An egui app for prompting local offline LLMs.

Description

coze is an egui application for prompting local offline LLMs using the Huggingface candle crate.

Currently it supports the following quantized models:

• Mistral Instruct 7B v0.2
• Mistral 7B v0.1
• Hugging Face Zephyr 7B β
• StableLM 2 Zephyr 1.6B

The first time a model is used its weights are downloaded from Huggingface and cached to the ~/.cache/coze folder for later use.

The current version supports:

• Prompt history navigation with fuzzy matching.
• History persistence across runs.
• Token generation modes.
• Copy prompts and replies to clipboard.
• Light/Dark mode.

See the app Help menu for usage details.

Installation

The latest version can be installed by getting the binaries for Linux, MacOS, and Windows from the releases page, or by using cargo:

cargo install --locked coze

To build locally (debug build may be very slow):

git clone https://github.com/vincev/coze
cd coze
cargo r --release