GitHub - abi/secret-llama: Fully private LLM chatbot that runs entirely with a browser with no server needed. Supports Mistral and LLama 3.
May 03, 2024 - github.com
The article introduces a fully private, in-browser chatbot called Secret Llama that supports Llama 3, Mistral, and other open source models. The chatbot is designed to ensure that no conversation data leaves the user's computer, does not require a server or installation, and can work offline. It also boasts an easy-to-use interface comparable to ChatGPT. The chatbot utilizes the inference engine provided by webllm and requires a modern browser with WebGPU support to run.
The article also provides a list of supported models, including TinyLlama, Llama-3, Phi1.5, and Mistral, along with their respective sizes. It invites contributions to improve the interface, support more models, speed up initial model loading time, and fix bugs. Additionally, it mentions other projects such as screenshot to code and Pico - an AI-powered app builder.
Key takeaways:
The Secret Llama is a fully private, in-browser chatbot that supports Llama 3, Mistral and other open source models.
The chatbot requires a modern browser with support for WebGPU and may have specific RAM requirements depending on the model used.
It supports various models including TinyLlama-1.1B-Chat-v0.4-q4f32_1-1k, Llama-3-8B-Instruct-q4f16_1, Phi1.5-q4f16_1-1k, and Mistral-7B-Instruct-v0.2-q4f16_1.
Contributions are welcomed to improve the interface, support more models, speed up initial model loading time and fix bugs.
