To use DistiLlama, one must first install and start Ollama, then clone the DistiLlama repository and install it using yarn. After enabling developer mode in Chrome, the user can load the unpacked extension from the cloned project's 'dist' folder. The extension can be customized by changing the LLM used, tweaking the summarization chain, or changing the model used. Future updates to DistiLlama may include making the summarization chain and LLM model configurable, improving the UI, and adding TTS support.
Key takeaways:
- DistiLlama is a Chrome extension that uses a locally running LLM instance to generate a summary of the current webpage.
- The project uses Ollama as the locally running LLM instance and uses Readability to extract text content from the page, which is then summarized using LangChain.
- To use DistiLlama, users need to install and start Ollama, clone the DistiLlama repo, and add the extension to their Chrome browser.
- Future improvements for DistiLlama include making the summarization chain and LLM model configurable, saving summaries in local storage, improving the UI, adding TTS support, and extending to chat with the page.