Ask HN: Local LLM Recommendation?

The author is a software engineer seeking advice on finding a suitable Language Model (LLM) that can be trained with their personal markdown notes to organize ideas and information. They want the LLM to identify topics and answer prompts such as deadlines, tasks, and meeting agendas. The LLM should run on a Linux machine with modern specs and operate in an "air gapped" way to prevent outside transfer of information for data privacy.

The author's main focus is on topic identification rather than powerful reasoning. They also emphasize the need for the system to work in an isolated environment due to the sensitive nature of the data they might use. They are open to suggestions and help in finding a solution that fits these requirements.

Key takeaways:

The individual is seeking a Language Model (LLM) that can be trained with personal notes to provide answers to prompts, helping to organize ideas and information.
The LLM should be able to identify topics correctly and does not necessarily need to have powerful reasoning capabilities.
The system should be able to run on a commodity Linux machine with relatively modern specs, and without the need for a high-end GPU.
The LLM should operate in an "air gapped" manner, meaning it should not transfer any information outside, to maintain the privacy of potentially sensitive data.

Ask HN: Local LLM Recommendation?

Key takeaways:

Comments (0)

Newsletter