Ask HN: Who is using small OS LLMs in production?

The article discusses the use of small open-source Language Learning Models (LLMs) in production, with a focus on models such as replit code 3b and llama 2 7b. The author seeks to understand the motivations behind choosing these models over the more popular prompted GPT API. The main question raised is whether these relatively small LLMs are being used in production or not.

The responses highlight several reasons for choosing small LLMs over the GPT API. These include privacy concerns, cost-effectiveness, and the ability to work locally. Some users also mention the reliability issues with OpenAI APIs, which have been known to go down without warning. Other users point out the need for data security and privacy in sectors such as aviation and finance, where legal and regulatory requirements necessitate the use of these models.

Key takeaways:

Small open source LLMs are being used in production, with privacy being a major motivation for their use.
Other reasons for using these models include cost-effectiveness and the ability to work locally in a short time.
Some users have found the performance of Llama 2 models comparable to GPT-3.5.
Reliability is another factor, with OpenAI known to have downtime issues.

Ask HN: Who is using small OS LLMs in production?

Key takeaways:

Comments (0)

Newsletter