AskHN: What's a real world production use case for Ollama?

The article discusses the use of Ollama, a wrapper around llama.cpp that simplifies the downloading of LLMs. The author is seeking real-world production use cases for Ollama, despite its hype and the fact that it hinders performance due to its model offloading capability. The author also mentions a preference for deploying thoroughly tested models in production.

In response, some users suggest that Ollama is not intended for production use, as explicitly stated in its documentation. Instead, it's more suitable for experimenting with LLMs and different models. Another user mentions its use in a research group to run large LLMs quickly on a decent server, highlighting its cost-effectiveness when the necessary hardware is already available.

Key takeaways:

Ollama is a wrapper around llama.cpp and is used to download LLMs.
It is not recommended for production use due to its model offloading capability that can hinder performance.
Ollama's primary use is for experimentation with LLMs and different models, not for commercial use.
Some research groups use Ollama to run large LLMs quickly on servers, and it is considered cost-effective for those who already have the hardware.

AskHN: What's a real world production use case for Ollama?

Key takeaways:

Comments (0)

Newsletter