ChatGPT is 1yo now. Which LLM framework(s) do you use in production and why?

The author discusses their experience with various LLM frameworks such as Langchain, LlamaIndex, LMQL, guidance, Marvin, Instructor, and others. They express uncertainty about the added value of these frameworks to LLM workflows, given the overlap between them, and question their maintainability and robustness. The author has been building their own libraries for use in some LLM applications, but is considering the potential benefits of using a framework due to recent advancements in the field.

These advancements include guaranteed function calling, improved RAG, and the use of agents and tools. The author is contemplating whether adopting one of these frameworks might be a more efficient approach than building everything independently. They are seeking opinions and comments on this matter.

Key takeaways

The author has used many LLM frameworks such as Langchain, LlamaIndex, LMQL, guidance, Marvin, Instructor, etc.
There is a lot of overlap between these frameworks and the author is unsure if they add value to LLM workflows in a maintainable and robust way.
The author has been building their own libraries for use in some LLM applications.
They are considering whether using one of these frameworks would be a better approach given recent advancements in the field.

ChatGPT is 1yo now. Which LLM framework(s) do you use in production and why?

Key takeaways

Discussion (0)