Cutting LLM Costs by 83% with LangSmith - Dataherald

The article discusses the use of LangChain and LangSmith in the development and maintenance of Dataherald, an open-source natural language-to-SQL engine. The authors express their satisfaction with LangChain's active community and integrations, but also their frustration with its bloated features. They have recently started using LangSmith to monitor their app, which has helped them identify and fix bugs more efficiently, and reduce token usage.

The authors also discuss the cost problem associated with using open-source language models, which are often less accurate and more expensive than GPT-4. They explain how LangSmith helped them identify a bug that was causing token usage to grow uncontrollably, and how fixing this bug reduced their costs by 83%. They conclude by stating that they are looking into using more of LangSmith's features, and recommend it to others who have built applications on LangChain.

Key takeaways:

Dataherald, an open source natural language-to-SQL engine, has been using LangChain to build their product and recently started using LangSmith to monitor their app in production.
LangSmith helped Dataherald identify and fix bugs quickly, reducing their token usage and costs significantly.
LangSmith's features for monitoring, debugging, testing, and evaluating applications powered by LLMs are proving to be invaluable for teams like Dataherald.
Dataherald is looking to further integrate LangSmith's management and evaluation features into their engineering process, particularly for regression testing.

Cutting LLM Costs by 83% with LangSmith - Dataherald - Medium

Key takeaways:

Comments (0)

Newsletter