Who is building LLM Chatbots, and what issues are you running into?

The article discusses the challenges faced while building an LLM chatbot for a technical book publisher with 20 years of data from books and conference talks. The difficulties include using LangChain, which isn't always reliable, a complex data pipeline, and the need for significant memory to index embeddings in PostGres.

The most challenging aspect, however, has been improving conversation quality. The use of LangSmith has been beneficial for tracing and observability, but it has its limitations. The author expresses a desire for better tools in the market to enhance the chatbot's performance.

Key takeaways:

The team is building an LLM chatbot that utilizes data from a technical book publisher's 20 years of books and conference talks.
They are using LangChain for the project, but it has its limitations.
The data pipeline and indexing embeddings in PostGres have been challenging due to their complexity and high resource requirements.
Improving conversation quality has been the most difficult task, despite the use of LangSmith for tracing and observability.

Who is building LLM Chatbots, and what issues are you running into?

Key takeaways:

Comments (0)

Newsletter