GitHub - llmonitor/llmonitor: Observability for AI apps and agent
Oct 28, 2023 - github.com
LLMonitor is a tool designed to assist AI developers in monitoring their applications in production. It offers features such as cost, token and latency analytics, user tracking, debugging traces, full request inspection, dataset labelling and exporting, user feedback collection, and soon to be available unit tests and prompt evaluations. The tool is compatible with any model, not just OpenAI, and is easy to integrate and self-host. It also offers modules for JavaScript and Python, and natively supports LangChain, OpenAI module, and LiteLLM.
The company provides a hosted version with a free plan that includes up to 1k requests per day. This version eliminates the need for devops or managing updates, offers priority support, and ensures data is stored safely in Europe. Users can reach out for support via Discord or email. The project is a creation of @vincelwt and Hugh, and is licensed under the Apache 2.0 License.
Key takeaways:
LLMonitor is a tool for AI developers to monitor their apps in production, offering features like cost, token & latency analytics, user tracking, debugging, and more.
The tool is designed to be usable with any model, easy to integrate, and simple to self-host.
LLMonitor offers a hosted version with a free plan of up to 1k requests per day, which includes priority support and secure data storage in Europe.
Support is available via Discord or email, and the project is licensed under the Apache 2.0 License.
