Langtrace also offers an observability client optimized for unique challenges associated with LLM-based apps, such as evaluations and prompt iteration. The client is SOC2 compliant and can be self-hosted for strict data privacy requirements. Both the SDK and the client are fully open source, and the team is seeking community feedback for improvement, particularly on standardizing trace attribute names following the OTEL rules.
Key takeaways:
- Ola and Karthik are developing Langtrace, an open-source, open telemetry based SDK and monitoring/evaluations client for LLM based applications.
- The SDK generates OTEL standard spans and traces for popular LLMs, frameworks, and vectorDBs, aiming to avoid vendor lock-in and allow for custom tooling development.
- Langtrace includes a Python and a TypeScript SDK, and an observability client that is optimized for LLM based apps. Both the SDK and the client are fully open source and SOC2 compliant.
- The team is seeking community feedback, particularly on Open Telemetry semantic conventions for LLMs, and acknowledges that the project is in its early stages with room for improvement.