Decipher's summarization pipeline involves identifying the last click event before an error, finding the clicked element, filtering events based on timestamps, and including relevant events. The company experimented with different LLMs and found a mix of Haiku and Sonnet to be the most effective. However, the summarizer can struggle with very long sessions where relevant events are widely dispersed. The company is considering additional enhancements, including the incorporation of new models and interspersing actual visuals for multimodal interpretation. They also provide a sandbox and an open-source library for users to try out their technology.
Key takeaways:
- Decipher uses AI to generate session summaries that make sense of error alerts, leveraging rrweb, an open-source library designed to record and replay user interactions on web applications, and LLMs to create concise, plain English summaries.
- rrweb captures a comprehensive series of events throughout the session, including DOM changes, user interactions, viewport changes, and form inputs. It uses incremental snapshots to efficiently manage the volume of data.
- Decipher's summarization pipeline involves several steps to ensure that only the most relevant events are included in the LLM context, including identifying the last click event before the error, traversing nodes to find the clicked element, filtering events based on timestamps, and including relevant events.
- Decipher is investigating additional enhancements including the incorporation of new models like Gemini 1.5 Flash and interspersing actual visuals for multimodal interpretation.