The tool also offers enterprise-grade features such as SOC 2 Type 1, admin/user controls, and a self-hosted option. SOC 2 Type 1 ensures that controls are in place and an audit has been completed. Admin/user controls allow for organization of members and control over read vs write access, with more fine-grained controls coming soon. The self-hosted option, which is also coming soon, will allow users to keep all their data within their own infrastructure. A 14-day trial of the tool is available without the need for a credit card.
Key takeaways:
- The service allows for continuous evaluation of generative AI quality with AI and heuristics, and observation of speed and cost in production.
- It offers automated grading with Evaluate, using AI and heuristic evaluators to automatically evaluate for regressions and hallucinations.
- With Observe, users can monitor production for speed and cost, and see detailed information about inputs, outputs, and evaluator scores for particular generations.
- The service is enterprise-grade, with SOC 2 Type 1 controls in place and an audit completed. It offers admin/user controls and a self-hosted option is coming soon.