The technical details of the project are also shared, including the use of GCP, Kubernetes, FastAPI, Celery, Next.js, OpenA, pgvector, and Postgres in the stack. The article also discusses potential future considerations such as a possible database migration from Postgres to ClickHouse, hosting open-source models, and the use of evaluation frameworks like RAGAS for evaluating LLM responses.
Key takeaways:
- Zelma Education is an AI-powered U.S. state assessment data repository developed by Luke Van Seters and his team at Novy, in conjunction with Emily Oster, an economist at Brown University.
- Initial challenges in the project included accuracy and speed issues, which were resolved by creating a structured JSON schema for queries and using optimization techniques to speed up LLM responses.
- The technical stack used for the project included GCP, Kubernetes, FastAPI, Celery, Next.js, OpenA, pgvector and Postgres.
- Future considerations for the project include a potential database migration from Postgres to ClickHouse, hosting open-source models for better control over generation processes and cost savings, and the use of RAGAS for evaluating LLM responses.