The usage of Gigax involves model instantiation and stepping an NPC. Various models are provided on the Huggingface hub, and users can instantiate one of them using outlines. The article provides a step-by-step guide on how to do this. Stepping an NPC is a one-liner action, and the platform provides classes to instantiate Locations, NPCs, etc. The article also mentions that users can contact Gigax to try their NPC API, which takes care of model serving, NPC memory, and more.
Key takeaways:
- Gigax offers NPCs that can perform various actions such as speaking, jumping, and attacking, with less than 1 second GPU inference on most machines.
- It provides open-weights models that are fine-tuned from Llama-3, Phi-3, Mistral, etc., and structured generation with Outlines, ensuring the output format is always respected.
- Upcoming features include a local server mode with a language-agnostic API, runtime quest generation for players and NPCs, and memory creation, storage, and retrieval with a Vector DB.
- Gigax provides various models on the Huggingface hub, such as NPC-LLM-7B and NPC-LLM-3_8B, and allows for model instantiation and NPC stepping using Python code.