The article also provides a detailed guide on how to set up and use AICI, including a walkthrough of an example where AICI is used to control LLM output. It further discusses the architecture, security, performance, and flexibility of AICI. The tool is currently a prototype and Microsoft Research is looking for a research intern to contribute to the project.
Key takeaways:
- The Artificial Intelligence Controller Interface (AICI) is a tool that allows developers to build Controllers that can direct and constrain the output of a Large Language Model (LLM) in real time.
- AICI is designed to simplify the development of Controllers, making it easier to write fast Controllers and ensure compatibility across LLM inference and serving engines.
- Controllers are implemented as light-weight WebAssembly (Wasm) modules which run on the same machine as the LLM inference engine, utilizing the CPU while the GPU is busy with token generation.
- AICI is a prototype designed and built at Microsoft Research, and it currently integrates with llama.cpp, HuggingFace Transformers, and rLLM, with vLLM in the works.