Nvidia has also revealed updates to TensorRT-LLM, initially launched for Windows to run large language models (LLMs) more efficiently on H100 GPUs. The update extends its compatibility to PCs powered by GeForce RTX 30 and 40 Series GPUs with 8GB of RAM or more. Nvidia will also make TensorRT-LLM compatible with OpenAI’s Chat API, allowing developers to run LLMs locally on their PCs. This is part of Microsoft's "hybrid loop" development pattern, aiming to enable AI development across the cloud and local devices.
Key takeaways:
- Microsoft has announced Windows AI Studio, a hub for developers to access and configure AI models, during the Microsoft Ignite event.
- Windows AI Studio will be rolled out as a Visual Studio Code extension in the coming weeks, and it will allow developers to test the performance of their models using Prompt Flow and Gradio templates.
- Nvidia has revealed updates to TensorRT-LLM, bringing it to PCs powered by GeForce RTX 30 and 40 Series GPUs with 8GB of RAM or more, and making it compatible with OpenAI’s Chat API.
- These developments are part of Microsoft's goal to create a 'hybrid loop' development pattern, enabling AI development across the cloud and locally on devices.