The extension can be installed directly inside Podman Desktop and supports AI applications as a set of containers connected together. It comes with a Recipes Catalog that helps navigate core AI use cases and problem domains such as Chat Bots, Code Generators, and Text Summarizers. The extension has specific software and hardware requirements, including Podman Desktop 1.8.0+, Podman 4.9.0+, and a minimum of 12GB of memory and at least 4 CPUs for the Podman machine. The roadmap for the extension includes expanded recipes, GPU acceleration, API/CLI interaction, enhanced playgrounds, fine-tuning with InstructLab, function calling, local RAG, and bridge with AI platforms.
Key takeaways:
- Podman AI Lab is an open-source extension for Podman Desktop that facilitates working with Large Language Models (LLMs) in a local environment. It includes a recipe catalog, a set of open-source models, and a playground for learning and experimentation.
- The extension uses Podman machines to run inference servers for LLM models and AI applications, supporting common formats like GGUF, Pytorch, or Tensorflow.
- Podman AI Lab provides a curated list of open-source AI models and LLMs, and supports AI applications as a set of containers that are connected together. It also includes a Recipes Catalog for core AI use cases and problem domains.
- The extension can be installed directly inside of Podman Desktop and is compatible with Windows, macOS, and Linux. It requires Podman Desktop 1.8.0+, Podman 4.9.0+, and a minimum of 12GB of memory and at least 4 CPUs for the Podman machine.