Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

GitHub - katanemo/archgw: AI-native (edge and LLM) proxy for agents. Move faster by letting Arch handle the pesky heavy lifting in building agentic apps -- ⚡️ query understanding and routing, seamless integration of prompts with tools, and unified access and observability of LLMs. Built by the contributors of Envoy proxy.

Mar 05, 2025 - github.com
Arch is an intelligent proxy designed for agentic applications, enabling secure, efficient, and personalized handling of API requests and LLM interactions. Built by contributors of Envoy Proxy, Arch focuses on nuanced prompt handling, offering features like intent-based routing, prompt guardrails, LLM traffic management, and observability using the W3C Trace Context standard. It operates alongside application servers, leveraging Envoy's HTTP management capabilities to enhance GenAI app performance.

The article provides a quickstart guide for using Arch Gateway to build AI agents, such as a currency exchange agent, and demonstrates its use as an LLM router. It includes steps for setting up Arch with Docker, creating configuration files, and interacting with the gateway using curl commands or the OpenAI Python client. Arch supports observability through integration with tools like Signoz and encourages community contributions to improve its features and documentation.

Key takeaways:

  • Arch is an intelligent proxy designed to enhance agentic applications by connecting existing APIs, providing features like intent-based prompt routing, prompt guard, LLM routing, and observability.
  • Arch Gateway is built on Envoy Proxy, offering secure handling, intelligent routing, and robust observability for prompt-based tasks, with purpose-built LLMs to manage prompts effectively.
  • The platform supports building AI agents with a simple configuration process, allowing integration with APIs like currency exchange services, and offers a free hosted version of its function-calling LLM in the US-central region.
  • Arch provides observability features using open standards, enabling request tracing, metrics monitoring, and integration with tools like Signoz for enhanced AI application performance.
View Full Article

Comments (1)