Overview of Arch Gateway: AI-Native Proxy for Agentic Applications

Arch Gateway is a sophisticated proxy designed specifically for agentic applications, facilitating smoother and more efficient app development by handling complex tasks such as input clarification, agent routing, and integration of prompts with common tools. This platform aims to accelerate the development process by providing a robust framework that manages the interaction between user inputs and large language models (LLMs).

Key Features

• Routing: Arch Gateway features purpose-built LLMs for rapid agent routing and hand-off scenarios, ensuring responses are delivered in less than 100 milliseconds.
• Tools Use: It simplifies the integration of prompts with tools and API calls, streamlining the creation of agentic scenarios.
• Guardrails: The platform includes centralized configurations to prevent harmful outcomes and ensure safe user interactions.
• Access to LLMs: Arch Gateway centralizes access to LLMs, managing traffic with smart retries to maintain continuous availability.
• Observability: It supports W3C-compatible request tracing and LLM metrics that integrate seamlessly with popular tools, providing clear insights into operations.
• Built on Envoy: Leveraging the proven HTTP management and scalability features of Envoy, Arch Gateway operates alongside application servers as a containerized process.

Usage Scenarios

Arch Gateway addresses common challenges faced by developers in agentic application development:

• Routing and Handoff: Developers transitioning from broad to specialized prompts often struggle with building efficient routing and handoff code. Arch Gateway automates this process.
• LLM Integration: The platform allows for the quick and safe addition of new LLMs without the need for extensive integration coding.
• Prompt Engineering: It assists in clarifying user intent and validating inputs, reducing the workload associated with prompt engineering.
• Observability: Developers can focus on core functionalities rather than integrating observability tools, as Arch Gateway handles this aspect transparently.

Quickstart Guide

To get started with Arch Gateway, developers need to set up their environment with Docker, Docker Compose, and Python. The platform offers a CLI for efficient management and interaction with the gateway. Here’s a brief guide on setting up a simple AI agent using Arch Gateway:

1. Environment Setup: Install Docker System (v24), Docker Compose (v2.29), and Python (v3.12). Create a Python virtual environment and install the Arch Gateway CLI.
2. Configuration: Create an arch_config.yaml file to specify the gateway settings, including listener address, port, message format, and LLM providers.
3. Start the Gateway: Use the Arch Gateway CLI to start the gateway with the specified configuration.

Demos and Documentation

Arch Gateway provides several demos and comprehensive documentation to help developers understand and utilize the platform effectively. These resources demonstrate the application of Arch Gateway in various scenarios, such as weather forecasting and network operations, and provide insights into connecting with third-party SaaS APIs.

For more detailed information and step-by-step guides, developers are encouraged to refer to the official documentation and explore the available demos.

Conclusion

Arch Gateway is designed to streamline the development of agentic applications by automating complex processes and providing a robust framework for managing interactions with LLMs. By handling the heavy lifting of application development, it allows developers to focus on creating more sophisticated and responsive AI agents.

Related Apps