LLM Gateway

A full-stack AI API gateway that gives teams a single OpenAI-compatible entry point for multiple clients, with API key lifecycle management, per-client limits, usage logging, and cost analytics.

Overview

LLM Gateway is designed for multi-tenant AI product workflows, especially cases where many client applications (such as WhatsApp chatbot deployments) need isolated access, usage controls, and billing visibility.

Instead of distributing provider API keys to each client, applications call the gateway with gateway-issued bearer keys. The gateway validates access, enforces limits, proxies requests to OpenAI, and records usage/cost telemetry for reporting.

This project was built specifically for companies selling n8n automation services. n8n handles workflow orchestration well, but it does not natively provide strong per-client LLM cost governance, usage accountability, and API key isolation for agency-style multi-tenant operations.

As those companies onboard more clients, manual tracking becomes error-prone and it gets difficult to control spend, enforce fair usage policies, and accurately attribute costs per client. LLM Gateway fills that operational gap.

Key Features

OpenAI-Compatible Gateway Endpoints

• Proxies /v1/chat/completions, /v1/completions, and /v1/embeddings
• Supports models listing endpoint for client integrations
• Keeps client apps integrated against a familiar OpenAI-style API surface

Multi-Tenant Client Management

• Create and manage clients with profile fields (name, email, company, phone)
• Assign lifecycle status (Active, Suspended, Inactive)
• Search/filter clients and monitor usage by client account

API Key Lifecycle

• Auto-generated gateway keys with llm_gw_... style format
• Optional key expiration
• Regenerate/revoke flows
• Last-used tracking and per-key usage visibility
• Keys stored hashed for safer persistence

Usage Tracking and Costing

• Logs every request with endpoint, model, token counts, latency, status, and computed cost
• Supports configurable model pricing (input/output) for accurate reporting
• Enables operational and billing analytics across date ranges
• Makes per-client cost attribution practical for n8n agencies managing many client automations

Rate Limiting

• Per-client limit controls across:
  • token volume
  • request count
  • credit budget
  • optional WhatsApp session count
• Configurable cycle-based enforcement
• Returns HTTP 429 when thresholds are exceeded

Analytics Dashboard

• Usage-over-time charts
• Model distribution and cost breakdowns
• Client-to-client usage comparison
• Date filters (24h, 7d, 30d, 90d) for operational and billing views

Settings and Provider Configuration

• User-level OpenAI API key management
• Configurable base URL and timeout for provider calls
• Model configuration table with enable/disable and per-model pricing

Authentication and Access Boundaries

• Dashboard protected by Better Auth
• Gateway access restricted to bearer API keys (no dashboard session dependency)

Architecture

Client Apps (Bearer key)
          |
          v
+-----------------------------------+
|            LLM Gateway            |
|     auth, limits, logging, cost   |
+-----------------------------------+
      |                        |
      v                        v
PostgreSQL                 OpenAI API
(clients, keys,            (or custom URL)
 usage, limits)

Gateway Request Flow

1. Client sends request with gateway bearer key.
2. Gateway validates key and client status.
3. Gateway evaluates rate limits (and optional WhatsApp session rules).
4. Gateway forwards request to OpenAI using configured provider credentials.
5. Gateway records tokens, latency, status, and cost.
6. Gateway returns provider response (or 429/4xx/5xx when blocked or failed).

Technical Stack

Frontend

• Next.js
• React
• TypeScript
• Tailwind CSS
• Radix UI
• TanStack Query
• Recharts

Backend

• Next.js API routes
• tRPC
• Prisma ORM
• Better Auth

Data and Infrastructure

• PostgreSQL
• OpenAI SDK
• Turborepo monorepo (apps/web, apps/server)

Production Highlights

• End-to-end type safety from database to UI via Prisma + tRPC + TypeScript
• Multi-tenant isolation through scoped API keys and per-client limits
• Operations-focused design with telemetry, pricing configuration, and analytics
• Extensible gateway layer ready for additional model providers and endpoints

Impact

LLM Gateway centralizes AI API governance for multi-client environments by combining access control, usage visibility, and billing-oriented analytics in one platform. It reduces key sprawl, improves operational observability, and provides a strong foundation for scaling client-facing AI workflows. For n8n service companies, it turns a hard-to-manage shared-API-cost problem into a controllable, client-level operating model.