kyush-llm-router/README.md

Kyush LLM Router

Multi-user LLM routing proxy with API key management and usage monitoring.

Features

• Multi-user support: Manage multiple users with individual API keys
• Backend routing: Route requests to multiple OpenAI-compatible backends (vLLM, SGLang, etc.)
• Permission management: Control which users can access which backends
• Usage monitoring: Track request logs, token usage, and backend metrics
• Admin dashboard: Web UI for managing users, backends, and permissions

Architecture

┌─────────────────┐     ┌──────────────────────────────────────┐
│  LLM Clients    │────▶│  Router Server (Port 3000)           │
│  (OpenAI SDK)   │     │  - Authentication                    │
└─────────────────┘     │  - Request Routing                   │
                        │  - Admin API                         │
                        └─────────────────┬────────────────────┘
                                          │
                        ┌─────────────────┴────────────────────┐
                        ▼                                      ▼
                ┌─────────────────┐                  ┌─────────────────┐
                │  core.db        │                  │  analytics.db   │
                │  - users        │                  │  - request_logs │
                │  - backends     │                  │  - usage_stats  │
                │  - permissions  │                  │  - metrics      │
                └─────────────────┘                  └─────────────────┘

Quick Start

Development

1. Install dependencies:

npm install

2. Start development servers:

npm run dev

This starts:

• Express API server on http://localhost:3000
• Vite admin dashboard on http://localhost:3001

Production with Docker

1. Build and run:

docker-compose up -d

2. Set admin password via environment variable:

ADMIN_PASSWORD=your_secure_password docker-compose up -d

Project Structure

kyush-llm-router/
├── server/           # Express backend
├── client/           # Solid.js admin dashboard
├── database/         # SQL schema files
├── scripts/          # Development scripts
└── docs/             # Documentation

API Endpoints

OpenAI-Compatible API (Port 3000)

POST /v1/chat/completions
POST /v1/completions
GET  /v1/models

Admin API (Port 3001)

# User Management
POST   /admin/users
GET    /admin/users
PUT    /admin/users/:id
DELETE /admin/users/:id

# Backend Management
POST   /admin/backends
GET    /admin/backends
PUT    /admin/backends/:id
DELETE /admin/backends/:id

# Permission Management
POST   /admin/permissions
DELETE /admin/permissions
GET    /admin/permissions

# Analytics
GET    /admin/analytics/usage
GET    /admin/analytics/requests

Environment Variables

Variable	Description	Default
SERVER_PORT	Express server port	3000
CLIENT_PORT	Vite dev server port	3001
CORE_DB_PATH	Core database path	./data/core.db
ANALYTICS_DB_PATH	Analytics database path	./data/analytics.db
ADMIN_PASSWORD	Admin dashboard password	Required

License

MIT