Relay

The problem

Relay is an async AI task agent platform that decouples HTTP intake, background workflows, and result delivery for observable long-running LLM jobs.

What I built

Built an async AI job platform in 2.6K+ lines of TypeScript that returns a job ID instantly while background workers run long LLM tasks with 3-attempt retries, linear backoff, cancellation, and per-job cost tracking.

How it works

• Next.js 14 App Router with TypeScript strict for API routes and dashboard UI.
• Render blueprint and workflow registration boundary; a fake in-process worker runs locally.
• Render Postgres schema in migrations/001_initial.sql; the local adapter is still in-memory.
• Server-Sent Events backed locally by an event bus; Postgres LISTEN/NOTIFY or Redis is still production follow-up work.
• Groq is the default LLM provider. OpenAI, Anthropic, and mock providers share the same interface.

Tradeoffs and limits

This project does not currently have a hosted demo. Run and verification instructions live in the repository when available.

What I learned

Unified Groq, OpenAI, and Anthropic behind one provider interface powering 5 Zod-validated workflow recipes, then hardened the API with idempotency keys, a 30 req/min token-bucket rate limiter, API-key scoping, and HMAC-signed webhooks.