AI Chat
AI Chat is a lightweight, responsive chat interface designed to run LLM conversations at the edge. Utilizing Server-Sent Events (SSE) to stream text responses from Cloudflare’s model registry, it provides a performant chatbot workspace with minimal load-time overhead and client-side history preservation.
Technical Stack
Edge AI & Stream
| Inference Engine | Cloudflare Workers AI LLMs |
| Models Supported | Llama-3-8B-Instruct, Mistral-7B |
| Stream API | Server-Sent Events (SSE) |
| Language | TypeScript |
Infrastructure
| Compute | Cloudflare Workers |
| AI Telemetry | Cloudflare AI Gateway (Rate limiting & logging) |
| Session Cache | Client-side browser state (sessionStorage/localStorage) |
Streaming with SSE: Instead of waiting for full LLM completions, the edge Worker streams individual tokens to the client as they are generated, improving perceived response times.
Core Features
- Responsive Chat Interface — Clean, mobile-friendly layout optimized for typing, reading, and viewing code blocks.
- AI Gateway Integration — Configured to run through Cloudflare AI Gateway to enable caching, logging, analytics, and custom rate limits.
- Local History Persistence — Stores conversation history directly on the user’s browser, preventing server leaks and maintaining chat continuity across reloads.
- Custom Prompts — Easily supports overriding the LLM’s system prompt or selecting different model categories directly from the web layout.