AI Incident Copilot
A Claude-powered production incident assistant. Feed it symptoms, service logs, and runbooks — it retrieves the relevant runbook guidance, then returns a structured investigation brief: probable cause, evidence, an investigation checklist, and stakeholder-ready update drafts.
The problem
Triaging a production incident usually means an engineer manually stitching together symptoms, raw logs, and whatever runbook happens to be relevant — under time pressure, and often before the first stakeholder update goes out. AI Incident Copilot turns that into a repeatable workflow instead of a one-off chat prompt: it grounds the model's answer in retrieved runbook evidence and always returns the same shape of output, so the result is easy to trust and act on.
Architecture
The frontend never talks to Claude directly. Every request goes through FastAPI, which checks the daily usage limit, retrieves the most relevant runbook chunks for the incident, folds that guidance into a structured prompt, and only then calls Claude — or, in mock mode, returns a canned response with no external call at all.
Key decisions
Runbook retrieval uses tokenization, sparse embeddings, and in-memory vector similarity search rather than a managed vector DB. Why: zero extra infrastructure or cost for a project that runs on Render's free tier. Tradeoff: embeddings for uploaded runbooks live in memory and don't survive a backend restart — fine for a demo, not for a real multi-user deployment.
The Anthropic key only ever lives in the FastAPI process's environment variables; the React
app never sees it. A CLAUDE_DAILY_LIMIT env var caps real Claude calls regardless
of traffic. Why: the frontend is a public demo — the worst-case spend needs a hard
ceiling that doesn't depend on anyone remembering to turn something off.
Mock mode exercises the full UI — retrieval, checklist, stakeholder drafts — with no API key and no cost. Why: anyone reviewing the project can see the entire workflow work end-to-end without needing an Anthropic API key or spending a cent; Claude mode is opt-in for people who want to see the real model output.
The AI orchestration (chunking, embeddings, PDF parsing) lives in a separate Python service rather than inside the frontend build. Why: keeps the Python RAG pipeline decoupled from frontend deploys. Tradeoff: Render's free tier means the backend can cold-start on the first request after inactivity.
A pytest suite checks that known incidents (pricing timeouts, inventory backlog, checkout regressions) actually retrieve the runbook guidance they should — and that irrelevant runbooks don't get pulled in. Why: RAG relevance is easy to eyeball as "looks right" in a demo and silently regress; treating retrieval as testable behavior catches that.
What I'd do differently
The in-memory embedding store is the honest limitation: uploaded runbook context resets on every backend restart or redeploy. That's acceptable for a demo but is the first thing I'd fix before this could hold a real, growing runbook library — moving to Postgres + pgvector so retrieval state survives deploys, which is already called out as the next step rather than something I discovered after the fact.