← All work
Incident triage agent
2025 · In Progress · Work-adjacent prototype · Python, Airflow, Datadog
Case study
Code is privateProblem
Most on-call pages are the same five failures wearing different hats. A human still has to wake up, orient, and diagnose from scratch every time.
What it does
Parses the alert, pulls logs and lineage, classifies the failure mode, and either retries safely or hands the on-call a briefing with the diagnosis already done.
Architecture
- 01Alert ingestion
- 02Log and lineage retrieval
- 03Failure classification
- 04Safe action or human briefing
Stack
- SourcesAirflow, Datadog, Rootly
- Agent logicTool-calling LLM with guarded actions
- RuntimeEvent-driven (SQS + Lambda)
Outcomes
- Target: triage the top failure classes without waking anyone
Code is private — happy to walk through it.