The AI Feature That Shipped Without a Kill Switch: A Post-Mortem

The Slack Thread That Went Sideways

10:42 AM — Support: "Getting reports that AI recommendations are way off. 12 tickets in the last hour."

10:44 AM — PM: "Checking… model accuracy looks normal on our dashboard. Probably user error?"

11:03 AM — Support: "Now 28 tickets. Users are angry. Can we turn this off?"

11:05 AM — PM: "Turn what off? The feature is hardcoded. We'd need to deploy a code change."

11:07 AM — Eng Lead: "Deploy takes 45 minutes (build + test + rollout). We're in the middle of a freeze."

11:15 AM — CEO: "This is trending on Twitter. Turn. It. Off. Now."

11:17 AM — PM: "We can't. No kill switch."

The Damage:

• 4 hours to emergency rollback (bypassed freeze, skipped tests, manual deploy)
• 200+ support tickets
• 3 enterprise customers escalated to exec team
• 1 customer threatened contract cancellation
• 1 PM learned a very expensive lesson

What Went Wrong (The Root Cause)

The Feature: AI-powered email auto-responder (suggests replies to customer support tickets).

The Model: Fine-tuned LLM, retrained weekly on new support ticket data.

The Deployment: Model update deployed Friday 6 PM. QA passed. Went live.

The Incident: Saturday morning, model started suggesting inappropriate responses (sarcastic tone, off-topic, occasionally rude).

Root Cause Analysis:

1. Training data included spam tickets (users testing the system, joking around)
2. Model learned sarcastic patterns from spam
3. Offline eval didn't catch this (eval set didn't include spam-like inputs)
4. No human-in-the-loop review for suggested responses
5. No kill switch to disable AI without code deploy

Why It Got Expensive:

• Kill switch would've taken 2 minutes (flip feature flag)
• Emergency rollback took 4 hours (code change, build, deploy, test)
• 120x slower response time

The Kill Switch Taxonomy (Three Levels)

Level 1: Feature Flag (Required for All AI)

What: Boolean toggle in config file or admin panel.

How:

if (featureFlags.aiEmailSuggestions === true) {
  return getAISuggestion(ticket);
} else {
  return null; // Fall back to manual response
}

Who Can Use: PM, eng lead, on-call engineer

Response Time: Under 2 minutes

When to Use: Model degrades, user complaints spike, unexpected behavior

Level 2: Confidence Threshold Dial (Precision Control)

What: Adjustable threshold for AI confidence. Only show suggestions above threshold.

How:

const confidenceThreshold = config.aiConfidenceThreshold; // default: 0.7

if (aiConfidence > confidenceThreshold) {
  return aiSuggestion;
} else {
  return null; // Don't show low-confidence suggestions
}

Who Can Use: PM, data scientist

Response Time: 5-10 minutes

When to Use: Model is mostly correct but has higher-than-usual error rate; want to reduce false positives without full shutdown

Level 3: Rollback to Previous Model Version (Nuclear Option)

What: Switch from new model (v2.3) back to old model (v2.2).

How:

const modelVersion = config.aiModelVersion; // "v2.3" or "v2.2"

const model = loadModel(modelVersion);

Who Can Use: ML engineer, PM (with approval)

Response Time: 10-30 minutes (model swap, cache clear)

When to Use: New model version is fundamentally broken; previous version was stable

The Incident Response Playbook (Copy-Paste)

Phase 1: Detect (0-10 minutes)

Signals:

• Support ticket volume spike (>3x normal)
• User reports of "AI is wrong/broken/weird"
• Monitoring alert (accuracy drop, error rate spike, latency spike)

Action:

• Page on-call PM + ML engineer
• Triage: Is this model degradation or user error?

Phase 2: Contain (10-20 minutes)

If Model Degradation Confirmed:

• Option A: Flip kill switch (disable AI feature)
• Option B: Raise confidence threshold (reduce false positives)
• Option C: Rollback to previous model version

Communication:

• Post in #incidents channel: "AI feature disabled due to quality issues. Investigating."
• Update status page: "AI suggestions temporarily unavailable. Manual responses working normally."

Phase 3: Diagnose (20 minutes - 2 hours)

Questions to Answer:

• What changed? (new model version, new training data, upstream API change)
• What's the failure mode? (hallucinations, bias, off-topic, latency)
• What's the scope? (all users, specific customer segment, specific input types)

Data to Collect:

• Recent model deployments (timestamps, versions)
• Recent training data changes (new sources, data quality issues)
• Sample of bad outputs (log 10-20 examples of errors)
• Offline eval results (did eval catch this? if not, why?)

Phase 4: Fix (2 hours - 2 days)

Temporary Fix (2-6 hours):

• Rollback to previous model version
• Re-enable feature with higher confidence threshold
• Add human-in-the-loop review for all suggestions (slow but safe)

Permanent Fix (1-2 days):

• Clean training data (remove spam, offensive content)
• Expand eval set (add failure cases to prevent regression)
• Retrain model
• Test on new eval set + manual QA
• Deploy with kill switch ready

Phase 5: Post-Mortem (Within 1 week)

Template:

INCIDENT: AI Email Suggestions Degraded (Aug 4, 2025)

IMPACT:
- Duration: 4 hours (10:42 AM - 2:47 PM)
- Users affected: ~2,000 (all users saw degraded suggestions)
- Support tickets: 200+
- Customer escalations: 3 (enterprise accounts)

ROOT CAUSE:
- Training data included spam tickets with sarcastic responses
- Model learned inappropriate tone patterns
- Offline eval set didn't include spam-like inputs
- No kill switch for rapid disable

TIMELINE:
- 10:42 AM: First support ticket
- 11:05 AM: PM confirms issue, realizes no kill switch
- 11:30 AM: Emergency rollback initiated (bypassed deploy freeze)
- 2:47 PM: Rollback complete, feature re-enabled with old model

WHAT WENT WELL:
- Team responded quickly once severity understood
- Support team communicated proactively with affected users

WHAT WENT POORLY:
- No kill switch → 4-hour response time (should be <10 minutes)
- Training data quality not monitored → spam leaked in
- Eval set didn't cover spam-like inputs → missed in QA

ACTION ITEMS:
- [PM] Add feature flag kill switch (due: Aug 6)
- [ML] Implement training data quality checks (due: Aug 10)
- [ML] Expand eval set to include edge cases (due: Aug 10)
- [Eng] Add confidence threshold dial (due: Aug 12)
- [PM] Update runbook: how to disable AI features (due: Aug 8)