Production checklist for data science

Moving from notebook to production means reliable data, tested pipelines, monitored models, and clear ownership—not just higher test accuracy.

Before shipping

• Feature definitions match between train and serve
• Train/test split methodology documented; no leakage
• Baseline and champion metrics on holdout and segments
• Ethics review for affected populations
• Rollback plan if model degrades

Engineering integration

• Versioned datasets and model artifacts
• Scheduled batch scoring or low-latency API
• Unit tests on transforms; integration tests on sample payloads
• Secrets and PII handled per policy—not in notebooks

Monitoring

• Data drift — feature distributions shift
• Concept drift — relationship to target changes
• Operational — latency, error rate, null rate spikes

Handoff documentation

Deliver: model card, feature list with SQL sources, retrain cadence, on-call runbook, and stakeholder metric dashboard.

Pair with AI track context when models feed product features or generative systems.

Important interview questions and answers

1. Q: Train-serve skew?
   A: Training features computed differently than production—silent metric collapse.
2. Q: Model rollback?
   A: Keep previous artifact and routing flag to revert without redeploying entire app.

Self-check

1. List five pre-ship checklist items.
2. What is train-serve skew?
3. Name two types of drift to monitor.

Tip: Monitor feature drift after deploy—not just accuracy once.