Before launch, walk this checklist—Generative features fail in ops and policy more often than in tensor math.
Checklist
- ☐ Secrets in vault; keys rotated; no client-side exposure
- ☐ Data retention and training opt-out documented
- ☐ RAG corpus owned, versioned, deletable
- ☐ Eval set + regression job in CI
- ☐ Moderation pre/post; injection test cases
- ☐ Cost and rate limits per user/tenant
- ☐ Fallback when API down; status page comms
- ☐ Human escalation path for high-risk intents
- ☐ Incident runbook (bad rollout, data leak)
Ongoing
Review thumbs-down clusters weekly; re-embed corpus on doc changes; track p95 latency and $/1k sessions.
Continue learning
Deepen security on Cybersecurity; scale infra on AWS; refresh ML math on AI and Data Science.
Important interview questions and answers
- Q: Minimum eval before ship?
A: Labeled questions with expected grounding and refusal behavior—not vibe checks alone.
Self-check
- Name five checklist items.
- What to do when corpus updates?
Tip: A Gen AI feature without rollback and owner is a demo, not production.
Interview prep
- Before ship?
Eval regression, moderation tests, secrets, fallback, owner, runbook.
- Corpus update?
Re-embed and run retrieval eval—version like schema migrations.