with the skill
## Pre-Mortem Analysis: Tallyhub Invoice Engine
**Planned launch:** Monday, November 9
**Migration:** All 48,000 customer accounts overnight on Sunday, November 8
**Recommendation:** **Do not approve the all-account cutover as currently planned.** Migration capacity and recovery readiness are launch-blocking gaps. Keep November 9 conditional on explicit go/no-go gates.
*Dates below refer to the supplied launch calendar; no year was provided.*
### Failure scenario
It is November 9 at 9:00 am. The marketing email arrives, but migration is still running. Some customers cannot access complete invoice histories; others encounter inconsistencies. Support is overwhelmed. The team cannot confidently restore service because rollback was never defined and new invoices have already been created.
The critical mistake: treating a successful 5% dry run as proof of production readiness.
### Tigers (Real Risks)
| Risk | Urgency | Assessment and mitigation |
|---|---|---|
| **Migration may take far longer than overnight** | **Launch-blocking** | The dry run processed **800 accounts/hour**. At that rate, 48,000 accounts take **60 hours**, excluding validation, retries, and cutover. Parallelism might improve this, but has not been demonstrated. Measure production-scale throughput and complete a timed rehearsal against a defined migration window. If it cannot fit with contingency time, reduce scope or postpone. |
| **Rollback is undefined** | **Launch-blocking** | A failed migration could leave customers without a trusted invoice system. Define and rehearse recovery: validated backups, reversal or restoration, routing back to the old engine, and treatment of invoices created after cutover. A backup alone is not a rollback plan. |
Show the rest (50 more lines)
| **“No errors” may not mean financial correctness** | **Launch-blocking** | The dry run shows no reported errors; it does not establish that every invoice, balance, tax amount, payment link, and history record remained correct. Define reconciliation checks and test representative complex accounts. Resolve unexplained discrepancies before enabling migrated accounts. |
| **Writes during migration could be lost, duplicated, or inconsistent** | **Launch-blocking** | The plan does not describe how invoice creation, edits, payments, scheduled jobs, or integrations behave during cutover. Establish a tested write-freeze or change-capture/replay strategy, including safe retries and a single authoritative system. |
| **The launch email is not tied to operational readiness** | **Launch-blocking** | A fixed 9:00 am send could announce availability while migration or validation is failing. Make sending contingent on an explicit operational sign-off, with a named person authorized to hold it. |
| **Support preparation is late and underspecified** | **Fast-follow; minimum coverage required before launch** | A November 6 briefing leaves little time to fix gaps. Before launch, support needs a known-issues guide, account-status visibility, escalation paths, customer messaging, and staffed cutover coverage. Expand training and self-service guidance within 30 days. |
| **Production performance may degrade after cutover** | **Fast-follow, unless core workflows fail readiness tests** | Migration success does not prove acceptable invoice creation, search, export, or integration performance under real load. Load-test critical workflows before launch, define acceptable limits, and monitor latency and failure rates afterward. Core workflow failures should block launch. |
### Paper Tigers (Overblown Concerns)
**“Ledgerfox might announce something similar, so we must launch on November 9 regardless.”**
The announcement is speculative, and a competitor announcement does not itself prevent Tallyhub customers from adopting the engine. It warrants lightweight competitive monitoring and messaging preparation—not accepting financial-data or recovery risks.
**Boundary:** Evidence of active deal losses or a contractual deadline would justify reassessment. Neither is provided here.
### Elephants (Unspoken Worries)
These are unresolved assumptions, not established failures. Investigate them before launch.
| Unspoken question | Investigation |
|---|---|
| **Why must all 48,000 accounts move at once?** | Engineering and Product should determine whether a small, representative canary cohort and staged rollout can isolate failures. Do not assume the architecture supports this. |
| **Was the 5% sample representative?** | Compare migrated accounts with the full population by invoice volume, history size, currencies, tax complexity, integrations, and other supported edge cases. Account count alone may hide migration cost. |
| **What customer benefit defines launch success?** | Product should identify the expected customer outcome and measure it alongside invoice completion rates, correctness, and support contacts. A completed migration is not sufficient proof of a successful launch. |
| **Have customers been told about possible disruption?** | Confirm whether maintenance notices, integration coordination, or contractual notification requirements apply. A launch-day marketing email is not advance operational notice. |
### Action Plans for Launch-Blocking Tigers
*Owners are proposed functions; assign a named accountable person to each.*
| Risk | Concrete mitigation and acceptance gate | Owner | Due date |
|---|---|---|---|
| **Migration capacity** | Define the available window; run a production-representative, timed rehearsal including validation and retry overhead. Demonstrate completion with an agreed contingency buffer. Otherwise stage or delay. | Engineering migration lead + SRE | **Nov 4** |
| **No rollback** | Document triggers, decision authority, recovery-time/data-loss limits, and handling of post-cutover writes. Rehearse recovery and verify restored data. | Engineering lead + SRE | **Nov 5** |
| **Financial correctness** | Reconcile source and destination records, amounts, relationships, and key histories. Include complex accounts and critical workflow tests; resolve unexplained discrepancies. | Data/QA lead + accounting domain owner | **Nov 5** |
| **Concurrent writes** | Test the chosen freeze or synchronization approach, including in-flight operations, scheduled jobs, integrations, duplicate prevention, and recovery. | Backend engineering lead | **Nov 5** |
| **Unconditional marketing send** | Add a manual release gate. Send only after migration, reconciliation, and customer-facing health checks pass for the audience being announced to. Prepare delay messaging. | Marketing lead + launch commander | **Nov 6** |
### Final Go/No-Go Gate — November 6
Proceed only if:
- Migration duration is demonstrated to fit the approved window.
- Recovery is rehearsed, not merely documented.
- Financial reconciliation and cutover-write handling pass.
- Minimum support coverage and escalation procedures are ready.
- One launch commander can stop the cutover and hold the email.
**If any gate fails, postpone the all-account launch or use a validated, isolated staged rollout. Do not discover the recovery strategy on November 9.**
*Suggested Markdown filename: `PreMortem-Tallyhub-invoice-engine-Nov-09.md`.*