ClientsFlow · Email-System Overhaul · W0 · EBO · FINAL

EBO — W0 · Observability & Watchdog

FINAL — owner comments folded in · 2026-07-12. The alarm layer that ships FIRST, as the rider for the whole drop (Q24=A): from the moment W0 is live, a dead schedule, a duplicate send, an accidentally armed auto-send flag, or hosting spend drifting past its budget each rings a bell on its own, within minutes, with zero polling by you. And the one thing that caused the July incident — a second copy of the app — is not monitored, it is made impossible. Scenarios S1 … S7.

What W0 owns
The system's alarms and nothing else: one health banner on the dash, one dedicated Slack alerts channel, the schedule-stall / auto-send-flag / duplicate-send / hosting-spend checks, the ready-to-paste rollback prompt inside every prod-misbehavior alarm — plus the one-time single-app guarantee (the old duplicate apps are deleted for good and deploys are pinned so no second copy can ever come back).
What W0 never does
W0 detects and alarms — it never intervenes. It never blocks or delays a send (send gating is W1's), never rolls anything back on its own (Q25: NO auto-rollback — it hands you the exact prompt instead), and — owner ruling, 2026-07-12 — never throttles, pauses, degrades or slows anything down to save money. It never touches a lead, a card, or the touchpoint history. It adds no ghost-app monitoring subsystem, no external dead-app monitor, and no log-scrubbing pass — all three were cut by the owner.
You do → click/action/hover You should see → on-screen result Element changes → copy · look · where What changes underneath → data/state Must NOT happen → bug guard 🕓 Touchpoint history → impact on the card's history (or —)
Owner rulings folded in (comments 2026-07-12 — these outrank round-3 answers, which outrank the handoff)
① No decoy/duplicate-app machinery. "Don't overcomplicate things with such copies of the app… make the main app so stable instead." The every-3-minutes ghost probe is deleted. In its place: the duplicate apps in the personal workspace are permanently deleted and every deploy is hard-pinned to the one production workspace, so a second copy cannot come back to life (S2). Its effects stay covered anyway by the duplicate-send scan (S6).
② No external dead-app monitor. "Don't build any infrastructure around the improbable case of the app dying. I will notice if it's dead and I will use Claude Code." The whole app-is-dead scenario and its outside-the-app monitor are deleted (this overrides round-3 Q13=A).
③ No log scrubbing. "Don't add this feature. Log everything as it is. Everything must appear verbatim." The log-hygiene audit/scrubber work is deleted (overrides round-3 D3 lineage). W0 adds no redaction work and changes no existing logging.
④ The $30 is HOSTING only, and it is a soft budget — not a cap. "It's only the hosting that should be around or below $30 a month… this is not a hard cap; we can go overboard. Therefore the app should never experience slowdown or pausing." So: the spend check watches hosting month-to-date only (not AI spend), fires one heads-up when it drifts past ~$30, and nothing anywhere is ever throttled, paused, capped or degraded to stay under it — including the hosting platform's own billing limit, which must be raised high enough that it can never reject a real lead's webhook.
⑤ The dash's look is not W0's business. Other workstreams change how the dash looks; W0's promise is only that on a clean day it adds no banner and no new surface — never "the dash looks exactly as today".
⑥ Slack is back (Q1/Q12): systemic alarms leave the shared lead-followup channel and go to one dedicated alerts channel + the in-app banner. No Slack for individual send failures (that stays a ring + banner, W1/W8 territory).
⑦ The decision register is CLOSED (DECISIONS_FINAL, 2026-07-12). D3 — staleness = 3× each schedule's own interval. D4 — the duplicate alarm fires at 2 sends of the same email to the same lead in a rolling 24h (not 3+): "one duplicate is already too many"; a legitimate manual resend may ring, and the owner accepts that. D5 — if the hosting month-to-date figure isn't readable live, the heads-up uses a computed estimate and says "estimated" in the message. Nothing in this EBO is open or defaulted any more.
Invariants that hold everywhere — English-only operator UI; alarms never block a click, a send, or a page (detection only); the banner is boring by default — nothing rendered when everything is clean, and it always names the specific problem, never a generic "something's wrong"; every alarm de-duplicates (one Slack message per episode, not one per polling cycle) and clears only when the next real check measures healthy; no new scheduled job slot is consumed — every check rides inside an already-scheduled run (the platform's 5-schedule cap is full); W0 never writes to a lead, a card, or the touchpoint history.
🌱 Seeded test identities + probe environment (staging only)
Every scenario is walked on staging against these stable ZZ identities — the seed re-creates them verbatim on every replay (never minted fresh), so the judge can tie each screenshot to the right card and a right-looking wrong screen fails:
• ZZ-W0-BASE Kft — the baseline lead card that must be visible on the staging board in every W0 frame (proves the walker is looking at the seeded staging board, not some other screen).
• ZZ-W0-DUP Kft — the lead seeded with the same email type sent exactly 2× in 24h — the threshold boundary itself (D4 = 2), so the seed proves the new line rather than merely clearing an old one (S6/S7: the duplicate alarm must name it and say "2×").
• ZZ-W0-REM Kft — the lead on the payment chase: one reminder per calendar day, on consecutive days — the single recognized cadence, exempt from the duplicate rule (S6.1: must NEVER be flagged as a duplicate, even though two of its sends fall inside one rolling 24h window).
All ZZ data is purged after the run. Values that change per replay (minutes since last run, dollar amounts, dates, counts) are asserted in the probes, never in the pixel checklist.
Probe environment: probes are shell commands — $BASE = the staging pipeline URL (https://matyas-staging--clientsflow-pipeline-web.modal.run), $DASH_TOKEN = the staging dash token. The only curly placeholder the walker binds is {lead_id}.

S1 — Clean day: everything healthy → the watchdog is invisible (happy path / baseline)

Who: Mátyás  ·  When: A normal working day. All five schedules ran on time, no duplicate sends, hosting spend under budget, auto-send flag off. The correct behavior of the whole alarm layer on a good day is: you can't tell it exists.

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Mátyás opens the dashboard on a healthy day No health banner anywhere — W0 adds nothing visible to a clean dash. (Whatever the dash looks like after the other workstreams land, W0's contribution on a clean day is zero pixels.) Copy: — · Look: no banner rendered at all when every check is green · Where: top of the dash (the banner's slot stays empty) The watchdog checks re-ran quietly in the background on their own schedules (all-5-schedule freshness, auto-send flag, hosting spend, duplicate-send scan) — all green, so the dash health payload's alerts list is empty and nothing renders Must NOT show a permanent "all systems OK" banner or badge (green noise trains him to ignore the slot); must NOT slow the dash load by even a beat — checks run on the server's own schedule, never on page-open — (W0 never writes history)

S2 — A second copy of the app cannot exist: the old duplicates are deleted for good, deploys are pinned (the July incident — closed by elimination, not by monitoring)

Who: System (verified by the build; no owner clicks)  ·  When: Always, from the moment W0 ships. In the 2026-07-06→07-11 incident a forgotten second copy of the app woke up in the wrong hosting workspace and, for 5 days, emailed real leads and hid cards. Owner ruling: do not build a watcher for that copy — make the copy impossible. The old apps are deleted, and the deploy path physically cannot create another one.

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Nothing — the build deletes every leftover copy of the app in the personal workspace, permanently Nothing on screen. The old addresses stay dead forever: asking either of them anything gets no answer at all (they no longer exist, rather than "existing but stopped") Copy: — · Look: — · Where: no UI change; the proof lives in the deploy log + the probe below The duplicate pipeline app and duplicate booking app are deleted from the personal hosting workspace (not merely stopped — a stopped app can be revived by a stray deploy, a deleted one cannot be woken by accident). The deletion is recorded in the ship ledger Must NOT leave a stopped-but-present copy behind (that is exactly what got revived in July); must NOT add any recurring "is the ghost back?" probe, banner, or Slack alarm — the owner cut that machinery; must NOT touch the real production apps —
2 Nothing — someone (or some future agent) runs a deploy without the pinned production workspace credentials The deploy refuses to run and says why, in English — it never falls back to whatever workspace happens to be configured in the shell. No second app is ever created Copy: the deploy script's refusal message names the missing production credential · Look: the deploy stops with a non-zero exit · Where: the terminal / the agent's deploy step The deploy script fails closed when the production workspace credentials are absent, so a bare or mis-configured deploy can no longer land in a personal workspace and mint a duplicate app — the exact 2026-07-06 mechanism. A regression test pins this behavior in the suite forever Must NOT deploy "somewhere" as a fallback; must NOT print the credentials; must NOT be bypassable by an env var set in the calling shell —

S3 — A background schedule stalls → the alarm names exactly which one, for how long, and what it costs him (failure path)

Who: System + Mátyás  ·  When: The app runs on 5 background schedules (the fast poll, the 10-minute poll, the CRM sync, the daily sweep, the morning report). Twice already (Jul 6, Jul 8) some of them silently died for days. Today one of the five — the CRM sync — isn't even watched. W0 watches all five and rings when any goes quiet.

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Nothing — one schedule (say the CRM sync) silently stops running Once it has been quiet for 3× its normal interval, a Slack alert in the dedicated alerts channel names it precisely and spells out the consequence — "⚠️ Schedule stalled: CRM sync — last ran 22 min ago (normal: every 5). Leads' CRM data is going stale." — and the dash shows the same as an amber health banner Copy: banner "⚠️ Schedule stalled: CRM sync" + the minutes + one plain consequence line · Look: amber banner strip at the dash top, above the tabs, on every tab · Where: dash top + the dedicated Slack alerts channel Every schedule stamps a "last ran" mark when it completes; the 10-minute watchdog compares all five stamps (including the CRM sync, which today is stamped but never read — that gap closes here) against 3× each one's own interval. Every alarm carries its own consequence sentence, fixed per schedule ("reminder emails are not going out", "the board is going stale", …) Must NOT watch only 4 of the 5 (today's bug); must NOT alert on a single slightly-late run (3× the interval is the line, so a busy cycle never cries wolf); must NOT say a vague "a cron is stale" — always the name + the minutes + the consequence line —
2 The schedule recovers (on its own, or after a fix run from the alarm's paste-prompt) The banner clears on its own within one watchdog cycle (≤10 min) and one recovery line lands in Slack — "✅ CRM sync running again (was quiet 41 min)." Copy: banner removed; one Slack recovery line with the total quiet gap · Look: clean dash again · Where: dash + Slack A fresh "last ran" stamp arrives; the stale signal clears; the alert de-dup marker resets so a FUTURE stall of the same schedule alarms again as a new episode Must NOT keep alerting every 10 minutes while stalled (one alert per episode, the banner carries the ongoing state); must NOT stay silent if the same schedule stalls AGAIN next week (each episode is new); must NOT require a manual "dismiss" click to clear a resolved alarm —

S4 — The auto-send master flag ever turns ON in production → alarm within 10 minutes (failure path — the July 8 incident)

Who: System + Mátyás  ·  When: On 2026-07-08 a leaked setting flipped the global auto-send flag on, and sequences emailed real leads with zero review — discovered by luck, not by an alarm. The flag's correct production state is OFF; W0 treats it being ON as a fire.

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Nothing — a deploy or a settings change flips the auto-send master flag ON in production Within 10 minutes: an urgent Slack alert — "🚨 Auto-send is ON in production. Emails can go out without your review. If this wasn't you, paste the prompt below into Claude Code." — plus a red dash banner saying the same, carrying the paste-prompt behind its "Details" link Copy: banner "🚨 Auto-send is ON in production" + Details · Look: red banner at the dash top · Where: dash top + the dedicated Slack alerts channel The 10-minute watchdog reads the flag's live production value each run; ON → one alert + banner, both carrying a paste-prompt composed from the live fact. W0 does not flip the flag back (it never intervenes) — it makes the state impossible to miss Must NOT auto-flip the flag off (one day he may genuinely mean it — the alarm asks, never decides); must NOT alert repeatedly while it stays on (one alert + the persistent banner); must NOT take days to notice (the July 8 failure) —

S5 — Hosting spend drifts past the ~$30/month budget → one heads-up, and nothing is ever throttled (guardrail — owner ruling: soft budget, never a cap)

Who: System + Mátyás  ·  When: Owner ruling (2026-07-12): the ~$30/month figure covers hosting only, and it is a soft budget, not a hard cap — "we can go overboard… the app should never experience slowdown or pausing". So this is a single informational heads-up, nothing more. AI spend is NOT part of this check. The one thing that must never happen again: the hosting platform hitting its own billing limit and silently rejecting real lead webhooks for hours (which happened once).

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Nothing — a normal month, hosting spend tracking under the ~$30 budget Nothing. No banner, no Slack line about money, all month. The app runs at full speed the entire time Copy: — · Look: — · Where: — (spend detail stays where it already lives, on the Usage tab) The 10-minute watchdog reads the hosting month-to-date figure (or, if it isn't readable live, a computed estimate — D5) and compares it with the ~$30 budget. Under budget → no signal. Separately, the hosting account's own billing limit is raised far above the budget, so the platform itself can never start rejecting work Must NOT surface routine spend numbers as alarms; must NOT include AI/Gemini spend in this figure (hosting only — owner ruling); must NOT leave the hosting platform's own billing limit anywhere near $30, where it could reject a real lead's webhook —
2 Nothing — hosting month-to-date drifts past the ~$30 budget One Slack heads-up — "💸 Hosting spend passed the $30/month guide: $32.40 month-to-date (day 19, ≈ $51 at this pace). Nothing has been throttled — this is FYI." — and an amber note on the dash health banner. Everything keeps running at full speed Copy: banner "💸 Hosting spend passed the $30/month guide" + the amount + the month-end pace projection · Look: amber banner · Where: dash + the dedicated Slack alerts channel The threshold-crossing fires exactly once per month; the marker remembers it fired, and resets automatically on the 1st. If the figure is an estimate rather than a live reading, the message says so Must NOT re-alert every 10 minutes while over budget; must NOT throttle, pause, block, degrade, or slow down ANY function, send, page or AI call to save money (owner ruling — the budget is a signal, never an enforcement); must NOT forget to reset the marker at month roll-over (or month 2 gets no heads-up) —

S6 — The same email hits the same lead repeatedly → detected within ~10 minutes, alarm names the lead and the email (failure path — now the primary net for a rogue sender)

Who: System + Mátyás  ·  When: The July incident's most damaging symptom was double-created drafts and repeated emails to the same leads, all week, undetected. Whatever the cause — a retry bug, a stuck schedule, anything — the same email hitting the same lead repeatedly must ring on its own. Since the ghost-app probe is gone (owner ruling ①), this scan is the net that catches a rogue sender by its effects, so it runs on the 10-minute watchdog, not once a day (owner: "apply" on the too-slow critique). Threshold (D4, settled): 2 sends of the same email type to the same lead inside a rolling 24h. One duplicate is already too many. The single exemption is the payment chase's designed cadence (one reminder per calendar day, W1's named 3-reminder constant); every other repeat rings — including a legitimate manual resend, which the owner explicitly accepted.

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Nothing — a normal day, including a lead legitimately in the daily payment-reminder chase Nothing — the scan runs silently every 10 minutes and finds no repeats. The payment chase's one-reminder-per-calendar-day cadence is never flagged, even when two of its reminders land inside one rolling 24h window Copy: — · Look: — · Where: — Every 10 minutes the watchdog scans the sent-email record for the same email type going to the same lead 2+ times inside a rolling 24h window (D4 = 2, settled). The payment-chase reminder template is the ONE exempt cadence: at most one such send per calendar day is by design (W1's named 3-reminder constant) and never counts as a duplicate — none found here, no signal Must NOT flag the payment chase's designed cadence (its exemption is what keeps this row green at a threshold of 2); must NOT widen that exemption to any other template — every other same-email repeat inside 24h rings, a manual resend included (owner-accepted) — (W0 reads the send record; it never writes it)
2 Nothing — something sends the same email to the same lead a second time inside 24h (the seeded ZZ-W0-DUP lead: 2× the same template) Within ~10 minutes a Slack alert naming both — "🚨 Duplicate sends: 'Follow-up #2' went to ZZ-W0-DUP Kft 2× in the last 24h. Possible retry bug or a second sender — open the lead's history." — plus a red dash banner with the lead's name, whose "Details" link carries the paste-prompt and a link straight to that lead's card Copy: banner "🚨 Duplicate sends detected — ZZ-W0-DUP Kft got the same email 2×" + Details · Look: red banner at the dash top · Where: dash + the dedicated Slack alerts channel The 10-minute scan found 2 sends of the same template to the same lead in a rolling 24h — the threshold (D4 = 2) — and raised one alert for the episode. Detection only — W0 does not retract, pause, or block anything (send gating is W1's territory) Must NOT wait for a third copy before ringing (3+ was the old default; 2 is the settled line); must NOT block or delay the send pipeline in any way; must NOT report the duplicate anonymously ("duplicates found") — always the lead name + which email + how many times; must NOT re-alert every 10 minutes for the same episode The duplicate rows are already in the lead's touchpoint history (written by the send path) — that history is exactly the evidence the alarm points him at. W0 adds nothing to it.

S7 — Every prod-misbehavior alarm carries the exact paste-prompt for Claude Code; nothing ever rolls back on its own (guardrail — Q25)

Who: Mátyás  ·  When: Any alarm above has fired about production misbehaving (a stalled schedule, auto-send armed, duplicate sends). His ruling: NO auto-rollback, ever — the system notifies and hands him the exact prompt to paste into Claude Code, and HE decides whether to fire it. This scenario is the shape of that payload, across every alarm.

#You doYou should see Element that changes
copy · look · where		 What changes underneathMust NOT happen 🕓 Touchpoint history
1 Mátyás reads any prod-misbehavior alarm (the Slack message, or the banner's "Details") Below the plain-language description, a ready-to-copy block: "Paste this into Claude Code →" followed by a complete, self-contained prompt naming the actual thing that broke — e.g. "Duplicate emails are going out: 'Follow-up #2' reached ZZ-W0-DUP Kft 2× in 24h on production. Find what is sending them, stop the repeats, and tell me which leads were affected." Copy: one copyable prompt block per alarm, in English, with no placeholders left unfilled · Look: a code block in the Slack message; a copy button in the banner's Details view · Where: inside every systemic alarm from S3 / S4 / S6 Each check composes its paste-prompt from the live facts it just measured (which schedule, which lead, which email) at alarm time — not a generic template. Each prompt describes the problem and the desired end-state, never step-by-step commands — so it cannot rot as the codebase moves; Claude Code works out the current steps fresh each time Must NOT execute any rollback, stop, or fix automatically (Q25 — the human fires the prompt, always); must NOT hand him a vague prompt ("something is wrong, investigate") — it names the exact object and the exact desired end-state; must NOT bake in step-by-step commands that go stale —
2 Mátyás pastes the prompt into Claude Code and lets it run Claude Code performs the fix; within one watchdog cycle the relevant banner clears itself and the recovery line lands in Slack — confirmation comes from the watchdog re-measuring reality, not from anyone declaring success Copy: the standard recovery lines from S3 / S4 / S6 · Look: banner clears on its own · Where: dash + Slack The watchdog keeps polling on its normal schedule; the all-clear appears only when the next real measurement is actually green Must NOT clear the banner because a fix was attempted — only because the next check measured healthy (the "deploy said success but wasn't" lesson from July 6) —
🕓 Touchpoint-history note for W0:
W0 is a pure observer — no scenario in this document ever creates, edits, or deletes a touchpoint-history entry, a lead, or a card. Every history column above is "—" by design. The one place history appears at all is S6, where the duplicate rows the send path already wrote are the evidence the alarm points at.

Work-item → scenario-step mapping

Each build work item for W0, and the scenario steps that prove it works when exercised on staging (failure modes are forced with ZZ-sentinel fixtures on staging only — never against real prod alarms; the seeded identities are the constants ZZ-W0-DUP Kft and ZZ-W0-REM Kft, re-created verbatim on every reseed).

Work itemWhat it deliversProven by (step ids)
WI-1 · Single-app guaranteeThe leftover duplicate pipeline + booking apps are deleted from the personal workspace (not stopped); the deploy script fails closed without the pinned production workspace credentials, with a regression test locking it. No ghost probe, no ghost alarm, no ghost banner.S2.1 · S2.2
WI-2 · Full schedule-staleness watchAll FIVE schedules watched (closing the unwatched-CRM-sync gap); stale = quiet for 3× its own interval; the alert names the schedule + the minutes + a fixed plain-language consequence line; recovery auto-clears and re-arms per episode.S3.1 · S3.2 · S1.1
WI-3 · Auto-send flag sentinelThe production auto-send flag is read every 10 minutes; ON → one urgent alert + red banner + paste-prompt; never auto-flipped back.S4.1
WI-4 · Hosting-spend heads-up (soft budget)Hosting month-to-date (AI spend excluded) vs the ~$30 soft budget → exactly one heads-up per month at the crossing, with the pace projection and an "estimated" caveat if the figure isn't live; the hosting account's own billing limit raised far above the budget so it can never reject a webhook; nothing is ever throttled.S5.1 · S5.2
WI-5 · Duplicate-send scan (10-minute)Every 10 minutes, scan for the same email type reaching the same lead 2+ times in a rolling 24h window (D4 = 2, settled — the threshold lives in ONE named constant); the alert names lead + email + count and deep-links the card; the payment chase's one-per-calendar-day cadence is the single exemption and is never flagged; detection only.S6.1 · S6.2
WI-6 · Health banner + health-payload extensionThe dash health payload gains an alerts list (staleness incl. CRM sync, auto-send flag, hosting spend, duplicates — each with severity, message, paste-prompt); one banner slot renders red/amber with the specific reason, renders nothing when the list is empty, and clears only by re-measurement.S1.1 · S3.1 · S3.2 · S4.1 · S5.2 · S6.2 · S7.2
WI-7 · Dedicated Slack alerts channelSystemic alarms move out of the lead-followup channel into one dedicated alerts channel; one message per episode, one recovery line; no Slack for individual send failures.S3.1 · S3.2 · S4.1 · S5.2 · S6.2
WI-8 · Paste-prompt alarm payloadsEvery prod-misbehavior alarm composes a specific, self-contained, end-state-phrased Claude Code prompt from the live facts; NO auto-rollback anywhere; the all-clear only ever comes from a fresh measurement.S7.1 · S7.2 · S4.1 · S6.2
Sign-off — acceptance oracle. By signing, Sarudi Mátyás locks scenarios S1 … S7 and the invariants above as the acceptance answer key for the W0 Observability & Watchdog workstream — the rider that ships FIRST, before any other workstream's "shipped" claim is trusted. The decision register is closed — D3/D4/D5 are settled in DECISIONS_FINAL and folded into the rows above; this EBO carries no open questions. No W0 code is built or deployed before this signature.
Sarudi Mátyás  ✔ FINAL · 2026-07-12 · awaiting sign-off