{"slug": "cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems", "title": "Cloudflare went down 9 times this week. My ghost wrote it 9 poems.", "summary": "A developer built the Downtime Ghost, a Bluesky bot that polls 37 official status pages hourly and uses a Gemma 3 4B model running via Ollama on a free GitHub CPU runner to write a poem whenever a service goes down and announce when it recovers. Over its first week the bot logged 22 outages and 16 recoveries, including nine poems for Cloudflare, with each thread quoting one line from the status page verbatim and closing with a timestamped Browserbase screenshot. The hourly pass runs as a Temporal workflow with one heartbeating activity per status feed, and incidents and poems are stored with embeddings in Tiger Data for semantic search.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge: Ghost in the Machine](https://dev.to/challenges/hacktoberfest-week4-2026-10-26).*\n\nAhoy, the packets stalled and slow,\n\nCloudflare, where did the data go?\n\n— The Downtime Ghost, 1:29 am Wednesday, about Network Performance Issues in Ashburn, IAD\n\nIs it down, or is it me? Everyone has typed it. You refresh, you blame the wifi, you restart the router, and twenty minutes later you find the status page that has said \"Elevated errors\" the whole time.\n\nSo I built a ghost that does the refreshing for me, and then rubs it in. Every hour it reads the official status pages of 37 services. When one goes down, a Gemma 3 4B on a free GitHub runner writes it a poem. When it comes back, the ghost announces the resurrection. All of it goes out on Bluesky as [@downtimeghost.bsky.social](https://bsky.app/profile/downtimeghost.bsky.social), where it has been haunting since Tuesday night.\n\nHere's how the week went. 22 hauntings. 16 resurrections. Nine poems for Cloudflare alone. One worker I killed on purpose to see if Temporal would save the night (it did). And one Throwback Thursday the ghost missed completely.\n\n| **Watches** | 37 official status feeds, hourly: AWS, Cloudflare, GitHub, Google, Apple, PayPal, Azure, Discord, Zoom, Shopify, OpenAI, Claude and 25 more | \n| **Gemma 3 4B** | Writes every poem through Ollama on GitHub's free CPU runner. No API, no account, no bill | \n| **Trust rule** | The poem can be silly. The quote can't: one line from the status page, word for word, linked | \n| **Temporal** | The hourly pass is a workflow with one heartbeating activity per status page; a dead worker is replaced mid-haunting | \n| **Browserbase** | Each thread closes with a real-browser screenshot of the live status page, timestamped | \n| **Tiger Data** | The graveyard: every incident and poem stored with an embedding, searchable by meaning | \n| **Result** | 22 hauntings, 16 resurrections, 3 graves, 11 quotes kept word for word, 0 posts written by me | \n\n**The Downtime Ghost is a Bluesky bot that notices when a big service breaks, writes it a poem with one true line in it, and tells you when it rises again.**\n\n**Who it's for:** anyone who has ever typed \"is X down\" into a search box. And, if I'm honest, me, because I find outages funnier than I should.\n\nThe ghost is live: [@downtimeghost.bsky.social](https://bsky.app/profile/downtimeghost.bsky.social). Everything below is on the timeline, in order, with the status page linked under each one.\n\nIts first post, Tuesday at 9:28 pm Montreal time:\n\n👻 An ode to Cloudflare, down since Oct 28, 2026, 00:17 UTC Oh, Cloudflare, a shimmer, a slight pause, Your Durable Objects wrestled with cause, Queues and Workflows, a moment's delay, Source: www.cloudflarestatus.com\n\n—[The Downtime Ghost 👻 (@downtimeghost.bsky.social)](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm?ref_src=embed) [2026-10-28T01:28:31.000Z](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm/post/3myvlxmr2gt22?ref_src=embed)\n\nFour hours later Cloudflare rose. Then it went down again. And again. By Friday the ghost had written Cloudflare nine poems, from a haiku to a sixteen-line epic, and no other service came close. Cloudflare is either very busy or very honest.\n\n👻 A sea shanty for Cloudflare, down since Oct 28, 2026, 04:39 UTC Ahoy, the packets stalled and slow, Cloudflare, where did the data go? Ashburn's network, a curious flow, Making connections rather low! We waited patiently, you see, Source: www.cloudflarestatus.com\n\n—[The Downtime Ghost 👻 (@downtimeghost.bsky.social)](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm?ref_src=embed) [2026-10-28T05:29:01.000Z](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm/post/3myvzfojdes25?ref_src=embed)\n\nThe thread's last post is the receipt:\n\nIs Cloudflare down? Its status page has reported “Network Performance Issues in Ashburn, IAD” since Oct 28, 2026, 04:39 UTC. Screenshot taken Oct 28, 2026, 05:28 UTC. Source: www.cloudflarestatus.com\n\n—[The Downtime Ghost 👻 (@downtimeghost.bsky.social)](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm?ref_src=embed) [2026-10-28T05:29:03.000Z](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm/post/3myvzfqgzbr24?ref_src=embed)\n\nMy favourite of the week is a haiku about one button:\n\nZoom's button's gone now,\n\nEvaluations hide from view,\n\nFixes soon will bloom.\n\nThat's about \"Service Degradation Affecting the Evaluations Acknowledge Button in Zoom Quality Management in the US Region\". Somebody at Zoom had a very specific bad Wednesday. It stayed down 16 hours and 57 minutes, the longest haunting of the week that ended (Figma's has been going for more than a day and still hasn't). The shortest was AWS: API Gateway in Oregon, back in exactly one hour, which earned a couplet.\n\nFor Día de los Muertos, GeoCities got an elegy:\n\n🕯️ Día de los Muertos: in memoriam GeoCities (1994 to 2009) A rainbow of squares, a digital dream, Where personal pages gleamed, it would seem. Neighbourhoods of wonder, a joyful spree, Building bright worlds for you and for me. Easy-to-use, a friendly embrace, Source: web.archive.org\n\n—[The Downtime Ghost 👻 (@downtimeghost.bsky.social)](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm?ref_src=embed) [2026-10-31T15:30:39.000Z](https://bsky.app/profile/did:plc:t7i2jrk5gtyedohlbyuzoikm/post/3mz6mgadb4f2n?ref_src=embed)\n\nThe scoreboard as of Saturday afternoon, printed by the ghost's own `graveyard` command:\n\n```\n22 hauntings, 16 resurrections, 3 graves visited, 11 quotes verified word for word.\n  Cloudflare: 9\n  Claude: 2\n  Discord: 2\n  OpenAI: 2\n  AWS, DigitalOcean, Epic Games, Figma, GitHub, Shopify, Zoom: 1 each\nStill haunted: Discord, Epic Games, Figma\n```\n\nWatch the public rules pages you care about without reading them. Once a day a job fetches each page. If nothing changed, the model is never called. If something changed, a local open-weight model (Gemma 3 4B through Ollama) summarises only the changed paragraphs, and the summary is kept only if its quote appears word for word in the new page text. Otherwise you get the raw change, clearly marked unverified. Once a week you read one short markdown digest.\n\nNothing leaves your machine except the page fetches.\n\nThe write-up, with the weekend's numbers (68 page checks, 5 model calls, 4 verified quotes, 1 thrown out): [lazywatch on DEV](https://dev.to/mtlmatt/lazywatch-68-page-checks-5-model-calls-0-bylaws-read-by-me-20ej).\n\n```\nollama pull gemma3:4b\ncopy sites.example.json sites.json        # then edit the URLs\npython lazywatch.py check                 # run daily; first run only stores snapshots\npython lazywatch.py digest --days 7 > digest.md\npython\n```\n\n…\nMIT. Same repo as my Week 3 watcher; the ghost is `lazywatch_ghost.py` (about 1,150 lines, standard library plus Playwright for the cards), `statuspages.json` (the 37 feeds), `ghost_card.py` (the cards), `lazywatch_ghost_temporal.py` (the durable version) and `.github/workflows/ghost.yml` (the hourly run). 60 tests (40 for the ghost) run with no model, no account and no network. Without a Bluesky password every command is a dry run that prints what it would post.\n\n**The open piece first.** Gemma 3 4B runs through Ollama on GitHub's free runner, CPU only. A poem takes 8 to 13 seconds. The runner caches the model between runs, so an hourly haunting costs a few minutes of free compute and nothing else.\n\n**Official feeds, five dialects.** Most services use Atlassian Statuspage (`summary.json`). Google publishes `incidents.json`. AWS has a health feed that turned out to be UTF-16 with a byte-order mark, which cost me an evening. Apple has its own JSON, PayPal a components endpoint, Azure an RSS feed. Each feed is one parser of twenty lines; adding a service is one line in `statuspages.json`.\n\nGemma is asked for the verse and for one quote from the status text. The quote is checked against the incident's update and title: curly quotes straightened, case ignored, a final full stop forgiven, and it has to be at least 12 characters long. If it isn't there word for word, the ghost quotes the incident title instead. The card and the closing post both carry the page's words and a link to the page, so a reader can check the ghost in one tap. 11 of 25 poems (22 hauntings and 3 graves) kept Gemma's quote; 14 fell back to the title. The poem is the fun. The quote is the fact.\n\nStatus pages are written by tired engineers at 3 am, so the ghost never mocks them. A poem is thrown out if it uses a mean word (`useless`, `incompetent`, `clowns`…), names a person, or reads like news; it gets another try next hour. In 95 hourly runs it threw out one poem for tone, about GitHub on Wednesday morning, and tried again an hour later. The filter earned its keep in a dry run of the Thursday graves: the archived Yahoo! Answers home page carried a news ticker, Gemma quoted a line about a real person, and the rule that an archived quote must mention the site itself threw it out before anything posted.\n\nThirty-seven feeds get loud. The day I added Twilio it had eight minor incidents open. So there is a noise gate: some feeds only count major outages, an incident already open for more than 48 hours when the ghost first sees it is old news and skipped, and the hard cap is 12 posts a day. Before every post the ghost reads its own feed and refuses to repeat itself. It has no code path for replies, follows or DMs.\n\nThe hourly pass is also a Temporal workflow, `Haunting`, with one `haunt_page` activity per status page, a heartbeat every 10 seconds, a 30-second heartbeat timeout and 5 retries. The point is that a haunting survives the haunted.\n\nOn Saturday I ran it on my own PC and killed the worker on purpose. Worker 1 finished GitHub and Cloudflare, then got 11 seconds into page 3 of 37, Discord, before I killed it at 3:22:57 pm. The Temporal UI showed Discord as a pending activity, \"attempt 2 of 5\", waiting for someone to pick it up:\n\nWorker 2 came up 27 seconds after the kill. Once the heartbeat timeout ran out, Temporal handed Discord to the new worker, which started it 39 seconds after the kill. Discord had a live incident, so worker 2 wrote it a hymn. Then it carried on through the other 34 pages. All 37 finished in 8 minutes 7 seconds. GitHub and Cloudflare were already in the history, so neither was read again; the only page that ran twice was the one that died with its worker.\n\nWhat the ghost needs, and the Temporal piece that does it:\n\n| What the ghost needs | What does it | \n|---|---|\n| Run every hour without a cron I have to babysit | a Temporal Schedule ( `schedule` command) | \n| A feed that times out shouldn't stop the other 36 | one activity per page, its own retry policy | \n| A model that is still loading | retries with backoff, the page waits | \n| A worker that dies mid-haunting | heartbeat timeout, the activity moves to the next worker | \n| Never read a page twice after a crash | finished results live in the workflow history | \n\nEvery haunting thread closes with a screenshot of the status page itself, taken in a real browser through a Browserbase session at the moment of posting. 20 haunting threads this week, and 19 closed with a Browserbase screenshot; the one that didn't lost its connection mid-session and posted without it, saying \"Status page checked\" instead of \"Screenshot taken\". So the one-off \"open the status page and look\" became a repeatable workflow, hourly. The screenshot post is written for search (\"Is Cloudflare down?\"), carries the timestamp it was taken, and is the ghost's receipt: the poem says what the ghost felt; the screenshot shows what the page said.\n\nEvery incident, every resurrection and every poem lands in a Tiger Cloud Postgres with pgvector, with an embedding from embeddinggemma. `search` fuses full-text rank and vector rank with reciprocal rank fusion, so you can ask the graveyard a question in plain words.\n\nThree questions I asked it on Saturday, top three rows each:\n\n```\n> search \"which service keeps going down\"\n2026-10-29 Cloudflare: ode: Issues with Durable Objects availability in Eastern Europe\n2026-10-28 Cloudflare: sea shanty: Network Performance Issues in Ashburn, IAD\n2026-10-29 Cloudflare: hymn: Increased error rates and reduced success rates for Workflows operations\n\n> search \"Durable Objects\"\n2026-10-29 Cloudflare: ode: Issues with Durable Objects availability in Eastern Europe\n2026-10-28 Cloudflare: epic: Increased Errors for Durable Objects in the Eastern North America region\n2026-10-28 Cloudflare: ode: Increased Errors for Durable Objects in the Eastern North America region\n\n> search \"voice calls\"\n2026-10-31 Discord: hymn: Issues establishing voice call\n2026-10-30 Discord: hymn: Issues establishing voice calls\n2026-10-28 Zoom: haiku: Service Degradation Affecting the Evaluations Acknowledge Button in Zoom Quality Management in the US Region\n```\n\nThe top five answers to the first question are all Cloudflare, which is the right answer. \"Voice calls\" finds both Discord outages, singular and plural, before anything else. The third row is the vector half reaching: a Zoom button isn't a voice call, but it is a Zoom problem.\n\nA 4B model on a CPU writes fast and sometimes stops early. One sea shanty ended its first post on \"We waited patiently, you see,\" and finished the thought in the reply, which reads like a ghost being interrupted, so I let it stand. 14 of 25 quotes (22 hauntings and 3 graves) weren't on the page word for word and fell back to the title. It can't tell a big outage from a small one: a broken button in one Zoom product in one region got the same treatment as AWS's API Gateway. And it asks things like \"Cloudflare, where did the data go?\" with a perfectly straight face.\n\nThe ghost missed its own Throwback Thursday. A busy Cloudflare morning spent all 12 of the day's posts by 8:34 am, so when the six graves came due at 10 am there was no room, and every run was green, so nothing told me. I found it Friday while counting posts for this write-up. The fix reserves room under the cap for the day's graves, and the three Día de los Muertos graves on Saturday were the make-up.\n\nSaturday found two more. The first ofrenda (the weekly altar of the fallen) called Cloudflare and OpenAI \"still haunted\" when both were long back up. On busy days the cap had held back some resurrection posts, and the code dropped them instead of saving them for later, so the ofrenda thought those outages never ended. I took it down, fixed it, and the ghost posted a corrected one. Then the grave screenshots: the first came out as a broken page, because the archive's raw copy points at image servers that died with the site. Once that was fixed, the Yahoo! Answers front page from 2021 rendered perfectly, with questions about real politicians and a real trial on it. The text filters never saw it, because the problem was in the picture. That screenshot is gone, and a grave now gets one only after a person has looked at the page.\n\nIt only knows what status pages admit. A service that doesn't update its page is, to the ghost, fine. And a GitHub cron is not a clock: GitHub skipped most of the scheduled runs on this quiet new repo, so a scheduled task of mine dispatches the workflow by hand whenever the hour passes without a run.\n\nBecause a joke account shouldn't need a credit card, and because every line the ghost posts can be checked. The model is open, the runner is free, the feeds are public, the quote links to its source, and the screenshot shows the page. Anyone can fork the repo, edit `statuspages.json` and have a ghost of their own by dinner, haunting whatever they depend on.\n\nDía de los Muertos is about remembering fondly, not mourning. That's the ghost's whole manner: when Cloudflare goes down it gets a shanty, not a scolding; when it comes back, the candles go out and the lights come on; and the old web gets its elegies, read from what the home pages used to say. On Día de los Muertos it visited GeoCities, AltaVista and Yahoo! Answers, and GeoCities got the line I'd want on my own stone: \"Neighbourhoods of wonder, a joyful spree, / Building bright worlds for you and for me.\"\n\nCloudflare's packets came back, the candles went out, and the ghost went quiet until the next one. That's the whole job.\n\nWhich outage would you have wanted a poem for?\n\nI built the ghost with Claude in about four days, in a running conversation rather than a single prompt. Three moments mattered:\n\nClaude drafted this post from the ghost's logs and I edited it. The numbers above come from the account's own timeline and the `graveyard` command.\n\n**Best Use of Gemma:** Gemma 3 4B writes every poem on a free CPU runner through Ollama, 8 to 13 seconds each: 22 hauntings in nine forms, 11 quotes kept word for word, one poem thrown out for tone in 95 runs, and the elegies written from archived home pages.\n\n**Best Use of Temporal:** the hourly pass is a `Haunting` workflow with one heartbeating activity per status page (10 s heartbeat, 30 s timeout, 5 retries); a worker killed 11 s into page 3 of 37 was replaced, that page ran again as attempt 2 on the new worker 39 s after the kill, all 37 finished in 8 min 7 s, and no finished page ran twice.\n\n**Best Use of Browserbase:** every haunting thread closes with a Browserbase screenshot of the live status page, timestamped and written for search; 19 of 20 haunting threads this week closed with one, a one-off look turned into a repeatable hourly workflow; the Temporal version takes the same screenshot inside its activity.\n\n**Best Use of Tiger Data:** every incident and poem is stored in Tiger Cloud with a pgvector embedding; hybrid keyword and vector search fused by rank, so \"which service keeps going down\" returns five Cloudflare hauntings in a row, without the word Cloudflare in the query.", "url": "https://wpnews.pro/news/cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems", "canonical_source": "https://dev.to/mtlmatt/cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems-5dci", "published_at": "2026-10-31 20:05:03+00:00", "updated_at": "2026-10-31 20:22:04.030509+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "generative-ai", "large-language-models", "ai-infrastructure"], "entities": ["Cloudflare", "Bluesky", "Gemma 3 4B", "Ollama", "GitHub", "Temporal", "Browserbase", "Tiger Data"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems", "markdown": "https://wpnews.pro/news/cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems.md", "text": "https://wpnews.pro/news/cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems.txt", "jsonld": "https://wpnews.pro/news/cloudflare-went-down-9-times-this-week-my-ghost-wrote-it-9-poems.jsonld"}}