{"slug": "from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-t", "title": "From Visual Workflows to Native Code in Production: The Complete Journey of an n8n Backend That Couldn't Stop Evolving", "summary": "A developer built a GitOps-based CI/CD pipeline to synchronize and version n8n workflows between staging and production environments for CoreAutoCRM, an auto repair shop management SaaS operating via WhatsApp. The solution downloads workflow JSONs from n8n's REST API into a Git repository, enables AI-assisted editing, and automatically publishes approved changes to production, ensuring traceability and consistency across 74 workflows.", "body_md": "I'll tell this story in the order it happened. It doesn't start with a compiler. It starts with two n8n servers and a synchronization problem I had to solve before I could sleep peacefully.\n\nCoreAutoCRM is an auto repair shop management SaaS operating entirely via WhatsApp. AI customer service, scheduling, quotes, yard management, intelligent follow-ups—all of this lives in n8n workflows. I chose n8n for the obvious reason: speed. In a few months, I went from zero to a real production product, built entirely alone.\n\nn8n is great for that. The problem starts when you have a real product, with a real client, and you need two environments: one to test without fear of breaking what works, and another for what the client actually uses.\n\nStaging and production. Two separate n8n servers.\n\nAnd then comes the problem nobody mentions: **how do you ensure that what you tested in staging is exactly what goes to production?**\n\nIn n8n, a workflow lives inside the server's database. It's not a file. It has no history. It has no diff. If you open the visual editor and change a `Switch`\n\ncondition, a SQL query parameter, or a node's AI prompt—that change exists only in that server's UI. There's no way to know what changed, who changed it, or when. There's no way to revert. There's no way to review before deploying to production.\n\nThe \"manual\" way to sync staging with production would be: open each workflow in staging, export the JSON, open the same workflow in production, import the JSON. With 74 workflows, this is unfeasible. And even if it were, it would be a manual process prone to human error—the kind of thing that wakes you up at 2 AM because someone (you) forgot to sync a critical workflow.\n\nI needed a solution that:\n\nThis didn't exist out of the box. I built it from scratch.\n\nn8n's REST API has endpoints for everything: listing workflows, fetching a specific workflow's JSON, creating, updating, activating, deactivating. What I needed was an automation that used this API in both directions.\n\nI built a synchronization skill that does the following:\n\n**Direction 1 — n8n → Git (download):**\n\nThe automation connects to the staging n8n server via API, downloads each workflow's JSON, organizes them into a folder structure in the local project (`/agente/workflows/`\n\n), and prepares a merge request with the changes. I can do this at any time—after a development cycle in the visual editor, when I want to \"commit\" the backend's current state.\n\n**Direction 2 — Git → n8n (upload):**\n\nThe reverse also works. I can edit a JSON directly in the repository—with AI assistance, since it's just structured text—and publish that change back to the n8n server. For quick tweaks, editing the JSON is sometimes faster than navigating the visual interface.\n\n**The CI/CD pipeline:**\n\nWhen a merge request is approved in Git, the CI/CD pipeline kicks in automatically. It uses the production n8n API to publish each changed workflow directly to the production server, with no manual intervention. What was tested in staging is exactly what hits production—because it's the exact same JSON, versioned, reviewed, and approved.\n\nThe result was a setup I didn't expect to be so good: **visual backend development with traditional software engineering discipline**. Each backend feature became a branch. Each change had a readable Git diff. Each deploy had full traceability. And all this without giving up the n8n visual editor, which remained the most productive tool for building and testing flows quickly.\n\nWith this structure running, something important happened quietly: I now had 74 workflows—the entire product backend at the time—as structured, updated, versioned, and synced JSON files in the repository. The backend source code existed in a machine-readable format.\n\nI still didn't know what to do with this besides versioning. But I was about to find out.\n\nThe GitOps structure completely solved governance. I never lost trace of a change again. I never feared syncing staging with production again. The pipeline worked beautifully.\n\nMeanwhile, the product kept growing. The original 74 workflows became 126—new features, new AI modules, new operational flows. All versioned, all synced, all going through the same pipeline. The GitOps structure scaled naturally with the product's growth.\n\nBut n8n was still the production runtime. And as the volume grew, the cost of that became more visible.\n\nEvery n8n node serializes and deserializes the entire state between executions. It's the inherent cost of any visual orchestration engine that needs to be generic enough to serve everyone. Every subworkflow call—and I had many, because my n8n microservices architecture used subworkflows heavily—turned into an internal HTTP call with authentication, serialization, and transport overhead. Result: 180ms to 450ms average latency, 1.2 GB to 2.5 GB of RAM per instance.\n\nFor a SaaS that handles real-time WhatsApp messages, this is a real problem. The feeling of \"taking too long to respond\" starts showing up in the product before you have enough volume to justify a traditional rewrite.\n\nThe conventional options:\n\n**Rewrite in native code.** Six months of work, a complete halt on new features, a risky migration, and—most importantly—the end of the visual development speed that got me where I was. Discarded.\n\n**Scale horizontally with more n8n instances.** Multiplies an already high cost without solving per-request latency. Discarded.\n\nAnd then the realization that changed everything: **I already had the 74 workflows as structured JSON in the repository**. If there is a program capable of reading these JSONs and understanding what each workflow does—what each node receives, processes, and outputs—that program can generate equivalent TypeScript code. Code that has no serialization between nodes. That makes no internal HTTP calls. That runs straight in the process, with no orchestration engine in the middle.\n\nThe GitOps I built to solve versioning had inadvertently created the prerequisite for the next step: a compiler.\n\nThe first version of the compiler followed the nodes in the order they appear in the JSON. It broke immediately.\n\nVisual workflows have no linear order. A JWT authentication node might appear at position 3 in the JSON, but needs to execute before the database node at position 1. An `If`\n\nnode triggers only one of two branches. A node receiving input from two different branches must wait for at least one of them to execute.\n\nVisual workflows are Directed Acyclic Graphs—DAGs. The only way to correctly resolve the execution order is with topological sorting.\n\nI used Kahn's Algorithm: start with nodes that have no predecessors (the triggers), process them, mark them as resolved, free the nodes that depended on them, repeat. The result is a linear queue ensuring every node only executes after all its predecessors have already executed.\n\nThe compiler also identifies SINK nodes—nodes that end execution and return the HTTP response (`Respond to Webhook`\n\n). These nodes are placed at the end of the topological queue, allowing the response to the client to be returned in less than 5ms while secondary asynchronous tasks continue in the background.\n\nMy first implementation of subworkflows used standard ES Module imports. It worked in development. It broke silently in the production bundle.\n\nThe problem is twofold.\n\nFirst: ES modules with relative paths of different depths can result in separate `Map`\n\ninstances in the same process. Two modules thinking they share the same registry might be talking to different registries with no visible errors.\n\nSecond: bundlers like `bun build`\n\nand `esbuild`\n\nperform tree-shaking—removing code unreferenced in static analysis. Subworkflows called dynamically by name (`executeSubworkflow(\"COREAUTOCRM-PANEL-ACTION-GET-OS-DETAILS\", ...)`\n\n) are invisible to the bundler. The name is a string at runtime. The bundler doesn't know that string corresponds to a function. The bundle reached production without the subworkflows, and the dynamic calls failed silently.\n\nSolution for both problems at once: Global Singleton Registry tied to `globalThis`\n\n.\n\n``` js\nconst g = globalThis as any;\nif (!g.__COREAUTO_WORKFLOWS_REGISTRY__) {\n  g.__COREAUTO_WORKFLOWS_REGISTRY__ = new Map<string, Function>();\n}\nexport const workflowsRegistry = g.__COREAUTO_WORKFLOWS_REGISTRY__;\n```\n\n`globalThis`\n\nis guaranteed to be unique per process. To solve tree-shaking, the compiler automatically injects a static export referencing all 126 workflows, forcing the bundler to include them all in the bundle.\n\nn8n uses its own expression syntax: `={{ $json.body.osId }}`\n\n, `={{ $('NodeName').item.json.field }}`\n\n, `Text: {{ $json.name }}`\n\n. Converting this to TypeScript has three distinct problems.\n\nFirst is safe chaining. `$json.user.store_id`\n\ncrashes with a `TypeError`\n\nif `user`\n\nis null. The compiler needs to convert to `item?.json?.user?.store_id`\n\nfor paths of arbitrary depth.\n\nSecond is mixing text and expressions. `\"Text: {{ $json.name }}\"`\n\nmust become ``Text: ${item?.json?.name ?? ''}``\n\n—a template string with a fallback.\n\nThird, I didn't expect: AI prompts. The prompts feeding Gemini in the workflows contain Markdown code blocks with triple backticks. When the compiler places these prompts inside TypeScript template strings, the internal backticks break the syntax. The parser had to learn to identify these tokens and escape each internal backtick with `\\`\n\n`.\n\nThe solution was a real tokenizer—not a regex over the entire text, but a parser that identifies the `{{`\n\nand `}}`\n\ndelimiters, extracts the expression content, converts n8n's grammar to TypeScript, and reconstructs the text with proper escaping for each context.\n\nGenerating code that compiles is not enough. I need to guarantee that the generated code produces exactly the same result n8n would with the same data.\n\nThe n8n API exposes the full execution history: `GET /api/v1/executions`\n\n. For each execution, it returns the input payload, intermediate data for each node, and final output payload.\n\nThe `TestGenerator`\n\nuses this API as an oracle: I run a workflow in staging n8n with real data, the generator fetches this execution, captures input and output for each node, and generates a `.test.ts`\n\nfile that verifies the compiled TypeScript function produces an identical output, field by field.\n\nIf any field diverges, the test fails, and the CI blocks the deploy. Staging n8n isn't just the visual IDE—it's the correctness oracle for everything going to production.\n\n| Metric | n8n on VPS | Compiled Fastify | Difference |\n|---|---|---|---|\n| Average Latency | 180ms – 450ms | 3ms – 8ms | ~35x faster |\n| RAM per instance | 1.2 GB – 2.5 GB | 80 MB – 120 MB | ~95% less |\n| Throughput per vCPU | ~120 req/s | > 4,500 req/s | ~37x more scale |\n| Compile Time (126 workflows) | — | 77ms | Instantaneous |\n\nThe production bundle is 2.6 MB. It boots with PM2, sits behind Nginx, and is deployed automatically by CI when all tests pass.\n\nLooking back, what happened was a sequence where each solved problem created the conditions for the next step:\n\nThe problem of syncing two n8n servers forced me to build GitOps automation. GitOps gave me 74 workflows as versioned JSONs in the repo. The JSONs in the repo created the prerequisite for the compiler. The compiler turned those JSONs into native code with automatic correctness guarantees.\n\nNone of these steps were planned from the start. Each solved a real problem and created, as a side effect, the infrastructure the next step needed.\n\nToday the full flow is:\n\n`bun run transpile`\n\ncompiles the JSONs into native TypeScriptn8n remains the development tool. Fastify is the production runtime. The JSONs in the repo are the contract connecting the two—and the same contract that, since Act 2, gave me governance, traceability, and peace of mind.\n\nA lean startup isn't one that chooses between speed and quality. It's the one that builds the infrastructure to have both—and, with any luck, discovers that every solved problem was the stepping stone for the next.\n\nCoreAutoCRM is already running in production across dozens of auto shops in Brazil. I built this engineering for my own ecosystem, but I know the pain of scaling automations on rigid servers keeps many CTOs and founders awake at night.\n\nMy main focus is the expansion of my SaaS, making my technical schedule almost non-existent. However, I've decided to open an exclusive time window this month to structure the architecture of only 2 operations that need to solve this exact problem: scale n8n to millions of requests, cut AWS/VPS costs by 90%, and drop latency to milliseconds.\n\nIf your n8n is eating up your server's memory, if your automation crashes during peak hours, or if you've simply hit the Low-Code ceiling and don't want to rewrite everything from scratch, text me on WhatsApp ([https://wa.me/556296232227](https://wa.me/556296232227)). Let's do a quick X-ray of your infrastructure and solve this.", "url": "https://wpnews.pro/news/from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-t", "canonical_source": "https://dev.to/rogeriomaciel/from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-n8n-backend-that-508j", "published_at": "2026-08-09 14:54:01+00:00", "updated_at": "2026-08-09 15:17:49.342086+00:00", "lang": "en", "topics": ["developer-tools", "mlops", "ai-products"], "entities": ["CoreAutoCRM", "n8n", "Git"], "alternates": {"html": "https://wpnews.pro/news/from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-t", "markdown": "https://wpnews.pro/news/from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-t.md", "text": "https://wpnews.pro/news/from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-t.txt", "jsonld": "https://wpnews.pro/news/from-visual-workflows-to-native-code-in-production-the-complete-journey-of-an-t.jsonld"}}