{"slug": "how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free", "title": "How to Pull Structured Public Data into n8n Without Community Nodes (2 Free Templates)", "summary": "A developer open-sourced two n8n workflow templates that pull public web data and podcast transcripts into automation pipelines using only native core nodes — Schedule Trigger, Edit Fields, HTTP Request, Code, Google Sheets and Slack — with no community node packages. The templates call Apify Actors through the synchronous run-sync-get-dataset-items endpoint and pass a maxTotalChargeUsd parameter to enforce a hard spending cap at the API gateway, defaulting to $0.10 per run. One template extracts Podcasting 2.0 <podcast:transcript> tags from RSS feeds to feed RAG pipelines without paying for speech-to-text, with the first 25 saved items per run free and pay-per-result pricing of $2.00 per 1,000 items beyond that.", "body_md": "When automating workflows in [n8n](https://n8n.io), bringing in public web data is often where setups become fragile. Custom Puppeteer or Playwright scripts break on site updates, headless browsers consume heavy system memory on self-hosted instances, and custom community nodes can create dependency issues or fail security reviews in production.\n\nA cleaner architectural pattern is to decouple data collection from workflow orchestration: let specialized, serverless cloud extractors handle the fetching, parsing, and schema normalization, while n8n handles what it does best: conditional logic, deduplication, and routing data into Google Sheets, Slack, databases, or AI vector pipelines.\n\nIn this tutorial, we will look at how to connect n8n directly to public Apify Actors using **only native core n8n nodes** (`Schedule Trigger`, `Edit Fields / Set`, `HTTP Request`, `Code`, `Google Sheets`, and `Slack`). Zero custom community nodes required. Everything runs seamlessly on both self-hosted n8n and n8n Cloud.\n\nWe have open-sourced two production-ready workflow templates on GitHub that you can import with a single click:\n\n`<podcast:transcript>` tag) for RAG pipelines and content research—without paying for audio speech-to-text models.\n**Full Transparency & Disclosure:** I am the author of these free, open-source n8n templates ([GitHub Repository](https://github.com/AHMEDFOUAAD/n8n-apify-data-templates)) and the maintainer of the two Apify Actors they invoke on the Apify Store. Every run's first 25 saved items are 100% free with $0 in Actor run charges. The default template configurations stay within this free tier. Beyond 25 items per run, usage is pay-per-result ($2.00 per 1,000 items; the Actor's Pricing tab on Apify is the authoritative source). Both workflows include hard cost caps to guarantee no unexpected charges occur.\n\nMany developers assume integrating a scraping platform into n8n requires installing a community node package. While community nodes exist, relying on them introduces several tradeoffs:\n\nBy using n8n's native **HTTP Request** node calling the synchronous Apify API endpoint, your workflow remains completely lightweight and portable:\n\n```\nhttps://api.apify.com/v2/acts/{actorId}/run-sync-get-dataset-items\n```\n\nTypically, running an asynchronous cloud task requires a webhook receiver or polling loop. However, for batch extractions that complete within standard timeout windows (under 60-120 seconds), Apify's `run-sync-get-dataset-items` API runs the Actor, waits for completion, and returns the final JSON dataset array directly in the HTTP response body.\n\nFurthermore, by appending `?maxTotalChargeUsd={{ $json.maxCostPerRunUsd }}`, Apify's execution engine enforces a hard spending ceiling at the API gateway level. Even if you accidentally configure a scrape over thousands of pages, the run automatically stops when reaching your specified limit (defaulting to `$0.10`).\n\nFeeding podcast knowledge into LLMs or RAG systems usually requires downloading audio files and running them through Whisper or another speech-to-text API. At typical API pricing ($0.36 to $1.50+ per hour of audio), transcribing daily podcasts across several shows quickly becomes expensive and computationally heavy.\n\nHowever, a large and growing number of podcast publishers (hosted on Buzzsprout, Omny Studio, Podbean, etc.) already attach human-verified or pre-generated transcripts directly to their RSS feeds using the standardized Podcasting 2.0 `<podcast:transcript>` tag (in VTT, SRT, JSON, HTML, or plain text formats).\n\n```\n[Schedule Trigger (Daily 07:00)]\n       │\n       ▼\n[Podcast Settings (Edit Fields)]\n       │\n       ▼\n[Validate Settings (Code)]\n       │\n       ▼\n[Fetch Transcripts (HTTP Request -> Apify)]\n       │\n       ▼\n[Read Saved GUIDs (Google Sheets)]\n       │\n       ▼\n[Filter New Episodes & Format (Code)]\n       │\n       ▼\n[Append New Transcripts (Google Sheets)]\n```\n\n`feedUrls`), backfill episode limits (` maxEpisodesPerFeed`, default 3), and maximum items per run (` maxTranscriptsPerRun`, default 25).` ahmed.fouaad82/podcast-published-transcripts`. It parses the feeds, resolves the linked transcript files, converts VTT/SRT timestamps into clean clean text paragraphs with speaker labels, and returns structured episode objects.`guid` column. A Code node compares incoming episode GUIDs against existing entries, ensuring only brand-new episodes are processed.`textTruncated = true`), while always preserving the permanent `transcriptUrl` link to the full raw file.\nCreate a tab named `Transcripts` with the following 15-column header row:\n\n```\nguid, publishedAt, podcastTitle, episodeTitle, episodeNumber, durationMinutes, wordCount, speakers, language, transcriptUrl, audioUrl, feedUrl, textTruncated, text, savedAt\n```\n\nKeeping tabs on competitive hiring markets and compensation trends requires monitoring dozens of company career pages. Most modern job boards are JavaScript Single Page Applications (SPAs) where raw HTML scraping fails. Furthermore, only a subset of postings include explicit numeric pay ranges.\n\n```\n[Schedule Trigger (Weekdays 08:00)]\n       │\n       ▼\n[Job Watch Settings (Edit Fields)]\n       │\n       ▼\n[Validate Board Slugs (Code)]\n       │\n       ▼\n[Fetch Lever Jobs (HTTP Request -> Apify)]\n       │\n       ▼\n[Read Known Job IDs (Google Sheets)]\n       │\n       ▼\n[Filter New Postings & Format (Code)]\n       │\n       ▼\n[Append Postings (Google Sheets)]\n       │\n       ▼\n[Send Digest (Slack)]\n```\n\n`leverdemo`, or direct URLs `https://jobs.lever.co/company`), minimum salary threshold (` minSalary`), and remote filters (` remoteOnly`).` ahmed.fouaad82/lever-salary-jobs`. The Actor interfaces with Lever's public career APIs, extracts structured compensation objects (currency, minimum, maximum, interval), and returns only qualifying postings.`jobId` column from Google Sheets. The first run populates the sheet; subsequent runs filter strictly for newly discovered postings.`#jobs` (up to 15 highlighted roles with direct application links).\nCreate a tab named `Salary watch` with the following header row:\n\n```\njobId, board, title, team, department, location, workplaceType, commitment, salaryMin, salaryMax, salaryCurrency, salaryInterval, salaryRange, postedAt, jobUrl, firstSeenAt\n```\n\nSetting up either template takes approximately 5 minutes. Here is the complete checklist:\n\n`Authorization`\n`Bearer YOUR_APIFY_API_TOKEN`\n`Apify Header Auth`.` podcast-transcripts-to-google-sheets/workflow.json``lever-salary-watch-sheets-slack/workflow.json`` Ctrl+V` / `Cmd+V`).` Podcast settings` or `Job watch settings`).`/d/` and `/edit`).\nWhen automating scraping, safeguarding against accidental runaway costs is critical. These templates implement a two-layer defense:\n\n`maxTotalChargeUsd` is dynamically bound to `{{ $json.maxCostPerRunUsd }}` (defaulting to `$0.10`). If a run attempts to backfill hundreds of episodes or thousands of postings beyond your budget, Apify terminates the job automatically before the charge can exceed 10 cents.\nBy combining n8n's visual orchestration engine with Apify's serverless scraping actors through standard HTTP requests, you get the reliability of a managed scraping platform without the operational overhead of maintaining custom headless browsers or third-party community nodes.\n\nAll resources are free and open source under the MIT license:\n\nIf you build on top of these templates—such as connecting the transcript output directly to a LangChain / Pinecone vector node for interactive Q&A—feel free to share your fork or open an issue on the GitHub repository!", "url": "https://wpnews.pro/news/how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free", "canonical_source": "https://dev.to/hermes_820f031ccd971d4dc5/how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free-templates-1mp0", "published_at": "2026-11-14 10:12:51+00:00", "updated_at": "2026-11-14 10:20:36.044009+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents"], "entities": ["n8n", "Apify", "GitHub", "Google Sheets", "Slack", "Buzzsprout", "Omny Studio", "Podbean"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free", "markdown": "https://wpnews.pro/news/how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free.md", "text": "https://wpnews.pro/news/how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free.txt", "jsonld": "https://wpnews.pro/news/how-to-pull-structured-public-data-into-n8n-without-community-nodes-2-free.jsonld"}}