🌐 An engineering-first, AI-era SEO manual that can take someone from zero to all-rounded SEO engineer. A developer has published an engineering-first SEO manual designed to take readers from beginner to all-rounded SEO engineer, covering ranking strategies for Google Search, AI Overviews, and LLM-powered engines like ChatGPT and Perplexity. The guide reframes modern SEO as systems engineering combined with information retrieval, addressing technical performance, content systems, data experimentation, and AI visibility. It includes a full implementation playbook for building AI-ready platforms. A full-stack, engineering-first blueprint for ranking in Google search, AI Overviews, and LLM-powered engines like ChatGPT, Claude, Perplexity, and Gemini. 1. Mental Model: From SEO to SEO Engineering https://gist.github.com/starred.atom 1-mental-model-from-seo-to-seo-engineering 2. How Search Engines and LLM Search Work https://gist.github.com/starred.atom 2-how-search-engines-and-llm-search-work 3. Technical SEO for Developers: An Engineering Checklist https://gist.github.com/starred.atom 3-technical-seo-for-developers-an-engineering-checklist 4. Content SEO: Information Architecture, On-Page, and E-E-A-T https://gist.github.com/starred.atom 4-content-seo-information-architecture-on-page-and-e-e-a-t 5. AI / LLM SEO: Showing Up in AI Answers https://gist.github.com/starred.atom 5-ai--llm-seo-showing-up-in-ai-answers 6. Tooling and Stack: Free vs Paid https://gist.github.com/starred.atom 6-tooling-and-stack-free-vs-paid 7. Measurement, Metrics, and Dashboards https://gist.github.com/starred.atom 7-measurement-metrics-and-dashboards 8. Learning Path: From Beginner to SEO Engineer https://gist.github.com/starred.atom 8-learning-path-from-beginner-to-seo-engineer 9. Implementation Playbook: From Idea to AI-Ready Platform https://gist.github.com/starred.atom 9-implementation-playbook-from-idea-to-ai-ready-platform 10. What "All-Rounded SEO Engineer" Means in the AI Era https://gist.github.com/starred.atom 10-what-all-rounded-seo-engineer-means-in-the-ai-era References https://gist.github.com/starred.atom references Traditional SEO was largely about keywords, backlinks, and on-page tweaks. Modern SEO is closer to systems engineering plus information retrieval applied to the open web and LLM ecosystems. 1 https://gist.github.com/starred.atom user-content-fn-1-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 SEO engineering combines: Information retrieval IR fundamentals : How systems like Google use crawlers, indexes, ranking functions BM25, vector search, learning-to-rank to order documents. 3 https://gist.github.com/starred.atom user-content-fn-3-66ff172076fd8e7acf65a3968d005083 4 https://gist.github.com/starred.atom user-content-fn-4-66ff172076fd8e7acf65a3968d005083 Web performance and UX engineering : Core Web Vitals LCP, INP, CLS and responsive, accessible UX as first-class ranking and satisfaction signals. 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 6 https://gist.github.com/starred.atom user-content-fn-6-66ff172076fd8e7acf65a3968d005083 7 https://gist.github.com/starred.atom user-content-fn-7-66ff172076fd8e7acf65a3968d005083 Content systems : Topic modeling, programmatic SEO, internal knowledge graphs, and scalable content generation processes. 8 https://gist.github.com/starred.atom user-content-fn-8-66ff172076fd8e7acf65a3968d005083 9 https://gist.github.com/starred.atom user-content-fn-9-66ff172076fd8e7acf65a3968d005083 Data and experimentation : Continuous measurement via GSC, GA4, and dashboards; running experiments on titles, internal links, and content structure. 9 https://gist.github.com/starred.atom user-content-fn-9-66ff172076fd8e7acf65a3968d005083 10 https://gist.github.com/starred.atom user-content-fn-10-66ff172076fd8e7acf65a3968d005083 1 https://gist.github.com/starred.atom user-content-fn-1-66ff172076fd8e7acf65a3968d005083 LLM / AI visibility : Generative Engine Optimization GEO , LLM SEO, and AI Overview optimization aimed at retrieval probability in AI answers, not just blue links. 11 https://gist.github.com/starred.atom user-content-fn-11-66ff172076fd8e7acf65a3968d005083 12 https://gist.github.com/starred.atom user-content-fn-12-66ff172076fd8e7acf65a3968d005083 6 https://gist.github.com/starred.atom user-content-fn-6-66ff172076fd8e7acf65a3968d005083 13 https://gist.github.com/starred.atom user-content-fn-13-66ff172076fd8e7acf65a3968d005083 14 https://gist.github.com/starred.atom user-content-fn-14-66ff172076fd8e7acf65a3968d005083 As a full-stack engineer, SEO should be treated as a distributed optimization problem across crawlability, indexation, ranking, engagement, and AI retrieval. Think of the SEO stack in layers: | Layer | Focus | Owner Mindset | |---|---|---| | Infrastructure | DNS, HTTPS, CDN, caching, logging | DevOps / SRE | | Crawl & Render | robots.txt, sitemaps, SSR/SSG, JS behavior | Backend / Frontend | | Content & Semantics | Information architecture, on-page, schema, E-E-A-T | Product + Content + Eng | | Authority & Reputation | Links, citations, brand mentions, social proof | Marketing / Founders | | Analytics & Feedback | GSC, GA4, Looker Studio, log analysis | Data / Growth / Eng | | AI & LLM Surfaces | GEO, AI Overviews, ChatGPT/Perplexity visibility | SEO Engineering | A well-rounded SEO engineer can operate across all layers and design constraints in the codebase that guarantee SEO hygiene by default. Google describes a three-stage pipeline: crawling, indexing, ranking . 4 https://gist.github.com/starred.atom user-content-fn-4-66ff172076fd8e7acf65a3968d005083 Crawling : Googlebot discovers URLs via links, sitemaps, and URL submission.- Respects robots.txt and crawl budget. - Uses different crawlers for mobile/desktop, images, etc. Indexing : Content is parsed, canonicalized, and deduplicated. Ranking : For a query, Google uses hundreds of signals and ranking methods BM25-like term scoring, vector similarity, learning-to-rank models to order candidate documents. 3 https://gist.github.com/starred.atom user-content-fn-3-66ff172076fd8e7acf65a3968d005083 4 https://gist.github.com/starred.atom user-content-fn-4-66ff172076fd8e7acf65a3968d005083 Key implications for engineering: - Googlebot primarily sees raw HTML , not your visual UI. 4 https://gist.github.com/starred.atom user-content-fn-4-66ff172076fd8e7acf65a3968d005083 - Server-side rendering or static generation dramatically simplifies indexing. 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Correct canonicalization and deduping are crucial to avoid index bloat. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 Core Web Vitals are a small set of UX metrics Google explicitly uses as ranking signals: Largest Contentful Paint LCP , Interaction to Next Paint INP , and Cumulative Layout Shift CLS . 6 https://gist.github.com/starred.atom user-content-fn-6-66ff172076fd8e7acf65a3968d005083 7 https://gist.github.com/starred.atom user-content-fn-7-66ff172076fd8e7acf65a3968d005083 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 - LCP loading : should be ≤ 2.5 seconds for a good experience. 17 https://gist.github.com/starred.atom user-content-fn-17-66ff172076fd8e7acf65a3968d005083 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 - INP interactivity : should be ≤ 200 ms. 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 17 https://gist.github.com/starred.atom user-content-fn-17-66ff172076fd8e7acf65a3968d005083 - CLS visual stability : should be ≤ 0.1. 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 17 https://gist.github.com/starred.atom user-content-fn-17-66ff172076fd8e7acf65a3968d005083 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 These metrics are measured from real-user data Chrome UX Report, RUM libraries and are correlated with both rankings and conversions. 7 https://gist.github.com/starred.atom user-content-fn-7-66ff172076fd8e7acf65a3968d005083 17 https://gist.github.com/starred.atom user-content-fn-17-66ff172076fd8e7acf65a3968d005083 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 LLM-driven engines ChatGPT, Claude, Perplexity, Gemini, Google AI Overviews typically follow a retrieval-augmented generation RAG style pipeline. 12 https://gist.github.com/starred.atom user-content-fn-12-66ff172076fd8e7acf65a3968d005083 11 https://gist.github.com/starred.atom user-content-fn-11-66ff172076fd8e7acf65a3968d005083 Fan-out query generation : A user question gets decomposed into one or more optimized search queries. 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 Web search via Bing/Google : The system queries Bing or other backends and receives a list of candidate URLs and snippets. 19 https://gist.github.com/starred.atom user-content-fn-19-66ff172076fd8e7acf65a3968d005083 20 https://gist.github.com/starred.atom user-content-fn-20-66ff172076fd8e7acf65a3968d005083 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 Metadata screening : Results are filtered using title, URL, snippet, and freshness before any page is read. 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 Chunked retrieval : The engine reads slices of text sliding window from chosen URLs, stripping layout and JS; only text content and simple markup matter. 20 https://gist.github.com/starred.atom user-content-fn-20-66ff172076fd8e7acf65a3968d005083 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 Passage-level ranking : Instead of ranking whole pages, the engine ranks passages using dense embeddings and probabilistic scoring, then stitches them into an answer. 11 https://gist.github.com/starred.atom user-content-fn-11-66ff172076fd8e7acf65a3968d005083 12 https://gist.github.com/starred.atom user-content-fn-12-66ff172076fd8e7acf65a3968d005083 Consequences: - Presence in Bing/Google index is a prerequisite for ChatGPT, Perplexity and Gemini visibility. 19 https://gist.github.com/starred.atom user-content-fn-19-66ff172076fd8e7acf65a3968d005083 20 https://gist.github.com/starred.atom user-content-fn-20-66ff172076fd8e7acf65a3968d005083 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 - Metadata quality title, description, URL heavily influences whether your page is even fetched. 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 - Clear, modular sections and headings increase the probability that relevant passages are retrieved and quoted. 14 https://gist.github.com/starred.atom user-content-fn-14-66ff172076fd8e7acf65a3968d005083 18 https://gist.github.com/starred.atom user-content-fn-18-66ff172076fd8e7acf65a3968d005083 GEO is the discipline of optimizing for generative engines, focusing on reference probability how often content is cited rather than only ranking position. 13 https://gist.github.com/starred.atom user-content-fn-13-66ff172076fd8e7acf65a3968d005083 6 https://gist.github.com/starred.atom user-content-fn-6-66ff172076fd8e7acf65a3968d005083 Research on GEO shows: - Adding citations, quotations, and statistics in content can increase source visibility in generative answers by up to ~40%. 13 https://gist.github.com/starred.atom user-content-fn-13-66ff172076fd8e7acf65a3968d005083 - GEO is model-agnostic and treats AI systems as black boxes optimized via experiments over prompts and content variations. 13 https://gist.github.com/starred.atom user-content-fn-13-66ff172076fd8e7acf65a3968d005083 - Brands need to track how often they are mentioned or cited in AI answers, not just SERP rankings. 6 https://gist.github.com/starred.atom user-content-fn-6-66ff172076fd8e7acf65a3968d005083 If pages are not discoverable and indexable, no other optimization matters. robots.txt - Allow Googlebot and Bingbot by default. - Block internal, staging, and admin routes. - Never block /css , /js , or critical assets. XML sitemaps Meta robots & canonical tags Status codes & redirects - Ensure important URLs resolve with 200; avoid long redirect chains. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Use 301s for permanent moves; 404 or 410 for removed content. - Ensure important URLs resolve with 200; avoid long redirect chains. Internationalization - Use hreflang for multi-language/country sites. - Keep country/language mappings consistent in sitemaps and markup. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Use Log file analysis provides ground truth for how bots crawl a site and where crawl budget is wasted. 22 https://gist.github.com/starred.atom user-content-fn-22-66ff172076fd8e7acf65a3968d005083 23 https://gist.github.com/starred.atom user-content-fn-23-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Track crawl frequency, depth, response codes, and wasted hits on irrelevant URLs. 22 https://gist.github.com/starred.atom user-content-fn-22-66ff172076fd8e7acf65a3968d005083 - Tools: Screaming Frog Log File Analyser, JetOctopus/Sitebulb cloud, Splunk/Loggly, or custom pipelines. 24 https://gist.github.com/starred.atom user-content-fn-24-66ff172076fd8e7acf65a3968d005083 23 https://gist.github.com/starred.atom user-content-fn-23-66ff172076fd8e7acf65a3968d005083 22 https://gist.github.com/starred.atom user-content-fn-22-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 JavaScript-heavy SPAs can hurt SEO if not carefully handled, because search engines must render JS on a secondary wave. 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 4 https://gist.github.com/starred.atom user-content-fn-4-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 Best practices: - Favor static generation SSG or server-side rendering SSR for primary marketing and content pages. 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Ensure the initial HTML contains meaningful content and links; avoid blank shells. - Avoid client-only rendering for core content; hydrating interactivity later is fine. - Test with curl and the URL Inspection tool in GSC to see what HTML Google actually indexes. 4 https://gist.github.com/starred.atom user-content-fn-4-66ff172076fd8e7acf65a3968d005083 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 A clean, logical information architecture helps both crawlers and users navigate content. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Use shallow hierarchies : important pages within 3–4 clicks of the homepage. - Use descriptive, keyword-aligned URLs : /blog/technical-seo-checklist instead of /page123 . 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Implement breadcrumbs and hub-and-spoke structures for topic clusters. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Ensure a consistent internal link graph from hubs to subpages and between related content. Key engineering levers for Core Web Vitals: 17 https://gist.github.com/starred.atom user-content-fn-17-66ff172076fd8e7acf65a3968d005083 7 https://gist.github.com/starred.atom user-content-fn-7-66ff172076fd8e7acf65a3968d005083 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 6 https://gist.github.com/starred.atom user-content-fn-6-66ff172076fd8e7acf65a3968d005083 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 LCP improvements INP improvements- Debounce expensive event handlers. - Avoid long synchronous tasks; split work into smaller chunks. - Use web workers for CPU-heavy logic. 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 CLS improvements Monitoring: - Use the web-vitals JS library in production to collect real-user monitoring data. 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 - Use PageSpeed Insights , Lighthouse , and Chrome UX Report for lab + field data. 17 https://gist.github.com/starred.atom user-content-fn-17-66ff172076fd8e7acf65a3968d005083 5 https://gist.github.com/starred.atom user-content-fn-5-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 Security signals influence trust, usability, and indexing. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Enforce HTTPS everywhere; enable HSTS. - Clean up mixed content, insecure iframes, and open redirects. - Set robust Content Security Policy CSP and X-Frame-Options to mitigate clickjacking. - Monitor for hacked content, malware, and manual actions in GSC. 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 Structured data helps search engines interpret content and unlock rich results. 25 https://gist.github.com/starred.atom user-content-fn-25-66ff172076fd8e7acf65a3968d005083 16 https://gist.github.com/starred.atom user-content-fn-16-66ff172076fd8e7acf65a3968d005083 2 https://gist.github.com/starred.atom user-content-fn-2-66ff172076fd8e7acf65a3968d005083 - Use JSON-LD in