RAG Without Leaving Snowflake Snowflake published a six-step SQL workflow for building retrieval-augmented generation (RAG) entirely inside a Snowflake account using its Cortex suite, eliminating the need to stand up five separate systems for document question-answering. The workflow uses AI_PARSE_DOCUMENT in LAYOUT mode to convert PDFs, Word documents and scanned images to Markdown, SPLIT_TEXT_RECURSIVE_CHARACTER at a 512-character chunk size with 128-character overlap, a CREATE CORTEX SEARCH SERVICE statement with TARGET_LAG = '1 hour' and the snowflake-arctic-embed-l-v2.0 embedding model, and AI_COMPLETE with claude-sonnet-4-6 to generate answers from retrieved context only. The article warns that Cortex Search's default embedding model has a 512-token context window, so oversized chunks are silently truncated before embedding while remaining keyword-searchable, and that a search service suspends indexing after five consecutive refresh failures while continuing to serve from the frozen index. A large language model is only as current as its training data — it has never seen your contracts, your policies, or last week’s support tickets. The usual fix, Retrieval-Augmented Generation, works by fetching the relevant facts right before the model answers. No re-training. The model answers from documents you hand it, which cuts hallucination sharply on questions about your own data. The problem isn’t the concept. It’s that “standing up RAG” traditionally means standing up five separate systems just to answer questions about your own documents. Snowflake’s Cortex suite replaces each piece with a managed, in-platform equivalent: TARGET LAG handles re-indexing instead of a manual pipeline, the documents never leave the account, and there's zero additional infrastructure to patch. The whole thing runs as six steps inside one account. Here’s the SQL behind each step. CREATE OR REPLACE STAGE docs stage DIRECTORY = ENABLE = TRUE ; PUT file://./contracts/ .pdf @docs stage/contracts/;-- or point at S3 / Azure Blob / GCS with an external stage CREATE OR REPLACE TABLE parsed docs ASSELECT relative path AS file name, PARSE JSON AI PARSE DOCUMENT TO FILE '@docs stage', relative path , {'mode': 'LAYOUT'} :content::string AS parsed textFROM DIRECTORY @docs stage ; CREATE OR REPLACE TABLE doc chunks ASSELECT d.file name, c.index::int AS chunk index, c.value::string AS chunk textFROM parsed docs d,LATERAL FLATTEN input = SNOWFLAKE.CORTEX.SPLIT TEXT RECURSIVE CHARACTER d.parsed text, 'markdown', 512, 128 c; AI PARSE DOCUMENT reads PDFs, Word docs, and scanned images directly off the stage. LAYOUT mode preserves headings and tables as Markdown, which chunks far more cleanly than raw OCR text. Note that it takes a FILE object built with TO FILE rather than a stage-and-path pair, and it returns JSON as a string — hence the PARSE JSON before pulling :content. SPLIT TEXT RECURSIVE CHARACTER splits on paragraph and sentence boundaries first, only hard-cutting when it has to, with 128-character overlap to preserve context across edges. The 512-character chunk size is deliberate. Cortex Search’s default embedding model has a 512- token context window, and anything longer gets truncated before it’s embedded — while still being fully searchable by keyword, so an oversized chunk looks like it’s working. At 512 characters you’re around 128 tokens, comfortably clear of that edge. If you raise the chunk size later, check it: SELECT COUNT IF AI COUNT TOKENS 'snowflake-arctic-embed-l-v2.0', chunk text 512 AS truncated chunksFROM doc chunks; CREATE OR REPLACE CORTEX SEARCH SERVICE doc qa search ON chunk text ATTRIBUTES file name WAREHOUSE = cortex wh TARGET LAG = '1 hour' EMBEDDING MODEL = 'snowflake-arctic-embed-l-v2.0'AS SELECT chunk text, file name, chunk index FROM doc chunks ; SELECT PARSE JSON SNOWFLAKE.CORTEX.SEARCH PREVIEW 'doc qa search', '{"query": "What is our refund policy?", "columns": "chunk text", "file name" , "limit": 5}' 'results' AS hits; SELECT AI COMPLETE 'claude-sonnet-4-6', CONCAT 'Answer using ONLY this context:\n', :retrieved chunks, '\n\nQuestion: ', :user question AS answer; One CREATE CORTEX SEARCH SERVICE statement builds and maintains a hybrid semantic-plus-keyword index — no embedding pipeline to write or host, and TARGET LAG re-indexes new documents automatically. The prompt in the final step instructs the model to answer only from the retrieved context, which is why hallucination drops sharply: it’s reading, not guessing. One operational note worth setting up on day one. A search service inherits dynamic-table refresh behaviour, which means that after five consecutive refresh failures it suspends indexing on its own — and keeps serving answers from the frozen index. Put this on an alert: SELECT name, indexing state, indexing errorFROM INFORMATION SCHEMA.CORTEX SEARCH SERVICESWHERE indexing state = 'ACTIVE'; AI COMPLETE takes the model name as a string, so swapping it is a one-line change, not a re-architecture. Start with a small model for simple lookups; reach for a larger one only when answers need deeper reasoning. Two things to check before you settle on one. Model availability varies by region, and some models require cross-region inference to be enabled — Snowflake publishes the matrix per region, and it’s worth confirming yours rather than copying a model name out of a blog post. And if you’re following an older tutorial, note that SNOWFLAKE.CORTEX.COMPLETE, PARSE DOCUMENT, and CLASSIFY TEXT are all documented as legacy now, slated for deprecation by the end of 2026. AI COMPLETE, AI PARSE DOCUMENT, and AI CLASSIFY are the current surface. The pipeline wraps in a Streamlit-in-Snowflake app — same compute, same account, zero extra hosting. A user asks “What’s our refund policy for enterprise plans?” and gets a grounded answer with sources listed underneath: enterprise msa 2025.pdf, refund policy v3.docx. A ten-minute result cache keeps repeat questions fast and cheap. If you want page numbers alongside the file names, parse with 'page split': TRUE and carry the page index through the chunk table — the citation is only as precise as the metadata you keep. Most of the governance story carries over for free. Data never leaves the account or region, ACCESS HISTORY logs every parse, index refresh, and completion call the same as any other query, and Snowflake's SOC 2, HIPAA, and FedRAMP certifications cover this pipeline too — there's nothing new to certify. Access control is the exception, and it’s worth being precise about because the intuition here is wrong. A Cortex Search service executes with owner’s rights . Snowflake’s documentation states the consequence directly: any role with sufficient privileges to query a service may query any of the data that service has indexed, regardless of that role’s privileges on the underlying tables and views. Row access policies and masking on the source table do not follow the text into the index. So you have one decision to make rather than none. Either index the security dimension as an ATTRIBUTE and require a matching filter on every query — cheap, but your access model now lives in application code, where a missing filter is a one-line bug with a wide blast radius. Or create a separate service per audience, each under a role scoped to only that audience's documents. That costs more, because serving is billed per gigabyte of indexed data and overlapping corpora get indexed twice, but the boundary is enforced by what was indexed rather than by what the caller remembered to pass. For anything with a real confidentiality gradient, take the second one. Then test it the way an auditor would: create a role with USAGE on the service and nothing else, and ask it the questions you'd least like answered. Six SQL statements, chained by a Task. No new infrastructure, no data leaving the platform, and the LLM is a parameter you can change on a whim. What changes here isn’t the quality of the answers. A well-built pipeline on Postgres and pgvector will answer the refund-policy question just as well. What changes is how much of your attention the pipeline demands after it works. The five-system version has five upgrade cycles, five sets of credentials, five places where a schema change breaks something at 2am, and a standing question about which copy of your contracts is the current one. That overhead never shows up in the demo. It shows up in month four, as the reason nobody has added the second document set. Collapsing it to six statements moves the hard question somewhere more useful. It stops being how do we stand up retrieval and becomes which documents actually deserve to be in the index — a question about your business rather than your infrastructure, and the one you wanted to spend your time on in the first place. Start with one folder of documents that people already ask questions about. The pipeline is the easy part now. RAG Without Leaving Snowflake https://pub.towardsai.net/rag-without-leaving-snowflake-d705bb7e3025 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.