A large language model is only as current as its training data — it has never seen your contracts, your policies, or last week’s support tickets. The usual fix, Retrieval-Augmented Generation, works by fetching the relevant facts right before the model answers. No re-training. The model answers from documents you hand it, which cuts hallucination sharply on questions about your own data.
The problem isn’t the concept. It’s that “standing up RAG” traditionally means standing up five separate systems just to answer questions about your own documents.
Snowflake’s Cortex suite replaces each piece with a managed, in-platform equivalent: TARGET_LAG handles re-indexing instead of a manual pipeline, the documents never leave the account, and there's zero additional infrastructure to patch.
The whole thing runs as six steps inside one account.
Here’s the SQL behind each step.
CREATE OR REPLACE STAGE docs_stage DIRECTORY = (ENABLE = TRUE);
PUT file://./contracts/*.pdf @docs_stage/contracts/;-- or point at S3 / Azure Blob / GCS with an external stage
CREATE OR REPLACE TABLE parsed_docs ASSELECT relative_path AS file_name, PARSE_JSON( AI_PARSE_DOCUMENT( TO_FILE('@docs_stage', relative_path), {'mode': 'LAYOUT'} ) ):content::string AS parsed_textFROM DIRECTORY(@docs_stage);
CREATE OR REPLACE TABLE doc_chunks ASSELECT d.file_name, c.index::int AS chunk_index, c.value::string AS chunk_textFROM parsed_docs d,LATERAL FLATTEN(input => SNOWFLAKE.CORTEX.SPLIT_TEXT_RECURSIVE_CHARACTER( d.parsed_text, 'markdown', 512, 128)) c;
AI_PARSE_DOCUMENT reads PDFs, Word docs, and scanned images directly off the stage. LAYOUT mode preserves headings and tables as Markdown, which chunks far more cleanly than raw OCR text. Note that it takes a FILE object built with TO_FILE() rather than a stage-and-path pair, and it returns JSON as a string — hence the PARSE_JSON before pulling :content.
SPLIT_TEXT_RECURSIVE_CHARACTER splits on paragraph and sentence boundaries first, only hard-cutting when it has to, with 128-character overlap to preserve context across edges.
The 512-character chunk size is deliberate. Cortex Search’s default embedding model has a 512-token context window, and anything longer gets truncated before it’s embedded — while still being fully searchable by keyword, so an oversized chunk looks like it’s working. At 512 characters you’re around 128 tokens, comfortably clear of that edge. If you raise the chunk size later, check it:
SELECT COUNT_IF(AI_COUNT_TOKENS('snowflake-arctic-embed-l-v2.0', chunk_text) > 512) AS truncated_chunksFROM doc_chunks;
CREATE OR REPLACE CORTEX SEARCH SERVICE doc_qa_search ON chunk_text ATTRIBUTES file_name WAREHOUSE = cortex_wh TARGET_LAG = '1 hour' EMBEDDING_MODEL = 'snowflake-arctic-embed-l-v2.0'AS (SELECT chunk_text, file_name, chunk_index FROM doc_chunks);
SELECT PARSE_JSON( SNOWFLAKE.CORTEX.SEARCH_PREVIEW('doc_qa_search', '{"query": "What is our refund policy?", "columns": ["chunk_text", "file_name"], "limit": 5}'))['results'] AS hits;
SELECT AI_COMPLETE('claude-sonnet-4-6', CONCAT('Answer using ONLY this context:\n', :retrieved_chunks, '\n\nQuestion: ', :user_question)) AS answer;
One CREATE CORTEX SEARCH SERVICE statement builds and maintains a hybrid semantic-plus-keyword index — no embedding pipeline to write or host, and TARGET_LAG re-indexes new documents automatically.
The prompt in the final step instructs the model to answer only from the retrieved context, which is why hallucination drops sharply: it’s reading, not guessing.
One operational note worth setting up on day one. A search service inherits dynamic-table refresh behaviour, which means that after five consecutive refresh failures it suspends indexing on its own — and keeps serving answers from the frozen index. Put this on an alert:
SELECT name, indexing_state, indexing_errorFROM INFORMATION_SCHEMA.CORTEX_SEARCH_SERVICESWHERE indexing_state != 'ACTIVE';
AI_COMPLETE() takes the model name as a string, so swapping it is a one-line change, not a re-architecture. Start with a small model for simple lookups; reach for a larger one only when answers need deeper reasoning.
Two things to check before you settle on one. Model availability varies by region, and some models require cross-region inference to be enabled — Snowflake publishes the matrix per region, and it’s worth confirming yours rather than copying a model name out of a blog post. And if you’re following an older tutorial, note that SNOWFLAKE.CORTEX.COMPLETE, PARSE_DOCUMENT, and CLASSIFY_TEXT are all documented as legacy now, slated for deprecation by the end of 2026. AI_COMPLETE, AI_PARSE_DOCUMENT, and AI_CLASSIFY are the current surface.
The pipeline wraps in a Streamlit-in-Snowflake app — same compute, same account, zero extra hosting. A user asks “What’s our refund policy for enterprise plans?” and gets a grounded answer with sources listed underneath: enterprise_msa_2025.pdf, refund_policy_v3.docx. A ten-minute result cache keeps repeat questions fast and cheap.
If you want page numbers alongside the file names, parse with 'page_split': TRUE and carry the page index through the chunk table — the citation is only as precise as the metadata you keep.
Most of the governance story carries over for free. Data never leaves the account or region, ACCESS_HISTORY logs every parse, index refresh, and completion call the same as any other query, and Snowflake's SOC 2, HIPAA, and FedRAMP certifications cover this pipeline too — there's nothing new to certify.
Access control is the exception, and it’s worth being precise about because the intuition here is wrong.
A Cortex Search service executes with owner’s rights. Snowflake’s documentation states the consequence directly: any role with sufficient privileges to query a service may query any of the data that service has indexed, regardless of that role’s privileges on the underlying tables and views. Row access policies and masking on the source table do not follow the text into the index.
So you have one decision to make rather than none. Either index the security dimension as an ATTRIBUTE and require a matching filter on every query — cheap, but your access model now lives in application code, where a missing filter is a one-line bug with a wide blast radius. Or create a separate service per audience, each under a role scoped to only that audience's documents. That costs more, because serving is billed per gigabyte of indexed data and overlapping corpora get indexed twice, but the boundary is enforced by what was indexed rather than by what the caller remembered to pass.
For anything with a real confidentiality gradient, take the second one. Then test it the way an auditor would: create a role with USAGE on the service and nothing else, and ask it the questions you'd least like answered.
Six SQL statements, chained by a Task. No new infrastructure, no data leaving the platform, and the LLM is a parameter you can change on a whim.
What changes here isn’t the quality of the answers. A well-built pipeline on Postgres and pgvector will answer the refund-policy question just as well. What changes is how much of your attention the pipeline demands after it works.
The five-system version has five upgrade cycles, five sets of credentials, five places where a schema change breaks something at 2am, and a standing question about which copy of your contracts is the current one. That overhead never shows up in the demo. It shows up in month four, as the reason nobody has added the second document set.
Collapsing it to six statements moves the hard question somewhere more useful. It stops being how do we stand up retrieval and becomes which documents actually deserve to be in the index — a question about your business rather than your infrastructure, and the one you wanted to spend your time on in the first place.
Start with one folder of documents that people already ask questions about. The pipeline is the easy part now.
RAG Without Leaving Snowflake was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.