cd /news/ai-tools/rag-without-leaving-snowflake · home › topics › ai-tools › article
[ARTICLE · art-171484] src=pub.towardsai.net ↗ pub= topic=ai-tools verified=true sentiment=↑ positive

RAG Without Leaving Snowflake

Snowflake published a six-step SQL workflow for building retrieval-augmented generation (RAG) entirely inside a Snowflake account using its Cortex suite, eliminating the need to stand up five separate systems for document question-answering. The workflow uses AI_PARSE_DOCUMENT in LAYOUT mode to convert PDFs, Word documents and scanned images to Markdown, SPLIT_TEXT_RECURSIVE_CHARACTER at a 512-character chunk size with 128-character overlap, a CREATE CORTEX SEARCH SERVICE statement with TARGET_LAG = '1 hour' and the snowflake-arctic-embed-l-v2.0 embedding model, and AI_COMPLETE with claude-sonnet-4-6 to generate answers from retrieved context only. The article warns that Cortex Search's default embedding model has a 512-token context window, so oversized chunks are silently truncated before embedding while remaining keyword-searchable, and that a search service suspends indexing after five consecutive refresh failures while continuing to serve from the frozen index.

recommendation

Size chunks to the embedding model's 512-token window—512 characters here—and run AI_COUNT_TOKENS to catch truncated chunks; alert on INFORMATION_SCHEMA.CORTEX indexing_state so a suspended index doesn't silently serve stale answers.

more in playbook: Retrieval and RAG
by read6 min views7 publishedNov 18, 2026

A large language model is only as current as its training data — it has never seen your contracts, your policies, or last week’s support tickets. The usual fix, Retrieval-Augmented Generation, works by fetching the relevant facts right before the model answers. No re-training. The model answers from documents you hand it, which cuts hallucination sharply on questions about your own data.

The problem isn’t the concept. It’s that “standing up RAG” traditionally means standing up five separate systems just to answer questions about your own documents.

Snowflake’s Cortex suite replaces each piece with a managed, in-platform equivalent: TARGET_LAG handles re-indexing instead of a manual pipeline, the documents never leave the account, and there's zero additional infrastructure to patch.

The whole thing runs as six steps inside one account.

Here’s the SQL behind each step.

CREATE OR REPLACE STAGE docs_stage    DIRECTORY = (ENABLE = TRUE);
PUT file://./contracts/*.pdf @docs_stage/contracts/;-- or point at S3 / Azure Blob / GCS with an external stage
CREATE OR REPLACE TABLE parsed_docs ASSELECT    relative_path AS file_name,    PARSE_JSON(        AI_PARSE_DOCUMENT(            TO_FILE('@docs_stage', relative_path),            {'mode': 'LAYOUT'}        )    ):content::string AS parsed_textFROM DIRECTORY(@docs_stage);
CREATE OR REPLACE TABLE doc_chunks ASSELECT    d.file_name,    c.index::int      AS chunk_index,    c.value::string   AS chunk_textFROM parsed_docs d,LATERAL FLATTEN(input => SNOWFLAKE.CORTEX.SPLIT_TEXT_RECURSIVE_CHARACTER(    d.parsed_text, 'markdown', 512, 128)) c;

AI_PARSE_DOCUMENT reads PDFs, Word docs, and scanned images directly off the stage. LAYOUT mode preserves headings and tables as Markdown, which chunks far more cleanly than raw OCR text. Note that it takes a FILE object built with TO_FILE() rather than a stage-and-path pair, and it returns JSON as a string — hence the PARSE_JSON before pulling :content.

SPLIT_TEXT_RECURSIVE_CHARACTER splits on paragraph and sentence boundaries first, only hard-cutting when it has to, with 128-character overlap to preserve context across edges.

The 512-character chunk size is deliberate. Cortex Search’s default embedding model has a 512-token context window, and anything longer gets truncated before it’s embedded — while still being fully searchable by keyword, so an oversized chunk looks like it’s working. At 512 characters you’re around 128 tokens, comfortably clear of that edge. If you raise the chunk size later, check it:

SELECT COUNT_IF(AI_COUNT_TOKENS('snowflake-arctic-embed-l-v2.0', chunk_text) > 512)    AS truncated_chunksFROM doc_chunks;
CREATE OR REPLACE CORTEX SEARCH SERVICE doc_qa_search    ON chunk_text    ATTRIBUTES file_name    WAREHOUSE = cortex_wh    TARGET_LAG = '1 hour'    EMBEDDING_MODEL = 'snowflake-arctic-embed-l-v2.0'AS (SELECT chunk_text, file_name, chunk_index FROM doc_chunks);
SELECT PARSE_JSON(    SNOWFLAKE.CORTEX.SEARCH_PREVIEW('doc_qa_search',      '{"query": "What is our refund policy?",        "columns": ["chunk_text", "file_name"], "limit": 5}'))['results'] AS hits;
SELECT AI_COMPLETE('claude-sonnet-4-6',  CONCAT('Answer using ONLY this context:\n', :retrieved_chunks,         '\n\nQuestion: ', :user_question)) AS answer;

One CREATE CORTEX SEARCH SERVICE statement builds and maintains a hybrid semantic-plus-keyword index — no embedding pipeline to write or host, and TARGET_LAG re-indexes new documents automatically.

The prompt in the final step instructs the model to answer only from the retrieved context, which is why hallucination drops sharply: it’s reading, not guessing.

One operational note worth setting up on day one. A search service inherits dynamic-table refresh behaviour, which means that after five consecutive refresh failures it suspends indexing on its own — and keeps serving answers from the frozen index. Put this on an alert:

SELECT name, indexing_state, indexing_errorFROM INFORMATION_SCHEMA.CORTEX_SEARCH_SERVICESWHERE indexing_state != 'ACTIVE';

AI_COMPLETE() takes the model name as a string, so swapping it is a one-line change, not a re-architecture. Start with a small model for simple lookups; reach for a larger one only when answers need deeper reasoning.

Two things to check before you settle on one. Model availability varies by region, and some models require cross-region inference to be enabled — Snowflake publishes the matrix per region, and it’s worth confirming yours rather than copying a model name out of a blog post. And if you’re following an older tutorial, note that SNOWFLAKE.CORTEX.COMPLETE, PARSE_DOCUMENT, and CLASSIFY_TEXT are all documented as legacy now, slated for deprecation by the end of 2026. AI_COMPLETE, AI_PARSE_DOCUMENT, and AI_CLASSIFY are the current surface.

The pipeline wraps in a Streamlit-in-Snowflake app — same compute, same account, zero extra hosting. A user asks “What’s our refund policy for enterprise plans?” and gets a grounded answer with sources listed underneath: enterprise_msa_2025.pdf, refund_policy_v3.docx. A ten-minute result cache keeps repeat questions fast and cheap.

If you want page numbers alongside the file names, parse with 'page_split': TRUE and carry the page index through the chunk table — the citation is only as precise as the metadata you keep.

Most of the governance story carries over for free. Data never leaves the account or region, ACCESS_HISTORY logs every parse, index refresh, and completion call the same as any other query, and Snowflake's SOC 2, HIPAA, and FedRAMP certifications cover this pipeline too — there's nothing new to certify.

Access control is the exception, and it’s worth being precise about because the intuition here is wrong.

A Cortex Search service executes with owner’s rights. Snowflake’s documentation states the consequence directly: any role with sufficient privileges to query a service may query any of the data that service has indexed, regardless of that role’s privileges on the underlying tables and views. Row access policies and masking on the source table do not follow the text into the index.

So you have one decision to make rather than none. Either index the security dimension as an ATTRIBUTE and require a matching filter on every query — cheap, but your access model now lives in application code, where a missing filter is a one-line bug with a wide blast radius. Or create a separate service per audience, each under a role scoped to only that audience's documents. That costs more, because serving is billed per gigabyte of indexed data and overlapping corpora get indexed twice, but the boundary is enforced by what was indexed rather than by what the caller remembered to pass.

For anything with a real confidentiality gradient, take the second one. Then test it the way an auditor would: create a role with USAGE on the service and nothing else, and ask it the questions you'd least like answered.

Six SQL statements, chained by a Task. No new infrastructure, no data leaving the platform, and the LLM is a parameter you can change on a whim.

What changes here isn’t the quality of the answers. A well-built pipeline on Postgres and pgvector will answer the refund-policy question just as well. What changes is how much of your attention the pipeline demands after it works.

The five-system version has five upgrade cycles, five sets of credentials, five places where a schema change breaks something at 2am, and a standing question about which copy of your contracts is the current one. That overhead never shows up in the demo. It shows up in month four, as the reason nobody has added the second document set.

Collapsing it to six statements moves the hard question somewhere more useful. It stops being how do we stand up retrieval and becomes which documents actually deserve to be in the index — a question about your business rather than your infrastructure, and the one you wanted to spend your time on in the first place.

Start with one folder of documents that people already ask questions about. The pipeline is the easy part now.

RAG Without Leaving Snowflake was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

── more in #ai-tools 4 stories · sorted by recency
── more on @snowflake 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/rag-without-leaving-…] indexed:0 read:6min 2026-11-18 · —