How to use website search functionality with my LLM

Hi! One approach that works well is to build a structured knowledge base from your website instead of relying only on raw page crawling. If your site already has FAQs, service pages, pricing information, and educational content, you can chunk that content, generate embeddings, and store it in a vector database for RAG.

For local business websites, users often ask the same question in different ways, such as: A combination of semantic search and keyword search usually returns more reliable results than using only one approach, especially for conversational queries.

I’m also interested to know what tools others are using to crawl websites and keep the knowledge base updated automatically.