Originally published on tamiz.pro.
The standard architecture for Retrieval-Augmented Generation (RAG) systems has ossified around a complex stack: an ingest pipeline, a heavy-duty vector database (Pinecone, Weaviate, Milvus), and an orchestration layer. This approach often introduces significant operational overhead. You must manage vector indexing, handle schema migrations, deal with embedding model drift, and provision additional infrastructure just to serve the data layer. For many use cases, particularly those involving unstructured enterprise documents (PDFs, legal contracts, technical manuals), this complexity is unnecessary. You don't always need to pre-compute embeddings and store them in a specialized index. You can use on-the-fly semantic search.
Enter Gemini File Search. Google's Gemini API offers a "File Search" tool that allows the model to search across a directory of unstructured files (PDFs, DOCX, TXT, HTML) without requiring you to host them in a vector database. It handles the chunking, embedding, and retrieval internally. By combining this capability with the Go programming language, developers can build production-grade RAG pipelines that are incredibly lean. Go's concurrency model and type safety make it ideal for handling the HTTP layers and data transformations required to glue your application logic to the Gemini API. This article explores how to architect and build such a system, focusing on eliminating the "vector DB tax" while maintaining high retrieval accuracy and system reliability.
Traditional RAG requires the client or a middle-tier service to know exactly what to look for. You must generate embeddings for your chunks and query a vector store using cosine similarity. This decouples the understanding of the query from the retrieval mechanism. If your user asks a nuanced question that implies a specific search strategy, a standard vector search might miss the context if the embeddings don't align perfectly.
With File Search, the LLM (Gemini) acts as the retrieval engine. It understands the intent of the question, reformulates search queries if necessary, and scans the file contents (using its internal vector indexing service) to find relevant context. This shifts the burden of "semantic matching" to the model.
It is crucial to understand that File Search is best suited for medium-to-large corpora of unstructured text. It is not a replacement for a structured relational database. If your RAG relies heavily on complex joins between tables or precise numeric filtering, a traditional vector DB or SQL engine is still superior. File Search is optimized for document retrieval, not data querying.
We will build a service that accepts user queries and returns grounded answers using Gemini File Search. The stack will consist of:
net/http (or Chi/Echo for routing)google.golang.org/genai
viper for environment management
The architecture follows a clean layered design:
First, we set up the configuration. Production-grade Go apps should never hardcode secrets. We use environment variables for the Gemini API key and the project ID.
package config
import (
"fmt"
"os"
)
type Config struct {
GeminiAPIKey string
ProjectID string
ModelName string
MaxTokens int
Temperature float64
}
func LoadFromEnv() (*Config, error) {
apiKey := os.Getenv("GEMINI_API_KEY")
if apiKey == "" {
return nil, fmt.Errorf("GEMINI_API_KEY must be set")
}
projectID := os.Getenv("GCP_PROJECT_ID")
if projectID == "" {
return nil, fmt.Errorf("GCP_PROJECT_ID must be set")
}
return &Config{
GeminiAPIKey: apiKey,
ProjectID: projectID,
ModelName: "gemini-1.5-pro", // or flash for lower latency
MaxTokens: 2048,
Temperature: 0.2,
}, nil
}
In Gemini, tools are defined within the GenerationConfig or ToolConfig. The File Search tool requires specifying the directory in Google Cloud Storage where your files reside. The files must be in supported formats (.pdf, .docx, .txt, .html).
We need to create a helper to generate the tool configuration. Note that the directory must be accessible to the service account used by the Gemini API.
package service
import (
"context"
"google.golang.org/genai"
"google.golang.org/genai/google"
)
type FileSearchConfig struct {
GCSBucket string
Directory string
}
func NewFileSearchTool(ctx context.Context, cfg *FileSearchConfig) (*google.Tool, error) {
// Construct the file search tool specification
// Note: The exact struct definitions may vary slightly based on SDK version
// This is a conceptual representation of the required fields
tool := &google.Tool{
FunctionDeclarations: []*google.FunctionDeclaration{},
// In recent SDKs, file search is handled via specific tool types or parameters
// We will use the built-in FileSearch capability if available in the SDK version used
}
return tool, nil
}
Correction: As of the latest stable SDKs, File Search is often enabled via specific model parameters or by using the SearchRetrieval or FileSearch tool types provided in the google package. Let's use the more standard approach of defining the RagConfig or passing the file directory explicitly if the SDK supports it. For this article, we will assume the use of the genai client where we can pass a FileSearch tool configuration.
In practice, you define the tool like this:
func CreateFileSearchTool(bucket, dir string) *google.Tool {
return &google.Tool{
FileSearch: &google.FileSearchToolConfig{
Directory: &google.FileSearchToolConfigDirectory{
Bucket: bucket,
Directory: dir,
},
},
}
}
The core of our system is the service that takes a prompt and a file search configuration. We must ensure that we are using a model that supports this tool. gemini-1.5-pro and gemini-2.0-flash are currently the primary candidates. gemini-2.0-flash is often preferred for its speed and cost efficiency in high-volume production environments.
package service
import (
"context"
"fmt"
"time"
"google.golang.org/genai"
"google.golang.org/genai/google"
)
type GeminiService struct {
client *genai.Client
model string
fsConfig *FileSearchConfig
}
func NewGeminiService(client *genai.Client, model string, fsConfig *FileSearchConfig) *GeminiService {
return &GeminiService{
client: client,
model: model,
fsConfig: fsConfig,
}
}
func (s *GeminiService) Query(ctx context.Context, prompt string) (string, error) {
// Create the tool instance
tool := CreateFileSearchTool(s.fsConfig.GCSBucket, s.fsConfig.Directory)
// Construct the request
req := &google.GenerateContentRequest{
Model: s.model,
Contents: []*google.Content{
{
Role: "user",
Parts: []*google.Part{
{
Text: prompt,
},
},
},
},
Tools: []*google.Tool{tool},
// System instruction to enforce grounding
SystemInstruction: &google.Content{
Parts: []*google.Part{
{
Text: "You are a helpful assistant. Answer questions based ONLY on the provided file context. If the context does not contain the answer, state that clearly. Do not use external knowledge.",
},
},
},
}
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
// Generate the content
res, err := s.client.Models.GenerateContent(ctx, req)
if err != nil {
return "", fmt.Errorf("failed to generate content: %w", err)
}
if len(res.Candidates) == 0 {
return "", fmt.Errorf("no candidates returned from model")
}
if len(res.Candidates[0].Content.Parts) == 0 {
return "", fmt.Errorf("empty response from model")
}
return res.Candidates[0].Content.Parts[0].Text, nil
}
To use File Search, your documents must be in GCS. A production pipeline needs a way to upload files. We can create a separate command or an HTTP endpoint that uses the google.golang.org/api/storage library to upload files to the designated directory.
package ingest
import (
"context"
"fmt"
"io"
"net/http"
"google.golang.org/api/option"
"google.golang.org/api/storage/v1"
)
type Ingestor struct {
svc *storage.Service
bucket string
dir string
}
func NewIngestor(ctx context.Context, projectID string) (*Ingestor, error) {
client, err := storage.NewService(ctx, option.WithCredentialsFile("/path/to/service-account.json"))
if err != nil {
return nil, fmt.Errorf("failed to create storage service: %w", err)
}
return &Ingestor{
svc: client,
}, nil
}
func (i *Ingestor) UploadFile(file io.Reader, filename string) error {
// In a real app, you'd set specific content types based on extension
obj := &storage.Object{
Name: i.dir + "/" + filename,
}
err := i.svc.Objects.Insert(i.bucket, obj).Media(file).Run()
if err != nil {
return fmt.Errorf("failed to upload file: %w", err)
}
return nil
}
Even with zero extra infrastructure, network calls to the LLM are expensive and slow. You must implement caching. In Go, go-cache or a Redis-based cache is suitable. The cache key should be a hash of the prompt and the specific file directory version.
import (
"github.com/patrickmn/go-cache"
)
type Cache struct {
c *cache.Cache
}
func NewCache() *Cache {
return &Cache{
c: cache.New(10*time.Minute, 10*time.Minute),
}
}
func (c *Cache) Get(key string) (string, bool) {
if item, found := c.c.Get(key); found {
return item.(string), true
}
return "", false
}
func (c *Cache) Set(key, value string) {
c.c.Set(key, value, cache.DefaultExpiration)
}
You need to track:
Integrate with OpenTelemetry. The genai library supports OpenTelemetry middleware, which makes it easy to inject spans into your Go application.
use google.golang.org/api/option
set option.WithTelemetryEnabled() // or similar OTel compatible options
Since File Search runs on GCS, you must strictly control access to the GCS bucket. Use IAM policies to restrict who can read the documents. Never expose the GCS bucket publicly. The Gemini API key should be stored in a secrets manager (e.g., GCP Secret Manager) and injected at runtime. In a multi-tenant scenario, you might need separate GCS directories or buckets per tenant, which complicates the FileSearch configuration. In that case, you must dynamically construct the FileSearch tool configuration per request, which requires careful state management in your Go service.
While File Search is powerful, it may miss specific metadata. You can combine it with a traditional vector search by implementing a custom tool that calls your vector DB and then passing the results to Gemini as context. However, this defeats the "zero extra infrastructure" goal. A better hybrid approach is to pre-process your files in GCS to include metadata in the file names or front-matter (if using TXT/HTML) that Gemini's indexer can pick up.
For better UX, implement streaming. The genai client supports StreamGenerateContent. This allows you to send the response to the user as it is generated.
func (s *GeminiService) StreamQuery(ctx context.Context, prompt string, w io.Writer) error {
req := /* ... same as Query ... */
stream, err := s.client.Models.StreamGenerateContent(ctx, req)
if err != nil {
return err
}
defer stream.Close()
for {
res, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
return err
}
if len(res.Candidates) > 0 && len(res.Candidates[0].Content.Parts) > 0 {
if _, err := w.Write([]byte(res.Candidates[0].Content.Parts[0].Text)); err != nil {
return err
}
w.Flush() // if w is an http.ResponseWriter
}
}
return nil
}
Building a RAG pipeline without a vector database is not just about simplifying infrastructure; it is about leveraging the semantic understanding of LLMs to handle the retrieval layer. By using Go for your application logic and Gemini's File Search for the data layer, you can create a robust, scalable, and maintainable system. The key to success lies in proper data hygiene (cleaning your documents before upload), rigorous caching, and monitoring the "retrieval quality" metrics. As models continue to improve, the boundary between "retrieval" and "reasoning" will blur further, making these server-side semantic tools increasingly viable for production workloads. For a deeper look at similar patterns, check out our previous analysis on stateless RAG architectures and optimizing LLM inference costs.
Q: Can I use File Search with structured data like CSVs?
A: Yes, CSVs are supported, but the semantic search performance depends on the column headers and content readability. For large structured datasets, a SQL engine or vector DB with metadata filtering is usually more accurate.
Q: What is the maximum file size for a single document?
A: Currently, the limit is 2GB per file. However, very large files may time out during indexing or search. It is best practice to chunk large PDFs into logical sections if possible.
Q: How does Gemini know which files to search?
A: It searches the entire directory specified in the FileSearch configuration. You cannot dynamically filter by metadata within the tool itself, so organize your GCS directories logically (e.g., /docs/2023, /docs/2024) if you need temporal separation.