# Serverless Multimodal Vector Search on Apache Iceberg via Google Apps Script

> Source: <https://dev.to/gde/serverless-multimodal-vector-search-on-apache-iceberg-via-google-apps-script-4fg>
> Published: 2026-09-08 05:06:23+00:00

*Turn Google Drive into an AI-Powered Lakehouse Vector Engine across Converted PDFs, Binary Images, and Text without Specialized Vector Databases.*

*Hero Infographic: Unified Multimodal Lakehouse Vector Search via Apache Iceberg & Google Apps Script. Consolidates Google Docs, Sheets, Slides, Forms, binary diagrams, and web-fetched assets into an open Parquet table, driven by Gemini embeddings and BigQuery serverless pushdown for sub-second semantic retrieval and live Google Drive discovery.*

## 
  
  
  Structural Analysis of the Hero Infographic:

The hero infographic visualizes the end-to-end paradigm shift enabling **unified multimodal storage and sub-second semantic discovery** across three synchronized operational zones:

- 
**Left Zone (Diverse Multimodal Ingestion)** : Ingests heterogeneous corporate knowledge spanning Google Drive assets (Docs with automated PDF normalization, Sheets data/PDFs, Slides presentations, and Forms intake/response structures), binary media (PNG/JPEG schematics and diagrams), and web-crawled/HTTP-downloaded external files into a unified ingestion pipeline.
- 
**Center Zone (Serverless Processing & Unified Iceberg Lakehouse)** : The**IcebergApp.js** engine running purely within Google Apps Script extracts payloads and routes textual representations to the**Gemini Embedding API (`text-embedding-004`)** to synthesize 768-dimensional normalized float vectors (` ARRAY<FLOAT64>` ). These are committed alongside raw binary streams (`data BYTES` ), metadata, and authentic Google Drive identifiers (`file_id STRING` ) into a**Single Apache Iceberg Table** hosted on Google Cloud Storage.
- 
**Right Zone (Serverless Query & Live File Discovery)** : When an operator issues a natural language query,**BigQuery** executes distributed`COSINE_DISTANCE` pushdown directly across the open Parquet files on Cloud Storage. Within 0.3 to 0.5 seconds, ranked results (with similarity metrics exceeding`0.87` ) are returned, enabling instant 1-click browser navigation back to the live collaborative document or triggering programmatic Workspace automation.

## 
  
  
  Abstract

Enterprise knowledge is notoriously fragmented across structured tabular records, raw text notes, binary diagrams, and rich documents stored in Google Drive. Traditional Retrieval-Augmented Generation (RAG) and semantic search architectures address this challenge by deploying dedicated vector database clusters (e.g., Pinecone, Milvus, Qdrant) or specialized Reverse ETL pipelines. However, these systems introduce steep subscription costs, operational complexity, and vendor lock-in, while isolating embeddings from core analytical storage.

This article introduces a **100% serverless, zero-maintenance Multimodal Lakehouse Vector Search architecture** built on **[IcebergApp](https://github.com/tanaikech/IcebergApp)**, Google Apps Script (GAS), and BigQuery. Leveraging Apache Iceberg's native support for binary data (`BYTES`), this system ingests heterogeneous assets—including live Google Docs automatically converted into PDF binary blobs, raw image diagrams, and direct plain text records—into an open Parquet table on Google Cloud Storage. Crucially, by persisting the authentic Google Drive `file_id` alongside the binary PDF snapshot, vector search results immediately map back to the live collaborative document (`https://docs.google.com/document/d/${file_id}/edit`), transforming Apache Iceberg into an enterprise-grade semantic file search engine for Google Workspace. Vector embeddings are generated on-the-fly using the Gemini API (`text-embedding-004`) and queried using distributed vector distance metrics (` COSINE_DISTANCE`) with sub-second latency. Furthermore, to enable developers to safely evaluate the pipeline within their own Google Cloud environment, the article includes an autonomous test suite equipped with an automated self-cleaning mechanism that guarantees zero-residue cleanup of temporary test assets, empowering Workspace developers to evaluate and deploy enterprise-grade multimodal search engines directly from Google Apps Script.

## 
  
  
  1. Introduction: The Fragmentation of Multimodal Knowledge

In modern enterprise operations, knowledge does not reside neatly in relational database rows. It lives in diverse multimodal formats scattered across Google Workspace and cloud storage:

- 
**Rich Documents** : Standard operating procedures, technical manuals, and architectural specifications in**Google Docs** ; financial models and analytical KPIs in**Google Sheets** ; pitch decks and visual proposals in**Google Slides** ; and questionnaire forms and intake responses in**Google Forms** .
- 
**Visual Artifacts** : System architecture schematics, flowcharts, and infographics saved as**PNG/JPEG images** .
- 
**Direct Operational Text** : Real-time event notes, error logs, and transactional summaries written directly in code or spreadsheets.

To visually capture this operational divide and contrast conventional, costly multi-tier architectures with our unified serverless design, **Figure 1** outlines the structural evolution from isolated enterprise silos to an integrated Apache Iceberg lakehouse.

*Figure 1: Fragmented Enterprise Data Silos & Traditional RAG vs. IcebergApp Unified Multimodal Serverless Lakehouse. Contrasts the high-cost, fragmented traditional RAG pipeline against the unified, zero-egress Apache Iceberg architecture powered by Google Apps Script and BigQuery on Google Cloud.*

### 
  
  
  Detailed Architectural Analysis of Figure 1:

As clearly mapped in the comparative tiers of **Figure 1**, enterprise multimodal knowledge management undergoes a fundamental structural paradigm shift:

1. 
**Top Section (Fragmented Enterprise Data Silos)** : In everyday operations, assets are created and stored in isolated systems: Google Docs, Sheets, Slides, and Forms inside Google Drive; PNG/JPEG diagrams residing in Cloud Storage or Drive; and operational event notes residing as raw text strings. Because these assets use incompatible storage primitives, traditional analytical systems cannot ingest or index them collectively.
2. 
**Middle Section (Traditional Complex RAG Stack - The Problem)** : As depicted by the red-bordered warning flow in**Figure 1** , conventional architectures attempt to bridge these silos by introducing third-party Python microservices and dedicated vector databases (e.g., Pinecone, Milvus, Qdrant). This approach introduces three fatal operational penalties:  - 
**Exorbitant Subscription Tax** : Dedicated vector database clusters impose recurring fixed commitments between $500 and $3,000 per month, regardless of actual query volume.
  - 
**Severe Egress Vulnerabilities** : Sensitive enterprise documents and intellectual property must leave the secure Google Cloud Virtual Private Cloud (VPC) and traverse external public networks to reach third-party SaaS platforms.
  - 
**Dual-Storage Synchronization Drift** : Storing vector embeddings in a standalone vector database while keeping binary PDFs and images in separate object storage invariably leads to broken links, synchronization lags, and metadata drift.
3. 
**Bottom Section (IcebergApp Unified Serverless Lakehouse - The Solution)** : Indicated by the glowing green and cyber-cyan borders in**Figure 1** , our architecture replaces this complex multi-tier stack with a clean, 100% serverless pipeline:  - 
**Google Apps Script (V8 Engine)** : Directly interacts with Google Workspace APIs via`DriveApp` and native services, exporting Docs, Sheets, and Slides into standardized PDF binary blobs on-the-fly (and extracting Forms structure via`FormApp` or linked response sheets) without intermediary servers.
  - 
**Apache Iceberg on Google Cloud Storage** : Eliminates storage fragmentation by persisting Text metadata, Binary BYTES (PDF, PNG), and 768-dimensional float embedding vectors into a**single, open Apache Parquet table** .
  - 
**BigQuery Vector Accelerator** : Leverages distributed`COSINE_DISTANCE` pushdown directly across open Parquet files, executing sub-second semantic retrieval with 100% data sovereignty entirely within the customer's GCP perimeter.

### 
  
  
  1.1 The Pitfalls of Traditional Vector Architectures

To perform semantic search across these mixed assets, engineering teams typically deploy complex multi-tier pipelines:

1. 
**Dedicated Vector Database Tax** : Specialized vector databases (Pinecone, Weaviate, Qdrant) cost hundreds to thousands of dollars per month, requiring dedicated network infrastructure, VPC peering, and credential rotation.
2. 
**Metadata Synchronization Drift** : When document content changes on Google Drive, keeping the vector database synchronized with external document stores requires complex webhook choreography and distributed transaction handling.
3. 
**Binary Data Isolation** : Most vector databases store only vector embeddings and minimal text metadata, forcing systems to maintain secondary blob storage links that frequently break or drift out of sync.

### 
  
  
  1.2 The Breakthrough: Apache Iceberg as a Multimodal Vector Store

[Apache Iceberg](https://iceberg.apache.org/) fundamentally redefines this paradigm:

- 
**Binary Data Support** : Iceberg tables are not restricted to scalar primitives; they natively store arbitrary binary payloads (`BYTES` ) and complex arrays (`ARRAY<FLOAT64>` ). A single open Parquet table can hold document metadata, raw PDF binaries, image bytes, and high-dimensional vector embeddings side by side.
- 
**Zero Vendor Lock-In** : Assets reside in open Parquet files on Google Cloud Storage, directly queryable by BigQuery, Apache Spark, Trino, or DuckDB.
- 
**Serverless Vector Compute** : BigQuery provides distributed vector distance functions (`COSINE_DISTANCE` ,`EUCLIDEAN_DISTANCE` ) and vector indexes over Iceberg tables, eliminating the need for standalone vector database servers.

### 
  
  
  1.3 From Vector Search to Semantic File Discovery: The Power of GAS & `file_id`

One of the most compelling enterprise advantages of implementing this pipeline in Google Apps Script is the direct bridge between AI vector search and live Google Drive file discovery:

- In conventional RAG pipelines, vector search engines return isolated text chunks, leaving users stranded with no direct path to the original source document.
- In contrast, because `IcebergApp` stores the authentic Google Drive`file_id` directly in the Iceberg table schema alongside the PDF binary snapshot,**every vector search result includes the authoritative Drive File ID** .
- This enables Workspace developers to build an **intelligent semantic file search engine** :  1. 
**One-Click Live Navigation** : Users can immediately navigate directly to the matching collaborative asset across formats (Google Docs:`https://docs.google.com/document/d/${file_id}/edit` , Sheets:`https://docs.google.com/spreadsheets/d/${file_id}/edit` , Slides:`https://docs.google.com/presentation/d/${file_id}/edit` , Forms:`https://docs.google.com/forms/d/${file_id}/edit` , or generic Drive link`https://drive.google.com/open?id=${file_id}` ).
  2. 
**Automated Workspace Actionability** : In GAS, receiving the`file_id` allows instant downstream scripting—such as updating document text via`DocumentApp` , recalculating spreadsheets via`SpreadsheetApp` , generating slides via`SlidesApp` , managing form intake via`FormApp` , or sharing access via`DriveApp.getFileById(file_id).addViewer()` .

By marrying Google Apps Script with Apache Iceberg via **[IcebergApp](https://github.com/tanaikech/IcebergApp)**, Workspace developers gain not only a multimodal lakehouse but an actionable AI-powered file search engine across their entire Google Drive ecosystem.

## 
  
  
  2. Architecture & Multimodal Pipeline

The end-to-end processing pipeline—from document ingestion and binary conversion to vector generation and distributed BigQuery search, as well as the autonomous verification flow executed by our test suite—is organized into the 6-stage workflow illustrated in **Figure 2**.

*Figure 2: End-to-End Multimodal Vector Search Lifecycle in Apache Iceberg via Google Apps Script. Displays the 6-stage operational workflow from document normalization to sub-second vector search and safe test suite resource cleanup.*

### 
  
  
  Detailed Operational Walkthrough of Figure 2:

As illustrated across the numbered progression of **Figure 2**, the system executes an automated, 6-step workflow that coordinates Google Workspace, Gemini AI, BigQuery, and Google Cloud Storage:

1. 
**Step 1: Document Normalization & Asset Ingestion** : The pipeline captures heterogeneous enterprise assets across Google Workspace. When processing live Google Docs,`DriveApp.getFileById(docId).getBlob()` converts collaborative documents into immutable, standardized PDF binaries (`application/pdf` ) without requiring headless Chromium or external conversion microservices. Concurrently, binary PNG images and raw operational text notes are loaded into memory.
2. 
**Step 2: Vector Embedding Synthesis** : Google Apps Script dispatches the textual payload of each asset (full text for documents and text files; semantic description metadata for images) to Google's Gemini Embedding API (`text-embedding-004` ). The model returns dense, L2-normalized 768-dimensional float arrays (`ARRAY<FLOAT64>` ) that capture semantic meaning across all modalities.
3. 
**Step 3: Iceberg Parquet Commit via IcebergApp** : Leveraging the core`IcebergApp.js` library, the engine prepares an open Apache Iceberg table on Google Cloud Storage. Through`table.insertBlobs()` , binary data payloads (`BYTES` ), descriptive schema metadata, and 768-dimensional vector arrays are committed simultaneously into open Parquet data files accompanied by hierarchical Avro metadata manifests.
4. 
**Step 4: Sub-Second Vector Query** : When an operator issues a natural language query (e.g.,*"How do I automate spreadsheets with JavaScript?"* ), the query is vectorized via Gemini. BigQuery then executes a distributed vector scan using`COSINE_DISTANCE(embedding, query_vector)` directly against the Parquet data files stored on Cloud Storage, completely bypassing intermediate database servers.
5. 
**Step 5: Ranked Multimodal Retrieval** : BigQuery ranks all assets deterministically by cosine distance (`ORDER BY distance ASC LIMIT 3` ). The search engine successfully retrieves the most relevant asset regardless of original format (whether PDF document, architectural diagram, or raw text note), achieving high semantic confidence (e.g., similarity scores exceeding`0.87` ).
6. 
**Step 6: Test Suite Zero-Residue Cleanup** : Upon completion of verification and retrieval, the autonomous test runner purges all temporary test assets: trashing test Google Docs on Google Drive, dropping the test table, and completely deleting the BigQuery dataset and Cloud Storage bucket within 35 seconds.*(Note: While production systems naturally retain their data, this self-cleaning teardown is built into the test suite so evaluators can run it with zero lingering cloud cost or clutter.)*

### 
  
  
  2.1 Component Breakdown

1. 
**Automatic Document Normalization (`Google Workspace Docs / Sheets / Slides -> PDF Blob`)** :
In Google Apps Script, calling`DriveApp.getFileById(fileId).getBlob()` on Google Docs,**Google Sheets, or Google Slides** automatically exports the live collaborative asset into an industry-standard, immutable PDF binary (`application/pdf` ) without external software. (For Google Forms, questions and configurations can be extracted via`FormApp` or linked response sheets into structured text/blobs.) This allows all dynamic Google Workspace documents to be archived and indexed as reproducible binary snapshots inside Apache Iceberg.
2. 
**Multimodal Schema Definition** :
The Iceberg table schema harmonizes structured metadata, binary assets, and vector embeddings:

Notice the `file_id STRING` column: this preserves the primary Google Drive document identifier, serving as the essential bi-directional bridge between the immutable lakehouse Parquet archive and the live Google Drive file.

1. 
**Automated Dense Vector Synthesis with Gemini (`text-embedding-004`)** :` IcebergApp` interfaces natively with Google's state-of-the-art Gemini Embedding API (`text-embedding-004` ). Extracted body text from Google Docs, tabular summaries from Sheets, slide contents, form structures, binary image metadata summaries, and external web payloads are synthesized into 768-dimensional normalized float vectors (`ARRAY<FLOAT64>` ), persisted directly into the Iceberg table's`embedding` column.
2. 
**Sub-Second Vector Search via BigQuery Acceleration** :
Queries are evaluated using serverless SQL pushdown:

### 
  
  
  2.2 How Gemini Powers the Architecture: 3 Core Operational Pillars

Within this architecture, **Google Gemini is not merely an optional add-on; it serves as the foundational intelligence engine** that transforms a passive storage repository into a cognitive, semantic discovery system. Gemini participates deeply across three key operational dimensions:

#### 
  
  
  1. Ingestion Phase: Harmonizing Heterogeneous Modalities into a Unified 768-D Semantic Space

Traditional storage engines cannot compare a PDF document, an architectural PNG image, a spreadsheet, and an external web page because their underlying data primitives are fundamentally incompatible.

- 
**Documents & Text Assets (Docs, Sheets, Slides, Forms, Web)** :
Within Google Apps Script,`DocumentApp` and native Workspace APIs extract raw text representations, which`IcebergApp` dispatches to the Gemini Embedding API (`text-embedding-004` ).
- 
**Visual Artifacts (Images & Diagrams)** :
Structural summaries and descriptive metadata are supplied to the embedding engine.
- 
**Gemini's Role** :
Gemini analyzes the semantic content and projects each disparate asset into an identical**768-dimensional L2-normalized float array (`ARRAY<FLOAT64>`)** . By doing so, every asset—regardless of whether it originated as a 30-page PDF, a flowchart image, or an HTML snippet—shares the exact same mathematical coordinate system inside the open Iceberg Parquet table.

#### 
  
  
  2. Query Phase: Real-Time Translation of Natural Language into Calculable Query Vectors

When an operator issues a natural language query from Google Apps Script (e.g., *"How do I automate spreadsheets with JavaScript?"*):

- 
`IcebergApp.getTableByName().searchSimilar(query)` immediately makes an outbound REST call to Gemini's`embedContent` endpoint via Apps Script's native`UrlFetchApp` .
- Within 200 to 300 milliseconds, Gemini returns a 768-dimensional query vector that captures the user's conceptual intent.
- This vector is injected into BigQuery's distributed SQL engine, allowing `COSINE_DISTANCE(embedding, [queryVector])` to scan thousands of records in parallel with sub-second execution latency.

#### 
  
  
  3. Breaking Keyword Barriers: True Semantic Understanding Without Exact-Word Matches

Legacy Google Drive search and relational SQL `LIKE` queries fail whenever documents use synonyms or alternative technical phrasing.

- 
**Gemini's Semantic Breakthrough** :
Gemini understands cross-domain conceptual hierarchies. As demonstrated in our empirical execution log (Section 5.1), when queried with the complex physical concept*"superconducting qubit coherence at millikelvin temperatures"* , the system correctly identified`TXT-001` (Quantum Computing Guide) with an**outstanding similarity score of 0.8669** , cleanly separating it from unrelated biological research or operational guides despite the lack of exact title matches.
- 
**Pure Serverless Simplicity** :
While conventional vectorization pipelines require heavyweight Python environments (PyTorch, Hugging Face Transformers) running on GPU clusters,`IcebergApp` achieves state-of-the-art embedding generation through a single serverless HTTPS call, incurring zero persistent infrastructure costs.

## 
  
  
  3. Implementation: The Test Suite & Code Essence

To provide a fully verified, turn-key demonstration, the complete standalone test script is implemented at [`src/multimodal_vector_search_test.js`](https://github.com/tanaikech/IcebergApp/blob/master/src/multimodal_vector_search_test.js).

### 
  
  
  3.1 Step 1: Install IcebergApp in Google Apps Script

To use IcebergApp in your project:

1. Open your Google Apps Script project (via `script.new` or container-bound to a Sheet/Doc).
2. Download the core library [`IcebergApp.js`](https://github.com/tanaikech/IcebergApp/blob/master/src/IcebergApp.js) from GitHub:

1. In the Apps Script editor, create a new script file named `IcebergApp.gs` (or`IcebergApp.js` ) and paste the complete library content.
2. Under **Services (+)** , add the**BigQuery API** (`v2` ).
3. Open **Project Settings (⚙️) > Script properties** , and configure:  - 
`PROJECT_ID` : Your Google Cloud Project ID.
  - 
`REGION` : Your target region (e.g.,`asia-northeast1` ).
  - 
`GEMINI_API_KEY` : Your Gemini API key from Google AI Studio.

### 
  
  
  3.2 Step-by-Step Test Execution: What Was Obtained at Each Stage

By inspecting the empirical execution log produced by `runMultimodalVectorSearchTest()`, we can trace exactly what technical artifacts and verification milestones were obtained across each of the 6 sequential stages:

1. 
**Stage 1 [00:00:04]: Ephemeral Infrastructure Provisioning**
   - 
**Action** : Provisioned an isolated BigQuery dataset`lakehouse_multimodal_xxxx` at`asia-northeast1` and a Cloud Storage bucket`gs://lakehouse-mm-xxxx` with Uniform Bucket-Level Access.
  - 
**What Was Obtained** : A completely isolated, timestamped GCP sandbox environment. This guarantees that test operations never interfere with production datasets or existing lakehouse catalogs.
2. 
**Stage 2 [00:00:08]: Apache Iceberg Table Initialization**
   - 
**Action** : Dispatched DDL statements via BigQuery's BigLake external catalog integration to establish the Iceberg table`multimodal_knowledge_assets` .
  - 
**What Was Obtained** : A unified, open table schema natively supporting binary payloads (`data BYTES` ), high-dimensional embeddings (`embedding ARRAY<FLOAT64>` ), and standard metadata (`id` ,`name` ,`category` ,`mime_type` ,`size` ,`description` ).
3. 
**Stage 3 [00:00:10]: Google Docs Generation & Cross-Modal Asset Staging**
   - 
**Action** : Dynamically generated 3 live Google Docs on Google Drive, immediately converted them to immutable PDF blobs via`DriveApp.getFileById().getBlob()` , and staged 2 raw text notes and 1 binary PNG diagram.
  - 
**What Was Obtained** : Six concrete multimodal assets across three distinct formats with verified byte-level sizes:    - 
`DOC-001` :`Google_Apps_Script_Automation_Guide.pdf` (27,177 bytes,`application/pdf` )
    - 
`DOC-002` :`Google_Sheets_Calculations_Analytics_Guide.pdf` (28,363 bytes,`application/pdf` )
    - 
`DOC-003` :`Google_Docs_Collaborative_Publishing_Guide.pdf` (20,757 bytes,`application/pdf` )
    - 
`TXT-001` :`quantum_cryogenic_computing_whitepaper.txt` (229 bytes,`text/plain` )
    - 
`TXT-002` :`marine_ocean_bioluminescence_study.txt` (218 bytes,`text/plain` )
    - 
`IMG-001` :`lakehouse_multimodal_architecture_diagram.png` (70 bytes,`image/png` )
4. 
**Stage 4 [00:00:15 – 00:00:24]: Gemini Vector Synthesis & Single-Table Parquet Commit**
   - 
**Action** : Dispatched the textual payloads and descriptive summaries to Google's Gemini Embedding API (`text-embedding-004` ), generated 768-dimensional float arrays, and executed`table.insertBlobs()` .
  - 
**What Was Obtained** : All 6 heterogeneous records—unifying binary bytes, structured metadata, and vector embeddings—were committed into a single open Parquet data file and updated Avro metadata manifests on Cloud Storage in just 9 seconds. The ingested inventory was printed and verified.
5. 
**Stage 5 [00:00:24 – 00:00:29]: Semantic Vector Searches across Heterogeneous Formats**
   - 
**Action** : Executed 4 distinct natural language queries using BigQuery distributed`COSINE_DISTANCE` pushdown directly over the Iceberg Parquet files.
  - 
**What Was Obtained** :**100% precision in cross-modal ranking** ; every target asset achieved Rank #1 with statistically significant margin over alternative files:    - 
*Query 1 (Automation)* : Isolated`DOC-001` (`0.7500` similarity) vs Sheets PDF (`0.6324` ).
    - 
*Query 2 (Formulas)* : Isolated`DOC-002` (`0.7935` similarity) vs GAS PDF (`0.5748` ).
    - 
*Query 3 (Quantum Qubits)* : Isolated`TXT-001` (`0.8669` similarity) vs Biology text (`0.5012` ).
    - 
*Query 4 (Visual Flow Diagram)* : Isolated`IMG-001` (`0.8762` similarity) vs Docs PDF (`0.6724` ).
6. 
**Stage 6 [00:00:29 – 00:00:35]: Absolute Zero-Residue Teardown**
   - 
**Action** : Automatically dropped the Iceberg table, trashed the 3 Google Docs by explicit Drive ID (`[xxxx-drive-doc1-id]` , etc.), deleted the BigQuery dataset, and destroyed the Cloud Storage bucket.
  - 
**What Was Obtained** :**100% eradication of all transient cloud resources** in only 6 seconds, leaving zero leftover data, zero storage costs, and zero orphaned Drive files.

### 
  
  
  3.3 Step 2: Code Essence – Ingesting Mixed Assets

The following concise snippet demonstrates how easy it is to ingest mixed Google Docs (PDFs), direct text, and binary images using `IcebergApp`:

### 
  
  
  3.4 Code Essence – Semantic Vector Search & Live File Discovery

Executing semantic vector search across both PDF documents and plain text files requires only a single method call. Crucially, because `file_id` is queried directly alongside distance metrics, the result connects straight back to the live Google Drive document:

## 
  
  
  4. Test Suite Safety Design: Automated Cleanup of Test Resources

For developers who wish to evaluate this multimodal lakehouse architecture within their own Google Cloud and Google Drive environments, the accompanying autonomous test script ([`multimodal_vector_search_test.js`](https://github.com/tanaikech/IcebergApp/blob/master/src/multimodal_vector_search_test.js)) incorporates an automated self-cleaning mechanism to ensure that temporary test assets (test Google Docs, temporary BigQuery datasets, and temporary GCS buckets) are completely scrubbed.

[!NOTE]

**Production vs. Testing Context**: While real-world production lakehouses naturally preserve accumulated tables and documents, this cleanup engine is implemented exclusively in the test suite so that developers can safely run and evaluate the pipeline without leaving orphaned test clutter or incurring unexpected cloud maintenance costs.

When running automated test suites in serverless environments, an execution timeout (e.g., Apps Script's 6-minute ceiling) or manual cancellation could risk leaving temporary test resources uncollected. To eliminate this risk, the test script implements a **Triple-Tiered Safety Architecture**:

1. 
**Persistent ScriptProperties Registry (`_ICEBERG_MULTIMODAL_TEST_REGISTRY_`)** :
The instant a test Google Doc, BigQuery dataset, GCS bucket, or table is created, its identifier is committed immediately to persistent`ScriptProperties` .
2. 
**Pre-Flight Sweep** :
Upon invoking`runMultimodalVectorSearchTest()` , the runner first reads the registry and scrubs any lingering resources from previously aborted runs, followed by a wildcard Drive search for test-pattern titles.
3. 
**Standalone Emergency Purge Function (`purgeResidualMultimodalTestResources`)** :
A dedicated function executable directly from the Apps Script toolbar dropdown to purge all tracked test resources on demand without re-running the full test suite.

## 
  
  
  5. Execution Results & Empirical Log Audit

When `runMultimodalVectorSearchTest()` is executed in the Apps Script Editor, it sequentially carries out:

1. Isolated infrastructure provisioning (dataset & bucket).
2. Creation of three live Google Docs on Google Drive and immediate conversion into PDF binary blobs.
3. Preparation of direct text files and image assets.
4. Batch ingestion into Apache Iceberg with Gemini 768-dimensional vector embeddings.
5. Execution of 4 diverse semantic vector queries targeting distinct domains (Google Apps Script, Google Sheets, Quantum Physics, Architecture Diagrams).
6. Comprehensive post-test zero-residue teardown.

### 
  
  
  5.1 Empirical Live Execution Log

The following execution log is the output produced by executing `runMultimodalVectorSearchTest` within the Google Apps Script V8 runtime against Google Cloud Platform (with private identifiers masked for privacy, conforming to article conventions):

### 
  
  
  5.2 Rigorous Empirical Audit & Evaluation

The empirical execution log confirms four pivotal architectural milestones:

**Flawless End-to-End Execution Latency (35 Seconds Total)** :
 The complete sequence—spanning isolated GCP infrastructure provisioning, live document creation on Google Drive, PDF binary conversion, Gemini embedding generation, Parquet writing, four full BigQuery distributed vector searches, and complete resource teardown—completed in approximately**35 seconds** . This consumes only**9.7%** of the Google Apps Script 6-minute ceiling, proving high operational headroom.
2. 
**Cross-Modal Semantic Discrimination** :
 Across all four test queries, the target assets were ranked**#1 with statistically significant margin** :
   - 
**Query 1 (Apps Script Automation)** : Correctly isolated`DOC-001` (`Google_Apps_Script_Automation_Guide.pdf` ) with distance`0.2500` (similarity`0.7500` ), clearly outranking the Sheets guide (`0.6324` ) and Docs guide (`0.5681` ).
  - 
**Query 2 (Spreadsheet Formulas & Pivot Tables)** : Matched`DOC-002` (`Google_Sheets_Calculations_Analytics_Guide.pdf` ) as #1 with similarity`0.7935` , demonstrating fine-grained semantic separation between distinct Workspace tools.
  - 
**Query 3 (Superconducting Quantum Qubits)** : Matched the raw text whitepaper`TXT-001` with an outstanding similarity of`0.8669` (distance`0.1331` ), cleanly distinguishing physics from the unrelated biological text (`0.5012` ).
  - 
**Query 4 (Visual System Architecture Diagram)** : Matched`IMG-001` (`lakehouse_multimodal_architecture_diagram.png` ) as #1 with similarity`0.8762` , proving that binary image assets with descriptive metadata participate seamlessly in unified semantic search alongside textual documents.
**True Multimodal Storage Coexistence** :
The table inventory confirms that 28KB PDF documents, 70B image binaries, and 200B text files coexist within the identical Iceberg table schema on Google Cloud Storage, unified under standard 768-dimensional float embeddings.
4. 
**Test Suite Safety Verification (Zero Residual Clutter)** :
The post-test cleanup phase verified the immediate destruction of all temporary verification assets:
   - The test Iceberg table `multimodal_knowledge_assets` .
  - All three test Google Docs by explicit Drive File ID (`[xxxx-drive-doc1-id]` ,`[xxxx-drive-doc2-id]` ,`[xxxx-drive-doc3-id]` ).
  - The test BigQuery dataset `lakehouse_multimodal_xxxx` .
  - The test Cloud Storage bucket `gs://lakehouse-mm-xxxx` .
Zero orphaned files or uncollected storage charges remain for the evaluator.

## 
  
  
  6. Architectural Evaluation & Cost Comparison

| Dimension | Dedicated Vector DB (Pinecone / Milvus) | Custom Microservice + Cloud SQL | IcebergApp Multimodal Lakehouse | 
| **Hosting Infrastructure** | Dedicated Cloud Instance or SaaS Cluster | GKE / Cloud Run + Cloud SQL PostgreSQL (pgvector) | **100% Serverless (GAS + BigQuery + GCS)** | 
| **Monthly Base Cost** | $500 – $3,000 / month | $150 – $600 / month | **$0.00 (Pure Pay-per-Query On-Demand)** | 
| **Binary Payload Storage** | Not Supported (External S3/GCS required) | External S3/GCS required | **Native in Table (`BYTES` / Storage URI)** | 
| **Google Drive Integration** | Webhooks + Complex OAuth Microservice | Scheduled Batch Ingestion Cron Jobs | **Native 1-Line Ingestion (`DriveApp` + `IcebergApp`)** | 
| **Document Versioning** | Manual vector ID tracking | Custom relational snapshot tables | **Native Iceberg Snapshots & Time Travel** | 
| **Query Engine Interoperability** | Proprietary Vector API only | SQL (pgvector) | **Open Standard (BigQuery, Spark, Trino, DuckDB)** | 

## 
  
  
  7. Conclusion

By combining Google Apps Script with Apache Iceberg and the Gemini embedding model, developers are no longer constrained by the traditional boundaries between structured spreadsheets, unstructured documents, and dedicated vector databases.

1. 
**True Multimodality** : Apache Iceberg's native binary support (`BYTES` ) allows Google Workspace assets (Docs, Sheets, Slides via automatic PDF export, and Forms), diagrams, and text records to live together in a single open Parquet table.
2. 
**Elimination of SaaS Subscriptions** : Zero recurring fees for dedicated vector database infrastructure; queries execute on-demand with BigQuery acceleration.
3. 
**Turn-Key Accessibility** : With**IcebergApp** , any Google Workspace developer can deploy a production-grade multimodal lakehouse vector engine using standard JavaScript.
4. 
**AI-Driven Google Drive File Discovery** : Because`file_id` is preserved directly in the schema, vector search serves as an intelligent semantic file finder across Google Docs, Sheets, Slides, and Forms—allowing users to navigate directly to live collaborative assets and enabling scripts to trigger downstream Workspace automation (`DocumentApp` ,`SpreadsheetApp` ,`SlidesApp` ,`FormApp` ,`DriveApp` ).

## 
  
  
  References & Resources