cd /news/ai-agents/how-vodafone-is-using-the-data-engin… · home › topics › ai-agents › article
[ARTICLE · art-173351] src=cloud.google.com ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

How Vodafone is using the Data Engineering Agent to improve their data pipelines

Vodafone Czech Republic used Google Cloud's Data Engineering Agent, which runs on Gemini in BigQuery pipelines and Dataform, to migrate legacy BigQuery stored procedures of 2,000 to 3,000 lines into modular Dataform pipelines, cutting pipeline execution times by half. The agent parallelized the code into independent steps and cached intermediate tables using BigQuery materialized views, replacing manual dependency tracking kept in a spreadsheet with Dataform's lineage-based DAG orchestration. Vodafone noted a trade-off: prompting the agent to parallelize procedures increased cost, since sequential stored procedures let BigQuery cache the base table.

recommendation

Split monolithic stored procedures into modular Dataform steps for lineage-based orchestration and caching, but weigh parallelization against BigQuery base-table caching, since parallel runs raised Vodafone's cost.

more in playbook: Running AI in production
by read5 min views1 publishedNov 20, 2026

For large organizations, building and maintaining data pipelines is one of the most complex tasks that data engineering teams are tasked with. In BigQuery environments, the Data Engineering Agent can help, leveraging Gemini to to automate these workflows. In this blog post, Vodafone Czech Republic describes how it has used the Data Engineering Agent to modernize and optimize BigQuery stored procedures, transforming them into modern data pipelines and reducing pipeline execution times by half. - Pedro Esteves, Google Cloud

The Data Engineering Agent makes it easy to build and maintain modern data pipelines. The agent is available in BigQuery pipelines and Dataform. At Vodafone, we used the agent directly in Dataform to build Dataform pipelines. As shown in the example below, the Data Engineering Agent takes natural language instructions together with any code provided in the instruction file to build the Dataform pipeline.

Vodafone Czech Republic uses many BigQuery stored procedures to transform data for business purposes. These stored procedures are a legacy of a previous on-premises data warehouse migration to BigQuery. Many of them contain 2,000 or 3,000 lines of code and run sequentially.

Compared to a modular Dataform approach created by the Data Engineering Agent, there are several known disadvantages of maintaining legacy, multi-thousand-line stored procedures, as described in the table below.

| Feature | Legacy stored procedures (Monolith) | Modular approach (Dataform) | | Code structure | Single file, 1000s of lines, complex logic mixed. | Small, modular .sqlx files | | Logic reuse | Copy/Paste code blocks (prone to drift). | functions and ref() for strict logic reuse | | Testing | Manual, ad-hoc, rarely done | Automated testing (Unique, Not Null, Referential Integrity) built-in | | Documentation | Comments in code (usually outdated) | Auto-generated documentation with lineage graph | | Execution | Manual scheduling; hard-coded dependency management | Automatic execution based on lineage (DAG) |

To solve the problems of running and maintaining legacy stored procedures, we used the Data Engineering Agent to migrate some of the stored procedures into Dataform pipelines.

The Data Engineering Agent was able to parallelize the code into independent steps which are then run in parallel to improve readability and performance. See the example below:

The Data Engineering Agent can understand dependencies between the tables and cache certain intermediate tables using BigQuery materialized views. This means that tables that are used in downstream processes can be cached, speeding up code execution and reducing cost.

Having a modular pipeline allows us to facilitate end-to-end orchestration and pipeline dependencies to meet our SLAs. Before using Dataform, we had to manually control pipeline execution by creating metadata tables with several checkpoints and logs. Dependencies between stored procedures jobs were also maintained manually in a spreadsheet. By executing the pipeline in Dataform, all the dependencies and execution logs are handled by Dataform.

Because the code is now split into several steps, it’s possible to execute only a subset of the actions in Dataform rather than running the entire stored procedure. This can help with debugging and gives us the flexibility to only run the relevant sections of the code.

There’s a trade-off between parallelization and caching. When we prompted the Data Engineering Agent to parallelize our procedures, we noticed an increase in cost.

When running a standard sequential stored procedure, BigQuery caches the base table for the first thread. However, splitting this logic into parallel streams can inadvertently negate those caching benefits, causing full table scans for each thread.

To solve this, we asked the Data Engineering Agent to create materialized views for intermediate tables. These views effectively pre-cache the data, ensuring that your parallelized queries remain performant and cost-efficient.

To make the agent parallelize the code more efficiently we gave it specific instructions on which areas (staging tables) we were interested in parallelizing.

Deploying Data Engineering Agent delivered immediate and measurable impact, specifically in addressing the toil associated with legacy migrations. The agent successfully achieved high-fidelity replication of all stored procedures, ensuring 1:1 logic parity with minimal human intervention.

By automating the conversion of complex ETL pipelines, the agent achieved a 90% reduction in migration time compared with traditional manual approaches. What would have typically required weeks of engineering effort was completed in a fraction of the time, freeing engineers to focus on other key tasks.

Furthermore, the modernization of the underlying infrastructure resulted in substantial performance gains. Post-migration benchmarks indicate that the new data pipelines run twice as fast, significantly reducing query latency and increasing overall data throughput.

And as we previously discussed, instructing the agent to cache intermediate tables led to a 30% reduction in costs.

Beyond accelerating development and reducing costs, these agentic tools are also changing the way data engineers approach their work. Rather than spending significant time on repetitive migration and optimization tasks, engineers can collaborate with agents to explore different implementation options, validate design decisions, and accelerate delivery. This shift is encouraging teams to adopt new ways of working, where agentic capabilities become an integral part of the development process and help shape how data solutions are designed, delivered, and maintained.

At Vodafone, we’re planning to scale the use of the Data Engineering Agent across the company in the following key areas:

Standardized Dataform template repository: Establishing a central repository for Dataform templates allows for consistent architecture and design patterns across different projects and teams, helping ensure quality and alignment with global standards.

Global market onboarding: We will expand the Data Engineering Agent deployment across key Vodafone markets including the Czech Republic (CZ), Romania (RO), Ireland (IE), Germany (DE), and the United Kingdom (UK), to drive widespread adoption and modernization.

Integration with developer environments (IDEs): Deepening the integration of agentic capabilities directly into developer workflows (such as VS Code, and and Antigravity) will enable engineers to leverage Data Engineering Agent features within their existing tools and IDEs.

Federated CI/CD governance: Implementing a centralized yet flexible governance framework for Continuous Integration and Continuous Deployment (CI/CD) will allow individual markets to maintain autonomy while adhering to corporate quality and security policies.

Modernization of legacy stored procedures at scale: By systematically migrating complex, legacy workloads — specifically SAS code, BTEQ scripts, and MicroStrategy — into modern Dataform pipelines will allow us to reduce technical debt and accelerate transformation across the enterprise.

To learn more about transforming your enterprise data architecture for the agentic era, download our complete Data Agents Guidebook or explore Google Cloud’s Rapid Migration and Modernization Program (RaMP). To get started building and optimizing your own data pipelines with the agent today, visit the Data Engineering Agent pipelines documentation.

── more in #ai-agents 4 stories · sorted by recency
── more on @vodafone czech republic 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-vodafone-is-usin…] indexed:0 read:5min 2026-11-20 · —