Starting and iterating on a Kaggle competition in Google Antigravity #
I spent a couple of days on a Kaggle Playground Series competition (S6E7, predicting a three-class health_condition
from tabular health data), mostly as an excuse to try Google Antigravity 2.0 with Gemini 3.1 Pro, as part of my Google Developer Expert activity. I wanted to see if an agentic IDE can take a competition from a cold start to a good leaderboard submission. Google Antigravity 2.0 is Google’s agentic development platform, launched a couple of months ago. Unlike a normal IDE, it is built around AI agents that plan tasks, write and run code, and operate the editor, terminal, and browser on their own, then present the work as reviewable artifacts. The agent runs on
, Google’s model for complex reasoning and coding. Gemini 3.1 Pro I did not start from scratch. Before opening Antigravity, I used Claude Code to build a starter kit for tabular competitions based on my previous experience: a general playbook, two prompts (one to run the pipeline end-to-end, one to drive an improvement loop), and a set of agent prompts/commands. Then I added the competition data and ran the initial prompt in Antigravity. The full code is available on GitHub.
This post is about the experience of running a Kaggle competition in an agentic environment and about the importance of setting correct constraints and validation checks.
The starting direction
A multi-agent pipeline only works if all models agree on the rules. If different model scripts create their own cross-validation splits, they can’t be compared and their out-of-fold predictions can’t be blended. That’s why I defined the rules beforehand: a fixed fold split that every model reads, a pre-defined out-of-fold and test-prediction schema that every model writes, and a “memory” (EXPERIMENTS_LOG.md
) that every run appends to.
My initial prompt included a standard approach to tabular competitions: EDA, feature engineering, three models trained in parallel (LightGBM, CatBoost, and a PyTorch MLP), a hill-climbing blend, and a submission to Kaggle. The most important rule was to spend considerable time setting up cross-validation and then using it for all subsequent runs. The second prompt was a loop that read the scoreboard, picked the highest-value ideas, implemented them, judged on CV, and logged the result. The loop repeated until the goal was met or ideas were exhausted.
Running the loop in Antigravity
I pasted the first prompt, and Gemini set up the folds, ran the EDA, built about 140 features (missing indicators, frequency encodings, numeric ratios, out-of-fold target encoding without leaks), and trained the three models. Antigravity runs long jobs as background tasks, so the three models trained concurrently, and Gemini could set a wake-up timer and check the results instead of monitoring a running process. Given the second prompt, the loop mostly ran itself: read the scoreboard, pick the highest-value ideas, implement them, evaluate on CV, log the result.
I used high reasoning to ensure the best quality, but it was at a cost. On the high reasoning setting, trivial edits took minutes and anything substantial took hours. Switching Gemini to low reasoning for implementation sped up the iterations significantly, and I kept high reasoning only for the few steps that actually needed it.
The wall
The first pipeline produced a CV of 0.967 accuracy. I submitted it and saw that something went wrong - the public leaderboard score was 0.874. A Kaggler who sees such a gap usually assumes the CV is wrong. However, the agent’s instinct was to get more data. It proposed down a Kaggle dataset of pseudo-labels and “known noisy IDs” for this competition, and it made the case with real confidence: this, it said, was “the only way to break past the 0.874 wall.”
The problem was that the dataset was just a user-uploaded file. Its pseudo-labels are one competitor’s model output and its “noisy ID” list is that model’s guesses, so training on them would have meant fitting our models to someone else’s solution, with no guarantee the labels were right. I told the agent it was unofficial and to stop. It stopped, reverted the change, and then did something more plausible: it dug up a public dataset the synthetic competition data had been inspired by and used that to augment instead.
That was the better hunch, but it was wrong again: the original competition stated that the second dataset was an inspiration, but the feature distributions were different. As expected, this external data didn’t improve CV and was dropped. Another attempt was confident-learning label cleaning - it was worse too, scoring 0.866 on the leaderboard, lower than the previous score.
The wall was two bugs
Both real causes were simple, and both were in the evaluation pipeline.
The first was the metric. The competition scored balanced accuracy, but the config and the EDA had been left on plain accuracy. With a 14.9x class imbalance (592561 at-risk
rows against 39803 fit
), plain accuracy rewards a model for predicting the majority class and ignoring the rest, which is what the models learned: a strong accuracy CV and a low balanced-accuracy leaderboard score. The fix was to go back to my original rules: use the competition metric and set up the validation based on it.
The second was subtler. An earlier label-cleaning experiment had written a file listing the rows to keep, and the data used it: load_train()
dropped the noisy rows and reset the index, so the training data became smaller, while the precomputed fold indices still held the original row references. Two files didn’t match, so fold 0’s out-of-fold predictions were zeros.
I admit it was my mistake not to notice it immediately. By the way, Claude failed to diagnose this issue either (I showed it the code repository and asked it to analyze possible causes of the CV-LB gap), so it is a common trap for agents.
The fix and score improvement
Both problems were simple to fix once found. I ensured the training data was used correctly, used the appropriate evaluation metric, added class weights to the model training, and optimized the blend based on balanced accuracy. The blend scored 0.97498 out-of-fold and 0.94886 on the public leaderboard. The models trained noticeably slower with class weight, because reweighting the rare class changes the gradient and early-stopping iterations changed too.
This score is far below public notebooks that have already been tuned, but it is a good score for several experiments. And my goal was to make a submission with a reasonable score and iterate on it.
Where Gemini and Antigravity helped, and where they didn’t
Where it worked well:
- Running sub-agents. Gemini spawned the EDA, feature, model, and blender sub-agents, and they shared the context.
- Background tasks and scheduling. It ran the three models in parallel, and I could check the results later.
- Systematic iteration. Given the loop prompt, it assessed the scoreboard and logged every run without being reminded.
Where it struggled:
- Speed on high reasoning. Minutes for small tasks, hours for large ones; low reasoning was a good trade-off between quality and speed.
- Lack of logging or intermediate outputs. It was hard to see what it was doing in the middle of a long run, and I had to wait for the final result to know if it worked.
- Confidence at wrong times. It was ready to train on an unofficial dataset and call it “the only way” forward.
The bug that never existed
I first drafted a write-up with Gemini, and it added one funny hallucination. While explaining the fold-0 zeros, it blamed a folds < f
typo in the training template, which was a plausible explanation that fits the symptom exactly. I almost believed it, as it convinced me that the bug was “pre-existing”. Only when I opened the template and showed Gemini folds != f
it grepped through the project history and confirmed folds < f
had never been there. The real cause was the row mismatch above. The agent invented a bug to fit a symptom I described, defended it when I pushed back, and I nearly believed it.
Conclusion
Antigravity and Gemini 3.1 Pro were fun to use, and they made starting and iterating on a competition much easier. But using them reinforced one of the oldest lessons on Kaggle: setting the correct cross-validation and using the correct metric is the foundation on which everything else is built. The agents have lowered the cost of participating in a competition, but they are not a substitute for understanding the problem and the data. The human in the loop is still essential for reviewing the code and ensuring that the agents are taking the right direction.