This is a submission for the Kaggle Benchmarking Challenge. As a C++ learner, I wanted to explore how well AI models can identify common logical errors in code and suggest appropriate fixes.
For the DEV × Kaggle Benchmarking Challenge, I created a small benchmark on Kaggle called C++ Logical Bug Detection Benchmark. The goal is to test whether AI models can recognize common programming mistakes, explain why they occur, and suggest corrections.
The benchmark contains three tasks:
This task checks whether a model can identify incorrect variable usage in a summation loop and suggest the correct fix.
This task evaluates whether a model can identify incorrect loop boundaries and indexing that may cause an off-by-one error.
This task checks whether a model can detect incorrect array index usage and explain how to correct it.
I tested three AI models:
I selected these models to compare how different AI systems handle simple C++ logical bug-detection tasks.
All three models achieved a score of 100.00 on the benchmark.
| Model | Score |
|---|---|
| Qwen 3 Coder 480B | 100.00 |
| GPT-5.4 mini | 100.00 |
| Gemini 3.7 Flash | 100.00 |
All three models passed the three tasks, giving a total of 9 successful model-task evaluations out of 9.
The results show that all three models handled the three simple bug-detection tasks successfully.
However, this is a small benchmark with only three tasks. It does not establish how well these models perform on complex C++ debugging, large codebases, or real-world software projects.
In the future, I would like to expand the benchmark with more challenging logical errors, nested loops, pointer-related bugs, and edge cases.
Building this benchmark helped me understand how AI models can be evaluated using specific programming tasks and measurable results.
It also encouraged me to think about how to design better tests that go beyond simple examples.
You can explore my benchmark here:
C++ Logical Bug Detection Benchmark on Kaggle I would appreciate feedback and suggestions for adding more challenging C++ debugging tasks.