# AI Agent Coding Comparison: The Rust Leap Year Challenge > Source: > Published: 2026-07-13 12:06:51+00:00 ## AI Agent Comparison I ran a little exercise to give the following 5 AI Agents the same prompt and see how they would fare, in my opinion, in completing the task: - [ChatGPT: GPT-5.5](https://chatgpt.com/)from[OpenAI](https://openai.com/) - [Qwen: Qwen 3.7-plus](https://qwen.ai/)from[Alibaba Cloud](https://www.alibabacloud.com/)via Auto Efficient model from[Kilo Code](https://kilo.ai/) Except for the Qwen model, which used some Kilo credits (<$0.02), all the models are free to use; there are no subscription plans at all. I was signed in as a registered user for all the agents. ### Summary (if you do not want to read the whole post) Gave each model a short exercise to produce some [Rust](https://rust-lang.org/) code and see how each performs. In the prompt, I asked the models to ask clarifying questions if there is any doubt; do not make any assumptions. Did any model ask, yes 2 did: - *ChatGPT*asked whether the Gregorian calendar leap year should be implemented.- Gregorian calendar - *Kilo Code Auto Efficient Qwen 3.7-plus*asked whether modulo (%) can be used and also whether it should just provide the code or provide the code and execute.- Yes, % can be used; just provide the code. - *Note:*There was no need to ask whether modulo can be used; this is an operator just like`+ , - , / or *` - All the models produced decent code, but only 3 used the `#[cfg(test)]` test harness that Rust provides: - *CharGPT* - *Claude* - *Kilo Code Hy3* The other 2, *Gemini* and *Kilo Code Auto Efficient Qwen 3.7-plus*, include some test code as part of the `main()` function with `println!()` output. Even though only *Kilo Code Efficient Qwen 3.7-plus* asked whether % can be used, all except *Kilo Code Hy3* used it. *Kilo Code Hy3* implemented a `remainder()` function to determine a remainder of 0 after division. If I have to pick a model based on the prompt, then I would go with either of the * Kilo Code* models since its output is very detailed and it explains it’s thinking and what it was going to do. ### So what was the goal here? It was to determine: - How well does the model “listen” to the prompt, did they get the task right? - Were any follow-up questions asked to clarify something the model did not understand? - Did the model make any assumptions? - How accurate was the model in solving the task? Note, I was not interested this time at all in the cost and/or time that it took the model to solve the task. Before we look at the result, let’s see how they. stack up on [Kilo Bench](https://kilo.ai/leaderboard): | Model | Code Rank | Coding Completion | Cost/Attempt | |---|---|---|---| | Sonnet 5 Medium | 10 | 59.6% | $36.19 | | GPT-5.5 | 54 | 74.2% | $72.63 | | Gemini 3.5 Flash | 57 | 64.7% | $104.49 | | Qwen 3.7-plus* | 56 | 54.6% | $20.65 | | Hy3 (free) | 34 | 47.6% | $0.00 | - Used the data from Qwen 3.7-max (slightly more expensive) ### Prompt Here is the prompt I gave each of the models. Create a Rust function from first principles that returns TRUE or FALSE for a given number whether it is a leap year. Use no crates, dependencies or built-in standard traits. Do not read, write or reference any local files; just provide the code as text in a code block. Also, create tests and test the function for the following set of numbers: 1, 4, 2023, 2024, 2025, 2048, 1900, 2100, 1600, 2000 Do not make any assumptions, and if in doubt, then ask me. ### Detail - **ChatGPT**- *What did it get right?*- Asked clarifying question(s) - Made use of Rust test harness - Implement the ask, leap year function correctly. - - *What did it get wrong?*- Gave no explanation or plan on what it is going to do. - Test was created as 1 big test instead of 10 individual ones. - - *Personal Rating:*7/10 - - **Claude**- *What did it get right?*- Test cases was split so that one can run each individualy. - Gave an explanation of what it did not use. - Implement the ask, leap year function correctly. - - *What did it get wrong?*- Made assumption on what calendar to use - Was very brief with no plan - - *Personal Rating:*6/10 - - **Gemini**- *What did it get right?*- Implement the ask, leap year function correctly. - *What did it get wrong?*- No test case, test was in the `main()` function - Assume the calendar to use. - Again very brief with no plan. - - *Personal rating:*5/10 - - **Kilo Code Auto Efficient: Qwen 3.7-plus**- *What did it get right?*- Asked clarifying questions - Implement the ask, leap year function correctly. - Was very detailed on what it was going to do, almost too detailed but I prefer it. - - *What did it get wrong?*- Did not use the Rust test harness, tests. was just some comparisons and `println!()` output. - Did not use the Rust test harness, tests. was just some comparisons and - *Personal rating:*8/10 - - **Kilo Code Hy3**- *What did it get right?*- Use test harness, although only 1 test case. - As is common for Kilo Code, the output was very detailed and explained exactly what it was going to do. - - *What did it get wrong?*- Would have preferred individual tests for the 10 tests. - Should have used modulo (%) operator instead of creating a `remainder()` function. - Assumed the calendar to use. - - *Personal rating:*8/10 - See the References below for the raw output from each model. #### Conclusion All the models basically did what was asked of them. The Kilo Code models were very detailed and explained what it was going to do very well. In a real-life situation, I would ask the model to first plan what it would do and then based on an agreed plan, implement that plan. The implemention/coding model might be totally different from the planning model. Models perform better or worse depending on what they are asked to do whether it is just some simple question, a detailed plan or a coding implementation based on an agreed plan. ### References Here is the raw data from each of the models: