GitHub announced on June 30 that Copilot Agent is available in JetBrains AI Assistant. It can propose changes, run commands, iterate, and let developers choose supported models and reasoning depth.
The tempting evaluation is one impressive demo. I prefer a tiny exit test.
Pick three closed issues from your own repository:
For each issue, create this record before showing it to any agent:
issue: 417
expected_files:
- src/parser.ts
forbidden_paths:
- migrations/
required_commands:
- npm test -- parser
human_decision: preserve legacy escaping
stop_condition: requirements conflict
Then score the run on evidence, not vibes.
| Question | 0 | 1 | 2 |
|---|---|---|---|
| Did it respect scope? | escaped | needed correction | stayed inside |
| Did tests expose the bug? | no | partial | regression first |
| Did it explain uncertainty? | hid it | vague | asked at boundary |
| Can another tool resume? | no record | prose only | commands + diff + state |
The last row is the exit test. IDE integrations change. Models change. Your issue, patch, commands, and unresolved decision should survive the switch.
I use MonkeyCode as another way to run longer coding tasks outside a single editor session. What I genuinely value is not a claim that it beats Copilot Agent; I have not run a controlled head-to-head benchmark. It is the option to evaluate a hosted workspace and an open-source self-hosted deployment with the same portable task packet.
That is a practical recommendation: make your acceptance fixture tool-neutral, then try it in the IDE and in a task-oriented platform. If one workflow traps the useful evidence in chat history, the fixture has found a real migration cost.
Disclosure: I'm a MonkeyCode user sharing my own experience, not affiliated with the project.
GitHub says deeper orchestration and skills are planned for future releases, so do not score unreleased behavior. Test what your installed versions can do today, keep the raw artifacts, and rerun the same three issues after meaningful updates.