The Model Scored 30%. The Harness Scored 100%. Which One Did You Benchmark?
A new analysis reveals that benchmark scores for ARC-AGI-3 vary by up to 70 points depending on the harness used, with the same model scoring 30.16% in the official harness and 100.00% in custom harne…