# Show HN: A benchmark for flight search intent

> Source: <https://fjmatrix.github.io/flight-search-intent-eval/>
> Published: 2026-09-03 13:59:00+00:00

| 1 |
**gpt-5.6-sol** medium |
87%
212/244
|
|
**86%**
*n=59*
|
**87%**
*n=153*
|
**67%**
*n=12*
|
**84%**
*n=116*
|
**83%**
*n=109*
|
**100%**
*n=7*
|
**100%**
*n=3*
|
— |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
240/244 | search type *oneway · roundtrip · multi* |
223/233 | origin *departure locations* |
225/233 | destination *arrival locations* |
220/233 | travel dates *departure and return* |
180/192 | stay length *nights* |
72/82 | passengers *adults, children, infants* |
15/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
28/32 | date validity *no past or out-of-order dates* |
229/229 | trip shape *leg count, return xor duration* |
228/229 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 742 / 242 / 120 |
| total tokens | 240k |
| total cost | no price set |
| cost per case | — |
| median latency | 3.9s |
| errored calls | 0 |
### By language| de |
43/50 |
|
86% | | en |
130/150 |
|
87% | | es |
39/44 |
|
89% |
|
| 2 |
**gpt-5.6-terra** medium |
86%
209/244
|
|
**93%**
*n=59*
|
**86%**
*n=153*
|
**17%**
*n=12*
|
**79%**
*n=116*
|
**79%**
*n=109*
|
**100%**
*n=7*
|
**100%**
*n=3*
|
— |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
237/244 | search type *oneway · roundtrip · multi* |
222/233 | origin *departure locations* |
222/233 | destination *arrival locations* |
223/233 | travel dates *departure and return* |
179/192 | stay length *nights* |
66/82 | passengers *adults, children, infants* |
16/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
31/32 | date validity *no past or out-of-order dates* |
226/226 | trip shape *leg count, return xor duration* |
225/226 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 742 / 205 / 86 |
| total tokens | 231k |
| total cost | no price set |
| cost per case | — |
| median latency | 3.3s |
| errored calls | 0 |
### By language| de |
45/50 |
|
90% | | en |
125/150 |
|
83% | | es |
39/44 |
|
89% |
|
| 3 |
**claude-opus-5** medium |
85%
207/244
|
|
**92%**
*n=59*
|
**86%**
*n=153*
|
**75%**
*n=12*
|
**86%**
*n=116*
|
**80%**
*n=109*
|
**0%**
*n=7*
|
**67%**
*n=3*
|
— |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
231/244 | search type *oneway · roundtrip · multi* |
224/233 | origin *departure locations* |
224/233 | destination *arrival locations* |
213/233 | travel dates *departure and return* |
182/192 | stay length *nights* |
79/82 | passengers *adults, children, infants* |
16/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
29/32 | date validity *no past or out-of-order dates* |
230/236 | trip shape *leg count, return xor duration* |
235/236 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 2725 / 272 / 0 |
| total tokens | 731k |
| total cost | $2.09 |
| cost per case | $0.0086 |
| median latency | 4.6s |
| errored calls | 0 |
### By language| de |
37/50 |
|
74% | | en |
137/150 |
|
91% | | es |
33/44 |
|
75% |
|
| 4 |
**claude-sonnet-5** medium |
79%
192/244
|
|
**86%**
*n=59*
|
**80%**
*n=153*
|
**46%**
*n=13*
|
**77%**
*n=116*
|
**73%**
*n=110*
|
**0%**
*n=7*
|
**67%**
*n=3*
|
2 |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
231/243 | search type *oneway · roundtrip · multi* |
218/232 | origin *departure locations* |
217/232 | destination *arrival locations* |
214/232 | travel dates *departure and return* |
179/190 | stay length *nights* |
75/82 | passengers *adults, children, infants* |
16/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
28/32 | date validity *no past or out-of-order dates* |
229/236 | trip shape *leg count, return xor duration* |
231/236 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 2725 / 302 / 0 |
| total tokens | 742k |
| total cost | $0.89 |
| cost per case | $0.0036 |
| median latency | 3.9s |
| errored calls | 2 |
### By language| de |
36/50 |
|
72% | | en |
125/151 |
|
83% | | es |
31/44 |
|
70% |
|
| 5 |
**gemini-3.5-flash-lite** medium |
74%
181/244
|
|
**85%**
*n=59*
|
**77%**
*n=153*
|
**8%**
*n=12*
|
**68%**
*n=116*
|
**64%**
*n=109*
|
**0%**
*n=7*
|
**67%**
*n=3*
|
25 |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
207/219 | search type *oneway · roundtrip · multi* |
201/211 | origin *departure locations* |
199/211 | destination *arrival locations* |
194/211 | travel dates *departure and return* |
156/171 | stay length *nights* |
61/69 | passengers *adults, children, infants* |
16/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
24/28 | date validity *no past or out-of-order dates* |
204/209 | trip shape *leg count, return xor duration* |
209/209 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 92 / 897 / 748 |
| total tokens | 241k |
| total cost | no price set |
| cost per case | — |
| median latency | 3.0s |
| errored calls | 25 |
### By language| de |
35/50 |
|
70% | | en |
120/150 |
|
80% | | es |
26/44 |
|
59% |
|
| 6 |
**gemini-3.5-flash** medium |
74%
181/244
|
|
**80%**
*n=59*
|
**78%**
*n=153*
|
**25%**
*n=12*
|
**72%**
*n=116*
|
**72%**
*n=109*
|
**0%**
*n=7*
|
**67%**
*n=3*
|
32 |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
207/212 | search type *oneway · roundtrip · multi* |
201/204 | origin *departure locations* |
196/204 | destination *arrival locations* |
189/204 | travel dates *departure and return* |
165/167 | stay length *nights* |
65/65 | passengers *adults, children, infants* |
15/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
24/26 | date validity *no past or out-of-order dates* |
204/209 | trip shape *leg count, return xor duration* |
209/209 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 92 / 986 / 830 |
| total tokens | 263k |
| total cost | no price set |
| cost per case | — |
| median latency | 3.8s |
| errored calls | 32 |
### By language| de |
30/50 |
|
60% | | en |
120/150 |
|
80% | | es |
31/44 |
|
70% |
|
| 7 |
**gemini-3.8-flash** medium |
74%
181/244
|
|
**83%**
*n=59*
|
**73%**
*n=153*
|
**8%**
*n=12*
|
**63%**
*n=116*
|
**63%**
*n=109*
|
**100%**
*n=7*
|
**100%**
*n=3*
|
44 |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
187/200 | search type *oneway · roundtrip · multi* |
175/189 | origin *departure locations* |
175/189 | destination *arrival locations* |
174/189 | travel dates *departure and return* |
141/150 | stay length *nights* |
49/57 | passengers *adults, children, infants* |
14/15 | cabin *cabin class* |
5/5 | filters *stops, price, bags, connections* |
20/21 | date validity *no past or out-of-order dates* |
176/176 | trip shape *leg count, return xor duration* |
176/176 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 92 / 834 / 704 |
| total tokens | 226k |
| total cost | no price set |
| cost per case | — |
| median latency | 2.8s |
| errored calls | 44 |
### By language| de |
31/50 |
|
62% | | en |
118/150 |
|
79% | | es |
32/44 |
|
73% |
|
| 8 |
**gpt-5.6-luna** medium |
71%
174/244
|
|
**81%**
*n=59*
|
**71%**
*n=153*
|
**17%**
*n=12*
|
**66%**
*n=116*
|
**64%**
*n=109*
|
**57%**
*n=7*
|
**100%**
*n=3*
|
— |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
234/244 | search type *oneway · roundtrip · multi* |
223/233 | origin *departure locations* |
204/233 | destination *arrival locations* |
205/233 | travel dates *departure and return* |
171/192 | stay length *nights* |
61/82 | passengers *adults, children, infants* |
15/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
28/32 | date validity *no past or out-of-order dates* |
226/229 | trip shape *leg count, return xor duration* |
228/229 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 742 / 257 / 135 |
| total tokens | 244k |
| total cost | no price set |
| cost per case | — |
| median latency | 3.4s |
| errored calls | 0 |
### By language| de |
36/50 |
|
72% | | en |
106/150 |
|
71% | | es |
32/44 |
|
73% |
|
| 9 |
**claude-haiku-4-5** |
53%
129/244
|
|
**41%**
*n=59*
|
**56%**
*n=153*
|
**17%**
*n=12*
|
**47%**
*n=116*
|
**31%**
*n=109*
|
**71%**
*n=7*
|
**100%**
*n=3*
|
— |
*▸* |
### Per-dimension *pass rate over the repeats that scored it*
action *search the query or reject it* |
204/244 | search type *oneway · roundtrip · multi* |
171/233 | origin *departure locations* |
180/233 | destination *arrival locations* |
174/233 | travel dates *departure and return* |
143/192 | stay length *nights* |
61/82 | passengers *adults, children, infants* |
14/16 | cabin *cabin class* |
6/6 | filters *stops, price, bags, connections* |
23/32 | date validity *no past or out-of-order dates* |
184/197 | trip shape *leg count, return xor duration* |
177/197 |
### Cost & latency
tokens in / out / reasoning
*mean per call* | 2373 / 112 / 0 |
| total tokens | 606k |
| total cost | $0.72 |
| cost per case | $0.0029 |
| median latency | 1.9s |
| errored calls | 0 |
### By language| de |
19/50 |
|
38% | | en |
79/150 |
|
53% | | es |
31/44 |
|
70% |
|