We saved ~half a million dollars last week with Jev.
And our products got much better at the same time.
Matthew Wilson, Co-founder/CEO @ Jack & Jill
Background
Jack & Jill is a talent marketplace brokered by AI agents. Jack is a career agent for top professionals and Jill is an agent for hiring teams. When Jack and Jill agree there is a mutual match, they introduce the candidate and the company directly, skipping the application altogether. To date, they’ve worked with over 5,000 companies and more than 400,000 professionals working in startups - and raised a total of $60M in the process.
Within 10 days of their first test, Jack & Jill had fully rolled out Jev in production - replacing Gemini 3.1 Flash Lite for 100% of calls for a key stage in their candidate matching pipeline.
Compared to Gemini, Jev maintained the same level of accuracy, with 88% lower cost and roughly 50% faster task completion time.
Cost savings are projected to be $500K per year - and more importantly, new use cases have been unlocked across the product, such as showing candidates matched jobs immediately after signup.
Impact
| Result | Details |
|---|---|
| ↓ $500K/year | Absolute projected savings |
| ↓ 88% cost | BEFORE | $0.755 with Google GeminiAFTER | $0.092 with TypeSafe Jev Cost per 1,000 candidates scored |
| 2x speed | BEFORE | 20.3 sec`` AFTER | 10.3 sec Median time to screen each candidate |
| No loss in quality | BEFORE | 93.9% BaselineAFTER | 94.6% Candidates requested by hiring managers |
The challenge: expensive to screen a large volume of candidates with Gemini #
Before Jev
- Candidate pool. Every day, 6,000 candidatesfor every open role are pulled from the database
- Initial scoring. A low-cost model (Gemini 3.1 Flash Lite) scores each candidate and creates a shortlist of the top 2,000 candidates
- Close review. A higher-power model assesses the top 2,000 candidates, ending with a list of a few hundred candidates for the hiring manager
Even with a “low-cost” model like Gemini Flash Lite, costs were adding up fast, due to the "Initial scoring" step that assessed 6,000 candidates for every open role.
Results with Jev: Similar quality to Gemini at 88% lower cost #
With Jev
Before switching, Jack & Jill tested both Gemini and Jev models on 150 live roles, and the same candidates for each role.
For each role, they included:
- every past “winner”: a candidate a hiring manager later asked to meet
- a sample of remaining candidates (weighted to accurately represent the original 6,000 candidates retrieved for each role)
The test counted how many winners each model kept on the shortlist. For example, a score of 95% means that for every 100 candidates that the hiring manager asked to be introduced to, the model correctly kept 95 of them in its shortlist.
Ultimately, Jack & Jill found that Jev was able to maintain similar quality to Gemini, with 88% lower cost and half the median time to complete scoring.
Overall, the team expects to save more than $500K based on projected volumes over the next 12 months.
Quality: Slightly higher than Gemini (Recall and AUC)
Jev kept slightly more winning candidates than Gemini (+0.7 percentage points), although the difference was not statistically significant (90% confidence interval of −0.4 to +1.8)
Jev also performed slightly better on AUC, a measure of how often a model ranks a winner above a randomly chosen candidate with no recorded hiring manager selection. 1.0 is perfect and 0.5 is a coin toss. Jev’s score of 0.933 means it ranked the winner higher in approximately 93.3% of those pairings, compared with 92.4% for Gemini — aligning better with the recorded hiring manager selections.
| Measure | Gemini 3.1 Flash Lite | Jev |
|---|---|---|
| % winners kept in the Initial Scoring of 2,000 candidates | 93.9% | 94.6% |
| Ranking quality (AUC) | 0.924 | 0.933 |
Speed: Roughly twice as fast
Median time to screen candidates dropped by 50% with Jev compared with Gemini.
| Measure | Gemini 3.1 Flash Lite | Jev |
|---|---|---|
| Median time to screen all candidates for a role | 20.3s | 10.3s |
Cost: 88% lower token cost, $500K in projected savings
Jev's input price is a sixth of Gemini's and its output is free. As a result, the cost per 1,000 candidates with Jev is just 9 cents - compared with 76 cents with Gemini.
Just at their current volume of roles, Jack & Jill will save $265K a year. Factoring in expected growth, the team expects to save $500K over the next twelve months.
| Measure | Gemini 3.1 Flash Lite | Jev |
|---|---|---|
| List price per million tokens | $0.25 in, $1.50 out | $0.042 in, output free |
| Cost per 1,000 candidates screened | $0.755 | $0.092 |
Improving user experience by unlocking new features #
Jev’s usage has expanded across more than 15 workflows across Jack & Jill's product, ultimately powering a better product experience for users. For instance:
- Matched roles at sign-up. Every new candidate is checked against every suitable open role straight after onboarding, for under $0.10 per candidate.
- A near-instant profile check. The check candidates wait on after onboarding dropped from about 3.6 seconds on Gemini to about 0.2 seconds, with better accuracy.
- Checks across the product, including the CV editor, company classification and marketplace filtering.
Savings from using Jev instead of Gemini in these additional features are excluded from the projections above.
Conclusion #
Within just 10 days, Jack & Jill used Jev as a “drop-in” replacement for Gemini 3.1 Flash Lite - achieving comparable results on quality, with 88% lower cost and 50% median completion time.
In addition, adopting Jev has unlocked new use cases that will power better user experience throughout Jack & Jill’s product, resulting in faster / better job matches for both hiring managers and professionals considering their next move.
Methodology: Quality figures come from an offline test on live roles. The $500K figure is a projection based on expected growth in search volume (cost savings based on current volume = $265K). Gemini list prices are as published by Google at the time of measurement.
∵ ⩆
⩆ ∵