Since we are talking about AI replacing all human jobs, let's discuss banana sorters. Somewhere in this world, there are people who are doing this task (all day?) of banana sorting. They find an overripe banana, they trash it (hopefully compost) or else they put it in the good banana basket. Let's say that they could make better use of their time. So, I'll build them a banana sorter.
Building the decision machine #
I am not a hardware expert, so I will just focus on the decision system. I am assuming that a conveyor belt system can be designed easily to serve our purpose. The Banana Intelligence System uses a camera to decide between trash and fresh.
Asking the machines #
Let's take five sample images of bananas and ask the models.
Models used:
| Model | Weights | Offline vs cloud |
|---|---|---|
| Qwen2.5-VL 3B | Open | Offline |
| Gemma 3 4B | Open | Offline |
| Liquid d1-2B | Open | Offline |
| gpt-6.1-sol | Closed | Cloud |
Prompt (Qwen2.5-VL, Gemma 3, gpt-6.1-sol), sent with the photoYou are the vision system on a fruit sorting machine. Bananas that are stale go to TRASH. All others go to the BASKET. Look at this banana. Answer with exactly one word: BASKET or TRASH. #
You are the vision system on a fruit sorting machine. Bananas that are stale go to TRASH. All others go to the BASKET. Look at this banana. Answer with exactly one word: BASKET or TRASH.
Prompt (Liquid d1-2B, as a choice between two options), sent with the photoYou are the vision system on a fruit sorting machine. Bananas that are stale go to TRASH. All others go to the BASKET. Where does this banana go? BASKET: The banana is not stale. TRASH: The banana is stale. #
You are the vision system on a fruit sorting machine. Bananas that are stale go to TRASH. All others go to the BASKET. Where does this banana go?
BASKET: The banana is not stale TRASH: The banana is stale
The results of the runs are below:
| TRASH, of 100 runs | 1 Yellow | 2 Sugar spots | 3 Heavy spots | 4 Mostly brown | 5 Black |
|---|---|---|---|---|---|
| Qwen2.5-VL 3B | 0 | 100 | 100 | 100 | 100 |
| Gemma 3 4B | 0 | 100 | 100 | 100 | 100 |
| Liquid d1-2B, P(TRASH) | 0 (0.005) | 100 (0.90) | 100 (0.84) | 100 (0.98) | 100 (0.92) |
| gpt-6.1-sol (low) | 0 | 0 | 16 | 100 | 100 |
| gpt-6.1-sol (high) | 0 | 0 | 38 | 100 | 100 |
My interpretation of machine responses #
- The smaller (offline) VLMs, Qwen and Gemma, don't try hard to save the banana. Any black dot makes them trash it.
- The frontier model is in a dilemma: it trashes the banana with heavy spots once in a while.
- Liquid d1 gives a probabilistic response; the response will depend on the threshold value. It assigns a higher trash probability to the sugar spots banana than to the heavy spots banana.
- With higher reasoning effort, the same model trashes the heavily spotted banana more often. But neither setting gives a dependable answer for that banana.
Where the line is a judgment #
Now, let's revisit the question whether AI can replace human judgment. Let's make the question more generic: Based on the above example, would you trust a high-end model to make a critical decision for you? It might come up with very smart and complex reasoning to convince you to eat a spotty banana (or not). I would not.
Back to the sorter question #
The right way to design and build a banana sorter AI machine is much more nuanced.
If you had to create a machine that sorts bananas, how would you do it? Banana photos from Wikimedia Commons, resized to 640 px: 1 by Evan-Amos, CC BY-SA 3.0 · 2 by Slewrate, CC BY-SA 2.0 (also in the animation) · 3 by Scot Nelson, CC0 · 4 by Park taeho, CC BY-SA 4.0 · 5 by TwoWings, public domain.
