{"slug": "sloth-fridge-a-fridge-photo-turns-into-a-shopping-list", "title": "Sloth Fridge: A Fridge Photo Turns Into a Shopping List", "summary": "A developer built Sloth Fridge, an open-source tool that uses a local vision-language model (qwen2.5vl:3b via Ollama) to turn a single fridge photo into a categorized shopping list and to read receipts so purchased items are ticked off the pantry. The project runs entirely on a laptop with no hosting, scanning four sample fridge photos in roughly 40-56 seconds each. The developer found that prompting the small model for JSON directly caused it to hallucinate and repeat a single item, and that switching to plain-English prompts produced accurate item detection.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady](https://dev.to/challenges/hacktoberfest-week3-2026-10-19)*\n\nI built **Sloth Fridge**. You take one photo of your fridge and it gives you a shopping list.\n\nHere is the chore it removes. You open the fridge, the cold air is on your legs, you are staring at half a lemon and a tub of something, and you are trying to remember what you already bought. Then you write a list on your phone, you forget half of it in the shop, and you come home with three bags of the same thing.\n\nNow you take one photo and close the door. While you are out, a small AI model that runs on your own laptop looks at the photo, works out what you have, compares it with what you had last time, and tells you what has run out. When you get home, the list is waiting for you.\n\nIt also reads your shopping receipt. Everything on the receipt gets ticked off and added to your pantry, so the list does not ask for milk right after you bought milk.\n\nThe people who get to be lazy here are anyone who shops. It is most useful for people who cook at home and shop for one or two people, because those are the people who end up buying the same six things every week.\n\nThere is no live link and no video, because the whole point is that it runs on your own machine and there is nothing to host.\n\nThe repo has four real fridge photos and a sample receipt. You can run the whole thing on your laptop with one command:\n\n```\ngit clone https://github.com/sonu7089/sloth-fridge.git\ncd sloth-fridge\npip install -e .\nollama pull qwen2.5vl:3b\nollama pull qwen2.5:3b\npython scripts/demo.py\n```\n\nThis is real output from my machine. It scanned four fridge photos in about 40 seconds each and printed the list:\n\n```\nscanning fridge-01-inside-double-sided-refrigerator.jpg ...\n  19 item(s) in 56s\n    - hot sauce x4 piece [condiment]\n    - mustard x2 container [condiment]\n    - mayo x2 pack [condiment]\n    - yogurt x1 container [dairy]\n    - milk x3 container [dairy]\n    - soy sauce x2 container [condiment]\n    - tomato ketchup x1 container [condiment]\n    - eggs x6 pack [eggs]\n    - bread x1 loaf [bread]\n    - cheese x1 pack [dairy]\n    ...\n\nDAIRY\n  [ ] butter x2 pack  (out of stock)\n  [ ] milk x2 container  (running low, 1 container left)\n  [ ] yogurt x2 container  (running low, 1 container left)\n\nMorning digest\ncorn, cucumber, lemon, hot sauce\nfridge smells like a tropical fruit salad\n```\n\nThen it reads the receipt and everything on it is ticked off:\n\n```\nbought milk\nbought yogurt\nbought eggs\nbought cheese\nbought bread\nbought banana\nbought lime\nbought olive oil\nbought tomato ketchup\nbought black pepper\nbought coffee\n```\n\nYou can also drive it from the command line:\n\n```\nsloth-fridge scan fridge.jpg          # scan a photo\nsloth-fridge list                      # show the list\nsloth-fridge digest                    # two line morning summary\nsloth-fridge receipt receipt.jpg       # read a receipt\nsloth-fridge watch ~/Pictures/fridge   # scan new photos as they arrive\n```\n\nThere is a web version too:\n\n```\nstreamlit run sloth_fridge/ui.py\n```\n\n[https://github.com/sonu7089/sloth-fridge](https://github.com/sonu7089/sloth-fridge)\n\nThese are the open source parts I used:\n\nHere is the thing that took me the longest, and it is the reason this project works at all.\n\n`qwen2.5vl:3b` is a small model. When I asked it for JSON straight away, it broke. It made up one item and then repeated it twenty times:\n\n```\n{\"items\": [{\"name\": \"jalapeno jelly\", \"qty\": 1}, {\"name\": \"jalapeno jelly\", \"qty\": 1}, ...]}\n```\n\nIt was not really looking at the photo either. The fix was three things.\n\n**One, let the model talk instead of making it fill in a form.** I asked the same question in plain English, \"look at this photo of a refrigerator shelf and tell me which food you can actually see\", and the answer changed completely. It gave me \"Mustard, Ketchup, Hot sauce, Pickles, Kikkoman soy sauce, Honey\", and all of those really were in the photo. So now the vision model writes notes and the text model takes notes. Two jobs, two models.\n\n**Two, cut the photo into small tiles.** A small carton at the back of a shelf is invisible in one wide photo. So I cut the photo into a 3 by 3 grid of overlapping tiles and look at each one. Then the votes decide. An item that shows up in three tiles is real. An item that shows up in one tile gets asked about again.\n\n**Three, do the boring work in normal Python.** Brand names, pack sizes, units and vague answers like \"yellow lid\" are handled by code, not by the model. The model gives me \"Kikkoman soy sauce\", the code turns that into \"soy sauce\", and \"free range eggs 12\" becomes \"eggs\".\n\nThe rule I ended up with is simple. The model is good at seeing. It is bad at formats and bad at counting. So let it see, and let the code do the rest.\n\nAfter all of that, a scan takes about 40 seconds on my laptop.\n\nOpen is the reason this project is possible, not just nice to have.\n\n**It works with no internet.** There is no API key and no bill. Pull the two models once and the tool works on a plane or on a cheap second hand laptop. A closed vision API charges you for every photo and needs a working connection every time.\n\n**Your fridge stays on your fridge.** A photo of your fridge tells people your diet, your money, your health and your family. Uploading it to a company you do not control means sending it somewhere you do not control. My code makes exactly one network call and it goes to `127.0.0.1`, which is my own Ollama process. You can check that yourself:\n\n```\ngrep -rn \"https\\?://\" sloth_fridge/\n# config.py:50: OLLAMA_HOST = os.environ.get(\"OLLAMA_HOST\", \"http://127.0.0.1:11434\")\n```\n\n**It costs nothing to run.** After the first download the cost of a scan is zero, so I can scan every single day. With a paid API, the thing I would use most is the thing that costs the most.\n\n**I could fix the model because the model was a file.** The hallucination loop above was not a support ticket. I changed the prompt, I changed the sampling, I built a whole pipeline around it, and I can try a different checkpoint with one command:\n\n```\nSLOTH_VISION_MODEL=qwen2.5vl:7b sloth-fridge scan fridge.jpg\n```\n\nWith a closed API I can only change the prompt. With an open model I can change how the model is used.\n\n**It is mine.** The list of things I always keep stocked is a JSON file I own. Nobody can change what ends up on my shopping list, because there is no app running on someone else's server.\n\nTo be honest, a big closed model would read a fridge photo better than a 3B model does. What open wins for me here is privacy, cost, and the fact that I can fix it myself.\n\nHere is the honest version.\n\nA scan takes about 40 seconds of GPU time and about 10 seconds of my time. The thing it replaces is standing in a cold kitchen with the fridge door open, trying to remember what is already in there.\n\nThe part I like most is not the scan. It is the receipt step. I buy milk, I take a photo of the receipt, and the list fixes itself. Before this I would come home, put the bags away, and still have no idea what was in the fridge before the shop.\n\nI did not run this for a full week of shopping, so I do not have a perfect number. What I can say is that on the four real fridge photos and the receipt I tested it with, it found the condiments, dairy, eggs and fruit correctly every time, and writing the whole shopping list took about 10 seconds.", "url": "https://wpnews.pro/news/sloth-fridge-a-fridge-photo-turns-into-a-shopping-list", "canonical_source": "https://dev.to/sonu7089/sloth-fridge-a-fridge-photo-turns-into-a-shopping-list-1fj4", "published_at": "2026-10-20 05:10:35+00:00", "updated_at": "2026-10-20 05:17:26.980448+00:00", "lang": "en", "topics": ["ai-tools", "computer-vision", "large-language-models", "ai-products", "developer-tools"], "entities": ["Sloth Fridge", "Ollama", "Qwen2.5-VL", "GitHub", "Streamlit", "Hacktoberfest"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/sloth-fridge-a-fridge-photo-turns-into-a-shopping-list", "markdown": "https://wpnews.pro/news/sloth-fridge-a-fridge-photo-turns-into-a-shopping-list.md", "text": "https://wpnews.pro/news/sloth-fridge-a-fridge-photo-turns-into-a-shopping-list.txt", "jsonld": "https://wpnews.pro/news/sloth-fridge-a-fridge-photo-turns-into-a-shopping-list.jsonld"}}