{"slug": "vybbench-is-now-on-github", "title": "VybBench Is Now on GitHub", "summary": "Coco Labs, the makers of Vyb, released VybBench 1.0 on GitHub, a design-fidelity benchmark built from 145 fonts, 100 colors and 200 edits, with each model performing each task three times and every generation scored individually. The benchmark compares ChatGPT Image (gpt-image-2.5-sunburst), Meta Muse and Recraft v4.1 alongside Coco Labs' own Vyb 1.0, with prompts and run results published in a HuggingFace dataset and scoring code installable via pip. Coco Labs disclosed it built the benchmark, and developers can add a model by implementing generate and edit methods and running the run, judge and score commands.", "body_md": "**Almost isn't good enough.** VybBench measures how well models maintain\ndesign fidelity by comparing how effectively they generate specific\nfonts and colors, and perform non-destructive edits.\n\nOur dataset consists of 145 fonts, 100 colors, and 200 edits. We have each model perform each task three times and score each generation individually.\n\nWe compare top image generation models:\nChatGPT Image (gpt-image-2.5-sunburst), Meta Muse, and Recraft v4.1. We also report\nthe results for [Vyb 1.0](https://cocolabs.ai/vyb).\n\nFull Disclosure: VybBench was built by Coco Labs, the makers of Vyb. The full\nmethodology and rationale are in\n[the launch post](https://cocolabs.ai/blog/introducing-vybbench).\n\nThis repo contains the code for scoring any model. The prompts and the results\nfrom our runs are available in the HuggingFace Dataset:\n([https://huggingface.co/datasets/cocolabs/vybbench](https://huggingface.co/datasets/cocolabs/vybbench)). They are downloaded automatically.\n\n```\npip install -e .\nvybbench score\n```\n\nWe have made it easy for you to test any model simply by doing the following:\n\n``` python\nfrom vybbench.model import Model, Start\n\nclass MyModel(Model):\n    name = \"mymodel\"\n\n    def generate(self, prompt: str, width: int, height: int) -> bytes: ...\n    def edit(self, instruction: str, start: Start) -> bytes: ...\nvybbench run --model mypkg.mymodel:MyModel\nvybbench judge\nvybbench score\n```\n\nOur judging and scoring is based on the returned PNGs. Starts contain both the starting png and the starting svg. Use whichever is better for your model.\n\nWe are happy to include your model. Please create an issue with the model code.\n\n```\n@misc{vybbench2026,\n  title = {VybBench 1.0: A Design Fidelity Benchmark},\n  author = {{Coco Labs}},\n  year = {2026},\n  url = {https://huggingface.co/datasets/cocolabs/vybbench}\n}\n```\n\n", "url": "https://wpnews.pro/news/vybbench-is-now-on-github", "canonical_source": "https://github.com/cocolabsai/vybbench", "published_at": "2026-11-05 19:43:48+00:00", "updated_at": "2026-11-05 19:50:42.127296+00:00", "lang": "en", "topics": ["ai-research", "generative-ai", "ai-products", "developer-tools"], "entities": ["Coco Labs", "VybBench", "Vyb 1.0", "GitHub", "HuggingFace", "ChatGPT Image", "gpt-image-2.5-sunburst", "Meta Muse"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/vybbench-is-now-on-github", "markdown": "https://wpnews.pro/news/vybbench-is-now-on-github.md", "text": "https://wpnews.pro/news/vybbench-is-now-on-github.txt", "jsonld": "https://wpnews.pro/news/vybbench-is-now-on-github.jsonld"}}