{"slug": "show-hn-turn-an-image-into-a-3d-blockbench-model", "title": "Show HN: Turn an image into a 3D blockbench model", "summary": "Orca released img2blockbench, an open-source tool that converts a Minecraft-style reference image into a native Blockbench .bbmodel file using agent vision, optional mesh guidance, and deterministic compilation. The tool offers three routes: mesh-guided cuboid reconstruction, direct cuboid and texture reasoning, and img2threejs scene conversion, all producing textured models with structural audits. Orca's broader Minecraft creation tools include the Orca CLI, Orca MCP, and orcaclient.com.", "body_md": "Turn one Minecraft-style reference image into a native Blockbench model using agent vision, optional mesh guidance, and deterministic compilation.\n\nimg2blockbench is Orca's open-source image-to-Blockbench benchmark and\ncompiler. Use it directly, or explore Orca's broader Minecraft creation tools\nthrough the [Orca CLI](https://orcaclient.com/minecraft-cli),\n[Orca MCP](https://orcaclient.com/minecraft-mcp), and\n[orcaclient.com](https://orcaclient.com).\n\n```\nRoute 1: image → provider-selected 3D mesh → cuboid reconstruction → .bbmodel\nRoute 2: image → cuboid + texture reasoning → density-locked .bbmodel\nRoute 3: image → img2threejs scene → semantic texture pass → .bbmodel\n```\n\nThe [ demo](/orca-gamedev/img2blockbench/blob/main/demo) app shows all three interactive\n\n`.bbmodel`\n\noutputs together.\nChoose an animal once, then compare mesh-guided, Direct, and img2threejs routes\nside by side. All 15 models are prefetched; drag to rotate, scroll to zoom,\nand double-click to reset.\n\n```\ncd demo\nnpm install\nnpm run dev\n```\n\nEvery cell below is a deterministic render of the linked `.bbmodel`\n\n. Route 1\nalso retains its benchmark source mesh, anatomy specification, and eight-view\nQA. Route 3 retains the official generated TypeScript factory and executable\nThree.js scene.\n\n| Animal | Minecraft-style reference | Route 1 · Mesh-guided | Route 2 · Direct | Route 3 · img2threejs |\n|---|---|---|---|---|\nPlatypus |\n|\n|\n|\n|\nChimpanzee |\n|\n|\n|\n|\nElephant |\n|\n|\n|\n|\nTiger |\n|\n|\n|\n|\nCoyote |\n|\n|\n|\n\nEvery output includes its embedded texture and a structural audit. Route 1 also preserves the source GLB and records its source-palette facial repair. Route 3 preserves the procedural scene, clustered albedo audit, semantic texture landmarks, and provenance.\n\nThe reusable image prompts are recorded in\n[ examples/lane1-five-animals-prompts.md](/orca-gamedev/img2blockbench/blob/main/examples/lane1-five-animals-prompts.md).\n\nRoutes 2 and 3 require no neural image-to-3D mesh. Route 1 uses the mesh as measured shape and texture evidence, then rebuilds it as native Minecraft cuboids. The compiler handles file structure, UV packing, texture transfer, auditing, Bedrock geometry, and reproducible manifests.\n\nRoute 1 does not require or call a particular image-to-3D service. The user chooses the generator and supplies its exported mesh. The included benchmark uses Trellis, but Trellis is an example provider rather than a dependency or default.\n\nThe provider boundary is intentionally small:\n\n- accept a textured\n`.glb`\n\nor`.gltf`\n\ncontaining one or more triangle meshes; - prefer UVs and a base-color texture, but retain geometry when textures are unavailable;\n- normalize the source with the anatomy spec's\n`canonical_transform`\n\n; - record the provider, model/version, source hash, and applicable license in provenance; and\n- reconstruct and validate native cuboids independently of the source generator.\n\nThis lets contributors use hosted, local, commercial, or open-source 3D generators without changing the Blockbench compiler.\n\nRoute 3 runs the official\n[ img2threejs/img2threejs](https://github.com/img2threejs/img2threejs) generator:\n\nIts explicit implementation dependencies are:\n\n, pinned to commit`img2threejs`\n\n`f1ade81d45252ede20323d74a5b269c819f75245`\n\n, for procedural TypeScript scene generation;`Three.js`\n\n`0.185.1`\n\nfor executing, serializing, and rendering the generated`Object3D`\n\n; and- this repository's\n`img2blockbench`\n\nadapter for converting compatible`BoxGeometry`\n\nand native material maps into a textured`.bbmodel`\n\n.\n\n```\nMinecraft-style image\n  → strict-quality ObjectSculptSpec\n  → official img2threejs TypeScript factory\n  → browser-executed THREE.Group\n  → visible Object3D scene\n  → box geometry adapter\n  → native material map and UV-transform transfer\n  → texture-only semantic landmarks\n  → nearest-neighbor Blockbench atlas\n  → .bbmodel\n```\n\nThe five-animal benchmark pins upstream commit\n`f1ade81d45252ede20323d74a5b269c819f75245`\n\nand preserves the\nspec, generated source, scene JSON, provenance, and converted model for every\nanimal. Eyes, nostrils, inner ears, and markings are declared texture-only\nbefore factory generation, rather than surviving as decorative boxes. Each\nremaining source component uses boxes, so dimensions and rotations transfer\ndirectly.\nThe five factories are generated at `optimization-pass`\n\nafter the ordered\nblockout, structural, form, material, surface, lighting, interaction, and\noptimization reviews. The adapter preserves each generated\n`MeshPhysicalMaterial.map`\n\nplus repeat, wrap, offset, rotation, and flip state\nwithout recoloring or palette reduction. The adapter samples those native maps\ninto one nearest-neighbor atlas, then paints each eye on exactly one face pair.\nThis avoids texture crushing, projection smearing, and duplicate facial marks.\n\nThree.js roughness, normal, and AO maps remain preview-only because Blockbench's Minecraft texture format has no equivalent PBR channels.\n\nThese scores compare the direct Route 2 and final Route 3 `.bbmodel`\n\ngeometry\nafter uniform normalization. They do not measure texture similarity.\n\n| Animal | Boxes R2/R3 | Shape IoU | Topology F1 | Box count | Dimensions | Rotations | Weighted |\n|---|---|---|---|---|---|---|---|\n| Platypus | 16 / 28 | 67.0% | 96.0% | 57.1% | 85.6% | 99.9% | 78.3% |\n| Chimpanzee | 22 / 22 | 83.5% | 92.1% | 100.0% | 96.2% | 99.7% | 91.9% |\n| Elephant | 21 / 21 | 90.5% | 100.0% | 100.0% | 97.5% | 99.7% | 96.2% |\n| Tiger | 20 / 20 | 80.5% | 95.8% | 100.0% | 93.5% | 99.2% | 90.9% |\n| Coyote | 20 / 20 | 82.9% | 100.0% | 100.0% | 94.1% | 99.7% | 92.8% |\n\nRun `cd demo && npm run benchmark:geometry`\n\nto reproduce the comparison.\nRotation scores are high because both baselines are predominantly axis-aligned.\n\nThese measurements cover all five animals. A UV-density outlier is a face whose texel density is outside the accepted range; a detail cuboid is geometry misused for a flat eye, nostril, marking, or stripe.\n\n| Metric | Route 1 · Mesh | Route 2 · Direct | Route 3 · img2threejs |\n|---|---|---|---|\n| Average cuboids | 23.2 | 19.8 | 22.2 |\n| UV-density outlier faces | 0 | 0 | 0 |\n| Flat detail cuboids | 0 | 0 | 0 |\n| Texture landmarks | 16 | 25 | 42 |\n| Explicit front axis | Yes | Yes | Yes |\n\nRun `python3 tools/benchmark-quality.py`\n\nto reproduce\n[ examples/quality-benchmark.json](/orca-gamedev/img2blockbench/blob/main/examples/quality-benchmark.json).\n\nGeneral image-to-3D tools produce triangle meshes. Minecraft mobs need something different:\n\n- a small set of meaningful cuboids;\n- connected, animatable bones and pivots;\n- one consistent pixel-art texture atlas;\n- collision metadata and Bedrock-compatible geometry;\n- visible checks from both sides, not only a plausible front render.\n\nDirect Route 2 intentionally starts with images that already look Minecraft-native: clear cuboid anatomy, crisp square-pixel materials, a full-body neutral pose, and separated appendages.\n\nA photograph or smooth character illustration is not a valid direct-route demo input. Restyle it into a Minecraft concept first, or use Route 1 when organic depth is the important signal.\n\n```\ngit clone https://github.com/orca-gamedev/img2blockbench.git\ncd img2blockbench\npython -m venv .venv\nsource .venv/bin/activate\npip install -e .\n```\n\nInstall the bundled skill for your agent:\n\n```\n# Codex\ncp -R skill/img2blockbench ~/.codex/skills/\n\n# Claude Code\ncp -R skill/img2blockbench ~/.claude/skills/\n```\n\nThen attach a reference image and ask:\n\n```\nUse $img2blockbench to rebuild this creature as a Minecraft Blockbench model.\n```\n\nThe agent supplies the visual reasoning. This repository supplies its workflow, model contract, deterministic compiler, and quality gates. It does not bundle or require a particular LLM API.\n\n```\n# Inspect the source image.\nimg2blockbench probe ./reference.png\n\n# Create a starter spec for the agent to complete.\nimg2blockbench new ./reference.png --id red-panda --output ./red-panda.json\n\n# Block shallow or malformed specifications.\nimg2blockbench validate ./red-panda.json --strict\n\n# Compile the accepted spec.\nimg2blockbench build ./red-panda.json --output ./dist\n\n# Optional Route 2 preview generated from the cuboid spec.\nimg2blockbench preview-threejs \\\n  ./red-panda.json \\\n  --output ./createRedPandaModel.ts\n\n# Route 3: import standard Three.js Object3D.toJSON output.\nimg2blockbench from-threejs \\\n  ./red-panda.three.json \\\n  --reference ./reference.png \\\n  --id red-panda-threejs \\\n  --description \"A Minecraft-style red panda\" \\\n  --output ./red-panda-threejs.json\n\n# Preserve and audit the native img2threejs base-color maps.\npip install -e '.[reference-projection]'\npython tools/img2threejs/bake-reference-faces.py \\\n  ./red-panda-threejs.json \\\n  --reference ./reference.png \\\n  --output-spec ./red-panda-imported.json \\\n  --audit ./red-panda-texture-transfer-audit.json\n\n# After defining a red-panda entry in semantic-recipes.json, remove\n# texture-only detail geometry and paint its landmarks.\npython tools/img2threejs/semanticize-model-spec.py \\\n  ./red-panda-imported.json \\\n  --animal red-panda \\\n  --recipes ./tools/img2threejs/semantic-recipes.json \\\n  --reference ./reference.png \\\n  --output ./red-panda-textured.json\n\n# Re-audit an existing Blockbench file.\nimg2blockbench audit ./dist/red-panda.bbmodel\n```\n\nThe build directory contains:\n\n```\nred-panda.bbmodel\nred-panda.png\nred-panda.geo.json\nred-panda.model-spec.json\nred-panda.reference.png\nred-panda.audit.json\nred-panda.manifest.json\nred-panda.zip\n```\n\nSee the original fox example in [ examples/fox](/orca-gamedev/img2blockbench/blob/main/examples/fox).\n\nAll three routes converge on the same validated Minecraft model specification and delivery bundle, but their upstream reasoning is separate.\n\n| Route 1 | Route 2 | Route 3 | |\n|---|---|---|---|\n| Route | Image → selected 3D generator → cuboid spec | Image → cuboid spec | Image → img2threejs → cuboid spec |\n| Intermediate | Textured GLB or GLTF | Native cuboid JSON | Generated TypeScript + Object3D JSON |\n| External 3D GPU | Provider-dependent | None | None |\n| Best fit | Ambiguous organic depth | Minecraft-native references | Cheaper procedural 3D reconstruction |\n| Platypus cuboids | 22 | 16 | 28 |\n| Shared output | `.bbmodel` , texture, `geo.json` , audit, bundle |\nSame | Same |\n\nRoute 2 may optionally render its cuboid spec through Three.js for review. That\npreview does not make it Route 3. Route 3 begins with the official img2threejs\nspec and generated Three.js scene, then ends as a native `.bbmodel`\n\n.\n\nThe repository retains its original `lane1`\n\nand `lane2`\n\nartifact paths so\nexisting links remain valid. The demo and documentation define their public\nroute order independently.\n\nThe current [platypus benchmark](/orca-gamedev/img2blockbench/blob/main/examples/platypus/benchmark.json) records\nartifact sizes, hashes, cuboid counts, bone counts, and external GPU\nrequirements. Generation latency, provider price, and LLM token usage were not\ncaptured for the existing runs, so the repository does not fabricate those\ncost numbers.\n\nThe benchmark meshes below were generated with Trellis. The provider-neutral audit rasterizes any supported source mesh and the reconstructed cuboids as front, side, top, and isometric silhouettes, then records intersection-over-union (IoU), source coverage, and model precision.\n\n| Model | Mean IoU | Source coverage | Evidence |\n|---|---|---|---|\n| Platypus | 0.608 | 0.916 |\n|\n\n[audit](/orca-gamedev/img2blockbench/blob/main/examples/chimpanzee/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/chimpanzee/lane2/overlap-sheet.png)[audit](/orca-gamedev/img2blockbench/blob/main/examples/elephant/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/elephant/lane2/overlap-sheet.png)[audit](/orca-gamedev/img2blockbench/blob/main/examples/tiger/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/tiger/lane2/overlap-sheet.png)[audit](/orca-gamedev/img2blockbench/blob/main/examples/coyote/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/coyote/lane2/overlap-sheet.png)White pixels are overlap, cyan is source-only, and orange is cuboid-only. These results prove the source meshes are genuinely used, but also show that the current reconstruction is approximate rather than an optimized silhouette fit.\n\n```\npip install -e '.[overlap-audit]'\nimg2blockbench-overlap \\\n  --source ./source.glb \\\n  --spec ./anatomy-spec.json \\\n  --bbmodel ./model.bbmodel \\\n  --json ./overlap-audit.json \\\n  --sheet ./overlap-sheet.png\n```\n\nA single image cannot reveal every hidden surface or guarantee exact depth. The workflow records uncertainty, mirrors bilateral anatomy when appropriate, and requires multi-view Blockbench review. It should request another view instead of pretending ambiguous anatomy is known.\n\n```\npip install -e .\npython -m unittest discover -s tests -v\npython skill/img2blockbench/scripts/img2blockbench.py validate \\\n  examples/fox/model-spec.json --strict\n```\n\nMIT licensed.", "url": "https://wpnews.pro/news/show-hn-turn-an-image-into-a-3d-blockbench-model", "canonical_source": "https://github.com/orca-gamedev/img2blockbench", "published_at": "2026-07-27 00:18:53+00:00", "updated_at": "2026-07-27 00:52:24.386630+00:00", "lang": "en", "topics": ["developer-tools", "computer-vision", "generative-ai"], "entities": ["Orca", "img2blockbench", "Blockbench", "Orca CLI", "Orca MCP", "Trellis", "img2threejs", "Three.js"], "alternates": {"html": "https://wpnews.pro/news/show-hn-turn-an-image-into-a-3d-blockbench-model", "markdown": "https://wpnews.pro/news/show-hn-turn-an-image-into-a-3d-blockbench-model.md", "text": "https://wpnews.pro/news/show-hn-turn-an-image-into-a-3d-blockbench-model.txt", "jsonld": "https://wpnews.pro/news/show-hn-turn-an-image-into-a-3d-blockbench-model.jsonld"}}