Show HN: Turn an image into a 3D blockbench model

Orca released img2blockbench, an open-source tool that converts a Minecraft-style reference image into a native Blockbench .bbmodel file using agent vision, optional mesh guidance, and deterministic compilation. The tool offers three routes: mesh-guided cuboid reconstruction, direct cuboid and texture reasoning, and img2threejs scene conversion, all producing textured models with structural audits. Orca's broader Minecraft creation tools include the Orca CLI, Orca MCP, and orcaclient.com.

Turn one Minecraft-style reference image into a native Blockbench model using agent vision, optional mesh guidance, and deterministic compilation. img2blockbench is Orca's open-source image-to-Blockbench benchmark and compiler. Use it directly, or explore Orca's broader Minecraft creation tools through the Orca CLI https://orcaclient.com/minecraft-cli , Orca MCP https://orcaclient.com/minecraft-mcp , and orcaclient.com https://orcaclient.com . Route 1: image → provider-selected 3D mesh → cuboid reconstruction → .bbmodel Route 2: image → cuboid + texture reasoning → density-locked .bbmodel Route 3: image → img2threejs scene → semantic texture pass → .bbmodel The demo /orca-gamedev/img2blockbench/blob/main/demo app shows all three interactive .bbmodel outputs together. Choose an animal once, then compare mesh-guided, Direct, and img2threejs routes side by side. All 15 models are prefetched; drag to rotate, scroll to zoom, and double-click to reset. cd demo npm install npm run dev Every cell below is a deterministic render of the linked .bbmodel . Route 1 also retains its benchmark source mesh, anatomy specification, and eight-view QA. Route 3 retains the official generated TypeScript factory and executable Three.js scene. | Animal | Minecraft-style reference | Route 1 · Mesh-guided | Route 2 · Direct | Route 3 · img2threejs | |---|---|---|---|---| Platypus | | | | | Chimpanzee | | | | | Elephant | | | | | Tiger | | | | | Coyote | | | | Every output includes its embedded texture and a structural audit. Route 1 also preserves the source GLB and records its source-palette facial repair. Route 3 preserves the procedural scene, clustered albedo audit, semantic texture landmarks, and provenance. The reusable image prompts are recorded in examples/lane1-five-animals-prompts.md /orca-gamedev/img2blockbench/blob/main/examples/lane1-five-animals-prompts.md . Routes 2 and 3 require no neural image-to-3D mesh. Route 1 uses the mesh as measured shape and texture evidence, then rebuilds it as native Minecraft cuboids. The compiler handles file structure, UV packing, texture transfer, auditing, Bedrock geometry, and reproducible manifests. Route 1 does not require or call a particular image-to-3D service. The user chooses the generator and supplies its exported mesh. The included benchmark uses Trellis, but Trellis is an example provider rather than a dependency or default. The provider boundary is intentionally small: - accept a textured .glb or .gltf containing one or more triangle meshes; - prefer UVs and a base-color texture, but retain geometry when textures are unavailable; - normalize the source with the anatomy spec's canonical transform ; - record the provider, model/version, source hash, and applicable license in provenance; and - reconstruct and validate native cuboids independently of the source generator. This lets contributors use hosted, local, commercial, or open-source 3D generators without changing the Blockbench compiler. Route 3 runs the official img2threejs/img2threejs https://github.com/img2threejs/img2threejs generator: Its explicit implementation dependencies are: , pinned to commit img2threejs f1ade81d45252ede20323d74a5b269c819f75245 , for procedural TypeScript scene generation; Three.js 0.185.1 for executing, serializing, and rendering the generated Object3D ; and- this repository's img2blockbench adapter for converting compatible BoxGeometry and native material maps into a textured .bbmodel . Minecraft-style image → strict-quality ObjectSculptSpec → official img2threejs TypeScript factory → browser-executed THREE.Group → visible Object3D scene → box geometry adapter → native material map and UV-transform transfer → texture-only semantic landmarks → nearest-neighbor Blockbench atlas → .bbmodel The five-animal benchmark pins upstream commit f1ade81d45252ede20323d74a5b269c819f75245 and preserves the spec, generated source, scene JSON, provenance, and converted model for every animal. Eyes, nostrils, inner ears, and markings are declared texture-only before factory generation, rather than surviving as decorative boxes. Each remaining source component uses boxes, so dimensions and rotations transfer directly. The five factories are generated at optimization-pass after the ordered blockout, structural, form, material, surface, lighting, interaction, and optimization reviews. The adapter preserves each generated MeshPhysicalMaterial.map plus repeat, wrap, offset, rotation, and flip state without recoloring or palette reduction. The adapter samples those native maps into one nearest-neighbor atlas, then paints each eye on exactly one face pair. This avoids texture crushing, projection smearing, and duplicate facial marks. Three.js roughness, normal, and AO maps remain preview-only because Blockbench's Minecraft texture format has no equivalent PBR channels. These scores compare the direct Route 2 and final Route 3 .bbmodel geometry after uniform normalization. They do not measure texture similarity. | Animal | Boxes R2/R3 | Shape IoU | Topology F1 | Box count | Dimensions | Rotations | Weighted | |---|---|---|---|---|---|---|---| | Platypus | 16 / 28 | 67.0% | 96.0% | 57.1% | 85.6% | 99.9% | 78.3% | | Chimpanzee | 22 / 22 | 83.5% | 92.1% | 100.0% | 96.2% | 99.7% | 91.9% | | Elephant | 21 / 21 | 90.5% | 100.0% | 100.0% | 97.5% | 99.7% | 96.2% | | Tiger | 20 / 20 | 80.5% | 95.8% | 100.0% | 93.5% | 99.2% | 90.9% | | Coyote | 20 / 20 | 82.9% | 100.0% | 100.0% | 94.1% | 99.7% | 92.8% | Run cd demo && npm run benchmark:geometry to reproduce the comparison. Rotation scores are high because both baselines are predominantly axis-aligned. These measurements cover all five animals. A UV-density outlier is a face whose texel density is outside the accepted range; a detail cuboid is geometry misused for a flat eye, nostril, marking, or stripe. | Metric | Route 1 · Mesh | Route 2 · Direct | Route 3 · img2threejs | |---|---|---|---| | Average cuboids | 23.2 | 19.8 | 22.2 | | UV-density outlier faces | 0 | 0 | 0 | | Flat detail cuboids | 0 | 0 | 0 | | Texture landmarks | 16 | 25 | 42 | | Explicit front axis | Yes | Yes | Yes | Run python3 tools/benchmark-quality.py to reproduce examples/quality-benchmark.json /orca-gamedev/img2blockbench/blob/main/examples/quality-benchmark.json . General image-to-3D tools produce triangle meshes. Minecraft mobs need something different: - a small set of meaningful cuboids; - connected, animatable bones and pivots; - one consistent pixel-art texture atlas; - collision metadata and Bedrock-compatible geometry; - visible checks from both sides, not only a plausible front render. Direct Route 2 intentionally starts with images that already look Minecraft-native: clear cuboid anatomy, crisp square-pixel materials, a full-body neutral pose, and separated appendages. A photograph or smooth character illustration is not a valid direct-route demo input. Restyle it into a Minecraft concept first, or use Route 1 when organic depth is the important signal. git clone https://github.com/orca-gamedev/img2blockbench.git cd img2blockbench python -m venv .venv source .venv/bin/activate pip install -e . Install the bundled skill for your agent: Codex cp -R skill/img2blockbench ~/.codex/skills/ Claude Code cp -R skill/img2blockbench ~/.claude/skills/ Then attach a reference image and ask: Use $img2blockbench to rebuild this creature as a Minecraft Blockbench model. The agent supplies the visual reasoning. This repository supplies its workflow, model contract, deterministic compiler, and quality gates. It does not bundle or require a particular LLM API. Inspect the source image. img2blockbench probe ./reference.png Create a starter spec for the agent to complete. img2blockbench new ./reference.png --id red-panda --output ./red-panda.json Block shallow or malformed specifications. img2blockbench validate ./red-panda.json --strict Compile the accepted spec. img2blockbench build ./red-panda.json --output ./dist Optional Route 2 preview generated from the cuboid spec. img2blockbench preview-threejs \ ./red-panda.json \ --output ./createRedPandaModel.ts Route 3: import standard Three.js Object3D.toJSON output. img2blockbench from-threejs \ ./red-panda.three.json \ --reference ./reference.png \ --id red-panda-threejs \ --description "A Minecraft-style red panda" \ --output ./red-panda-threejs.json Preserve and audit the native img2threejs base-color maps. pip install -e '. reference-projection ' python tools/img2threejs/bake-reference-faces.py \ ./red-panda-threejs.json \ --reference ./reference.png \ --output-spec ./red-panda-imported.json \ --audit ./red-panda-texture-transfer-audit.json After defining a red-panda entry in semantic-recipes.json, remove texture-only detail geometry and paint its landmarks. python tools/img2threejs/semanticize-model-spec.py \ ./red-panda-imported.json \ --animal red-panda \ --recipes ./tools/img2threejs/semantic-recipes.json \ --reference ./reference.png \ --output ./red-panda-textured.json Re-audit an existing Blockbench file. img2blockbench audit ./dist/red-panda.bbmodel The build directory contains: red-panda.bbmodel red-panda.png red-panda.geo.json red-panda.model-spec.json red-panda.reference.png red-panda.audit.json red-panda.manifest.json red-panda.zip See the original fox example in examples/fox /orca-gamedev/img2blockbench/blob/main/examples/fox . All three routes converge on the same validated Minecraft model specification and delivery bundle, but their upstream reasoning is separate. | Route 1 | Route 2 | Route 3 | | |---|---|---|---| | Route | Image → selected 3D generator → cuboid spec | Image → cuboid spec | Image → img2threejs → cuboid spec | | Intermediate | Textured GLB or GLTF | Native cuboid JSON | Generated TypeScript + Object3D JSON | | External 3D GPU | Provider-dependent | None | None | | Best fit | Ambiguous organic depth | Minecraft-native references | Cheaper procedural 3D reconstruction | | Platypus cuboids | 22 | 16 | 28 | | Shared output | .bbmodel , texture, geo.json , audit, bundle | Same | Same | Route 2 may optionally render its cuboid spec through Three.js for review. That preview does not make it Route 3. Route 3 begins with the official img2threejs spec and generated Three.js scene, then ends as a native .bbmodel . The repository retains its original lane1 and lane2 artifact paths so existing links remain valid. The demo and documentation define their public route order independently. The current platypus benchmark /orca-gamedev/img2blockbench/blob/main/examples/platypus/benchmark.json records artifact sizes, hashes, cuboid counts, bone counts, and external GPU requirements. Generation latency, provider price, and LLM token usage were not captured for the existing runs, so the repository does not fabricate those cost numbers. The benchmark meshes below were generated with Trellis. The provider-neutral audit rasterizes any supported source mesh and the reconstructed cuboids as front, side, top, and isometric silhouettes, then records intersection-over-union IoU , source coverage, and model precision. | Model | Mean IoU | Source coverage | Evidence | |---|---|---|---| | Platypus | 0.608 | 0.916 | | audit /orca-gamedev/img2blockbench/blob/main/examples/chimpanzee/lane2/overlap-audit.json · sheet /orca-gamedev/img2blockbench/blob/main/examples/chimpanzee/lane2/overlap-sheet.png audit /orca-gamedev/img2blockbench/blob/main/examples/elephant/lane2/overlap-audit.json · sheet /orca-gamedev/img2blockbench/blob/main/examples/elephant/lane2/overlap-sheet.png audit /orca-gamedev/img2blockbench/blob/main/examples/tiger/lane2/overlap-audit.json · sheet /orca-gamedev/img2blockbench/blob/main/examples/tiger/lane2/overlap-sheet.png audit /orca-gamedev/img2blockbench/blob/main/examples/coyote/lane2/overlap-audit.json · sheet /orca-gamedev/img2blockbench/blob/main/examples/coyote/lane2/overlap-sheet.png White pixels are overlap, cyan is source-only, and orange is cuboid-only. These results prove the source meshes are genuinely used, but also show that the current reconstruction is approximate rather than an optimized silhouette fit. pip install -e '. overlap-audit ' img2blockbench-overlap \ --source ./source.glb \ --spec ./anatomy-spec.json \ --bbmodel ./model.bbmodel \ --json ./overlap-audit.json \ --sheet ./overlap-sheet.png A single image cannot reveal every hidden surface or guarantee exact depth. The workflow records uncertainty, mirrors bilateral anatomy when appropriate, and requires multi-view Blockbench review. It should request another view instead of pretending ambiguous anatomy is known. pip install -e . python -m unittest discover -s tests -v python skill/img2blockbench/scripts/img2blockbench.py validate \ examples/fox/model-spec.json --strict MIT licensed.