Turn one Minecraft-style reference image into a native Blockbench model using agent vision, optional mesh guidance, and deterministic compilation.
img2blockbench is Orca's open-source image-to-Blockbench benchmark and compiler. Use it directly, or explore Orca's broader Minecraft creation tools through the Orca CLI, Orca MCP, and orcaclient.com.
Route 1: image → provider-selected 3D mesh → cuboid reconstruction → .bbmodel
Route 2: image → cuboid + texture reasoning → density-locked .bbmodel
Route 3: image → img2threejs scene → semantic texture pass → .bbmodel
The demo app shows all three interactive
.bbmodel
outputs together. Choose an animal once, then compare mesh-guided, Direct, and img2threejs routes side by side. All 15 models are prefetched; drag to rotate, scroll to zoom, and double-click to reset.
cd demo
npm install
npm run dev
Every cell below is a deterministic render of the linked .bbmodel
. Route 1 also retains its benchmark source mesh, anatomy specification, and eight-view QA. Route 3 retains the official generated TypeScript factory and executable Three.js scene.
| Animal | Minecraft-style reference | Route 1 · Mesh-guided | Route 2 · Direct | Route 3 · img2threejs |
|---|---|---|---|---|
| Platypus | ||||
| Chimpanzee | ||||
| Elephant | ||||
| Tiger | ||||
| Coyote | ||||
Every output includes its embedded texture and a structural audit. Route 1 also preserves the source GLB and records its source-palette facial repair. Route 3 preserves the procedural scene, clustered albedo audit, semantic texture landmarks, and provenance.
The reusable image prompts are recorded in examples/lane1-five-animals-prompts.md.
Routes 2 and 3 require no neural image-to-3D mesh. Route 1 uses the mesh as measured shape and texture evidence, then rebuilds it as native Minecraft cuboids. The compiler handles file structure, UV packing, texture transfer, auditing, Bedrock geometry, and reproducible manifests.
Route 1 does not require or call a particular image-to-3D service. The user chooses the generator and supplies its exported mesh. The included benchmark uses Trellis, but Trellis is an example provider rather than a dependency or default.
The provider boundary is intentionally small:
- accept a textured
.glb
or.gltf
containing one or more triangle meshes; - prefer UVs and a base-color texture, but retain geometry when textures are unavailable;
- normalize the source with the anatomy spec's
canonical_transform
; - record the provider, model/version, source hash, and applicable license in provenance; and
- reconstruct and validate native cuboids independently of the source generator.
This lets contributors use hosted, local, commercial, or open-source 3D generators without changing the Blockbench compiler.
Route 3 runs the official img2threejs/img2threejs generator:
Its explicit implementation dependencies are:
, pinned to commitimg2threejs
f1ade81d45252ede20323d74a5b269c819f75245
, for procedural TypeScript scene generation;Three.js
0.185.1
for executing, serializing, and rendering the generatedObject3D
; and- this repository's
img2blockbench
adapter for converting compatibleBoxGeometry
and native material maps into a textured.bbmodel
.
Minecraft-style image
→ strict-quality ObjectSculptSpec
→ official img2threejs TypeScript factory
→ browser-executed THREE.Group
→ visible Object3D scene
→ box geometry adapter
→ native material map and UV-transform transfer
→ texture-only semantic landmarks
→ nearest-neighbor Blockbench atlas
→ .bbmodel
The five-animal benchmark pins upstream commit
f1ade81d45252ede20323d74a5b269c819f75245
and preserves the
spec, generated source, scene JSON, provenance, and converted model for every
animal. Eyes, nostrils, inner ears, and markings are declared texture-only
before factory generation, rather than surviving as decorative boxes. Each
remaining source component uses boxes, so dimensions and rotations transfer
directly.
The five factories are generated at optimization-pass
after the ordered
blockout, structural, form, material, surface, lighting, interaction, and
optimization reviews. The adapter preserves each generated
MeshPhysicalMaterial.map
plus repeat, wrap, offset, rotation, and flip state without recoloring or palette reduction. The adapter samples those native maps into one nearest-neighbor atlas, then paints each eye on exactly one face pair. This avoids texture crushing, projection smearing, and duplicate facial marks.
Three.js roughness, normal, and AO maps remain preview-only because Blockbench's Minecraft texture format has no equivalent PBR channels.
These scores compare the direct Route 2 and final Route 3 .bbmodel
geometry after uniform normalization. They do not measure texture similarity.
| Animal | Boxes R2/R3 | Shape IoU | Topology F1 | Box count | Dimensions | Rotations | Weighted |
|---|---|---|---|---|---|---|---|
| Platypus | 16 / 28 | 67.0% | 96.0% | 57.1% | 85.6% | 99.9% | 78.3% |
| Chimpanzee | 22 / 22 | 83.5% | 92.1% | 100.0% | 96.2% | 99.7% | 91.9% |
| Elephant | 21 / 21 | 90.5% | 100.0% | 100.0% | 97.5% | 99.7% | 96.2% |
| Tiger | 20 / 20 | 80.5% | 95.8% | 100.0% | 93.5% | 99.2% | 90.9% |
| Coyote | 20 / 20 | 82.9% | 100.0% | 100.0% | 94.1% | 99.7% | 92.8% |
Run cd demo && npm run benchmark:geometry
to reproduce the comparison. Rotation scores are high because both baselines are predominantly axis-aligned.
These measurements cover all five animals. A UV-density outlier is a face whose texel density is outside the accepted range; a detail cuboid is geometry misused for a flat eye, nostril, marking, or stripe.
| Metric | Route 1 · Mesh | Route 2 · Direct | Route 3 · img2threejs |
|---|---|---|---|
| Average cuboids | 23.2 | 19.8 | 22.2 |
| UV-density outlier faces | 0 | 0 | 0 |
| Flat detail cuboids | 0 | 0 | 0 |
| Texture landmarks | 16 | 25 | 42 |
| Explicit front axis | Yes | Yes | Yes |
Run python3 tools/benchmark-quality.py
to reproduce examples/quality-benchmark.json.
General image-to-3D tools produce triangle meshes. Minecraft mobs need something different:
- a small set of meaningful cuboids;
- connected, animatable bones and pivots;
- one consistent pixel-art texture atlas;
- collision metadata and Bedrock-compatible geometry;
- visible checks from both sides, not only a plausible front render.
Direct Route 2 intentionally starts with images that already look Minecraft-native: clear cuboid anatomy, crisp square-pixel materials, a full-body neutral pose, and separated appendages.
A photograph or smooth character illustration is not a valid direct-route demo input. Restyle it into a Minecraft concept first, or use Route 1 when organic depth is the important signal.
git clone https://github.com/orca-gamedev/img2blockbench.git
cd img2blockbench
python -m venv .venv
source .venv/bin/activate
pip install -e .
Install the bundled skill for your agent:
cp -R skill/img2blockbench ~/.codex/skills/
cp -R skill/img2blockbench ~/.claude/skills/
Then attach a reference image and ask:
Use $img2blockbench to rebuild this creature as a Minecraft Blockbench model.
The agent supplies the visual reasoning. This repository supplies its workflow, model contract, deterministic compiler, and quality gates. It does not bundle or require a particular LLM API.
img2blockbench probe ./reference.png
img2blockbench new ./reference.png --id red-panda --output ./red-panda.json
img2blockbench validate ./red-panda.json --strict
img2blockbench build ./red-panda.json --output ./dist
img2blockbench preview-threejs \
./red-panda.json \
--output ./createRedPandaModel.ts
img2blockbench from-threejs \
./red-panda.three.json \
--reference ./reference.png \
--id red-panda-threejs \
--description "A Minecraft-style red panda" \
--output ./red-panda-threejs.json
pip install -e '.[reference-projection]'
python tools/img2threejs/bake-reference-faces.py \
./red-panda-threejs.json \
--reference ./reference.png \
--output-spec ./red-panda-imported.json \
--audit ./red-panda-texture-transfer-audit.json
python tools/img2threejs/semanticize-model-spec.py \
./red-panda-imported.json \
--animal red-panda \
--recipes ./tools/img2threejs/semantic-recipes.json \
--reference ./reference.png \
--output ./red-panda-textured.json
img2blockbench audit ./dist/red-panda.bbmodel
The build directory contains:
red-panda.bbmodel
red-panda.png
red-panda.geo.json
red-panda.model-spec.json
red-panda.reference.png
red-panda.audit.json
red-panda.manifest.json
red-panda.zip
See the original fox example in examples/fox.
All three routes converge on the same validated Minecraft model specification and delivery bundle, but their upstream reasoning is separate.
| Route 1 | Route 2 | Route 3 | |
|---|---|---|---|
| Route | Image → selected 3D generator → cuboid spec | Image → cuboid spec | Image → img2threejs → cuboid spec |
| Intermediate | Textured GLB or GLTF | Native cuboid JSON | Generated TypeScript + Object3D JSON |
| External 3D GPU | Provider-dependent | None | None |
| Best fit | Ambiguous organic depth | Minecraft-native references | Cheaper procedural 3D reconstruction |
| Platypus cuboids | 22 | 16 | 28 |
| Shared output | .bbmodel , texture, geo.json , audit, bundle |
||
| Same | Same |
Route 2 may optionally render its cuboid spec through Three.js for review. That
preview does not make it Route 3. Route 3 begins with the official img2threejs
spec and generated Three.js scene, then ends as a native .bbmodel
.
The repository retains its original lane1
and lane2
artifact paths so existing links remain valid. The demo and documentation define their public route order independently.
The current platypus benchmark records artifact sizes, hashes, cuboid counts, bone counts, and external GPU requirements. Generation latency, provider price, and LLM token usage were not captured for the existing runs, so the repository does not fabricate those cost numbers.
The benchmark meshes below were generated with Trellis. The provider-neutral audit rasterizes any supported source mesh and the reconstructed cuboids as front, side, top, and isometric silhouettes, then records intersection-over-union (IoU), source coverage, and model precision.
| Model | Mean IoU | Source coverage | Evidence |
|---|---|---|---|
| Platypus | 0.608 | 0.916 | |
audit·sheetaudit·sheetaudit·sheetaudit·sheetWhite pixels are overlap, cyan is source-only, and orange is cuboid-only. These results prove the source meshes are genuinely used, but also show that the current reconstruction is approximate rather than an optimized silhouette fit.
pip install -e '.[overlap-audit]'
img2blockbench-overlap \
--source ./source.glb \
--spec ./anatomy-spec.json \
--bbmodel ./model.bbmodel \
--json ./overlap-audit.json \
--sheet ./overlap-sheet.png
A single image cannot reveal every hidden surface or guarantee exact depth. The workflow records uncertainty, mirrors bilateral anatomy when appropriate, and requires multi-view Blockbench review. It should request another view instead of pretending ambiguous anatomy is known.
pip install -e .
python -m unittest discover -s tests -v
python skill/img2blockbench/scripts/img2blockbench.py validate \
examples/fox/model-spec.json --strict
MIT licensed.