# Show HN: Turn an image into a 3D blockbench model

> Source: <https://github.com/orca-gamedev/img2blockbench>
> Published: 2026-07-27 00:18:53+00:00

Turn one Minecraft-style reference image into a native Blockbench model using agent vision, optional mesh guidance, and deterministic compilation.

img2blockbench is Orca's open-source image-to-Blockbench benchmark and
compiler. Use it directly, or explore Orca's broader Minecraft creation tools
through the [Orca CLI](https://orcaclient.com/minecraft-cli),
[Orca MCP](https://orcaclient.com/minecraft-mcp), and
[orcaclient.com](https://orcaclient.com).

```
Route 1: image → provider-selected 3D mesh → cuboid reconstruction → .bbmodel
Route 2: image → cuboid + texture reasoning → density-locked .bbmodel
Route 3: image → img2threejs scene → semantic texture pass → .bbmodel
```

The [ demo](/orca-gamedev/img2blockbench/blob/main/demo) app shows all three interactive

`.bbmodel`

outputs together.
Choose an animal once, then compare mesh-guided, Direct, and img2threejs routes
side by side. All 15 models are prefetched; drag to rotate, scroll to zoom,
and double-click to reset.

```
cd demo
npm install
npm run dev
```

Every cell below is a deterministic render of the linked `.bbmodel`

. Route 1
also retains its benchmark source mesh, anatomy specification, and eight-view
QA. Route 3 retains the official generated TypeScript factory and executable
Three.js scene.

| Animal | Minecraft-style reference | Route 1 · Mesh-guided | Route 2 · Direct | Route 3 · img2threejs |
|---|---|---|---|---|
Platypus |
|
|
|
|
Chimpanzee |
|
|
|
|
Elephant |
|
|
|
|
Tiger |
|
|
|
|
Coyote |
|
|
|

Every output includes its embedded texture and a structural audit. Route 1 also preserves the source GLB and records its source-palette facial repair. Route 3 preserves the procedural scene, clustered albedo audit, semantic texture landmarks, and provenance.

The reusable image prompts are recorded in
[ examples/lane1-five-animals-prompts.md](/orca-gamedev/img2blockbench/blob/main/examples/lane1-five-animals-prompts.md).

Routes 2 and 3 require no neural image-to-3D mesh. Route 1 uses the mesh as measured shape and texture evidence, then rebuilds it as native Minecraft cuboids. The compiler handles file structure, UV packing, texture transfer, auditing, Bedrock geometry, and reproducible manifests.

Route 1 does not require or call a particular image-to-3D service. The user chooses the generator and supplies its exported mesh. The included benchmark uses Trellis, but Trellis is an example provider rather than a dependency or default.

The provider boundary is intentionally small:

- accept a textured
`.glb`

or`.gltf`

containing one or more triangle meshes; - prefer UVs and a base-color texture, but retain geometry when textures are unavailable;
- normalize the source with the anatomy spec's
`canonical_transform`

; - record the provider, model/version, source hash, and applicable license in provenance; and
- reconstruct and validate native cuboids independently of the source generator.

This lets contributors use hosted, local, commercial, or open-source 3D generators without changing the Blockbench compiler.

Route 3 runs the official
[ img2threejs/img2threejs](https://github.com/img2threejs/img2threejs) generator:

Its explicit implementation dependencies are:

, pinned to commit`img2threejs`

`f1ade81d45252ede20323d74a5b269c819f75245`

, for procedural TypeScript scene generation;`Three.js`

`0.185.1`

for executing, serializing, and rendering the generated`Object3D`

; and- this repository's
`img2blockbench`

adapter for converting compatible`BoxGeometry`

and native material maps into a textured`.bbmodel`

.

```
Minecraft-style image
  → strict-quality ObjectSculptSpec
  → official img2threejs TypeScript factory
  → browser-executed THREE.Group
  → visible Object3D scene
  → box geometry adapter
  → native material map and UV-transform transfer
  → texture-only semantic landmarks
  → nearest-neighbor Blockbench atlas
  → .bbmodel
```

The five-animal benchmark pins upstream commit
`f1ade81d45252ede20323d74a5b269c819f75245`

and preserves the
spec, generated source, scene JSON, provenance, and converted model for every
animal. Eyes, nostrils, inner ears, and markings are declared texture-only
before factory generation, rather than surviving as decorative boxes. Each
remaining source component uses boxes, so dimensions and rotations transfer
directly.
The five factories are generated at `optimization-pass`

after the ordered
blockout, structural, form, material, surface, lighting, interaction, and
optimization reviews. The adapter preserves each generated
`MeshPhysicalMaterial.map`

plus repeat, wrap, offset, rotation, and flip state
without recoloring or palette reduction. The adapter samples those native maps
into one nearest-neighbor atlas, then paints each eye on exactly one face pair.
This avoids texture crushing, projection smearing, and duplicate facial marks.

Three.js roughness, normal, and AO maps remain preview-only because Blockbench's Minecraft texture format has no equivalent PBR channels.

These scores compare the direct Route 2 and final Route 3 `.bbmodel`

geometry
after uniform normalization. They do not measure texture similarity.

| Animal | Boxes R2/R3 | Shape IoU | Topology F1 | Box count | Dimensions | Rotations | Weighted |
|---|---|---|---|---|---|---|---|
| Platypus | 16 / 28 | 67.0% | 96.0% | 57.1% | 85.6% | 99.9% | 78.3% |
| Chimpanzee | 22 / 22 | 83.5% | 92.1% | 100.0% | 96.2% | 99.7% | 91.9% |
| Elephant | 21 / 21 | 90.5% | 100.0% | 100.0% | 97.5% | 99.7% | 96.2% |
| Tiger | 20 / 20 | 80.5% | 95.8% | 100.0% | 93.5% | 99.2% | 90.9% |
| Coyote | 20 / 20 | 82.9% | 100.0% | 100.0% | 94.1% | 99.7% | 92.8% |

Run `cd demo && npm run benchmark:geometry`

to reproduce the comparison.
Rotation scores are high because both baselines are predominantly axis-aligned.

These measurements cover all five animals. A UV-density outlier is a face whose texel density is outside the accepted range; a detail cuboid is geometry misused for a flat eye, nostril, marking, or stripe.

| Metric | Route 1 · Mesh | Route 2 · Direct | Route 3 · img2threejs |
|---|---|---|---|
| Average cuboids | 23.2 | 19.8 | 22.2 |
| UV-density outlier faces | 0 | 0 | 0 |
| Flat detail cuboids | 0 | 0 | 0 |
| Texture landmarks | 16 | 25 | 42 |
| Explicit front axis | Yes | Yes | Yes |

Run `python3 tools/benchmark-quality.py`

to reproduce
[ examples/quality-benchmark.json](/orca-gamedev/img2blockbench/blob/main/examples/quality-benchmark.json).

General image-to-3D tools produce triangle meshes. Minecraft mobs need something different:

- a small set of meaningful cuboids;
- connected, animatable bones and pivots;
- one consistent pixel-art texture atlas;
- collision metadata and Bedrock-compatible geometry;
- visible checks from both sides, not only a plausible front render.

Direct Route 2 intentionally starts with images that already look Minecraft-native: clear cuboid anatomy, crisp square-pixel materials, a full-body neutral pose, and separated appendages.

A photograph or smooth character illustration is not a valid direct-route demo input. Restyle it into a Minecraft concept first, or use Route 1 when organic depth is the important signal.

```
git clone https://github.com/orca-gamedev/img2blockbench.git
cd img2blockbench
python -m venv .venv
source .venv/bin/activate
pip install -e .
```

Install the bundled skill for your agent:

```
# Codex
cp -R skill/img2blockbench ~/.codex/skills/

# Claude Code
cp -R skill/img2blockbench ~/.claude/skills/
```

Then attach a reference image and ask:

```
Use $img2blockbench to rebuild this creature as a Minecraft Blockbench model.
```

The agent supplies the visual reasoning. This repository supplies its workflow, model contract, deterministic compiler, and quality gates. It does not bundle or require a particular LLM API.

```
# Inspect the source image.
img2blockbench probe ./reference.png

# Create a starter spec for the agent to complete.
img2blockbench new ./reference.png --id red-panda --output ./red-panda.json

# Block shallow or malformed specifications.
img2blockbench validate ./red-panda.json --strict

# Compile the accepted spec.
img2blockbench build ./red-panda.json --output ./dist

# Optional Route 2 preview generated from the cuboid spec.
img2blockbench preview-threejs \
  ./red-panda.json \
  --output ./createRedPandaModel.ts

# Route 3: import standard Three.js Object3D.toJSON output.
img2blockbench from-threejs \
  ./red-panda.three.json \
  --reference ./reference.png \
  --id red-panda-threejs \
  --description "A Minecraft-style red panda" \
  --output ./red-panda-threejs.json

# Preserve and audit the native img2threejs base-color maps.
pip install -e '.[reference-projection]'
python tools/img2threejs/bake-reference-faces.py \
  ./red-panda-threejs.json \
  --reference ./reference.png \
  --output-spec ./red-panda-imported.json \
  --audit ./red-panda-texture-transfer-audit.json

# After defining a red-panda entry in semantic-recipes.json, remove
# texture-only detail geometry and paint its landmarks.
python tools/img2threejs/semanticize-model-spec.py \
  ./red-panda-imported.json \
  --animal red-panda \
  --recipes ./tools/img2threejs/semantic-recipes.json \
  --reference ./reference.png \
  --output ./red-panda-textured.json

# Re-audit an existing Blockbench file.
img2blockbench audit ./dist/red-panda.bbmodel
```

The build directory contains:

```
red-panda.bbmodel
red-panda.png
red-panda.geo.json
red-panda.model-spec.json
red-panda.reference.png
red-panda.audit.json
red-panda.manifest.json
red-panda.zip
```

See the original fox example in [ examples/fox](/orca-gamedev/img2blockbench/blob/main/examples/fox).

All three routes converge on the same validated Minecraft model specification and delivery bundle, but their upstream reasoning is separate.

| Route 1 | Route 2 | Route 3 | |
|---|---|---|---|
| Route | Image → selected 3D generator → cuboid spec | Image → cuboid spec | Image → img2threejs → cuboid spec |
| Intermediate | Textured GLB or GLTF | Native cuboid JSON | Generated TypeScript + Object3D JSON |
| External 3D GPU | Provider-dependent | None | None |
| Best fit | Ambiguous organic depth | Minecraft-native references | Cheaper procedural 3D reconstruction |
| Platypus cuboids | 22 | 16 | 28 |
| Shared output | `.bbmodel` , texture, `geo.json` , audit, bundle |
Same | Same |

Route 2 may optionally render its cuboid spec through Three.js for review. That
preview does not make it Route 3. Route 3 begins with the official img2threejs
spec and generated Three.js scene, then ends as a native `.bbmodel`

.

The repository retains its original `lane1`

and `lane2`

artifact paths so
existing links remain valid. The demo and documentation define their public
route order independently.

The current [platypus benchmark](/orca-gamedev/img2blockbench/blob/main/examples/platypus/benchmark.json) records
artifact sizes, hashes, cuboid counts, bone counts, and external GPU
requirements. Generation latency, provider price, and LLM token usage were not
captured for the existing runs, so the repository does not fabricate those
cost numbers.

The benchmark meshes below were generated with Trellis. The provider-neutral audit rasterizes any supported source mesh and the reconstructed cuboids as front, side, top, and isometric silhouettes, then records intersection-over-union (IoU), source coverage, and model precision.

| Model | Mean IoU | Source coverage | Evidence |
|---|---|---|---|
| Platypus | 0.608 | 0.916 |
|

[audit](/orca-gamedev/img2blockbench/blob/main/examples/chimpanzee/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/chimpanzee/lane2/overlap-sheet.png)[audit](/orca-gamedev/img2blockbench/blob/main/examples/elephant/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/elephant/lane2/overlap-sheet.png)[audit](/orca-gamedev/img2blockbench/blob/main/examples/tiger/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/tiger/lane2/overlap-sheet.png)[audit](/orca-gamedev/img2blockbench/blob/main/examples/coyote/lane2/overlap-audit.json)·[sheet](/orca-gamedev/img2blockbench/blob/main/examples/coyote/lane2/overlap-sheet.png)White pixels are overlap, cyan is source-only, and orange is cuboid-only. These results prove the source meshes are genuinely used, but also show that the current reconstruction is approximate rather than an optimized silhouette fit.

```
pip install -e '.[overlap-audit]'
img2blockbench-overlap \
  --source ./source.glb \
  --spec ./anatomy-spec.json \
  --bbmodel ./model.bbmodel \
  --json ./overlap-audit.json \
  --sheet ./overlap-sheet.png
```

A single image cannot reveal every hidden surface or guarantee exact depth. The workflow records uncertainty, mirrors bilateral anatomy when appropriate, and requires multi-view Blockbench review. It should request another view instead of pretending ambiguous anatomy is known.

```
pip install -e .
python -m unittest discover -s tests -v
python skill/img2blockbench/scripts/img2blockbench.py validate \
  examples/fox/model-spec.json --strict
```

MIT licensed.