{"slug": "sony-arw6-llvc-v3-reverse-engineered-decoder", "title": "Sony ARW6 (LLVC v3) reverse-engineered decoder", "summary": "A developer has reverse-engineered and released an open-source decoder for Sony's ARW6 lossy raw format, used by the ILCE‑7RM6 and ILCE‑7M5 cameras, with the initial implementation done using Claude (Fable 5 and Opus 4.8). The decoder successfully processes APS-C and full-frame images and outputs DNG files, though the author notes potential errors in the format description.", "body_md": "This is a decoder for Sony's ARW6 lossy raw format, used by (at least) ILCE‑7RM6 and ILCE‑7M5 when shooting lossy compressed raw (lossless compressed ARW is a different, LJPEG-based format), along with a format description as far as I could figure it out.\n\nIt successfully decodes APS-C and full-frame images from rawdb, along with the ones I took with my 7RM6.\n\nMost of the research and the initial implementation was done with Claude (Fable 5 and Opus 4.8).\n\nUnfortunately, I don't remember much about image compression, and this format gave me lots of trouble to understand (the canvas was a pain!), so I can't promise I haven't missed or misexplained something.\n\nAny errors made here are mine to blame on!\n\n```\nnix develop\npython -m arw6_proto FILE.ARW OUT.dng\n```\n\nThe image is split into one or several tiles with specified offsets, which can be decoded separately; their layout is described in the bitstream section.\n\nThe pixels from the Bayer grid are rearranged into (we call these *components*):\n\n| component | what it is |\n|---|---|\n`green_lo` |\nthe low‑pass half of the green channel |\n`green_hi` |\nthe high‑pass half of the green channel |\n`chroma_r` |\nred pixels as a residual against the neighbouring greens |\n`chroma_b` |\nblue pixels as a residual against the neighbouring greens |\n\nThe green channel is split once by a single horizontal 5/3 DWT into `green_lo`\n\nand `green_hi`\n\n. This split is a transform\nin its own right and is *not* the 2‑D wavelet described below. The halves are then coded very differently:\n\n`green_lo`\n\n: processed like chromas;`green_hi`\n\n: stored as‑is.\n\nA red or blue pixel is stored as a residue from the two greens of its own 2×2 Bayer cell: one beside it, one above/below. After the image is split by channels, they are the two adjacent columns (left and right). R and B are stored as:\n\n```\nchroma_r = (R − mean(g1, g2)) / 2\nchroma_b = (B − mean(g1, g2)) / 2\n```\n\nGreen is stored as signed integers, offset by 2048. When we restore, we clip all values to 12 bit.\n\nThe Bayer mosaic is then assembled (`decode.assemble_tile`\n\n):\n\n```\nR = clip( (g1 + g2)//2 + 2·chroma_r , 0, 4095 )\nB = clip( (g1 + g2)//2 + 2·chroma_b , 0, 4095 )\n```\n\nThe bit depth seems to be a codec constant, 12 bits, and the component values are compressed. A fixed curve maps each\ncode back to a linear sensor value. The curve is logarithmic (Cineon / S‑Log family) with a linear toe below a knee `K`\n\n:\n\n```\nV(c) = c                          for c ≤ K\nV(c) = (K − A) + A·exp((c − K)/A)  for c > K\n```\n\nwith two design inputs — the log rate `A ≈ 658.4`\n\nand the white level `VMAX = 39003`\n\nreached at the top code. The body\nabove already meets the toe with slope 1 at the knee; `K`\n\nthen follows from `V(4095) = VMAX`\n\n.\n\nThese two constants were fit against a code-value table generated by running samples through Adobe DNG Converter\n(`scripts/delin_fit_table.csv`\n\n; the fit is `scripts/fit_delin_curve.py`\n\n). Adobe's actual curve is still a bit different\nand I couldn't figure out how exactly it is constructed.\n\nDelinearisation is the last decoding step, applied to the assembled mosaic.\n\nWhile `green_hi`\n\nis stored as-is, `green_lo`\n\nand the two chromas are each put through a 3‑level 2‑D 5/3 DWT.\n\nA 2‑D wavelet level filters both axes into Low/High, producing four sub‑bands named by (horizontal, vertical):\n\n**LL**— low both ways: the blurry core, split again and stored at the next level;** HL**— vertical edges,** LH**— horizontal edges,** HH**— diagonal detail.\n\nThe three high‑frequency sub‑bands (HL, LH, HH) are the *orientations* stored at each detail level; the LL is fed to the\nnext level. After the coarsest level you are left with a single small `LL`\n\n. Numbering the detail levels finest‑first:\n\n```\ncomponent  =  D1 (finest)  +  D2  +  D3 (coarsest)  +  LL\n              └── detail levels, 3 orientations each ──┘     └ 1 orient\n```\n\nEvery one of `D1`\n\n, `D2`\n\n, `D3`\n\n, `LL`\n\nis stored on its own; they differ only in how many orientations they hold — each\ndetail level three, `LL`\n\none. We call these *regions*; a region holds its level's data for all the components that go\nthrough the DWT.\n\nThe final `LL`\n\nis additionally stored as horizontal differences; an inverse DPCM undoes it\n(`pixelops.predict_horizontal`\n\n).\n\nDecoding combines the levels coarsest→finest (`idwt.idwt`\n\n). There is a possible parity flip, see below.\n\nThe region-component rows are split into chunks which are encoded separately. Also, some of the detail levels are merged\nat flipped parity during the IDWT (the *flips* below). Both are derived from the number of detail levels and the tile\nheight. Because none of this is stored on disk, we need to derive it too to decode properly.\n\nAll regions use a shared canvas:\n\nEach orientation uses a *lattice* — a strided subset of this canvas's rows. The chunks are sized so that the coarsest\nlattice has one row per chunk: with three detail levels the coarsest step is 2³, so chunks are 8 canvas rows tall.\nLattices can be odd or even; we know which orientations use which. Lattices correspond to the detail levels after the\nDWT; each level halves the resolution, so its lattice is twice as sparse. A lattice can be reused for different data\n(then exactly the same layout is used for it).\n\nThe coordinate axis is shared among regions. Two parameters are set:\n\n`chunk_origin`\n\n— where does the first chunk start on the coordinates;`comp_top`\n\n— where does the*first row*start on the coordinates.\n\n`chunk_origin`\n\nis set to the first row of the coarsest level's detail (odd) lattice — half the coarsest step. `comp_top`\n\nis set such that given a known tile height, the first and the last chunks are symmetrical in amount of data they have.\n\nTo be clear, these coordinates never appear in the bitstream; the canvas only tells the decoder how many rows of each region to decode from each chunk.\n\nFor instance, given the diagram above, the first chunk contains everything at canvas rows 10 and 11:\n\n- 2\n`green_hi`\n\nrows; - for D1: 1 HL row (canvas row 10) and 1 LH and HH row each (row 11), for the 3 components (\n`green_lo`\n\n,`chroma_r`\n\n,`chroma_b`\n\n); - for D2: 1 LH and HH row each, again per component — canvas row 10 is also on D2's odd lattice (rows 2, 6, 10, …);\n- no D3 or LL rows.\n\nChunks are encoded separately by regions and components.\n\nAn orientation only owns the lattice points that fall inside the real image. So, for a given lattice we calculate:\n\n`top`\n\n/`bottom`\n\n— the first and one‑past‑last*lattice‑row indices*that are inside the image;`row_base`\n\n— the first lattice row at/after the chunk origin: chunk 0's window starts there.\n\nFor 3 orientations, the lattices used are:\n\n- HL:\n*even*lattices; - LH, HH:\n*odd*lattices.\n\nFinally, the *flips*. DWT splits the rows into high and low; the order may be reversed. During IDWT, we must merge in\nthe same order. The level's rows alternate between its two lattices; the merge is flipped when the topmost row inside\nthe image falls on the odd lattice rather than the even one. We calculate the flip bit per detail level.\n\nThe horizontal passes never flip. The one exception is recombining the green halves (`decode.combine_green`\n\n), which is\n*always* flipped, putting `green_hi`\n\non the even columns.\n\nAs to *why* this canvas exists and why Sony doesn't just encode every region and component by chunks starting from row\n0, I have no idea. Maybe this is somehow tied to how the encoder works.\n\nThe encoder quantises the wavelet coefficients; this is the only lossy step. Each chunk carries its own per‑orientation\nquantiser `q`\n\n.\n\nDecoding dequantises with a mid‑rise reconstruction of each magnitude to its bin centre, `(mag + 0.5)·2^q`\n\n; magnitude 0\nstays 0, q == 0 means \"no quantisation\", signs are preserved.\n\nAn `.ARW`\n\nis a TIFF file containing a compression tag value for ARW6 (32766) in one of its IFDs. It can also contain a\nJPEG preview in a separate IFD.\n\nThe standard TIFF geometry tags are used:\n\n| tag | meaning |\n|---|---|\n`ImageWidth` / `ImageLength` (0x100 / 0x101) |\ndimensions |\n`StripOffsets` / `StripByteCounts` (0x111 / 0x117) |\nlocation + size of the compressed strip |\n`BlackLevel` (0x7310, Sony‑private) |\nblack level per Bayer channel, R G1 G2 B (expected uniform) |\n`WhiteLevel` (0xc61d) |\nwhite (saturation) level |\n`WB_RGGBLevels` (0x7313, Sony‑private) |\nwhite-balance level per Bayer channel, R G1 G2 B |\n\nThe raw image is stored as a single strip (`StripOffsets[0]`\n\n/`StripByteCounts[0]`\n\n).\n\nThe black and white levels are stored halved (I'm not sure why).\n\nSee also: `container.read_tiff`\n\nThe image (strip in TIFF terminology) is split into independent, rectangular tiles, not to be confused with the TIFF tiles. The strip begins with a little-endian header describing them:\n\n```\nstrip:\n  u32  count            number of tiles\n  u32  _unknown         unknown\n  count × TileRecord:\n        u64  offset     byte offset of this tile's payload within the strip\n        u32  x, y       tile's top-left corner in the Bayer mosaic\n        u32  w, h       tile's size, in mosaic pixels\n```\n\nTiles do not necessarily have the same size — e.g. a 10016‑wide image may split `5024 + 4992`\n\n. Full‑frame files that\nI've observed have 4 tiles; APS‑C crop files have 1. Each tile's `offset`\n\nmarks the start of its self‑contained\ncompressed blob.\n\nSee also: `container.parse_tiles`\n\nEach tile starts with this header. Some sizes here and below are stored in *blocks*: units of 16 bytes. Everything is\nbit‑packed MSB‑first.\n\n```\ntile blob:\n  0x00  sub-header block (0x10 bytes):\n          u32   magic          ASCII \"0000\" (0x30303030)\n          u32   _unknown0      unknown (increments across a frame's tiles)\n          u16   width          tile width, in mosaic pixels\n          u16   comp_height    component height = tile height / 2\n          u32   _unknown1      unknown (0x3d006e00 in every sample)\n  0x10  region-totals area (0x20 bytes = 2 blocks):\n          5 × u24   total block count of each coding region\n  0x30  5 × record block (0x10 bytes each), one per coding region:\n          u8        component count (number of components in this region)\n          count × u24   block count of each component's data in this region\n  0x80  [component data begins here]\n```\n\nThe number of regions is not in the file and is always five, in this order:\n\n| region | contents | components |\n|---|---|---|\n| 0 | `LL` |\n3 |\n| 1 | `D3` — coarsest detail |\n3 |\n| 2 | `D2` |\n3 |\n| 3 | `D1` — finest detail |\n3 |\n| 4 | `green_hi` |\n1 |\n\nWithin the shared regions the components are stored in the order `green_lo`\n\n, `chroma_r`\n\n, `chroma_b`\n\n.\n\nThe region totals and per‑component sizes are plain sizes; the byte offset of any one component's data is the sum of everything before it. A region's total may be larger than the sum of its components' sizes — the next region then starts past the padding.\n\nEach region‑component starts with a 16‑byte header followed by a chunk table, then the entropy‑coded chunk data:\n\n```\ncomponent:\n  0x00  header block (0x10 bytes):\n          u16   table_blocks   chunk-table size in blocks (header + table = table_blocks+1 blocks)\n          u24   chunk_blocks   entropy-chunk data size in blocks\n          u8    _unknown0      unknown (constant format byte; 0x40 in every sample)\n          u2    orient_count   orientation count (high bits of its byte): 1 = LL/green_hi, 3 = detail\n          u6    _unknown1      unknown (rest of the orientation-count byte — 0 in every sample, not assumed)\n          u16   chunk_count    number of chunk-table entries\n          u8    _unknown2      unknown (constant format byte; 0x10 in every sample)\n          u48   (padding)\n  0x10  chunk table (table_blocks blocks):\n          chunk_count × chunk entry:\n                u16  length             byte length of chunk i (0 ⇒ empty chunk)\n                orient_count × u4  q    per-orientation quantiser, applied to that chunk's coefficients\n  (table_blocks+1)·0x10  [entropy chunks begin here]\n```\n\nChunk `i`\n\nis the data for the `i`\n\n‑th 8‑row cut of the canvas (component rows — 16 mosaic rows), anchored at\n`chunk_origin`\n\n. A chunk length of 0 marks a chunk that has none of this component's rows (e.g. the topmost chunks of the\ncoarse regions, which fall in the margin \"air\" above the image).\n\nEach chunk is an independent bitstream holding its rows of each orientation in turn, in the HL, LH, HH order (also the\norder of the `q`\n\nvalues in the chunk table). Each row is a *line* of `ceil(width / 4)`\n\ngroups of four coefficients (the\nlast group's excess coefficients are discarded), read with an adaptive Golomb‑Rice code (`bitstream.py`\n\n):\n\nThe Rice parameter `k`\n\nis updated with a variable-length code (skipped after the line's last group): `0`\n\nkeeps the\ncurrent `k`\n\n; `10`\n\nfollowed by a unary number (zeroes ending with one) raises it by number + 1; `11`\n\nfollowed by a unary\nnumber lowers it by number + 1, the run capped so `k`\n\nnever drops below zero (at the cap the terminating one is\nomitted).\n\nEach line starts by reading an update code for the Rice parameter `k`\n\n, initially 0. While `k = 0`\n\n, the line is coded as\nruns of all‑zero groups (an Elias‑gamma‑style exponent + mantissa), each run followed — unless it ends the line — by an\nunary code that raises `k`\n\nby at least 1.\n\nOtherwise, each group reads four `k`\n\n‑bit values, then an update code for `k`\n\n, then one sign bit per non‑zero value is\nstored.\n\nThe values are then dequantised with the chunk's own quantiser values from the chunk table.\n\n| module | role |\n|---|---|\n`decode.py` |\ntop‑level decoder: `decode_tile` (one tile → mosaic) and `decode` (full mosaic) |\n`container.py` |\nbyte structure: TIFF/strip/tile parsing, the tile sub‑header, per‑component headers |\n`geometry.py` |\nthe canvas coordinate frame and every region's derived geometry + component count |\n`bitstream.py` |\nGolomb‑Rice entropy decoding of the chunks |\n`idwt.py` |\nthe inverse 5/3 wavelet (IDWT) |\n`pixelops.py` |\ndequant, LL predictor, delinearisation curve |\n`utils.py` |\nMSB bit readers, integer helpers |\n`dngwriter.py` |\nwrap a decoded mosaic into a mosaiced DNG |\n`convert.py` |\nthe CLI (`python -m arw6_proto` ) |", "url": "https://wpnews.pro/news/sony-arw6-llvc-v3-reverse-engineered-decoder", "canonical_source": "https://github.com/abbradar/arw6_decode", "published_at": "2026-07-21 08:42:44+00:00", "updated_at": "2026-07-21 08:52:52.640369+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["Sony", "ARW6", "ILCE‑7RM6", "ILCE‑7M5", "Claude", "Adobe DNG Converter"], "alternates": {"html": "https://wpnews.pro/news/sony-arw6-llvc-v3-reverse-engineered-decoder", "markdown": "https://wpnews.pro/news/sony-arw6-llvc-v3-reverse-engineered-decoder.md", "text": "https://wpnews.pro/news/sony-arw6-llvc-v3-reverse-engineered-decoder.txt", "jsonld": "https://wpnews.pro/news/sony-arw6-llvc-v3-reverse-engineered-decoder.jsonld"}}