{"slug": "fx2-cmix-transformer", "title": "Fx2-Cmix-Transformer", "summary": "The fx2-cmix-transformer program achieved the top ranking on the Large Text Compression Benchmark, compressing the first 10^9 bytes of the English Wikipedia XML dump (enwik9) to 96,996,198 bytes, with a total size of 96,996,198 bytes including the decompresser, as of Sept 2, 2026. The benchmark, maintained by Matt Mahoney, ranks lossless compression programs by compressed size plus decompresser size, and aims to encourage research in artificial intelligence and natural language processing.", "body_md": "Matt Mahoney\n\nLast update: Sept 2, 2026. [history](#history)\n\nThis competition ranks lossless data compression programs by the compressed\nsize (including the size of the decompression program) of the first \n10<sup>9</sup> bytes of the XML text dump of the English version of Wikipedia on Mar. 3, 2006.\n[About the test data](textdata.html).\n\nThe goal of this benchmark is not to find the best overall compression program, but\nto encourage research in artificial intelligence and natural language processing (NLP).\nA fundamental problem in both NLP and text compression is modeling: the ability to\ndistinguish between high probability strings like *recognize speech* and low \nprobability strings like *reckon eyes peach*.\n[Rationale](rationale.html).\n\nThis is an open benchmark.  Anyone may contribute results.  Please read the\n[rules](textrules.html) first.\n\nOpen source compression improvements to this benchmark with certain hardware\nrestrictions may be eligible for the [Hutter Prize](http://prize.hutter1.net/).\n\nCompressors are ranked by the compressed size of enwik9 (10<sup>9</sup> bytes) \nplus the size of a zip archive containing the decompresser.  Options are selected\nfor maximum compression at the cost of speed and memory.  Other data in the table does not\naffect rankings.  This benchmark is for informational purposes only.  There is no prize money\nfor a top ranking.  Notes about the table:\n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp   Mem Alg Note\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----   --- --- ----\nfx2-cmix-transformer                                       96,996,198          0 xd  96,996,198 133766 131561  9547 Tr  108\naltxs 1.0.0                                                93,434,410 13,490,401 xd 106,924,811 227372 1189976 88000 Tr 110\nnncp v3.2                                     14,915,298  106,632,363    628,955 xd 107,261,318 241871 238670  7600 Tr  88\nfx-deepmix                                                107,828,411          0 xd 107,828,411        301180 10233 LSTM 106\ncmix-obias 18 Aug 2026                                    108,009,834          0 xd 108,009,834        119920  9836 LSTM  108\ncmix v21          -t                          14,623,723  107,963,380    281,387 sd 108,244,767 622949 638442 30950 CM  83\nforge-cmix v2                                             109,079,343          0 xd 109,079,343        182008  8505 LSTM  111\ncmix-lex                                      14,952,093  109,190,109          0 xd 109,190,109        265512  9407 LSTM  106\nfx2-cmix                                                  110,351,665          0 xd 110,351,665        272072  8811 LSTM  97\njax-compress                                  15,505,441  113,393,442     60,872 sd 113,393,442 110013 112140 41900 LSTM 105\ntensorflow-compress v4                        15,905,037  113,542,413     55,283 sd 113,597,696 291394 290803 45360 LSTM 94\ncmix-hp 10 Jun 2021                           15,957,339  113,712,798          0 xd 113,712,798 189420 194280  6873 CM  89\nfast-cmix                                                 113,746,218          0 xd 113,746,218        121971  8027 CM  99\nstarlit 31 May 2021                           15,215,107  114,951,433          0 xd 114,951,433 173953 171682 10233 CM  89\nphda9 1.8                                     15,010,414  116,544,849     42,944 xd 116,587,793  86182  86305  6319 CM  83\ngmix v1                                       16,246,629  122,336,013    182,085 sd 122,518,098  73986  70552  3751 CM  83\npaq8px_v206fix1   -12L                        15,849,084  124,696,410    402,949 s  125,099,359 291916 294847 28151 CM  93\ndurilca'kingsize  -m13000 -o41 -t2            16,209,219  127,376,595    406,700 xd 127,783,295   1341   1466 13000 PPM 109\nfxv v1            -1 -j -w -cfxcm1.pxv        15,946,608  128,804,372    494,797 x  129,299,169  20763         1628 CM  102\ncmve 0.2.0        -m2,3,0x7fed7dfd            16,424,248  129,876,858    307,787 x  130,184,645 1140801       19963 CM  81\npaq8hp12any       -8                          16,230,028  132,045,026    330,700 x  132,375,726  37660  37584  1850 CM  41\ndrt|emma 1.23                                 16,523,517  134,164,521  1,358,251 xd 135,522,772  73006  67097  3800 CM  81\nzpaq 6.42         -m s10.0.5fmax6             17,855,729  142,252,605      4,760 sd 142,257,365   6699  14739 14000 CM  61\ndrt|lpaq9m        9                           17,964,751  143,943,759    110,579 x  144,054,338    868   898 1542 CM   41\nmcm 0.83          -x11                        18,233,295  144,854,575     79,574 s  144,934,149    394   281 5961 CM   72\nnanozip 0.09a     -cc -m32g -p1 -t1 -nm       18,594,163  148,545,179    783,642 x  149,328,821   1149  1141 32000 CM  74\nxwrt 3.2    -l14 -b255 -m96 -s -e40000 -f200  18,679,742  151,171,364     52,569 s  151,223,933   2537  2328 1691 CM\nfp8 v3            -8                          18,438,169  153,188,176     50,068 s  153,238,244  20605 22593 1192 CM   26\nWinRK 3.03        pwcm +td 800MB SFX          18,612,453  156,291,924     99,665 xd 156,391,589  68555        800 CM   10\nppmonstr J        -m1700 -o16                 19,055,092  157,007,383     42,019 x  157,049,402   3574 ~3600 1700 PPM\nstc                                           19,996,028  157,388,188    263,902 s  157,388,188   1031   475 11874 BWT 106\nzcm 0.93          -m8 -t1                     19,572,089  159,135,549    227,659 x  159,363,208    421   411 3100 CM   48\nslim 23d          -m1700 -o12                 19,077,276  159,772,839     69,453 x  159,842,292   5232 ~5400 1700 PPM\nbsc-m03 0.4.0     -b1000000000                20,293,393  160,258,936   105,456 xd  160,364,392    160   135 13000 BWT 96\nbwmonstr 0.02                                 20,307,295  160,468,597     69,401 x  160,537,998 331801 156147 590 BWT  30\nglza 0.12         c -x -o0.6 -p4 -r16000      20,068,917  161,678,356     56,526 s  161,678,356  11771   9.4 15331 Gram 67\nnanozipltcb 0.09                              20,537,902  161,581,290    133,784 x  161,715,074     64    30 3350 BWT  40\nkanzi    -b 1024M -t RLT+TEXT+UTF -e TPAQX    19,098,186  161,690,495    397,855 x  162,088,350    490   480 3100 CM   103\nM03 1.1b          1000000000                  20,710,197  163,667,431     50,468 x  163,717,899    457   406 5735 BWT  52\nbcm 2.03          -b1000x-                    20,738,630  163,646,387    125,866 x  163,772,253     63    34 4096 BWT  98\nbsc 3.25          -b1000 -e2                  20,786,794  163,884,462     74,297 xd 163,958,759     23     8 5000 BWT  96\nbbb               m1000                       20,847,290  164,032,650     11,227 s  164,043,877   4524  2619 1401 BWT\nppmx 0.10         c16                         20,705,277  164,250,527     93,482 x  164,344,009    251   252 39727 PPM 107\npcompress 3.1     -c libbsc -l14 -s1000m      20,769,968  163,391,884  1,370,611 x  164,762,495    359    74 3300 BWT  48\npaq9a             -9                          19,974,112  165,193,368     13,749 s  165,207,117   3997  4021 1585 CM\nuda 0.300                                     19,393,460  166,272,261     11,264 x  166,283,525  25282 25174  180 CM\nBWTmix v1         c10000                      20,608,793  167,852,106      9,565 x  167,861,671   1794   690 5000 BWT  49\nlrzip 0.612       -z -L 9 -p 1                19,847,690  169,318,794     99,363 x  169,418,157   2987  2929 2700 CM   33\nbzip3             -b 511                      20,749,611  169,990,721    368,033 s  170,358,754    175   146 3700 BWT  103\ncm4_ext                                       20,188,048  170,566,799    204,782 x  170,771,581   4123  4130 1906 CM   26\nM1x2 v0.6         7 enwik7.txt                20,723,056  172,212,773     38,467 s  172,251,240    711   715 1051 CM   26\ncmm4 v0.1e        96                          20,569,034  172,669,955     31,314 x  172,701,269   2052  2056 1321 CM\nlstm-compress v3                              20,318,653  173,874,407    144,567 s  174,018,974  92342 91876    9 LSTM 83\nccmx 1.30         7                           20,857,925  174,142,092     15,014 x  174,157,106   1313  1338 1332 CM\nbit 0.7           -p=5                        20,823,204  174,425,039     62,493 x  174,487,532   2050  2100  663 CM   26\nmcomp 2.00        -mw -M320m                  21,103,670  174,388,351    172,531 x  174,560,882    473   399 1643 BWT  26\nepmopt|epm r9     -m800 -n20 --fixedorder:12  19,713,502  174,817,424    141,101 x  174,958,525   3179  3376  800 PPM\nWinUDA 2.91       mode 3 (194 MB)             20,332,366  174,975,730     17,203 x  174,992,933  23610 23473  194 CM\ndark 0.51         -b333mf                     21,169,819  175,471,417     34,797 x  175,506,214    533   453 1692 BWT\nFreeArc 0.40pre-4 -mppmd:1012m:o13:r1         20,931,605  175,254,732    748,202 x  176,002,934   1175  1216 1046 PPM\nhook v1.4         1700                        21,990,502  176,648,663     37,004 x  176,685,667    741   695 1777 DMC  26\n7zip 4.46a        -m0=ppmd:mem=1630m:o=10 ... 21,197,559  178,965,454          0 xd 178,965,454    503   546 1630 PPM  23\nrings 2.5         -m8 -t1                     20,873,959  178,747,360    240,523 x  178,987,883    280   163 2518 BWT  48\npimple2                                       20,871,457  180,251,530     78,642 x  180,330,172  18474 17992  128 CM\nash 04a           /m700 /o10                  19,963,105  180,735,542     11,137 x  180,746,679   6100  5853  700 CM\nbce3                                          22,729,148  180,732,702     19,889 s  180,752,591   1151  2444 5000 CM   71\nocamyd LTCB 1.0   -s0 -m3                     21,285,121  182,359,986     21,030 x  182,381,016 108960~110000 300 DMC   6\nbee 0.79 b0154    -m3 -d8                     20,975,994  182,373,904     57,046 x  182,430,950   9295  9285  512 PPM\nuhbc 1.0          -m3 -b100m                  20,930,838  182,918,172     56,242 x  182,974,414   1569   809  800 BWT\nsmac 1.20                                     21,781,544  183,190,888      4,356 x  183,195,244   4249  4399 1542 CM   26\nppmd J1           -m256 -o10 -r1              21,388,296  183,964,915     11,099 s  183,976,014    880   895  256 PPM\ntc 5.2 dev 2                                  21,481,399  184,939,711     41,112 x  184,980,823   3637  3655  230 CM\nbwtsdc v1                                     23,414,955  185,709,858      8,421 s  185,718,279   2100   420 5213 BWT  47\nfbc v1.1          333333334                   22,554,133  185,975,548     23,576 x  185,999,124    451   415 1647 BWT  55\nppmvc v1.1        -m256 -o8 -r1               21,484,294  186,208,405     25,241 x  186,233,646    898   913  272 PPM\nchile 0.4         -b=244141                   22,218,917  186,979,614     11,530 s  186,991,144   2513   512 1426 BWT\nbwtdisk 0.9.0     -b 2 -m 3500                24,725,277  190,004,306    169,579 s  190,173,885   1124       3500 BWT  48\nCTXf 0.75 pre b1  -me                         22,072,783  191,008,871     57,337 x  191,066,298   1112  1037   78 PPM\nm03exp 2005-02-15 32MB blocks                 21,948,192  191,250,500     44,593 x  191,295,093  ~4800 ~2100  256 BWT\nStuffit 12.0.0.17 -m=4 -l=16 -x=30            22,105,654  190,372,707  2,658,122 xd 193,030,829    628   658 1062 PPM\nplzma v3b         c2 ... (see below)          24,206,571  193,240,160    101,221 x  193,341,381   8889    55 10110 LZ77 58\ncrook v0.1        -m1600 -O8                  22,503,627  193,333,159      8,539 s  193,341,698    483   513 1641 PPM  26\nlzturbo 1.1       -49 -b1000 -p0              24,416,777  194,681,713    110,670 x  194,792,383   1920     9 14700 LZ77 59\nenc 0.15          aq                          22,156,982  195,604,166     94,888 x  195,699,054   6843  6868   50 CM\ncomprolz 0.11.0-bugfix1  -b250 -f             22,813,215  196,651,379     29,453 x  196,680,832    984   308  688 ROLZ 26\nsbc 0.970r2       -ad -m3 -b63                22,470,539  197,066,203     99,094 xd 197,165,297   1733   313  224 BWT\nxz 5.2.1--lzma2=preset=9e,dict=1GiB,lc=4,pb=0 24,703,772  197,331,816     36,752 xd 197,368,568   5876    20 6000 LZ77 73\nWinRAR 3.60b3     -mc7:128t+ -sfxWinCon.sfx   22,713,569  198,454,545          0 xd 198,454,545    506   415  128 PPM\nquark v0.95r beta -m1 -d25 -l8                22,988,924  198,600,023     80,264 x  198,680,287  27952   217  534 LZ77\nlzip 1.14-rc3     -9 -s512MiB                 24,756,063  199,410,543     21,682 s  199,432,225   2409    21 5632 LZ77 57\ncomprox 0.11.0-bugfix1 -b250 -f -m100         23,064,386  199,515,912     34,176 x  199,550,088    917   153  688 LZ77 26\nbssc 0.95 alpha   -b16383                     23,117,061  201,810,709     45,489 x  201,856,198    578   217  140 BWT   4\nflashzip 1.0.0    -mx7 -b7                    23,869,034  202,363,445    123,053 x  202,486,498   1296   122  802 ROLZ 26\nlzham 1.0         -d29 -x                     25,002,070  202,237,199    191,600 s  202,428,799   1096   6.6 7800 LZ77 70\ncsarc 3.3         -m5 -d1024m                 24,516,202  203,995,005     69,848 s  204,064,853    621    22 2463 LZ77 48\npacket 1.9        -mx -b512 -h8               24,968,492  204,195,438    261,967 x  204,457,405    974    14 2824 LZ77 48\nuharc 0.6b        -mx -md32768                23,911,123  208,026,696     73,608 xd 208,100,304   1666  1330   50 PPM\nTarsaLZP Jan 29 2012                          24,751,389  208,867,187     13,081 s  208,880,268    203      ~2000 LZP  54\nGRZipII 0.2.4     -b8m                        23,846,878  208,993,966     41,645 s  209,035,641    312   216   58 BWT\n4x4 0.2a          4t (grzip:m1:h18)           23,833,244  208,787,642    317,097 x  209,104,739    386   240  269 BWT\nrzm 0.07h                                     24,361,070  210,126,103     17,667 x  210,143,770   2336    81  160 ROLZ\npim 2.50          best                        24,303,638  210,124,895    330,901 x  210,455,796    764  ~764   88 PPM\nCTW 0.1           -d6 -n16M -f16M             23,670,293  211,995,206     43,247 x  212,038,452  19221 19524  144 CM\nboa 0.58b         -m15                        24,322,643  213,845,481     55,813 x  213,901,294   3953 ~4100   17 PPM\nyxz 0.11          -m9 -b7 -h6                 25,754,856  214,317,684    131,062 x  214,448,746    642    77 1590 LZ   26\nzstd 0.6.0        -22 --ultra                 25,405,601  215,674,670     69,687 s  215,744,357    701   2.2  792 LZ77 76\ntornado 0.6       -16                         25,768,105  217,749,028     83,694 s  217,832,722   1482     9 1290 LZ77 48\nLZPXj 1.2h        9                           25,205,783  217,880,584      4,853 s  217,885,437    783   717 1316 PPM  \nscmppm 0.93.3     -l 9                        25,198,832  217,867,392     37,043 s  217,904,435    708   644   20 PPM\nacb 2.00c         u                           25,063,656  218,473,968     38,976 x  218,512,944  10656 10883   16 LZ77 26\ncrushm                                        25,013,576  218,656,416     30,097 x  218,686,513    617   649   39 CM   26\nPX v1.0                                       24,971,871  219,091,398      3,054 s  219,094,452   1838  1809   66 CM    3\nDGCA 1.10         default+SFX                 25,203,248  219,655,072          0 xd 219,655,072    858   270   76\nSqueez 5.20.4600  sqx2.0 32MB Ultra           25,118,441  220,004,873     91,019 xd 220,095,892   2575   116  365\nfpaq2                                         25,287,775  221,242,386      3,429 s  221,245,815  20183 20186  131 CM\nTinyCM 0.1        9                           25,913,605  221,773,542     12,553 x  221,786,095   1342  1330 1083 CM   26\ndmc               c 1800000000                25,320,517  222,605,607      2,220 s  222,607,827    676   721 1800 DMC\nlza 0.82b         -mx9 -b7 -h7                26,396,613  222,808,457    285,766 x  223,094,223    449   9.7 2000 LZ77 48\nbrotli 18-Feb-2016 -q 11 -w 24                25,764,698  223,597,884    542,385 s  224,140,269   3400   5.9  437 LZ77 48\nszip 1.12a        -b41o16                     26,120,472  227,586,463     31,708 x  227,618,171   1191   289   21 BWT  26\nbalz 1.13         ex                          26,421,416  228,337,644     49,024 x  228,286,668   3700   190  206 ROLZ\nlzpm 0.11         9                           26,501,542  229,083,971     46,824 x  229,130,795  15395    57  740 ROLZ\nqazar 0.0pre5     -l7 -d9 -x7                 26,455,170  229,846,871     71,959 x  229,918,830   5738   903  105 LZP\nKuaiZip 2.3.2 x86                             25,895,915  227,905,650  3,857,649 x  231,763,299   1061    47  197 LZ77 26\nqc 0.050          -8                          26,763,343  232,784,501     46,100 x  232,830,601   8218  1503  151\nppms J            -o5                         26,310,248  233,442,414     16,467 x  233,458,881    330   354  1.8 PPM\ndzo beta                                      26,616,115  235,056,859    618,883 x  235,675,742   1088   159  200 LZ77 26\ncomprox_ba 20110929                           27,828,189  242,846,243      4,134 s  242,850,377    397   101  226 BWTS 48\nWinTurtle 1.60    512 MB buffer               28,379,612  245,217,944    160,090 x  245,378,034    273   237  583 PPM\ndiz                                           26,545,256  246,679,382     12,945 s  246,692,327  21240 22746 1350 PPM  26\ncabarc 1.00.0601  -m lzx:21                   28,465,607  250,756,595     51,917 xd 250,808,853   1619    15   20 LZ77\nsr3                                           28,926,691  253,031,980      9,399 s  253,054,625    148   160   68 SR   26\nbzip2 1.0.2       -9                          29,008,736  253,977,839     30,036 x  254,007,875    379   129    8 BWT\nrh5_x64           -window:27 c6               29,078,552  254,220,469     36,744 x  254,257,213    196   9.4  145 ROLZ 48\nRangeCoderC v1.7  c7 26                       28,788,013  254,527,369      7,858 x  254,535,227   2460  2436 1116 CM   26\nquad v1.11        -x                          29,110,579  256,145,858     13,387 s  256,159,245    956   116   34 ROLZ\nWinACE            -sfx -m5 -d4096             29,481,470  257,237,710          0 xd 257,237,710   1080    77    4\nlzsr 0.01                                     29,433,834  258,912,605     40,287 x  258,952,892    194    88    6 LZ77 26\nlibzling 20160107 e4                          29,721,114  259,475,639     35,582 s  259,511,221     83    27   28 ROLZ 48\nxpv5              c2                          29,963,217  262,525,246     14,371 x  262,539,617   2359   516    9 ROLZ 26\nsr3c 1.0                                      29,731,019  266,035,006      7,701 x  266,042,707    160   145    5 SR   26\nlzc v0.08         10                          30,611,315  266,565,255     11,364 x  266,576,619    302    63  550 LZ77\nnakamichi 2019-Jul-01                         32,917,888  277,293,058    112,899 s  277,405,957 8200000  1.3 302000 LZSS 85\ncrush 1.00        cx                          31,731,711  279,491,430      2,489 s  279,493,919    948   2.9  148 LZ77 60\nxeloz 0.3.5.3     c889                        32,441,272  283,621,211     18,771 s  283,639,982   1079     8  230 LZ77 48\nbzp 0.2                                       31,563,865  283,908,295     36,808 x  283,945,103    110   120    3 LZP\nlzwg              -27                         34,423,369  284,356,322     19,828 xd 284,376,150    135    41 1744 LZW  95\nha 0.98           a2                          31,250,524  285,739,328     28,404 x  285,767,732   2010  1800  0.8 PPM\nulz 0.06          c9                          32,945,292  291,028,084     49,450 x  291,077,534    325   1.1  490 LZ77 82\nirolz                                         33,310,676  292,448,365      4,584 s  292,452,949    274   144   17 ROLZ 26\nlcssr 0.2         -b7 -l9                     34,549,048  296,160,661      8,802 x  296,169,463   8186  8281 1184 SR\nzlite                                         33,975,840  298,470,807      4,880 s  298,475,687     61    28   36 ROLZ 26\nlazy 1.00         5                           35,024,082  306,245,949      5,986 s  306,251,935    273    24   96 LZ77 26\nzhuff 0.97 beta   -c2                         34,907,478  308,530,122     63,209 x  308,593,331     24   3.5   32 LZ77 48\nlzhhf                                         34,848,933  308,825,079     24,576 xd 308,849,655    392    12   14 LZ77 95\nslug 1.27                                     35,093,954  309,201,454      6,809 x  309,208,263     32    28   14 ROLZ\nect 0.9.5         -9 -zip --mt-deflate        34,950,275  309,402,124    352,904 s  309,755,028   1340       2900 LZ77 103\nlzuf62                                        34,960,889  309,837,920     24,576 xd 309,862,496    375    11   14 LZ77 95\npigz 2.3          -11                         35,002,893  309,812,953     52,717 s  309,865,670   2237    13   25 LZ77 48\nkzip May 13 2006  /b1024                      35,016,649  310,188,783     29,184 xd 310,217,967   6063    62  121 LZ77  2\nuc2 rev 3 pro     -tst                        35,384,822  312,767,652    123,031 x  312,890,683    360    63    4 LZ77\nqbp                                           35,434,557  313,013,756      3,232 xd 313,013,756     44    30    1 LZSS 104\nthor 0.95         e4                          35,795,184  314,092,324     49,925 x  314,142,249     64    34   16 LZP\netincelle a3                                  35,776,971  314,801,710     44,103 x  314,845,813     29    18  976 ROLZ 26\nlz5 1.3.3         -18                         36,514,408  319,510,433    138,210 s  319,648,643  10578   3.7 1139 LZ77 48\ngzip124hack 1.2.4 -9                          36,273,716  321,050,648     62,653 x  321,113,301    149    19    1 LZ77 \ndoboz 0.1                                     36,367,430  322,415,409     83,591 x  322,499,000    533   3.4 1200 LZ77 48\ngzip 1.3.5        -9                          36,445,248  322,591,995     38,801 x  322,630,796    101    17  1.6 LZ77\nInfo-ZIP 2.3.1    -9                          36,445,373  322,592,120     57,583 x  322,649,703    104    35  0.1 LZ77\npkzip 2.0.4       -ex                         36,556,552  323,403,526     29,184 xd 323,432,710    171    50  2.5 LZ77\njar (Java) 0.98-gcc  cvfM                     36,520,144  323,747,582     19,054 x  323,766,636    118    95  1.2 LZ77\nPeaZip            better, no integrity check  36,580,548  323,884,274    561,079 x  324,445,353    243   243    8 LZ77 20\narj 3.10          -m1                         37,091,317  328,553,982    143,956 x  328,697,938    262    67    3 LZ77 26\nlzgt3a                                        37,444,440  334,405,713      4,387 xd 334,410,100   1581  2886    2 LZ77\npucrunch          -d -c0                      39,199,165  350,265,471     34,359 s  350,299,830   2649   463    2 LZ77\npackARC v0.7RC11  -sfx -np                    38,375,065  361,905,425          0 xd 361,905,425   1359  1486   23 CM\nurban                                         38,215,763  362,677,440      4,280 s  362,681,720    381   450    6 o2   48\nlzop v1.01        -9                          41,217,688  366,349,786     54,438 x  366,404,224    289    12  1.8 LZ77\nlzw 0.2                                       41,960,994  367,633,910        671 s  367,634,581   3597    31   18 LZW\nMTCompressor v1.0                             41,295,546  370,152,396      3,620 x  370,156,016    173   117   74 LZ77 26\nlz4x 1.02         c4                          41,950,112  372,068,437     48,609 x  372,117,046     79   1.4  114 LZ77 68\narbc2z                                        38,756,037  379,054,068      6,255 sd 379,060,323   2659  2674   68 PPM\nlz4 v1.2          -c2                         42,870,164  379,999,522     49,128 x  380,048,650     91     6   20 LZ77 26\nlzss 0.02         cx                          42,874,387  380,192,378     48,114 x  380,240,492    107   2.3  145 LZSS 63\nxdelta 3.0u       -9                          44,288,463  389,302,725    107,985 x  389,410,710   1021    30   47 LZ77\nbrieflz 1.1.0                                 43,300,800  390,122,722     14,907 s  390,137,629     21   7.5    3 LZ77 48\nmtari 0.2                                     41,655,528  397,232,608      4,156 s  397,236,764     80    99   18 CM   26\nlzf 1.02          cx                          45,198,298  406,805,983     48,359 x  406,854,342     68   2.2  151 LZ77 68\nsrank 1.1         -C8                         43,091,439  409,217,739      6,546 x  409,224,285     51    45    2 SR\nQuickLZ 1.30b     (quick3)                    46,378,438  410,633,262     44,202 x  410,677,464     48    12    3 LZ77\nstz 0.7.2         -c2                         47,192,312  416,524,596     41,941 x  416,566,537     14    13    3 LZ77 26\ncompress 4.3d                                 45,763,941  424,588,663     16,473 x  424,605,136    103    70  1.8 LZW\nlzrw3-a                                       48,009,194  438,253,704      4,750 x  438,258,454     38    17    2 LZ77\nfcm1                                          45,402,225  447,305,681      1,116 s  447,306,797    228   261    1 CM1\nruncoder1                                     46,883,939  458,125,932      5,488 s  458,131,420    140   156    4 o1   26\ndata-shrinker 23Mar2012                       51,658,517  459,825,318      3,706 s  459,829,024     14     4    2 LZ77 26\nlzwc_bitwise 0.7                              46,639,414  463,884,550      4,183 x  463,888,733    123   134   71 LZW  26 \nexdupe 0.3.3                                  53,717,422  478,788,378  1,092,986 x  479,881,364     27     5 1000 LZ77 48\nlzv 0.1.0                                     54,950,847  488,436,027     10,385 x  488,446,412      4   2.6    3 LZ77 48\nFastLZ Jun 12 2007                            54,658,924  493,066,558      7,065 xd 493,073,623     18    13    1 LZ77\nsharc 0.9.11b     -c2                         53,175,042  494,421,068     81,001 s  494,502,069     15    14    6 LZP  26\nflzp v1                                       57,366,279  497,535,428      3,942 s  497,539,370     78    38    8 LZP\nalba 0.5.1        cd                          52,728,620  515,760,096      4,870 s  515,764,966    239    10    4 BPE  48\nlzpgt6                                        56,113,248  522,877,083     27,136 x  522,904,219      6     5    6 LZP  95\nsnappy 1.0.1                                  58,350,605  527,772,054     23,844 s  527,795,898     25    12  0.1 LZ77 26\nbpe               5000 4096 200 3             53,906,667  532,250,688      1,037 sd 532,251,725    639    28  0.5 Dict 26\nkwc                                           54,097,740  532,622,518     15,186 x  532,637,704    438   145  668 Dict 26\nbpe2 v3                                       55,289,197  542,748,980      2,979 s  542,751,959    518   132  0.5 Dict 26\nfpaq0f2                                       56,916,872  558,645,708      3,066 x  558,648,769    222   207  0.4 o0\nghost             456 5                       55,357,196  568,004,779        696 sd 568,005,475 172800   245 88000 Dict 100\nppp                                           61,657,971  579,352,307      1,472 s  579,353,779     80    59    1 SR\nksc               4                           59,511,259  580,557,413     13,507 x  580,570,920  40050  7917 1700 SR   48\nlzbw1 0.8                                     67,620,436  590,235,688     21,751 x  590,257,439     15    12   55 LZP  26\nlzp2 0.7c                                     67,909,076  598,076,882     40,819 x  598,117,701     11     8   15 LZP  26\nNTFS              LZNT1                       76,955,648  636,870,656          0    636,870,656     10     9  0.1 LZ77 26\nshindlet_fs                                   62,890,267  637,390,277      1,275 xd 637,391,552    113   103  0.6 o0\narb255                                        63,501,996  644,561,595      4,871 sd 644,566,466   2551  2574  1.6 o0\ncompact                                       63,862,371  648,370,029      3,600 sd 648,373,629    216   164  0.2 o0\nTinyLZP 0.1                                   79,220,546  694,274,932      2,811 s  694,277,743     32    38   10 LZP  26\nsmile                                         71,154,788  695,562,502        207 xd 695,562,709  10517 10414  0.6 MTF  26\nbarf              (2 passes)                  76,074,327  758,482,743    983,782 s  759,466,525    756    53    4 LZ77\narb2x v20060602                               99,642,909  995,674,993      3,433 sd 995,678,426   2616  2464  1.6 o0b\nCompression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nhipp 5819         /o8                         20,555,951  (fails)         36,724 x                5570  5670  719 CM\nppmz2                                         23,557,867  (fails)         29,362 s               92210 88070 1497 PPM  26\nXMill 0.8         -w -P -9 -m800              26,579,004  (fails)        114,764 xd                616   530  800 PPM\nlzp3o2                                        33,041,439  (fails)         23,427 xd                230   270  151 LZP\n```\n\nPrograms that properly decompress enwik9 and don't use external dictionaries are still eligible for the Hutter Prize.\n\n```\n                       Compression               Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                  Options                enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                  -------              ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nrdmc 0.06b                                    33,181,612                                          1394  1381      DMC  6\nESP v1.92                                     36,651,292                                           223            LZ77 16\n```\n\nPareto frontier: compressed size vs. compression time as of Aug. 18, 2008 from the main table (options for maximum compression).\n\nPareto frontier: compressed size vs. memory as of Aug. 18, 2008\n(options for maximum compression).\n I only test the latest supported version of a program.  I attempt to find the\noptions that select the best compression, but will not generally do an exhausitve\nsearch.  If an option advertises maximum compression or memory, I don't try the alternatives.\nIf you know of a better combination, please let me know.\nI will select the maximum memory setting that does not cause disk thrashing, usually about 1800 MB.\nIf the compressor is not downloadable as a zip file then I will compress the source or\nexecutable (whichever archive is smaller) plus any other needed files (dictionaries) into a single zip\narchive using 7zip 4.32 -tzip -mx=9.\nIf no executable is available I will attempt to compile in C or C++ \n(MinGW 3.4.2, Borland 5.5 or Digital Mars), Java 1.5.0, MASM, NASM, or gas.\n \n1. Reported by Guillermo Gabrielli, May 16, 2006.  Timed on a Celeron D325 2.53Ghz Windows XP SP2 256MB RAM. I have not verified results submitted by others.  Timing information, when available,\nmay vary widely depending on the test machine used.\n The numbers in the headings are the compression ratios on enwik9.\n The bug was fixed on Aug. 21, 2026. The Intel test runs slowly (8 days) because only 12 GB are available to Linux under WSL2\non the test machine for memory mapping the PPM model to disk to stay within the 10 GB Hutter prize limit.\nThe compressor size is shown because it is used for scoring in the Hutter prize, although not in this benchmark.\nThe original submission was posted to Google Drive: Moved to Github on Sept. 2, 2026: \n\n```\nVersion      archive9    C size     C time  D time  C mem  D mem  Note\n24 Jul 2026  96,996,198  3,428,474  133766  131561   9663   9547  108 (AMD)\n21 Aug 2026  96,994,188  3,426,642          695558          8418  106 (Intel)\n```\n\n is a free, open source (MIT+GPL)\ncompressor tuned to enwik9 by Conrad Lippert-Zajaczkowski, Aug. 25, 2026. It is based\non cmix-lex plus a transformer with externally supplied weights trained on enwik9.\nCompression and decompression require a CUDA compatible GPU. It was tested by the\nauthor on enwik9 (but not enwik8) on a 40 core AMD EPYC 9v84 with 320 GB\nand a GH100 GPU with 94 GB under Linux 24.04 taking 63 hours to compress and 330 hours to decompress\nusing 8 GB of CPU memory and about 80 GB of GPU memory.\n## Notes about compressors\n\n2. Decompression size and time for pkzip 2.0.4.  kzip only compresses.\n\n3. Reported by Ilia Muraviev (author of PX, TC, pimple), June 10-July 18, 2006.  Timed on a P4 3.0 GHz, 1GB RAM, WinXP SP2.\n\n4. enwik9 reported by Johan de Bock, May 19, 2006.  Timed on Intel Pentium-4 2.8 GHz 512KB L2-cache, 1024MB DDR-SDRAM.\n\n5. Compressed with paq8h (VC++ compile) and decompressed with paq-8h (Intel compile of same source code).\n   Normally compression and decompression are the same speed.\n\n6. ocamyd 1.65.final and LTCB 1.0 reported by Mauro Vezzosi, May 30-June 20, 2006.\n   Timed on a 1.91 GHz AMD Athlon XP 2600+, 512 MB, WinXP Pro 2002 SP2 \n   using timer 3.01.  ocamyd 1.66.final reported Feb. 3, 2007.\n   Times are process times.\n\n7. Under development by Mauro Vezzosi, May 24, 2006.\n\n8. Reported by Denis Kyznetsov (author of qazar), June 2, 2006.\n\n9. Reported by sportman, May 24, 2006.  Timed on a Intel Pentium D 830 dual core 3.0GHz, \n   2 x 512MB DDR2-SDRAM PC4300 533Mhz memory timing 4-4-4-12 (833.000KB free),\n   Windows XP Home SP2.  CPU was at 52% so apparently only one of 2 cores was used.\n   Decompression verified on enwik8 only (not timed, about 2.5 hours).\n   WinRK compression options: Model size 800MB,\n   Audio model order: 255,\n   Bit-stream model order: 27,\n   Use text dictionary: Enabled,\n   Fast analyses: Disabled,\n   Fast executable code compression: Disabled\n\n10. Reported by Malcolm Taylor (author of WinRK), May 24, 2006.  \n    Timed on an Athlon X2 4400+ with 2GB, running WinXP 64.  Decompression not tested.\n    decompresser size is based on SFX stub size reported by Artyom (A.A.Z.), Sept. 2, 2007,\n    although it was not tested this way.\n\n11. Reported by sportman, May 25, 2006.  CPU as in note 9.\n\n12. Reported by sportman, May 30, 2006.  CPU as in 9 (50% utilized).\n\n13. xwrt 3.2 options are -2 -b255 -m250 -s -f64.  ppmonstr J options are -o10 -m1650.\n\n14. Reported by Michael A Maniscalco, June 15, 2006.\n\n15. Reported by Jeremiah Gilbert on the Hutter group, Aug. 18, 2006.  Tested under Linux on a dual Xeon\n    1.6 GHz(lv) (overclocked to 2.13 GHz) with 2 GB memory.  Time is user+sys (real=196500 B/ns).\n\n16. Reported by Anthony Williams, Aug. 19-22. 2006.  Timed on a 2.53 GHz Pentium 4 with 512 MB under WinXP Home SP2.\n\n17. Tested Aug. 20, 2006 under Ubuntu Linux 2.6.15 on a 2.2 GHz Athlon-64 with 2 GB memory.  Time is approximate\n    wall time due to disk thrashing.  User+sys time is 153600 ns/byte compress, 148650 decompress.\n\n18. Reported by Dmitry Shkarin (author of durilca4linux), Aug. 22-23, 2006 for durilca4linux_1;\n    and Oct. 16-18, 2006 for durilca4linux_2.  3 GB memory usage is RAM + swap.\n    Tested on AMD Athlon X2 4400+, 2.22 GHz, 2 GB memory under SuSE Linux AMD64 v10.0.\n    durilca4linux_3 reported Feb. 21, 2008 using 4 GB RAM + 1 GB swap. v2 reported Apr. 22, 2008.\n    v3 reported May 22, 2008.\n\n19. enwik8 confirmed by sportman, Sept. 20, 2006.  Compression time 61480 ns/byte timed on a\n    2 x dual core (only one core active) Intel Woodcrest 2GHz with 1333MHz fsb and 4GB 667MHz CL5 memory under\n    SiSoftware Sandra Lite 2007.SP1 (10.105).  Drystone ALU 37,014 MIPS, Whetstone iSSE3 25,393 MFLOPS,\n    Integer x8 iSSE4 220,008 it/s, Floating-point x4 iSSE2 119,227 it/s.\n\n20. Reported by Giorgio Tani (author of PeaZip) on Nov. 10, 2006. Tested on a MacBook Pro, \n    Intel T2500 Core Duo CPU (one core used),\n    with 512 MB memory under WinXP SP2.  Time is combined compression and decompression.\n\n21. enwik9 -8 reported by sportman, Dec. 12-13, 2006.  Hardware as note 19.  enwik9\n    decompression not verified.  paq8hp7 -8 enwik8 compression was reported as 16,417,650\n    (4 bytes longer; the size depends on the length of the input filename, which was\n    enwik8.txt rather than enwik8).\n    I verified enwik8 -7 and -8 decompression.\n\n22. paq8hp8 -8 enwik9 reported by sportman, Jan. 18, 2007.\n    paq8hp10 -8 enwik9 on Apr. 2, 2007.  paq8hp11 -8 enwik9 on May 10, 2007.\n    paq8hp12 -8 enwik8/9 on May 20, 2007.\n    Hardware as in note 19.  Decompression verified for enwik8 only.\n\n23. 7zip 4.46a options were -m0=PPMd:mem=1630m:o=10 -sfx7xCon.sfx\n\n24. paq8o8-intel (intel compile of paq8o8) -1, paq8o8z-jun7 (DOS port of paq8o8) -1\n    reported by Rugxulo on Jun 10, 2008.\n    Timed on a AMD64x2 TK-53 Tyler 1.7 GHz laptop with Vista Home Premium SP1.\n\n25. paq8o8z -1 enwik8 (DJGPP compile) reported by Rugxulo on Jun 17, 2008.\n    Tested on a 2.52 Ghz P4 Northwood, no HTT, WinXP Home SP2.\n\n26. Tested on a Gateway M-7301U laptop with 2.0 GHz dual core Pentium T3200\n    (1MB L2 cache), 3 GB RAM, Vista SP1, 32 bit. Run times are similar to my\n    older computer.\n\n27. enwik9 size reported by Eugene Shelwien, Mar. 5, 2009.\n    enwik8 size and all speeds are tested as in note 26.\n\n28. Reported by Eugene Shelwien on a Q6600, 3.3 GHz, WinXP SP3, ramdrive:\n    bcm 0.06 on Mar. 15, 2009, bcm 0.08 on June 1, 2009.\n\n29. Reported by kaitz (KZ): paq8p3 on Apr. 19, 2009, v2 on Apr. 21, 2009, paq8pxd on Jan. 21, 2012,\n    v2 on Feb. 11, 2012, v3 on Feb. 23, 2012, v4 on Apr. 23, 2012.\n    2012 tests on a Core2Duo T8300 2.4 GHz, 2 GB.\n\n30. Reported by Sami Runsas (author of bwmonstr), July 14, 2009. Tested on an Athlon XP 2200 (Win32).\n\n31. Reported by Dmitry Shkarin, July 21, 2009, Nov. 12, 2009. Tested on a 3.8 GHz Q9650 with 16 GB\n    memory under Windows XP 64bit Pro SP2. Requires msvcr90.dll.\n\n32. Reported by Mike Russell, Sept. 11, 2009.\n    Tested on an 2.93 GHz Intel Q6800 with 3.5 GB memory.\n\n33. Reported by Con Kolivas (author of lrzip) on Nov. 27, 2009 (lrzip 0.40),\n    Nov. 30, 2009 (lrzip 0.42), Mar. 17, 2012 (lrzip 0.612). Tested on a 3 GHz\n    quad core Q9650, 8 GB, 64 bit debian linux.\n\n34. Reported by sportman, Nov. 29, 2009 (durilca'kingsize), Nov. 30, 2009 (durilca'kingsize4),\n    Apr. 8, 2010 (bsc 1.0.0). Test hardware:\n    2 x 2.4GHz (overclocked at 2.53 GHz) quad core Xeon Nahalem,\n    24GB DDR3 1066MHz, 8 x 2TB RAID5, Windows 2008 Server R2 64bit\n\n35. Reported by zody on Dec. 12, 2009. Tested in Windows 7, x64, 3.6 GHz e8200, 4 GB 1066 MHz RAM.\n\n36. Reported by Ilia Muraviev on Dec. 16, 2009. Tested on a 2.40 GHz Core 2 Duo,\n    DDR2-800 4GB RAM, Windows7 x64.\n\n37. Reported by Sami Runsas, Mar. 3, 2010. Tested under Win64 on a Q6600 at 3.0 GHz.\n\n38. Reported by Ilya Grebnov, Apr. 7, 2010. Tested on an Intel Core 2 Duo E8500, 8 GB memory,\n    Windows 7.\n\n39. Reported by Ilya Grebnov, Apr. 8, 2010. Tested on an Intel Core 2 Quad Q9400, 8 GB memory,\n    Windows 7. bsc 2.00 on May 3, 2010. bsc 2.2.0 on June 15, 2010.\n\n40. Reported by Sami Runsas, May 10, 2010. Tested on an overclocked Intel Core i7 860. nanozip 0.08a\n    tested June 6, 2010. nanozip 0.09a on Nov. 5, 2011.\n\n41. lpaq9m reported by Alexander Rhatushnyak on June 9, 2010. Tested on an Intel Core i7 CPU 930\n    (8 core), 2.8 GHz, 2.99 GB RAM. paq8hp12any tested June 28, 2010.\n\n42. Reported by Michal Hajicek, June 4, 2010 on an AMD Phenom II 965, 64 bit Windows.\n    WinRK, ppmonstr on June 14.\n\n43. Reported by Ilia Muraviev, June 26, 2010. Tested on a Core 2 Quad Q9300, 2.50 GHz,\n    4 GB DDR2, Windows 7.\n\n44. Timed on a Dell Latitude E6510 laptop Core I7 M620, 2.66 GHz, 4 GB, Windows 7 32-bit.\n\n45. Reported by Richard Geldreich (lzham author) on Aug. 30, 2010. Tested on a\n    2.6 GHz Core i7 (quad core + HT), 6 GB, Win7 x64.\n\n46. Reported by Stefan Gedo (ST author) on Oct. 14, 2010. Tested on Athlon II X4 635\n    2.9 GHz, 4 GB memory, Windows 7.\n\n47. Reported by David A. Scott on Dec. 15, 2010. Tested on a I3-370 with 6 GB DDR3\n    1033 MHz memory.\n\n48. Timed on a Dell Latitude E6510 laptop Core I7 M620, 2.66 GHz, 4 GB, Ubuntu Linux 64-bit.\n\n49. Tested by the author on a Q9450, 3.52 GHz = 440x8, ramdrive.\n\n50. Tested by the author on an Intel Core i7-2600, 3.4 GHz, Kingston\n    8 GB DDR3, WD VeloicRaptor 10000 RPM 600 GB SATA3, Windows 7 Ultimate SP1.\n\n51. Tested by Bulat Ziganshin on i7-2600, 4.6 GHz with 1600 MHz RAM (8-8-8-21-1T)\n    and NVIDEA GeForce 560Ti at 900/2000 MHz.\n\n52. Tested by Michael Maniscalco on an 8 core Intel Xeon E5620, 2.40 GHz,\n    12 GB memory running Windows 7 Enterprise SP1, 64 bit.\n\n53. Tested by the author on a Core i7-2600K @ 4.6GHz, 8GB DDR3 @ 1866MHz,\n    240GB Corsair Force GT SSD.\n\n54. Tested by Piotr Tarsa on a Core 2 Duo E8400, 8 GiB RAM, Ubuntu 11.10 64-bit,\n    OpenJDK 7.\n\n55. Tested by David Catt on a 64 bit Windows 7 laptop, 2.33 GHz, 4 GB, 4 cores.\n\n56. Reported by the author on a Athlon II X4 635 2.9 GHz, 4GB, Windows 8 Enterprise.\n\n57. Reported by the author on a x86_64 Athlon 64 X2 5200+ with 8 GiB of RAM running GNU/Linux 2.6.38.6-libre.\n\n58. Reported by the author on a 4 GHz i7-930 from ramdrive.\n\n59. [Reported](http://encode.su/threads/1716-LZTURBO?p=33118&viewfull=1#post33118)\n    by the author on a I7-2600, 4.6 GHz, 16 GB RAM, Ubuntu 13.04.\n\n60. Tested by Ilia Muravyov on an Intel Core i7-3770K, 4.8 GHz, 16 GB Corsair Vengeance LP 1800\n    MHz CL9, Corsair Force GS 240 GB SSD, Windows 7 SP1.\n\n61. Tested by Matt Mahoney on a dual Xeon E-2620, 2.0 GHz, 12+12 hyperthreads,\n    64 GB RAM (20 GB usable), Fedora Linux.\n\n62. Tested by Valéry Croizier on a 2.5 GHz Core i5-2520M, 4 GB memory, Windows 7 64 bit.\n\n63. Tested by Ilia Muravyov on an Intel i7-3770, 4.7 GHz, Corsair Vengenance LP 1600 MHz CL9 16 GB RAM,\n    Samsung 840 Pro 512 GB SSD, Windows 7 SP1.\n\n64. Tested by Kennon Conrad on a 3.2 GHz AMD A8-5500.\n\n65. Tested by sportman on an Intel Core i7 4960X 3.6GHz OC at 4.5GHz - 6 core (12 threads) 22nm Ivy Bridge-E,\n    Kingston 8 x 4GB (32GB) DDR3 2400MHz 11-14-14 under clocked at 2000MHz 10-11-11.\n    Windows 8.1 Pro 64-bit, SoftPerfect RAM Disk 3.4.5 64-bit.\n\n66. Tested by Byron Knoll on a Intel Core i7-3770, 31.4 GB memory, Linux Mint 14.\n\n67. Tested by Kennon Conrad on a 4.0 GHz i4790K, 16 GB at 1866 MHz, 128 GB SSD Windows 8.1.\n\n68. Tested by Ilia Muraviev on an Intel Core i7-3770K @ 4.8GHz, 8GB 2133 MHz CL11 DDR3,\n    512GB Samsung 840 Pro SSD, Windows 7 Ultimate SP1.\n69. Tested by Nania Francesco Antonio on a Intel Core i7 920 2.67 ghz 6GB ram.\n\n70. Tested by Richard Geldreich on a Core i7 Gulftown 3.3 Ghz, Win64.\n\n71. Tested by Christoph Diegelmann on a Core i7-4770K, 8 GB DDR3, Samsung 840Pro 128 GB, Fedora 21 64 bit, gcc 4.9.2.\n\n72. Tested by Skymmer on a i7-2770K, WinXP x64 SP2.\n\n73. Tested by Andreas M. Nilsson on a 1.7 GHz Intel Core i7, 8 GB 1600 MHz DDR3, Mac OS X 10.10.3 (14D136).\n\n74. Tested by Michael Crogan on a Core i7-3930K, 3.20 GHz, 6+HT, 64 MB, Linux64.\n\n75. Tested by Mauro Vezzosi on a Core i7-4710HQ 2.50-3.50 GHz, 8 GB DDR3, Windows 8.1 64 bit.\n\n76. Tested by Yann Collet on Core i7-3930K, 4.5 GHz, Linux 64, gcc 5.2.0-5.3.1.\n\n77. Tested by Darek on a Core i7 4900 MQ, 2.8 GHz overclocked to 3.7 GHz, 16 GB, Win7Pro 64.\n\n78. Tested by mpais on a Core i7 5820K 4.4 GHz, Windows 10.\n\n79. Tested by Sportman on2 x Intel Xeon E5-2643 v3 6 cores (12 threads) 3.4GHz, 3.7GHz turbo, 20MB L3 cache,\n      8 x 32GB DDR4 2133MHz CAS 15, SoftPefect RAM Disk 3.4.7, Windows Server 2012 R2 64-bit.\n\n80. Tested by kaitz on an Intel Celeron G1820 DDR3 8GB PC3-12800 (800 MHz).\n\n81. Tested by Darek on Core i7 4900MQ 2.8GHz ovwerclocked to 3.8GHz, 32GB, Win7Pro 64.\n\n82. Tested by Ilia Muraviev on an Intel Core i7-4790K @ 4.6GHz, 32GB @ 1866MHz DDR3 RAM, RAMDisk.\n\n83. Tested by Byron Knoll on an Intel Core i7-7700K, 32 GB DDR4, Ubuntu 16.04-18.04.\n\n84. Tested by Fabrice Bellard on 2 x Xeon E5-2640 v3 @ 2.6 GHz, 196 GB RAM, Linux.\n\n85. Tested by Georgi Marinov on a Windows 10 Laptop: Lenovo Ideapad 310; \n      i5-7200u @2.5GHz; 8GB DDR4 @1066MHz (2133MHz) CL15 CR2T; L2 cache: 2x256KB; L3 cache: 3MB; SSD: Crucial MX500 500GB\n\n86. Tested by Byron Knoll on an Intel Xeon 2.30 GHz, 13 GB, Tesla P100 GPU.\n\n87. Tested by Byron Knoll on an Intel Xeon 2.00 GHz, 13 GB, Tesla V100 GPU.\n\n88. Tested by Fabrice Bellard on an Intel Xeon E3-1230 v6, 3.5 GHz, RTX 3090 GPU.\n\n89. Tested by Matt Mahoney on a Lenovo Intel i7-1165G7 (4 core, 8 thread) 2.80 GHz, 16 GB, Windows 10/Ubuntu 20.04.\n\n90. Tested by Artemiy Margaritov on an Intel Xeon Silver 4114, 2.20 GHz, Ubuntu 18.\n\n91. Tested by Zoltán Gotthardt on an Intel Core i7-8700K @ 3.70GHz, HyperX Fury 32GB 2666MHz DDR4 CL16 (2x16GB kit), Windows 10 Pro 64 bit. The system was not completely idle during the tests.\n\n92. Tested by Darek on a DELL Precision 7730, Intel Core i9-8950HK, 32GB RAM (2400MHz), Windows 10 Pro for Workstations (21H2). The system was not completely idle during the tests.\n\n93. Tested by Sportman on an Intel Core i9 12900KS 16 cores (8 efficient cores disabled, hyper-threading disabled) 3,4GHz, 5.5GHz turbo, 30MB L3 cache, 14MB L2 cache, 2 x 16GB DDR5 6400MHz (PC5-51200) timings 32-39-39-102, Windows 10 Pro 64-bit.\n\n94. Tested by Byron Knoll on an Intel Xeon 2.2 GHz, 83 GB, A100 GPU.\n\n95. Tested by Gerald R. Tamayo on a Dell Inspiron 3881 Intel Core i3-10100 16GB RAM @ 3.60GHz (Windows 10).\n\n96. Tested by Ilya Grebnov on an Intel 9700K CPU (5GHz all cores) with 2x8 GB DDR4 RAM (4133 MHz with 17-17-17-37-400-2T timings) running Microsoft Windows 10 Pro (64 Bit).\n\n97. Tested by Matt Mahoney on a Lenovo Core i7-1165G7 2.80 GHz 16 GB, SSD, Windows 11 or Ubuntu.\n\n98. Tested by Ilia Muraviev on an Intel Core i7-12700K (stock), 32 GB DDR5 5200 MHJz, 1 TB M.2 NVMe SSD.\n\n99. Tested by James Bowery on an AMD Ryzon 7-3700x, 3.6 GHz, 8 cores, 16 threads, 64 GB.\n\n100. Tested by Andrea Barbato on a AMD Ryzen 9 5950X 3.4 GHz 32core processor Patriot Viper Steel RAM DDR4 3600 Mhz 32GB (4x32GB)\n\n101. Tested by James Bowery on Amazon AWS with --machine-type=c2-standard-4 (Geekbench 5 score 790).\n\n102. Tested by kaitz on a Intel Core i5-4460, 3.20 GHz, 32 GB (4x8 GB) PC3-12800 (800 MHz) RAM, Windows.\n\n103. Tested by HyperSoop on an AMD FX 4300 CPU with 6 (4+2) GB of DDR3 RAM, zram + zswap configured. OS is Arch Linux with kernel 6.15.0-rc3\n\n104. Tested by Alexey Simbarsky on an Intel Core i7 9700K / 64 GB RAM / Windows 10 Pro.\n\n105. Tested by Byron Knoll on an AMD EPYC 9B14, 172.9 GB RAM, v6e-1 TPU.\n\n106. Tested by Matt Mahoney on a Lenovo SSD, Core i7-1165G7, 2.80 GHz, 16 GB, Windows 11 (11.8 Gb under WSL2 Ubuntu Linux), Geekbench 5=1499\n\n107. Tested by Ilia Muraviev on a Ryzen 9 9950X, 64GB of DDR5 6000MHz CL30. \n\n108. Tested by James Bowery on a AMD Ryzen 9 5900X, 12 core, 24 thread, 4.95 GHz, 64 GB, Ubuntu 22.04 LTX (Geekbench 5=1648).\n\n109. Tested by Zeelin on a 2 x Intel Xeon Gold 6342 @ 2.80GHz, 96 logical CPUs, 377 GiB RAM, Linux x86_64 + Wine64.\n\n110. Tested by Conrad Lippert-Zajaczkowski on a 40-core AMD EPYC 9V84, 320 GB and a GH100 GPU, 94 GB, Linux 24.04.\n\n111. Tested by Dirk Nesner on a i9-10900K, 62 GB, Ubuntu 24.04. (Geekbench 5=1478 single core).\n\n## About the Compressors\n\n## .0969 fx2-cmix-transformer\n\nfx2-cmix-transformer is a Hutter prize entry submitted as a Linux self extracting archive (archive9)\nby Vladimir Ivanov on July 24, 2026. It is derived from fx2-cmix by replacing the LSTM model with\na transformer with 6M parameters pre-trained on enwik9 on 8 RTX 5090 GPUs for 26 hours, adding 2.9 MB\nto the compressor and decompressor. It does not need a GPU to run.\nThis version was successfully tested on an AMD Ryzen 7-3700 but gave\na segmentation fault on an Intel Core-i7 1165G7.[archive9](https://drive.google.com/file/d/1JsmbKfdxmq0bv7YE7ZKtzpHPVbottFly/view?usp=drive_link) Linux self extracting archive\n\n[compressor](https://drive.google.com/file/d/1bZ957n049YTJgzD3LQUvooEZn9_ISLDo/view?usp=drive_link) Linux 64 bit executable\n\n[source code](https://drive.google.com/file/d/11Vljjo9pAY7nPgSDa-apjnltRO2bWh-4/view?usp=drive_link)\n\n[documentation](https://drive.google.com/file/d/1iFjIvcjAs1zU51nGPQgpzxXrwsPhw1_r/view?usp=drive_link).\n\n[archive9](https://drive.google.com/file/d/1-y6FveFx-yiAo7hPaeSbPzjvdq2VFOY_/view?usp=sharing)\n\n[compressor](https://drive.google.com/file/d/1ZWsVvu4dCo7r_3ZP3RgLzUsXeKDvz8_g/view?usp=drive_link)\n\n[source code](https://drive.google.com/file/d/1vuKsfrrxF6eJAArxsi8WHehzBk4EvCRV/view?usp=drive_link)\n\n[documentation](https://drive.google.com/file/d/17c8hMeitD2kdJpxOtoNCQRmIwQeG9qKd/view?usp=drive_link).\n\n[archive9](https://github.com/astOwOlfo/fx2-cmix-transformer-v1/releases/download/binaries/archive9)\n\n[compressor](https://github.com/astOwOlfo/fx2-cmix-transformer-v1/releases/download/binaries/cmix)\n\n[source code](https://github.com/astOwOlfo/fx2-cmix-transformer-v1/)\n\n[documentation](https://github.com/astOwOlfo/fx2-cmix-transformer-v1/blob/main/writeup.md)\n\n## .1069 altxs\n\n[altxs](https://github.com/altvectra/altxs) 1.0.0\n\n```\nVersion                  enwik9     decompressor  total size  Comp   Decomp  Mem   Alg Note\naltxs 1.0.0 25 Aug 2026  93,434,410 13,490,401 xd 106,924,811 227372 1189976 88000 Tr 110\n./preprocess c out.words enwik9 out.pre 16384 512\n./nncp -n_layer 7 -hidden_size 384 -n_embed_out 5 -n_symb 16388 -full_connect 1 -lr 6e-3 c out.pre out.bin\n```\n\nVersion 2019-11-16 was released Nov. 16, 2019. It was run in 8 threads.\n\nVersion 2 was released Jan. 3, 2021. It uses a\n[transformer](<https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)>)\narchitecture, a recurrent neural network with attention mechanism to allow parallelism.\n The algorithm\nis described briefly [here](https://bellard.org/nncp/nncp_v2.pdf).\nIt uses the same dictionary preprocessing as earlier versions.\nIt was tested with an\n[Intel Xeon E3-1230 v6](https://ark.intel.com/content/www/us/en/ark/products/97474/intel-xeon-processor-e3-1230-v6-8m-cache-3-50-ghz.html) at 3.5 GHz and a\n[Geforce RTX 3090 GPU](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090/) with 10,496 Cuda cores and 24 GB RAM.\n\nnncp v2.1 was released Feb. 6, 2021. It is the same code as v2 except for a larger model and slightly different hyperparameters.\n\nnncp v3 was released Apr. 24, 2021. This new version is coded in C and supports recent NVIDIA GPUs. It is much faster (3x) due to algorithmic improvements and requires less memory. The Transformer model is similar (199M parameters) but the hyperparameters have been tuned.\n\nnncp v3.1 was released June 1, 2021.\n\nnncp v3.2 was released Oct. 23, 2023.\n\n```\n            Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem  Alg Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  ----- --- -----\nnncp 2019-05-08          16,791,077  125,623,896    161,133 xd 125,785,029  420168 602409   2040 LSTM 84\nnncp 2019-11-16          16,292,774  119,167,224    238,452 xd 119,405,676  826048 1156467  5360 LSTM 84\nnncp v2                  15,600,675  114,317,255     99,671 xd 114,317,255  308645 313468  17000 Transformer 88     \nnncp v2.1                15,020,691  112,219,309    100,046 xd 112,319,355  508332 515401  23000 Transformer 88\nnncp v3                  15,206,966  110,034,293    197,491 xd 110,231,784  161812 158982   6000 Transformer 88\nnncp v3.1                14,969,569  108,378,032    201,620 xd 108,579,652  212766 210970   6000 Transformer 88\nnncp v3.2                14,915,298  106,632,363    628,955 xd 107,261,318  241871 238670   7600 Transformer 88\n./cmix -e enwik9 archive9\n./archive9\n```\n\nfx-deepmix is derived from cmix-lex with the following changes:\n\n```\nVersion                  archive9     C size   C time  D time  D mem  Note\nfx-deepmix 28 Jul 2026   107,828,411  458,116          301180  10233  106\nVersion                  archive9     C size  C time  D time  Mem (MB) Note\ncmix-obias 19 Jul 2026   108,009,834  459,989 121900  119920  9836     108 (AMD)\ncmix-obias 11 Aug 2026   108,010,938                  228793  9861     106 (Intel)\n```\n\ncmix v2 was released May 29, 2014.\n\ncmix v3 was released June 27, 2014.\n\ncmix v4 was released July 22, 2014. It uses 28,976,428 KiB memory (29.7 GB).\n\ncmix v5 was released Aug. 13, 2014. The decompressor size is a zip archive containing the source code, makefile, and a dictionary compressed with cmix from 465211 to 90065 bytes.\n\ncmix v6 was released Sept. 3, 2014. The decompressor size includes the dictionary compressed with cmix from 465211 to 90207 bytes.\n\ncmix v7 was released Feb. 4, 2015.\n\ncmix v8 was released Nov. 10, 2015.\n\ncmix v9 was released Apr. 8, 2016.\n\ncmix v10 was released June 17, 2016.\n\ncmix v11 was released July 3, 2016. It incorporates a modification originally developed by Eugene Shelwien in which PPMd is included as a model.\n\ncmix v12 was released Nov. 7, 2016. It includes a LSTM model.\n\ncmix v13 was released Apr. 24, 2017.\n\ncmix v14 was released Nov. 22, 2017.\n\ncmix v15 was released May 19, 2018.\n\ncmix v16 was released Oct 6, 2018.\n\ncmix v17 was released Mar. 24, 2019.\n\ncmix v18 was released Aug. 2, 2019.\n\ncmix v19 was released Aug. 29, 2021. It has improvements based on the startlit (article reordering) and cmix-hp Hutter prize entries. It has a separate decompressor.\n\ncmix v20 was released Nov. 5, 2023.\n\ncmix v21 was released Sept. 17, 2024.\n\n```\n            Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem   Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  -----  -----\ncmix v1                  16,076,381  128,647,538    279,185 x  128,926,723  181924 179706  20785  66\ncmix v2                  15,863,623  126,323,656    310,068 x  126,633,724  580083 577626  28152  66\ncmix v3                  15,809,519  125,971,560    274,992 x  126,246,552  267978 266622  26681  66\ncmix v4                  15,784,946  125,621,620    278,375 x  125,899,995  284243 282390  28976  66\ncmix v5                  15,769,367  125,526,628    163,552 s  125,690,180  282056 282647  28865  66\ncmix v6                  15,738,922  124,172,611    161,908 s  124,334,519  280749 282137  30882  66\ncmix v7                  15,738,825  124,168,463    166,785 s  124,335,248  280416 280904  30600  66\ncmix v8                  15,709,216  123,930,173    164,882 s  124,095,055  344244 346641  30311  66\ncmix v9                  15,627,536  123,874,398    161,911 s  124,036,309  346436 345681  26929  66\ncmix v10                 15,587,868  123,257,156    164,263 s  123,421,419  355721 355850  29924  66\ncmix v11                 15,566,358  122,977,954    172,261 s  123,150,215  377529 374440  27745  66\ncmix v12                 15,440,186  121,718,424    175,953 s  121,894,377  571339 574522  27865  66\ncmix v13                 15,323,969  120,480,684    177,979 s  120,658,664  617346 615987  27803  66\ncmix v14                 15,210,458  119,017,492    203,717 s  119,221,209  631838 627802  28287  83\ncmix v15                 15,111,677  117,959,016    217,830 s  118,176,846  650055 651716  28365  83\ncmix v16                 14,955,482  116,912,035    226,121 s  117,138,156  613898 658679  27708  83\ncmix v17                 14,877,373  116,394,271    208,263 s  116,602,534  641189 645651  25258  83\ncmix v18                 14,838,332  115,714,367    208,961 s  115,923,328  602867 601569  25738  83\ncmix v19                 14,837,987  111,470,932    223,485 sd 111,694,417  605110 601825  25528  83\ncmix v20                 14,760,552  109,877,715    241,725 sd 110,119,440  621780 619024  31650  83\ncmix v21       -t        14,623,723  107,963,380    281,387 sd 108,244,767  622949 638442  30950  83\n```\n\n[forge-cmix-v2](https://github.com/dirknesner/forge-cmix/releases/tag/submission-s20-2026-08-29)\nwas released Aug. 29, 2026.\n\n```\nVersion                     archive9     C size   C time   D time  D mem  Note\nforge-cmix v1  27 Jul 2026  109,079,791  462,290           269605  8356   106 (Core i7 1165G7, 2.80 GHz, Ubuntu 24.04, 12 GB under WSL2).\nforge-cmix v2  29 Aug 2026  109,079,343  461,842           156219  8812   111 (pinned to 1 core)\n                                                           182008  8505   111 (1 core, cgroup 12 GB limit)\n```\n\nIn a separate test I used cmix-lex as compiled for the official submission to compress enwik8. This requires the dictionary english.dic from the source code to compress and decompress. The cmix executable is compressed with UPX to 459,938 bytes. english.dic is a plain text list of words of size 411,966 bytes. Both files compress together to a zip -9 archive of size 635,437 bytes. I tested enwik9 decompression only from the official archive9, which does not require english.dic and is slightly smaller than the AMD compile and required 73.75 hours (182,561 s user, 18,200 sys, at 75% CPU) in Ubuntu with 12 GB memory available.\n\nThe official size for the Hutter prize is 109,201,040 (archive9) + 470,599 (compressor) = 109,671,639.\n\n```\n            Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem   Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  -----  -----\ncmix   -e                           109,201,040         0 xd   109,201,040  145029 147029  9993   108\ncmix   -c english.dic   14,952,093                 635437 x                 165441 165963  8998   106\narchive9                            109,190,109         0 xd   109,190,109         265512  9407   106\n```\n\n[fx2-cmix](https://github.com/kaitz/fx2-cmix)\n([discussion](<https://encode.su/threads/4308-fx2-cmix-(HP)>))\nis an update to fx-cmix and a [Hutter Prize winner](http://prize.hutter1.net/)\nby Byron Knoll and Kaido Orav (Kaitz), submitted Aug. 11, 2024 and accepted Oct. 8, 2024\nwith a total size of 110,351,665 (self extracting archive) + 441,468 (compressor) = 110,793,128.\nChanges from fx-cmix are described in the\n[readme](https://github.com/kaitz/fx2-cmix/blob/main/README.md) file.\n\n```\n            Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem  Alg Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  ----- --- -----\nfx-cmix                              112,142,259          0 xd 112,142,259         216836   8869  CM   97\nfx2-cmix                             110,351,665          0 xd 110,351,665         272072   8811  CM   97\n                                     110,351,665          0 xd 110,351,665         289079   9522  CM   101\n```\n\nv2 was released Sept. 7, 2020. It runs on a V100 GPU using\n2669 MB CPU RAM and 15621 MB GPU RAM. The decompressor contains\na colab notebook, NNCP preprocessor source code and makefile,\nand a dictionary created by the NNCP preprocessor.\n v3 was released Nov. 29, 2020. It uses 3252 MiB of CPU RAM\non a 2.00 GHz Xeon and 15621 MiB of GPU RAM on a Tesla V100.\n v4 was released Aug. 10, 2022. It uses 5696 MiB of CPU RAM\nand 39664 GPU RAM on an Intel Xeon 2.2 GHz, 83 GB RAM, A100 GPU.\n \n\n```\nProgram                   enwik8      enwik9       Prog      Total      Comp  Deco   Mem   Note\n---------               ----------  -----------  --------  ---------    ----  ----   ----  ----\ntensorflow-compress v1  20,119,747  159,716,240  88,870sd  159,805,110  72260 82259  25663 86\ntensorflow-compress v2  16,828,585  127,146,379 175,047sd  127,321,426 157196 142820 18290 87\ntensorflow-compress v3  16,128,954  118,938,744  54,597sd  118,993,341 300104 300408 18873 87\ntensorflow-compress v4  15,905,037  113,542,413  55,283sd  113,597,696 291394 290803 45360 94\n```\n\n cmix-hp v2 was released Aug. 1, 2021.\n cmix-hp v3 was released Aug. 9, 2021.\n \n\n```\n            Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem  Alg Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  ----- --- -----\ncmix-hp v1               15,957,339  113,712,798          0 xd 113,712,798  189420 194280  6873  CM  89\ncmix-hp v2               15,221,487  113,816,319          0 xd 113,816,319  198900         6720  CM  89\ncmix-hp v3                           113,788,598          0 xd 113,788,598  188460 188040  6693  CM  89\n```\n\n starlit compresses by first reordering the articles in enwik9 to maximize mutual information\nbetween consecutive articles, then uses the dictionary preprocessor from phda9 and compresses\nusing a reduced version of cmix to decrease memory usage from 32 GB to 10 GB and increase speed.\nThe compressor is built from the supplied bash scripts\nby compiling with clang++-12 in Linux with different parts optimized for size or speed.\nThen the dictionary and article order list (both text files) are compressed with the newly\ncreated cmix and appended to the executable. The size is 124,984 bytes before appending and\n401,505 bytes afterward. (A precompiled cmix is supplied optimized for an AMD Zen 2 with size 114,012\nbytes before appending, which I did not use). The new executable then compresses enwik9 by\nextracting the compressed article order and dictionary and an additional 17 GB of temporary\nfiles to produce an executable file named archive9. To decompress, archive9 is run, which extracts\nthe dictionary, article order list, and 17 GB of temporary files, and 2 days later, the\noutput as a file named enwik9_uncompressed. No other files are required to decompress. The\noriginal article order is restored by sorting the titles alphabetically.\n To compress enwik8, the command is  Modifications to cmix_v18 (from README.md).\n Changes to HP-2017 (phda9) enwik8-specific transforms \n \n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp   Mem Alg Note\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----   --- --- ----\nstarlit 10 May 2021                                       115,093,300          0 xd 115,093,300        273600  9910 CM  90\nstarlit 31 May 2021                           15,215,107  114,951,433          0 xd 114,951,433 173953 171682 10233 CM  89\n```\n\n[jax-compress](https://github.com/byronknoll/jax-compress) is a free, open source experimental\nfile compressor by Byron Knoll, Mar. 15, 2026. It uses a neural network architecture running\non a TPU based on a port of tensorflow-compress with article reordering from fx2-cmix.\nIt can be run on Google Colabs through a web interface. It was tested on an\nAMD EPYC 9B14 with 172.9 GB RAM and a v6e-1 TPU, taking 31 hours to compress or decompress using 41.9 GB memory.\n## .1135 tensorflow-compress\n\n[tensorflow-compress](https://github.com/byronknoll/tensorflow-compress) v1 is a free, open source experimental file\ncompressor by Byron Knoll, July 20, 2020. It uses a LSTM neural network\naccelerated by a GPU if available. It uses a dictionary and preprocessor\nfrom NNCP by default, or from cmix. The test results for v1 use the\ndefault settings and were tested by the author on an Intel Xeon 2.30 GHz,\n13 GB RAM with a Tesla P100 GPU. It uses 10138 MiB CPU RAM and 15525 MiB\nGPU RAM. It is run as a Colab notebook.\n## .1137 cmix-hp\n\n[cmix-hp](https://github.com/byronknoll/cmix-hp)\n[(mirror)](cmix-hp.zip) is a Hutter prize submission by Byron Knoll, June 10, 2021.\nIt is a simple modification\nto startlit (May 31 2021 submission) to enlarge the PPMD model and map it to 21.4 GB virtual memory.\nto meet the Hutter prize requirement of using at most 10 GB RAM and 100 GB disk. It uses 94% CPU\non the SSD swapping 45 GB.\n## .1137 fast-cmix\n\nfast-cmix-hp\n[(source code)](https://drive.google.com/drive/folders/15ChUkq7RmmoShSn_xD9K2ZRLIHK7GfIP)\n[(Linux self extracting archive)](fastcmix.archive9) is\na free, open source Hutter prize entry by Saurabh Kumar, Apr. 20, 2023. It is a speed optimization of\ncmix-hp and starlit. It is a self extracting archive (archive9) for 64 bit Linux, size 113,746,218 bytes,\nthat creates enwik9. Compression was not tested.\nExtraction time was 190,507 s at 88% CPU on a Lenovo 82HT Core i7-1165G7, 2.80 GHz, 4 cores, 8 threads, 16 GB under Ubuntu in Windows 11/WSL.\nExtraction time was 121,971 s at 99% CPU on an AMD Ryzon 7-3700x, 3.6 GHZ, 8 cores, 16 threads, 64 GB.\nIn both cases, it runs in 1 thread.\n## .1149 starlit\n\n[starlit](https://github.com/amargaritov/starlit) is a Hutter prize submission by\nArtemiy Margaritov on May 10, 2021, updated May 31, 2021. It is a free, open source, Linux compressor\nthat produces a self extracting archive for enwik9 as a special case. It satisfies the Hutter prize\nrules of using less than 10 GiB of memory (the figure shown is in 1000 KiB), and 20 GB of disk space\nand compressing and decompressing in less than 50,000/(geekbench 5 score) hours each. I tested on a\nLenovo Intel Core i7-1165G7, 2.80 GHz, 16 GB (geekbench 5 = 1427 single thread, 4667 multithreaded)\nin an Ubuntu 20.04 shell window under Windows 10 with the screen/sleep saver and WiFi turned off\nfor 2 days each to compress and decompress.\n`cmix -e .dict enwik8 enwik8.cmix`. To decompress:\n`cmix -d .dict enwik8.cmix enwik8_uncompressed`, where `.dict` is the\nuncompressed dictionary file. Articles are not reordered. `cmix` is the reduced\ncmix with or without the appended compressed files.\n\n## .1165 phda9\n\n[phda](http://qlic.altervista.org/phda9.zip) 1.0\n[(discussion)](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=55210&viewfull=1#post55210)\nis the public version of a winning Hutter prize submission\ndated Dec. 15, 2017 by Alexander Rhatushnyak. There are Windows and Linux\nexecutables, no source.\n\nThe [original prize winning version](HP_2017_October.rar)\nis a 64 bit Linux decompressor (no source) and compressed enwik8 as a RAR archive,\nawarded Nov. 4, 2017, posted Aug. 12, 2019.\nArchive plus decompressor size is 15,284,944 bytes. It uses 1 GB memory\nand a 176 MB scratch file. There is a version that uses only RAM.\n\n[phda9 1.2](http://qlic.altervista.org/phda9.zip)\n[(discussion)](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=56332&viewfull=1#post56332) was released Mar. 13, 2018.\n\n[phda9 1.3](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=56588&viewfull=1#post56588) was released Apr. 21, 2018. The decompressor size for enwik8\nis different (557050 bytes) because the dictionary is loosely compressed in the\ndecompressor instead of in the compressed file.\n\n[phda9 1.4](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=56819&viewfull=1#post56819) was released May 20, 2018. This is mainly a bug fix version.\n\n[phda9 1.5](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=57502&viewfull=1#post57502) was released Aug. 1, 2018. enwik8 uses a separate decompressor\nwith a size of 557415 bytes.\n\n[phda9 1.6](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=58505&viewfull=1#post58505) was released Oct. 20, 2018. enwik8 uses a separate decompressor\nwith a size of 564616 bytes.\n\n[phda9 1.7](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=59310&viewfull=1#post59310) was released Feb. 18, 2019. enwik8 uses a separate decompressor\nwith a size of 565,352 bytes.\n\n[phda9 1.8](https://encode.su/threads/2858-Hutter-Prize-4-17-improvement-is-here?p=60777&viewfull=1#post60777) was released July 4, 2019. enwik8 uses a separate decompressor\nwith a size of 558,298 bytes.\n\n```\n            Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem   Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  -----  -----\nphda9  1.0               15,173,565  118,658,060     41,994 xd 118,700,054   56815  55201   5031  83\nphda9  1.2               15,144,786  118,335,817     42,745 xd 118,378,562   60726  61586   4992  83\nphda9  1.3               15,069,752  117,617,185     42,108 xd 117,659,293   86557  87375   4996  83\nphda9  1.4               15,074,624  117,603,125     42,110 xd 117,645,235   87520  87909   4992  83\nphda9  1.5               15,063,267  117,223,130     42,428 xd 117,265,558   85877  86365   4995  83\nphda9  1.6               15,040,647  117,039,346     41,911 xd 117,081,257   84713  88401   4996  83\nphda9  1.7               15,023,870  116,940,874     43,274 xd 116,984,148   83712  87596   4996  83\nphda9  1.8               15,010,414  116,544,849     42,944 xd 116,587,793   86182  86305   6319  83\nCompression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp   Decomp   Mem   Notes\n-------       -------    ----------  -----------  -----------  -----------  ------ ------  -----  -----\ngmix v1                  16,246,629  122,336,013    182,085 sd 122,518,098  73986   70552  3751 CM  83\n```\n\npaq8px_v206fix1 is the latest versions in the following PAQ series of open source (GPL) context mixing archivers.\n\np5, p6, and p12 (Matt Mahoney, May 13, 2000) use a neural network\nwith 256K or 4M inputs, no hidden layer and a single output to predict \nthe next bit of input, \ngiven hashes of various contexts to select active inputs.  The output\nis arithmetic coded.  p5 uses 1 MB memory\nand context orders 0 to 3.  p6 uses 16 MB and orders 0-5.  p12\nuses 16 MB, orders 1-4 and word-level orders 0-1 as an optimization\nfor text.  The programs take no options.  The algorithm is described in\nM. Mahoney,\n[*Fast Text Compression with Neural Networks*](mmahoney00.pdf), Proc. AAAI FLAIRS, Orlando, 2000\n(C) 2000, AAAI.\n\npaq1 (Matt Mahoney, Jan. 6, 2001) replaces the neural network in p5, p6, p12 \nwith a fixed weighted averaging\nof model outputs.  Described in an unpublished report, M. Mahoney,\n[*The PAQ1\nData Compression Program*](paq1.pdf), 2002.\n\npaq6 (Matt Mahoney and Serge Osnach, Dec. 30, 2003) evolved as a series\nof improvements to paq1.  It is described in\nM. Mahoney, \n[*Adaptive Weighing of Context Models for Lossless Data Compression*](cs200516.pdf), \nFlorida Tech. Technical Report CS-2005-16, 2005.  The most significant\nimprovements are replacing the fixed model weights with adaptive linear\nmixing (Matt Mahoney), and SSE (secondary symbol estimation) postprocessing\non the output probability, and modeling of sparse contexts (Serge Osnach).\nOther models were added for x86 executable code, and automatic detection\nof fixed length records in binary data. Intermediate versions can be found\n[here](paq.html).\n\npaqar 4.5 (Alexander Rhatushnyak, Feb. 13, 2006)\nis the last of a long series of improvements to paq6 by\nAlexander Rhatushnyak (paqar: multimixer model, .exe preprocessor, other model\nimprovements), Przemyslaw Skibinski (WRT text preprocessing), Berto Destasio (model tuning),\nFabio Buffoni (speed optimizations), David. A Scott\n(arithmetic coder optimizations), Jason Schmidt (model improvements), and\nJohan de Bock (compiler optimizations).  For text, the biggest improvement was from\nWRT (Word Reducing Transform), \nwhich replaces words with shorter codes from an external English dictionary\nto PAsQDa 1.0 on Jan. 18, 2005.\nWRT is described in \nP. Skibiński, Sz. Grabowski, and S. Deorowicz, \n[*Revisiting\ndictionary-based compression*](http://www.ii.uni.wroc.pl/~inikep/papers/05-RevisitingDictCompr.pdf), Software - Practice & Experience, 35 (15), \npp. 1455-1476, December 2005.\nThere were a great number of versions by many contributors, mostly in 2004 when the PAQ \nseries moved to the top of most compression benchmarks and attracted interest.\nPrior to PAQ, the top ranked programs were generally closed source.\n\n[paq8f](paq8f.zip) (Matt Mahoney, Feb. 28, 2006) evolved from paq7 (Dec. 24, 2005) as a\ncomplete rewrite of paq6/paqar.  The important improvements were replacing the\nadaptive linear mixing of models with a neural network (coded in MMX assembler),\na more memory-efficient mapping of contexts to bit histories using a cache-aligned\nhash table, adaptive mapping of bit histories to probabilities,\nand models for bmp, tiff, and jpeg images.  It models text using whole-word\ncontexts and case folding, like all versions back to p12, but lacks WRT text\npreprocessing.  It served as a baseline for the Hutter prize.  Details are\nin the source code comments.\n\npaq8g (Przemyslaw Skibinski, Mar. 3, 2006) adds back WRT text preprocessing.\n\npaq8h (Alexander Rhatushnyak, Mar. 24, 2006) added additional contexts to the neural network mixer. It was top ranked on enwik9 (but not enwik8) when the Hutter prize was launched on Aug. 6, 2006. This is the 78'th version since p5.\n\n[raq8g](http://cilibrar.com/projsup/hutter/) by Rudi Cilibrasi,\nreleased 0721Z Aug. 16, 2006, is a modification of paq8f.  It adds\na NestModel to model nesting of parenthesis and brackets.\nThe test below for -7 is based on\na Windows compile, [raq8g.exe](raq8g.exe).\nThe test for -8 was under Linux.  The unzipped Linux executable is 27,660 bytes.\n\n[paq8j](http://ilovemyking.googlepages.com/paqpage) by Bill Pettis,\nNov. 13, 2006, is based on paq8f (no dictionary) with model improvements taken\nfrom paq8hp5.  It is a general purpose compressor like paq8f, not specialized for text.\n\n[paq8ja.zip](paq8ja.zip) by Serge Osnach, Nov. 16, 2006, is an improvement\nof paq8j, using additional contexts based on character classifications.\n\n[paq8jb.zip](paq8jb.zip) by Serge Osnach, Nov. 22, 2006, adds\ncontexts using the distance to an anchor byte (x00, space, newline, xff)\ncombined with previous characters.  \nThe -8 test caused some minor disk thrashing at 2 GB\nmemory under WinXP Home (82% CPU usage).  Time reported is wall time.\n\n[paq8jc.zip](paq8jc.zip) by Serge Osnach, Nov. 28, 2006, improves the\nrecord model for better compression of some binary files, although it is\nslightly worse for text.  Time for -8 is wall time at 72% CPU usage.\n\n[paq8jd](paq8jd.zip) by\n[Bill Pettis](http://ilovemyking.googlepages.com/paqpage),\nDec. 30, 2006, improves on paq8j with additional SSE (APM) stages.\nenwik8 -8 caused some disk thrashing at 2 GB memory.\n\n[paq8k](http://ilovemyking.googlepages.com/paq8k.zip) is by\nBill Pettis, Feb. 13, 2007.\n\n[paq8l](paq8l.zip) by\nMatt Mahoney, Mar. 8, 2007, is based on paq8jd.  It adds a DMC model\nand minor improvements.\n\n[paq8fthis2](paq8fthis2.zip)\nby Jan Ondrus, Aug. 12, 2007, is paq8f with an improved model for compressing JPEG\nimages.  It is otherwise archive compatible with paq8f for data without JPEG images (such as\nenwik8 and enwik9).\n\n[paq8n](paq8n.zip) by Matt Mahoney,\nAug. 18, 2007, combines paq8l with the JPEG model from paq8fthis2.\n\n[paq8o and paq8osse](paq8o.zip) by\nAndreas Morphis, Aug 22 2007, is paq8n with an improved model for .bmp images.\nThere are two executables that produce identical archives.  paq8o.exe is for\nPentium MMX or higher.  paq8osse.exe is for newer processors that support SSE2 instructions\nlike the Pentium 4.  It is about 8% faster, but uses more memory.\nBoth use the same C++ source but use\ndifferent (but equivalent) assembler code to implement the neural network mixer.\npaq8osse.exe was compiled with Intel C++, which produces slightly faster executables than\ng++ used in earlier versions.  The current version is\n[paq8o ver. 2](paq8o2.zip) (Aug. 24, 2007),\nwhich fixes the file name extension (was .paq8n) but does not change compression.\nThe benchmark is based on the first version.\n\n[paq8o3](paq8o3.zip) by KZ, Sept. 11, 2007, \ncombines paq8o with an improved JPEG model from paq8fthis3 (Jan Ondrus, Sept. 8, 2007)\nand an improved model for grayscale PGM images from paq8i\n(Pavel Holoborodko, Aug. 18, 2006).  Text compression is unchanged from paq8l, paq8m,\npaq8o, or paq8o2.\n\n[paq8o4 v1](http://code.google.com/p/paq8/source) by KZ, Sept. 15, 2007,\nincludes a grayscale .bmp model (based on the grayscale PGM model).  Text compression\nis unaffected.  It was compiled with Intel C++.\n[paq8o4 v2](paq8o4v2.zip) by\nMatt Mahoney, Sept. 17, 2007,\nis a port to g++ which allows wildcards, directory traversal, and directory creation,\nbut is 8% slower.  It is archive compatible with v1.\n\n[paq8o6](paq8o6.zip) by KZ,\nSept. 28, 2007, is based\non [paq8o5](paq8o5.zip) by KZ,\nSept. 21, 2007 with the improved JPEG model from \n[paq8fthis4](paq8fthis4.zip)\nby Jan Ondrus, Sept. 27, 2007.  paq8o5 is paq8o4 with an improved StateMap\nfrom lpaq1.  The improved compression of enwik8 comes from this StateMap.\nCompression of enwik8 is unchanged from paq8o5 to paq8o6.\n\n[paq8o7](paq8o7.zip) by\nKZ, Oct. 16, 2007, improves paq8o6 with improved JPEG compression and support\nfor 4 and 8 bit BMP images.  Text is not affected.\n\n[paq8o8](paq8o8.zip) by\nKZ, Oct. 23, 2007, improves paq8o7 with improved JPEG compression further.\n\n[paq8o8-jun7](paq8o8z-jun7.zip)\nis a DOS port of paq8o8 by Rugxulo, June 7, 2008.\n\n[paq8o10t](paq8o10t.zip)\nis by KZ, June 11, 2008.\n[Discussion](http://www.encode.su/forum/showthread.php?t=81).\n\n[paq8p3](http://www.encode.su/forum/showthread.php?p=6440) is\nby KZ, Apr. 19, 2009.\n\n[paq8p3 v2](http://www.encode.su/forum/showthread.php?p=6440) is\nby KZ, Apr. 21, 2009.\n\n[paq8px_v60_turbo](http://www.encode.su/forum/attachment.php?attachmentid=909&d=1247346512)\n[(source code and discussion)](http://www.encode.su/forum/showthread.php?t=342)\nwas by Jan Ondrus (with contributions from many others),\nJune 20, 2009, and speed optimized by LovePimple on July 11, 2009.\nBy default the turbo version\nruns in high priority under Windows, but was tested at normal priority.\nThe v60 version was released after a long period of development beginning with v1\non Apr. 25, 2009. Development was aimed mostly at improving x86,\nimage and wav compression. Decompression was not verified.\n\n[paq8px_v69](http://dhost.info/paq8/) was released\nApr. 26, 2010.\n\n[paq8pxd](http://encode.su/threads/1464-Paq8pxd-dict) by kaitz, Jan. 21, 2012,\nmodifies paq8px_v69 by adding dynamic dictionary preprocessing\n(based on XWRT), UTF-8 detection, and an alternating byte sparse model.\n\n[paq8pxd_v2](http://encode.su/threads/1464-Paq8pxd-dict?p=28321#post28321) by kaitz (KZo) was released Feb. 11, 2012.\n\n[paq8pxd_v3](http://encode.su/attachment.php?attachmentid=1855&d=1330038239) by kaitz (KZo) was released Feb. 23, 2012. Modified im8model,\nbase64 in email model, and fixes false image detection in enwik9.\n\n[paq8pxd_v4](http://encode.su/attachment.php?attachmentid=1930&d=1334855042) by kaitz was released Apr. 19, 2012. Adds 4 bit bmp model,\nbase64 fixes, combines WRT source code and has other fixes.\n\n[paq8pxd_v5](http://encode.su/attachment.php?attachmentid=2281&d=1366302563) by kaitz was released Apr. 18, 2013.\n\n[paq8pxd_v7](http://encode.su/threads/1464-Paq8pxd-dict) by kaitz was released Aug. 14, 2013.\n\n[paq8pxd_v8](http://encode.su/threads/1464-Paq8pxd-dict?p=38824&viewfull=1#post38824) by kaitz was a temporary release on June 16, 2014.\nIt was still under development to fix bugs causing it to fail\non JPEG and WAV input, but there were no errors for enwik8 or enwik9.\nTo test, it was compiled from source under 64 bit Ubuntu using g++ 4.8.1 -O3.\n\n[paq8pxd_v10fix](http://encode.su/threads/1464-Paq8pxd-dict?p=38948&viewfull=1#post38948) by kaitz was released June 21, 2014. It was compiled from source\nunder 64 bit Ubuntu, g++ 4.8.1 -O3.\n\n[paq8pxd_v12](http://encode.su/threads/1464-Paq8pxd-dict?p=39668#post39668) by kaitz was released July 28, 2014. It was compiled from source\nunder 64 bit Ubuntu, g++ 4.8.1 -O3.\n\n[paq8pxd_v12-skbuild](http://encode.su/threads/1464-Paq8pxd-dict?p=39814&viewfull=1#post39814), Aug. 9, 2014,\nis a 64 bit port of paq8pxd_v12 by Skymmer with work by AlexDoro adding\noptions -9 and -10, each of which doubles memory usage from the previous level.\n\n[paq8pxd_v13_x64](http://encode.su/threads/1464-Paq8pxd-dict?p=40113&viewfull=1#post40113) is the 64 bit compile by Skymmer of\n[paq8pxd_v13fix3](http://encode.su/threads/1464-Paq8pxd-dict?p=40109&viewfull=1#post40109) by kaitz on Aug. 26, 2014. It supports levels up to 15\nusing 25955 MB memory.\n\n[paq8pxd_v15](http://encode.su/threads/1464-Paq8pxd-dict?p=40523&viewfull=1#post40523) was released Sept. 17, 2014. It has options -s1...-s15 and -f1...-f15\nwhich mean slow or fast respectively. Higher levels use more memory. Faster methods\nuse fewer models. Levels 9 and higher require a 64 bit compile. To test, the\nprogram was compiled with g++ 4.8.2 for 64 bit Ubuntu with option -O3.\n\n[paq8pxd_v12_biondivers1_x64](http://encode.su/threads/1464-Paq8pxd-dict?p=40972&viewfull=1#post40972) is a 64 bit build of v12 by Luca Biondi,\nOct. 27, 2014.\n\n[paq8pxd_v18](http://encode.su/threads/1464-Paq8pxd-dict?p=49033&viewfull=1#post49033) by kaitz was released July 18, 2016. Options -{qfs} select quick,\nfast, slow, followed by a number selecting memory usage.\n\n[paq8px_v77](https://encode.su/threads/342-paq8px?p=53244&viewfull=1#post53244) was released July 10, 2017.\n\n[paq8px_v32 and pax8pxd_v96](https://encode.su/threads/1464-Paq8pxd-dict?p=53999&viewfull=1#post53999) with DRT and split preprocessing of enwik9\nwere released Aug. 29, 2017.\n\n[paq8pxd_v47](https://encode.su/threads/1464-Paq8pxd-dict?p=56327&viewfull=1#post56327) was released Mar. 18, 2018.\n\n[paq8pxd_v48_bwt1](https://encode.su/threads/1464-Paq8pxd-dict?p=57634&viewfull=1#post57634) was released Aug. 9, 2018.\n\npaq8pxd_v61 was released Feb. 23, 2019. \n[Resplit package](https://www.datafilehost.com/d/010c3d90).\n\n[paq8px_v206fix1](https://encode.su/threads/342-paq8px?p=74585&viewfull=1#post74585)\n was released on June 6, 2022 by Zoltán Gotthardt. As most paq8px versions it doesn't\n specifically target enwik9. That is, it doesn't have enwik-specific models,\n it doesn't preprocess enwik8/enwik9 and it doesn't reorder articles either.\n Memory options range from -1 (146 MB) to -12 (28 GB), optional parameters may be\n added to use an LSTM model (L), to use adaptive learning rate\n (A, which hurts enwik results), to load dictionary files to pre-train the Normal,\n Word and Text models before compression (T), to load a pre-trained LSTM repository\n to pre-train the LSTM model before compression (E) or to use the paq8px executable\n itself to pre-train the Normal model. All these optional switches (and some of their\n combinations) were tested at memory level 12. About the results:\n\n```\ns  = zipped source\nsd = zipped source + paq8px-compressed dictionary and repository files\n\nZipped source files (original, files not needed to compile such as CHANGELOG and README are excluded):\n402,949 = 7z.exe a -mm=Deflate -mfb=258 -mpass=15 -mx9 paq8px_v206fix1_src.zip\n\nPaq8px-compressed dictionary files used for text pre-training with the command line switch \"T\":\n109,478 = paq8px -12 @_list.txt (english.dic + english.exp + english.emb in multiple file mode)\n\nPaq8px-compressed LSTM repository, used with the command line switch \"R\":\n221,311 = paq8px -12 english.rnn\n```\n\nOptions select memory usage as shown in the table. Early versions took no options. Most versions were not tested on enwik9 due to their slow speed.\n\n```\n           Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram      Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp  Decomp  Mem Note\n-------      -------    ----------  -----------  -----------  -----------  -----  -----  --- ----\np5                      31,255,092                   9,298 s                3421           1   6\np6                      25,377,998                   9,421 s                4190          16   6\np12                     24,714,219                   9,598 s                4160          16   6\npaq1                    22,156,982                  16,436 s                7800   7790   50\npaq6 v2         -8      19,589,267                  26,548 s               47624         808\npaqar 4.5       -7      18,388,609                 414,164 s              118690 119010  470\npaq8f           -7      18,289,559                  34,371 x               68960         854\n                -8      18,075,265                  34,371 x               69170        1693\npaq8g           -7      17,817,246                 804,867 s               44130         854\npaq8h           -7      17,674,700  147,195,723    801,612 s  147,997,335  56511  57278  854   5\nraq8g           -7      18,132,399                  33,483 x               84555  84793 1089\n                -8      17,923,022                  27,660 x              337430~330000 2095  17\n                -8      17,923,022                  27,660 x              196540~196000 2095  15\npaq8j           -7      18,208,284                  39,366 s              138030 138260  959\n                -8      17,991,628                  39,366 s              138990 136500 1896\npaq8ja          -7      18,184,224                  39,781 s              148560 143200  993\n                -8      17,968,233                  39,781 s              154700 153990 1965\npaq8jb          -7      18,180,081                  39,982 s              148570 148200 1009\n                -8      17,964,363                  39,982 s              188590 190190 1999\npaq8jc          -7      18,185,705                  40,064 s              150910 152080 1017\n                -8      17,970,943                  40,064 s              224410 234900 2015\npaq8jd          -7      18,158,159                  40,460 s              157340 156350 1030\n                -8      17,943,042                  40,460 s              406730        2028\npaq8k           -8      18,239,915                  41,881 s              457150        1463\npaq8l           -6      18,518,485                  35,955 x              133910         435\n                -7      18,168,563                  35,955 x              134770         837\n                -8      17,916,450                  35,955 x              136000 136390 1643\npaq8fthis2      -8      18,075,265                  34,846 x               69100  69310 1693\npaq8n           -8      17,916,420                  37,402 x              134880 135480 1643\npaq8o           -8      17,916,451                  42,389 s              135850 135260 1643\npaq8osse        -8      17,916,451                  42,290 s              125260 124570 1778\npaq8o3          -8      17,916,450                  43,745 s              134580 134530 1636\npaq8o4 v1       -8      17,916,450                  43,876 s              126780 126560 1636\npaq8o6          -8      17,904,721                  44,883 s              139530 139520 1712\npaq8o7          -8      17,904,756                  45,979 s              139140 138530 1574\npaq8o8          -8      17,904,756                  46,381 s              139370 139150 1574\npaq8o8-intel    -1      22,260,679                  46,381 s               24687          37  24\npaq8o8z-jun7    -1      22,260,679                  49,085 s               25919          37  24\n                -1      22,260,680                                         29639          37  25\npaq8o10t        -8      17,772,821                  50,865 s              144250 143720 1591\npaq8p3          -7      18,044,229  150,709,834     57,288 s  150,767,122  72412         803  29\npaq8p3 v2       -7      17,990,788                                         86891         803  29\n                -8      17,759,875                                         87305        1574  29\npaq8px_v60_turbo -8     17,733,057  146,272,609     53,846 s  146,326,455 143846        1643  26\npaq8px_v69      -7      17,939,225                                         20170         878  26\npaq8pxd_v1      -7      17,596,170  144,773,408     83,547 s  144,856,955  63302         811  29\npaq8pxd_v2      -7      17,045,653                                         94280         853  29\n                -8      16,848,214                                         95350        1658  29\npaq8pxd_v3      -7      17,045,354  140,110,094     72,976 s  140,183,094  80069         853  29\n                -8      16,847,903  136,777,893     72,976 s  136,850,869  82822        1658  29\npaq8pxd_v4      -8      16,642,941  135,027,170     67,766 s  135,094,936  88409        1633  29\npaq8pxd_v5      -8      16,699,597                  67,745 s              114960 116450 1633  26\npaq8pxd_v7      -8      16,606,773  134,791,909     70,210 s  134,862,119  93751        1633  29        \npaq8pxd_v8      -8      16,607,759  134,781,085     72,059 s  134,853,144  59387  54611 1521  48\npaq8pxd_v10fix  -8      16,607,760  134,780,308     72,382 s  134,852,690  37177  54433 1633  48\npaq8pxd_v12     -8      16,577,460  134,452,453     81,196 s  134,533,649  54812  54506 1586  48\npaq8pxd_v12-skbuild -10 16,372,331  129,827,930    422,400 s  130,250,330  28313        6500  65\npaq8pxd_v13_x64 -15     16,595,606  131,598,576     83,499 s  131,682,075  29924       25955  65\npaq8pxd_v15     -s9     16,437,892  131,992,226     88,538 s  132,080,764  54993  55067 3243  48\n                -f9     17,838,013                                         11980  11760 1555  48\npaq8pxd_v12_bio -11     16,361,221  129,435,477     82,111 s  129,517,588  30537       13000  65\npaq8pxd_v18     -q8     27,789,833  237,862,503    100,521 s  237,963,024    738         144  80\n                -q9     27,674,156  235,259,956    100,521 s  235,360,477    794         288  80\n                -f8     17,896,675  146,238,833    100,521 s  146,339,354   8725         762  80\n                -f9     17,814,539                                          6696        1482  80\n                -f10    17,790,248                                          7401        2666  80\n                -f11    18,081,957                                          7082        5034  80\n                -f12    18,078,461                                          8755        5674  80\n                -s8     16,516,558  134,561,662    100,521 s  134,662,183  75267        2298  80\n                -s9     16,370,991                                         65814        4552  80\n                -s10    16,308,754                                         65233        7448  80\n                -s15    16,345,626  129,125,083    100,521 s  129,225,607  46698 46608 37878  79\npaq8px_v77      -8      17,629,076  145,454,919     62,154 s  145,517,073  86266 86192  1625  48\ndrt|paq8px_v96  -8      16,704,802  137,170,609    167,886 s  137,338,495  63618 64113  1700  81\npaq8pxd_v32     -s15    16,254,271  128,209,407    144,756 s  128,354,163  41418 43518 27278  81\npaq8pxd_v47     -s15    16,080,717  127,404,715    139,841 s  127,544,556  75022 75611 27500  81\npaq8pxd_v48_bwt1 -s14   16,004,759  126,183,029    153,295 s  126,336,324 579894       51865  81\npaq8pxd_v61     -15     15,968,477  126,587,796    194,704 s  126,782,500  98571 98751 41200  81\npaq8px_v206fix1 -12     16,046,995  126,486,867    402,949 s  126,889,816  151474 ------  28138  91\npaq8px_v206fix1 -12A    16,068,251  ---,---,---    402,949 s  ---,---,---  ------ ------  28151  91\npaq8px_v206fix1 -12LRET 15,820,862  ---,---,---    733,738 sd ---,---,---  ------ ------         92\npaq8px_v206fix1 -12T    15,995,416  126,407,894    512,427 sd 126,920,321  161336 ------  28138  92\npaq8px_v206fix1 -12LT   15,799,749  124,619,348    512,427 sd 125,131,775  507007 514197  28151  92\npaq8px_v206fix1 -12L    15,849,084  124,696,410    402,949 s  125,099,359  291916 294847  28151  93\n```\n\n[durilca](http://www.compression.ru/ds/) and durilca'light 0.5 by Dmitry Shkarin\n(Apr. 1, 2006) are closed source, experimental command line file compressors\nbased on ppmd/ppmonstr with filters for text,\nexe, and data with fixed length records (wav, bmp, etc).  durilca'light is a faster\nversion with less compression.  Unfortunately both\ncrash on enwik9.  Decompression is verified on enwik8.\n\nThe -m700 option selects 700 MB of memory. (It appears to use substantially more for enwik9 according to Windows task manager). -o12 selects PPM order 12 (optimal for enwik9 -t0). -t0 (default) turns off text modeling, which hurts compression but is necessary to compress enwik9 (although decompression still crashes). -t2(3) turns on text preprocessing (dictionary; thus the increased decompresser size). -t2 also supports 3 additive flags (4, 8, 16) which have no effect on this data, thus -t2(31) or -t2 (default is 31) give the same compression as -t(3).\n\n[durilca 0.5(Hutter)](durilca05h.rar)\nwas released 1457Z Aug. 16, 2006.  It does not use external dictionaries.\nWhen run with 1 GB memory (-m700), -o13 is optimal.  With 2 GB (-m1650), -o21 is optimal.\nThe unzipped .exe file is 86,016 bytes.\n\n[durilca4linux_1](DURILCA4Linux_1.rar)\n(0825Z Aug 23 2006)\nis a Linux version of durilca 0.5(Hutter) which successfully compresses enwik9 and\ndecompresses with [UnDur](UnDur.zip)\n(23,375 bytes zipped, 42,065 bytes uncompressed).  All\nversions of durilca require memory specified by -m plus memory to read the input file\ninto memory.  In Windows, this exceeds the 2 GB process limit regardless of available\nRAM and swap.  Thus, enwik9 compresses\nonly under Linux with 2 GB real memory and 1 GB additional swap.  \nThe -o12 option is optimal for enwik9 (tested under 64 bit SuSE 10.0 by the author), \n-o24 for enwik8 (verified by me under 64 bit Ubuntu 2.6.15).\n\n[durilca4linux_2](DURILCA4Linux_2.rar)\n(Oct. 16, 2006)\nis a closed source Linux version specialized for this benchmark.\nIt includes a warning that use on other files may cause data loss.\nIt requires AMD64 Linux and 3 GB of memory (2 GB for enwik8).\nThe decompresser files (EnWiki.dur and UnDur)\nare contained within a 241,322 byte zip file in the rar distribution.  To compress:\n\n```\n  ./DURILCA d EnWiki.dur\n  ./DURILCA e -m1800 -o10 -t2 enwik9\n```\n\nTo decompress:\n\n```\n  ./UnDur EnWiki.dur\n  ./UnDur enwik9.dur\n```\n\nThe first step extracts a compressed dictionary. It is organized in a similar manner to paq8hp2-paq8hp5 in that syntactically related words and words with the same suffix are grouped together. Results are reported by the author under Suse Linux 10.0. I verified enwik8 only (6480 ns/b to compress on a 2.2 GHz Athlon 64 with 2 GB memory under Ubuntu Linux). enwik9 caused disk thrashing.\n[durilca4linux_3](DURILCA4Linux_3.rar)\n(dictionary version v1)\nwas released Feb. 21, 2008.  Like version 2, it requires extraction of EnWiki.dur\nbefore compressing or decompressing, and may not work with files other than\nenwik8 and enwik9.  As tested, requires 64-bit Linux, 4 GB RAM, and 5 GB RAM+swap.\n\n[undur3 v2](undur3v2.zip) contains\nan improved dictionary (version v2), released Apr. 22, 2008,\nfor DURILCA4Linux_3.  The compression\nand decompression programs are the same.  The decompression program UnDur (Linux\nexecutable) is included.  To compress, download durilca4linux_3 and replace the\ndictionary (EnWiki.dur) with this one.  The options are -m3600 (3600 MB memory),\n-o14 (order 14 PPM), -t2 (text model 2).\n\n[undur3 v3](undur3v3.zip),\nreleased May 22, 2008,\nuses an improved dictionary but the same compressor and decompresser as v1 and v2.\nThe dictionary contains 123,995 lowercase words separated by NUL bytes.\nOf these, 5579 words occur more than once (wasted space?)\nI tested options -m1500 under Ubuntu Linix with 2 GB memory.\nAt -m1500 `top` reports 2157 MB virtual memory and 1894 MB real memory. -m1600\ncaused disk thrashing.\n\n[durilca\nkingsize](durilca_kingsize.rar) (July 21, 2009) runs under 64 bit Windows and requires 13 GB memory.\nIt is designed to work only on this benchmark and not in general. The dictionary\nfile EnWiki.fsd must be extracted first from EnWiki.dur before compression or\ndecompression. Requires msvcr90.dll. enwik8 can be compressed with -m1200 (1.2 GB).\n\n[durilca4_decoder](durilca4_decoder.zip) is a new dictionary for\ndurilca'kingsize (above), Nov. 12, 2009. It is reported as\n\"durilca'kingsize_4\" below. Decompression time is reported to be\n1411.88 sec with \"durilca d\" and 1796.98 sec with \"UnDur\". enwik8\ncompresses with 1200 MB (-m1200) in 157.38 sec.\n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp   Notes\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----   -----  \ndurilca'light 0.5   -m650  -o12               21,089,993  178,562,475  1,495,422 x  180,057,897   1227 (fails)  \ndurilca 0.5         -m700  -o12 -t0           19,227,202  162,117,578     74,292 x  162,191,870   4140 (fails)\n                    -m800  -o128              19,321,003  164,298,178     74,292 x  165,372,470   7718 (fails)\n                    -m700  -o12 -t2(3)        18,520,589    (fails)    1,507,312 x                3330  3940\ndurilca 0.5(Hutter) -m700  -o13 -t2           18,128,339    (fails)       77,295 x                5905\n                    -m1650 -o21 -t2           17,958,687    (fails)       77,295 x                6140  6140\ndurilca4linux_1     -m700  -o13 -t2           18,128,334                  23,375 xd               5950  5880\n                    -m1750 -o12 -t2           18,027,888  146,521,559     23,375 xd 146,544,934   5500  7301    18\n                    -m1750 -o24 -t2           17,949,422                  23,375 xd               6190  6780\ndurilca4linux_2     -m1800 -o10 '-t2(11)'     17,002,831  136,536,189    241,322 xd 136,777,511   4249  4827    18\n                    -m1800 -o10 -t2           16,998,300  136,596,818    241,322 xd 136,838,140   4405  4894    18\ndurilca4linux_3 v1  -m3600 -o14 -t2           16,356,063  129,933,145    345,957 xd 130,279,102   3649  3715    18\n                    -m1200 -o32 -t2           16,348,796                                          4170  4178    18\ndurilca4linux_3 v2  -m3600 -o14 -t2           16,323,581  129,670,441    344,525 xd 130,014,966   3628  3639    18\n                    -m1200 -o32 -t2           16,316,255                                          4148  4157    18\ndurilca4linux_3 v3  -m3600 -o14 -t2           16,292,414  129,469,384    339,990 xd 129,809,374   3624  3627    18\n                    -m1200 -o32 -t2           16,285,285                                          4135  4138    18\n                    -m1500 -o6  -t2           16,517,051  133,674,565                             3852\n                    -m1500 -o7  -t2           16,418,799  132,239 495                             4006\n                    -m1500 -o8  -t2           16,368,632  131,722,213                             4149\n                    -m1500 -o9  -t2           16,335,259  131,549,901    339,990 xd 131,889,891   4261  4344\n                    -m1500 -o10 -t2           16,316,775  131,574,739                             4405\n                    -m1500 -o11 -t2           16,306,086  131,707,901                             4544\n                    -m1500 -o12 -t2           16,299,411  131,807,298                             4554\n                    -m1500 -o14 -t2           16,292,414  132,238,662                             4763\n                    -m1500 -o16 -t2           16,289,512  132,516,825                             4879\n                    -m1500 -o32 -t2           16,285,285  134,238,759                             5440\ndurilca'kingsize    -m13000 -o40 -t2          16,258,380  127,695,666    333,790 xd 128,029,456   1413 1805     31\n                    -m22500 -o40 -t2                      127,695,666                             1806 1814     34\ndurilca'kingsize_4  -m13000 -o40 -t2          16,209,167  127,377,411    407,477 xd 127,784,888   1398 1797     31\n                                              16,209,167  127,377,411                             1788 1802     34\n                    -m13000 -o41 -t2          16,209,219  127,376,595    406,700 xd 127,783,295   1341 1466     109\n```\n\n[fxv](https://github.com/kaitz/fxv) [(discussion)](https://encode.su/threads/4339-fxv)\nis a free, open source (GPL v2) experimental archiver by kaitz, Nov. 26, 2024. It is based on paq8pxv\nwith configurable models described in a .pxv file (selected by -c) and just in time model compilation for x86 (selected by -j).\n-t1 says to run in 1 thread. -w says not to store the models in the archive, making it necessary to specify\nthe .cfg file during decompression. There can be multiple streams with different models. The\ndictionary (included in the zipped compressor size) must be present at decompression with the fxcm1 model. -1 is the command to compress.\nThe first two rows of the table are results for config files simulating lpaq1 and fpaq0, and give similar compression ratios.\n\n```\nProgram         Options              enwik8      enwik9     zip size      Total      Comp   Deco  Cmem Dmem  Alg  Note\n--------        -----------        ----------  -----------  ---------  -----------   ----   ----  ---- ---- ----  ----\nfxv v1   -1 -t1 -j -clpaq1.pxv     19,709,887  163,977,016    494,797  164,471,813   2908         1627      CM    102\nfxv v1   -1 -t1 -w -j -cfpaq0.pxv  63,376,137  641,256,112    494,797  641,256,112    555            7      CM    102\nfxv v1   -1 -j -w -cfxcm1.pxv      15,946,608  128,804,372    494,797  129,299,169  20763         1628      CM    102\n```\n\n[cmv](http://encode.su/threads/2284-CMV) 00.01.00 is a free, closed source,\nexperimental file compressor for 32 bit Windows by Mauro Vezzosi, Sept. 6, 2015.\nIt uses context mixing. Option \"2,3,+\" selects max compression (2), max\nmemory (3), and a large set of models (+). A hex bitmap for this argument turns\nindividual models on or off. Note 48 timings are for enwik8 only.\n\ncmv 00.01.01 was released Jan. 10, 2016. It is compatible with 00.01.00 and does not change the compression ratio.\n\n[cmve 0.2.0](https://encode.su/threads/2284-CMV?p=55104&viewfull=1#post55104) was released Nov. 28, 2017.\n\n```\nProgram         Options              enwik8      enwik9     zip size      Total      Comp   Deco  Cmem Dmem  Alg  Note\n--------        -----------        ----------  -----------  ---------  -----------   ----   ----  ---- ---- ----  ----\ncmv 00.01.00    -m2,3,+            18,218,283  150,226,739   77,404 x  150,304,143 285750 293090  2817 2817   CM   48,75\n                                               150,226,739   77,404 x  150,304,143 216000         2801        CM   75\n                -m2,3,0x03ededff   18,153,319                                      720000        ~3900        CM   75\ncmv 00.01.01    -m2,3,0x03ed7dfb   18,122,372  149,357,765   77,404 x  149,435,169 426162 394855  3335 3335   CM   75\ncmve 0.2.0      -m2,3,0x7fed7dfd   16,424,248  129,876,858  307,787 x  130,184,645 1140801       19963        CM   81\n```\n\npaq8hp12any was developed as a fork of the \n[PAQ series](index.html) of open source context mixing compressors by Alexander\nRhatushnyak. It was forked from the paq8 series\ndeveloped largely by Matt Mahoney, and uses a dictionary\npreprocessor (xml-wrt) originally developed by Przemyslaw Skibinski as a separate\nprogram and later integrated. All versions are optimized for\nthe Hutter prize. Thus, they are tuned for enwik8. The 12 versions are\ndescribed below in chronological order. They originally were located\n[here](http://start.binet.com.ua/~artest/HKCC/) (link broken)\nand can now be found\n[here](http://artst.narod.ru/binet/all_HKCC.rar)\n[(as a zpaq archive)](all_HKCC.zpaq) (as of Sept. 16, 2009).\nAll programs are free, GPL open source, command line archivers.  Most take a\nsingle option controlling memory usage.\n\nNote: these programs are compressed with upack, which compresses better than upx. Some virus detectors give false alarms on all upack-compressed executables. The programs are not infected.\n\npaq8hp1 by Alexander Rhatushnyak, 1945Z Aug. 21, 2006. It is a modification of paq8h using a custom dictionary tuned to enwik8 for the Hutter prize. Because the Hutter prize requires no external dictionaries, the dictionary is spliced into the .exe file during the build process. When run, it creates the dictionary as a temporary file. The program must be run in the current directory (not in your PATH or with an explicit path), or else it can't find this file. The unzipped paq8hp1.exe is 206,764 bytes. Decompression was verified for enwik8 (60730 ns/b for -8, 60660 ns/b for -7). enwik9 is pending.\n\npaq8hp2\n([source code](paq8hp2s.rar))\nby Alexander Rhatushnyak, 0233Z Aug. 28, 2006 is an improved version of paq8hp1\nsubmitted for the Hutter prize.  paq8hp2.exe size is 205,276 bytes.\nIt differs from paq8hp1 mainly in that the 43K word dictionary for 2-3 byte codes is sorted alphabetically.\nThe 80 most frequent words, coded as 1 byte before compression, are grouped by syntactic type\n(pronoun, preposition, etc).\n\npaq8hp3\n([source code](paq8hp3s.rar))\nby Alexander Rhatushnyak, released Aug. 29, 2006 is an improved version of paq8hp2\nsubmitted for the Hutter prize on Sept. 3, 2006.\nThe 80 dictionary words coded with 1 byte and 2560 words coded with 2 bytes\nare organized into semantically related groups or by common suffixes.\nThe 40,960 words with 3 byte codes are sorted from the last character in reverse\nalphabetical order.  paq8hp3.exe is 178,468 bytes unzipped.\nenwik9 decompression is not yet verified.  For enwik8, decompression is verified\nwith time 60300 ns/b compression, 60220 ns/b decompression.\n\npaq8hp4\n([source code](paq8hp4s.rar))\nby Alexander Rhatushnyak, released and submitted for the Hutter prize on\nSept. 10, 2006, is an improved version of paq8hp3.\nThe dictionary is further organized into semantically related groups among 3-byte codes.\nThe unzipped size of paq8hp4.exe is 206,336 bytes.\n\npaq8hp5\n([source code](paq8hp5s.rar))\nby Alexander Rhatushnyak, released Sept. 20, 2006, is an improved version of paq8hp4,\nsubmitted for the Hutter prize on Sept. 25, 2006.\nThe unzipped size of paq8hp5.exe is 174,616 bytes (in spite of a slightly larger dictionary).\nThe dictionary size is optimized for enwik8; a larger dictionary would improve compression\nof enwik9.  Decompression is verified for enwik8 only (-8 at 74640 ns/b).\nA [Linux port](http://linux.tu-varna.acad.bg/~lig/UNIX/paq8hp5l.rar) of paq8hp5 is by\nЛъчезар Илиев Георгиев (Luchezar Georgiev), Oct 26, 2006\n([mirror](paq8hp5l.rar)).\n\npaq8hp6\n([source code](paq8hp6s.rar))\nby Alexander Rhatushnyak, released Oct. 29, 2006, is an improved version of paq8hp5.\nIt was submitted as a Hutter prize candidate on Nov. 6, 2006.\nUnzipped paq8hp6.exe size is 170,400 bytes.\nThe -8 option was not tested on enwik9 due to disk thrashing on my 2 GB PC.  Compression was about\n25% finished after 9 hours.\n\npaq8hp7a by Alexander Rhatushnyak, Dec. 7, 2006, was intended to supercede paq8hp6 as a Hutter prize entry, then was withdrawn on Dec. 10, 2006 with the release of paq8hp7. Unzipped executable size is 151,664 bytes. -8 for enwik9 (but not enwik8) caused disk thrashing on my computer (2 GB, WinXP).\n\npaq8hp7\n([source code](paq8hp7s.rar)) by\nAlexander Rhatushnyak, Dec. 10, 2006, as a Hutter prize entry.\nUnzipped paq8hp7.exe size is 152,556 bytes.\n\npaq8hp8\n[(source code)](paq8hp8s.rar)\nby Alexander Rasushnyak, Jan. 18, 2007, as a Hutter prize entry\n(replacing an incorrect version posted 2 days earlier).\nUnzipped size is 152,692 bytes.  The dictionary is identical to paq8hp7.\n\npaq8hp9\n[(mirror)](http://artst.narod.ru/b/paq8hp9.zip)\n[(source code)](paq8hp9s.rar)\nby Alexander Rhatushnyak, Feb. 20, 2007, is a Hutter prize entry.\nOnly the -7 option works.\nThe unzipped size of paq8hp9.exe is 112,628 bytes.\n\npaq8hp9any (Feb. 23, 2007) by Alexander Rhatushnyak is a paq8hp9 -7 compatible version with external dictionary where all options work. However the zipped program is larger and -8 was not tested due to disk thrashing, so results are unchanged.\n\npaq8hp10 (Mar. 26, 2007) by Alexander Rhatushnyak was derived from paq8hp9 as a Hutter prize entry. The unzipped size is 103,224 bytes. Only the -7 option works.\n\npaq8hp10any\n[(source code)](paq8hp10s.rar),\nMar. 31, 2007, by Alexander Rhatushnyak is archive compatible with paq8hp10 -7 but\nworks with other memory options.  When run, paq8hp10.exe and both dictionary\nfiles should be in the current directory.  This program is not a Hutter prize entry.\n\npaq8hp11\n[(mirror)](http://artst.narod.ru/b/paq8hp11.zip)\nby Alexander Rhatushnyak, Apr. 30, 2007, is a Hutter prize entry.\npaq8hp11.exe is 99,816 bytes.  Like paq8hp10, it works only with the -7 option.\n\n```\n  To compress:   paq8hp11 -7 enwik8.paq8hp11 enwik8\n  To decompress: paq8hp11 enwik8.paq8hp11\n```\n\npaq8hp11any\n([source code](paq8hp11any_src.rar))\nby Alexander Rhatushnyak, May 2, 2007, is a paq8hp11 variant\nthat accepts any memory option.  It was optimized for\nspeed rather than size.  It includes two dictionary files which must\nbe present in the current directory when run, unlike paq8hp11 where the\ndictionary is self extracted.  -8 selects 1850 MB memory.  -7 produces\nthe same archive as paq8hp11.  Run speeds for -8 enwik8 are 76770+76820 ns/B.\n\npaq8hp12\n[(mirror)](http://artst.narod.ru/b/paq8hp12.zip)\nby Alexander Rhatushnyak, May 14, 2007, is a Hutter prize entry.\npaq8hp12.exe size is 99,696 bytes.  It works only with the -7 option like paq8hp11.\n\npaq8hp12any\n([source code](paq8hp12any_src.rar))\nby Alexander Rhatushnyak, May 20, 2007, is a paq8hp12 variant that accepts\nany memory option (like paq8hp11any).  The -7 option produces an archive\nidentical to that of paq8hp12.\n\n[paq8hp12any](paq8hp12any_src.zip) was updated\non Jan. 9, 2009\nto fix a compiler issue and add a 64 bit Linux version.\nCompressed file format was not changed. It was not retested.\n\nOptions select memory usage as shown in the table.\n\n```\n           Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram      Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp  Decomp  Mem Note\n-------      -------    ----------  -----------  -----------  -----------  -----  -----  --- ----\npaq8hp1         -7      17,566,769                 205,783 x               60170  60660  748\n                -8      17,397,023  142,477,977    205,783 x  142,683,760  63317        1595\npaq8hp2         -7      17,390,490                 204,557 x               62000  62330  747\n                -8      17,223,661  141,145,684    204,557 x  141,350,241  65323        1584\npaq8hp3         -7      17,241,280                 177,477 x               61360  59690  742\n                -8      17,085,021  139,905,045    177,477 x  140,082,522  63420        1586\npaq8hp4         -7      17,039,173                 198,525 x              ~65000  65110  755\n                -8      16,889,237  138,188,695    198,525 x  138,387,220  67956  68120 1598\npaq8hp5         -7      16,898,402                 161,887 x               76300  77710  900  19\n                -8      16,761,044  137,017,311    161,887 x  137,179,198 ~85153  75162 1787\npaq8hp6         -7      16,731,800  138,828,889    166,715 x  138,995,604  74953  73707  941\n                -8      16,568,451  135,281,289    166,715 x  135,448,004  60865        1807  21\npaq8hp7a        -7      16,592,672  137,441,743    150,678 x  137,592,421  79795         940\n                -8      16,431,239                 150,678 x               76940  77600 1790\npaq8hp7         -7      16,579,500                 151,633 x               79620  79660  940\n                -8      16,417,646  133,835,408    151,633 x  133,987,041  66074        1850  21\npaq8hp8         -7      16,528,353                 151,711 x               79580  79970  940\n                -8      16,372,960  133,271,398    151,711 x  133,423,109  64639        1849  22\npaq8hp9         -7      16,516,789  136,676,674    111,653 x  136,788,327  84529  85957  940\npaq8hp10        -7      16,490,947                 102,256 x               86720  88890  940\npaq8hp10any     -8      16,335,197  132,979,531    333,925 x  133,313,456  55639        1849  22\npaq8hp11        -7      16,459,515                  98,851 x              129540 128530  947\npaq8hp11any     -8      16,304,862  132,757,799    327,608 s  133,085,407  57503        1850  22\npaq8hp12        -7      16,381,959                  98,745 x              130820 131480  936\npaq8hp12any     -7      16,381,959                 330,700 x               78860  76190  941\n                -8      16,230,028  132,045,026    330,700 x  132,375,726  56993        1850  22\n                -8      16,230,028  132,045,026    330,700 x  132,375,726  37660  37584 1850  41\n```\n\npaq8hp1 through paq8hp12 can be used as a preprocessor to other compressors by compressing with option -0. In the following tests on ppmonstr, options were tuned for the best possible compression of enwik8 with 2 GB memory (1.65 GB available under WinXP). The xml-wrt 2.0 options are -l0 -w -s -c -b255 -m100 -e2300 (level 0, turn off word containers, turn off space modeling, turn off containers, 255 MB buffer for dictionary, 100 MB buffer, 2300 word dictionary). The xml-wrt 3.0 options are -l0 -b255 -m255 -3 -s -e7000 (-3 = optimize for PPM).\n\nxml-wrt prepends the dictionary to its output. To make the comparison fair, the compressed size of the dictionary must be added. This is done in two ways, first by compressing the preprocessed text and dictionary and adding the compressed sizes, and second by prepending the dictionary to the preprocessed text before compression. The first method compresses about 1-2 KB smaller.\n\nThe uncompressed size of each dictionary for paq8hp1 through paq8hp4 is 398,210 bytes. They contain identical words, but in different order. The first two dictionaries are identical. They compress smaller because they are sorted alphabetically. The dictionary for paq8hp5 is 411,681 bytes. It contains all of the words in the first 4 dictionaries plus 1280 new words (44,880 total).\n\n```\nPreprocessor    Compressor                 enwik8     dict      total    dict+enwik8\n------------    ----------               ----------  -------  ----------  ---------\npaq8hp1 -0    | ppmonstr J -m1650 -o64   18,322,077   81,190  18,403,267  18,403,991\npaq8hp2 -0    | ppmonstr J -m1650 -o64   18,266,424   81,190  18,347,614  18,349,587\npaq8hp3 -0    | ppmonstr J -m1650 -o64   18,197,797  107,583  18,305,380  18,306,690\npaq8hp4 -0    | ppmonstr J -m1650 -o64   18,170,944  107,590  18,278,534  18,280,098\npaq8hp5 -0    | ppmonstr J -m1650 -o64   18,154,921  111,935  18,266,856  18,267,556\nxml-wrt 2.0   | ppmonstr J -m1650 -o64   18,625,624\nxml-wrt 3.0   | ppmonstr J -m1650 -o64   18,494,374\n (none)         ppmonstr J -m1650 -o16   19,062,555\n                ppmonstr J -m1650 -o32   19,084,964\n                ppmonstr J -m1650 -o64   19,098,634\n```\n\nThe transform done by paq8hp1 through paq8hp5 is based on WRT by Przemyslaw Skibinski, which first appeared in PAsQDa and paqar, and later in paq8g and xml-wrt. The steps are as follows:\n\n[emma](http://encode.su/threads/2459-EMMA-Context-Mixing-Compressor) v0.1.3 is a free, closed source file compressor for 32 bit\nWindows by mpais, Mar. 8, 2016. It uses context mixing. It has a\nGUI-only interface to select compression options. For testing, all settings\nwere for maximum compression as follows: Memory usage 512 Mb, maximum order 9, ring\nbuffer 32 Mb, probability refinement level 3, mixing complexity insane,\nadaptive learning rate on, fast mode on long matches off, ludicrous complexity\nmode on, match model on, 32 Mb, high complexity; text model on, 128 Mb, high;\nsparse model on, 16 Mb, high; sparse model on, 16 Mb, high; indirect model on,\n16 Mb, high; x86/64 model on, 64 Mb, insane; image models on, 80 Mb, high;\naudio models on, 32 Mb, high; record model on, 16 Mb, high; distance model on,\n8 Mb; JPEG model on, 40 Mb, high; GIF model on, 32 Mb, high; executable code (x86/64)\ntransform on; process conditional jumps on; colorspace (RGB) on; delta coding on;\ndictionaries: English on, Spanish off, Italian off, French off, Portugese off.\n\nemma v0.1.4 was released Mar. 13, 2016. For testing, the text model was increased to 256 MB. A DMC model (8 MB) was added. The non-text related models were turned off: x86, image, audio, JPEG, GIF. All transforms (x86, RGB, delta) were turned off.\n\nemma 0.1.6\n(\n[discussion](http://encode.su/threads/2459-EMMA-Context-Mixing-Compressor?p=47301&viewfull=1#post47301)) was released Mar. 27, 2016. It was tested by splitting enwik9 into\nparts using hsplit to move the highly compressible middle part to the end.\nThen the reordered file was then processed using drt dictionary processing\n(see [lpaq9m](<#1440\\<h3>))\ninstead of emma's built in dictionary and then compressed with emma with maximum compression\nand memory options (like below) except that dictionary processing was turned off.\nThe decompressor size includes drt.exe, lpqdict0.dic, hsplit.exe and a BAT file\nto restore the original order, all compressed with emma, then\nthose files plus emma.exe (without dictionaries) compressed into a zip archive.\nSpecifically, enwik9 was prepared:\n\n```\nfsplit32 enwik9 en1 586000000\nfsplit32 en1.1 en2 480000000\nfsplit32 en2.1 en3 424000000\ncopy /b en3.1+en1.2+en3.2+en2.2 enwik9o\ndel en1.1\ndel en1.2\ndel en2.1\ndel en2.2\ndel en3.1\ndel en3.2\ndrt enwik9o enwik9o.drt\ndel enwik9o\n```\n\nbefore compression with emma, then restored after decompression:\n\n```\ndrt enwik9o.drt enwik9o d\nfsplit32 enwik9o en1o 894000000\ndel enwik9o\nfsplit32 en1o.1 en2o 838000000\nfsplit32 en2o.1 en3o 424000000\ncopy /b en3o.1+en2o.2+en1o.2+en3o.2 enwik9\ndel en1o.1\ndel en1o.2\ndel en2o.1\ndel en2o.2\ndel en3o.1\ndel en3o.2\n```\n\nThe command\nemma 0.1.12 was released July 10, 2016. There are 32 and 64 bit versions. The 64 bit version can use more memory. Settings were as follows:\n\n```\n\t\t\tx64\t\tx86\nMemory\t\t\t2048 MB\t\t512 MB\nMax order\t\t10\t\t9\nRing buffer size   \t128 MB\t\t32 MB\nProbability refinement\tlevel 3\t\tlevel 3\nMixing complexity\tinsane\t\tinsane\nAdaptive learning rate\ton\t\toff\nFast mode long matches\toff\t\toff\nLudicrous complexity\ton\t\ton\nMatch model\t\t128 MB, high\t32 MB, high\nText model\t\t1024 MB, high\t256 MB, high\nSparse model\t\t64 MB, high\t16 MB, high\nIndirect model\t\t64 MB, high\t16 MB, high\n86/x64 model\t\toff\t\toff\nImage models\t\toff\t\toff\nAudio models\t\toff\t\toff\nRecord model\t\t64 MB, high\t16 MB, high\nDistance model\t\t32 MB\t\t8 MB\nDMC model\t\t32 MB\t\t8 MB\nJPEG model\t\toff\t\toff\nGIF model\t\toff\t\toff\nXML model\t\t16 MB\t\t4 MB\nRAW models\t\toff\t\toff\nTransforms exec code\toff\t\toff\nColerspace RGB\t\toff\t\toff\nDelta coding\t\toff\t\toff\nDictionaries\t\tEnglish\t\tEnglish\n```\n\n[emma 0.1.22](https://encode.su/threads/2459-EMMA-Context-Mixing-Compressor?p=51783&viewfull=1#post51783) was released Feb. 12, 2017.\nSettings: all settings = MAX, eceept: image and audio models = off, \nuse fast mode on long matches = off, xml=on, x86model=off,\nx86 exe code = off, delta coding = off, dictionary = off, ppmd memory = 1024, ppmd order = 14\n\n[emma 1.23](https://encode.su/threads/1464-Paq8pxd-dict?p=53999&viewfull=1#post53999) was released Aug. 29, 2017. It uses ppmd_mod v3a by Shelwein\nand is preprocessed with DRT.\nEMMA 1.23 settings: all settings = MAX, eceept: image and audio models = off,\nuse fast mode on long matches = off, xml=on, x86model=off, x86 exe code = off,\ndelta coding = off, dictionary = off, ppmd memory = 1024, ppmd order = 14\n\n```\nProgram               enwik8      enwik9    program size      total     Comp  Decomp Mem  Alg Note\n-------             ----------  ----------  ------------  -----------   ----- ------ ---- --- ----\nemma 0.1.3          17,971,713  149,864,553  1,844,505 x  151,709,068  110458 113839 1336 CM  77\nemma 0.1.4          17,865,328  148,887,824  1,848,033 x  150,735,857   58141         980 CM  78\ndrt|emma 0.1.16 x64 16,855,079  136,393,547  1,257,839 x  137,651,386   64341  62102 3800 CM  77\nemma 0.1.12 x86     17,824,974  148,403,034  1,878,971 x  150,282,005   62639         986 CM  78\nemma 0.1.12 x64     17,468,937  142,416,812  2,105,286 x  144,522,098   95997        3688 CM  78\nemma 0.1.22         16,679,420  135,169,967  1,302,363 xd 136,472,330   86187        3824 CM  81\ndrt|emma 1.23       16,523,517  134,164,521  1,358,251 xd 135,522,772   73006 67097  3800 CM  81\n```\n\nA ZPAQ archive is organized into independently compressed blocks. Each block is divided into one or more segments which must be decompressed in sequence. Each segment represents a file or a part of a file. The standard supports both archivers and single file compressors. In the case of a compressor, no filenames are stored in the segment headers, and all the blocks and segments are concatenated to a single output file specified by the user.\n\nZPAQ uses a streaming format that can be read or written in a single pass. The arithmetic coded data is designed so that the end of a segment can be found by scanning quickly without decoding. There is no central directory information to update when blocks are added, removed, or reordered.\n\nThe ZPAQ standard requires that the decompression algorithm be described in the block headers. The header describes a collection of bitwise predictive models based loosely on PAQ components, a program to compute the bytewise contexts for each model, and a second program to perform arbitrary postprocessing on the output data. The two programs are written in an interpreted bytecode language called ZPAQL.\n\nA ZPAQ model specifies a list of 1 to 255 components. Each component outputs a prediction or probability that the next bit will be a 1. Each component may receive as input a computed 32-bit context and the output predictions of earlier components on the list. The last component's prediction is fed to an arithmetic coder to encode or decode the next bit. The components are as follows:\n\nThere are two ZPAQL virtual machines, one (HCOMP) to compute contexts, and one (PCOMP) to postprocess the decoded data. Each program is called once per decoded byte with that byte as input. A ZPAQL machine has the following state:\n\nzpaq 1.03 takes as input a configuration file which describes the arrangement of components, their parameters, and the ZPAQL program HCOMP written one token per byte in a C-like syntax (e.g. \"A=B\" to assign B to A). PCOMP is not specified because in general the preprocessing step by the compressor is different (and usually more complex) than the postprocessing step. Instead, zpaq 1.03 provides the option of two built-in preprocessors, LZP and E8E9. If selected, the preprocessing is done in C++ by the compressor, and the compressor generates ZPAQL code to perform the inverse transform and insert it into the archive block header. (PCOMP is actually appended to the beginning of the input data and compressed with it. HCOMP is not compressed).\n\nE8E9 is used to improve compression of 32 bit x86 executable files. It replaces the 32 bit relative address after a CALL or JMP (0xE8 or 0xE9) x86 instruction by adding the offset from the beginning of the file. This improves compression because often there are several calls to the same target. PCOMP performs the inverse transform in ZPAQL by subtracting the offset.\n\nLZP encodes long string matches as an escape byte and length byte. The decompresser maintains a rolling context hash which indexes a pointer table (the H array) into the output buffer (the M array) pointing to the previous context match. If an escape is present, then the indicated number of bytes are copied from the previous context match. In zpaq 1.03, the user can specify the sizes of M and H, the hash multiplier (effectively choosing the context length), the value to use as the escape byte (preferably occurring rarely in the input), and minimum match length. Escape bytes in the input are encoded as an escaped 0 length.\n\nzpaq 1.03 is distributed with three configuration files, min.cfg (for speed), mid.cfg (the default), and max.cfg (for good compression). However, the user can also write their own config files.\n\n[o0.cfg](o0.cfg), [o1.cfg](o1.cfg), and\n[o2.cfg](o2.cfg) are order 0, 1, and 2 models with a single\nCM and direct context lookup with no hashing. o0 is equivalent\nto fpaq0. In each of the models the asymptotic learning rate\nwas tuned for maximum compression. Other values are given as\ncomments in the sources. The CM uses 2KB, 512KB and 128MB respectively.\n\nmin.cfg uses LZP preprocessing with a minimum match length of 3 and an order 4 context hash, followed by compression by single CM with an order 3 context and 512K entries. The LZP has a 1 MB output buffer and 256K index. It uses 4 MB memory.\n\nmid.cfg (the default) does no preprocessing. It has an order 0 ICM, a chain of ISSE with context orders 1 through 5, each taking the previous ISSE as input, a MATCH with an order 7 context, and a final MIX with an order 1 context taking input from all other models. It uses 111 MB memory.\n\nmax.cfg does no preprocessing. It has 21 components: an order 0 ICM, a chain of order 1, 2, 3, 4, 5, 7 ISSE, an order 8 MATCH, a wordwise order 0-1 ICM-ISSE chain (for text), sparse order 1 ICM with gaps of 1, 2, and 3, a partially masked order 2 ICM with a gap of 216 for CCITT images (calgary/pic), order 0 and 1 mixers taking a CONST and all previous components as input and averaged together with a context free MIX2, followed by a chain of order 0 and 1 SSE each partially bypassed by a context free and order 0 MIX2, and a final context free MIX of all other components. The two wordwise contexts depend on the current and previous case insensitive sequences of letters in the range a-z. It uses 278 MB memory.\n\n[max3.cfg](max3.cfg) is a variation of max.cfg by Jan Ondrus\n(Sept. 10, 2009) using 550 MB memory and without a CCITT model.\n\n[max4.cfg](max4.cfg) is a variation of max3.cfg\n(Sept. 15, 2009) using 1465 MB memory.\n\ndrt is the dictionary preprocessor from lpaq9m by Alexander Rasushnyak. The results include the dictionary file lpqdict0.dic compressed from 465,210 to 88,759 bytes in 8 seconds as a separate archive with max4.cfg and decompressed in 7 seconds, and drt.exe with a size of 15,548 bytes (whether uncompressed or as a zip file) with 38 seconds to encode enwik9 and 38 seconds to decode.\n\n[max_enwik9.cfg](max_enwik9.cfg) is a variation of max.cfg\nby Mike Russell, Sept. 11, 2009. It adds 5 more models for higher order\ncontexts using an ISSE chain after the first order 5 mixer.\n\n[max_enwik9drt.cfg](max_enwik9drt.cfg) is a variation of\nmax_enwik9.cfg, Sept. 18, 2009, modified to define word contexts\nfor ASCII range 65-255 instead of A-Z,a-z because DRT encodes words\nusing bytes in the range 128-255. The compressed size of lpqdict0.dic\nis 86810 bytes, 12+9 sec, compressed separately and added to the\ncompressed sizes.\n\n[zpipe 1.00](zpipe100.zip) is a ZPAQ compatible streaming\nfile compressor that compresses or decompresses from standard input\nto standard output. It takes no options.\nIt compresses equivalently to mid.cfg without storing\na filename or comment. The decompresser outputs the contents of archives\nto a single file by concatenation.\n\n[bwt_j2.cfg](bwt_j2.zip) implements an inverse BWT transform.\nIt was writen by Jan Ondrus, Oct. 6, 2009. The forward transform is\nimplemented by an external preprocessor, bwtpre (included above) by\nMatt Mahoney, Oct. 6, 2009. bwtpre is based on BBB fast mode compression\nbut does not itself compress. The argument \",18\" tells bwt_j2.cfg to use a\nblock size of 2<sup>10+18</sup>-256 bytes. Memory usage is 5x blocksize for both\nthe preprocessor and postprocessor, plus 100 MB for the model.\nThe ability of config files to call external\npreprocessors was added to zpaq v1.05 on Sept. 28, 2009. The ability\nto pass arguments was added to zpaq v1.07 on Oct. 2, 2009.\n\n[zpaq v1.08](zpaq108.zip) (Oct. 14, 2009)\nadds the capability to compile\nZPAQL configuration files and corresponding archive headers to C++ and\nlink to a copy of itself to speed up compression and decompression.\nThe program first looks for an optimized version of the program,\nwrites and compiles it if needed, then runs it to compress or decompress.\nSome tests are shown for speed comparison. max.cfg was modified to\nuse less memory. \nThe arguments to min.cfg, mid.cfg, and max.cfg have the effect of improving\ncompression at the cost of doubling memory for each increment.\n\n[bwt_slowmode1_1GB_block.cfg](bwt_slowmode1.zip) implements\nslow mode BWT transform using 1.25x blocksize memory based on BBB.\nThe inverse transform was re-implemented in ZPAQL by Jan Ondrus,\nOct. 15, 2009.\n\n[zpaq v1.09](zpaq109.zip) is mainly a Linux port of v1.08 with some\ncosmetic improvements. Times for obwt_j2.cfg,18 are shown for comparison\nto v1.07 without optimization. Memory usage is 1838 MB for compression\n(includes preprocessor) and 1443 MB for decompression.\n\nThe c command followed by the name of a configuration file creates a new archive using that file. By default the archive header includes the file name (6 bytes), size (10 bytes), and SHA1 checksum (20 bytes). There are options to omit these and save 36 bytes. The \"oc\" command in zpaq v1.08 optimizes for speed.\n\n[zp 1.00](zp100.zip) is a ZPAQ compatible archiver by\nMatt Mahoney, May 7, 2010. It is designed to have fewer options so it is\neasier to use. It has 3 compression levels: 1=fast, 2=mid, 3=max.\nIt uses compiled ZPAQL code (like zpaq oc/ox) but without requiring\nan external C++ compiler to be installed. It automatically detects when\nan archive is compressed with one of these three models and decompresses\nwith compiled code. Otherwise, it will decompress all other ZPAQ compatible\narchives with slower, interpreted code.\nLevels 2 and 3 are the same as zpaq mid.cfg and max.cfg.\nOnly level 1 (fast) was tested because\nit uses a new model, [fast.cfg](fast.cfg), an ICM chain of length\n2 with order 2 and 4 contexts. It is equivalent to compressing with\nzpaq ocfast.cfg.\n\n```\n                  Compression                          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram            Options                           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------            -------                         ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nzpaq 1.03          co0.cfg                         61,217,687  620,040,242     14,317 xd 620,054,559    441   453  0.4 o0  26\n                   co1.cfg                         46,083,596  454,040,416     14,317 xd 454,054,733    459   480  0.6 o1  26\n                   co2.cfg                         36,694,483  346,551,263     14,317 xd 346,565,580    557   560  134 o2  26\n                   cmin.cfg                        33,460,947  294,281,789     14,317 xd 294,296,106    438   513    4 LZP 26\n                   cmid.cfg                        20,941,558  180,279,221     14,317 xd 180,293,538   3521  3652  111 CM  26\n                   cmax.cfg                        19,412,353  165,191,085     14,317 xd 165,205,402  12211 12204  278 CM  26\n                   cmax3.cfg                       19,179,311  161,604,379     14,317 xd 161,618,696  14108 13609  550 CM  26\n                   cmax4.cfg                       18,986,507  157,246,349     14,317 xd 157,260,666  14061 13077 1465 CM  26\n                   cmax_enwik9.cfg                 18,238,435  149,376,058     14,317 xd 149,390,375  11961       2002 CM  32\ndrt|zpaq 1.03      cmax4.cfg                       18,400,773  149,761,125     29,865 xd 149,790,990   8663  8547 1465 CM  26\n                   cmax_enwik9drt.cfg              18,022,167  146,078,502     29,865 xd 146,108,367  11494 11614 1952 CM  26\nzpipe 1.00                                         20,941,543  180,279,205     13,421 x  180,292,626   3540  3480  111 CM  26\nzpaq 1.07          cbwt_j2.cfg,18                  20,756,888  174,171,969     13,421 x  174,185,390   5593  4347 1838 BWT 26\nzpaq 1.08          ocbwt_slowmodel_1GB_block.cfg   20,756,996  163,565,006     29,153 x  163,594,159   7957  3875 1443 BWT 26\n                   oco0.cfg                        61,217,687                                           335   407  0.4 o0  26\n                   ocmin.cfg                       33,460,960                                           414   383    4 LZP 26\n                   ocmid.cfg                       20,941,558                                          2392  2456  111 CM  26\n                   ocmax.cfg                       19,448,650                                          6569  6641  246 CM  26\n                   ocmax.cfg,3                     18,977,961                                          6667  6640 1861 CM  26\nzpaq 1.09          ocbwt_j2.cfg,18                 20,756,883  174,171,965     31,744 x  171,203,709   4529  1847 1838 BWT 26\nzp 1.00            c1                              24,837,469  222,310,430     26,815 s  222,337,245    688   776   37 CM  26\n                                                                                                        587   688          44\n```\n\n[pzpaq](http://mattmahoney.net/dc/zpaq.html) 0.01\n(a predecessor to zp 1.02) is a free, open source\nfile compressor and archiver by Matt Mahoney, Jan. 21, 2011. It uses a ZPAQ compatible\nformat with speed optimizations for the 3 default compression levels supported by libzpaq,\nzpaq, and zpipe. It supports parallel compression and decompression by dividing the\ninput into blocks which are compressed or decompressed\nat the same time in separate threads, writing\nthe result to temporary files, and then comcatenating them when done.\nFor compression with N threads,\nthe input is divided into N blocks of equal size by default, although a different block\nsize can be specified.\nLarger blocks make compression better but reduce the number of threads that can\nrun at the same time. Using more threads also increases the memory required.\npzpaq can also compress\nor decompress multiple files at once to separate archives or pack them into a solid\narchive or an archive with the packed files split across blocks within the archive.\n\nThe version 0.01 distribution includes a 32 bit Windows executable and source code\nto compile for Windows or Linux. For Windows, the code must be linked with\n[Pthreads-Win32](http://sourceware.org/pthreads-win32/) and pthreadGC2.dll\nis required at run time.\nThe program size was calculated from the source code (including libzpaq)\nrequired for Linux, which has pthreads installed by default and is not included in\nthe size.\n\nThe test results shown below are for 2 machines, a 2.67 GHz Intel Core i7 M620 with 2 cores and 2 hyperthreads per core, running 64 bit Linux (note 48), and a 2.0 GHz Intel T3200 with 2 cores without hyperthreading running 32 bit Windows (note 26). The Linux version was compiled with g++ 4.4.4 -O3 -s -march=native -DNDEBUG. The Windows version used the distributed pzpaq.exe and pthreadGC2.dll. It was compiled with g++ 4.5.0 -O2 -s -march=pentiumpro -fomit-frame-pointer. Times shown are wall (real) times, not process times, in nanoseconds per byte.\n\nWe observe the normal 3 way tradeoff between speed, memory, and compression. Compression levels -1, -2, and -3 require 38 MB, 112 MB, and 247 MB per thread respectively. The default is -2. -t selects the number of threads. The default is -t2. -b selects the block size. The default is the input size divided by the number of threads. The -m option limits memory usage in MB by reducing -t. The default is -m500. Selecting larger -m than required has no effect on compression, speed, or actual memory used. -m is only required with -3 -t3 or higher.\n\n```\n                                            C/D time     C/D time\nLev Thr Block      Memory      enwik8       Note 48      Note 26\n-------------------------    ----------   -----------   -----------\n-1 -t2 -b1000000     -m76    28,176,221                  471\n-1 -t2 -b2500000     -m76    26,915,416                  443\n-1 -t2 -b5000000     -m76    26,236,689                  436\n-1 -t2 -b10000000    -m76    25,728,498                  429\n-1 -t4 -b25000000    -m152   25,253,629    210    220\n-1 -t3 -b33333334    -m114   25,144,587    220    240\n-1 -t2 -b50000000    -m76    25,009,236    240    290    410   430\n-1 -t1 -b100000000   -m38    24,837,482    420    470    750   800\n\n-2 -t2 -b1000000     -m224   24,582,373                 1440\n-2 -t2 -b2500000     -m224   23,374,191                 1396\n-2 -t2 -b5000000     -m224   22,644,738                 1417\n-2 -t2 -b10000000    -m224   22,044,838                 1430\n-2 -t2 -b25000000    -m224   21,438,679                 1382\n-2 -t4 -b25000000    -m448   21,438,679    720    730\n-2 -t3 -b33333334    -m336   21,303,705    790    820\n-2 -t2 -b50000000    -m224   21,138,877    950    980   1300  1310\n-2 -t1 -b100000000   -m112   20,941,571   1510   1560   2350  2330\n\n-3 -t2 -b1000000     -m494   23,281,943                 4142\n-3 -t2 -b2500000     -m494   22,105,128                 3896\n-3 -t2 -b5000000     -m494   21,371,902                 3866\n-3 -t2 -b10000000    -m494   20,745,064                 3854\n-3 -t2 -b25000000    -m494   20,073,978                 3816\n-3 -t4 -b25000000    -m988   20,073,978   1900   1950\n-3 -t3 -b33333334    -m741   19,914,412   2070   2120\n-3 -t2 -b50000000    -m494   19,710,450   2180   2250   3670  3990\n-3 -t1 -b100000000   -m247   19,448,663   3780   3910   6080  6200\n\n                                            C/D time     C/D time\nLev Thr Block      Memory     enwik9        Note 48      Note 26\n-------------------------   -----------   -----------   -----------\n-1 -t2 -b1000000     -m76   254,931,717                  582\n-1 -t2 -b10000000    -m76   232,278,737                  425\n-1 -t2 -b100000000   -m76   224,233,690                  392\n-1 -t2 -b250000000   -m76   223,043,964                  393\n-1 -t4 -b250000000   -m152  223,043,964    198    223\n-1 -t3 -b333333334   -m114  222,789,971    224    254\n-1 -t2 -b500000000   -m76   222,544,698    236    276    408   556\n-1 -t1 -b1000000000  -m38   222,310,443    410    470    758   800\n\n-2 -t2 -b1000000     -m224  216,322,292                 1377\n-2 -t2 -b10000000    -m224  192,436,071                 1286\n-2 -t2 -b100000000   -m224  182,293,069                 1275\n-2 -t2 -b250000000   -m224  180,995,559                 1278\n-2 -t4 -b250000000   -m448  180,995,559    710    742\n-2 -t3 -b333333334   -m336  180,716,954    768    811\n-2 -t2 -b500000000   -m224  180,516,414    854    881   1275\n-2 -t1 -b1000000000  -m112  180,279,234   1487   1532   2231\n\n-3 -t2 -b1000000     -m494  203,976,295                 3824\n-3 -t2 -b10000000    -m494  180,499,077                 3657\n-3 -t2 -b100000000   -m494  168,839,648                 3611\n-3 -t2 -b250000000   -m494  167,036,071                 3635\n-3 -t4 -b250000000   -m988  167,036,071   1881   1926\n-3 -t3 -b333333334   -m741  166,567,322   2025   2158\n-3 -t2 -b500000000   -m494  166,324,415   2172   2236   3599\n-3 -t1 -b1000000000  -m247  165,887,518   3708   3846   5989\n```\n\nOption -m2 selects the better BWT mode (bwt2), which drops the RLE step and uses an order 0-1 ISSE chain. The order-1 ISSE adjusts the order-0 ICM prediction by mixing it in the logistic domain with a constant, such that the pair of weights is selected by an 8-bit bit history, which is selected by an order 1 context of the BWT output. After coding, the mixing weights are adjusted to reduce the prediction error.\n\nOptions -m3 and -m4 select the \"mid\" and \"max\" modes, the same as -4 and -5 respectively in pzpaq. The option -bN selects a block size of N*2^20 - 256 bytes. Memory usage per thread for the two BWT modes is 5 times the block size after rounding up to a power of 2. The default is -b32 which uses 160 MB per thread for -m1 and -m2. Memory usage for -m3 and -m4 is not affected by block size. Usage is 111 MB and 246 MB per thread for -m3 and -m4 respectively.\n\nOther changes: there is no longer an option to limit memory. The default number of threads (-t option) is the number of cores. There is no solid mode compression because BWT requires that each block contain only one whole or part of a file. There is a separate decompresser, unzp, which is optimized for fast, mid, max, bwtrle1, and bwt2 modes, and can be configured to optimize for other models by generating, compiling, linking, and running C++ code for an optimized version of itself. Compressed sizes are based on the unzp source code (37,967 bytes).\n\n[zpaq 4.00](../zpaq) was released Nov. 13, 2011. It uses libzpaq v4.00,\nwhich internally\ntranslates ZPAQL into just-in-time (JIT) x86-32 or x86-64, which runs about as fast\nas the previous version that translated ZPAQL to C++ and compiled it.\nUnlike the earlier version, it correctly handles all legal ZPAQL, such as jumps into\nthe middle of a 2 byte instruction, such as occurs in max_enwik9.cfg.\nLike zp 1.02, it uses multi-threading and the same build-in compression\nlevels -m1 through -m4.\n\nResults are shown below for a 4 GB 2.66 GHz Core I7 M620 (note 40), which has 2 cores with 2 hyperthreads each. Run under Ubuntu 64 bit Linux. Compression and decompression times (wall times, ns/byte) are shown for 1 through 4 threads (-t1 through -t4) as the compression method (-m) and block size (-b) are varied. max_enwik9 runs in one thread in a single block.\n\n```\nCompressor  Options        enwik8       enwik9       -t1        -t2        -t3        -t4     MB/thread\n----------  --------     ----------  -----------  ---------  ---------  ---------  ---------  ----------\nzp 1.02     -m1 -b32     24,091,153  210,224,876   264  313   144  184   131  170   120  165   160\n            -m1 -b128    22,823,452  197,571,474   264  335   163  208   137  187   136  179   640\n            -m1 -b256    22,823,452  191,741,553              167  218                        1280\n            -m2 -b32     22,440,353  195,887,789   446  514   259  304   237  274   231  267   160\n            -m2 -b128    21,246,043  184,023,690   467  543   291  343   250  295   248  294   640\n            -m2 -b256    21,246,043  178,551,919              304  351                        1280\n            -m3 -b32     21,301,940  185,584,854  1420 1478   805  856   760  790   713  745   111\n            -m3 -b128    20,941,571  181,908,375  1430 1491   851  897   772  823   723  758   111\n            -m3 -b1024   20,941,571  180,279,234  1446 1503                                    111\n            -m4 -b32     19,912,920  172,989,918  3567 3695  2075 2145  1966 2011  1868 1906   246\n            -m4 -b128    19,448,663  168,312,889  3578 3706  2156 2234  1984 2043  1875 1925   246\n            -m4 -b1024   19,448,663  165,887,518  3597 3732                                    246\n\n             Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------     ------------     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nzpaq 4.00   -mmax_enwik9     18,238,435  149,376,058     66,958 s  149,440,016   6327  6528 2002 CM   48\n```\n\n[zpaq](http://mattmahoney.net/dc/zpaq.html)\n[v6.12](http://mattmahoney.net/dc/zpaq612.zip), Oct. 19, 2012,\nis a journaling, deduplicating, incremental archiver. These features were\nadded in zpaq v6.00 on Sept. 26, 2012.\nIt implements the level 2 ZPAQ standard introduced with libzpaq v5.00\non Feb. 1, 2012. The level 2 standard allows for uncompressed\n(but possibly pre/post-processsed) data. The format is described\nin the [ZPAQ specification v2.01](http://mattmahoney.net/dc/zpaq201.pdf).\n\nzpaq v6.12 is designed for large backups. It will compress 100 GB to an external drive in a few hours, then perform daily incremental backups of files whose dates have changed in a few minutes. It recursively traverses directories, storing last-modified dates and attributes of added files.\n\nA journaling archive is append-only. When a journaling archive is updated, it keeps both the old and new versions of each file or directory. The old version can be extracted by specifying a dated version, and any later updates are ignored.\n\nInput is deduplicated before compression by dividing input files into fragments averaging 64 KB on content-dependent boundaries that move when data is inserted or removed. The archive stores fragment SHA-1 hashes and stores any fragment with a matching hash as a pointer to an existing fragment. Any remaining fragments are packed into 16 MB blocks in memory and compressed by multiple threads in parallel to memory buffers before being appended to the archive. After compression is completed, the fragment sizes and hashes are appended, and then a list of index updates in separately compressed blocks. Each update is either a deletion (filename only) or an update (filename, date, attributes, and list of fragment pointers).\n\nAn update is performed as a transaction by first appending a temporary header, then the compressed data and index, and then finally going back and updating the header to store the compressed data size so that it can be skipped over when listing the archive contents or preparing a list of files to add or extract. If compression is interrupted or an error occurs, then the temporary header is not updated. If zpaq encounters a temporary header then it assumes that any data following it is corrupted and ignores it during extraction or listing, and overwrites it during the next update.\n\nzpaq also has features to summarize the contents of archives containing millions of files, show update history and version dates, and compare and extract individual files and directories and rename them. Archives can be encrypted.\n\nThe deduplication algorithm uses a rolling hash of the input that depends on the last 32 bytes that are not predicted in an order-1 context. Missed predictions (from a 256 byte table) are counted as a heuristic to guess whether a block can be compressed. If not, then it is stored without compression as a speed optimization. There are 4 compression levels (-method 1 through 4). The threshold for compressing a block is 1/16, 1/32, 1/64, and 1/128 of bytes predicted by the order 1 model, respectively. Like earlier versions of zpaq, it also accepts configuration files and external preprocessors. These are always compressed.\n\nThe journaling format is not compatible with zpaq versions prior to 6.00. Older versions would decompress a journaling archive to a set of jDC* files that could in theory reconstruct the data. To support older versions, there are three additional modes: streaming, solid, and tiny. In streaming mode, each file is compressed in parallel in a separate block, and large files are split into 16 MB blocks. In solid mode, all files are compressed to a single block in a single thread. Tiny mode is like solid mode except that comments (uncompressed sizes), checksums, and header locator tags (for error recovery) are not stored, saving a few bytes each. None of these modes support journaling, incremental backup, or deduplication, and do not save file attributes or empty directories. An update appends to an archive without checking whether the files have been added before.\n\nThere are 4 built in methods. Method 1 is equivalent to \"lazy\" level 3. It is LZ77 using variable length codes to represent the lengths of literal byte strings or the length and offset of matches to earlier occurrences of the same string in a 16 MB output block. Matches are found by indexing a hash of the next 4 bytes in the input buffer into a table of size 4M which is grouped into 512K buckets of 8 pointers each. The longest match is coded, provided the length is at least 4, or 5 if the offset is greater than 64K and the last output was a literal. Ties are broken by favoring the smaller offset. Bucket elements are selected for replacement using the low 3 bits of the output count.\n\nLiteral lengths are coded using \"marked binary\" Elias gamma codes, where the leading 1 bit of the number is dropped and a 1 bit is inserted in front of the remaining bits and a 0 marks the end. For example, 1100 is coded as 1,1,1,0,1,0,0. Matches are coded as a length and an offset. The length is at least 4. All but the last 2 bits are coded as a marked binary. The number of match bits is given in the first 5 bits of the code. If the code starts with 00, then a literal length and string of literal follow. Otherwise the 5 bits code a number from 0 to 23, and that number of bits, with an implied leading 1 give the offset.\n\nThe codes are not compressed further. They are stored in the ZPAQ level 2 format, consisting of a sequence of sub-blocks each preceded by a 4 byte header giving the sub-block size.\n\nMethod 2 is also LZ77, but the codes are byte aligned and context modeled rather than coded directly. It also searches 4 order-7 context hashes and 4 order-4 hashes, rather than 8 order-4 hashes like method 1. Method 2 first codes as follows, according to the high 2 bits of the first byte:\n\n```\n  00 = literal of length 1..64, followed by uncompressed bytes.\n  01 = match of length 4..11 and offset 1..2048.\n  10 = match of length 1..64 and offset of 1..65536.\n  11 = match of length 1..64 and offset of 1..16777216.\n```\n\nThese codes are arithmetic coded using an indirect context model. The context depends on the parse state and in the case of literals, on the previous byte. An indirect context model maps a context into a bit history (represented as an 8 bit state) and then to a bit prediction. The model is updated by adjusting the prediction to reduce the error by 0.1%. A bit history represents a bounded pair of bit counts (n0,n1) and the value of the most recent bit. The bounds for (n0,n1) and (n1,n0) are (20,0), (48,1), (15,2), (8,3), (6,4), (5,5).\nMethod 3 uses a Burrows-Wheeler transform (BWT) using libdivsufsort-lite v2.0. This is equivalent to -m2 in older zpaq versions. The input bytes are sorted by their right contexts and compressed using an order 0-1 ICM-ISSE chain. The order 0 ICM (indirect context model) works as in method 2, taking only the previous bits of the current byte (MSB first) as context. The prediction is adjusted by an order-1 indirect secondary symbol estimator (ISSE). An ISSE maps its context (the previous byte and the leading bits of the current byte) to a bit history, and the history selects a pair of mixing weights to compute the weighted average of the constant 1 and the ICM output in the logistic domain, log(p/(1-p)). The output is converted back to linear, and the two weights are updated to reduce the prediction error in favor of the better model. In other words, the output is:\n\n```\n  p' := 1/(1 + exp(-w1*1 - w2*log(p/(1-p))))\n```\n\nand after the bit is arithmetic coded, the weights w1 and w2 are updated:\n\n```\n  w1 := w1 + 1            * 0.001 * (bit - p')\n  w2 := w2 + log(p/(1-p)) * 0.001 * (bit - p')\n```\n\nMethod 4 is equivalent to mid.cfg or -m3 in older zpaq versions. It directly models the data using an order 0-5 ICM-ISSE chain, an order 7 match model, and an order 1 mixer which produces the bit prediction by mixing the predictions of all other components. The 6 components in the chain each mix the next lower order prediction using a hash of the next higher order context to select a bit history for that context, which selects the mixing weights. A match model has a 16 MB history buffer and a 4M hash table of the previous occurrence of the current context. If a match is found, it predicts the bit that followed the match with probability 1 - 1/(length in bits). The outputs of all 7 models are then mixed as with an ISSE except with a vector of 7 weights selected by an order 1 (16 bit) context, and with a faster weight update rate of about 0.01.\n\nWith method 4 you can give an argument like \"-method 4 1\" to double the memory allocated to the components to improve compression. The same extra memory is needed to decompress. The default is 111 MB per thread. An argument n multiplies memory usage by 2^n. n can be negative.\n\nMethods 1, 2, and 3 only work in journaling and streaming mode, since they have a 16 MB block size limit. Method 4 and configuration files work in all modes.\n\nThe following tests are on a 2.0 GHz T3200 with 2 cores. zpaq will automatically detect the number of cores and use the same number of compression or decompression threads, although this can be overridden.\n\n```\n             Compression                 Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------     ------------             ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nzpaq 6.12   -method 1                37,397,857  328,974,375    104,067 s  329,078,442     93    53  152 LZ77  26\n            -method 1 -streaming     37,359,931  328,618,875    104,067 s  328,722,942     85    28  151 LZ77  26\n            -method 2                31,765,035  281,184,939    104,067 s  281,289,006    196   108  153 LZ77  26\n            -method 2 -streaming     31,730,884                                           218   126  151 LZ77  26\n            -method 3                23,341,562  203,365,453    104,067 s  203,469,520    429   369  238 BWT   26\n            -method 3 -streaming     23,328,888                                           425   375  238 BWT   26\n            -method 4                21,768,810                                          1403  1371  299 CM    26\n            -method 4 -streaming     21,744,770                                          1403  1356  299 CM    26\n            -method 4 -solid         20,941,591                                          2036  2056  109 CM    26\n            -method 4 1 -solid       20,740,920                                          2338  2197  216 CM    26\n            -method 4 4 -solid       20,581,270                                          2356  2289 1482 CM    26\n            -method 4 4 -tiny        20,581,208  173,028,477    104,067 s  173,132,544   2107  2230 1654 CM    26\n```\n\n[zpaq](http://mattmahoney.net/dc/zpaq.html)\n[v6.19](http://mattmahoney.net/dc/zpaq619.zip), Jan. 23, 2013,\nmoves the -solid and -tiny modes into a separate program, zpaqd, and\neliminates -streaming. It adds 5 more compression levels (0 through 9).\n-method 5 is max.cfg, a 22 component CM with some of the component sizes\nreduced to use about 225 MB per thread. -methods 6 through 9 each double\nthe memory size (450 MB to 1.8 GB) and block size (32 MB to 256 MB).\nAll levels except 0 (store uncompressed) have an E8E9 pre/post-processor.\n-methods 0 through 4 are unchanged.\n\n```\n             Compression                 Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------     ------------             ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nzpaq 6.19   -method 0 -threads 2    100,050,464                                            37    42  169 copy  26\n            -method 1 -threads 2     37,398,697                                           143    61  225 LZ77  26\n            -method 2 -threads 2     31,766,023                                           294   185  225 LZ77  26\n            -method 3 -threads 2     23,342,327                                           635   548  322 BWT   26\n            -method 4 -threads 2     21,770,084                                          1319  1331  378 CM    26\n            -method 5 -threads 2     20,491,832                                          3778  3773  563 CM    26\n            -method 6 -threads 2     19,901,321                                          4446  4615  991 CM    26\n            -method 7 -threads 2     19,497,869                                          4625  4711 1845 CM    26\n            -method 8 -threads 1     19,038,853  164,475,887     95,914 s  164,571,801   6153  6296 1911 CM    26\n            -method 8 -threads 2     19,038,853                                          3553  3551 3800 CM    48\n            -method 9 -threads 1     19,004,217  161,001,056     95,914 s  161,096,970   3468  3521 3800 CM    48\n```\n\n[zpaq v6.34](http://mattmahoney.net/dc/zpaq634.zip) has 7 compression\nmethods as follows:\n\nMethods 0 and 1 use 16 MB blocks by default. Methods 2..6 use 64 MB blocks.\nThe size can be specified by a second digit N which specifies 2<sup>N</sup> MB\nblocks. Thus, the defaults are 04, 14, 26, 36, 46, 56, 66. Larger blocks\ncompress better but require more memory per thread.\n\nMethods 1..6 use heuristics to detect already compressed data and either store it or compress it with a fast method like 1 depending on the degree of compressibility. The heuristic depends on the 256 byte order-1 prediction table that is used to compute the rolling hash used in the fragmentation algorithm. The table is initialized to all zeros at each fragment boundary, and contains the last byte seen in each of 256 possible 1 byte contexts. If the data is random, then at each fragment boundary (average size 64K), the following properties are expected:\n\nIn addition, the order 1 tables are used to detect text and x86 (.exe) data types. Text is detected if at least 5 letter, digit, period, or comma contexts predict a space, minus any predicted characters in the range 1..8, 11, 12, 14..31, which normally do not appear in text files. If at least 1/4 of the fragments are detected as text, then methods 5 and 6 add extra models for it. x86 is detected if at least 5 contexts predict a 139 (an x86 MOV reg, r/m instruction). If at least 1/8 of the fragments are detected as x86, then a E8E9 pre/post processor is used in methods 1..6.\n\nLZ77 and BWT removed the 16 MB block size limitation of the previous version.\nVariable length LZ77 adds an extra field of rb = 1..8 bits to represent the low\nbits of an offset up to 32 bits, where rb increases by 1 for each doubling of\nthe block size over 16 MB. 2<sup>rb</sup> - 1 is added to the offset, so that\nit requires a rb..rb+23 bit code.\n\nByte aligned LZ77 removed the limitation by eliminating the short code (3 bit length and 11 bit offset) and adding a code with 4 offset bytes. Lengths range from m..m+63 where m is the mininum match length, normally 8 when used with an order-1 context model.\n\nBWT removes the block size limitation by removing the IBWT optimization of packing pointers and the byte pointed to into a single 32 bit linked list element when the block size is over 16 MB. No changes were required for higher compression levels.\n\nzpaq versions since v6.22 support custom context models through the command line. When compressing enwik8 and enwik9 the following models are automatically generated:\n\n```\nOption    Equivalent\n------    ----------\n  -m 0    -m x4,0\n  -m 1    -m x4,1,4,0,3,24,16,18\n  -m 18   -m x8,1,4,0,3,27,16,18\n  -m 2    -m x6,1,4,8,4,26,16,18\n  -m 28   -m x8,1,4,8,4,27,16,18\n  -m 3    -m x6,2,8,0,4,26,16,24c0,0,511\n  -m 38   -m x8,2,8,0,4,26,16,24c0,0,511\n  -m 4    -m x6,3ci1\n  -m 48   -m x8,3ci1\n  -m 5    -m x6,0ci1,1,1,1,2awm\n  -m 58   -m x8,0ci1,1,1,1,2awm\n  -m 6    -m x6,0w2c0,1010,255i1c256ci1,1,1,1,1,1,2ac0,2,0,255i1c0,3,0,0,255i1c0,4,0,0,0,255i1mm16ts19t0\n  -m 68   -m x8,0w2c0,1010,255i1c256ci1,1,1,1,1,1,2ac0,2,0,255i1c0,3,0,0,255i1c0,4,0,0,0,255i1mm16ts19t0\n```\n\nThe meaning is as follows.\n\n**x** (experimental) rather than a digit\nselects a specific method which is the same for every block. It can also be s\nto add in streaming mode with each file in a separate block and large files\nsplit into blocks with no deduplication.\n\nThe first digit N1 after x selects a maximum block size of 2<sup>N1+20</sup> - 4096\nbytes. This is selected by the second digit of the method, if present, or else it\ndefaults to 6 for methods 2..6 or 4 otherwise.\n\nThe second digit N2 selects the pre/post processing step. 0 means none. 1 means LZ77 with variable length codes. 2 means LZ77 with byte aligned codes. 3 means BWT. 4..7 means 0..3 with E8E9 filtering.\n\nN3..N8 apply to the LZ77 modes only. N3 (4 or 8) is the minimum match length.\nN4 (8 or 0) if not 0 specifies a context order to search first. N5 (3 or 4) says\nto search 2<sup>N5</sup> contexts of each order to look for matches.\nN6 (24..27) specifies 2<sup>N6</sup> elements in the hash table for lookups. Each entry\nrequires 4 bytes of memory. It defaults to the block size up to N1=26, then N1-1.\nN7 and N8 specify that the minimum match (N3) should be increased by 1 after a\nliteral or match, respectively, when the match offset is greater than 2<sup>N7</sup>\nor 2<sup>N8</sup> respectively.\n\nThe sequence of strings starting with letters followed by a comma-separated list\nof numbers specifies various context models used by methods 3 and higher. **c0** specifies\nan ICM (indirect context model: context to bit history to prediction). **c1**...** c256**\n(used in -m 6) specifies a CM (context to prediction) with an update\nrate of 1/count and maximum count of N1*4-4, e.g. c256 specifies 1020.\nThe remaining arguments to c default to 0. N2 describes any special contexts.\nN2 in 1..255 (e.g. c0,2) means offset mod N2. N2 in 1000..1255 means the distance\nto the last occurrence of N2-1000 (e.g. c0,1010 means how far from the last linefeed).\nN3 and up specifies byte masks starting with the most recent context byte\n(e.g. c0,2,0,255 means offset mod 2 combined with the second context byte (sparse\nmodel)). A value of 256..511 includes the byte aligned LZ77 parse state if applicable\n(e.g. c0,0,511 means the order 1 context plus parse state hashed together).\n\n**i** followed by a list specifies a chain of ISSE components with each context order\nincreasing by the specified amount by hashing it with the previous component,\n(e.g. ci1,1,1,1,2 specifies an order 0 ICM chained with order 1, 2, 3, 4, 6 ISSE).\nEach ISSE (indirect secondary symbol estimator) adjusts the prediction of the\nprevious component in the bit history of the current context (hashed together with the\nprevious component's context).\n\n**a** specifies a match model, which predicts the bit which followed the most\nrecent occurrence of the current (normally high order) context. It can take\nparameters specifying buffer size, hash table index size and context order.\n\n**w** N1 specifies a word model, an ICM-ISSE chain of increasing order from\n0 to N1-1 in words rather than bytes. A word is defined as a sequence of\nletters converted to upper case, ignoring all other characters\n(e.g. w2 specifies an order 0 ICM and order 1 ISSE). It can take additional\nparameters specifying an alphabet range and a mask to convert case.\n\n**m** specifies a mixer, which adaptively averages the predictions of all\nprior components. It can take a parameter (default 8) which is the number of bits\nof context to select the mixing weights (e.g. m16 is a byte-wise order 1 context).\nIt takes additional parameters specifying update rate.\n\n**t** is a MIX2 2-input mixer which averages just the last 2 components.\n\n**s** is a SSE which adjusts the prevous prediction like an ISSE but using a\ndirect context instead of a bit history. It takes parameters specifying the\nnumber of context bits (e.g. s19 selects the current and previous bytes and the\n3 high bits of the second byte), and additional parameters specifying initial\nand final update rates.\n\n-m is short for -method. -th 1 (-threads 1) selects 1 thread. The default on the test machine is 4 (2 cores + 2 hyperthreads). It is also used in decompression to reduce memory.\n\n```\n             Compression                 Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------     ------------             ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nzpaq 6.34   -m 1                     36,720,879  322,717,507                               38    15  456 LZ77  48\n            -m 18 -th 1              36,174,283  316,439,766                               85    25 1200 LZ77  48\n            -m 2                     32,785,291  287,047,166                               76    17 1500 LZ77  48\n            -m 28 -th 1              32,123,217  279,231,899                              159    25 1200 LZ77  48\n            -m 3                     30,759,444  270,317,562                               89    56 1500 LZ77  48\n            -m 38 -th 1              30,216,795  264,333,006                              198   106 1200 LZ77  48\n            -m 4                     21,982,505  189,860,169                              285   224 1800 BWT   48\n            -m 48 -th 1              21,293,686  179,016,475                              596   512 1400 BWT   48\n            -m 5                     20,742,462  179,365,293                              937   658 2100 CM    48\n            -m 58 -th 1              20,214,879  172,645,399                             1931  1430 2400 CM    48\n            -m 6                     19,627,225  168,583,236                             2348  2356 3300 CM    48\n            -m 68 -th 1              18,998,601  160,541,121     118,086 s  160,659,207  4300  4408 3200 CM    48\n```\n\nThe following table shows compression with the config file\n[max5.cfg](max5.cfg) (Oct. 14, 2013). This is the same model as\n[max_enwik9.cfg](max_enwik9.cfg) except that it was\nmodified to take an argument to double memory usage for most of the\ncomponents for each increment. With argument 0, it is the same as max_enwik9.\nCompression was with\n[zpaqd 6.33](zpaqd633.zip) (June 20, 2013), which is the developement tool that\naccompanies zpaq and produces streaming mode archives from a config file.\nThus, the command \"zpaqd c max5 3 archive enwik9\" compresses to archive.zpaq\nwith 3 passed to $1 in max5.cfg. This has the effect of using almost 8 times as\nmuch memory for both compression and decompression as max_enwik9.\nThe archive was decompressed with both [zpaq 6.42](zpaq642.zip)\n(Sept. 26, 2013)\nand with [tiny_unzpaq](tiny_unzpaq.cpp) (Mar. 21, 2012, public\ndomain) compiled with g++ 4.1.2 -O3\nunder Linux on the test machine, which has 20 GB of available memory.\nzpaq 6.42 is an archiver like zpaq 6.33 with a number of added features\nand bug fixes unrelated to compression. tiny_unzpaq is a stand-alone program\nthat extracts only streaming mode archives and is designed so that the source\ncode is as small as possible. It does not support JIT compilation of the\nZPAQL code, or multithreading and has no error checking or help message.\nIt takes an archive as an argument with no options\nand extracts to the saved names.\n\n[max6.cfg](max6.cfg) (Oct. 15, 2013) modifies max5 by\nrewriting the word model and adding models that count brackets (\"[\" minus \"]\"\nin range 0..2) and a column model (counts bytes after the last linefeed\nin range 0..64). It also changes the memory parameter from $1 to $3\nso it can be passed to zpaq like \"-m s10.0.5fmax6\". This means to choose\nstreaming mode (s), a block size of 2^10 MB (10), no preprocessing (0),\npass 5 as $3 selecting 14 GB (or 1 selecting 1.4 GB) using max6.cfg.\nFor this test, tiny_unzpaq is used to extract when the decompresser\nis given as \"sd\" although either program could be used.\n\n```\n             Compression                 Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------     ------------             ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nzpaqd 6.33  max5 0                   18,238,448                                          5960        2000 CM   61\n            max5 1                   18,135,013  146,750,019                             6309        3400 CM   61\n            max5 2                   18,095,676  144,918,290                             6521        6600 CM   61\n            max5 3                   18,084,027  143,757,714       4,760 sd 143,762,474  5894 13173 13100 CM   61\nzpaq 6.42                                        143,757,714     125,670 s  143,883,384        5985 13500 CM   61\nzpaq 6.42   -m s10.0.1fmax6          18,167,158  150,622,666     125,670 s  150,748,336  6368  6475  1400 CM   61\n            -m s10.0.5fmax6          17,855,729  142,252,605       4,760 sd 142,257,365  6699 14739 14000 CM   61\n```\n\n[zpaq 6.50](zpaq650.zip), Mar. 21, 2014, uses 5 compression\nlevels instead of 6. LZ77 when used in methods 2 and higher uses a suffix\narray to find matches. There are also other improvements in sorting files,\ngrouping into blocks, detecting file type, detecting random data, and\nselecting compression algorithm based on type. Tests below used 4 threads.\n\n```\n             Compression                 Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------     ------------             ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nzpaq 6.50   -method 1                35,691,734  314,117,968     137,993 s 314,255,964     35    23  512 LZ77  48\n            -method 2                31,184,422  271,626,606     137,993 s 271,764,602    150    24 1800 LZ77  48\n            -method 3                21,980,366  189,875,990     137,993 s 190,013,986    222   220 1600 BWT   48\n            -method 4                20,740,505  179,455,249     137,993 s 179,593,245    665   670 2200 CM    48\n            -method 5                19,625,015  168,590,741     137,993 s 168,728,730   2410  2419 3400 CM    48\n```\n\nlpaq versions 1 through 8 may be downloaded [here](index.html#lpaq).\nlpaq9* can be downloaded [here](http://binet.com.ua/~artest/paqar/)\nor as a [zpaq archive](lpaq9.zpaq).\nThe decompr8 series of Hutter prize entries (decompresser and enwik8 archive) are also\nlisted here because they followed a period of development of the lpaq series.\n\nNote: some of these programs are compressed with upack, which compresses better than upx. Some virus detectors give false alarms on all upack-compressed executables. The programs are not infected.\n\n[lpaq1](lpaq1.zip) is a free,\nopen source (GPL) file compressor by Matt Mahoney, July 24, 2007.  It uses context mixing.\nIt is a \"lite\" version of paq8l, about 35 times faster at the cost of about\n10% in compression.  The \"9\" option selects maximum memory.  The options\nrange from 0 (6 MB) to 9 (1.5 GB).  Memory usage is 3 + 3*2<sup>N</sup> MB,\nN = 0..9.\n\nThe compressor mixes 7 contexts: orders 1, 2, 3, 4, 6, a unigram word context (consecutive letters, case insensitive), and a matched bit context. The contexts (except the matched bit) are mapped to nonstationary bit histories using nibble-aligned hash tables, then mapped to bit prediction probabilities using stationary adaptive tables with bit counts to control adaptation rate. The matched bit context maps the predicted bit (based on a context match), match length and order-1 context (or order 0 if no match) to a bit prediction. The probabilities are combined in the logistic domain (log(p/(1-p)) using a single layer neural network selected by a small context (3 high bits of last byte + context order), then passed through 2 SSE stages (orders 0 and 1) and arithmetic coded. Except for one model for ASCII text, there are no specialized models for binary data, .exe, .bmp, .jpeg, etc.\n\n[lpaq2](lpaq2.zip) by\nAlexander Rhatushnyak, Sept. 20, 2007, contains some speed optimizations.\n\n[lprepaq 1.2](http://schnaader.info/) by Christian Schnaader, Sept. 29, 2007,\nis lpaq1 combined with precomp as a preprocessor.  precomp compresses JPEG files\nand also expands data segments compressed with zlib, often making them more\ncompressible.  This preprocessing has no effect on text files.\n\n[lpaq3 and elpaq3](lpaq3.zip) by\nAlexander Rhatushnyak, Sept. 29, 2007, has two versions with the same source\ncode.  When compiled with\n-DWIKI, the result is elpaq3 which is tuned for large text files.  The normal\ncompile produces lpaq3.\n\n[lpaq3a](lpaq3a.zip) by\nAlexander Rhatushnyak, Sept. 30, 2007, improves compression on some files\nover lpaq3 (but not enwik8/9).  The archive also contains lpaq3e.exe, which is\nan archive compatible (Intel compile) of elpaq3.exe.\n\n[lpaq4 and lpaq4e](http://binet.com.ua/~artest/paqar/lpaq4.zip)\n([mirror](lpaq4.zip))\nare by Alexander Rhatushnyak, Oct. 1, 2007.  lpaq4e is tuned for large text files.\n\n[lpaq5 and lpaq5e](lpaq5.zip)\nare by Alexander Rhatushnyak, Oct. 16, 2007.  Option 9 selects 1542 MB memory.  \nlpaq5e is tuned for large text files.  It includes separate programs\nfor compression only (lpaq5e-c.exe) and decompression only (lpaq5e-d.exe).\nTests were done with these programs, rather\nthan the version that does both (lpaq5e.exe).\n\n[lpaq6 and lpaq6e](lpaq6.zip)\nare by Alexander Rhatushnyak, Oct. 22, 2007.  Option 9 selects 1542 MB memory.\nlpaq6e is tuned for large text files.  lpaq6 includes a E8E9 transform for\ncompressing x86 executables.\n\n[lpaq7 and lpaq7e](http://www.binet.com.ua/~artest/paqar/?M=D)\n[(mirror)](lpaq7.zip)\nare by Alexander Rhatushnyak, Oct. 31, 2007.\n\n[lpaq8 and lpaq8e](lpaq8.zip)\nare by Alexander Rhatushnyak, Dec. 10, 2007.  The executables are packed with upack.\nzip -9 would make them larger.\n\n[lpaq1a](lpaq1a.zip) by\nMatt Mahoney, Dec. 21, 2007, uses the same model as lpaq1 but replaces the\narithmetic coder with the asymmetric binary coder from fpaqb.\n\n[lpq1](lpq1.zip) by\nMatt Mahoney, Dec. 23, 2007, is an archiver (not a file compressor) based\non lpaq1 option 7.\n\n[drt|lpaq9e](http://binet.com.ua/~artest/paqar/lpaq9e.zip)\nis by\nAlexander Rhatushnyak, Feb. 20, 2008.  It is specialized for English text.\nIt includes a separate program drt.exe (without source code) which performs\na dictionary transform prior to compression with lpaq9e.  The option 9 is\nfor lpaq9e which selects maximum memory.  The program size is computed by adding\nlpaq9e.exe, drt.exe, and the compressed dictionary, which must be uncompressed\nwith lpaq9e before running.  The size is smaller without a zip archive.\nDecompression consists of uncompressing the dictionary with lpaq9e,\nuncompressing the transformed file with lpaq9e, and reversing the transform\nwith drt.  Run times are for the sum of all three operations\n(1+62+2943, 1+2929+45 sec).\n\n[lpaq9f](http://binet.com.ua/~artest/paqar/lpaq9f.zip) by\nAlexander Rasushnyak, Apr. 27, 2007, works like lpaq9e.  Run times are\n(2+55+2801, 2+2819+38 sec). drt uses 8 MB for compression and 4 MB\nfor decompression.\n\n[lpaq9g](http://binet.com.ua/~artest/paqar/lpaq9g.zip) by\nAlexander Rasushnyak, May 23, 2008, works like lpaq9e.  Run times are\n(2+51+2691, 2+2682+38 sec).  \n\n[lpaq9h](http://binet.com.ua/~artest/paqar/lpaq9h.zip) by\nAlexander Rasushnyak, June 3, 2008, works like lpaq9e.  Run times are\n(2+53+2530, 2+2529+44 sec).\n\nlpaq9i by Alexander Rasushnyak, June 13, 2008, works like lpaq9e. Run times are (2+59+2425, 2+2453+46 sec). drt.exe and the dictionary file (tmpdict0.dic) are unchanged in all versions starting with lpaq9f.\n\n[lpaq9j](http://www.binet.com.ua/~artest/paqar/lpaq9j.zip)\nby Alexander Rhatushnyak, Aug. 17, 2008, has a new version of drt.exe and\ndictionary. Run times are (2+58+2365, 2+2358+48 sec).\n\n[lpaq9k](http://www.binet.com.ua/~artest/paqar/lpaq9k.zip)\nis by Alexander Rhatushnyak, Sept. 30, 2008. Run times are (2+59+2336,\n2+2346+47 sec). decompresser size is as 3 files (not zipped).\n\n[lpaq9l](http://www.binet.com.ua/~artest/paqar/lpaq9l.zip)\nis by Alexander Rhatushnyak, Dec. 2, 2008. Run times are (2+41+2132,\n2+2179+40 sec) on the computer described in note 26, and\n(2+58+2338, 2+2422+50) on the computer used to test all the earlier versions.\ndecompresser size is as 3 files (not zipped).\n\n[lpaq9m](http://artst.narod.ru/b/lpaq9m.zip)\n[(zpaq archive)](lpaq9m.zpaq)\nis by Alexander Rhatushnyak, Feb. 20, 2009. Run times are\n(2+38+2067, 2+2111+38). decompresser size is 3 files (not zipped).\n\n[decomp8](decomp8.zpaq) is a Hutter\nPrize entry by Alexander Rhatushnyak, Mar. 23, 2009. It consists of a\ndecompresser (Windows executable only) and an archive (archive8.bin) which\ndecompresses to enwik8. There is no compressor. During decompression, the\nprogram creates a temporary file containing a dictionary similar to the one\nused in paq8hp12 and by drt. The command to decompress is \"decomp8 archive8.bin enwik8\".\nThe total size (not zipped) is 15,986,677 bytes.\n\n[decomp8b](decomp8b.zpaq) is\nan update to the Hutter prize entry\ndecomp8 by Alexander Rhatushnyak, Apr. 22, 2009. Total size\n(not zipped) is 15,958,674 bytes.\n\n[decmprs8](decmprs8.zpaq) is\nan update to the Hutter prize entry\ndecomp8b by Alexander Ratushyak, May 23, 2009. Total size\n(not zipped) is 15,949,688 bytes. To decompress: decmprs8.exe archive8.dat enwik8\n\n```\nProg       Opt     enwik8      enwik9         prog       Total       Comp  Deco Mem  Alg Note\n----       ---   ----------  -----------      ----     -----------   ----  ---- ---- --- ----\nlpaq1       9    19,755,948  164,508,919      6,676 x  164,515,595   3646  3594 1539 CM\nlpaq2       9    19,755,471  164,496,295      6,888 x  164,503,183   3260  3354 1539 CM\nlprepaq 1.2 9    19,755,989  164,509,300    189,891 x  164,699,191   8696  7888 1582 CM\nlpaq3       9    19,580,276  165,600,121      7,514 x  165,607,635   3695  3735 1542 CM\nelpaq3      9    19,392,604  160,081,507      7,377 x  160,088,884   3411  3454 1542 CM\nlpaq3a      9    19,585,951  165,661,890     12,004 s  165,673,894   4177  4163 1542 CM\nlpaq3e      9    19,392,604  160,081,507     12,004 s  160,093,511   3967  3932 1542 CM\nlpaq4       9    19,583,905  165,603,612      7,117 x  165,610,729   3693  3697 1542 CM\nlpaq4e      9    19,358,662  159,675,213      6,990 x  159,682,203   3383  3422 1542 CM\nlpaq5       9    19,455,395  161,410,276      8,382 x  161,418,658   3614  3630 1542 CM\nlpaq5e      9    19,078,767  156,194,860      7,841 xd 156,202,701   3428  3605 1542 CM\nlpaq6       9    19,562,861  165,224,012      8,848 x  165,232,860   3586  3624 1542 CM\nlpaq6e      9    19,054,076  155,943,020      8,866 x  155,951,886   3420  3478 1542 CM\nlpaq7       9    19,557,894  162,359,435      9,078 x  163,368,513   3922  3850 1542 CM\nlpaq7e      9    19,039,516  155,840,757      8,570 x  155,849,327   3477  3490 1542 CM\nlpaq8       9    19,523,803  161,987,713      9,676 x  161,997,389   3682  3718 1542 CM\nlpaq8e      9    18,982,007  155,232,477      8,888 x  155,241,365   3424  3475 1542 CM\nlpaq1a      9    19,759,778  164,547,926      8,558 x  164,556,484   3462  3423 1540 CM\nlpq1             19,888,399  168,467,267      9,151 x  168,476,408   3389  3402  387 CM\ndrt|lpaq9e  9    18,151,024  145,628,635    110,844 x  145,739,479   3006  2975 1542 CM\ndrt|lpaq9f  9    18,079,247  144,877,844    110,864 x  144,988,708   2858  2859 1542 CM\ndrt|lpaq9g  9    18,069,107  144,838,636    110,318 x  144,948,954   2744  2722 1542 CM\ndrt|lpaq9h  9    18,067,711  144,763,248    110,376 x  144,873,624   2585  2575 1542 CM\ndrt|lpaq9i  9    18,065,347  144,752,858    110,149 x  144,863,007   2486  2501 1542 CM\ndrt|lpaq9j  9    18,056,997  144,687,646    110,135 x  144,797,781   2425  2408 1542 CM\ndrt|lpaq9k  9    18,007,677  144,277,379    110,785 x  144,388,164   2397  2395 1542 CM\ndrt|lpaq9l  9    17,979,724  144,082,479    110,479 x  144,192,958   2398  2474 1542 CM\ndrt|lpaq9l  9    17,979,724  144,082,479    110,479 x  144,192,958   2175  2221 1542 CM  26\ndrt|lpaq9m  9    17,964,751  143,943,759    110,579 x  144,054,338   2107  2151 1542 CM  26\ndrt|lpaq9m  9    17,964,751  143,943,759    110,579 x  144,054,338    868   896 1542 CM  41\ndecomp8          15,970,425                  16,252 xd                    78180  936 CM  26\ndecomp8b         15,942,290                  16,384 xd                    74790  934 CM  26\ndecmprs8         15,932,968                  16,720 xd                    76080  936 CM  26\n```\n\ndrt may be combined with other compressors to improve compression. The following were obtained using drt and tmpdict0.dic (from lpaq9i) with ppmonstr J (PPM). Option -m1650 selects 1650 MB memory. -r1 partially rebuilds the model when memory is exhausted. -o select the PPM model order. Compression time is for ppmonstr only. Mem8 is actual memory used to compress enwik8.drt. enwik9.drt always uses 1650 MB. As a separate compressor, the compressor size would be 147,915 for a zip file containing drt.exe, ppmonstr.exe, and tmpdict0.pmm (tmpdict0.dic compressed with ppmonstr -m1650 -r1 -o64). Total size would be 148,047,289.\n\nFor drt 9j, the decompresser size is 149,468 and total size is 147,196,757.\n\n```\n    Compressors          options         enwik8    enwik9       Comp Mem8\n-------------------  ----------------  ----------  -----------  ---- ----\ndrt 9i | ppmonstr J  -m1650 -r1 -o10   18,185,633  147,936,682  2509  825\n                     -m1650 -r1 -o11   18,166,961  147,899,374  2634  895\n                     -m1650 -r1 -o12   18,152,982  147,907,628  2661  953\n                     -m1650 -r1 -o16   18,142,625  148,306,179  2888 1109\n                     -m1650 -r1 -o32   18,124,722  149,857,650  3361 1371\n                     -m1650 -r1 -o64   18,122,785  151,343,426  3870 1554\n                     -m1650 -r1 -o128  18,130,333                    1650\ndrt 9j | ppmonstr J  -m1650 -r1 -o11   18,165,440  147,859,151  2636\n                     -m1650 -r1 -o64   18,120,770               2603\n```\n\nThe following shows the effects of drt from lpaq9m on enwik8. The first numeric column is the compressed size of enwik8. The second is the compressed size of the uncompressed dictionary (lpqdict0.dic, 465,210 bytes) concatentated with enwik8.drt (61,289,634 bytes) using compressor versions that were current as of June 26, 2010 unless indicated. The ratio shows the improvement due to preprocessing. The dictionary contains 44880 lowercase words. DRT replaces word occurrences with codes of 1 to 3 bytes and uses codes to indicate capitalized words or letters.\n\n```\nCompressor    enwik8   dic+drt   ratio   Options (version)\n----------   -------   --------  ------  -----------------\npaq8px_v67   18293940  17342041  0.9480  -6\npaq8l        18518485  17560378  0.9483  -6\nnanozip      18826931  18633832  0.9897  -cc (v0.08a)\nlpaq9m       19072743  18077356  0.9478  8\nzpaq         19448650  18928856  0.9733  ocmax.cfg\npmm          19701161  18650601  0.9467  (J)\nlpaq1        19796957  18905483  0.9550\npaq9a        20129573  19374291  0.9625\npaq6         20303336  19439547  0.9575  -6\ncmm4         20548514  19133313  0.9311  (v0.1e)\nzpaq         20941558  19447733  0.9287  ocmid.cfg\nnz           20948832  20588807  0.9828  (v0.08a)\nbwt.fpaq0f2  21798843  21406906  0.9820\npaq1         22156982  21437426  0.9675\nbwt.fpaq0p   23809591  22855730  0.9599\ngrzip        23846878  22379326  0.9385  (0.2.4)\nbbb          24576921  22701384  0.9237\nzpaq         24837469  21559014  0.8680  ocfast.cfg\ntarsalzp     25134862  22773386  0.9060\nlzpxj        25251404  21877402  0.8664  8 (1.2h)\np6           25377998  23078246  0.9094\nctw          25453025  24454785  0.9608\n7z           25895909  23487746  0.9070  (9.12b)\nszip         26120472  24045552  0.9206  -b41 -o16\nppmd         26275353  23448205  0.8924  (J)\nppms         26310248  23824677  0.9055  (J)\ndmc          28402672  25532850  0.8990  100000000\ncabarc       28465607  25963613  0.9121  -m lzx:21\nbzip2        29008758  25612712  0.8829  -9\nsr2          30432506  26328768  0.8652\nRAR          35107917  30132497  0.8583  -m5 (v2.50)\nHA           36379137  30633820  0.8421  (0.98)\ngzip         36445248  30902821  0.8479  -9 (1.3.5)\nzip          36445470  30903043  0.8479  -9 (2.32)\nlzop         41217688  33358696  0.8093  -9 (1.01)\nsrank        43091439  38492535  0.8933  -C8\nfcm1         45402225  29581661  0.6515\ncompress     45763941  37478724  0.8190\nlzrw3-a      48009194  38635335  0.8047\nbpe          53906667  41403271  0.7681  5000 4096 200 3\nfastlz       54658924  42337322  0.7746\nlzrw2        55360907  41854974  0.7560\nfpaq0f2      56916872  40415334  0.7101\nflzp         57366279  43944882  0.7660\nlzrw5        59375192  46019812  0.7751\nlzrw1-a      59471657  43184084  0.7261\nfpaq0p       61457810  44979267  0.7319\nppp          61657971  44103741  0.7153\nfpaq0        63391013  47589951  0.7507\n            100000000  61289634  0.6129  (uncompressed)\nbwt         100000004  61289638  0.6129  (msufsort 3.1b)\n```\n\n[mcm v0.0](http://encode.su/threads/1732-New-CM-compressor-in-development?p=33358&viewfull=1#post33358) is a free, experimental, closed source file compressor\nby Mathieu Chartier, June 4, 2013. It uses CM. Options -1 ... -9 select\n8 MB to about 1500 MB memory.\n\n[mcm v0.2](http://encode.su/threads/1732-New-CM-compressor-in-development?p=33480&viewfull=1#post33480), June 11, 2013,\nhas automatic detection of text and binary files with UTF modeling\nin text mode and sparse models in binary mode, an improved match model,\nand cache optimizations.\n\n[mcm v0.3](http://encode.su/threads/1732-New-CM-compressor-in-development?p=33536&viewfull=1#post33536) was released June 17, 2013.\n\n[mcm 0.4](https://github.com/mathieuchartier/mcm) was released\nas open source on July 17, 2013. To test, it was compiled with g++ 4.8.0 using\nthe supplied make.bat file.\n\n[mcm 0.8](https://github.com/mathieuchartier/mcm)\n[(discussion)](http://encode.su/threads/2127-MCM-LZP?p=42513&viewfull=1#post42513), was released Feb. 5, 2015. It uses LZP preprocessing\nwith fast and high modes. The high mode (default, as tested)\nuses 8 context models and the fast uses 6. It was compiled in Linux/g++ 4.8.2\nusing the supplied make.bat file. Option -10 uses 2.9 GB memory.\nOption -11 (5.5 GB) was not tested.\n\n[mcm 0.82](http://encode.su/threads/2127-MCM-LZP?p=42648&viewfull=1#post42648) was released Feb. 16, 2015. -max selects best compression\n(default is -high).\n\n[mcm 0.83](http://encode.su/threads/2127-MCM-LZP?p=43220&viewfull=1#post43220) was released Apr. 5, 2015. -x10 and -x11 select the memory used\nfor max compression. To test -x10, I compiled from source using the supplied make.sh\nin Ubuntu, g++ 4.8.2. -x11 was tested using\n[optimized source](http://encode.su/threads/2127-MCM-LZP?p=43403&viewfull=1#post43403) with comments removed.\n\n```\n        Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram  Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------  -------    ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nmcm v0.0   -9       19,842,740  166,276,589    116,198 x  166,392,787   1425  1449 1447 CM   26\nmcm v0.2   -9       19,768,502  165,480,329    137,308 x  165,617,637   1453  1468 1451 CM   26\nmcm v0.3   -9       19,707,487  164,464,527    122,205 x  164,586,732   1387  1435 1452 CM   26\nmcm v0.4            19,858,418                                          1718  1623  735 CM   26\n           -9       19,762,418  165,009,983     43,479 s  165,053,462   1552  1494 1457 CM   26\nmcm v0.8   -10      19,434,824  158,421,558    107,815 s  158,529,373    710   727 2901 CM   48\nmcm v0.82  -10      19,281,673  157,501,519    111,681 s  157,613,200    636   642 2854 CM   48\n           -10 -max 19,173,407  156,544,880    111,681 s  156,656,561    692   689 2855 CM   48\nmcm v0.83  -x10     18,286,400  146,525,446    127,052 s  146,652,498    706   569 3056 CM   48\n           -x11     18,233,295  144,854,575     79,574 s  144,934,149    394   281 5961 CM   72\n```\n\n[nanozip](http://www.nanozip.net/) 0.01a is a free, experimental,\nclosed source GUI and command line archiver by Sami Runsas, July 14, 2008.\nFor these tests, the command line version (smaller executable) was used. It compresses\nusing several algorithms (fastest to best): LZP (options -cf and -cF), LZ77\n(-cd, -cD), BWT (-co, -cO, uses 5N block size)\nand CM (-cc). The uppercase options (-cF, -cD, -cO) compress\nbetter but slower than the corresponding lowercase options and may use more memory.\nThe default compression mode is -co (fast BWT).\n-m1500m selects 1500 MB memory, although the reported memory usage may differ and\nthe actual memory usage (Cmem, Dmem, in MB)\nmeasured with Task Manager is usually lower than reported.\nThe program will use less memory depending on available physical memory when run.\n-forcemem was used to override this.\nFor all tests, -nm was used to turn off checksums and not store timestamps or file\npermissions. For -cO, the program uses a LZ77 variant (called LZT)\ninstead of BWT for binary files. -txt is an optimization for text files with -co or -cO.\n\nnanozip 0.03a was released July 31, 2008. Only -cc was tested.\n\nnanozip 0.05a was released Oct. 20, 2008. Options are as in 0.01a and include -nm -forcemem.\n\nnanozip 0.06a was released Feb. 13, 2009. Options are as in 0.01a and include -nm -forcemem. w32c creates a self extracting archive (.exe file).\n\nnanozip 0.08a was released June 3, 2010. _64 refers to the Windows 64 bit version. w32c means to produce a self extracting archive. -nm means do not store metadata or redundancy information. -cc selects a context mixing model. -m2.6g means use 2.6 GB memory. enwik8 was tested with -m2g (uses 1670 MB).\n\n[nanozip 0.09a](nz.exe) was released Nov. 4, 2011. Option w32c selects a self extracting\narchive, so the decompresser size is 0. Option -p4 runs multithreaded compression\non 4 processors. Tested under 64 bit Linux.\n\n```\nProgram       Options                enwik8      enwik9     zip size      Total     Comp  Deco  Cmem Dmem (reported) Alg  Note\n--------    -----------            ----------  -----------  ---------  -----------  ----  ----  ---- ---- ---- ----  ---  ----\nnz 0.01a    -cf                    46,381,713                                         24    24    96       404  404  LZP\n            -cf -m1500m            46,381,713  417,351,980  266,797 x  417,618,777    26    31   975  978 1476 1476  LZP\n            -cF                    40,733,125                                         62    43   155       404  404  LZP\n            -cF -m1500m            40,733,125  359,192,720             359,459,517    63    40  1040 1045 1476 1476  LZP\n            -cd                    33,241,150                                        127    28    89       422  402  LZ77\n            -cd -m1500m            33,001,952  292,180,617             292,447,414   156    28   768  687 1546 1474  LZ77\n            -cD                    29,384,997                                        288    27   282       466  258  LZ77\n            -cD -m1500m            29,253,158  258,513,190             258,779,987   323    31  1020  693 1314  994  LZ77\n            -co                    21,838,721                                        391   186   333       431  336  BWT\n            -co -m1500m            20,503,629  176,470,974             176,737,771   448   221  1667 1160 1810 1294  BWT\n            -co -m1500m -txt       20,503,629  170,711,387             170,978,184   336   234  1074 1120 1471 1463  BWT\n            -cO                    21,623,801                                        465   247   333       431  266  BWT\n            -cO -m1500m            20,306,489  174,770,662             175,037,459   511   269  1378 1135 1810 1294  BWT\n            -cO -m1500m -txt       20,306,489  169,092,652             169,359,449   393   280  1074 1274 1471 1463  BWT\n            -cO -m1670m -txt       20,306,489  167,509,921             167,776,718   403   284  1170 1325 1633 1625  BWT\n            -cc                    18,994,349                                       2975  2910   360       436  435  CM\n            -cc -m1500m            18,723,413  152,654,332             152,921,129  3147  3091  1556 1556 1524 1523  CM\nnz 0.03a    -cc -m1670m            18,679,094  151,668,563  263,953 x  151,932,516  3058  3003  1700 1700 1700 1699  CM\nnz 0.05a    -cf -m1670m            46,381,713                                         18    22   100                 LZP\n            -cF -m1670m            40,608,638                                         66    41   164                 LZP\n            -cd -m1670m            31,555,257                                         96    29   289                 LZ77\n            -cD -m1670m            27,811,031                                        182    35   170                 LZ77\n            -co -m1670m            20,499,411                                        351   177   626                 BWT\n            -cO -m1670m            20,302,501                                        422   240   642                 BWT\n            -cc -m1670m            18,638,419  151,176,555  288,449 x  151,465,004  3032  2975  1668                 CM\nnz 0.06a    -co -m1670m            20,499,412                                        250   183   441                 BWT   26\n            -cO -m1670m            20,302,502                                        300   243   457                 BWT   26\n            -cc -m1670m            18,636,515  151,177,510  336,273 x  151,513,783  2143  2137  1670                 CM    26\n            w32c -cc -m1670m       18,754,787  151,295,782        0 xd 151,295,782  2156  2173  1670                 CM    26\nnz 0.08a_64 w32c -nm -cc -m2.6g    18,752,842  150,441,103        0 xd 150,441,103  1109  1086  2760                 CM    40\n            -cc -m2g               18,623,317  150,375,385  459,607 x  150,834,992  1616        2088                 CM    42\nnz 0.09a    w32c -cc -m3g -nm      18,723,846  150,037,341        0 xd 150,037,341  1110  1084  2693                 CM    40\n            w32c -cc -m3g -nm -p4              158,107,738        0 xd 158,107,738   299        3124                 CM    40\n            -cc -m32g -p1 -t1 -nm  18,594,163  148,545,179  783,642 x  149,328,821  1149  1141 32000     13285 13282 CM    74\nProgram         Options              enwik8      enwik9     zip size      Total      Comp   Deco  Cmem Dmem  Alg  Note\n--------        -----------        ----------  -----------  ---------  -----------   ----   ----  ---- ---- ----  ----\ncmv 00.01.00    -m2,3,+            18,218,283  150,226,739   77,404 x  150,304,143 285750 293090  2817 2817   CM   48,75\n                                               150,226,739   77,404 x  150,304,143 216000         2801        CM   75\n                -m2,3,0x03ededff   18,153,319                                      720000        ~3900        CM   75\ncmv 00.01.01    -m2,3,0x03ed7dfb   18,122,372  149,357,765   77,404 x  149,435,169 426162 394855  3335 3335   CM   75\ncmve 0.2.0      -m2,3,0x7fed7dfd   16,424,248  129,876,858  307,787 x  130,301,106 1140801       19963        CM   81\n```\n\n[xml-wrt 2.0](http://www.ii.uni.wroc.pl/~inikep/research/XML/XML-WRT20.zip)\nis a free command line file compressor with source available, by Przemyslaw Skibinski,\nJune 19, 2006.  It uses LZMA (LZ77 + arithmetic coding) with preprocessing for modeing text,\nXML tags, dates, and numbers.  It may also be used as a preprocessor for input\nto other compressors.  Version 1.0 was strictly a preprocessor without built-in compression.\n\nThe -l6 option selects maximum LZMA compression. -b255 selects maximum buffer size of 255 MB for building a dynamic dictionary. -m255 selects maximum memory. -s turns off spaces modeling. -f8 sets the minimum word frequency for dictionary inclusion to 8 (default is 6).\n\n[xml-wrt 3.0](http://sourceforge.net/project/showfiles.php?group_id=176333)\n(Sept. 14, 2006)\nincludes a stripped-down version of PAQ8 (-l11 option) in addition to LZMA compression.\n\n[xwrt 3.2](http://sourceforge.net/project/showfiles.php?group_id=176333)\n(Oct. 29, 2007) is a dictionary preprocessor frontend to LZMA, PPMVC and lpaq6 as\nwell as a standalone preprocessor.  Option -l14 selects lpaq6 option 9 (1542 MB).\n-b255 selects 255 MB memory (maximum) for building the dictionary.  -m96 selects\n96 MB buffer during compression.  (Higher values cause out of memory error).\n-s turns of space modeling.  -e40000 limits the dictionary size to 40000 words.\n-f200 limits the dictionary to words that occur at least 200 times.\n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nxml-wrt 2.0  -l6 -b255 -m255 -s -f8           23,199,202  196,914,328     25,354 s  196,939,682    905    70  525 LZ77\nxml-wrt 3.0  -l11 -b255 -m255 -f24            19,663,305  165,274,422     40,447 s  165,314,869   4398  4317  416 CM\nxwrt 3.2     -l14 -b255 -m96 -s -e40000 -f200 18,679,742  151,171,364     52,569 s  151,223,933   2537  2328 1691 CM\n```\n\nxml-wrt 2.0 and higher and xwrt 3.2\ncan be used as either a standalone compressor or as a preprocessor to other compressors.\nThe table below shows the best known settings for enwik9 and enwik8 for xml-wrt 3.0 and 2.0 as\na preprocessor to ppmonstr var. J, the best known combination for which xml-wrt improves compression.\nxml-wrt 1.0 is a preprocessor only.\nSee also [xml-wrt and xwrt as a standalone compressor](#1512).\n\n```\n                                                                         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram/options                                                         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------------------------------------------------------------------   ----------  -----------  -----------  -----------  ----- -----  --- ---\nxml-wrt 3.0 -l0 -b255 -m255 -3 -s -e20000    | ppmonstr J -m1650 -o10 18,592,499  150,004,636     82,466 sx 150,087,102   3067  2708 1650 PPM\nxml-wrt 3.0 -l0 -b255 -m255 -3 -s -e7000     | ppmonstr J -m1650 -o64 18,494,374                  82,466 sx               3500  3340 1650 PPM\nxml-wrt 2.0 -l0 -w -s -c -b255 -m100 -e10000 | ppmonstr J -m1700 -o10 18,794,295  150,651,873     67,309 sx 150,719,182   2715 ~2650 1700 PPM\nxml-wrt 2.0 -l0 -w -s -c -b255 -m100 -e2300  | ppmonstr J -m1650 -o64 18,625,624                  67,309 sx               3550  3360 1650 PPM\nxml-wrt 2.0 -l0 -w -s -c -b255 -m100 -e10000 | ppmonstr J -m800 -o8   18,863,790  154,223,582     67,309 sx 154,290,891   2820        800 PPM\nxml-wrt 1.0 -f800                            | ppmonstr J -m800 -o8   19,043,178  154,749,585     56,837 sx 154,806,422   2702 ~2700  800 PPM\n```\n\n[xml-wrt 1.0](http://www.ii.uni.wroc.pl/~inikep/research/XML/XML-WRT10.zip)\n(XML Word Reducing Transform)\nis a free command line single file preprocessor with source code\nby Przemyslaw Skibinski, May 10, 2006.\nIt is not intended to compress files by itself (although it does somewhat).\nRather, it is intended to improve the compressibility of text and XML files by replacing\ncommon words and XML substrings with shorter symbols.  (So it is actually LZW with a\nstatic dictionary prepended to the output).  \nIt improves compression for most programs except for those\nthat already have English text models such as paq8h.  Some additional results\nare shown below for combinations with some other compressors.\n\n```\n                     Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Notes\n-------                -------                     ----------  -----------  -----------  -----------  ----- -----  -----\nxml-wrt 1.0|ppmonstr J -f1800 | -m800 -o10         18,965,658  155,066,074     56,837 sx 155,122,911   2905  2809\nxml-wrt 1.0|slim23d    -f1800 | -m700 -o12         19,163,987  156,734,571     69,453 x  156,804,024   4702  4717\nxml-wrt 1.0|ppmd J1    -f1800 | -m256 -o8 -r1      21,128,019  178,154,529     25,917 s  178,180,446    717   722\n```\n\nThe following table shows the compressed size (without decompresser except SFX) of enwik8 before and after the XML-WRT transform with option -f180 for several compressors. A ratio less than 1 means that XML-WRT improves compression.\n\n```\nProgram           Options                       enwik8   enwik8.xwrt  Ratio   Alg\n-------           -------                    -----------  ----------  ------  ---\npaq8h             -7                          17,674,700  18,341,959  1.0378  CM\nppmonstr J        -o10 -m800                  19,338,065  18,886,224  0.9766  PPM\nslim23d           -m700 -o10                  19,264,094  18,938,602  0.9830  PPM\nWinUDA 2.91       mode 3 (194 MB)             20,332,366  20,859,165  1.0259  CM\nppmd J1           -o10 -m256 -r1              21,388,296  20,945,220  0.9793  PPM\nuhbc 1.0          -m3 -b100m                  20,930,838  21,171,204  1.0115  BWT\nM03exp            32 MB                       21,948,192  21,583,059  0.9834  BWT\nsbc               -ad -m3 -b63                22,470,539  22,216,425  0.9887  BWT\nWinRAR 3.60b3     -mc7:128t+ -sfxWinCon.sfx   22,713,569  22,457,785  0.9887  PPM\nPX 1.0                                        24,971,871  22,818,070  0.9137  CM\nuharc 0.6b        -mx -md32768                23,911,123  22,915,299  0.9583  PPM\nchile 0.3d-1      -b=40000                    23,408,335  22,884,519  0.9776  BWT\ncabarc 1.00.0601  -m lzx:21                   28,465,607  25,739,214  0.9042  LZ77\nWinACE            -sfx -m5                    30,919,182  27,112,651  0.8769\nbzip2 1.0.3                                   29,008,758  27,339,845  0.9425  BWT\ngzip 1.3.5        -9                          36,445,248  30,403,738  0.8342  LZ77\npkzip 2.0.4                                   36,934,712  30,729,525  0.8432  LZ77\nthor 0.9a         ex                          41,670,916  32,586,444  0.7820\ncompress 4.3d                                 45,763,941  38,485,494  0.8409  LZW\nOriginal size                                100,000,000  52,174,989  0.5217\n```\n\nThe -f option (default -f6) selects the minimum word frequency required to have it added to the dictionary. The optimal setting depends on the input size. When used with ppmd or ppmonstr (the best compressors improved by XML-WRT), the optimal settings are about -f180 for enwik8 and -f1800 for enwik9, which results in a dictionary of 7697 words for enwik8 and 6657 words for enwik9. The following table shows the effect of the -f and -o options for ppmonstr -m800 enwik9. The best combination found is -f1800 -o8.\n\n```\n -f       -o7          -o8          -o9          -o10        -o11         -o12         -o16         -o32\n ---  -----------  -----------  -----------  -----------  -----------  -----------  -----------  -----------\n 100                                                                   155,908,621\n 200                                                                   155,775,164\n 300                                                                   155,653,815\n 500               154,884,542               155,367,681  155,465,355  155,547,660\n 600               154,787,455                                         155,497,645\n 800               154,749,585\n1000  154,909,136  154,794,501  154,951,751  155,122,278  155,306,526  155,409,926  155,948,066  157,901,320\n1500  155,092,513  154,895,455  154,999,654  155,073,186  155,306,526  155,301,322\n1800  155,191,178  154,924,936  155,036,534  155,066,074  155,366,281  155,297,828\n2000               154,998,528                                         155,296,112\n3000                                                                   155,379,959\n```\n\nThe following table shows that the optimal setting for -f is lower for smaller files (with ppmd):\n\n```\n              Compression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  \n-------         -------         ----------  -----------  -----------  -----------  ----- -----  \n  xml-wrt 1.0   -f1800         (70,826,140)(532,089,443)   (14,818 s)(532,104,261)  (115) (103)\n+ ppmd J        -m256 -o8 -r1   21,128,019  178,154,529     41,653 sx 178,196,182    712   723\n  xml-wrt 1.0   -f180          (52,174,989)(468,964,104)   (14,818 s)(468,978,922)  (113) (103)\n+ ppmd J        -m256 -o8 -r1   20,910,527  178,215,315     41,653 sx 178,256,968    690   699\nppmd J          -m256 -o10 -r1  21,388,296  183,964,915     26,835 x  183,991,750    880   895\n```\n\nThe default values of -s (disable spaces model) and -t (disable try smaller word) appear to work best on this data.\n\n```\nxml-wrt -f1800 enwik9 | ppmonstr -m800 -o12\n-------------------------------------------\n(default)   154,924,936\n-s          155,040,558\n-t          155,421,035\n-s -t       155,542,575\n```\n\n[xml-wrt 2.0](http://www.ii.uni.wroc.pl/~inikep/research/XML/XML-WRT20.zip)\nreleased June 14, 2006 (updated June 19, 2006)\nhas additional transform options, and also includes LZ77 (zlib)\nand LZMA (LZ with arithmetic coding) compression.  When used as a preprocessor,\nthis compression is turned off.  enwik9 was compressed using the options:\n\n```\n  xml-wrt -l0 -w -s -c -b255 -m100 -e10000 enwik9\n  ppmonstr e -o8 -m800 enwik9.xwrt\n```\n\nThe option -l0 turns off compression. -w turns off word containers. -s turns off space modeling (this hurts compression in version 1.0 but helps in 2.0). -c turns off word and number containers (independent of -w and -n. -n hurts compression). -b255 sets memory for the dictionary to 255 MB, the maximum. -m100 sets the memory buffer to 100 MB, which is not maximum (255 MB), but larger values hurt compression. -e10000 sets the dictionary size to 10000 words. (The dictionary size can also be controlled with -f as in version 1.0, but using -e is less dependent on input size so it helps with enwik8). Additional tests showing the effects of -e, -m, and -o:\n\n```\nxml-wrt 2.0 options                ppmonstr J     enwik9\n--------------------------------   ----------   -----------\n-l0 -w -s -c -b255 -m100 -e10000 | -m800 -o8    154,223,582\n-l0 -w -s -c -b255 -m100 -e8000  | -m800 -o8    154,234,621  (smaller -e)\n-l0 -w -s -c -b255 -m100 -e12000 | -m800 -o8    154,239,769  (larger -e)\n-l0 -w -s -c -b255 -m50  -e10000 | -m800 -o8    154,259,117  (smaller -m)\n-l0 -w -s -c -b255 -m100 -e10000 | -m800 -o7    154,322,272  (smaller -o)\n-l0 -w -s -c -b255 -m150 -e10000 | -m800 -o8    154,426,554  (larger -m)\n-l0 -w -s -c -b255 -m100 -e10000 | -m800 -o9    154,445,811  (larger -o)\n```\n\nThe optimal values of -w -c -s -n (turn off number containers) and -t (turn off try shorter words) was determined on enwik7 and enwik8 but not tested on enwik9.\n\nA bug fix for LZMA compression, released June 19, 2006, does not change any values for the June 14, 2006 version (using the -l0 option). However the compressed source code increases from 25,290 bytes to 25,354 bytes. The June 14 version is no longer published. The URL is unchanged.\n\n[xml-wrt 3.0](http://sourceforge.net/project/showfiles.php?group_id=176333)\n(Sept. 14, 2006) option -3 means to optimize the default settings for PPM compressors.\nVersion 3.0 also has a FastPAQ8 compressor for standalone compression\nwhich was tested separately.\n\nxwrt 3.2 (see below) with ppmonstr J has the following results.\n\n```\nxwrt 3.2 options        ppmonstr J opt    enwik8      enwik9        program size      total        Comp    Decomp   Mem\n----------------------  --------------  ----------  -----------  -----------------  -----------  --------  ------- ----\n-2 -b255 -m255 -s -f64   -o10 -m1650    18,456,706  148,915,761  52,569s + 26,835x  148,995,165  475+2512  43+2503 1650\n-2 -b255 -m255 -s -f64   -o64 -m1650    18,397,126                                               210+2810  50+2884 1527\n```\n\nppmonstr option -o64 is optimal for enwik8, but -o10 is optimal for enwik9.\n-m1650 selects 1650 MB memory.\nxwrt option -2 optimizes for PPM. -b255 selects buffer size 255 MB for building\nthe dictionary. -m255 selects 255 MB memory buffer. -s turns off space modeling.\n-f64 sets minimum word frequency for the dictionary to 64.  Program size and\ntimes are xwrt + ppmonstr.  Memory usage is 512 MB for xwrt, 1650 MB for ppmonstr.\n \n\n```\nProgram   options    enwik8      enwik9    program size      total    Comp  Decomp Mem  Alg Note\n-------   -------  ----------  ----------  ------------  -----------  ----- ------ ---- --- ----\nfp8 v1       -8    18,573,126                  49,865 s               20010        1150 CM   26\nfp8 v2       -8    18,556,327  154,359,664     49,964 s  154,409,626  19059  21196 1192 CM   26\nfp8 v3       -8    18,438,169  153,188,176     50,068 s  153,238,244  20605  22593 1192 CM   26\ntangelo 1.0        18,593,738  156,355,536      8,365 s  156,363,901  19849  19977  567 CM   26\ntangelo 2.0        20,202,547  171,678,313      6,275 s  171,684,588   6028   6007  362 CM   26\ntangelo 2.1        21,021,150  179,879,607     11,320 s  179,890,927   2275   2262  361 CM   26\ntangelo 2.3        20,921,619  178,497,116     11,687 s  178,508,803   2172   2194  361 CM   26\nCompression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp Mem   Alg  Notes\n-------           -------                     ----------  -----------  -----------  -----------  ----- ----- ----  ---  -----\nWinRK 3.03        PWCM (800MB +td)            18,612,453  156,291,924  3,017,362 x  159,309,286  68555             CM   10\nWinRK 3.03        PWCM                        18,612,551  156,349,910  3,017,362 x  159,367,272 102973~90000       CM    9\nWinRK 3.03        FPW1 (800MB +td)            19,035,564                                         24950                  10\nWinRK 3.03        PWCM (800MB -td)            19,060,620                                         88310             CM   10\nWinRK 3.03        Efficient                   21,157,165                                          5380             PPM  10\nWinRK 3.03        Normal (PPMd)               22,322,981                                           620             PPM  10\nWinRK 3.03        PWCM (800MB +td)            18,612,453  156,291,924     99,665 xd 156,391,589  68555        800  CM   10\nWinRK 3.03 x64    PWCM (2047MB +td o28)       18,101,637  150,481,300                                        3053  CM   42\n```\n\n RK and RKC are predecessors of WinRK so I don't plan to test them.\n ppms accepts only options -o2 through -o8. The default is -o5. This also gives\nthe best compression on enwik8. Task Manager shows 1.8 MB memory used.\n \n\n```\n              Compression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Note\n-------         -------         ----------  -----------  -----------  -----------  ----- -----  ----  \nppmonstr J      -m1700 -o16     19,055,092  157,007,383     42,019 x  157,049,402   3574 ~3600\nppmonstr J      -m800 -o16      19,230,657  161,496,685     42,019 x  161,538,704   3783 ~3800\nppmonstr J      -m1863 -o16     19,040,451  156,578,769     42,019 x  156,620,788               42\nppmd J          -m256 -o10 -r1  21,388,296  183,964,915     11,099 s  183,976,014    880   895\nppmd J          -m10 -o4 -r0    26,275,353  236,509,791     11,099 s  236,520,890    194   206\nppms J          -o5             26,310,248  233,442,414     16,467 x  233,458,881    330   354\n                -o2             36,866,748                                           102\n                -o3             30,242,535                                           135\n                -o4             27,030,761                                           246\n                -o6             26,644,863                                           449\n                -o7             27,028,318                                           492\n                -o8             27,343,283                                           532\n```\n\n ppmd was updated to J1 on May 10, 2006 to fix a bug.  Compression benchmarks are unchanged \nexcept the size of the compressor (11,099 bytes as zipped source code).  \nppmonstr is unchanged.\n zcm v0.02 was released Dec. 23, 2011.\n zcm v0.03 was released Dec. 28, 2011.\n zcm v0.04 was released Jan. 30, 2012. (Program banner says v0.03).\n zcm 0.30 was released May 2, 2012.\n zcm 0.40 was released May 16, 2012. It is described as using CM with\n6 contexts, a mixer, and one re-mixer (APM or SSE) to adjust the mixer\noutput. It uses LZP preprocessing.\n \n\n```\nProgram      Option     enwik8      enwik9       Prog      Total      Comp  Deco  Mem   Note\n---------    ------   ----------  -----------  --------  ---------    ----  ----  ----  ----\nzcm v0.01    c1       23,914,413                                      2260  2730    35   26\n             c7       20,093,284  169,397,795  47,975 x  169,445,770  2965  2883  1486   26\nzcm v0.02    c7       20,277,130  170,848,574                         2419  2396  1470   26\nzcm v0.03    c7       20,159,212  169,368,119  27,589 x  169,395,708  2416  2369  1476   26\nzcm v0.04    c7       20,853,133  173,956,638  27,731 x  173,984,369  1462  1459  1520   26\nzcm v0.11    c0       23,963,073                                      1230  1210    22   26\n             c1       22,937,669                                      1280          35   26\n             c2       22,076,074                                      1290          62   26\n             c3       21,362,445                                      1330         115   26\n             c4       20,810,077                                      1370         222   26\n             c5       20,447,150                                      1390         401   26\n             c6       20,215,116                                      1400         697   26\n             c7       20,078,151  165,518,908  31,576 x  165,550,484  1275  1190  1716   26\nzcm 0.20b    -m7      20,204,267  167,177,534 161,122 x  167,338,656  1199  1204  1657   26\nzcm 0.30     -m7      20,237,368  167,198,948 161,558 x  167,360,506   949   970  1720   26\nzcm 0.40     -m7      20,200,819  167,138,719 161,502 x  167,300,221   904   929  1511   26\nzcm 0.50a    -m7      19,966,605  164,661,654 161,614 x  164,823,268   947   971  1579   26\nzcm 0.60d    -m7 -t1  19,786,363  162,731,120 171,517 x  162,902,637   915   960  1662   26\n             -m1 -t1  23,374,636                                       890   920    46   26\n             -m1 -t2  23,440,140                                       830   910    97   26\n             -m4 -t1  20,698,415                                       950  1000   226   26\n             -m4 -t2  20,925,875                                       940   990   389   26\n             -m6 -t1  19,933,151                                      1030  1050   651   26\n             -m6 -t2  20,359,596                                      1070   990  1160   26\n             -m7 -t2  20,267,309                                      2080  1130  2450   26\nzcm 0.70b    -m7 -t1  20,065,306  166,373,795 159,493 x  166,532,988   870   884  1412   26\nzcm 0.80     -m7 -t1  19,937,741  164,724,585 110,565 x  164,835,150   552   557  1700   48\n             -m7 -t2  20,554,326  166,468,556 110,565 x  166,579,121   414   415  1990   48\nzcm 0.88         -t1  21,383,928                                       940   930   196   26\n                 -t2  25,767,005                                       800   820   120   26\n             -m7 -t2  20,418,171                                      1110   890  1400   26\n             -m7 -t1  19,970,859  164,702,310 162,136 x  164,864,446   910   891  1434   26\n             -m7 -t1  19,970,859  164,702,310 162,136 x  164,864,446   546   527  1434   48\nzcm 0.90     -m7 -t1  20,006,179  165,266,797 164,361 x  165,431,158   511   516 ~1700   48\nzcm 0.92     -m7 -t1  19,803,545  163,246,657 166,763 x  163,413,420   500   512  1546   48\nzcm_x64 0.92 -m7 -t1  19,803,545  163,246,657 225,205 x  163,471,862   488   471  1549   48\n             -m8 -t1  19,700,970  160,848,578 225,205 x  161,073,783   489   474  2400   48\nzcm 0.93     -m8 -t1  19,572,089  159,135,549 227,659 x  159,363,208   421   411  3100   48\n```\n\n As with other PPM compressors (ppmd, ppmonstr), using a higher order improves\ncompression but consumes memory faster.  For enwik8, -o32 is optimal with 700MB available,\nbut lower orders are better for enwik9.\n \n\n```\n              Compression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  \n-------         -------         ----------  -----------  -----------  -----------  ----- -----  \nslim23d         -m1700 -o12     19,077,276  159,772,839     69,453 x  159,842,292   5232 ~5400\nslim23d         -m700 -o32      19,226,339  (failed)        69,453 x                6530  6770\nslim23d         -m700 -o10      19,264,094  162,529,098     69,453 x  162,598,551   5175  5360\nCompression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem  Alg Note\n-------         -------         ----------  -----------  -----------  -----------  ----- ----- ----- --- --- \nbsc-m03 v0.4.0  -b1000000000    20,293,393  160,258,936  105,456 xd   160,364,392   160   135  13000 BWT 96\n```\n\n bwmonstr 0.01 was released Mar. 18, 2009.\n bwmonstr 0.02 was released July 8, 2009. It uses a compressed representation internally,\nthus memory usage is less than the 1 GB block size. It compresses the entire input file in\na single block and requires enough memory to hold the file. The program is multi-threaded\neven on a single block. Times shown are for a single core processor, but would be faster on\na multi-core processor.\nreorder2 is an alphabet reordering program by Eugene Shelwien.\ndrt is the dictionary preprocessor from lpaq9m by Alexander Rhatushnyak\n \n\n```\n              Compression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp  Decomp  Mem Alg Note\n-------         -------         ----------  -----------  -----------  -----------  -----  ----- ---- --- ---- \nbwmonstr 0.00                   20,401,888  161,249,951     27,772 x  161,277,723  15638  13028 1224 BWT  26\nbwmonstr 0.01                   20,379,365  161,026,258     32,163 x  161,058,420  15695  14135 1224 BWT  26\nbwmonstr 0.02                   20,307,295  160,468,597     69,401 x  160,537,998 331801 156147  590 BWT  30\nreorder2|bwmonstr 0.02          20,229,555                                                       590 BWT  30\ndrt|bwmonstr 0.02               19,750,461                                                       450 BWT  30\nCompression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- --- ----\ntree 0.1                                    187,985,256      6,656 sd 187,985,256 337287    22 1850 Gram 64\n                                23,660,364  187,933,399      6,656 sd 187,940,055 174589    12 1850 Gram 65\ntree 0.2                        23,250,856  185,311,980                           337287    22 1850 Gram 64\ntree 0.3                        23,233,932  184,838,711      6,591 sd 184,845,302 105728    23 1850 Gram 64\ntree 0.4                        23,178,500  184,312,072      7,216 sd 184,319,288  68866    22 1850 Gram 64\ntree 0.5                        23,084,884  181,375,076      8,271 sd 181,383,347  68869    22 1850 Gram 64\ntree 0.9                        22,366,748  181,324,992      7,104 sd 181,332,096  70723    15 1850 Gram 64\ntree 0.10                       22,072,432  178,949,848     12,174 sd 178,962,022  72663    18 1850 Gram 64\ntree 0.11                       22,076,556  178,773,808      9,645 sd 178,783,453  72659    13 1850 Gram 64\n                                22,076,556  178,782,844      9,645 sd 178,792,489  36765     8 1750 Gram 67\ntree 0.12                       21,974,704  177,542,704     10,121 sd 177,552,493 226279    17 1800 Gram 48\n                                            177,321,380                                      7      Gram 65\ntree 0.13                       21,976,316  177,340,072     10,525 sd 177,350,597 116473   7.5 1700 Gram 67\n               (P+W+C)          22,075,700  178,774,864     10,525 sd 178,785,389  36652   7.3 1700 Gram 67\ntree64 0.13                     22,196,288  180,516,660     10,525 sd 180,516,660  39417   7.4 6000 Gram 67\n               (P+W+C)          22,304,524  180,941,504     10,525 sd 180,952,029  20834   7.4 6000 Gram 67\ntree64 0.14                     22,124,900  178,839,408     10,525 sd 178,849,933   9364   7.5 5100 Gram 67\n               (P+W+C)          22,229,468  179,806,072     10,525 sd 179,816,597   6899   7.4 3800 Gram 67\ntree 0.15a                      21,922,356  176,896,672     11,203 sd 176,907,875 114733   6.9 1800 Gram 67\n               (P+W+C)          22,023,144  178,321,588     11,203 sd 178,332,791  36828   6.6 1800 Gram 67\ntree64 0.15a                    22,140,724  177,974,208     11,203 sd 177,985,411   9542   7.0 5200 Gram 67\n               (P+W+C)          22,155,772  178,874,272     11,203 sd 178,885,475   7269   6.8 3900 Gram 67\ntree 0.16b                      21,602,648  173,848,464     13,395 sd 173,861,859 114739   8.1 1693 Gram 67\ntree64                                      174,825,152     13,395 sd 174,838,547   9362   8.2 5002 Gram 67\ntree 0.17                       21,564,704  173,461,100     13,563 sd 173,474,663 115000   7.1 1700 Gram 67\ntree64                          21,772,096  174,399,062     13,563 sd 174,412,625   9400   7.2 5000 Gram 67\ntree64 0.18                     21,639,204  174,357,336     13,463 sd 174,370,799   4901   7.2 6009 Gram 67\n               -r100                        173,856,720     13,463 sd 173,870,183  14965   7.2 15370 Gram 67\ntree 0.19                       21,497,672  173,210,648     14,625 sd 173,225,273 119742   7.1 1692 Gram 67\ntree64 0.19                     21,547,924  173,803,292     14,625 sd 173,817,917   4702   7.2 6023 Gram 67\nglza 0.1                        21,225,310  171,131,068     14,391 sd 171,145,459   4716  12.5 6027 Gram 67\nglza 0.2                        20,806,740  167,274,338     15,218 sd 167,289,556   4713  16.4 6027 Gram 67\nglza 0.3                        20,541,988  165,419,346     18,982 sd 165,438,328   4156  14.9 6026 Gram 67\nglza 0.3b GLZAcompressFast      22,021,734  180,243,710     18,982 sd 180,262,692    786  15  11395 Gram 67\nglza 0.4                        20,497,514  165,117,094     20,056 sd 165,137,150   5971  14.9 6024 Gram 67\nglza 0.8                        20,472,828  164,943,294     64,327 sd 165,007,621   9328  15.8 12673 Gram 67\n               -p3              20,442,490  164,634,038     64,327 sd 164,698,365  10106  15.8 12369 Gram 67\nglza 0.10.1                     20,753,713  167,832,309     69,935 s  167,902,244    594  11.9 7452 Gram 67\n               -x -p3           20,356,097  163,768,203     69,935 s  163,838,138   8184  11.9 8205 Gram 67\nglza 0.12 -x -o0.6 -p4 -r16000  20,068,917  161,678,356     56,526 s  161,678,356  11771   9.4 15331 Gram 67\n          c                     20,382,236  164,700.487     56,526 s  164,757,487    563   9.3  7777 Gram 67\nCompression            Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options             enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------       ----------------   ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nnanozip 0.01  -cO -m1670m -txt   20,306,489  167,509,921    266,797 x   167,776,718   403   284 1325 BWT\nnanozipltcb                      20,494,670  166,251,135    239,124 x   166,490,259   348   185 1729 BWT\nnanozipltcb 0.08                 20,626,962  166,571,051          0 xd  166,571,051    93    53 1729 BWT 37\nnanozipltcb 0.09                 20,537,902  161,581,290    133,784 x   161,715,074    64    30 3350 BWT 40\nbash\n$ kanzi -c -f -b 1024M -i enwik8 -t RLT+TEXT+UTF -e TPAQX\nCompressed enwik8:  100000000 => 19098186 (19.10%) in 50414 ms\n\n$ kanzi -c -f -b 1024M -i enwik9 -t RLT+TEXT+UTF -e TPAQX\nCompressed enwik9:  1000000000 => 161690495 (16.17%) in 490.6 s\n```\n\n The program itself when statically compiled off the master branch with -Oz and zipped with `ect -9 -zip\n(https://github.com/fhanau/Efficient-Compression-Tool) is 235259 bytes compressed, 636296 uncompressed.\nThe compile used for the tests above (-Ofast) is 397855 bytes compressed, 1015688 uncompressed.\nTested on an AMD FX 4300 CPU with 6 (4+2) GB of DDR3 RAM, zram + zswap configured. OS is Arch Linux with kernel 6.15.0-rc3\nThe option -c means compress, -f means force overwrite, -b 1024M means divide the input into 1024 MB blocks to\ncompress in parallel. -i specifies the input, -t RLT-TEXT-UTF selects transforms (run length, text, UTF).\n-e selects the TPAQX encoding, a modified PAQ based encoder with arithmetic coding.\n \n\n```\nM99.exe e|d -switches blocksize input output \n\nswitches are:\n-r = post BWT run length encoding\n-a = arithmetic coding instead of M99 style bit packing\n-f = fast mode\n-m = max compression mode (implies -a).\n```\n\n Version 2.1 was released Apr. 19, 2007.\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options        enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------      ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nM99               e -m 239m    21,431,211  180,477,144     67,697 x  180,544,841    674   496 1500 BWT\nM99 v2.1          e -m 239m    21,251,170  178,910,174     68,052 x  178,978,226    713   535 1500 BWT\nM99 v2.2.1        e -m 239m    21,251,171  178,910,175     72,245 x  178,982,420    704   520 1500 BWT\nM03 0.2a          e 250000000  20,713,383  173,944,553     95,699 x  174,040,252    868   624 1470 BWT  26\nM03 1.1b          e 1000000000 20,710,197  163,667,431     50,468 x  163,717,899    457   406 5735 BWT  52\n```\n\n bcm 0.10  \n\n```\n                  Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram             Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------             -------       ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nbcm 0.03                          22,007,655  192,194,478     67,988 x  192,262,466    517   437  164 BWT  26\nbcm 0.04                          21,450,604  185,368,446     69,553 x  185,455,999    578   486  329 BWT  26\nbcm 0.05            -b327680      20,770,671  172,180,796     69,040 x  172,249,836    684   535 1642 BWT  26\n                    -b406991                  171,857,720     69,040 x  171,926,760              2030 BWT  27\nbcm 0.07            -b327680      20,770,673  172,180,037     60,990 x  172,241,027    818   578 1642 BWT  26\n                    -b488282                  169,396,680     60,990 x  169,457,670    472   341 2440 BWT  28\nbcm 0.08            e370          20,744,613  171,891,509     61,666 x  171,953,175    948   709 1900 BWT  26\n                    e477          20,744,613  169,179,098     61,666 x  169,232,764    545   418 2385 BWT  28\nreorder_v2|bcm 0.08 e477          20,677,205  168,694,909     80,149 x  168,775,058    548   422 2385 BWT  28\nreorder_V2|bcm 0.08 e477 xlt      20,665,536  168,598,121     80,661 x  168,678,782    552   420 2385 BWT  28\nbcm 0.09            -b328         20,625,697  170,913,486     63,704 x  170,977,190   1342  1053 1652 BWT  26\nbcm 0.10 x86        -b370         20,811,710  172,570,245     63,788 x  172,634,033    758   483 1899 BWT  26\nbcm 0.10 x64        -b512                     169,871,532     72,366 x  169,943,898    362       2560 BWT  35\nbcm 0.10 x64        -b477                     169,843,006     72,366 x  169,915,372    522   373 2500 BWT  36\nbcm 0.11            -b328         20,773,468  172,267,889     70,936 x  172,338,825    798   548 1552 BWT  26\n                    -b477         20,773,468  169,466,640     70,936 x  169,537,576    611   423 2500 BWT  43\nbcm 0.12            -b328         20,825,972  172,665,135     61,874 x  172,727,009    637   414 1683 BWT  26\n                    -b1000        20,825,972  164,654,285     61,974 x  164,716,259    281   214 5000 BWT  50\nbcm 0.14            c1000         20,736,614  163,885,873     74,569 x  163,960,442    162   153 5000 BWT  60\nbcm 1.00            -b500         20,792,796  169,489,509     15,187 s  169,504,696    251   250 2500 BWT  48\n                    -b1000        20,792,796  164,251,284     15,187 s  164,266,471    147   142 5000 BWT  60\nbcm 2.03            -b1000x-      20,738,630  163,646,387    125,866 x  163,772,253    106    67 4096 BWT  97\n                                              163,646,387                               62    34           98\n```\n\n Other options select LZP table size (default 2 bsc 1.03  bsc 2.20, June 15, 2010, has speed improvements for multi-core support.\n-b1000p means use 1000 MB\nblock size (-b1000, requires 5 GB memory) with no preprocessing (-p).\n-b80p uses 80 MB block size with no preprocessing. -m2f means use\nsort transform order 5 (-m2) and fast compression (-f).\nenwik8 was tested as in note 26 on bsc-x32 replacing -b1000p with -b100p.\n bsc 2.26, July 26, 2010, has some speed improvements but retains compatibility\nwith version 2.25. -b328 selects a block size of 328 MB, which divides enwik9\ninto 3 blocks. This is the fewest number of blocks supported by the x86 version\nbecause of a 2 GB process limit. The x64 version does not have this limit but\nrequires 64 bit Windows. -t disables parallel block processing, which would double the\nmemory requirement. -T disables all multicore processing. This gives a smaller compressed\nsize but is slower than -t. -T or -t must be specified during decompression\nto prevent an out of memory error. With -t, CPU usage is 156% for compression and 129%\nfor decompression on a dual core T3200 (2 GHz, 3 GB, Vista 32 bit).\n bsc 2.4.5, Jan. 3, 2011, improves the speed of decompression. It remains\ncompatible with the previous version.\n bsc 2.5.0, Mar. 20, 2011, had no significant changes for the tests performed.\nMinor performance enhancements. CRC32 is replaced with Adler32.\n bsc 3.0.0, Aug. 27, 2011 adds experimental NVIDEA (CUDA) GPU acceleration for\nforward sort transforms ST5 through ST8. ST7 and ST8 are GPU only. There are\n32 and 64 bit versions. For the test shown, the 64 bit version was used.\n-b32 means to select 32 MB block\nsize, -p disables preprocessing, -m8 selects order 8 sort transform, and\n-f selects fast compression. The test machine is a Core-i7 2600K (4 cores,\n8 threads, 8 MB cache) overclocked from 3.4 GHz\nto 4.6 GHz, with a 384 CUDA processor GeForce 560Ti GPU, overclocked from 822\nMHz to 900 MHZ, with 2000 MHz memory speed.\nCompression takes 8.705 seconds using 1129 MB CPU memory\nand about 1 GB GPU memory. Decompression uses only the CPU, taking 18.595 seconds\nusing 1395 MB memory.\n \n\n```\n              Compression       Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram        Options        enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------        -------      ----------  -----------  -----------  -----------  ----- ----- ---- ---  ----\nbsc-x64 1.00   -b1024       20,769,550  163,820,253    274,197 x  164,094,450    311   212 6000 BWT  34\nbsc-x64 1.00   -b1000 -p    20,787,437  163,882,152    274,197 x  164,156,349    271   209 6000 BWT  38\nbsc-x86 1.00   -b250 -t     20,769,550  174,337,692    258,824 x  174,596,616    473   276 1504 BWT  28\nbsc-x64 1.00   -b79 -p -m2  22,864,952  200,607,811    274,197 x  200,882,008     36    71 1896 ST5  39\nbsc-x86 1.03   -b250 -t     20,769,550  174,337,692    261,058 x  174,598,750    470   280 1504 BWT  28\nbsc-x64 2.00   -b1000p      20,789,147  163,888,465    122,581 s  164,011,046    237   199 5095 BWT  39\nbsc-x64 2.20   -b1000p      20,789,228  163,888,858    149,153 s  164,038,011    238    93 5095 BWT  39\n               -b80p -m2f   23,031,164  201,321,919    149,153 s  201,471,072     27    68 1624 ST5  39\nbsc-x86 2.26   -b328 -t     20,774,446  171,826,969    138,293 s  171,965,262    386   183 1667 BWT  28\n               -b328 -T     20,772,543  171,820,075    138,293 s  171,958,368    438   274 1663 BWT  28\nbsc-x86 2.45   -b328 -t     20,774,446  171,826,969    130,327 s  171,957,296    382   141 1667 BWT  28\n               -b328 -T     20,772,543  171,820,075    130,327 s  171,950,402    443   195 1667 BWT  28\nbsc-x86 2.50   -b328 -t     20,774,446  171,826,969    129,593 s  171,956,562    398   139 1670 BWT  28\n               -b328 -T     20,772,543  171,820,075    129,593 s  171,949,668    444   195 1670 BWT  28\nbsc-x64 3.00   -b32p -m8f   22,461,680  196,398,933    934,176 x  197,333,109      8    18 3129 ST8  51\nbsc-x86 3.10   -b328 -T     20,920,018  173,026,090    241,476 s  173,267,566    390   149 1712 BWT  28                \nbsc 3.25       -b1000 -e2   20,786,794  163,884,462     74,297 xd 163,958,759     23     8 5000 BWT  96\n```\n\n The m1000 command selects 1000 MB block size.  Thus, enwik9 is suffix sorted in one block.\nThis is accomplished by sorting 16 smaller blocks, writing the pointers to 4 GB\nof temporary files, and merging them.  The inverse transform is done in memory without\nbuilding a linked list.  Rather, the next position is found by looking up the\napproximate location in an index of size n/16 and finding the exact location by\nlinear search.\n \n\n```\n  g++ -Wall -O2 -Os -march=pentiumpro -fomit-frame-pointer -s -o bbb.exe\n  upx bbb.exe\n```\n\n bbb has a faster mode for both compression and decompression that does a \"normal\"\nBWT using 5x blocksize in memory.  Output format is the same for fast and slow mode\nfor both compression and decompression.  A file compressed in fast mode can be\ndecompressed in slow mode on another computer with less memory, and vice versa.\nThe mode has no effect on the compressed file contents.\n Recommended usage for best compression:  For files smaller than 20% of available\nmemory, use fast mode and one block.  For example, if you have 1 GB memory (800 MB\navailable under Windows) and  \n\n```\n  bbb cfm100 foo foo.bbb  (c = compress, f = fast, m100 = 100 MB blocks)\n  bbb df foo.bbb foo.out  (d = decompress, f = fast)\nbbb cm500 foo foo.bbb\n  bbb d foo.bbb foo.out\nbbb cm640 foo foo.bbb\n  bbb d foo.bbb foo.out\n```\n\n bbb results by block size are shown below.\nGain is the compression improvement obtained by using a larger block size.\nGain(blocksize) is defined as C(blocksize/10)/C(blocksize) - 1 where\nC(x) means the compressed size of enwik9 with block size x.\nCompression times are fast modes for block sizes 10 through 10 \n\n```\nBlock   enwik8      enwik9     Gain  Comp ns/b\n----  ----------  -----------  ----  ----\n101   66,414,034  646,449,572        4359\n102   56,241,619  542,912,447  .191  2169\n103   45,500,201  435,597,745  .246  1907\n104   37,006,646  343,663,203  .267  1802\n105   30,946,413  275,172,983  .249  1838\n106   26,661,555  233,555,297  .178  2095\n107   23,460,457  204,355,672  .142  2499\n108   20,847,290  182,162,626  .122  3106\n109   20,847,290  164,032,650  .110  4524\nCompressed size      Decompresser  Total size   Time (ns/byte)\nProgram                  enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nppmx 0.01              24,369,312  213,206,926     51,454 x  213,258,380    557   515  550 PPM 26\nppmx 0.02              22,580,291  194,298,469     53,511 x  194,351,980    874   888  609 PPM 26\nppmxcore2duo 0.02      22,580,291                  55,824 x                 871   949  609 PPM 26\nppmx 0.03              22,572,808  193,643,464     54,964 x  193,698,428    777   784  609 PPM 26\nppmx 0.04              23,150,510  201,384,355     52,406 x  201,436,761    791   801  280 PPM 26\nppmx 0.05              22,905,422  196,548,444     53,476 x  196,601,920    863   882  576 PPM 26\nppmx 0.06              26,131,726  235,257,572     54,596 x  235,312,168    276   317   71 PPM 26\nppmx 0.07              23,941,730  211,671,802     44,104 x  211,715,906    314   352  302 PPM 26\nppmx 0.08              23,204,040  202,868,559     54,098 x  202,922,657    397   420  355 PPM 26\n                       23,204,040  202,868,559     54,098 x  202,922,657    107   127  355 PPM 53\nppmx 0.09              25,952,954  232,581,333     50,873 x  232,632,206    122   150  279 PPM 48\n                       25,952,954  232,581,333     50,873 x  232,632,206     57    69  279 PPM 63\nppmx 0.10 c16          20,705,277                  93,482 x                 479   486 6382 PPM 106\n                       20,705,277  164,250,527     93,482 x  164,344,009    251   252 39727 PPM 107\nCompression                Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                  enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------                ----------  -----------  -----------  -----------  ----- -----  --- --- ----\npcompress 3.1  -c libbsc -l14 -s1000m    20,769,968  163,391,884   1,370,611 x 164,762,495    359    74 3300 BWT  48\n```\n\n## .1532 fp8_v3\n\n[fp8](http://encode.dreamhosters.com/showthread.php?t=613) v1 (fast paq)\nis a free, open source archiver by Jan Ondrus, May 2, 2010. It is derived\nfrom pax8px_v68. It has fewer models than paq8px for better\nspeed but retains the models for wav, bmp, and jpg. The option -8 selects maximum memory.\n[fp8 v2](http://encode.su/attachment.php?attachmentid=1919&d=1334082590),\nApr. 10, 2012, has some modeling improvements.\n[fp8 v3](http://encode.su/attachment.php?attachmentid=1946&d=1336895016),\nMay 13, 2012, has some more compression improvements (at a slight cost in speed)\nand a JPEG bug fix.\n[tangelo 1.0](<http://encode.su/threads/1738-TANGELO-new-compressor-(derived-from-PAQ8-FP8)>), June 17, 2013, is a single-file compressor based on fp8. It removes\nspecialied models and preprocessors for exe, bmp, wav and jpeg types. It takes no\noptions. It uses fixed memory of 567 MB, equivalent to fp8 -7.\n[tangelo 2.0](<http://encode.su/threads/1738-TANGELO-new-compressor-(derived-from-PAQ8-FP8)?p=33735&viewfull=1#post33735>), July 6, 2013, removed some models and made other\nsimplifications for better speed and less memory but worse compression.\n[tangelo 2.1](<http://encode.su/threads/1738-TANGELO-new-compressor-(derived-from-PAQ8-FP8)?p=33933&viewfull=1#post33933>), July 20, 2013, faster with less compression.\n[tangelo 2.3](<http://encode.su/threads/1738-TANGELO-new-compressor-(derived-from-PAQ8-FP8)?p=33955#post33955>), July 22, 2013, re-added APM for better compression, and minor\nchanges for better speed.\n## .1563 WinRK\n\n[WinRK](http://www.msoftware.co.nz/) 3.0.3 is a commercial\nGUI archiver by Malcolm Taylor\n(Mar. 6, 2006).  It is top ranked on some benchmarks.\nUnfortunately it is not available for free download (as of May 16, 2006).  The\n\"free trial\" expires as soon as you install it.\n(Update, Sept. 11, 2006: versions 3.0.2 and 3.0.3 are no longer available for download.  \nThey appear to have been withdrawn last month).\nWinRK in PWCM mode (Paq Weighted Context\nModeling) is based on the paq7/8 algorithm with text dictionary preprocessing\nand specialized models for wav, bmp, and exe files.  Version 3.0.2 was based on\nthe earlier paq6 algorithm which uses adaptive linear model mixing rather than\na neural network which mixes bitwise predictions from models \nin the logistic (log p/(1-p)) domain.  The +td and -td options turns English dictionary\npreprocessing on or off respectively.  800MB selects the memory limit.  When not\nspecified, PWCM appears to allocate all available memory except leaving 8 MB.\n## .1570 ppmonstr, ppmd, ppms\n\n[ppmonstr, ppmd, and ppms](http://www.compression.ru/ds/) var. J are\nfree command line file compressors by Dmitry Shkarin (model) and\nDmitry Subbotin (range coder), Feb. 16, 2006. (ppms on Feb. 21, 2006).\nppmonstr is a slower, experimental version of ppmd with better compression.\nSource code is available for ppms and ppmd but not ppmonstr.\nppms is a small memory (1 MB) version of ppmd.\nThey all use PPMII (PPM with information inheritance).  The -m256\noption selects 256 MB memory (maximum for ppmd).  The -o10 option selects\nPPM order 10.  (Higher orders use up memory faster which hurts\ncompression).  When ppmd runs out of memory, it discards the\nmodel and starts over.  The -r1 option (default in ppmonstr)\ntells ppmd to back up and partially rebuild the model before resuming compression.\nThe default options for ppmd are -m10 -o4 -r0 which are designed for reasonably\ngood compression with high speed and low memory usage (see table below).\n## .1573 STC\n\n[STC](https://github.com/thu-nmrc/STC-for-BWT-FamilyText-Compression) is a free, open source (GPL, Apache)\nfile compressor by 杜靖洋 , May 23, 2026, and updated June 8, 2026. It uses BWT, derived from BSC-M03 by Ilia Grebnov.\nI tested it under Ubuntu Linux under WSL2 compiling with g++ as directed on the website. The updated version reduces\nmemory usage to 11.8 GB for enwik9 but does not change compression.\n## .1593 zcm\n\n[zcm v0.01](http://encode.su/attachment.php?attachmentid=1771&d=1324079623)\n[(discussion)](http://encode.su/threads/1432-Experimental-new-ZCM-file-compressor!)\nis a free, experimental, closed source compressor for 32 bit Windows\nby Nania Francesco Antonio,\nDec. 16, 2011. It uses context mixing. Commands c1 through c7 select memory\nusage for compression. Decompression uses the same memory. c7 uses the most\nmemory and gets the best compression.\n[zcm v0.11](http://heartofcomp.altervista.org/zcmo.zip)\nwas released Feb. 19, 2012.\nIt is described as mixing 6 contexts. It detect file type and uses exe, delta,\nand LZP preprocessors. It has separate models for text and binary data.\nSpeed and memory usage are the same for compression and\ndecompression. Commands c0 through c7 select memory usage. Each increment\ndoubles memory, resulting in better compression.\nMemory is used slowly as the program runs up to a maximum value which is not reached on\nenwik8 for c5 and higher. For enwik8, c7 uses 1286 MB rather than 1716 MB.\n[zcm 0.20b](http://heartofcomp.altervista.org/zcm20.zip) was released\nApr. 4, 2012. It is an archiver rather than a single file compressor.\nOption -m7 selects maximum memory usage (range 32 MB to 1.7 GB).\n[zcm 0.50a](http://heartofcomp.altervista.org/ZCM/home.htm)\nwas released June 2, 2012.\n[zcm 0.60d](http://heartofcomp.altervista.org/zcm060.zip)\nadds multithreading and other improvements. The -t option selects the\nnumber of tasks. -t0 auto-detects the number of cores, which is equivalent\nto -t2 on the dual core test machine (T3200, 3 GB). The default is -t1.\nThe -m option selects memory usage from -m1 (46 MB per task)\nto -m7 (1.6 GB per task). The default is -m4. Parallel compression is\nperformed by separate processes that can independently access 2 GB of memory\neach in 32 bit Windows. When run with -t2, there is also a third task using\n5 MB of memory. All three tasks saturate one CPU core each.\nIt was found that -t2 makes compression worse (probably by splitting the\ninput in half and compressing each separately) and is not much faster than\n-t1. The -t option can also be given during extraction. If the archive\nwas compressed with -t2 then extraction with -t2 doubles memory usage\nbut only improves speed slightly. If compressed with -t1 then extraction\nwith -t2 is 4 seconds slower for enwik8 than with -t1 because the extra task exits\nimmediately and the third 5 MB task continues to run.\n[zcm 0.70b](http://heartofcomp.altervista.org/zcm070.zip)\nwas released Oct. 14, 2012.\n[zcm 0.80](http://heartofcomp.altervista.org/ZCM/home.htm)\nwas released May 15, 2013. It was tested in Linux under Wine.\nWhen -t2 was used to compress in 2 threads, it was also used to extract.\n[zcm 0.88](http://heartofcomp.altervista.org/ZCM/home.htm)\n[(discussion)](http://encode.su/threads/1432-Experimental-new-ZCM-file-compressor!?p=33572&viewfull=1#post33572) was released June 21, 2013. It was tested both in Windows and\nin Linux under wine.\n[zcm 0.90](http://encode.su/threads/1432-Experimental-new-ZCM-file-compressor!?p=37835&viewfull=1#post37835) was released May 3, 2014.\n[zcm 0.92](http://encode.su/threads/1432-Experimental-new-ZCM-file-compressor!?p=38117&viewfull=1#post38117) was released May 16, 2014.\nA \n[64 bit Windows version](http://encode.su/threads/1432-Experimental-new-ZCM-file-compressor!?p=39271&viewfull=1#post39271) was released July 3, 2014. It supports the undocumented -m8 option\nusing up to 3 GB memory.\n[zcm 0.93](http://encode.su/threads/1432-Experimental-new-ZCM-file-compressor!?p=43665&viewfull=1#post43665) was released May 12, 2015.\n## .1598 slim\n\n[slim](http://www.bars-trade.com/slim/) 23d is a free, closed source command line\narchiver by Serge Voskoboynikov, Sept 21, 2004.  It uses a PPMII core\n(ppmd/ppmonstr) by Dmitry Shkarin with filters for special file types including text.\nThe -m700 option selects 700 MB of memory.  (I found -m800 causes\ndisk thrashing at 1 GB).  The -o10 option selects order 10 PPM.  (-o12 and -o16\ncaused slim to fail on enwik9, creating an empty archive and exiting after about 60% completion with 1 GB.  \nSmaller files were OK.  There was no error with 2 GB).\n## .1603 bsc-m03\n\n[bsc-m03](https://github.com/IlyaGrebnov/bsc-m03)\n[(discussion)](<https://encode.su/threads/3763-bsc-m03-(experimental-M03-sorting-compressor)>)\nis a free, experimental, \nopen source (GPL v3) file compressor by Ilya Grebnov, November 20, 2022.\nIt is a practical implementation of Compression via Substring Enumeration\nusing Burrows-Wheeler transform and M03 context aware compression algorithm.\nIts purpose is to have the highest compression ratio among BWT based compressors\nwithout using preprocessing. The compressor only uses a single thread.\n## .1605 bwmonstr\n\n[bwmonstr](http://www.nanozip.net/bwmonstr.html) 0.00 is a free, experimental,\nclosed source\nfile compressor by Sami Runsas, Mar. 10, 2009. It uses BWT. The program takes no\noptions. It loads the input file into a single block and allocates 1.25 times the\nblock size in memory for either compression or decompression.\nThus, it is able to transform enwik9 in a single block.\n## .1616 glza\n\n[tree 0.1](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=37386#post37386) is a free, experimental, open source compressor by\nKennon Conrad, Mar. 31, 2014. It is a general purpose compressor\noptimized to compress text. The compressor is\n3 separate programs. The first, TreeCapEncode.c, converts upper case\nletters to lower case plus special symbols. It takes 4 minutes.\nThe second, TreeCompress.c,\nuses a suffix tree to parse the input into tokens.\nIt takes 3 days, 21 hours, 37 minutes and uses 1850 MB memory.\nThe third, TreeBitEncode.c\nencodes the tokens using variable length codes. This takes\n27 seconds. The decoder, TreeDecode.c, takes 22 seconds using\n400 MB memory. Compressed size depends on available memory; thus\nresults below are machine dependent.\n[tree 0.3](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=37735&viewfull=1#post37735) was released Apr. 27, 2014. It uses a model that only parses\nwhole words with a leading space.\n[tree 0.4](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=38235&viewfull=1#post38235) was release May 21, 2014.\n[tree 0.5](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=38336&viewfull=1#post38336) was released May 25, 2014.\n[tree 0.9](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=39327&viewfull=1#post39327) was released July 5, 2014. It includes a multi-threaded\ndecompression program for better speed. TreeCapEncode.c is now TreePreEncode.c\nand run in 11 seconds.\n[tree 0.10](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=39868&viewfull=1#post39868) was released Aug. 15, 2014. Timings for each step are:\nTreePreEncode 20 s, TreeParagraphs 1485 s, TreeWords 393 s, TreeCompress 70732 s,\nTreeBitEncode 33 s, total 72663 s.\n[tree 0.11](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=40204&viewfull=1#post40204) was released Sept. 2, 2014. It uses extra symbol tables to improve\ncompression ratio and decompression speed.\n[tree 0.12](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=40699&viewfull=1#post40699) was released Sept. 29, 2014 with a bug fix on Oct. 1, 2014.\nFor note 48, the program was compiled with gcc 4.8.2 -O3.\n[tree 0.13](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=40829&viewfull=1#post40829) was released on Oct. 12, 2014. There is a 32 bit version that uses\n1700 MB memory and a 64 bit version of TreeCompress.exe that uses 6x the input\nsize in memory. The option (P+W+C) means that the two preprocessing stages\nTreeParagraph.exe and TreeWords.exe (same for 32 and 64 bit) were run on the input\nprior to TreeCompress.exe or TreeCompress64.exe. Otherwise only the last stage\nis run. The preprocessing stages make compression worse but faster.\n[tree 0.14](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=41016&viewfull=1#post41016) was released Oct. 29, 2014. The 64 bit version was tested.\n[tree 0.15](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=41481&viewfull=1#post41481) was released Nov. 21, 2014. 0.15a, Nov. 22, 2014,\nhas a faster decompressor.\n[tree 0.16b](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=41699&viewfull=1#post41699) was released Dec. 9, 2014.\n[tree 0.17](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=41799&viewfull=1#post41799) was released Dec. 16, 2014. Compression times an memory\nusage are approximate (unchanged since last version).\n[tree 0.18](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=42134&viewfull=1#post42134) was released Jan. 17, 2005 with improvements to the 64 bit\nversion. The -r option controls memory usage.\n[tree 0.19](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=42494&viewfull=1#post42494) was released Feb. 4, 2015.\n[glza 0.1](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=43464&viewfull=1#post43464) is the new name of the tree program, released Apr. 27, 2015.\nIt uses adaptive order 0 arithmetic coding of dictionary symbols and other changes.\n[glza 0.2](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=43851&viewfull=1#post43851) was released May 24, 2015.\n[glza 0.3](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=44410#post44410) was released July 13, 2015. Decompression requires 330 MB memory.\n[glza 0.3b](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=45627&viewfull=1#post45627) was released Nov. 16, 2015. It contains the same files\nas v0.3a (a bug fix for v0.3) except that it also contains GLZAcompressFast (.c and .exe),\nwhich was tested below.\n[glza 0.4](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=47019&viewfull=1#post47019) was released Mar. 11, 2016.\n[glza 0.8](http://encode.su/threads/1909-Tree-alpha-v0-1-download?p=50293&viewfull=1#post50293) was released Sept. 27, 2016. The option -p3 selects a factor to favor\nlonger strings over more compressive.\n[glza 0.10.1](https://encode.su/threads/1909-Tree-alpha-v0-1-download?p=55433#post55433) was released Jan. 6, 2018.\n[glza 0.12](https://encode.su/threads/1909-Tree-alpha-v0-1-download?p=87384&viewfull=1#post87384) was released Mar. 23, 2026. The program learns grammar production rules and encodes\nthem at their first occurrence.\n## .1617 nanozipltcb\n\n[nanozipltcb](http://nanozip.net/nanozipltcb.zip) is a free file compressor\nby Sami Runsas, July 25, 2008. It uses BWT. It takes no options. It is a customized version of\n[nanozip](http://nanozip.net/), similar to -cO -txt -m1700m, but\ntuned to this benchmark. Files compressed with\nnanozipltcb are not compatible with nanozip.\n[nanozipltcb 0.08](http://nanozip.net/nanozipltcb-0.08_64.zip),\nMar. 3, 2010. is multithreaded and has other optimizations. Size is based\non a self extracting archive. Only a 64 bit Windows version\nexists. Tested by the author on a quad core Q6600 at 3.0 GHz.\nThe older version is withdrawn.\n[nanozipltcb 0.09](http://nanozip.net/nanozipltcb-0.09.linux64.zip), was relased\nMay 10, 2010. It has only a 64 bit Linux executable version.\n[`foo`\nis 500 MB:\n`foo` is 1 GB:]\n## .1620 kanzi\n\n[kanzi](https://github.com/flanglet/kanzi-cpp) is a free, open source (Apache) file compressor\nby Frederic Langlet. The version released Apr. 23, 2025 was reported. It uses a variety of compression\noptions: LZ, BWT, and CM with preprocessors for text, exe, and others. The reported results:\n## .1637 M03\n\n[M99](http://www.michael-maniscalco.com/compression.htm)\n([mirror](m99.zip)) is a free\nfile compressor by Michael Maniscalco, originally written in 1999 and ported\nto Windows on Mar. 27, 2007.  It uses BWT, based on MSufSort 3.1.\nM99 is a predecessor to M03.  Command line is:\n[M99 2.2.1](http://www.michael-maniscalco.com/downloads/m99.v2.2.zip),\nreleased July 18, 2008,\nhas an optimization to compress the contents of TAR files separately. For other files,\nit increases the size by 1 byte.\n[M03 v0.2a](http://www.michael-maniscalco.com/downloads/M03.0.2a.zip),\nOct. 10, 2009,\ntakes just one option, which is the block size in bytes. Memory usage is 6x\nblock size for compression and 5x for decompression.\n[M03 v1.1 beta](http://michael-maniscalco.com/downloads/M03.64.rar)\nwas released Oct. 24, 2011 for 64 bit Windows.\nIt includes some new, fully parallel\nsuffix sorting and BWT construction algorithms. The option 1000000000\nspecifies a single block requiring 5 GB memory to compress or decompress.\n## .1637 bcm\n\n[bcm](http://encode.narod.ru/)\n[0.03](http://encode.su/forum/attachment.php?attachmentid=359&d=1234186006)\n([discussion](http://encode.su/forum/showthread.php?t=273)) is a free\ncommand line compressor by Ilia Muraviev, Feb. 9, 2009. It uses BWT with a fixed\nblock size of 32 MB and an order 0 CM back end. It takes no command line options.\n[bcm 0.04](http://www.encode.su/forum/attachment.php?attachmentid=362&d=1234361923)\n([discusion](http://www.encode.su/forum/showthread.php?t=279)) was released\nFeb. 11, 2009. It increases the block size to 64 MB and has modeling improvements\nincluding interpolated SSE.\n[bcm 0.05](http://www.encode.su/forum/attachment.php?attachmentid=379&d=1236285486)\n([discussion](http://www.encode.su/forum/showthread.php?p=5364#post5364))\nwas released Mar. 5, 2009. The option -b327680 selects 327680 KB block size. It uses\n5x block size memory.\n[bcm 0.07](http://www.encode.su/forum/attachment.php?attachmentid=392&d=1237134593)\n[(discussion)](http://www.encode.su/forum/showthread.php?t=304)\nwas released Mar. 15, 2009.\n[bcm 0.08](http://encode.su/forum/attachment.php?attachmentid=752&d=1243801483)\n[(discussion)](http://encode.su/forum/showthread.php?t=382)\nwas released May 31, 2009. The command e370 means to use a block size of 370 MB.\nMemory usage is 5 times block size. Larger values gave an \"out of memory\" error\nunder 32 bit Windows Vista with 3 GB memory.\n[reorder v2](http://ctxmodel.net/files/BWT_reorder_v2.rar)\n[(discussion)](http://www.encode.su/forum/showthread.php?t=375)\nis an alphabet reordering preprocessor for BWT compressors by Eugene Shelwien,\nMay 26, 2009.\n[xlt](http://www.encode.su/forum/attachment.php?attachmentid=793&d=1244103104)\nis a pair of 256 byte files that defines the alphabet permutation used\nby reorder, released June 4, 2009 by Eugene Shelwien.\n[bcm 0.09](http://encode.su/forum/attachment.php?attachmentid=997&d=1250710056)\n[(discussion)](http://encode.su/forum/showthread.php?p=9679#post9679)\nwas released Aug. 19, 2009. Option -b328 selects a block size of 328 MB. Memory usage is\n5 times block size for both compression and decompression.\n[x64](http://encode.dreamhosters.com/attachment.php?attachmentid=1166&d=1260562113)\n[x86](http://encode.dreamhosters.com/attachment.php?attachmentid=1167&d=1260562119)\nwas released Dec. 11, 2009.\n[Discussion](http://encode.dreamhosters.com/showthread.php?t=519)\nThe x64 version is for 64 bit Windows. The x86 version is for 32 bit Windows.\nThe -b option gives the block size in MB. Memory usage is 5x block size.\n[bcm 0.11](http://encode.su/attachment.php?attachmentid=1332&d=1277207844)\n[(discussion)](http://encode.su/threads/1087-BCM-0.11-A-high-performance-BWT-compressor)\nwas released June 22, 2010. It is described as a complete rewrite.\n[bcm 0.12](http://encode.su/attachment.php?attachmentid=1400&d=1288545534)\n[(discussion)](http://encode.su/threads/1149-BCM-0.12-is-here!)\nwas released Oct. 31, 2010. A 64 bit version was tested by the author\nwith -b1000 on June 1, 2011.\n[bcm 0.14](http://encode.narod.ru/)\n[(discussion)](http://encode.su/threads/1739-BCM-The-ultimate-BWT-based-file-compressor?p=33579&viewfull=1#post33579)\nwas released June 22, 2013. Only a 64 bit Windows version was released. Command c1000 means to compress\nin 1000 MB blocks.\n[bcm 1.00](http://sourceforge.net/projects/bcm/)\n[(discussion)](http://encode.su/threads/1739-BCM-The-ultimate-BWT-based-file-compressor?p=42828&viewfull=1#post42828)  was released\nas open source (public domain), Mar. 2, 2015. It was tested by compiling\nwith g++ 4.8.2 -O3 in Linux.\n[bcm 2.03](http://compressme.net/bcm203.zip) was released Feb. 27, 2023\nas a Windows .exe only. Option -b1000x- selects 1000 MB block size and no x86\npreprocessing. Timing on my system (97) uses 1 of 8 threads.\n## .1639 bsc\n\n[bsc](http://encode.dreamhosters.com/showthread.php?p=11633)\n[1.00 x86](http://encode.dreamhosters.com/attachment.php?attachmentid=1248&d=1270631017)\n[x64](http://encode.dreamhosters.com/attachment.php?attachmentid=1249&d=1270631025)\nis a free, experimental file compressor by Ilya Grebnov, Apr. 7, 2010. It uses BWT with LZP\npreprocessing. The option -b1000t selects a block size of 1000 MB and turns off multithreading\n(parallel compression on multiple cores). Memory requirements is 6x block size times number of\nthreads. Multithreading was turned off (-t) for both compression and decompression in order\nto maximize compression. Nevertheless, compression shows CPU utilization of 109% on 2 cores\neven with -t set. -p turns off LZP preprocessing. -m2 selects a sort (Schindler) transform\nof order 5.\n<sup>18</sup> bytes, range 10..28), LZP\nmatch length (default 128, range 4..255), block sorting algorithm (default BWT, possible\norder 4 or 5 sort (Schindler) transform), and preceding or following context for sorting\n(default following). Only the defaults were tested, which may not be optimal. There\nare two versions: x86 for 32 bit Windows with a 2 GB memory limit, and x64 for 64 bit\nWindows with no memory limit. Notes apply to enwik9. enwik8 size is tested as in note 26.\n[x86](http://encode.dreamhosters.com/attachment.php?attachmentid=1257&d=1271017191) and\n[x64](http://encode.dreamhosters.com/attachment.php?attachmentid=1258&d=1271017197)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?p=11701#post11701),\nApr. 11, 2010,\nare bug fixes that do not change results except for the size of the program.\nThe x64 version is 276,292 bytes.\n[bsc](http://libbsc.com/default.aspx) 2.00, May 3, 2010,\nis available with source code licensed under LGPL.\n[bsc 3.1.0](http://encode.su/threads/586-bsc-new-block-sorting-compressor?p=29904&viewfull=1#post29904)\nwas released July 8, 2012.\n[bsc 3.25](https://github.com/IlyaGrebnov/libbsc)\n[discussion](https://encode.su/threads/586-bsc-new-block-sorting-compressor)\nwas released Nov. 28, 2022. It has the same compression but improved performance.\nMain contribution is new fast linear time suffix array and Burrows-Wheeler transform construction\n[library](https://github.com/IlyaGrebnov/libsais).\n## .1640 bbb\n\n[bbb](bbb.cpp) ver. 1\nis a free, open source (GPL) command line file compressor by Matt Mahoney, Aug. 31, 2006.\nIt uses a memory efficient BWT allowing blocks up to 80% of available memory.\nThe transformed data is compressed with an order 0 PAQ like model: the previous\nbits of the current byte are mapped first to a bit history, then through a 6 level\nprobability correcting adaptive chain before bitwise arithmetic coding.\n[bbb.exe](bbb.exe) Win32 executable\ncompiled with MinGW g++ 3.4.2 and UPX 1.24w.\n[bbb](bbb) Linux executable, supplied by\nPhil Carmody (Aug. 31, 2006).  Compiled with `g++-4.1 -Wall -O2 -o bbb bbb.cpp; strip bbb`\n`foo` is 100 MB:\n<sup>8</sup>\nand slow mode for 10<sup>9</sup> on a 2.2 GHz Athlon-64 with 2 GB memory under WinXP Home SP2.\n## .1643 ppmx\n\n[ppmx 0.01](http://www.encode.su/forum/attachment.php?attachmentid=241&d=1227642517)\nis a free, experimental, closed source file compressor by Ilia Muraviev,\nreleased Nov. 25, 2008. It uses PPM with no filters. It takes no options.\n[ppmx 0.02](http://www.encode.su/forum/attachment.php?attachmentid=242&d=1228240119)\nwas released Dec. 2, 2008. It uses order 9 PPM with hashed context tables,\nas discussed [here](http://www.encode.su/forum/showthread.php?t=225).\nThere is also a\n[core 2 duo version](http://www.encode.su/forum/attachment.php?attachmentid=243&d=1228240123)\nwhich is faster, although it runs on only one core, and has a slightly larger\nexecutable. Note that the table below is misleading because on enwik8 the regular\nversion compressed at 976 ns/byte (12% longer) and decompressed at 992 ns/byte\n(4.5% longer) than the core 2 duo version.\n[ppmx 0.03](http://encode.su/forum/attachment.php?attachmentid=257&d=1229960531)\n(discussed [here](http://encode.su/forum/showthread.php?t=235)) was\nreleased Dec. 22, 2008.\n[ppmx 0.04](http://www.encode.su/forum/attachment.php?attachmentid=285&d=1231172607)\n(discussed [here](http://www.encode.su/forum/showthread.php?t=248))\nwas released Jan. 5, 2008. It uses order 12-5-3-2-1-0 PPM and 280 MB.\n[ppmx 0.05](http://encode.dreamhosters.com/attachment.php?attachmentid=311&d=1232383290)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?t=257),\nJan 19, 2010, adds SEE (secondary escape estimation), more memory, and some optimizations.\n[ppmx 0.06](http://encode.su/threads/1103-PPMX-0.06-has-been-released!),\nreleased July 27, 2010, is designed for improved speed and less memory usage rather\nthan compression ratio. It removes SEE and uses only a fixed order 4-2-1-0 model\nwith hash tables. It has a P4 version for Pentium-4 and higher that is about 12% faster.\nThis is the version tested. It has a larger executable (54,496 vs. 45,216).\n[ppmx 0.07](http://encode.su/threads/1223-PPMX-0.07-is-here!), Feb. 20, 2011,\nuses order 5-3-2-1-0-(-1) PPM with hash tables. Memory usage is increased to 302 MB.\n[ppmx v0.08](http://encode.narod.ru/)\n[(discussion)](http://encode.su/threads/1448-PPMX-0-08-is-here!?p=27791#post27791),\nJan. 1, 2012, uses order 6-4-2-1-0-(-1) PPM with hash tables and SEE improvements.\n[ppmx](http://compressme.net/)\n[0.09](http://compressme.net/ppmx009.zip)\n[(discussion)](http://encode.su/threads/1448-PPMX-0-08-is-here!?p=37265#post37265)\nwas released Mar. 24, 2014.\n[ppmx](https://compressme.net/) [0.10](https://compressme.net/ppmx010.zip)\nwas released June 20, 2026. Option c16 selects order 16, requiring 40 GB memory for enwik9. Default is order 5.\n## .1647 pcompress\n\n[pcompress](https://github.com/moinakg/pcompress) 3.1 is a free,\nopen source (LGPLv3 and MPLv2) deduplicating archiver and file compressor\nby Moinak Ghosh. A\n[Ubuntu build](http://encode.su/threads/1639-pcompress-a-deduplication-compression-utility?p=42457&viewfull=1#post42457) released Feb. 2, 2015 and\n[updated](http://encode.su/threads/1639-pcompress-a-deduplication-compression-utility?p=42539&viewfull=1#post42539) Feb 6, 2015 was tested. The option \"-c libbsc\" means to compress\na single file using libbsc (BWT). -l14 selects maximum compression\n(default -l6). -s1000m selects 1000 MB block size (default -s60m). The\ncompression algorithm is deduplication followed by\ndictionary preprocessing and BWT.\n## .1652 paq9a\n\n[paq9a](paq9a.zip) is a free,\nopen source, command line archiver by Matt Mahoney, Dec. 31, 2007.\nIt is a context mixing compressor with an LZP preprocessor to improve speed\nfor highly redundant files.  Matches to a context length of 12 or more are\ncoded as 1 bit, and literals as 9 bits.  Context mixing differs from paq8 in\nthat it uses a chain of 2-input mixers rather than one mixer with many inputs.\nIt mixes sparse order-1 contexts with gaps of 3, 2, 1, 0, then orders 2 through 6,\nthen text word orders 0 and 1.  Option -9 selects maximum memory.\n\n```\n        Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram  Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------  -------    ----------  -----------  -----------  -----------  ----- -----  --- ---\npaq9a    -9         19,974,112  165,193,368     13,749 s  165,207,117   3997  4021 1585 CM\nProgram      Option     enwik8      enwik9     Comp  Deco  Mem   Note\n---------    ------   ----------  -----------  ----  ----  ----  ----\nbwtmix v1    c3334    20,608,793  170,596,616  3413  1253  1670   26\n             c10000   20,608,793  167,978,527  1793   690  5000   49\n```\n\n[uda 0.300](http://www.wex.cn/dwing/download/uda0300.zip) is a free, experimental\nfile compressor by dwing, July 16, 2006.  It is a modification of PAQ8H with optimizations\nfor speed.  It takes no options.  The decompresser size is for uda.exe, since this is smaller\nthan the corresponding zip file.\n## .1678 BWTmix\n\n[BWTmix v1](http://ctxmodel.net/files/mix_test/BWTmix_v1.rar)\n(from [here](http://ctxmodel.net/)) is a free, open source, experimental\nfile compressor by Eugene Shelwien, June 28, 2009. It uses BWT (implemented using\nquicksort) followed by an 8 model CM mixed using a tree of 2-input mixers.\nThe option c10000 selects a block size of 10000 * 100KB. The default block\nsize is 100 MiB. Memory usage is 5x block size.\n## .1694 lrzip\n\n[lrzip](http://ck.kolivas.org/apps/lrzip/) 0.40 is a free, open\nsource file compressor by Con Kolivas, Nov. 26, 2009. It uses a range\ndictionary preprocessor to remove long range redundancies (based on rzip),\nfollowed by lzma (7zip) compression. It also has options to compress with\nlzo (lzop) or bzip2 after preprocessing, or to output the preprocessed\ndata for compression with other programs. It runs under Linux.\n\n[lrzip 0.42](http://freshmeat.net/projects/lrzip/releases/308861)\nadds zpipe (zpaq cmid.cfg) as a back end compressor\nusing option -z. It was tested in this mode.\n\n[lrzip](http://lrzip.kolivas.org) 0.612\n\n```\nProgram     Options            enwik8      enwik9         prog       total       Comp  Deco Mem  alg  note\n----------  ------------     ----------  -----------     --------  -----------   ----  ---- ---- ---- ----\nlrzip 0.40                   25,190,577  214,903,304     38,173 x  214,941,477    843    31 1700 LZ77 33\nlrzip 0.42  -z               21,327,441  183,609,156     49,881 x  183,659,037   2173  2230 1800 CM   33\nlrzip 0.612 -z -L 9 -p 1     19,847,690  169,318,794     99,363 x  169,418,157   2987  2929 2700 CM   33\n```\n\n[bzip3](https://github.com/kspalaiologos/bzip3) is a free, open source file compressor\nby Kamila Szewczyk. The tested version is as of Apr. 27, 2025. It uses BWT. The option -b 511\nselects the maximum block size of 511 MB. The program size was after removing the large\ntext files shakespeare.txt and shakespeare.txt.bz3 from the zipped download.\n## .1707 cm4_ext\n\n[cm0](http://encode.su/attachment.php?attachmentid=2582&d=1386201812),\n[cm0_ext](http://encode.su/attachment.php?attachmentid=2583&d=1386201812),\n[cm1](http://encode.su/attachment.php?attachmentid=2622&d=1386604070)\n[(discussion)](http://encode.su/threads/1835-CM0), and\n[bwcm](http://encode.su/attachment.php?attachmentid=2601&d=1386380149)\n[(discussion)](http://encode.su/threads/1834-BWCM)\nare a series of free file compressors for Windows by Nauful.\ncm0 is a context mixing compressor released Dec. 4, 2013.\ncm0_ext is a slower version of the same program with better compression\nreleased Dec. 4, 2013.\ncm1 uses ROLZ and was released Dec. 5, 2013.\nbwcm used BWT and was released Dec. 6, 2013.\nOnly bwcm takes any options. The command c128 uses a 128 MB block size.\nThe default is c16. It requires 12x block size in memory for compression\nand 5x for decompression. All programs are single-threaded.\n\n[cm4_ext](http://encode.su/threads/1866-CM4?p=36346#post36346) was released Jan. 21, 2014. It is an order 10 CM with\na match model and SSE.\n\n```\n                Compression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- --- ----\ncm0                              23,276,242  206,929,764     201,213 x 207,130,977   1731  1791   68  CM  26\ncm0_ext                          21,156,055  181,772,665     201,303 x 181,973,968   4206  4250  516  CM  26\ncm1                              28,092,863  243,631,412     202,038 x 243,833,450    391   226  211  CM  26\nbwcm                             23,265,333  204,416,216     202,803 x 204,619,019   1142   335  184  CM  26\nbwcm              c128           21,278,364  185,473,048     202,803 x 185,675,851   1525   407 1469  CM  26\ncm4_ext                          20,188,048  170,566,799     204,782 x 170,771,581   4123  4130 1906  CM  26\n```\n\n[M1 0.2a](http://freenet-homepage.de/toffer_86/m1_0.2a_081002.7z)\nis a free, open source (GPL) file compressor by Christopher Mattern,\nreleased Oct. 3, 2008. It uses context mixing with only two contexts.\nThe contexts are 64 bits with some bits masked out. The masks and several other\nparameters were selected by a combination of a genetic and hill climbing\nalgorithms running for several hours to 3 days\nto optimize compression on this benchmark as discussed\n[here](http://encode.su/forum/showthread.php?t=159).\n\n[M1 0.3](http://freenet-homepage.de/toffer_86/m1_0.3_090101.7z)\nwas released Jan. 2, 2009.\n\n[M1 0.3b](http://freenet-homepage.de/toffer_86/m1_0.3b_090412.7z)\nwas released Apr. 12, 2009. This version takes a configuration file created\nby an optimization version of the program. The configuration file is required\nby the decompresser (and is included in the program size).\n\n[e8-m103b1-mh](http://www.encode.su/forum/attachment.php?attachmentid=476&d=1239907880)\nis a parameter file for M1 0.3b obtained by mhajicek after about 3 days of CPU time\nrunning M1's genetic optimization program on enwik8.\n\n[M1x2 v0.5-1](http://freenet-homepage.de/toffer_86/m1x2_0.5_091208.7z)\nwas released Dec. 8, 2009. The option 6 means to use 48 x 2<sup>6</sup> MB memory.\nThe option enwik7.txt is an optimization file which resulted from tuning parameters\non the first 10 MB of the benchmark by a separate optimization process. It must be\nspecified during decompression. The file size (242 bytes) is included in the\ndecompresser size. The program includes source code and compiled Windows and Linux\nversions. The Windows version was tested. The program is described as follows\nby the author:\n\nM1x2 mixes two ordinary M1 models in the logistic domain (thus four models in\ntotal). Data is processed bitwise with a flat decomposition. Contexts are\nmapped to states, which represent bit histories encountered under the\ncorresponding context. In this implementation contexts are restricted to byte\nmasks with some tweaks for text; the context mapping is implemented using hash\ntables. Two bit history states s1, s2 are quantised\nQ(.,.) and mapped to a linear counter to produce a prediction p = P(y=1|Q(s1,\ns2)), where y is the next bit. Afterwards two predictions are transformed into\nthe logistic domain and mixed linearily. The final prediction is: p = Sq[\n(St(p2)-St(p1))*w + St(p1) ]; St(.) and Sq(.) name stretch and squash (see PAQ)\nThere is just a single weight w in [0, 1]. The Predictions and the weight are\nupdated to minimize coding\ncost. As in previous versions a genetic optimzier can tune all degrees of\nfreedom\nto a training data set. Parameters include: contexts, state machine structure,\ncounter and mixer settings.\n\n[m1x2 v0.6](http://freenet-homepage.de/toffer_86/m1x2_0.6_100206.7z)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?p=11055#post11055),\nFeb. 8, 2010,\npreprocesses the input by pre-compressing it with an order-1 12 bit length limited\nHuffman code prior to compression with the context mixing model of v0.5-1.\nThis improves speed by reducing the size of the input and improves compression\nbecause the context hash tables are not filled as quickly.\nThe 7 option says to use 8 x 2<sup>7</sup> MB memory. The decompresser size includes\nthe 242 byte configuration file enwik7.txt.\nThe length limited Huffman codes are generated using an algorithm described by\nA. Turpin and A. Moffat in\n[Practical Length-Limited Coding for Large Alphabets](http://citeseer.ist.psu.edu/old/turpin95practical.html),\nThe Computer Journal, 38, (5), 339-347, 1995.\n\n```\n                Compression     Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Notes\n-------           -------    ----------  -----------  -----------  -----------  ----- -----  --- ---  -----\nM1 0.2a                      24,656,008  219,115,069     25,336 s  219,140,405    452   447   33 CM    26\nM1 0.3                       24,004,989  215,101,056     24,596 s  215,125,652    395   404   33 CM    26\nM1 0.3b       text2.txt      23,506,215  209,057,165     23,150 s  209,080,315    377   403   33 CM    26\nM1 0.3b       text.txt       23,558,990                                           360   390   33 CM    26\nM1 0.3b       e8-m103b1-mh   23,456,037  207,931,967     23,150 s  207,955,117    383   412   33 CM    26\nM1x2 v0.5-1   6 enwik7.txt   20,812,625  172,771,031     47,608 x  172,818,639   1019  1091 1576 CM    26\nM1x2 v0.6     7 enwik7.txt   20,723,056  172,212,773     38,467 s  172,251,240    711   715 1051 CM    26\n```\n\n[cmm1](http://freenet-homepage.de/toffer_86/cmm1-src.7z) is a free,\nopen source (GPL) file compressor by Christopher Mattern, Sept. 18, 2007.\nIt uses context mixing with LZP preprocessing.\n\n[cmm2](http://freenet-homepage.de/toffer_86/cmm2_09122007.7z)\nwas released Dec. 10, 2007 without source code.\n\n[cmm2 080113](http://freenet-homepage.de/toffer_86/cmm2-080113.7z)\nwas released Jan. 13, 2008 without source code.\n\n[cmm3 080207](http://freenet-homepage.de/toffer_86/cmm3_080207_test.7z)\n(test release) was released Feb. 7, 2008 without source code.\n\n[cmm4 v0.0](http://freenet-homepage.de/toffer_86/cmm4.exe)\n(test release) was released Mar. 14, 2008 without source code.\n\n[cmm4 v0.1e](http://freenet-homepage.de/toffer_86/cmm4_01e_080420.7z)\nwas released Apr. 20, 2008 without source code.  It takes a 2 digit option \"wm\"\n(e.g. 96 meaning w=9, m=6).  Memory usage is 2<sup>w</sup> MB for a sliding\nwindow, and 12*2<sup>m</sup> MB for a context mixing model\n(order 1,2,3,4,6).  On my machine m=7 caused disk thrashing.\n\nDescription by the author: CMM4 0.1e Is a variable order context mixing coder, it predicts using the four \"highest\" (ranking: 643210) models in each bit coding step and, in addition, the match model input. Orders 0 and 1 are implemented using a table lookup, all higher orders use nibble based hashing. Matches are found using order 4 and 6 LZP, the pointers and a quick exclusion hash are stored within the model's hashing tables. The mixer joins the 4 (or 5 in presence of a match model) predictions and outputs them to a SSE stage. A mixer (similar to (L)PAQ) is selected based on the last byte's 4 MSBs and on the coding order. The SSE context is made of an order 0 context and qunatized combination of the previous symbol rank, the match length and partially matched symbol. This results in a notable compression increase on redundant data. The model's counters are quantized using the PAQ's state machine since CMM4 (will be replaced). Despite the use of hashing most data structures are tuned to never cross a cache line per nibble (the models) or octet (the mixer) (only SSE does). The core compression performance is equivalent to LPAQ1/2, while being faster. In addition there's a filter framework, which currently implements an x86 transform and will be extended.\n\n```\nCompression           Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram      Opt     enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------      ---   ----------  -----------  -----------  -----------  ----- -----  --- ---\ncmm1               23,495,627  207,266,867     18,785 x  207,285,652   1165  1198   50 CM\ncmm2               23,477,008  208,268,161     17,901 x  208,286,062   1756  1849   32 CM\ncmm2 080113        22,303,128  191,477,052     18,263 x  191,495,315   2180  2127  329 CM\ncmm3 080207        21,212,766  179,633,451     18,700 x  179,652,151   2328 ~2609  395 CM\ncmm4 v0.0          21,459,665  186,395,591     18,042 x  186,413,633   1807  1849  116 CM\ncmm4 v0.1e   96    20,569,034  172,669,955     31,314 x  172,701,269   2052  2056 1321 CM\ncmm4 v0.2b   87    20,550,129  171,969,035                                        1803 CM  42\n```\n\n[lstm-compress](https://github.com/byronknoll/lstm-compress) is a free, experimental open source\nfile compressor by Byron Knoll, June 15, 2017. It takes no options. It uses the LSTM neural network\nmodel and dictionary preprocessor from CMIX but omits the other models.\n\nA new version v2 of lstm-compress was released Dec. 12, 2017.\n\nv3 was released Mar. 30, 2019.\n\n```\n                             Compression      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                        Options       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                        -------     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nlstm-compress v1                           20,488,816  175,708,405    154,379 s  175,862,784 433968 433783  10 LSTM 66   \nlstm-compress v2                           20,494,577  174,868,709    157,238 s  175,025,947 114764 114908   9 LSTM 83\nlstm-compress v3                           20,318,653  173,874,407    144,567 s  174,018,974  92342  91876   9 LSTM 83\n```\n\n[ccm 1.1.1a](http://www.encode.su/downloads/ccm_1.1.1a.zip)\n(Feb. 23, 2007) has only one version.\n\n[ccm 1.1.2a](http://www.encode.su/downloads/ccm112a.zip)\n(Mar. 2, 2007) includes a ccm_low version using less memory, which was not tested.\n\n[ccm 1.20a](http://www.encode.su/downloads/ccm120a.zip)\n(Mar. 21, 2007) has only one version.\n\n[ccm 1.20d](http://www.mytempdir.com/1286551) (Apr. 8, 2007)\nhas two versions: ccm using 99MB memory and ccmx using 210 MB for better\ncompression.  Only ccmx was tested.\n\n[ccm 1.21](http://www.mytempdir.com/1304617)\n[(mirror)](http://www.geocities.com/lovepimple_mail/)\n(Apr. 22, 2007) \nincludes an option to select memory usage.  7 selects maximum memory, 1300 MB.\nOnly the high compression version (ccmx) was tested.\n\n[ccm 1.30](http://rapidshare.com/files/81969962/ccm130.zip.html)\n[(mirror)](http://www.geocities.com/lovepimple_mail/)\nwas released Jan. 7, 2008.  Only ccmx 7 (high compression version, \nmaximum memory) was tested.\n\n```\nCompression           Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram              enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------            ----------  -----------  -----------  -----------  ----- -----  --- ---\nccm       1.0.3a   27,667,346  240,296,736      7,217 x  240,303,953    676   679   17 CM\nccm_high  1.0.3a   25,412,726  221,177,776      7,229 x  221,185,005   1119  1171   17 CM\nccm_extra 1.0.3a   24,027,805  207,273,926      7,230 x  207,281,156   1341  1353  100 CM\nccm       1.1.1a   22,824,629  197,271,467      9,019 x  197,280,486   1247  1252   82 CM\nccm       1.1.2a   22,675,768  195,965,427      8,502 x  195,973,929   1161  1183   83 CM\nccm       1.20a    21,350,295  182,784,655     13,346 x  182,798,001   1794  1801  210 CM\nccmx      1.20d    21,310,303  182,379,461     13,468 x  182,392,929   1383  1485  210 CM\nccmx 7    1.21     20,819,656  174,161,536     21,139 x  174,182,675   1521  1493 1324 CM\nccmx 7    1.30     20,857,925  174,142,092     15,014 x  174,157,106   1313  1338 1332 CM\n```\n\n[bit 0.1](http://www.osmanturan.com/bit.zip)is a free, closed\nsource file compressor by Osman Turan, Dec. 19, 2007.  It uses ROLZ optimized\nfor binary files.  It takes no options.\n\n[bit 0.2b](http://www.osmanturan.com/bit02.zip) is an archiver,\nreleased June 14, 2008.\nOption -m lwcm selects the compression type (lightweight context mixint).\nThis is the only type supported. Option -mem 9 selects maximum memory.\nThis option ranges from 0 to 9 and uses 3 + 2<sup>opt</sup> MB memory.\nThe program uses order 1, 2, 3, 4, and 6 context mixing with 2 SSE stages\nas discussed [here](http://www.encode.su/forum/showthread.php?p=1052#post1052).\nComments by author:\n\nLWCX (Light-Weight Context Mixing) is a codec of BIT Archiver. It's designed for getting high compression ratio with acceptable speed (Not enough fast currently). LWCX is a bit-wise context mixing schema which tries to mix order-n models (order 012346). The statistics are gathered by the counters which predict next bit by semi-stationary update rule. After gathering the predictions from all models, a neural network (similar to PAQ's neural network) tries to output a new mixed prediction. The mixed prediction is processed by a 2D SSE stage which have 32 vertices. Finally, a carryless arithmetic coder codes the given bit with final prediction.\n\nMost of data structures are designed for avoiding cache misses. Order-0 and order-1 models' statistics stored in a direct lookup table. Higher orders (order 2346) models' statistics stored in a large hash table. Hash table size can be selected by \"-mem N\" option (memory usage is 3+2^(N+1) MB, N ranges 0 to 9). The codec locates a hash entry per only coding nibble.\n\n[bit 0.7](http://www.osmanturan.com/bit07.zip) has options\n-p=1 through -p=5 to select memory usage of 10 + 20*2<sup>p</sup> MB.\n\n```\nCompressor       Opt      enwik8      enwik9         Prog      Total       Comp Decomp  Mem Alg  Note\n---------        ---    ---------   -----------     -------  -----------   ----  ----   --- ---- ----\nbit 0.1                 31,186,930  271,705,328    35,400 x  271,740,728    535    83    35 ROLZ\nbit 0.2b -m lwcm -mem 9 21,971,587  189,881,180    63,665 x  189,944,845   2708  2747  1052 CM\nbit 0.7  -p=5           20,823,204  174,425,039    62,493 x  174,487,532   2050  2100   663 CM   26\n```\n\n[mcomp](http://www.msoftware.biz/blog/2008/08/winrk-progress-and-new-codecs)\n[x32 v2.00](http://www.msoftware.biz/webfm_send/16) is a free, closed source,\ncommand line file\ncompressor by Malcolm Taylor (author of WinRK), released Aug. 23, 2008. It uses a large\nnumber of algorithms, although not the same ones as WinRK. There is a 32 bit version\n(mcomp_x32.exe) and a 64 bit version (mcomp_x64.exe) for Windows. Only the 32 bit\nversion was tested (in 32-bit Vista). It displays the following help message:\n\n```\nLibMComp Demo Compressor (v2.00).\nCopyright (c) 2008 M Software Ltd.\n\nmcomp [options] pofile(s)\n\nOptions:\n    -m[..]    Compression method:\n              b    - BZIP2.\n              c    - Experimental DMC codec.\n              d    - Optimised deflate (df - fast, dx - max)\n              d64  - Optimised deflate64 (d64f - fast, d64x - max)\n              lz   - Optimised LZ (lzf - fast, lzx - max)\n              f    - Optimised ROLZ (ff - fast, fx - max)\n              f3   - Optimised ROLZ3 (f3f - fast, f3x - max)\n              p    - PPMd var.J.\n              sl   - Bitstream (LSB first).\n              sm   - Bitstream (MSB first).\n              w    - Experimental BWT codec.\n    -MNN[k,m] Model size (in kb (default) or Mb, default 64M).\n    -oNN      Order (for Bitstream and PPMd).\n    -np       Display no progress information.\n```\n\npofile(s) means input file and output file. When run with no compression options, the program decompresses. Test results are as follows on a dual core 2 GHz Pentium T3200 with 3 GB as in note 26.\n\n```\nCompressor Opt                 enwik8      enwik9         Prog      Total       Comp Decomp  Mem Alg  Note\n---------  ---               ---------   -----------     -------  -----------   ----  ----   --- ---- ----\nmcomp_x32  -mb               29,997,076                                         2070   970     4 BWT  -M has no effect\n           -mc               23,546,185                                         1350  1410    50 DMC\n           -mc -M512m        22,561,089                                         1520         322 DMC  max memory\n           -mdf              fails\n           -md               35,436,114                                         2140  1421     4 LZ77 fails\n           -mdx              35,383,881                                         2240  1420     4 LZ77 fails\n           -md64f            fails\n           -md64x            32,983,178                                        28930  1310     4 LZ77 fails\n           -mlz              24,648,445                                         3090    50   595 LZ77\n           -mf               24,331,132                                         2240    78   149 ROLZ\n           -mf -M1800m       23,187,091                                         3320    77   414 ROLZ\n           -mfx -M1800m      23,182,541                                         3410    81   414 ROLZ\n           -mf3x -M1800m     23,098,116                                         3850   112   415 ROLZ\n           -mp -M1800m -o10  21,039,213  177,948,781   172,531 x  178,121,312   4580 12180  1847 PPM\n           -mp -M1800m -o12  20,917,657  179,193,238   172,531 x  179,365,769   5180        1847 PPM\n           -mp -M1800m -o16  20,868,127  181,150,814   172,531 x  181,323,345   5750        1847 PPM\n           -msl -M1800m -o12 54,428,147                                         6510  6480     1 CM?  -M has no effect\n           -msm              59,731,673                                         5880  5810     1 CM?  -M has no effect\n           -mw               21,805,857  188,095,082   172,531 x  188,267,613    356   232   660 BWT  2 cores\n           -mw -M180m        21,103,670  179,838,392   172,531 x  180,010,923    329   284  1850 BWT  2 cores\n           -mw -M320m        21,103,670  174,388,351   172,531 x  174,560,882    473   399  1643 BWT  1 core\n```\n\n-mb produces bzip2 compatible format. -M has no effect. Memory usage is fixed at 4 MB.\n\n-mc uses DMC. If memory is greater than -M512, then the program aborts with an assertion failed.\n\n-md and -md64 are supposed to generate deflate and deflate64 formats (zip or gzip). However -mdf and -md64f (fast modes) crash immediately during compression. The other modes decompress to files that are the correct size but not identical to the original. Run times are very slow due to most of the CPU time spent in the kernel (up to 90%) as reported by timer 3.01.\n\n-mp used PPMD var. J, but allows more memory (up to about 1800 MB). The original program was limited to 256 MB. The optimal orders are different for enwik8 and enwik9. Higher orders help compression, but lower orders save memory on larger files. The maximum order is -o16. Higher values have no effect. Decompression is slow due to 55% of the CPU time spent in the kernel. Normally this is around 1% and decompression speed would be the same as compression.\n\n-msl and -msm ignore the -M option and use 1 MB memory, resulting in poor compression.\n\n-mw (experimental BWT) is the only option that uses both cores. All others result\nin 50% CPU usage on a 2 core processor. The -M option actually\nselects the block size, not total memory usage. Memory usage is 5x block size if one core is used,\nor 10x if both are used. Both are used only if enough memory is available. The default is to\nsplit the file in half and compress the two halves in parallel. However, better but slower compression\ncan be obtained by using -M to select one block for the whole file. Maximum memory is 2 GB, even\nif more is available. For enwik9, -M320 selects 3 blocks, which are compressed in series on one core.\nFor two cores, time reported is wall time.\nProcess time for -mw -M320m is 187% of wall time for compression and 139% for decompression.\n \n\n```\n  epmopt -m800 -n20 --fixedorder:12 enwik6 .\n  epm c01286014321245957352513 enwik9 enwik9.epm -m800\n  epm d01286014321245957352513 enwik9.epm enwik9.tmp -m800\n```\n\n Warning: epm failed to decompress correctly on enwik7 (first\n10 \n\n```\n                             Compression      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                        Options       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                        -------     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\ndark 0.32b  July  9, 2006      -b128m      21,414,479  185,844,554     31,076 x  185,875,590    481   407  790 BWT\ndark 0.40b  Aug. 14, 2006      -b128mf0    21,243,259  184,271,115     34,688 x  184,305,803    471   316  790 BWT\ndark 0.46   Aug. 23, 2006      -b160mf0    21,231,325  181,904,374     40,780 x  181,945,154    488   404  813 BWT\n                               -b333mf0    21,231,325  175,955,412     40,780 x  175,996,192    432   425 1692 BWT\nopendark A  Nov. 14, 2006      -b333m      21,432,727    (fails)       10,089 s                 450   390 1692 BWT\n                               -b127m      21,432,727  185,985,101     10,089 s  185,995,190    389   331  652 BWT  26\ndark 0.51   Jan.  2, 2007      -b333mf     21,169,819  175,471,417     34,797 x  175,506,214    533   453 1692 BWT\n```\n\n ppmd generally gives the best compression for text.  It will also call ppmonstr\nas an external program, but this mode was not tested, even though it compresses better.\n For this test, the Windows command line version was tested.  The option \n-mppmd:1012m:o13:r1 is equivalent to ppmd -m1012 -o13 -r1, selecting 1012 MB memory,\norder 13, and partial reinitialization of the model when memory is exhausted.\nNote that ppmd normally allows only up to -m256.  This program was tested with 2 GB\nmemory but values higher than -m1012 caused the program to crash during compression.\n \n\n```\n                             Compression      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                        Options       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                        -------     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nFreeArc 0.36        -m9 -lc1600000000      21,153,231  184,498,111    372,457 s  184,870,568    665   517 1600 PPM\nFreeArc 0.40 pre-4  -mppmd:1012m:o13:r1    20,931,605  175,254,732    748,202 x  176,002,934   1175  1216 1046 PPM\nFreeArc 0.666       -m9                    21,659,587  189,696,374  1,214,530 x  190,910,004    524   416  785 PPM  26\nFreeArc 0.67a       -m1                    39,485,049                                            25    27  191      26\n                    -m2                    26,831,928                                            59   121  117      26\n                    -m3                    25,221,359                                           147   100  157      26\n                    -m4                    24,285,483                                           174   132  155      26\n                    -m5                    23,020,671                                           410   443  311      26\n                    -m6                    21,659,587                                           570   471  463      26\n                    -m7                    21,659,587                                           592   477  463      26                     \n                    -m8                    21,659,587                                           604   495  448      26\n                    -m9                    21,659,587  189,696,374    148,665 xd 189,845,039    519   420  813      26\n                    -m1x                   39,485,049                                            27    25  194      26\n                    -m2x                   34,307,417                                            73    28  170      26\n                    -m3x                   27,336,122                                           269    32  186      26\n                    -m4x                   25,652,947                                           357    45  189      26\n                    -m5x                   24,897,495                                           564    43  204      26\n                    -m6x                   23,870,179                                           522    41  453      26\n                    -m7x                   23,788,636                                           546    41  599      26\n                    -m8x                   23,788,633                                           565    41  584      26\n                    -m9x                   23,788,633                                           567    41  584      26\n```\n\n[<sup>6</sup> bytes\nof the input file.  epmopt compressed this about 100 times in\n368 seconds with different options, making 35 passes through\nthe list of 20 undocumented parameters, adjusting each one up\nor down one at a time.  The fixed parameters\nwere -m800 (800 MB memory limit) and PPM order 12 (--fixedorder:12,\nalso the first 3 digits of the parameter string.  Allowing epmopt\nto set the PPM order on a smaller training file will cause it to\nchoose too large a value, hurting compression.  I only tested\norders 10, 12, and 20 on enwik8 and 12 gave the best compression).\nThe -n20 option tells epm to tune all 20 parameters.  The parameter\nstring is written to the file enc.ini.  The -m800 option need\nnot be the same for epmopt and epm but must be the same\nfor epm during compression and decompression.]\n## .1749 epmopt | epm\n\n[epmopt + epm](http://www.compression.ru/so/) r9 is an experimental,\nclosed source\ncommand line optimizer and file compressor by Serge Osnach, Oct. 16, 2003.  It was\nintended for enc r16, but development on that project has stopped at enc r15, according\nto the web page (in Russian).  The program has two parts: epm, a\nPPM compressor with text preprocessing, and epmopt, which attempts to optimize\nthe parameters to epm by compressing repeatedly and varying the options one at a\ntime until there is no more improvement.  The input to epmopt may be different\nthan epm, and supports optimization on sets of files matching patterns in\nspecified sets of directories.  The options to epm are memory limit, PPM order,\nand 20 undocumented options each specified by a single digit.  The exact same options\nmust be passed to the decompresser.  In the results, I added 27 bytes to the\ncompressed file sizes to account for this information.  enwik9 was compressed\nand decompressed as follows:\n<sup>7</sup> bytes).  In the output, some linefeeds were changed\nto spaces.  This happened with all parameter combinations I\ntested including defaults: `epm c enwik7 enwik7.epm`.\nDecompression was bit-exact for enwik5, enwik6, enwik8 and enwik9.\n## .1749 WinUDA\n\n[WinUDA](http://dwing.51.net/download.htm) 0.291 is a\nfree, closed source GUI\narchiver by dwing, July 4, 2005.  It uses context mixing and is\nderived from paq6.  Mode 3 is the slowest (about 3x slower than\nmode 0) and uses the most memory, 194 MB.\n## .1755 dark\n\n[dark](http://darchiver.narod.ru/) v0.51 is a free, closed source\narchiver by Malyshev Dmitry Alexandrovich, Jan. 2, 2007.  It uses BWT + distance coding without preprocessors.\nThe -b333m option selects 333 MB\nblocks. -f (-f0 in 0.40 and 0.46, not supported in 0.32) forces no segmentation. \nMemory usage is 5 times the block size for compression\n(6x prior to v0.46).\n[opendark ver. A](http://code.google.com/p/adark/)\nis an open source version of dark.  The supplied Windows dark.exe\ncrashed when decompressing enwik9 (size is 177,675,818).\nDecompression works up to -b127m. opendark does not support the -f option.\n is a free, open source\narchiver by Bulat Ziganshin, Dec. 16, 2007.  It compresses using ppmd, GRZipII, and LZMA\nalong with multimedia filters, a dictionary preprocessor and a REP preprocessor\nfor removing repeating strings.  It has Windows and Linux versions and an optional GUI.\n## .1760 FreeArc\n\n[FreeArc](http://www.haskell.org/bz/) 0.36 is a free, open source archiver\nby Bulat Ziganshin, Feb. 21, 2007.  It incorporates 7 compression libraries - PPMd, \nGRZipII, LZMA (7zip), plus BCJ (7zip), REP (rzip-like), dynamic dictionary and LZP \npreprocessors. The option -m9 selects maximum compression (dict + LZP + PPMd for text \nfiles, REP+LZMA for binary).  -lc1600000000 limits\nmemory to 1.6 GB (same as -lc1600m).  There is an option to use ppmonstr as an external\ncompressor, which was not included in the test.\n[FreeArc](http://www.haskell.org/bz/) 4.0 pre-4[FreeArc](http://freearc.org/)\n0.666 was released May 19, 2010. The 32 bit Windows console version was\ntested. -m9 selects maximum compression. There are many other compression options\nbut these were not tested.\n[freearc 0.67a](http://freearc.org/download/testing/FreeArc-console-0.67-alpha-win32.exe) was released Mar. 15, 2014. Options -m1 to -m9 select the compression\nlevel from fastest to best. -m1x to -m9x select levels with fast decompression.\nDecompression was tested with the separate unarc.exe program.\n## .1766 hook\n\n[hook](hook.zip) v0.2 is a free, \nopen source (GPL) command line file\ncompressor by Nania Francesco Antonio, Jan. 8, 2007.  It uses DMC: a state machine\nin which each state represents a bitwise context.  Each state has 2 outgoing\ntransitions corresponding to next bits 0 and 1, and a count n0 or n1 associated\nwith each transition.  Bit y (0 or 1) is compressed by arithmetic coding with probability\nny/(n0+n1) (where ny is n0 or n1 according to y), and then ny is incremented.\n\nAfter each input bit, the next state represents a context obtained by appending that bit on the right and possibly dropping bits on the left. States are cloned (copied) whenever the incoming and outgoing counts exceed certain limits. This has the effect of creating a new context in which no bits are dropped. In the example below, the state representing context 110 (dropping 2 bits from the previous context) is cloned by creating a new state 11110 because the incoming 0 transition count (ny for y=0) from state 1111 exceeded a limit. The new context is longer because it does not drop any bits. This transition is moved to point to the new state. Other incoming transitions (not shown) remain pointing to the original state. The outgoing transitions are copied. The counts of the original state are distributed to the new state in proportion to the moved transition's contribution to those counts, which is w = ny/(n0+n1).\n\n``` php\n                n0 ----> 1100           n0*(1-w) ----> 1100\n         ny       /                             /     /\n   1111 -----> 110               1111        110     /\n        (y=0)     \\                 |           \\   /\n                n1 ----> 1101       |   n1*(1-w) ----> 1101\n                                    |             /    /\n                                    |     n0*w   /    /\n                                    | ny        /    /\n                                    +----> 11110    /\n                                                \\  /\n                                          n1*w   --\n\n        Before cloning            After cloning 110 to 11110\n```\n\nNormally, the initial set of contexts begin on byte boundaries. The cloning mechanism ensures that new contexts also have this property.\n\nIn hook v0.2, the counts are 32 bit floating point numbers initialized to 0.1. The initial state machine has 256*255 states representing bytewise order 1 contexts with uniform statistics. When memory is exhausted, the model is discarded and the state machine is reinitialized. A new state is cloned when ny > limit and n0+n1-ny > length, where limit and length are parameters. The optimal parameters for enwik8 and enwik9 are \"c 7 2 6\", c means compress, 7 selects the maximum of 1 GB memory (64M states at 16 bytes each, minimum is 8 MB memory), 2 is the limit (range 1 to 7), and 6 selects a length of 32 (possible values are 1, 2, 3, 4, 8, 16, 32, 64). Larger lengths are better for large files because they conserve memory at the expense of compression.\n\nhook v0.3 (Jan. 11, 2007) allows up to 1.8 GB memory (first option = 9) and uses double precision predictions in the 32 bit arithmetic coder.\n\nhook v0.3a (Jan. 12, 2007) initializes the counts to 0.125 (instead of 0.1) and uses 24 bit precision in the arithmetic coder (instead of 32 bit).\n\nhook v0.4 (Jan. 15, 2007) initializes counts to 0.1. Argument 2 selects length 3 (not 2).\n\nhook v0.5b (Jan. 22, 2007) adds an LZP preprocessor. If the next byte to be coded is the same as the byte that occurred in the last matching 3 byte context, then this is indicated by coding a flag bit in an order 3 model (32 MB memory), and a match length coded by DMC with a fixed size of 128 MB. If there is no match, then the literal byte is coded by another variable sized DMC model. The parameters \"c 1600000000 2 64 1 6\" select compression (c), 1.6 GB for the DMC literal model (1600000000), a limit of 2 (minimum count for the cloned state), length of 64 (minimum remaining count for the state to be cloned), LZP selected (1), and a minimum match length of 6.\n\nhook v0.6 (Feb. 7, 2007) removes the \"length\" parameter (effectively infinite). The arguments \"c 1600 4 1 6\" mean to compress (c), use 1600 MB memory, set the \"limit\" parameter to 4, turn on LZP preprocessing (1) with a minimum match length of 6. The \"limit\" parameter is the minimum count for an outbound DMC state transition to clone the state. Limit was tuned on enwik8.\n\nhook v0.6b (Feb. 8, 2007) includes support for files up to 2<sup>64</sup> bytes (compiled\nby Ilia Muraviev.  Earlier versions were compiled with MinGW g++ 3.4.5 by Matt Mahoney.)\n\"limit\" was tuned on both enwik8 and enwik9.  Higher values\nconserve memory at the expense of compression on smaller files.\n\nhook v0.6c (Feb. 14, 2007) stores the input filename in the compressed file and uses it during decompression.\n\nhook v0.7 (Mar. 10, 2007) uses 325 MB more memory than advertised so it was tested with a lower option.\n\nhook v0.7b (Mar. 12, 2007) reduces the excess memory to 94 MB.\n\nhook v0.8 was released Mar. 17, 2007. Some additional results on enwik9 decreasing the rate at which the state machine fills up and is flushed:\n\n```\nhook08 params    enwik9\n------------  -----------\nc 1700 1 1 6  183,175,857\nc 1700 2 1 6  181,578,888\nc 1700 3 1 6  181,220,553\nc 1700 4 1 6  181,268,867\nc 1700 5 1 6  181,197,310\nc 1700 6 1 6  181,567,697\nc 1700 7 1 6  181,813,763\nc 1700 8 1 6  182,360,391\n```\n\nhook v0.8b (Mar. 18, 2007) has some LZP improvements.\n\nhook v0.8c (Mar. 19, 2007) is a minor bug fix. Compressed sizes are 1 byte larger than v0.8b.\n\nhook v0.8d was released Mar. 21, 2007.\n\nhook v0.8e was released Mar. 27, 2007.\n\nhook v0.9 (Apr. 6, 2007) is closed source. It requires a processor that supports SSE instructions. It has some speed improvements and a E8/E9 filter for improved compression of .exe files. Memory usage is the second argument + 60MB.\n\n[freehook 0.2](http://eugene.ath.cx/projects/freehook-0.2.zip)\nis an open source port of hook v0.8e from C++ to C by Eugene Ortmann, Apr. 7, 2007.\nThe supplied .exe file requires SSE instructions (Pentium 3 or higher),\nbut the source can be recompiled for other processors.\n\nhook v0.9b (Apr 10, 2007) replaces floating point arithmetic with integer arithmetic, so that archives are compatible across different processors. Note: I reduced the memory setting from 1800 to 1700 to prevent disk thrashing, which was a problem in earlier tests. I will do this from now on. This hurts enwik9 compression (but not enwik8) slightly, from 180,444,546 to 180,582,601. Actual memory usage is 60 MB over.\n\n[freehook 0.3](http://eugene.ath.cx/projects/freehook-0.3.zip)\n(Apr 10, 2007) has only very minor changes from 0.2 but is\nslightly faster due to different g++ compiler options.  Compression is the\nsame as 0.2.  Memory usage is about 160 MB over.\n\nhook v0.9c (May 8, 2007) has some speed improvements in the arithmetic coder. It compresses the same size as v0.9b.\n\nhook v1.0 (Sept. 20, 2007) is closed source. The only option is memory size in MB.\n\nThe zip file linked above contains all versions (C++ source and Win32 .exe).\n\nhook 1.1 (Nov. 13, 2007) improves BMP and WAV compression.\n\nhook 1.3 was released Dec. 14, 2007, modified Dec. 15, 2007.\n\n[hook 1.4](http://heartofcomp.altervista.org/) was\nreleased Apr. 29, 2009.\n\n```\nCompression                             Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options                  enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------       -------                ----------  -----------  -----------  -----------  ----- -----  --- ---\nhook v0.2     c 7 2 6                23,628,061  208,211,084      2,556 s  208,213,640    772   779 1052 DMC\nhook v0.3     c 9 2 6                23,548,017  202,024,740      3,567 s  202,028,307    849   864 1764 DMC\nhook v0.3a    c 9 2 6                23,499,700  201,934,976      3,555 s  201,938,531    862   832 1764 DMC\nhook v0.4     c 9 2 6                23,349,695  199,829,234      4,112 s  199,833,346    934   959 1764 DMC\nhook v0.5b    c 1600000000 2 64 1 6  22,806,402  193,227,085      5,113 s  193,232,198   1084  1029 1764 LZP+DMC\nhook v0.6     c 1600 4 1 6           22,472,884  191,733,561      5,112 s  191,738,673   1146  1034 1600 LZP+DMC\nhook v0.6b    c 1600 4 1 6           22,535,069  189,932,778      5,174 s  189,937,952   1040       1600 LZP+DMC\n              c 1600 6 1 6           22,776,927  188,384,238      5,174 s  188,389,412   1090  1026 1600\nhook v0.6c    c 1600 6 1 6           22,561,621  188,081,694      5,878 s  188,087,572   1131  1092 1600 LZP+DMC\nhook v0.7     c 1000 6 1 6           22,410,669  191,516,313      6,195 s  191,522,508   1360  1353 1375 LZP+DMC\nhook v0.7b    c 1700 6 1 6           22,404,817  184,765,030      6,195 s  184,771,225   1516  1655 1794 LZP+DMC\nhook v0.8     c 1700 5 1 6           22,290,033  181,197,310      6,686 s  181,203,996   1110  1118 1700 LZP+DMC\nhook v0.8b    c 1700 5 1 6           22,399,354  180,335,788      6,944 s  180,342,732    988  1033 1700 LZP+DMC\nhook v0.8c    c 1700 5 1 6           22,399,355  180,335,789      7,071 s  180,342,860   1043  1005 1700 LZP+DMC\nhook v0.8d    c 1700 5 1 6           22,399,027  180,319,203      7,037 s  180,326,240    928   915 1700 LZP+DMC\nhook v0.8e    c 1700 3 1 6           22,039,935  178,140,788      7,263 s  178,148,051    952  1009 1700 LZP+DMC\nhook v0.9     c 1800 2 1 6           21,969,342  178,932,435     10,069 x  178,942,435    869       1860 LZP+DMC\n              c 1800 3 1 6           22,077,883  178,599,478     10,069 x  178,609,547    833   916 1860 LZP+DMC\nfreehook 0.2  c 1700 3 1 6           22,039,914  178,141,036      7,386 s  178,148,422    813   855 1860 LZP+DMC\nhook v0.9b    c 1700 3 1 6           22,496,910  180,582,601      9,278 x  180,591,879    810   810 1721 LZP+DMC\nfreehook 0.3  c 1600 3 1 6           22,039,914  178,619,149      7,352 s  178,626,501    789   818 1713 LZP+DMC\nhook v0.9c    c 1700 3 1 6           22,496,910  180,582,601      8,506 x  180,591,107    774   791 1721 LZP+DMC\nhook v1.0     c 1700                 22,122,484  177,843,658     11,163 x  177,854,821    865   879 1739 LZP+DMC\nhook v1.1     c 1700                 22,122,484  177,843,658     25,854 x  177,869,512    877   872 1739 LZP+DMC\nhook v1.3     c 1700                 22,030,108  178,216,980     13,870 x  178,230,850    825   835 1736 LZP+DMC\nhook v1.4     c 1700                 21,990,502  176,648,663     37,004 x  176,685,667    741   695 1777 LZP+DMC\n```\n\n[7zip](http://www.7-zip.org/) 4.42 is an open source GUI and command line archiver\nby Igor Pavlov, May 14, 2006.  It compresses to 7z, zip, gzip, ppmd.H and tar format,\noptionally encrypts with AES, and will uncompress several other formats.\n\n7z is the default format.  It uses LZMA compression, a variation of LZ77.\nThe option -mx=9 selects ultra (maximum) compression in this mode.  The option\n-sfx7zCon.sfx creates a console-based self extracting executable by prepending\na 131,584 byte decompresser.  This is slightly smaller than the Windows GUI version\n(132,096 bytes) and much smaller than the decompression program itself as a zipped\nself extracting download (817,795 bytes).  The best compression is with ppmd.\nThe options are -m0=ppmd:mem=768m:o=10 equivalent to ppmd var H (with minor changes)\norder 10 with 768 MB memory.\n[7zip 4.46a](http://www.7-zip.org/alpha/7z446a1.msi) was announced May 21, 2007.\n(The improved compression is due to testing with more memory).\n\n7zip 9.04a was released Dec. 3, 2009. It gave an out of memory error with mem=1630.\n\n7zip 9.20 was released Nov. 18, 2010. Default (LZMA) mode was tested. It uses 196 MB for compression using 75% of 2 cores, and 18 MB for decompression on a 2.0 GHz T3200 under Windows.\n\nThe following include the best known option combinations for 7zip on enwik8 in ppmd (PPM), 7z (LZMA), bzip2 (BWT) and zip (LZ77) formats.\n\n```\n                Compression                         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Alg  Notes\n-------           -------                        ----------  -----------  -----------  -----------  ----- -----  ---  -----\n7zip 4.42 -m0=ppmd:mem=768:o=10 -sfx7xCon.sfx    21,375,060  185,043,783          0 xd 185,043,783    505  ~500  PPM\n7zip 4.42 -m0=ppmd:mem=293m:o=7                  21,791,628                                           647   655  PPM   6\n7zip 4.42 -mx=9 -sfx7zCon.sfx                    24,996,113  213,490,979          0 xd 213,490,979   2286    63  LZMA\n7zip 4.42 -tbzip2 -mpass=2                       29,003,844                                          1974   176  BWT   6\n7zip 4.42 -tzip -mm=deflate64 -mfb=153 -mpass=8  33,727,442                                          2803    28  LZ77  6\n7zip 4.42 -tzip -mm=deflate -mfb=171 -mpass=8    35,056,389                                          2672    27  LZ77  6\n7zip 4.42 -tzip -mm=deflate -mfb=258 -mpass=8    35,057,040                                          2664    29  LZ77  6\n7zip 4.42 Zip/Ultra (in GUI)                     35,057,347                                          4307        LZ77  1\n7zip 4.46a -m0=ppmd:mem=1630m:o=10 -sfx7xCon.sfx 21,197,559  178,965,454          0 xd 178,965,454    503   546  PPM\n7zip 9.04a -m0=ppmd:mem=1500m:o=10 -sfx7zCon.sfx 21,211,895  179,209,403          0 xd 179,209,403    506   520  PPM   26\n7zip 9.12b -m0=ppmd:mem=2048m:o=10               21,060,863  177,187,967                                         PPM   42\n7zip 9.20                                        25,895,909  227,905,645    518,536 x  228,424,181   1031     42 LZMA  26\n```\n\n[rings](http://heartofcomp.altervista.org/) 0.1 is a free, closed\nsource, experimental file compressor by Nania Francesco Antonio, Sept. 21, 2007.\nIt uses LZP with order-2 coding of literals and arithmetic coding.\nIt takes no command line options.\n\nrings 0.2 (Nov. 16, 2007) includes improved BMP, WAV, TIFF, and PGM filters.\n\nrings 0.3 was released Dec. 21, 2007.\n\nrings 1.0 was released Feb. 8, 2008. It uses 50 MB for compression and 43 MB for decompression.\n\nrings 1.1 was released Feb. 13, 2008 with same memory usage. It uses CM with LZP preprocessing for faster compression.\n\nrings 1.2 was released Mar. 4, 2008 with the same memory usage.\n\nrings 1.3 was released Apr. 2, 2008. It uses 54 MB for compression and 47 MB for decompression.\n\nrings 1.4c was released Apr. 14, 2008. It has an option (1-9) which selects memory usage. Each increment doubles usage. Memory usage and run time are greater for decompression than compression. For option 9, compression uses 526 MB and decompression uses 789 MB. The program uses BWT. The transformed data is encoded using MTF (move to front), pre-Huffman coding followed by arithmetic coding.\n\nrings 1.5 was released Apr. 21, 2008. It improves compression and is symmetric with regard to memory usage. Options are like 1.4c. The table below compares timing results on my old and new computers.\n\nrings 1.6 was released Aug. 16, 2009. The option ranges from 1 to 10, where 10 uses the most memory. It includes a Linux version (18,348 bytes zipped) which was not tested.\n\n[rings 2.0](http://heartofcomp.altervista.org/RINGS/home.htm)\n[(discussion)](http://encode.su/threads/1749-Rings-BWT-Archiver-Now!)\nis a multi-threaded archiver rather than a file compressor. It uses BWT.\nIt has an interface similar to zcm. Option -m7 selects maximum block\nsize of 100 MB using 500 MB memory per thread.\nOption -t1 or -t2 selects 1 or 2 threads. On a 2 core machine,\nselecting 2 threads shows 3 processes in Windows Task Manager, two of which\nuse 500 MB memory and I/O dividing the input and output files, and\none process using 7 MB with several GB of input and a lot of kernel CPU\ntime. These 3 processes must\nshare 2 cores. As a result, it runs slower than 1 thread.\n\n[rings](http://heartofcomp.altervista.org/)\n[2.1](http://heartofcomp.altervista.org/rings2.1.zip)\n[(discussion)](http://encode.su/threads/1749-Rings-BWT-Archiver-Now!?p=43840&viewfull=1#post43840)\nwas released May 23, 2015.\n\n[rings 2.2](http://heartofcomp.altervista.org/rings2.2.zip)\nwas released May 28, 2015.\n-o option enables multi-threaded compression.\n\n[rings 2.5](http://heartofcomp.altervista.org/rings2.5.zip)\nwas released June 6, 2015. Option -o was removed. The 64 bit version was tested.\n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nrings 0.1                       35,693,969  314,161,660     11,271 x  314,172,931    187   179   16 LZP\nrings 0.2                       35,693,969  314,161,660     25,832 x  314,187,492    192   167   16 LZP\nrings 0.3                       35,151,555  309,179,126     32,132 x  309,211,258    188   154   16 LZP\nrings 1.0                       26,384,013  235,897,616     25,585 x  235,923,201    221   321   50 CM\nrings 1.1                       26,793,247  238,353,988     27,513 x  238,381,501    151   255   50 CM\nrings 1.2                       25,873,235  229,695,548     30,484 x  229,726,032    120   175   50 CM\nrings 1.3                       25,873,235  229,695,548     43,329 x  229,738,877    104   163   54 CM\nrings 1.4c        9             24,591,826  217,427,384     39,149 x  217,466,533    103   287  789 BWT\nrings 1.5         9             21,848,093  191,067,972     44,565 x  191,112,537    172   189  426 BWT\nrings 1.5         9             21,848,093  191,067,972     44,565 x  191,112,537    144   188  425 BWT  26\nrings 1.6         10            21,918,217  189,242,552     47,618 x  189,290,170    165   192  795 BWT  26\nrings 2.0         -m7 -t2       21,195,013  185,258,194    164,995 x  185,423,189    398   223  986 BWT  26\nrings 2.0         -m7 -t1       21,194,965  185,256,848    164,995 x  185,421,843    375   206  493 BWT  26\nrings 2.1         -m7 -t1       20,967,373  183,891,457    230,702 x  184,122,159    195   188 1859 BWT  48\nrings 2.2         -m7 -o        20,938,029  183,531,002    341,445 x  183,872,447    202   179 1859 BWT  48\nrings 2.5         -m8 -t1       20,873,959  178,747,360    240,523 x  178,987,883    280   163 2518 BWT  48\n```\n\n[pimple](http://lovepimple.110mb.com/) 1.43 beta is\na free, closed source GUI archiver by Ilia Muraviev, Apr. 24, 2006.  It uses\ncontext mixing.\n\n[pimple2](http://www.encode.su/downloads/pimple2.zip) is a\ncommand line file compressor, June 11, 2007.\n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----  --- --- ----\npimple 1.43 beta  512MB, order 8, match 32    20,992,830  181,998,817    353,472 x  182,352,259   9638 10112  512 CM    3\npimple2           (none)                      20,871,457  180,251,530     78,642 x  180,330,172  18474 17992  128 CM\n```\n\n[ash 04a](http://compression.ru/sh/ash04a.rar)\nis a free, experimental command line file compressor by\nEugene D. Shelwien, Dec. 5, 2003.  The /m700 option\nselects 700 MB memory limit.  (/m800 causes disk thrashing with 1 GB).\n/o10 selects model order 9.\nThis gives good results on smaller files when memory\nis constrained, but I did not try to optimize it.\nThere is a /s option to select SSE depth that\ngives good results for the default value of /s5\nso I did not try to optimize it either.  Other results:\n\n```\nash04a options           enwik9    Comp (ns/byte)\n----------            -----------  ----\n/m700 /o8  (order 7)  180,830,523  5883\n/m700 /o10 (order 9)  180,735,542  6011\n```\n\nNote: the acutal memory usage (commit charge) for enwik9 /m700 /o8 was 1910 MB at the end of compression, minus 257 MB for other programs, according to Windows task manager. This is generally not a problem if your swap file is large enough. It appears to be a slow memory leak (recovered when program exits) and does not cause thrashing.\nash /m1700 /o10 and /o12 failed to compress enwik9 with 2 GB memory\n(error: could not allocate a block).\nenwik8 compressed to 19,713,239 using /o10 and\n19,446,859 using /o12.\n## .1807 bce3\n\n[bce3](http://encode.su/threads/2150-A-new-algorithm-for-compression-using-order-n-models) is a free, open source (Apache), experimental file compressor\nby Christoph Diegelmann, Mar. 16, 2015. It uses an order-n bitwise context model\nwhere the model is computed using BWT and encoded and transmitted to the\ndecoder. Memory usage is 5 times the file size. The program takes no\noptions. I tested by compiling with g++ 4.8.3 in Ubuntu Linux.\n\n```\n                Compression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nbce3                             22,729,148  180,732,702     19,889 s  180,752,591   1151  2444  5000 CM  71\n                                 22,729,148                                          1230  2020   500 CM  48\n```\n\nocamyd LTCB 1.0 is a modification by Mauro Vezzosi on June 20, 2006 of Frank Schwellinger's ocamyd-1.65-final. The option -s0 selects maximum compression. -m3 selects 300 MB memory (the maximum for the test machine), but it supports up to -m8.\n\nocamyd 1.66.final, by Frank Schwellinger, Feb. 1, 2007, includes the -f option to prevent flushing and rebuilding the DMC model when memory is exhausted.\n\n```\n                Compression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nocamyd 1.65.final -s0 -m8        21,456,536  185,727,437     20,618 x  185,748,055  50782 50935  800 DMC\nocamyd LTCB 1.0   -s0 -m3        21,285,121  182,359,986     21,030 x  182,381,016 108960~110000 300 DMC   6\nocamyd 1.66.final -s0 -m3 -f     21,123,280  182,410,035     20,636 x  182,430,561  59130 59637  300 DMC   6\n```\n\nThe following table shows the effect of the -s and -m options on ocamyd 1.65.final on enwik8. Times are in ns/byte, process (kernel+user) time by timer 3.01, ~ indicates global (wall) time.\n\n```\nOptions    enwik8    Comp  Decomp  Notes\n-------  ----------  -----  -----  -----\n-s0 -m8  21,456,536  42030  42010\n\n-s0 -m4  22,073,527  70482  70538  6 (400 MB) (~101015 ~92921 global time)\n-s1 -m4  23,944,647 ~33535         6\n-s2 -m4  26,345,297  ~1940         6\n-s3 -m4  28,060,900  ~1826         6\n\n-s0 -m3  22,296,826 ~70960         6 (300 MB)\n-s1 -m3  24,114,574 ~33818         6\n-s2 -m3  26,911,154  ~1603         6\n-s3 -m3  28,278,662  ~1514         6\n\n-s0 -m2  22,688,950 ~70172         6 (200 MB)\n-s1 -m2  24,511,065 ~33771         6\n-s2 -m2  27,614,083  ~1562         6\n-s3 -m2  28,928,850  ~1448         6\n\n-s0 -m1  23,487,047 ~68522         6 (100 MB)\n-s1 -m1  25,280,406 ~33277         6\n-s2 -m1  29,045,902  ~1509         6\n-s3 -m1  30,080,719  ~1408         6\n\n-s0 -m0  24,210,216 ~66463         6 (64 MB)\n-s1 -m0  25,882,226 ~33121         6\n-s2 -m0  30,591,255  ~1481         6\n-s3 -m0  31,276,535  ~1377         6\n```\n\n[bee](http://www.compression.ru/fa/) 0.78 build 0154\nis an open source (Delphi Object Pascal)\ncommand line archiver (with optional GUI)\nby Andrew Filinsky and Melchiorre Caruso, Sept. 23, 2005.\nIt uses PPM.  The -m3 option select maximum compression (default\nis -m1).  The -d8 option selects 512 MB memory, the maximum that does\nnot cause disk thrashing (default is -d2 = 10 MB).\n\nbee includes beeopt, a parameter optimizer similar to epmopt.\nThis was not tested.  bee comes preconfigured with parameters\ntrained on .txt and .xml files (and other types) in file bee.ini.  This was tested by renaming\nenwik7 (first 10<sup>7</sup> bytes)\nto enwik7.txt and enwik7.xml but compression was worse.  \nThe executable size is a zip archive containing\nbee.exe and bee.ini.  This is much smaller than the zipped source code download.\n## .1829 uhbc\n\n[uhbc 1.0](ftp://ftp.elf.stuba.sk/pub/pc/pack/uhbc10.zip) is\nan experimental, closed source command line file compressor\nby Uwe Herklotz, June 30, 2003.  It uses BWT.  The -b100m option\nselects 100 MB block size, which requires 800 MB for compression\nand 500 MB for decompression.  -m3 selects maximum compression\nfor the entropy coding stage, which consists of run length coding\n(RLE) + DWFC (double weighted frequency counting) + entropy coding.\nWFC is described in \n[Deorowicz, S., \n*Improvements to Burrows–Wheeler compression algorithm*, \nSoftware–Practice and Experience, 2000; 30(13):1465–1483](http://www-zo.iinf.polsl.gliwice.pl/~sdeor/pub.htm).\n\nAdditional results on enwik8:\n\n```\nOptions                                     enwik8 size  Comp  Decomp (ns/byte)\n-----------------------------------------   -----------  ----  ------\n-m3 -b100m (one 100 MB block)                20,930,838  1145   858\n-m3 (default block size is 5 MB)             24,296,345   914   733\n-m2 (RLE + WFC + entropy coding, default)    24,411,843   806   644\n-m2 -cp (prefix sort, default is suffix)     24,589,110   813   578\n-m1 (RLE + MTF (move to front) + entropy)    25,021,683   680   547\n-m0 (RLE + direct entropy coding)            25,341,274   603   500\n```\n\n[smac v1.8](http://encode.su/attachment.php?attachmentid=2175&d=1358903568)\n[(discussion)](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding) is a free, experimental file compressor for Windows by\nJean-Marie Barone, Jan. 22, 2013. It uses an order-4 bitwise context\nmodel and arithmetic coding. It takes no options. Source code is\nin x86 assembler.\n\n[smac v1.9](http://encode.su/attachment.php?attachmentid=2179&d=1359609467), Jan. 31, 2013, uses an order 4 and order 6 context model\nand chooses at each bit the model whose prediction is further away from 1/2.\n\n[smac v1.10](http://encode.su/attachment.php?attachmentid=2185&d=1360213568), Feb. 7, 2013, uses a nonstationary model like PAQ6.\nWhen a bit count is incremented, half of the count over 2 of the other bit\nvalue is discarded.\n\n[smac v1.11](http://encode.su/attachment.php?attachmentid=2198&d=1361242474), Feb. 18, 2013, switches between order 6, 4, and 3 context\nmodels depending on which prediction is furthest away from 1/2. For files\nsmaller than 5 MB, it switches between lower order contexts.\n\n[smac v1.12a](http://encode.su/attachment.php?attachmentid=2224&d=1363004347), Mar. 11, 2013, uses indirect context models. The context\nis mapped to a 16 bit state representing the number of 0 and 1 bits\nas 7 bit counters, plus the last 2 bits. When the counters reach the\nmaximum value of 127, they are both halved and incremented. v1.12a is\na speed improvement over v1.12 (released the day before) using prefetch\ninstructions.\n\n[smac v1.13](http://encode.su/attachment.php?attachmentid=2238&d=1363926894), Mar. 22, 2013, mixes the order 6, 4, and 3 indirect context\nmodels in the logistic domain, log(p(1)/p(0)). Each prediction has a fixed\nweight of 1/3.\n\n[smac v1.14](http://encode.su/attachment.php?attachmentid=2283&d=1366465608), Apr. 20, 2013, uses adaptive mixer weight update with\na learning rate of 0.002.\n\n[smac v1.15](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=33183#post33183), May 19, 2013, uses an order 6-4-3-2-1 context mixing\nalgorithm.\n\n[smac v1.16](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=34049#post34049), July 30, 2013, has improvements to the context bit history\nmodel and match model.\n\n[smac 1.17](http://encode.su/attachment.php?attachmentid=2539&d=1383358731)\n[(discussion)](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=35051&viewfull=1#post35051),\nNov. 1, 2013, has some speed optimizations and small changes in the bit history counter\nrounding and use of floating point lookup tables.\n\n[smac 1.17a](http://encode.su/attachment.php?attachmentid=2554&d=1384743548)\n[(discussion)](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=35271#post35271), Nov. 17, 2013,\nhas some speed improvements with no change in compression.\n\n[smac 1.18](http://encode.su/attachment.php?attachmentid=2619&d=1386558145)\n[(discussion)](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=35674#post35674), Dec. 8, 2013,\nuses a polynomial function to compute squash() to improve speed.\n\n[smac 1.19](http://encode.su/attachment.php?attachmentid=2649&d=1387326373)\n[(discussion)](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=35845#post35845), Dec. 17, 2013, has a speed optimization of the squash function.\n\n[smac 1.20](http://encode.su/threads/1633-SMAC-SMall-Arithmetic-Coding?p=36196&viewfull=1#post36196), Jan. 16, 2014, improves modeling of 0 frequency counts\nusing a Laplace estimator, p=(n0+1)/(n0+n1+2).\n\n```\nCompression         Compressed size      Decompresser  Total size   Time (ns/byte)\n  Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem  Alg Note\n  -------        ----------  -----------  -----------  -----------  ----- -----  ---  --- ----\nsmac 1.8         29,143,755  265,303,304     2,713 x   265,306,017   1917  1935 1691  o4  26\nsmac 1.9         26,888,498  242,014,586     2,832 x   242,017,418   3168  3266 1690  CM  26\nsmac 1.10        26,398,662  230,781,496     2,791 x   230,784,287   2917  3085 1649  CM  26\nsmac 1.11        25,633,348  223,294,431     2,831 x   223,297,262   3930  4331 1616  CM  26\nsmac 1.12a       24,948,001  216,016,106     2,833 x   216,018,939   4463  4568 1565  CM  26\nsmac 1.13        23,322,767  202,011,435     2,818 x   202,014,253   6801  6502 1613  CM  26\nsmac 1.14        22,675,896  193,797,222     2,965 x   193,800,187   5943  6148 1577  CM  26\nsmac 1.15        22,303,381  191,064,676     3,074 x   191,067,750   6518  7313 1658  CM  26\nsmac 1.16        21,831,822  183,551,384     3,465 x   183,554,849   6949  7285 1542  CM  26\nsmac 1.17        21,816,272  183,459,153     3,429 x   183,462,582   5672  5867 1542  CM  26\nsmac 1.17a       21,816,272  183,459,153     3,429 x   183,462,582   5335  5613 1542  CM  26\nsmac 1.18        21,816,285  183,459,860     4,522 x   183,464,382   4901  5137 1544  CM  26\nsmac 1.19        21,816,323  183,459,942     4,361 x   183,464,303   4211  4257 1542  CM  26\nsmac 1.20        21,781,544  183,190,888     4,356 x   183,195,244   4249  4399 1542  CM  26\n```\n\n[TC](http://www.encode.su/forums/index.php) 5.2 dev 2\nis an experimental command line file compressor, currently under development\nby Ilia Muraviev.  It takes no options.  \n\n```\n                                   Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                         ----------  -----------  -----------  -----------  ----- -----  --- --- ----\ntc 5.0 dev 1  (May  26 2006)    33,774,535  295,836,604     23,681 x  295,860,285    236   204      LZP   3\ntc 5.0 dev 2  (June 10 2006)    32,417,139  283,039,249     22,659 x  283,061,908    270   244      LZP   3\ntc 5.0 dev 4  (June 21 2006)    32,417,139  283,039,249     22,496 x  283,061,745    224   206      LZP   3\ntc 5.0 dev 6  (July  6 2006)    29,544,971  257,416,397     28,528 x  257,444,925    279   279      PPM   3\ntc 5.0 dev 7  (July  9 2006)    28,111,955  250,077,573     30,058 x  250,107,631    285   325   20 PPM   3\ntc 5.0 dev 9  (July 18 2006)    27,801,253  246,923,158     30,106 x  246,953,264    363   385   24 PPM   3\ntc 5.0 dev 11 (July 24 2006)    27,293,396  242,199,762     31,074 x  242,230,836    446   393   56 PPM   3\ntc 5.1 dev 1  (Oct.  1 2006)    31,708,176  280,007,538     26,578 x  280,034,116    289   154   25 LZ\ntc 5.1 dev 2  (Oct.  2 2006)    31,155,963  274,831,393     24,620 x  274,856,013    344   147   25 LZ\ntc 5.1 dev 5  (Oct. 13 2006)    28,567,681  247,853,181     26,659 x  247,879,840    951   439  148 CM\ntc 5.1 dev 7  (Dec. 18 2006)    27,934,960  241,898,216     40,104 x  241,938,320   1864   639  148 CM\ntc 5.1 dev 7x (Jan. 13 2007)    27,888,899  241,088,655     41,265 x  241,129,920   1974   638  609 CM\ntc 5.2 dev 2  (Feb.  7 2007)    21,481,399  184,939,711     41,112 x  184,980,823   3637  3655  230 CM\n```\n\n5.0 Dev 1 uses LZP. Dev 4 includes an improved hash table to conserve memory and a faster range coder compared to dev. 2, but compression is the same. Starting with 5.0 dev 6, LZP literals and match lengths are encoded using PPMC (PPM with fixed escape probabilities to lower orders). Dev 7 and 9 use order 3-1-0 PPMC.\n\n[tc 5.0 dev 11](http://www.encode.su/downloads/tc-5.0dev11.zip)\n(July 24, 2006) is the last of this series.\n\n[tc 5.1 dev 1](http://www.encode.su/downloads/tc-5.1dev1.zip)\nuses ROLZ (reduced offset LZ) with PPM order 1-0 for literals,\noffset set reduced with order 2 context, and a 16 MB dictionary.\n\n[tc 5.1 dev 2](http://www.encode.su/downloads/tc-5.1dev2.zip) has improved\nparsing and is archive compatible with dev 1.\n\n[tc 5.1 dev 5](http://www.encode.su/downloads/tc-5.1dev5.zip) uses\nROLZ plus context mixing (instead of PPM) for order 2 literals.\n\n[tc 5.1 dev 7](http://www.encode.su/downloads/tc-5.1dev7.zip)\nuses improved parsing (flexible parsing) and adds SSE.\n\n[tc 5.1 dev 7x](http://www.encode.su/downloads/tc-5.1dev7x.zip)\nuses a larger dictionary.\n\nfbc v1.1, Mar. 2, 2012,\nfixes a memory allocation bug that caused decompression to fail\nfor a block size of 333 MB. It automatically selects between 32 and 64 bit\nversions of divsufsort. Results are shown for the 64 bit version.\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nfbc v1.0        250000000       22,554,133  188,976,445     21,244 x  188,997,689    541   480 1225 BWT  26\nfbc v1.1        333333334       22,554,133  185,975,548     23,576 x  185,999,124    451   415 1647 BWT  55\n```\n\n The command line options are the same as in PPMd: -o8 selects order 8, -m256 selects\n256 MB memory, -r1 partially rebuilds the model when memory is exhausted.  I tuned\nthe compressor to -o8 on enwik8.  There are additional options related to VC\ncompression (which must be specified during decompression), but I used the\ndefaults since there is no guidance on how to set them in the program documentation.\nThe paper suggests that the best values (and defaults) are to encode matches\nof context length order+1 with a minimum match length of 2*order, searching the\nlast 8 to 16 contexts for the longest match.\nThe effect is usually greatest for low order PPM.\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- ---\nchile 0.3d-1      -b40000       23,408,335  203,451,387     11,298 s  203,462,685   4957   435  785 BWT\nchile 0.4         -b=244141     22,218,917  186,979,614     11,530 s  186,991,144   2513   512 1426 BWT\n```\n\n The program is supplied as source code only. It was compiled with g++ 4.6.3 using\nthe supplied Makefile in Ubuntu on a Core i7 M620, 4 GB.\nThere are two programs, the compressor \"bwte\" and decompresser \"unbwti\".\nThe compressor computes a low memory BWT using at most the memory specified\nby the -m option (in MB). The -b option specifies how the BWT transformed input\nis to be compressed. -b 1 specifies zlib, -b 4 specifies lzma, and -b 2 specifies\nrun length coding and range coding. There is no block size parameter. The input\nis compressed in a single block. Decompression requires 4 times the file size\nin memory, which used all of the test machine for enwik9 so was tested for enwik8 only.\nCompression of enwik9 with -b 4 failed (cannot create pipe).\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nbwtdisk 0.9.0   -b 1 -m 3500    27,173,252                                           245   234  500 BWT  48\n                -b 1 -m 3500    27,173,252  214,137,751    169,579 s  214,342,831   1124       3500 BWT  48\n                -b 2 -m 3500    24,725,277                                           186   255  500 BWT  48\n                -b 2 -m 3500    24,725,277  190,004,306    169,579 s  190,173,885   1124       3500 BWT  48\n                -b 4 -m 3500    26,975,980                                           270   247  500 BWT  48\nBlock size    enwik8    Comp  Decomp (ns/byte approx)\n----------  ----------  ----  ------\n8 MB        23,461,984  3860   1840\n32 MB       21,948,192  4800   2100\n```\n\n Stuffit\n12.0.0.17 (compression technology version 12.0.0.21) was released Jan. 31, 2008.\nIt includes lossless compression of JPEG and MP3 files and lossy recompression\nof zip archives, GIF, TIFF, PNG, and PDF files.  It supports a native SITX format\nas well as zip, gzip, rar, bzip2, compress, tar, cab, and some more obscure\nformats.  It is multithreaded for multicore support, although I tested it on\na single core processor.  I only tested\nthe native general-purpose formats.  For these tests, I used the command\nline programs console_stuff.exe and console_unstuff.exe to reduce the executable size\nand measure run time more accurately.  The options are\n-m=1 (LZ77-Huffman), -m=2 (LZ77-arithmetic), -m=4 (PPM), -m=8 (BWT), -l (level 2-16,\nhigher is slower but better), -x (memory extents, max 30, higher uses more memory).\nThe best compression for text is -m=4 (PPM) with maximum\nmemory -x=30.  (In the GUI but not the command line, above 29 causes an out of memory\nerror with 2 GB RAM).  The -l option apparently has no effect on PPM.\nThe decompresser size is based on console_unstuff.exe and the minumum set of\n5 .dll files needed to run it (4 common plus Plugins/sitx.dll).\nThe full GUI installer (without Office plugins)\nzips to 17,051,856 bytes.  The tested version was a complimentary copy provided\nby the company.\n Stuffit 2009 13.0.0.19 (compression technology 13.0.0.24) was released Dec. 19, 2008.\nI tested as with Stuffit 12, however the technique of finding the minimal set\nof .dll files that I used in Stuffit 12 did not work (internal error)\nso I had to include the zipped distribution\nsize (StuffIt2009.exe), which includes many other compression formats and a GUI.\nThe tested version was a complimentary copy provided by the company.\n \n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Notes\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----  --- ---  -----\nStuffit 9.0.0.21  Method 4 (best text)        24,310,583  210,801,103  1,015,808 x  211,816,911    542   503   36      12\n                  Method 6 (auto-pick best)   24,419,299  212,392,465  1,015,808 x  213,408,273   2149         68      12\nStuffit 12.0.0.17 -m=1 -l=16 -x=30            25,926,107                                          2540   420  298 LZ77\n                  -m=2 -l=16 -x=27            24,874,987                                          3080    90  881 LZ77\n                  -m=8 -l=16 -x=30            25,574,676                                           560   230  229 BWT\n                  -m=4 -l=16 -x=28            23,482,855                                           730   694  274 PPM\n                  -m=4 -l=16 -x=29            22,744,155                                           770   720  537 PPM\n                  -m=4 -l=16 -x=30            22,105,654  190,372,707  2,658,122 xd 193,030,829    628   658 1062 PPM\nStuffit 13.0.0.19 -m=4 -l=16 -x=30            22,105,658  190,372,711 21,611,401 x  211,984,112    567   604 1060 PPM  26\nCompressed size      Decompresser  Total size   Time (ns/byte)\nProgram       Options                                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  CMem Dmem Alg  Notes\n-------   ---------------                                 ----------  -----------  -----------  -----------  ----- -----   --- ---- ---  -----\nplzma_v3b c2  1000000000 999999999 273 8 0 0 6000 1 1 1 7 24,206,571  193,240,160    101,221 x  193,341,381   8889    55 10110  975 LZMA 58\n          c2                                              24,778,033                                          2110   167   394   54 LZMA 26\nplzma_v3c e                                               25,182,314                                          2050    39   394   54 LZMA 26\n          c                                               24,866,192                                          2060   164   394   54 LZMA 26\n          c2                                              24,778,037  213,154,428     55,974 x  213,210,402   2086   149   394   54 LZMA 26\n```\n\n Bits are modeled MSB first.\nContexts are stored in a binary tree where the two child nodes\nrepresent the current context extended by one bit on the right. Each node also has a\npointer to a suffix node, representing the current context shortened by\none byte on the left. Contexts always begin on byte boundaries.\nEach context maps to a 22 bit prediction for the next bit\n(initialized to 0.5) and a count. When a bit is coded, the\ncurrent node and all of its suffix nodes are updated by adjusting the\nprediction to reduce the error by 1/count and the count is incremented\nby 1 up to a limit of 32. The initial tree is bytewise order 0 (255 contexts)\nwith initial counts of 12. Subsequent nodes are added with a count of 1.5\nand a prediction inherited from its suffix node\nwhenever there is no node to represent the 1 bit extension, and the new node becomes\nthe current context.\n The option -m1600 limits memory usage to 1600 MiB. When memory is exhausted,\nno new nodes are added to the tree, but predictions and counts of existing\nnodes continue to be updated. The current context then becomes the suffix\nnode if needed. The option -O8 limits the tree depth to bytewise\norder 8 (found to be optimal for both enwik8 and enwik9). When the current\nnode reaches this depth, no child nodes are added, but existing nodes and\ntheir suffixes continue to be updated, just as if the memory limit were reached.\nIncreasing the model order improves\ncompression but also causes the tree to grow faster, which sometimes makes compression\nworse if the memory limit is reached sooner. The defaults are -m128 -O4.\n Compression and decompression require the same time and memory. Also,\nthe same compression options must be given again during decompression.\n(I added 10 bytes to the decompresser size to account for this). The compressed file\nis arithmetic coded with the original file size saved in the first 4 bytes.\nFile sizes are limited to less than 2 GiB. The program is distributed as\nsource code only. To test, I compiled with g++ 4.6.1 in 32 bit Windows\nusing the options recommended in the source comments.\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Note\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- ----\ncrook v0.1       -m1600 -O4     25,693,515  229,770,948                              379   393  781  26\n                 -m1600 -O5     23,664,987  207,093,726                              423   442 1641  26\n                 -m1600 -O6     22,793,009  197,202,156                              446   475 1641  26\n                 -m1600 -O7     22,505,951  193,896,089                              462   496 1641  26\n                 -m1600 -O8     22,503,627  193,333,159      8,539 s  193,341,698    483   513 1641  26\n                 -m1600 -O9     22,620,471  193,912,162                              479   519 1641  26\n                 -m1600 -O10    22,752,285  194,794,021                              488   511 1641  26\n                 -m1600 -O12    22,957,581  196,397,188                              492   505 1641  26\n                 -m1600 -O16    23,105,056  197,631,364                              477   503 1641  26\n```\n\n It uses LZ77 with arithmetic coding.  The option -49 selects method 4 (1, 2, 4)\nand level 9 (1..9) for best compression.  Other combinations were not tested.\nThere is also a Linux version which was not tested.\nMemory usage fluxuates but peaks at 654 MB for compression and 90 MB for decompression.\nThe Windows version produces read-only output files that must be set with\n\"attrib -r\" before they can be modified or deleted.\n lzturbo 0.1 (Oct. 5, 2007)\nis threaded for parallel execution on multicore machines.  The maximum\ncomprssion level is -59 where it uses 248 MB for compression and a peak\nof 72 MB for decompression.  Other modes compress much faster.  The read-only\nbug was fixed.\n \n\n```\nProg           Opt              enwik8      enwik9         prog       Total       Comp  Deco  Mem Alg  Note\n------------   ---            ----------  -----------     ------    -----------   ----  ----  --- ---- ----\nlzturbo 0.01   -49            26,678,709  233,322,999     68,561 x  233,391,560   1412    50  654 LZ77\nlzturbo 0.1    -59            26,616,816  232,708,136    129,344 x  232,837,480   1385    49  248 LZ77\nlzturbo 0.9    -59            26,616,278  232,701,587    116,508 x  232,818,095   1420    52  248 LZ77\nlzturbo 0.94   -59 -b100 -p0  24,763,542  217,342,694    152,254 x  217,494,948   5196    20 1450 LZ77 26\n               -10            51,426,368                                            10     8   78 LZ77 26\n               -14            38,325,178                                            74    10  171 LZ77 26\n               -39 -b50       26,123,933                                          1290    16 1450 LZ77 26\n               -41            36,615,397  325,577,604    152,254 x  325,729,858     29    23  203 LZ77 26\n```\n\n Option -b1000 selects a block size of 1000 MB. The default is -b24.\nSeparate blocks can be compressed and decompressed in parallel. The\ntest machine automatically selects 4 threads. Larger blocks improve\ncompression but use more memory and allow fewer threads to be allocated.\n-b1000 causes it to use 1 thread since there is a single block.\nAt level 9 (-19, -29, -39, -49), it is not possible to compress enwik9\nwith -b1000 on the 4 GB test machine because it will use over 6 GB memory\nand start disk thrashing. -p1 selects 1 thread. -p0 disables multi-threading.\n \n\n```\n             Compression       Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram        Options        enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Note\n-------        -------      ----------  -----------  -----------  -----------  ----- ----- ---- ----\nlzturbo 1.1    -10 -b24     53,199,932                                             3     2       48\n               -10 -b1000   53,194,540                                             6     2       48\n               -11 -b24     47,619,485                                             6     2       48\n               -11 -b1000   47,611,974                                            12     2       48\n               -12 -b24     44,421,925                                            18     2       48\n               -12 -b1000   44,413,087                                            36     2       48\n               -19 -b24     41,929,879                                           493     2       48\n               -19 -b1000   41,920,122                                          1610     2       48\n\n               -20 -b24     49,736,192                                             3     2       48\n               -20 -b1000   49,725,239                                             6     3       48\n               -21 -b24     42,628,330                                             6     2       48\n               -21 -b1000   42,538,087                                            12     3       48\n               -22 -b24     39,541,490                                            18     2       48\n               -22 -b1000   39,210,560                                            35     3       48\n               -29 -b24     32,919,788                                           543     2       48\n               -29 -b1000   31,370,930                                          1760     4       48\n\n               -30 -b24     39,036,288                                             5     3       48\n               -30 -b1000   39,023,229                                            10     6       48\n               -31 -b24     35,632,652                                             7     3       48\n               -31 -b1000   35,572,973                                            13     6       48\n               -32 -b24     31,266,016                                            18     3       48\n               -32 -b1000   30,753,365                                            38     6       48\n               -39 -b24     26,892,107                                           573     3       48\n               -39 -b1000   25,298,784                                          1838     6       48\n\n               -49 -b24     25,870,196  225,397,956     110,565 x  225,508,521   792    13 1702  48\n           -p1 -49 -b200    24,416,777  207,335,845     110,565 x  207,446,410  2566    17 3200  48\n               -49 -b1000   24,416,777                                          2110    20       48\n           -p0 -49 -b1000   24,416,777  194,681,713     110,670 x  194,792,383  1920     9 14700 59\n\nlzturbo 1.2    -10 -b24     52,703,759                                             3.2   1.6     48\n               -10 -b1000   52,698,226                                             7.0   2.2     48\n               -11 -b24     47,619,370                                             6.0   1.5     48\n               -11 -b1000   47,611,859                                            11     1.9     48\n               -12 -b24     44,421,812                                            17     1.4     48\n               -12 -b1000   44,412,974                                            31     1.8     48\n               -19 -b24     41,933,864                                           515     1.4     48\n               -19 -b1000   41,924,186                                          1577     1.9     48\n\n               -20 -b24     48,387,089                                             3.2   1.8     48\n               -20 -b1000   48,374,729                                             6.8   5.8     48\n               -21 -b24     42,628,216                                             5.9   1.7     48\n               -21 -b1000   42,537,971                                            11     2.9     48\n               -22 -b24     39,394,820                                            18     2.0     48\n               -22 -b1000   39,022,094                                            30     7.1     48\n               -29 -b24     32,922,201                                           545     2.4     48\n               -29 -b1000   31,372,980                                          1755     4.7     48\n\n               -30 -b24     39,147,401                                             5.3   2.5     48\n               -30 -b1000   39,138,118                                            11     5.1     48\n               -31 -b24     35,618,016                                             7.3   2.3     48\n               -31 -b1000   35,563,249                                            17     4.2     48\n               -32 -b24     30,979,376                                            19     2.7     48\n               -32 -b1000   30,258,461                                            41     5.3     48\n               -39 -b24     26,915,461                                           582     2.8     48\n               -39 -b1000   25,330,833                                          1873     5.1     48\n\n               -49 -b24     25,812,200                                           656     8       48\n           -p1 -49 -b200    24,416,777  206,359,193     125,174 x  206,484,367  2319    14 3200  48\n```\n\n Only source code was provided. It was compiled for 32 bit Windows Vista\nusing MinGW 4.6.1 using \"gcc -O3 *.c\".\n \n\n```\nCompressor         Opt            enwik8      enwik9         Prog      Total        Comp Decomp  Mem Alg  Note\n---------          ---          ---------   -----------     -------  -----------    ----  ----   --- ---- ----\ncomprolz 0.1.0     -b256        24,835,082  215,770,703     41,170 s 215,811,873     595   262   602 ROLZ 26\ncomprolz 0.2.0     -b250 -f     24,280,609  210,255,761     43,899 s 210,299,660    1415   319   666 ROLZ 26\ncomprolz 0.10.0    -b250 -f     23,050,103  198,635,448     82,824 x 198,718,272    1086   333   595 ROLZ 26\ncomprolz 0.11.0    -b250 -f     23,687,477  213,585,466     29,509 x 213,614,975    1608   324   866 ROLZ 26\ncomprolz 0.11.0b1  -b250 -f     22,813,215  196,651,379     29,453 x 196,680,832     984   308   688 ROLZ 26\nCompression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  \n-------         -------         ----------  -----------  -----------  -----------  ----- -----  \nsbc 0.970r2     -ad -m3 -b63    22,470,539  197,066,203     99,094 xd 197,165,297   1733   313\nsbc 0.970r2     -ad -m1 -b31    23,288,217                  99,094 xd                620   230\nsbc 0.970r2     -ad -m1 -b1     27,087,118                  99,094 xd                300   180\n```\n\n Program size is based on xz.exe. There is a separate decompressor (xzdec.exe)\nwhich is smaller and decompresses to standard output, but the Windows version\ndoes not work because it outputs in text mode. Additional results are shown below\nfor enwik8 for compression and decompression time (ns/byte) and compression\nand decompression memory (in MB).\n \n\n```\nVersion     Options                                  enwik8      enwik9     size (zip) enwik9+prog Ctime Dtime Cmem Dmem Note\n--------    -------                                ----------  -----------  ---------  -----------  ----- ---- ---- ---- ----\nxz 5.0.1    -9 -e                                  24,831,648                                       2310   40   690   66   26\n            -9                                     24,865,244                                       2600   40   690   66   26\n                                                   26,375,764                                       2020   45    95    8   26\nxz 5.2.1    --lzma2=preset=9e,dict=1GiB,lc=4,pb=0  24,703,772  197,331,816  36,752 xd  197,368,568  5876   20  6000 1025   73\n```\n\n The model order was tuned on enwik8.  Additional results are shown\nfor order 10,\nfor -m5 (maximum compression), and for normal compression as a .exe and\n.rar file.  The decompresser in the last case is zipped unrar.exe.\n WinRAR 4.20 was released June 9, 2012. It costs $29 with a 40 day free trial\nas of Feb. 1, 2013. Options are the same. -m1 through -m5\nselect compression level. The default is -m3. The algorithm is LZ77 with a 4 MB\nwindow. -mc7:128t+ selects PPM, order 7,\nwith maximum 128 MB memory. Time and memory to decompress with PPM is about the\nsame as compression.\n WinRAR 5.00b2 was released Apr. 29, 2013. It includes a larger dictionary,\nup to 1 GB for the 64 bit version and 256 MB for the 32 bit version.\nOption -ma5 selects the new archive format, which is not compatible\nwith v4.20 or earlier. The default is the older format. In the newer\nformat, option -mc is silently ignored. Option -m3 is the default\ncompression level.\n \n\n```\n                      Compression              Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                 Options               enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------         --------------------------  ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\nWinRAR 3.60b3   -mc7:128t+  -sfxWinCon.sfx  22,713,569  198,454,545          0 xd 198,454,545    506   415\n                -mc10:128t+ -sfxWinCon.sfx  23,233,523                       0 xd                770\n                -m5         -sfxWinCon.sfx  24,832,649                       0 xd                680   520\n                            -sfxWinCon.sfx  29,828,890                       0 xd                780    40\n                                            29,749,530                  98,888 xd                780    40\nWinRAR 4.20     -m1                         40,234,511                                            36    32   99 LZ77  26\n                -m2                         30,564,700                                           180    29   99 LZ77  26\n                -m3 (default)               29,671,175                                           325    30   99 LZ77  26\n                -m4                         29,329,237                                           484    30   99 LZ77  26\n                -m5                         29,225,016                                           590    30   99 LZ77  26\n                -mc5:128t+                  23,440,773                                           358        229 PPM   26\n                -mc6:128t+                  22,701,033                                           418        229 PPM   26\n                -mc7:128t+                  22,635,718  198,372,701    141,019 xd 198,513,720    440   373  229 PPM   26\n                -mc8:128t+                  22,769,557                                           518   456  229 PPM   26\n                -mc10:128t+                 23,153,065                                           582        229 PPM   26\n                -mc12:128t+                 23,401,290                                           609        229 PPM   26\nWinRAR 5.00b2   -mc7:128t+                  22,635,718  198,372,701    153,763 x  198,526,464    433   368  226 PPM   26\n                -ma5 -m1                    40,565,268                                            54    31  406 LZ77  26\n                -ma5 -m2                    29,758,785                                           228    30  435 LZ77  26\n                -ma5 -m3                    28,662,794                                           439    32  435 LZ77  26\n                -ma5 -m4                    28,072,832                                           751    31  435 LZ77  26\n                -ma5 -m5                    27,835,431                                          1004    31  435 LZ77  26\n```\n\n lzip and plzip are written for Linux. A\n lzip 1.14-rc3 was released Jan. 15, 2013.\n \n\n```\nCompressor     Opt            enwik8      enwik9         Prog       Total       Comp Decomp  Mem  Alg  Note\n---------      ---          ---------   -----------     -------   -----------   ----  ----   ---  ---- ----\nplzip          -9           25,578,352  221,845,216     56,614 x  221,901,830   1308    37   1028 LZ77  26\nlzip 1.14-rc3  -9 -s512MiB  24,756,063  199,410,543     21,682 s  199,432,225   2409    21   5632 LZ77  57\nplzip 1.5      -9           25,518,871  221,179,984    336,294 x  221,213,608    425    13   2048 LZ77  48\n```\n\n Version 20110928 was released Sept. 28, 2011. Compression runs in 2 threads.\nBoth the Windows and Linux versions were tested (on different computers).\n Version 20110929 was released Sept. 29, 2011. Decompression also runs in\n2 threads. Compression is slightly improved.\n comprox version 0.1.1, Oct. 10, 2011, replaces comprox_sa. It is a rewrite\nusing LZ77 (instead of LZSS) and arithmetic coding. It takes a compression\nlevel 0 (fastest) to 9 (best) with a default of 5. All levels use the same\nmemory, 218 MB for compression and 44 MB for decompression. The Linux\nversion reports the same resident memory as Windows but higher virtual memory:\n236 MB to compress and 284 MB to decompress. Both compression and decompression\nrun in 2 threads. Reported times are real times.\n comprox 0.8.0-bugfix1, Sept. 27, 2012, fixed a bug that caused compression\nto crash on some input files. It was compiled with MinGW 4.6.1 with\n\"gcc -O3 -msse2 -s -Wl,--stack,8000000 *.c -lpthread\".\n \n\n```\nCompression                                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram    Version          Opt                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  CMem Dmem Alg Note\n-------    --------         ---               ----------  -----------  -----------  -----------  ----- -----  ---- ---- --- ----\ncomprox_sa 20110927 (Win32)                   32,654,393  287,588,097      3,791 s  287,591,888    398   101    60    6 LZSS 26\ncomprox_sa 20110928 (Win32)                   32,654,718  287,590,343      3,790 s  287,594,133    205   101   122   10 LZSS 26\ncomprox_sa 20110928 (Linux)                   32,654,718  287,590,343      3,790 s  287,594,133    126    59   141   10 LZSS 48\ncomprox_sa 20110929 (Win32)                   32,652,597  287,575,768      3,774 s  287,579,542    209    71   122   12 LZSS 48\ncomprox_sa 20110929 (Linux)                   32,652,597  287,575,768      3,774 s  287,579,542    116    37   145   36 LZSS 48\ncomprox 0.1.1       (Win32)  0                29,463,135                                           146    65   219   44 LZ77 26\n                    (Win32)  5                28,836,139                                           290    65   218   43 LZ77 26\n                    (Win32)  9                28,586,545  250,565,797      5,430 s  250,571,227    768    57   218   43 LZ77 26\n                    (Linux)  9                28,586,545  250,565,797      5,430 s  250,571,227    496    29   218   44 LZ77 48\ncomprox 0.6.0       (Win32) e200              25,504,328  221,405,873     23,367 s  221,429,240    484    92  1567  590 LZ77 26\n                            e16               26,816,904                                           395   132   169   68 LZ77 26\ncomprox 0.7.0       (Linux) e200              25,068,368  217,403,007     36,702 s  217,439,709    225    52  1000  410 LZ77 48\n                    (Win32) e200              25,068,368  217,403,007     36,702 s  217,439,709    390   126  1107  472 LZ77 26\n                    (Linux) e500              25,068,368  212,824,614     36,702 s  212,861,316    260    57  2500 1100 LZ77 48\n                    (Linux) e700              25,068,368  212,348,904     36,702 s  212,385,606    309    49  3400 1500 LZ77 48\ncomprox 0.8.0       (Win32) e200              24,537,383  212,651,678     42,764 s  212,694,442    460   128  1143  279 LZ77 26\n                    (Linux) e500              24,537,383  208,328,173     42,764 s  208,370,937    296    49  2500  558 LZ77 48\ncomprox 0.8.0-bugfix1 (Win) e200              24,537,453  212,652,159     42,804 s  212,694,963    480   145  1108  281 LZ77 26\ncomprox 0.9.0       (Win32) -b250 -f -m100    24,243,078  208,369,181     46,387 s  208,415,568   1657   130  1405  326 LZ77 26\n                            -b250 -f          24,281,529                                           748   161   733  164 LZ77 26\n                            -b250             24,486,987                                           398   160   733  164 LZ77 26\n                                              25,494,243                                           317   167   151   86 LZ77 26\ncomprox 0.10.0      (Win32) -b250 -f -m100    23,332,113  201,288,183     86,687 x  201,374,870   1209   151  1271      LZ77 26\ncomprox 0.11.0      (Win32) -b200 -f -m100    23,990,134  217,340,709     34,176 x  217,374,885   2115   144  1211      LZ77 26\n                    (Win32)                   25,003,709  234,265,741     34,176 x  234,299,917    436   145   269      LZ77 26\ncomprox 0.11.0-bugfix1(Win) -b250 -f -m100    23,064,386  199,515,912     34,176 x  199,550,088    917   153   688      LZ77 26\n                                              23,861,257  209,481,309     34,176 x  209,515,485    307   162   196      LZ77 26\n```\n\n For this test, lzhamtest_x86 was used. There is a _x64 version for 64 bit machines\nwhich is faster. The library supports different speeds and dictionary sizes, but\nthe test program does not have any options to select them, so none were used.\nDecompression uses 67 MB memory vs. 609 MB for compression.\nCompression uses both cores on the test machine\nbut decompression uses only one.\n Version alpha 3, Aug. 30, 2010, supports all of the options suppored by\nthe library. Option -d26 selects 64M dictionay, the largest supported by\nthe x86 version. (The x64 version supports up to -d29 = 512M). -m4 selects \"uber\"\ncompression mode. There are 5 compression levels from -m0 through -m4.\nThe highest two levels use Huffman codes rather than Polar codes. -t2 says\nto use 2 helper threads (to match the number of cores on the test machine). The\ndefault is to use 1 less than the number of cores, up to 16 threads.\nDecompression is not multi-threaded.\n The x64 version was  \n\n```\n                       Compression               Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                  Options                enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                  -------             ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nlzham alpha 2 x86                             25,907,665  224,554,163     95,922 x  224,650,085   2485    21  609 LZ77 26\nlzham alpha 3 x86     -m4 -d26 -t2            24,991,681  213,868,601    139,694 x  214,008,295   2970    22  611 LZ77 26\nlzham alpha 3 x64     -m4 -d29                24,954,329  206,393,809    155,282 x  206,549,091    595     9 4800 LZ77 45 \nlzhamtest v1.0                                25,064,179  207,094,787    191,600 s  207,286,387    553     9 2392 LZ77 48\n                      -d26                    25,091,033                                           279   7.3      LZ77 70\n                      -d26 -x                 24,990,739                                           722   7.2      LZ77 70\n                      -d29                                204,325,043    191,600 s  204,516,643    339   6.6      LZ77 70\n                      -d29 -x                             202,237,199    191,600 s  202,428,799   1096   6.6      LZ77 70\n                      -d29 -x                 25,002,070                                          1761   9.5  911 LZ77 48\n```\n\n flashzip 0.2 was released Jan. 11, 2008.  It is compatible with version 0.1 but faster.\nNote: in both versions, CPU utilization during compression is about 28% to 35%.  Times\nshown are process times.\n flashzip 0.3 was released Feb. 4, 2008.  It uses ROLZ plus arithmetic coding.  It\ntakes an option x for better compression (slower) and 1 through 5, \nwhere 5 is the slowest (best compression).\n flashzip 0.9 was released June 28, 2008. Option -m2 selects method 2 (default\nis -m1). -b1 through -b5 select buffer size, which affects memory usage.\nDefault is -b3. -s1 through -s7 selects match length and speed. Default is\n-s1 (fastest, worst compression).\n flashzip 0.91 was released Aug. 17, 2008. Options are like version 0.9.\nMemory usage was increased\nto 198 MB for compression and 138 MB for decompression using settings for\nbest compression.  Minimum requirement is 10 MB and 6 MB.\n flashzip 0.94 was released Mar. 25, 2009.\n flashzip 0.99 was released July 23, 2009.\n flashzip 0.99b4 (Aug. 25, 2009) is an archiver rather than a compressor. The -s\noption was renamed to -c and the -b option was increased to -b8 to allow more\nmemory usage. For enwik8, memory usage for both -m1 and -m2 is 182 MB for\ncompression and 162 MB for decompression.\nFor enwik9, memory usage for -m2 is 609 MB for compression and 592 MB for decompression.\n flashzip 0.99b8 (Feb. 28, 2010) has 4 compression levels from -m0 (fastest) to\n-m3 (best). The buffer size option was increased to -b9 (1 GB).\nMemory usage depends on the input size.\nFor -m0 -c7 -b7 enwik8, compression takes 214 MB and decompression takes 195 MB.\nFor -m1 through -m3 -c7 -b8, enwik8 compression takes 231 MB and decompression takes 195 MB.\nFor -m3 -c7 -b8, enwik9 compression takes 658 MB and decompression takes 625 MB.\nChanging -b8 to -b9 has no effect on size, speed, or memory usage for enwik8,\nbut for enwik9 it improves compression and increases memory usage to 1111 MB for\ncompression and 1078 MB for decompression. The -s1 option enables the -b9 option.\nOtherwise -b9 will cause a \"no memory\" error.\n flashzip 0.99c1 (June 1, 2011) improves compression and speed. The option ranges\nare -m0...-m3, -c1...-c7 and -b1...-b7. Only the maximum compression options were tested.\n flashzip 0.99c3 (Oct. 10, 2011) is multi-threaded for compression\nin modes -m1, -m2, -m3. Decompression runs in a single thread.\nThe archive is compatible with the previous version.\nIn the tested mode (maximum compression), memory usage depends on the file\nsize and climbs steadily during compression or decompression. It is the\nsame for either, and same as the previous single threaded version.\n flashzip 0.99d1 was released Oct. 31, 2011. It has only two\noptions, -m0...-m9 (default -m4) for compression method (fastest...best)\nand -b1...-b7 (default -b1) for buffer size. Memory usage ranges from\n30 MB at -b1 to 1100 MB at -b7.\n flashzip 1.0.0 was released Oct. 3, 2012. Options -m1 to -m7 select compression\n-mx7 compresses best. Higher levels compress\nslower and use more memory but have little effect on decompression\nspeed, which is generally faster. Decompression uses the same memory\nas compression, up to 1.1 GB depending on the file size.\nOptions -b1 to -b7 select buffer size. Larger values\nuse more memory but don't affect speed. The default is -b4.\nThe program can use up to 8\nthreads and auto-detects the number of available cores. In the\nhigh compression modes tested, only 1 of 2 available cores was used.\n-e creates a self extracting archive. It extracts to the saved name\nusing both cores.\n \n\n```\n                Compression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------           -------         ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nflashzip 0.1                      34,053,198  299,443,551     25,734 x  299,469,285     67    51   47 LZP\nflashzip 0.2                      34,053,198  299,443,551     25,257 x  299,468,808     62    52   47 LZP\nflashzip 0.3      5               28,541,292  248,094,851     26,738 x  248,121,589    297    73   86 ROLZ\n                  x 5             27,845,033  241,997,412     26,738 x  242,024,150    673    72   86 ROLZ\nflashzip 0.9      (-m1 -s1 -b3)   31,856,012                                           141   124   83 ROLZ\n                  -b1             32,088,940                                           148   125   70 ROLZ\n                  -b5             31,764,213                                           143   119  132 ROLZ\n                  -s4             29,235,064                                           269    99   83 ROLZ\n                  -s7             28,370,670                                           928    87   83 ROLZ\n                  -m2             31,641,305                                           188   121   83 ROLZ\n                  -m2 -s7         27,665,526                                          2081    97   83 ROLZ\n                  -m2 -s7 -b5     26,737,801  230,987,395     30,052 x  231,017,447   2476    75  132 ROLZ\nflashzip 0.91     -m2 -s7 -b5     26,068,507  227,945,252     34,222 x  227,979,474   3560   112  198 ROLZ\n                  -m1 -s7 -b5     26,851,582                                          1305   127  198 ROLZ\nflashzip 0.93a    -m2 -s7 -b5     26,243,745  227,048,196     36,367 x  227,084,563   1458    95  132 ROLZ\n                  -m1 -s7 -b5     27,004,639                                          1030   140  198 ROLZ 26\nflashzip 0.94     -m2 -s7 -b5     26,236,095  226,981,882     35,996 x  227,017,878   2451    87  132 ROLZ 26\n                  -m1 -s7 -b5     26,662,405  230,985,291     35,996 x  231,021,287   1275    84  198 ROLZ 26\nflashzip 0.99     -m2 -s7 -b5     26,027,791  224,648,225     37,361 x  224,685,586   2399   110  198 ROLZ 26\n                  -m1 -s7 -b5     26,305,210                                          1230   160  132 ROLZ 26\nflashzip 0.99b4   -m2 -c7 -b8     25,804,706  218,328,751    141,207 x  218,469,958   3037    86  609 ROLZ 26\n                  -m1 -c7 -b8     26,255,893                                          1580    97  182 ROLZ 26\nflazhzip 0.99b8   -m0 -c7 -b8     29,191,973                                           200   110  214 ROLZ 26\n                  -m1 -c7 -b8     27,752,588                                           510   110  231 ROLZ 26\n                  -m2 -c7 -b8     26,351,718                                          1420   110  231 ROLZ 26\n                  -m3 -c7 -b8     26,008,189  220,193,756    119,185 x  220,312,941   3281    84  658 ROLZ 26\n              -s1 -m3 -c7 -b9     26,008,189  218,405,144    119,185 x  218,524,329   3531    89 1111 ROLZ 26\nflashzip 0.99c1   -m3 -c7 -b7     24,840,311  206,005,639    131,128 x  206,136,767   2139   117 1050 ROLZ 26\nflashzip 0.99c3   -m3 -c7 -b7     24,840,025  205,992,947    246,816 x  206,239,763   1925   112 1050 ROLZ 26\nflashzip 0.99d1                   28,022,537                                           253    92   46 ROLZ 26\n                          -b7     28,088,756                                           542   102  127 ROLZ 26\n                  -m9     -b7     24,363,049  207,354,714    170,353 x  207,525,067   1180    94 1100 ROLZ 26\nflashzip 1.00                     26,788,895                                           168   127   37 ROLZ 26\n                          -b7     26,761,559                                           174   123   91 ROLZ 26\n                  -m7     -b7     26,761,559                                           762   130  136 ROLZ 26\n                  -mx7    -b7     23,869,034  202,363,445    123,053 x  202,486,498   1296   122  802 ROLZ 26\n                  -mx7 -e -b7     23,995,498  202,489,909          0 x  202,489,909   1123   123  840 ROLZ 26\nflashzip 1.12     -mx3 -k7 -b1024 24,726,693  211,104,283    151,961 x  211,256,255    581    94 1152 ROLZ 26\nOptions         enwik8    Comp  Decomp (ns/byte)\n-------       ----------  ----  ------\n-mx -md32768  23,911,123  1830  1510\n-mx           23,952,039  1832  1546\n-m3           27,957,245  1840   110\n-m2           28,459,084  1726   110\n-m1           29,660,279  1242   121\n-mz           30,429,795   191   236\nCompression              Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options               enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------           -------             ----------  -----------  -----------  -----------  ----- -----  --- ---- ----\ncsc2                                  34,119,354  298,385,256      9,092 x  298,394,348    141   201   49 LZP  26\ncsc3 2009.08.12   -m1 -d1             33,920,768                                           150    59   15 LZ77 26\n                  -m1 -d7             33,510,724                                           320    59  511 LZ77 26\n                  -m2 -d4             31,627,835                                           660    56   93 LZ77 26\n                  -m2 -d7             31,460,838                                           730    55  511 LZ77 26\n                  -m3 -d7             30,430,159  263,485,695     14,027 x  263,499,722   1514    43  675 LZ77 26\ncsc31             -m3 -d7             28,984,849  250,172,831     64,214 x  250,237,045   1045    33  791 LZ77 26\ncsc32 a2          -m3 -d9             30,304,020  262,999,383    111,571 x  263,110,954    340    35  528 LZ77 26\ncsc32 final       -m1 -d128           28,973,600                                           178    49  166 LZ77 26\n                  -m2 -d128           28,624,802                                           283    52  166 LZ77 26\n                  -m3 -d4             27,776,206                                           416    52   24 LZ77 26\n                  -m3 -d128           26,842,072  232,326,926     53,665 s  232,380,591    420    46  201 LZ77 26\n                  -m3 -d512           26,842,072  229,929,654     53,665 s  229,983,319    423    47  660 LZ77 26\ncsarc 3.3         -m1 -p4 -t4 -d256m  29,160,344  250,618,458     69,848 s  250,688,306     32    12 1340 LZ77 48\n                  -m3 -p4 -t4 -d256m  27,130,418  232,020,894     69,848 s  232,090,742     95    12 1581 LZ77 48\n                  -m5 -d1024m         24,516,202  203,995,005     69,848 s  204,064,853    621    22 2463 LZ77 48\n```\n\n packet 0.02, May 16, 2008,\nimproves compression for .wav files and supports files over 2 GB.\n packet 0.03b, May 20, 2008,\nuses LZ77, 3 MB for compression, and 1 MB for decompression.  It takes an optional\nargument 'x' meaning better but slower compression, and a level 1 through 6, where\n6 is slowest with best compression.\n packet 0.90b, June 18, 2008,\nhas options -m1 to -m4 (method) and -s0 to -s9 (intensity).  All options use\n10 MB for compression and 2 MB for decompression.\n packet 0.91b, Aug. 6, 2009 has methods -m1 through -m6, where\n-m6 is maximum compression. Decompression requires 1.5 MB.\n \n\n```\n               Compression           Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram          Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------          -------         ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\npacket 0.01                      37,637,275  334,473,465     30,508 x  334,503,973     50    43    4 LZP\npacket 0.02                      37,637,276  334,473,466     27,900 x  334,501,366     58    42    4 LZP\npacket 0.03b     1               35,576,495                                           140    20    3 LZ77\n                 x 1             34,792,199                                           170    20    3 LZ77\n                 6               34,563,297                                           450    20    3 LZ77\n                 x 6             33,752,502  297,266,174     26,435 x  297,292,609    594    18    3 LZ77\npacket 0.90b     -m1 -s0         35,426,140                                           199    28   10 LZ77\n                 -m1 -s9         32,780,039                                          2887    26   10 LZ77\n                 -m2 -s0         34,281,503                                           274    24   10 LZ77\n                 -m2 -s9         31,968,711                                          4527    25   10 LZ77\n                 -m3 -s0         34,966,621                                           236    56   10 LZ77\n                 -m3 -s9         32,199,212                                          2965    51   10 LZ77\n                 -m4 -s0         33,612,046                                           307    61   10 LZ77\n                 -m4 -s3         32,033,412                                           861    57   10 LZ77\n                 -m4 -s6         31,367,386                                          2411    57   10 LZ77\n                 -m4 -s9         31,208,752  273,176,127     32,305 x  273,208,432   3871    48   10 LZ77\npacket 0.91b     -m6 -s9         31,306,703  274,033,491     45,358 x  274,078,849   3669    36   10 LZ77 26\npacket 1.0       -m4             28,349,717                                           487    37  416 LZ77 26\n                 -m4 -t2         28,789,607                                           385    53  500 LZ77 26\n                 -m9             27,439,216                                          4530    37  425 LZ77 26\n                 -mx9            26,895,256  232,428,377    114,566 x  232,542,943  19749    34  429 LZ77 26\npacket 1.1                       26,848,041  233,803,751    265,102 x  234,068,853    295    26  335 LZ77 48\n                 -m9 -b512 -h4   25,624,659  216,849,389    265,102 x  217,114,491    647    26 1500 LZ77 48\n                 -mx -b512 -h4   25,348,872  213,722,850    265,102 x  213,987,952    767    26 1500 LZ77 48\npacketx64 1.2    -mx -b512 -h4   24,664,592  204,646,570    314,885 x  204,961,455    601    21 1619 LZ77 48   \npacket_x64 1.9   -mx -b512 -h8   24,968,492  204,195,438    261,967 x  204,457,405    974    14 2824 LZ77 48\n```\n\n Older versions used order 3 LZP to code the last 16 matches at order 3,\nfollowed by order 2 PPM encoding of literals. \nIt takes no command line options but compression/decompression settings may be specified in\nan initialization file.  For this test, default settings were used and others were not tried.\n The Jul 30 2007 version uses 2 LZP models, one with a 4 byte context and one 8 byte.\nThe program selects the one that gives a higher probability of a match.  There is no\ninitialization file.\n The Aug 8 2007 version uses 341 MB memory for compression and 333 MB for decompression.\n The interim  \n\n```\n                          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram                  enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------                ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nTarsaLZP Jul  4 2006   35,745,297  334,661,013      2,255 sd 334,663,268    149   163   54 LZP\nTarsaLZP Jul 30 2006   34,321,697  320,160,237      1,455 xd 320,161,692    110   117   54 LZP\nTarsaLZP Aug  5 2006   32,270,002  295,312,202      1,579 xd 295,313,781    110   127   70 LZP\nTarsaLZP May  6 2007   32,461,606  297,130,840      1,580 xd 297,132,420     97   121   71 LZP\nTarsaLZP Jun 17 2007   31,233,381  283,895,945      1,604 xd 283,897,549    100   122   71 LZP\nTarsaLZP Jul 18 2007   31,363,533  285,248,058      2,365 xd 285,250,423     88   105   71 LZP\nTarsaLZP Jul 30 2007   26,664,933  233,613,937      2,472 xd 233,616,409    247   255   42 LZP\nTarsaLZP Aug  8 2007   25,134,862  215,301,412      2,843 xd 215,304,255    249   287  341 LZP\nTarsaLZP Aug 10 2007   25,135,357  215,301,079      3,546 xd 215,304,626    269   322  341 LZP\nTarsaLZP Jan 29 2012   24,751,389  208,867,187     13,081 s  208,880,268    203      ~2000 LZP 54\nTarsaLZP Nov 18 2012   24,860,676  211,990,481     20,303 s  212,010,784    244   277  330 LZP 26\nCompression               Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------     -------------------------- ----------  -----------  -----------  -----------  ----- -----  --- ---\n4x4 0.2a    1 (tor:1:4m)               59,711,544                                            17    13   54 LZ77\n            7 (tor:7:64m)              32,433,532                                           197    24  230 LZ77\n            8 (lzma:fast:128m:ht4:mc8) 32,698,603                                           292    43  230 LZ77\n           12 (lzma:128m:ht4:mc128)    27,307,504                                          4354    43  230 LZ77\n           1t (grzip:m4)               26,576,294                                           167   232  128 BWT\n           4t (grzip:m1:h18)           23,833,244  208,787,642     317,097 x 209,104,739    386   240  269 BWT\nCompression            Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options             enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------           -------           ----------  -----------  -----------  -----------  ----- -----  --- ---\nrzm 0.06c                           24,429,597  210,719,085     12,903 x  210,731,988   2216    92   180 ROLZ\nrzm 0.07h                           24,361,070  210,126,103     17,667 x  210,143,770   2336    81   160 ROLZ\n```\n\n pim 2.04 beta was released July 21, 2007.  It has PPMd as its only option.\n pim 2.10 was released July 31, 2007.  Older versions are no longer supported.\n pim 2.50 was released July 22, 2008. It supports 3 compression modes: store,\nnormal, and best. Only best was tested. It compresses in PPMd, bzip2 and DCL\nformats and extracts BALZ, QUAD, ZIP, JAR, PK3, PK4 and QUAKE PAK archives.\n \n\n```\n                Compression            Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options             enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------           -------           ----------  -----------  -----------  -----------  ----- -----  --- ---\npim 2.01   PPMd, no exe, no color   24,303,638  210,124,895    340,951 x  210,465,846   ~600   639   92 PPM\npim 2.04b  PPMd                     24,303,638  210,124,895    335,004 x  210,459,899    900   780   84 PPM\npim 2.10   PPMd                     24,303,638  210,124,895    335,374 x  210,460,269    895  ~900   84 PPM\npim 2.50   best                     24,303,638  210,124,895    330,901 x  210,455,796    764  ~764   88 PPM\n```\n\n The -d6 option selects order 6 (depth of context tree).  -n16M selects the maximum of 16M nodes\nfor the tree (using 128 MB memory).  -f16M selects the maximum 16 MB file buffer \n(for rebuilding pruned contexts).  The default values of all other options were tested on\nenwik6 and found optimal.  For -d, there is a tradeoff between compression and memory usage\nas with PPM compressors.  -d6 was found optimal on both enwik7 and enwik8.\n \n\n```\nOption    enwik7     enwik8      enwik9     Comp (ns/byte)\n------  ---------  ----------  -----------  -----\n-d5     2,490,460  24,174,511               11340\n-d6     2,438,708  23,670,293  211,995,206  19221\n-d7     2,455,765  23,689,423               24680\n-d9     2,494,767\n-d12    2,531,284\n```\n\n yzx 0.04 was released\nMay 27, 2010. Decompression memory remains at 268 MB.\n \n\n```\nCompressor   Opt            enwik8      enwik9         Prog      Total        Comp Decomp  Mem Alg  Note\n---------    ---          ---------   -----------     -------  -----------    ----  ----   --- ---- ----\nyzx 0.01     -b5          28,984,962  249,903,552    116,793 x  250,020,345    395    73   732 LZ   26\nyzx 0.02     -m2 -c8 -b5  27,293,259  229,890,264    116,795 x  230,007,059  10927    67   732 LZ   26\nyzx 0.03     -m2 -c5 -b6  28,132,853  241,790,934    116,141 x  241,907,075    911    71   404 LZ   26\nyzx 0.04     -m2 -c5 -b6  27,670,096  235,198,449    116,507 x  235,314,956    833    69   535 LZ   26\nyzx 0.11                  27,694,742                                           258    85   293 LZ   26\n             -m9 -b8 -h5  25,768,724                                           518    81   636 LZ   26\n             -m9 -b7 -h6  25,754,856  214,317,684    131,062 x  214,448,746    642    77  1590 LZ   26\n```\n\n zstd 0.4.2 was released Dec. 2, 2015.\n zstd 0.4.2_no_legacy (NL) was released Dec. 6, 2015. It is the same\nprogram with reduced source code size by dropping legacy support.\n \n\n```\n         Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram   Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------   -------       ----------  -----------  -----------  -----------  ----- -----  --- ---  ----\nzstd                    40,024,854  354,602,693     91,253 s  354,693,946    7.7   3.8  1.6 LZ77  48\nzstd                    40,024,854  354,602,694     91,253 s  354,693,947   23.3  13.9  1.1 LZ77  26\nzstd 0.4.0  -f20        27,195,437  233,505,508    301,514 s  233,807,022  432     1.6      LZ77  76\nzstd 0.4.2  -1          40,799,603  358,186,203    289,735 s  358,475,938    7.1   3.6    2 LZ77  48\n            -9          31,789,761  278,571,002    289,735 s  278,860,737   79     3.7   11 LZ77  48\n            -20         27,195,437  233,505,508    289,735 s  233,795,243  699     6.5  721 LZ77  48\n            -20         27,195,437  233,505,508    289,735 s  233,795,243  423     1.7  722 LZ77  76\nzstd NL 0.4.2  -20      27,195,437  233,505,508     59,431 s  233,564,939  423     1.7  722 LZ77  76\nzstd 0.5.1  -21         25,571,637  219,432,125     67,144 s  219,499,269  608     1.8      LZ77  76\n            -21         25,571,637                                         998     6.5  722 LZ77  48\nzstd 0.6.0  -22                     236,376,273     69,687 s  236,445,960  473     1.6      LZ77  76\n            -22 --ultra 25,405,601  215,674,670     69,687 s  215,744,357  701     2.2  792 LZ77  76\nCompression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp   Mem Alg  Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----   --- ---- ----\ntornado     0.1     -9           34,491,218  303,034,530     20,336 s  303,054,866    204    25   210 LZ77\ntornado     0.3     -1           59,790,826                                            18             LZ77\n                    -2           44,570,662                                            22             LZ77\n                    -3           40,173,986                                            28             LZ77\n                    -4           37,849,654                                            60             LZ77\n                    -5           34,206,892                                            81             LZ77\n                    -6           33,319,753                                           130             LZ77\n                    -7           32,346,652                                           195          96 LZ77\n                    -8           31,659,225                                           304         192 LZ77\n                    -9           30,967,871                                           506         384 LZ77\n                    -10          30,614,648                                           802         768 LZ77\n                    -11          30,274,896  259,412,590     45,833 s  259,458,423   1646    25  1510 LZ77\n                    -12          30,057,549                                          3700    28  1768 LZ77\ntornado     0.4a    -11          30,157,610  258,761,459     42,516 s  258,803,975    783    25  1513 LZ77\n                    -12          30,026,843                                          3200    29 >1800 LZ77\ntornado     0.6     -1           59,790,838  531,349,003                                8     5     2 LZ77  48\n                    -2           49,093,116                                             8     6     3 LZ77  48\n                    -3           39,510,585                                            14     9     5 LZ77  48\n                    -4           38,018,770                                            18     9    11 LZ77  48\n                    -5           34,175,257  300,482,758                               41     9    25 LZ77  48\n                                 34,175,257  300,482,758                               93    24    29 LZ77  26       \n                    -6           32,921,124                                            57    10    97 LZ77  48\n                    -7           30,131,376                                           134    10   229 LZ77  48\n                    -8           29,507,281                                           290    11   613 LZ77  48\n                    -9           29,327,427                                           392    11   613 LZ77  48\n                    -10          29,048,467                                           371    11   628 LZ77  48\n                    -11          30,108,427                                           270    10   356 LZ77  48\n                    -12          28,596,548                                           397     9   356 LZ77  48\n                    -13          28,042,448                                           503     9   484 LZ77  48\n                    -14          27,129,826                                           672     9   614 LZ77  48\n                    -15          26,762,749                                           985    10   614 LZ77  48\n                    -16          25,768,105  217,749,028     83,694 s  217,832,722   1482     9  1290 LZ77  48\n```\n\n LZPXj 1.2h, Mar. 6, 2007, uses LZP + PPM with a preprocessor for x86 executables.\nIt has just one option (1-9) which select memory usage.\nThe default is 6.  The maximum is 9.  Each increment doubles usage.\n \n\n```\n                Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Notes\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----  --- ---  -----\nLZPXj 1.1b        -s (best, = -r4 in 1.1d)    28,387,611                                           674            LZP\nLZPXj 1.1b           (default)                28,440,958                                           677            LZP\nLZPXj 1.1d        -m3 -r4 -a0 -e0             28,386,512  246,468,866      6,534 s  246,475,400    362   402  216 LZP\nLZPXj 1.2h        9                           25,205,783  217,880,584      4,853 s  217,885,437    783   717 1316 PPM\n```\n\n The -l 9 option selects maximum compression.\n fpaq0s2 is a\nfree, open source (GPL) file compressor by Nania Francesco Antonio, Sept, 29, 2006.\nIt is an order 2 model based on the order 0 compressor fpaq0s by David A. Scott, \nwhich is based on fpaq0 by Matt Mahoney by modifying the arithmetic coder.  \nfpaq0x is the same order 2 model based directly on fpaq0.\n fpaq0x1a is an order 3 model (hashed context) using fpaq0's arithmetic coder.\nfpaq0s2b is a similar model based on fpaq0s.  Both were released Oct. 1, 2006.\n fpaq0x1b (Oct. 6, 2006) switches between different models up to order 3.\n fpaq0s3 (Oct. 8, 2006) uses a simple order 0 model on groups of 3 bytes.\n fpaq0s4 (Oct. 12, 2006) uses a combined order 0-1-2, PPM and LZ model.\n fpaq0s5 (Oct. 15, 2006) improves on fpaq0s4.  Memory usage is 200 MB when\nrun at normal priority and 160 MB when run at below normal priority (WinXP Home).\n fpaq2 (Oct. 21, 2006) uses a combination context mixing and PPM algorithm.\n fpaq0s6 (Oct. 30, 2006) improves on fpaq0s5.\n fastari (Nov. 7, 2006) is an order 2 compressor with an all new arithmetic coder\nand greater speed.\n fpaq3 (Nov. 20, 2006) is an order 3 compressor.\n fpaq3b (Dec. 2, 2006) is a bitwise order 28 compressor.\n fpaq3c (Dec. 21, 2006) is an improved bitwise order 28 compressor.\n fpaq3d (Dec. 28, 2006) adds an option to fpaq3c to select memory\nusage from 16 MB to 2 GB.  Option 6 selects 1 GB memory (the highest tested).\n All programs are  \n\n```\nProgram Opt   enwik8      enwik9     prog (zip)   enwik9+prog  Comp Decomp  Mem Alg\n------- --- ----------  -----------  -----------  -----------  ----- -----  --- --\nfpaq2       25,287,775  221,242,386      3,429 s  221,245,815  20183 20186  131 CM\nfpaq3d    6 26,656,082  233,750,402      3,309 s  233,753,711   1922  1938 1050 o28b\nfpaq3c      27,978,995  248,253,886      2,535 s  248,256,421   1446  1456  268 o28b\nfpaq0s6     30,012,650  263,438,012      4,150 s  263,442,162    547   505  174 PPM\nfpaq0s5     30,374,122  266,244,843      4,027 s  266,248,870    480   419  200 PPM\nfpaq3b      29,992,583  270,804,549      2,926 s  270,807,475   1526  1517  256 o28b\nfpaq3       31,176,104  282,922,749      8,820 x  282,931,569   1770  1807  250 o3\nfpaq0x1b    30,860,828  283,001,299      2,727 s  283,004,026   1178  1180 1094 PPM\nfpaq0s4     33,327,611  311,104,858      3,528 s  311,108,386    477   473  147 PPM\nfpaq0x1a    36,186,433  339,131,763      2,561 s  339,134,324    621   623 1052 o3\nfpaq0s2b    35,934,548  343,603,459      3,029 s  343,606,488    599   605 1052 o3\nfastari     39,392,220  371,909,475      2,287 s  371,911,762    224   261  133 o2\nfpaq0s2     38,812,873  375,050,952      2,982 s  375,053,934    591   595  131 o2\nfpaq0x      38,845,305  375,276,899      2,482 s  375,279,381    631   631  263 o2\nfpaq0s3     49,728,923  490,781,136      3,000 s  490,784,136    525   475   32 o2\nCompressor   Opt         enwik8      enwik9         Prog       Total       Comp Decomp  Mem Note\n---------    ---       ---------   -----------     -------   -----------   ----  ----   --- ----\nTinyCM 0.1     9      25,913,605   221,773,542      12,553 x 221,786,095   1342  1330  1083 26\ngcc -O -s -Dexp=expand dmc.c\n```\n\n lza 0.51 was released Sept. 8, 2014. A\n \n\n```\n                Compression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nlza 0.01                         39,644,188  302,602,114                              142     9  111 LZ77 48\n              -m5 -b6 -h6 -t1    32,766,063  275,376,918    159,693 x  275,536,611    345    11 1024 LZ77 48\n              -m5 -b6 -h6 -t2    33,496,841  277,860,891                              237    20 2048 LZ77 48\nlza 0.10      -mx5 -b6 -h6       29,052,976  250,653,981    159,953 x  250,813,934    238    11 1012 LZ77 48\nlza_x64 0.10  -mx5 -b7 -h7       28,835,165  246,671,312    259,425 x  246,930,737    265    12 1800 LZ77 48\nlza 0.51      -mx5 -b6 -h6 -t1   28,365,587  242,852,984    179,415 x  243,032,399    243    10 1065 LZ77 48\nlza_x64 0.51  -mx5 -b7 -h8 -t1   27,992,585  234,652,984    218,944 x  234,871,928    261    14 2998 LZ77 48\nlza_x64 0.61  -mx5 -b7 -h7       28,019,802  236,604,708    218,090 x  236,822,798    279    10 1999 LZ77 48\nlza_x64 0.62  -mx5 -b7 -h9       27,870,452  231,801,036    219,447 x  232,020,483    409   9.5 5000 LZ77 69\nlza_x64 0.70b -mx9 -b7 -h7       27,111,239  229,073,644    260,686 x  229,334,330    378    10 2000 LZ77 48\nlza 0.80      -mx9 -b7 -h7       27,148,092  229,483,126    284,285 x  229,764,411    456    12 2152 LZ77 48\nlza 0.82b     -mx9 -b7 -h7       26,396,613  222,808,457    285,766 x  223,094,223    449   9.7 2000 LZ77 48\n```\n\n The test was repeated on the release as of Feb. 18, 2016. -w 24\nselects the window size. Default is -w 22.\n \n\n```\nCompression                         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  CMem Dmem Alg Note\n-------           --------       ----------  -----------  -----------  -----------  ----- -----  ---- ---- --- ----\nbro 21 Sep 2015   -q 1           36,893,038  326,282,447    514,344 s  326,796,791     18   5.2    10    5 LZ77 48\n                  -q 5           33,414,623  292,394,323    514,344 s  292,908,667     57   4.8    23    5 LZ77 48\n                  -q 9           30,227,230  264,047,624    514,344 s  264,561,968    361   5.0    77    6 LZ77 48\n                  -q 11          27,721,194  240,891,082    514,344 s  241,405,426   4386   5.0   294    6 LZ77 48\nbro 18 Feb 2016   -q 1           38,802,994  343,293,825    542,345 s  343,836,170     12.4 6.3     8    5 LZ77 48\n                  -q 5           33,414,209                                            59   4.5    38    6 LZ77 48\n                  -q 9           30,227,246                                           407   4.7    68    6 LZ77 48\n                  -q 11          27,076,871  235,560,131    542,345 s  236,102,476   3171   5.1   292    6 LZ77 48\n                  -q 11 -w 24    25,764,698  223,597,884    542,385 s  224,140,269   3400   5.9   437   18 LZ77 48\n```\n\n The option -b41o16 selects a block size of 4.1 MB (the maximum) and order 16, the maximum\nlength of string comparisons. Memory usage is 17 MB (4x block size) for compression\nand 21 MB (5x block size) for decompression. o0 means unbounded order, which is the\nsame as a normal BWT. The default is -b16o6.\n \n\n```\nCompressor   Opt         enwik8      enwik9         Prog       Total       Comp Decomp  Mem Note\n---------    ---       ---------   -----------     -------   -----------   ----  ----   --- ----\nszip 1.12a   -b41o16   26,120,472  227,586,463     31,708 x  227,618,171   1191   289    21  26\n             -b41o4    27,561,829                                            70   210    21  26\n             -b16o6    27,666,448                                           270   220     8  26\n             -b41o6    26,365,058                                           360   240    21  26\n             -b41o8    26,185,222                                           530   250    21  26\n             -b41o32   26,128,020                                          2550   400    21  26\n             -b41o64   26,130,850                                          5210   600    21  26\n             -b41o0    26,130,985                                           750   200    21  26\n```\n\n balz 1.02 is a free,\nclosed source file compressor by Ilia Muraviev, Mar. 8, 2008.  It uses LZ77\nwith arithmetic coding, a 512K buffer with Storer and Symanski parsing.\nIt takes no options.  Memory usage is 346 MB for compression and 18 MB for\ndecompression.\n balz  balz  balz  balz  balz  balz  \n\n```\nCompressor   Opt     enwik8      enwik9         Prog      Total       Comp Decomp  Mem Alg  Note\n---------    ---   ---------   -----------     -------  -----------   ----  ----   --- ---- ----\nbalz 1.02          30,634,726  268,552,062     48,030 x  268,600,092  21804    58  346 LZ77\nbalz 1.06    e     28,674,640                                          1580    79   67 ROLZ\nbalz 1.06    ex    28,234,913  245,288,229     48,937 x  245,337,166   2440    75   67 ROLZ\nbalz 1.07    e     28,271,200                                          1060    96  132 ROLZ\nbalz 1.07    ex    27,416,245  237,492,151     49,082 x  237,541,233   2106    77  132 ROLZ\nbalz 1.08    ex    26,534,890  229,477,116     49,351 x  229,526,467   4431   126  200 ROLZ\nbalz 1.09    ex    26,534,257  229,476,459     49,928 x  229,526,387   4049   128  201 ROLZ\nbalz 1.12    e     27,522,348                                          1800   177  201 ROLZ\nbalz 1.12    ex    26,522,258  229,347,434     48,400 x  229,395,834   3989   148  201 ROLZ\nbalz 1.13    e     27,405,650                                          1670   221  206 ROLZ\nbalz 1.13    ex    26,421,416  228,337,644     49,024 x  228,286,668   3700   190  206 ROLZ\nbalz 1.15    ex    28,232,824  245,218,274      4,045 s  245,222,319   1064    95   67 ROLZ\nbalz 1.20    c     30,056,097  261,416,611      3,499 s  261,420,110     53            ROLZ  68\nbalz 1.20    cx    28,232,824  245,218,274      3,499 s  245,221,773    193    22      ROLZ  68\nCompression            Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram      Opt      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------      ---    ----------  -----------  -----------  -----------  ----- -----  --- ----\nlzpm 0.02           29,274,461  254,596,796     26,078 x  254,622,874    612    59   83 LZ77\nlzpm 0.03           29,248,641  254,378,973     26,089 x  254,405,062    749    59  181 LZ77\nlzpm 0.04           29,297,905  254,793,933     25,333 x  254,819,266    665    60   83 ROLZ\nlzpm 0.06           28,896,680  251,111,835     25,369 x  251,137,204    852    58   83 ROLZ\nlzpm 0.07           28,385,939  246,426,198     46,692 x  246,472,890   2185    56  280 ROLZ\nlzpm 0.08           28,259,984  245,221,254     48,122 x  245,269,376   2754    59  280 ROLZ\nlzpm 0.09           27,986,111  242,929,442     46,933 x  242,976,375   2451    56  280 ROLZ\nlzpm 0.10           27,849,915  241,719,857     46,871 x  241,766,728   2598    57  280 ROLZ\nlzpm 0.11     1     29,728,112                                          1162    76  723 ROLZ\n              2     27,967,747                                          3746    66  723 ROLZ\n              3     27,424,937                                          5204    68  723 ROLZ\n              4     27,239,304                                          6488    66  723 ROLZ\n              5     27,134,495                                          7446    63  723 ROLZ\n              6     27,038,405                                          8143    64  723 ROLZ\n              7     26,962,337                                          8761    63  723 ROLZ\n              8     26,890,422                                          9330    62  723 ROLZ\nlzpm 0.11     9     26,501,542  229,083,971     46,824 x  229,130,795  15395    57  723 ROLZ\nlzpmlite 0.11 1     30,136,214                                           627    69  362 ROLZ\n              3     27,918,695                                          2620    64  362 ROLZ\nlzpmlite 0.11 9     27,096,516  235,135,224     48,144 x  235,183,368   6235    59  362 ROLZ\nlzpm 0.12     9     27,391,197  237,915,048     47,030 x  237,962,078   4501    57  280 ROLZ\nlzpm 0.13     9     27,318,013  237,241,658     47,129 x  237,288,787   4543    59  280 ROLZ\nlzpm 0.14     9     27,091,358  235,074,141     48,790 x  235,122,931   6467    73  428 ROLZ\nlzpm 0.15     9     27,145,224  235,567,823     48,401 x  235,616,224   6557    62  427 ROLZ\n```\n\n The -d9 option selects maximum dictionary size.  -x7 selects\nmaximum hash level (most memory).  -l7 selects maximim search level\n(slowest).\n \n\n```\nCompression                         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram    Version                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  CMem Dmem Alg Note\n-------    --------              ----------  -----------  -----------  -----------  ----- -----  ---- ---- --- ----\nKuaiZip    2.3.2 x86             25,895,915  227,905,650  3,857,649 x  231,763,299   1061    47   197   19 LZMA  26\n```\n\n See  The program takes no arguments. It uses 103 MB (24x block size) for compression and\n25 MB (6x block size) for decompression. There is a Windows and a Linux version.\nOnly the Windows version was tested.\n comprox_ba 20110928 was released Sept. 28, 2011. Compression runs in 2 threads. Both the Windows\nand Linux versions were tested (on different computers).\n comprox_ba 20110929 was released Sept. 29, 2011. Compression is slightly improved.\nBoth compression and decompression are now multi-threaded.\n \n\n```\nCompression                         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram    Version                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  CMem Dmem Alg Note\n-------    --------              ----------  -----------  -----------  -----------  ----- -----  ---- ---- --- ----\ncomprox_ba 20110927              27,831,722  242,858,769      4,165 s  242,862,934   1500   227   103   25 BWTS 26\ncomprox_ba 20110928 (Win32)      27,831,722  242,858,769      4,151 s  242,862,920    957   227   206   25 BWTS 26\ncomprox_ba 20110928 (Linux)      27,831,722  242,858,769      4,151 s  242,862,920    363   168   226   30 BWTS 48\ncomprox_ba 20110929 (Win32)      27,828,189  242,846,243      4,134 s  242,850,377    984   152   206   50 BWTS 26\ncomprox_ba 20110929 (Linux)      27,828,189  242,846,243      4,134 s  242,850,377    397   101   226   76 BWTS 48\n```\n\n WinTurtle 1.2 is a Windows GUI\nversion of turtle, released Aug. 16, 2007.  It uses PPM with LZP preprocessing.\nIt detects .tar, .iso, .nrg, .wav, .aiff, .bmp, .exe, .pdf, .log and text files.\nCompression times are wall times.  Note: the user interface is not fully functional.\nTo compress a file, click \"Drive\", click on \"Buffer\" until it is set to 512 MB (it does not\nwork until you click \"Drive\" first, also 1 GB caused program to crash on enwik8), \nselect \"File/compress single file\" from the upper menu,\nthen select the input file and output archive from the two file dialogs.\nThe program adds a .tur extention to the output archive.  To decompress,\nselect File/open archive, click on the file name, click Select, click Extract,\nand select an output folder from the file dialog.\n WinTurtle 1.21, Aug. 16, 2007,\nfixes an unrelated bug but is otherwise the same as 1.2.\n WinTurtle 1.30 was released Aug. 30, 2007.\n \n\n```\nCompressor   Opt       enwik8      enwik9         Prog      Total       Comp Decomp  Mem Alg\n---------    ---     ---------   -----------     -------  -----------   ----  ----   --- ----\nturtle v0.01         31,314,961  274,696,820     5,079 x  274,701,899    187   178   122 PPM\nturtle v0.02         31,314,961  274,696,820     4,637 x  274,701,457    196   175   122 PPM\nturtle v0.03         31,287,161  274,649,069     7,111 x  274,656,180    142   129   122 PPM\nturtle v0.04         31,137,531  273,100,225     7,808 x  273,108,033    141   128   122 PPM\nturtle v0.05         28,860,689  251,626,176     9,779 x  251,635,955    242   203   174 PPM\nturtle v0.07         28,669,320  250,600,644    10,625 x  250,611,269    217   175   206 PPM\nWinTurtle 1.2  8MB   29,601,717  258,927,402   238,080 x  259,164,482    248   242    31 PPM\n               512MB 28,814,475  250,364,644   238,080 x  250,598,724    264   240   548 PPM\nWinTurtle 1.21 512MB 28,814,475  250,364,644   225,123 x  250,589,767    255   219   548 PPM\nWinTurtle 1.30 512MB 28,814,478  250,364,647   239,247 x  250,603,594    243   240   597 PPM\nWinTurtle 1.60 512MB 28,379,612  245,217,944   160,090 x  245,378,034    273   237   583 PPM\n```\n\n Compression is as follows.\nA 20-bit hashed order-4 context is mapped into the last 3 bytes seen\nin that context in a move-to-front queue, plus a consecutive hit count.\nQueue positions (hits) or literals (misses) are arithmetic coded using\nthe count and an an order-1 context (order-0 if the count is more than 3)\nas secondary context.  After a byte is coded, it is moved to the front of the queue.\nThe hit count is updated as follows: incremented (max 63) if the first byte\nis matched, set to 1 if any other byte is matched, or set to 0 in case of a miss.\n sr3.exe was recompiled on July 23, 2009\nwithout upack to remove antivirus false alarms, resulting\nin a larger executable. The new size is shown using source code.\n \n\n```\nProgram    enwik8      enwik9         prog       Total      Comp  Deco Mem Alg\n-------  ----------  -----------      ----   ------------   ----  ---- --- --- \nsr2      30,432,506  273,906,319     2,831 sd 273,909,150     99   111   6 SR\nsr3      28,926,691  253,031,980     5,611 x  253,037,591    130   146  68 SR\nsr3      28,926,691  253,031,980     9,399 s  253,054,625    148   160  68 SR  26\nCompression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp\n-------           -------       ----------  -----------  -----------  -----------  ----- -----\nbzip2 1.0.2       -9            29,008,736  253,977,839     30,036 x  254,007,875    379   129\nbzip2 1.0.3       -9            29,008,758  253,977,891     56,082 xd 254,033,973    334   120\n```\n\n The 32 and 64 bit Windows .exe versions produce\nincompatible archives. The 32 bit version was tested in Windows.\nThe 64 bit version was tested in Ubuntu under Wine 1.6.\n \n\n```\n               Compression     Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram          Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------          -------    ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\nRH_x86                      35,675,086                  91,772 x                  78    47    8 ROLZ  26\nRH2_x86            c1       34,857,781                                            67    27   64 LZP   26\n                   c2       31,957,388                                           149    28   64 ROLZ  26\n                   c3       31,937,059  279,524,710     93,364 x  279,618,074    152    28   64 ROLZ  26\nRH2_x64            c3       31,937,063  279,524,714     97,016 x  279,621,730     72    20   64 ROLZ  48\nRH2_x64 20Feb2014  c1       34,816,471  306,646,293                               32    15   64 LZP   48\n                   c2       32,215,361  282,209,254                               48    14   64 ROLZ  48\n                   c3       30,960,001  271,181,799                               67    17   64 ROLZ  48\n                   c4       30,787,281  269,670,002                               76    15   64 ROLZ  48\n                   c5       30,543,306  267,344,532     53,408 x  267,397,940    447    18   64 ROLZ  48\nRH4_x64 22Mar2014  c1       32,664,118                                            44    13\n                   c2       31,309,650                                            47    12\n                   c3       30,906,206                                            61    12\n                   c4       30,872,697                                            64    12\n                   c5       30,030,867                                           128    11\n                   c6       29,553,289  258,411,625     79,155 x  258,490,780    301    12   27 ROLZ  48\nRH4_x64 24Apr2014  c2       31,309,670  274,101,406     90,071 x  274,191,477     44     9   31 ROLZ  48\n                   c6       29,553,309  258,411,645     90,071 x  258,501,716    287     9   31 ROLZ  48\nRH5_x64            c2       31,798,141  278,822,435     36,744 x  278,859,179     28    11   22 ROLZ  48\n                   c6       29,878,256  261,791,548     36,744 x  261,828,292    153    11   22 ROLZ  48\n        -window:27 c6       29,078,552  254,220,469     36,744 x  254,257,213    196   9.4  145 ROLZ  48\n0 - Simple Bitwise Model (default)\n1 - Indirect Bitwise Model\n2 - Indexed Bitwise Model Array\n3 - Hashed Bitwise Model\n4 - Bitwise Linear CM\n5 - Bitwise Linear CM With SSE\n6 - Bytewise Hashed Model\n7 - Combined Model\n0 - Bytewise Hashed Model\n1 - Simple Bitwise Model (default)\n2 - Adaptive Bitwise Model\n3 - Indexed Bitwise Model Array\n4 - Hashed Bitwise Model\n5 - Combined Model\nCompression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram             Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp Mem  Note\n-------             -------       ----------  -----------  -----------  -----------  ----- ----- ---  ----\nRangeCoderC v1.2    0             99,801,301\n                    1             99,660,153\n                    2             97,987,717\n                    3             96,963,829\n                    4             95,670,157\n                    5             94,154,825\n                    6             87,831,925\n                    7             80,009,581\n                    8             73,016,189\n                    16            46,805,877\n                    24            35,625,897\n                    25            34,635,889\n                    26            33,761,533  320,897,805      4,120 x  320,901,925   1324  1348 1050  26\nRangeCoderC v1.3    27            33,225,249  314,021,089      3,977 x  314,025,066   1210  1234 1050  26\n                    26 (Double)   30,934,993  285,258,957      4,052 x  285,263,009   1501  1488 1100  26\nRangeCoderC v1.4    27 (Hashed)   30,371,685  271,371,793      4,407 x  271,376,200   1809  1658 1050  26\n                    26 (Double)   30,934,989                   4,359 x                1560  1650 1116  26\n                    27 (Indirect) 36,108,281                   4,773 x                2700  3090 1182  26\n                    27 (Standard) 33,225,245                   4,288 x                1210  1270 1050  26\nRangeCoderC v1.5    c3 27         30,371,685                   5,747 x                1740  1810 1050  26\nRangeCoderC v1.6    c0 26         33,761,529                   7,028 x                1200  1230  525  26\n                    c0 27         33,225,245                                          1280  1330 1050  26\n                    c2 26         30,934,989                                          1610  1680 1116  26\n                    c3 26         30,832,497                                          1610  1720  525  26\n                    c3 27         30,371,685                                          1740  1790 1050  26\n                    c4 26         29,269,185                                          5320  5880 1642  26\n                    c5 26         28,461,477  260,009,661      7,028 x  260,016,689   5752  5833 1642  26\nRangeCoderC v1.7a   c1 27         36,108,281                   7,060 x                2570  3000 1182  26\nRangeCoderC v1.7    c0 27         33,225,245                                          1300  1330 1050  26\n                    c1 27         36,108,281                                          2490  2420 1182  26\n                    c2 26         30,934,989                                          1590  1660 1116  26\n                    c3 27         30,371,685                                          1710  1980 1050  26\n                    c4 26         29,269,185                                          5120  5130 1641  27\n                    c5 26         28,461,477  260,009,661      7,858 x  260,017,519   5832  5779 1642  26\n                    c6 27         35,265,593                                           990  1020 1050  26\n                    c7 26         28,788,013  254,527,369      7,858 x  254,535,227   2460  2436 1116  26\nRangeCoderC v1.8    c2 28         32,432,825  285,488,437      6,537 x  285,494,974   1338  1363 1050  26\n```\n\n As described by the author:\nQUAD uses ROLZ compression (Reduced Offset LZ). It makes use of an order-2 context to \nreduce the offset set that is matched to. This can be regarded as a fast large \ndictionary LZ. Literals and Match Lengths fits in a single alphabet which is coded \nusing an order-2-0 PPM with Full Exclusion. Match indexes are coded using an order-0 \nmodel. QUAD uses a 16 MB dictionary. For selectable compression speed and ratio, QUAD \nuses different parsing schemes: with Normal mode (Default) QUAD uses a Lazy Matching; \nwith Max mode (-x option) QUAD uses a variant of Flexible Parsing. In addition, QUAD \nhas an E8/E9 transformer for better executable compression which is always enabled. \n quad 1.01a (Dec. 24, 2006) used LZ77.  It was closed source and took no options.\n quad 1.04a (Feb. 8, 2007) used LZP.  Memory was expanded for this version\nonly, however it is no longer supported.\n quad 1.07beta (Feb. 22, 2007)\nincluded the \"x\" option for better compression.\n quad 1.08 was released Mar. 12, 2007.  Quad became open source.\n quad 1.10 was released Mar. 19, 2007.  -x selects maximum compression.\n quad 1.11 (Apr. 4, 2007) uses ROLZ.\n quad 1.12 was released Apr. 7, 2007.\n \n\n```\nCompression            Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram              enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------            ----------  -----------  -----------  -----------  ----- -----  --- ----\nquad v1.01a        29,930,547  263,137,995     26,927 x  263,164,922   1281   168   33 LZ77\nquad v1.04a        27,712,832  239,596,416     38,552 x  239,634,968    933   748  165 LZP\nquad v1.07b     x  29,360,404  258,361,092     61,067 x  258,422,159   1282   146   33 LZP\nquad v1.08      x  29,171,593  256,664,803     13,042 s  256,677,845   1206   164   33 LZP\nquad v1.10      -x 29,152,166  256,486,470     13,288 s  256,499,758   1007   117   34 LZP\nquad v1.11      -x 29,110,579  256,145,858     13,387 s  256,159,245    956   116   34 ROLZ\nquad v1.11HASH2 -x 29,110,519  256,145,858     30,129 x  256,175,987    705   117   42 ROLZ\nquad v1.12      -x 29,110,519  256,145,858     13,516 s  256,159,334    527   120   34 ROLZ\nCompression         Compressed size      Decompresser  Total size   Time (ns/byte)\n  Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp\n  -------        ----------  -----------  -----------  -----------  ----- -----\n-sfx -m5 -d4096  29,481,470  257,237,710          0 xd 257,237,710   1080    77\n-sfx -m5         30,919,182  270,578,538          0 xd 270,578,538    738    79\n-sfx             30,937,342                                          ~770   ~40\nCompression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nzling Nov-01-2013                33,297,650  292,746,596      5,468 s  292,752,064     80    21   37 ROLZ 26\nzling Dec-25-2013                32,222,737  282,435,374     12,807 s  282,448,181     33     8   27 ROLZ 48\nzling Jan-21-2014                32,189,336  281,869,136     14,886 s  281,884,022     78    21   29 ROLZ 26\nzling Jan-21-2014                32,189,336  281,869,136     14,886 s  281,884,022     29     7   29 ROLZ 48\nzling_demo Feb-19-2014           31,310,257  274,180,830     32,046 s  274,212,876     56    14   27 ROLZ 48\nzling_demo Mar-24-2014   e0      33,391,083                                            24     9   27 ROLZ 48\n                         e1      32,613,829                                            29     9   27 ROLZ 48\n                         e2      31,732,466                                            33     9   27 ROLZ 48\n                         e3      31,310,257                                            40     9   27 ROLZ 48\n                         e4      30,861,848  270,258,636     32,421 s  270,291,057     51     9   27 ROLZ 48\nzling_demo 201401414     e0      32,456,306  284,804,449                               23     9   27 ROLZ 48\n                         e1      31,800,497  278,703,086                               28     9   27 ROLZ 48\n                         e2      31,419,861  275,231,487                               32     9   27 ROLZ 48\n                         e3      31,064,418  271,969,050                               36     9   27 ROLZ 48\n                         e4      30,782,340  269,496,300     31,644 s  269,527,944     42     9   27 ROLZ 48\nzling_demo 20140430-bugfix e0    32,378,187                                            29    11   27 ROLZ 48\n                         e1      31,720,214                                            30    11   27 ROLZ 48\n                         e2      31,340,822                                            34    11   27 ROLZ 48\n                         e3      30,979,872                                            39    11   27 ROLZ 48\n                         e4      30,707,022  268,793,105     32,148 s  268,825,253     40    10   27 ROLZ 48\nzling_demo 20160107      e0      31,455,205                                            93    29   22 ROLZ 48\n                         e4      29,721,114  259,475,639     35,582 s  259,511,221     83    27   28 ROLZ 48\nCompression         Compressed size      Decompresser  Total size   Time (ns/byte)\n  Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem  Alg Note\n  -------        ----------  -----------  -----------  -----------  ----- -----  ---  --- ----\nxpv5 c0          31,675,180  277,174,541     14,371 x  277,188,912   908    534    9 ROLZ  26\n     c1          30,297,863  265,643,665     14,371 x  265,658,036  1236    515    9 ROLZ  26\n     c2          29,963,217  262,525,246     14,371 x  262,539,617  2359    516    9 ROLZ  26\n```\n\n lzc 0.03 was released May 11, 2007.\n lzc 0.04 was released May 16, 2007.  All versions up to 0.04\nuse 107 MB memory for decompression.\n lzc 0.05b was released May 26, 2007.  It has options from 1 (fastest)\nto 16 (best compression).  It uses 771 MB to compress and 390 MB to decompress.\n All versions through 0.05b are linked in the above archive.\n lzc 0.06b was released Aug. 27, 2007.\nIt uses 790 MB (peak) for compression and 409 MB (peak) for decompression.\n lzc 0.07 was released Oct. 24, 2007.\nOptions range from 1 (fastest) to 10 (slowest).\n \n\n```\nCompressor   Opt     enwik8      enwik9         Prog      Total       Comp Decomp  Mem Alg\n---------    ---   ---------   -----------     -------  -----------   ----  ----   --- ----\nlzc v0.01     4    40,312,925  363,504,638     7,656 x  363,512,294    238    61   360 LZ77\nlzc v0.03     4    37,908,748  341,811,895     8,268 x  341,820,163    182    61   515 LZ77\nlzc v0.04     4    37,779,426  340,628,765     8,869 x  340,637,634    142    59   540 LZ77\nlzc v0.05b    1    44,893,624                                          117    54       LZ77\nlzc v0.05b   16    30,611,315  267,784,591     9,158 x  267,793,749    365    82   771 LZ77\nlzc v0.06b   16    30,611,315  267,784,590    12,170 x  267,796,760    347    68   790 LZ77\nlzc v0.07     1    40,554,444                                          110    60    70 LZ77\nlzc v0.07    10    30,611,315  266,565,255    28,997 x  266,594,252    309    67   584 LZ77\nlzc v0.08    10    30,611,315  266,565,255    11,364 x  266,576,619    302    63   550 LZ77\nCompressor             Opt     enwik8      enwik9       Prog       Total       Comp Decomp     Mem Alg Note\n---------              ---   ---------   -----------   -------   -----------   ----  ----     --- ---- ----\nnakamichi 2019-Jul-01        32,917,888  277,293,058   112,899 s 277,405,957 8200000  1.3  302000 LZSS 85\n0,xxxxxxxx              - literal byte x\n  1,1,xx                  - match length x+3   (3..6)\n  1,0,1,xx                - match length x+7   (7..10)\n  1,0,0,1,xx              - match length x+11  (11..14)\n  1,0,0,0,1,xxx           - match length x+15  (15..22)\n  1,0,0,0,0,1,xxxxx       - match length x+23  (23..54)\n  1,0,0,0,0,0,xxxxxxxxx   - match length x+55  (55..566)\n```\n\n The compressor maintains an index for finding matches consisting of two\nhash tables of size 2 To save memory, only the last 2 \n\n```\n               Compression     Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram          Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------          -------    ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\ncrush 0.01       cf         37,401,090  330,975,986     46,879 x  331,022,865     94  17.2  143 LZ77 26\ncrush 0.01       cf         37,401,090  330,975,986     46,879 x  331,022,865     21   4.2  143 LZ77 50\ncrush 0.01       c          33,618,865                                          1040  13    143 LZ77 26\ncrush 0.01       c          33,618,865  297,103,092     46,879 x  297,721,957    129   3.9  143 LZ77 50\ncrush 0.01       cx         32,577,338                                          4490  13    143 LZ77 26\ncrush 0.01       cx         32,577,338  287,333,602     46,879 x  287,380,481    532   3.8  143 LZ77 50\ncrush 0.01       cx         32,577,338  287,333,602      2,469 s  287,336,071    532   3.8  143 LZ77 50\n\ncrush 1.00       cf         37,308,893                                           132  15    148 LZ77 26\ncrush 1.00       c          32,878,537                                          1541  15    148 LZ77 26\ncrush 1.00       cx         31,731,537                                          7916  15    148 LZ77 26\ncrush 1.00       cx         31,731,711  279,491,430      2,489 s  279,493,919    948   2.9  148 LZ77 60\nCompression     Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram          Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------          -------    ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\nxeloz 0.3.5.3    c          35,504,888  312,908,049     18,771 s  312,926,820    238     6    8 LZ77 48\n                 c889       32,441,272  283,621,211     18,771 s  283,639,982   1079     8  230 LZ77 48\nxeloz 0.3.5.3a   C          37,343,227  329,469,433     18,849 s  329,488,282    134     7   24 LZ77 48\n```\n\n lzwhc is a single program that both compresses and decompresses. It was released\nAug. 9, 2023. It uses a hash table for better speed but produces the same output as lzwg.\nThe option -c28 selects 2^28 hash table entries of 9 bytes each for 2.4 GB.\n \n\n```\n               Compression     Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram          Options      enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------          -------    ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\nlzwg             -24        34,423,369                                          102     26  218 LZW  95\nlzwg             -27                    284,356,322    19,828 xd  284,376,150   135     41 1744 LZW  95\nlzwhc            -c28       34,423,369  284,356,322    30,720 x   284,387,042    95     42 2400 LZW  95\nVersion     Options      enwik8    Comp  Decomp Notes\n--------    -----      ----------  ----  ----   -----\nha 0.98      a1        36,379,137   873   257          ns/byte\nha 0.98      a2        31,250,524  2080  1850\nha 0.999b    a21       31,250,523  2447          16    DOS compile, 1995\nha 0.9991a   a21       31,250,524  1551          16    DOS (.com) compile, 1995\nha 0.999b    a21       31,250,524  1290          16    Compiled for NT by Michael Markowsky at Apr 30 1997\nlgha v1.1    a21       31,250,524  1110          16    ha v.0999c DOS compile by Lyapko George, 1999\nlgha v1.1              31,250,524  1068  1114    16\nProgram   Options    enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg  Note\n-------   -------  ----------  -----------   ----------   -----------   ---- ----- --- ---- ----\nulz 0.01  c1       45,751,335  411,826,108     47,809 x   411,873,917     50    11  43 LZ77 26\n          c2       41,677,764                                             77    10  43 LZ77 26\n          c3       39,368,127                                            145     9  43 LZ77 26\n          c4       37,861,566                                            581     9  43 LZ77 26\n          c5       37,652,826  332,626,591                332,674,400   1077     9  43 LZ77 26\nulz 0.02  c1       50,382,083                                             37    10  43 LZ77 26\n          c2       45,751,335                                             52    10  43 LZ77 26\n          c3       41,677,764                                             74     9  43 LZ77 26\n          c4       39,368,127                                            139     8  43 LZ77 26\n          c5       37,861,566                                            576     8  43 LZ77 26\n          c6       37,652,826  332,626,591     47,833 x   332,674,424   1056     8  43 LZ77 26\nulz 0.03  cf       45,613,380  402,610,627     48,583 x   402,659,210     13   3.2  29 LZ77 48\n          c        39,946,599  353,878,403     48,583 x   353,926,986     54   3.3  29 LZ77 48\n          cu       37,199,413  329,119,609     48,583 x   329,168,192    192   3.2 228 LZ77 48\n          cu       37,199,413  329,119,609     48,583 x   329,168,192    115   1.4 228 LZ77 68\nulz 0.06  c1       47,674,405  421,011,442     49,450 x   421,060,892    7.4   1.0  94 LZ77 82\n          c        41,660,387  365,851,618     49,450 x   365,901,068     30   1.1  94 LZ77 82\n          c9       32,945,292  291,028,084     49,450 x   291,077,534    325   1.1 490 LZ77 82\n```\n\n Only source code is available. For this test, the program irolz.cpp was\ncompiled using g++ 4.5.0 on a 2 GHz T3200 under 32 bit Vista with\noptions -O2 -march=pentiumpro -fomit-frame-pointer -s.\n lcssr 0.2 (Dec. 3, 2007, same website) \n \n\n```\n                Compression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nsymbra 0.2        -c4 -m0 -p1    38,308,164  352,524,859     11,299 s  352,536,158    245   282   68 SR  26\nsymbra 0.2        -c4 -m5 -p2    34,644,072  302,948,753     11,299 s  302,960,062   4669  4633  112 SR  26\nsymbra 0.2        -c5 -m5 -p2    34,683,661  302,656,095     11,299 s  302,667,394   4700  4622  112 SR  26\nlcssr 0.2         -b7 -l9        34,549,048  296,160,661      8,802 x  296,169,463   8186  8281 1184 SR  26\n```\n\n The LZ77 format codes literals uncompressed after a length code.\nMatches can have an offset in the range 1 to 2 Compression is achieved in a 16 MB sliding window implemented as a\npair of buffers. A hash table of 2 \n\n```\n                Compression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options          enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------           -------        ----------  -----------  -----------  -----------  ----- -----  --- ---\nlazy 1.00         1              40,518,222  359,237,695      5,986 s  359,243,681     57    25   36 LZ77\n                  2              38,580,043  340,152,648      5,986 s  340,158,634     75    25   40 LZ77\n                  3              37,074,105  325,609,617      5,986 s  325,615,603    104    29   48 LZ77\n                  4              35,908,430  314,545,955      5,986 s  314,551,941    166    25   64 LZ77\n                  5              35,024,082  306,245,949      5,986 s  306,251,935    273    24   96 LZ77\n```\n\n zhuff 0.7 \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- ---- ----\nzhuff 0.1                       43,299,291  384,578,436      9,626 x  384,588,062     16    10  1.4 LZ77 26\nzhuff 0.7         -t1           40,974,542  365,122,888     45,522 x  365,168,410     17    10   12 LZ77 26\n                  -t2           40,974,542  365,122,888     45,522 x  365,168,410     17    10   19 LZ77 26\nzhuff 0.8         -c0           40,990,942  365,277,964     50,939 x  365,328,903     18    13   19 LZ77 26\n                  -c1           36,235,017  320,629,066     50,939 x  320,680,005     73    12   19 LZ77 26\n                  -c2           35,078,148  309,881,876     50,939 x  309,932,815    111    11   19 LZ77 26\nzhuff 0.95b       -c0           40,615,710  362,653,616     61,684 x  362,715,300      6.5   4.2 32 LZ77 48\n                  -c1           35,973,813  319,010,291     61,684 x  319,071,975     15     3.6 32 LZ77 48\n                  -c2           35,022,597  309,639,139     61,684 x  309,700,823     24     3.6 32 LZ77 48\nzhuff 0.97 beta   -c0           37,076,873  328,438,763     63,209 x  328,501,972     10     4.0 32 LZ77 48\n                  -c1           35,864,003  317,929,499     63,209 x  317,992,708     16     3.7 32 LZ77 48\n                  -c2           34,907,478  308,530,122     63,209 x  308,593,331     24     3.5 32 LZ77 48\nCompression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram   Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------   -------       ----------  -----------  -----------  -----------  ----- -----  --- ---  --\nlzhhf                   34,848,933  308,825,079    24,576 xd  308,849,655    392   12    14 LZ77 95\nCompression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram   Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------   -------       ----------  -----------  -----------  -----------  ----- -----  --- ---\nslug 1.1b               45,274,048  404,250,979      5,836 x  404,256,815     18    14    1 LZ77\nslug 1.27               35,093,954  309,201,454      6,809 x  309,208,263     32    28   14 ROLZ\nProgram           enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg\n-------         ----------  -----------   ----------   -----------   ---- ----- --- ----\nlzuf            38,036,810  338,488,945      4,070 xd 338,493,015    446    40    2 LZ77 26\nlzuf62          34,960,889  309,837,920     24,576 xd 309,862,496    375    11   14 LZ77 95\n```\n\n pigz is distributed as source code only. It requires linking with  pigz 2.3, Mar. 4, 2013, adds option -11 implelemting Google's\n \n\n```\n                      Compression                 Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram                 Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Notes\n-------                 -------               ----------  -----------  -----------  -----------  ----- ----- ---- ---- --\ngzip 1.3.5              -9                    36,445,248  322,591,995     34,408 x  322,626,403     55    22  4.5 LZ77 48\npigz 2.2.3              -9                    36,490,716  322,926,625     36,521 xd 322,963,146     31    10  115 LZ77 48\npigz 2.3                                      36,565,142  324,081,152     52,717 s  324,133,869     25    12    3 LZ77 48\n                        -9                    36,490,716  322,926,625     52,717 s  322,979,342     29    13    3 LZ77 48\n                        -11                   35,002,893  309,812,953     52,717 s  309,865,670   2237    13   25 LZ77 48\n```\n\n[tc 5.2 dev 2](http://www.encode.su/downloads/tc52dev2.zip) uses FPW\n(fast PAQ weighting).\n## .1857 bwtsdc\n\n[bwtsdc v1](http://bijective.dogma.net/bwtsdcV1.7z)\n([discussion](http://encode.su/threads/1179-BIJECTIVE-BWTS-DC-FIB))\nis a free, experimental file compressor with source code\nby David A. Scott and Yuta Mori. It takes no options. Memory usage is\n5 times the file size.\nThe program\nis bijective, meaning that any file is valid input to the decompresser,\nand no two inputs will decompress to the same file. In other words, there is\nan exact 1 to 1 mapping between uncompressed files and compressed files.\nThe compressor uses multiple\nstages, each of which is bijective. The first stage is a BWT variant called\n[BWTS](dce.html#Section_559)\n(BWT Scottified) developed by Scott. In this variation, it is not necessary\nto store the starting point for the inverse BWT. This is achieved by dividing\nthe input into a lexicographically nonincreasing sequence of Lyndon words.\nA Lyndon word is any subsequence that lexicographically precedes\nany of its rotations. The block is then sorted using contexts that wrap\nwithin Lyndon words rather than the whole block.\nThe BWTS is followed by distance coding (DC, developed in part by Mori),\nand Fibonacci coding, where each stage is also bijective.\nThe compressor is implemented as 3 programs called from a .bat file.\n## .1859 fbc\n\n[fbc](http://encode.su/threads/1502-FBC-Compressor) v1.0 is\na free, experimental file compressor for Windows by David Catt, Feb. 29, 2012.\nIt is described as using BWT (divsufsort) with a fast adapting (rate 1/16)\n14 bit context model consisting of an 11 bit history and 3 bits\nto encode the position in the current byte. The input is preprocessed using\nEugene Shelwein's alphabet reordering preprocessor, BWT_reorder_v2.\nThe argument 250000000\nselects the block size in bytes. Memory usage if 5 x block size.\n## .1862 ppmvc\n\n[ppmvc v1.1](http://www.ii.uni.wroc.pl/~inikep/research/PPMVC/PPMVC11.zip)\nis a free, command line file compressor by Przemysław Skibiński, May 12, 2006,\nbased on PPMd var. J by Dmitry Shkarin.  It uses variable length contexts\nas described in the paper,\nP. Skibinski and Sz. Grabowski.\n[Variable-length contexts for PPM](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.97.5623&rep=rep1&type=pdf).\nProceedings of the IEEE Data Compression Conference (DCC04), pp. 409-418, 2004.\nLong matching strings are encoded as in high order ROLZ,\nencoded as an index to a matching context and a length.\n## .1869 chile\n\n[chile](http://mosoi.lumina.ro/index.php/plain/projects/chile) 0.3d-1 is a free,\ncommand line file compressor as C source code by Alexandru Mosoi, May 29, 2006.\nIt uses BWT.  The option -b40000 selects a block size of 40000 KB, which requires about\n785 MB of memory for compression and 240 MB for decompression.  Version 0.3d1\nis identical to version 0.3d except that the maximum block size\nwas increased from 2048 KB to 99999 KB.  For this test the program was compiled for Windows\nusing MinGW 3.4.2 as specified in the Makefile.\n[chile 0.4](http://alexandru.mosoi.googlepages.com/chile-0.4.tar.gz)\n(Jan. 27, 2007)\nintroduces a faster algorithm for building suffix arrays that uses less memory (7N).\nThe option -b=244141 selects the block size in Kb (to split enwik9 in 4 equal parts).\nIt was compiled using MinGW gcc 3.4.5 with options -W -Wall -fomit-frame-pointer -g -O3\nand tested in WinXP Home with 2 GB memory.\n## .1901 bwtdisk\n\n[bwtdisk](http://people.unipmn.it/manzini/bwtdisk/) 0.9.0, is a free, experimental,\nopen source (GPL v3) file compressor by Giovanni Manzini, July 7, 2010.\nIt uses BWT. Its purpose is to test the techniques for low memory BWT\ndescribed in the paper\n[Lightweight Data Indexing and Compression in External Memory](http://people.unipmn.it/manzini/bwtdisk/bwte.pdf) by\nFerrangina, Gagie, and Manzini, Proc. LATIN 2010. The forward BWT computes the suffix\narray in small segments, then makes multiple passes over the BWT output to merge\nthe result. The external disk usage can be further reduced by compressing the input first\nwith zlib or lzma and decompressing the input on each pass.\nThe program is single threaded.\n## .1910 CTXf\n\n[CTXf](http://www.nlo-one.narod.ru/ctx/download.html) 0.75 pre-beta 1\nis a free, closed source command line archiver by Nikita Lesnikov, Sept. 20, 2003.\nIt uses PPM with preprocessing for text, exe and multimedia files.\nThe option -me selects extreme (best) compression.  It uses about 78 MB memory\nin Windows task manager.\n## .1912 M03exp\n\n[m03exp-2005-01-27](http://mij4x.datacompression.jp/?date=20040720) is an\nexperimental, closed source GUI file compressor by mij4x, Jan. 27, 2005.  \nIt uses BWT implementing the M03 algorithm by Michael A Maniscalco. \nwith a maximum block size of 8MB.  (Note on the GUI: to compress\nor decompress, drop a file on the program window.  Right click to\nselect options).\n[m03exp-2005-02-15](http://mij4x.datacompression.jp/junk/m03exp-2005-02-15.zip)\n(Feb. 15, 2005) supports blocks up to 32MB but is otherwise identical.\n,\nMar. 15, 2011, is multithreaded.\nIt automatically detects the number of cores and compressses or decompresses in parallel,\nor the number can be changed with -t.\nHowever, since the program is already faster than disk I/O with one thread, using more\nthreads makes no difference in practice. Speeds shown below are total process times.\nActual times are 17 seconds to compress and 44 to decompress\nwith either -t1 or -t2. Compressed size is the same either way\nand the archives are compatible but not identical.\n## .1930 Stuffit\n\n[Stuffit](http://my.smithmicro.com/stuffit-deluxe-windows.html) 9.0 is a commercial GUI archiver by Allume Systems,\nnow Smith Micro.  This was the current version as of May, 2006.\nNote: their free 30 day trial required registration and a credit card number\nwhich was charged if you forgot to cancel.  The options tested were:\n## .1933 plzma\n\n[plzma_v3b](http://nishi.dreamhosters.com/u/plzma_v3b.rar)\n(\n[discussion](http://encode.su/threads/1297-plzma-codec?p=25335)) is a free, closed source, experimental file compressor for Windows\n(32 and 64 bit versions) by\nEugene Shelwien, Oct. 8, 2011. It uses LZMA (7zip equivalent) with a modified entropy encoder.\n[plzma_v3c](http://nishi.dreamhosters.com/u/plzma_v3c.rar)\nwas released Mar. 19, 2012. Options are\nas follows:\n\n<sup>32</sup> - 1.\n## .1933 crook\n\n[crook](https://github.com/valdmann/crook) v0.1\n[(discussion)](http://encode.su/threads/1504-Crook-a-new-binary-PPM-compressor)\nis a free, open source file compressor by Jüri Valdmann, Mar. 5, 2012.\nIt uses bit-level PPM.\nBecause it predicts bits rather than bytes, there is no\nescape modeling. This is like DMC in that each bit-level context\nis mapped to a next-bit prediction and a\ncount (equvalent to two counts of zeros and ones). But \nunlike DMC, it avoids the problem of duplicate states representing\nthe same contexts, which would dilute the statistics and waste memory.\n## .1947 lzturbo\n\n[lzturbo 0.01](http://www.consultant-berater.de/lzturbo-0.0.1.tar.gz) is\na free, experimental, closed source file compressor by Hamid Bouzidi, Aug. 15, 2007.\nThere is some controversy over the origin of the source code.\n[Discussion](http://encode.su/threads/1695-LZTURBO).\n[Discussion](http://encode.su/threads/1722-lzturbo-is-an-ILLEGAL-tornado-clone).\n[lzturbo 0.9](http://consultant-berater.de/lzturbo/)\nwas released Feb. 25, 2008.  Decompression memory peaks at 79 MB.\n[lzturbo 0.94](http://lzturbo.web.officelive.com/documents/lzturbo.zip)\nwas released Apr. 11, 2009. The option -b59 selects method 5, compression level 9\nfor maximum compression. -b100 selects a block size of 100 MB for independent\ncompression in separate threads. The default is 32 MB. -p0 forces the\ncompressor to run on one core. By default the program runs on on all cores, but\nthis causes the program to run out of memory with -59 because each thread uses 1450 MB.\nDecompression ran on 2 cores with a process time of 20 seconds per core\nand wall time of 28 seconds using about 300 MB memory. Faster modes tested\nbelow are run on 2 cores with average process time per core shown.\n[lzturbo](https://sites.google.com/site/powturbo/home) 1.1, Apr. 29, 2013,\nruns only on 64 bit Windows and 64 bit Linux.\nThe Linux version was tested under Ubuntu (note 48) using the non-static\n(smaller) executable. The 2 digit options -11...-49 select the compression\nmethod and level. The first digit can be 1..4 with higher numbers\ncompressing better. The second digit can be 0, 1, 2, or 9 with higher\nnumbers compressing slower without affecting decompression speed.\nThe program gave an error during compression with -40, -41, -42.\n[lzturbo 1.2](https://sites.google.com/site/powturbo/)\nwas released Aug. 7, 2014 with updates on Aug. 10 and 11, 2014,\nwith compression ratio and decompression speed improvements.\nMethods -30, -31, -32, -39 use ANS (Asymmetric numeric system) encoding instead\nof arithmetic coding with SSE/AVX code selected at run tim. The updates fixed an\n\"illegal instruction\" error during compression in these modes on the test machine\nand some other processors. The other modes were tested on the Aug. 7 release.\nOptions are like v1.1.\n## .1956 enc\n\n[enc](http://www.compression.ru/so/) 0.15 is an experimental,\nclosed source command line archiver by Serge Osnach, Feb. 14, 2003.  It uses PPM and CM (in PaQ mode).\nIt tries up to 5 different compression\nmethods (depending on options) and chooses the best one.  The methods are (\"a\" means \"add to archive\"):\n\nMethods ae and ab with options -o8 -d256 were found to give the best compression on enwik7 (first 10<sup>7</sup>\nbytes).  These methods discard the model when the memory limit is reached, and this was observed to happen\n(in task manager), so these options should hold for larger files.  However with -d127 (necessary to decompress),\nmethod aq gives the best compression.\n## .1966 comprolz\n\n[comprolz 0.1.0](http://comprox.googlecode.com/files/comprolz-0.1.0.tar.gz)\n[(discussion)](http://encode.su/threads/1615-A-nooblike-ROLZ-compressor?p=30726#post30726)\nis a free, open source, experimental file compressor by\nZhang Li, Oct. 7, 2012. It uses ROLZ. The option -b256 selects the maximum\nblock size. During compression it uses 60-65% of two cores. Decompression\nuses one core.\n[comprolz 0.2.0](http://comprox.googlecode.com/files/comprolz-0.2.0.tar.gz) was released Oct. 16, 2012. It includes the -f\noption to select flexible parsing. It is slower but compresses better.\n[comprolz 0.10.0](http://comprox.googlecode.com/files/comprox-0.10.0.tar.gz)\n[(discussion)](http://encode.su/threads/1623-comprox-comprolz-updates?p=31518&viewfull=1#post31518) was released Nov. 25, 2012. It includes a dictionary\nderived from the first 10 MB of enwik8. To test, it was compiled\nas suggested in the documents using gcc 4.7.0 with options\n\"-O3 -fomit-frame-pointer -mno-ms-bitfields\". Source code is shared\nwith comprox 0.10.0. The executable, packed with UPX, is smaller.\n[comprolz 0.11.0](http://comprox.googlecode.com/files/comprox-0.11.0.tar.gz) was released Dec. 17, 2012.\nThe program builds a dictionary from the input instead of using\na static dictionary. 32 bit executables\nare included for Windows and Linux. The Windows version was tested.\n[comprolz 0.11.0-bugfix1](http://comprox.googlecode.com/files/comprox-0.11.0-bugfix1.tar.gz), Dec. 18, 2012,\nfixes a bug that caused poor compression.\n## .1971 sbc\n\n[sbc](http://sbcarchiver.netfirms.com/) 0.970r2\nis a free, closed source command line archiver and file encryptor\nby Sami, June 27 2005.  Compression options suggest it uses BWT.\nThe -m3 option selects maximum compression, requiring 32 MB memory \n(-m1 is minimum).  The -b63 option\nselects maximum block size (32 MB, requiring 192 MB additional memory).\n-ad disables adaptive block size reduction\nfor homogeneous data.  SBC runs faster with smaller block sizes and minimum\ncompression as shown:\n## .1973 xz\n\n[xz](http://tukaani.org/xz/) 5.0.1 is a free, open source file compressor,\nJan. 29, 2011. xz specifies a container format written by Lasse Collin. It uses\nthe public domain LZMA2 compressed format from 7zip by Igor Pavlov. There are\nversions for most operating systems including Windows and Linux. The Windows\nversion was tested. The option -9 specifies maximum compression and\nmemory. The default is -6. The option -e (extreme) specifies better compression\nat a cost in compression (but not decompression) time.\n[xz 5.2.1](http://tukaani.org/xz/xz-5.2.1-windows.zip)\nwas released Feb. 26, 2015.\n## .1984 WinRAR\n\n[WinRAR](http://www.rarlab.com/) 3.60 beta 3 is a commercial (free trial)\nWindows GUI and command line archiver by Eugene Roshal, May 8, 2006.  \nIt produces rar and zip archives \nand decompresses many other formats.  It also encrypts and performs other functions.\nThe best compression mode uses PPM (actually ppmd var. I, an earlier version of ppmd J)\nwith optimizations for text and other\nformats (exe, wav, bmp).  The -mc7:128t+ option says to use PPM order 7,\n128 MB memory (maximum) and force text preprocessing.  The -sfxWinCon.sfx\noption says to produce a self extracting console executable\n(adding 79,360 bytes).\n## .1986 quark\n\n[quark v0.95r beta](http://www.maximumcompression.com/quark095r.rar) is a free,\nclosed source command line file compressor by Frederic Bautista, Mar. 10, 2006.\nIt uses LZ.  It is characterized by high compression and fast decompression.\nThe -m1 option selects relative mode compression, which is normally best, but slowest.  The\n-d25 option selects a dictionary size of 2<sup>25</sup> which is the largest that will\nrun without thrashing with 1 GB RAM.  The -l8 option selects the search depth.\nHigher values normally improve compression (up to -l13, default -l4), but -l8 was the highest\npractical value for reasonable compression speed (7.5 hours).  Also, larger values were \nfound to hurt compression on enwik5.\nCompression time increases approximately exponentially with the -l value.\nThe compression speed with -l13 is 6,100,000 ns/byte.\n## .1994 lzip\n\n[plzip](http://www.nongnu.org/lzip/plzip.html) is a free, open source\nfile compressor by Antonio Diaz Diaz, Feb. 16, 2010. It is \"parallel lzip\", compatible\nwith [lzip](http://www.nongnu.org/lzip/lzip.html), but multi-threaded for\nparallel execution. It uses LZMA (LZ77 with arithmetic coding). The -9 option\nselects maximum compression. It has a command line interface similar to gzip.\nWhen it compresses, it removes the original file and adds a .lz extension.\n[Windows port](http://encode.dreamhosters.com/showthread.php?t=570) by\nChristian Schnaader on May 2, 2010 was tested. On my test computer (2 core T3200, 2 GHz),\ncompression showed 180% CPU and decompression showed 117%.\n[plzip 1.5](http://encode.su/threads/570-NEW-plzip-a-massively-parallel-OPEN-SOURCE-compressor-based-on-LZMA?p=48278&viewfull=1#post48278)\nwas released June 2, 2016. I tested the 64 bit Windows compile in Linux.\n## .1995 comprox\n\n[comprox_sa](http://code.google.com/p/comprox/) 20110927\n([discussion](http://encode.su/threads/1368-Some-of-my-toy-compressors)) is a free,\nexperimental, open source file compressor by Zhang Li, Sept. 27, 2011. It uses LZSS\n(in 4 MB blocks) followed by arithmetic coding. The program takes no arguments. It uses\n60 MB memory for compression and 6 MB for decompression. It runs in both Windows and\nLinux. Only the Windows version was tested.\n[comprox 0.6.0](https://comprox.googlecode.com/files/comprox-0.6.0.tar.gz) was released Aug. 24, 2012. It uses static 4K dictionary\nencoding followed by LZ77 and arithmetic coding. It was released as open\nsource (3 clause BSD) C code only. For testing, it was compiled using g++ 4.6.1\nas \"gcc -O3 *.c\" under 32 bit Windows. The option e200 means to use a 200 MiB\nblock size. The default is e16. Larger blocks improve compression but use\nmore memory. The program crashed with e250 or larger.\n[comprox 0.7.0](https://comprox.googlecode.com/files/comprox-0.7.0.tar.gz)\n[(discussion)](http://encode.su/threads/1378-comprox-0-1?p=30333&viewfull=1#post30333)\nwas released Sept. 10, 2012. It includes multi-threaded\ncompression and other improvements. It includes a static English\ndictionary with about 3000 common words.\nIt was tested in 64 bit Linux compiled\nwith \"gcc -O3 *.c -lpthread\" and in 32 bit Windows compiled with\n\"gcc -O3 *.c -lpthread -Wl,--stack,8000000\".\n[comprox](http://code.google.com/p/comprox/) v0.8.0 was\nreleased Sept. 26, 2012 with better compression. The Linux version\nwas compiled with \"gcc -O3 -march=native *.c -lpthread\". The Windows\nversion was compiled as before.\n[comprox 0.9.0](http://comprox.googlecode.com/files/comprox-0.9.0.tar.gz) was released Oct. 16, 2012. The -b option sets the\nblock size in MB. Default is -b16. -m sets number of matches to\ncheck. Default is -m40. -f selects flexible parsing. To test, the\nprogram was compiled \"gcc -O3 -march=native -s *.c\" as above.\n[comprox 0.10.0](http://comprox.googlecode.com/files/comprox-0.10.0.tar.gz)\n[(discussion)](http://encode.su/threads/1623-comprox-comprolz-updates?p=31518&viewfull=1#post31518) was released Nov. 25, 2012. It includes a dictionary\nderived from the first 10 MB of enwik8. To test, it was compiled\nas suggested in the documents using gcc 4.7.0 with options\n\"-O3 -fomit-frame-pointer -mno-ms-bitfields\". Source code is shared\nwith comprolz 0.10.0. The executable, packed with UPX, is smaller.\n[comprox 0.11.0](http://comprox.googlecode.com/files/comprox-0.11.0.tar.gz)\nwas released Dec. 17, 2012. It builds a dictionary from the input\nrather than use a static dictionary. Executables are included for\n32 bit Windows and Linux. These compressed smaller than the source code.\nThe compressor crashed with -b250 (250 MB block size)\non enwik9, but -b200 worked. -m100 selects the match search limit\n(default -m40). -f selects flexible parsing. Using large -m makes\ncompression time nonlinear, i.e. increasing from 75s to 2115s from\nenwik8 to enwik9.\n[comprox 0.11.0-bugfix1](http://comprox.googlecode.com/files/comprox-0.11.0-bugfix1.tar.gz), Dec. 18, 2012,\nfixes a bug that caused poor compression.\n## .2018 bssc\n\n[bssc](http://ssergeo.narod.ru/bssc.htm) 0.95a is a free command line file compressor\nby Sergeo Sizikov, 2005.\nIt uses BWT.  The -m16383 option selects the maximum block size of 16383 KB (uses 140 MB memory).\n## .2024 lzham\n\n[lzham](https://code.google.com/p/lzham/) alpha 2 is a free,\nopen source (MIT license) file compressor and library by Richard Geldreich Jr.,\nAug. 21, 2010. LZHAM is short for LZMA-Huffman-Arithmetic-Markov. It is based\non LZMA (7zip) but instead of using arithmetic coding throughout, it uses them\nonly for binary decisions and uses Huffman or\n[Polar](http://www.ezcodesample.com/prefixer/prefixer_article.html)\ncodes for literal and match codes. A Polar code is similar to a Huffman code\nbut is simpler to calculate at a cost of 0.1% in compression. Polar codes\nare calculated as follows:\n\nFor example, if the symbols and their frequencies are A=3, B=2, C=1, then the sum (6) is\nrounded up to 8 and the individual frequencies are rounded down to A=2, B=2, C=1, which\nsums to 5. We then double A=4, which sums to 7. We cannot double B=4 because the sum\nwould exceed 8, so we continue to C. At this point we have A=4, B=2, C=2, which\nsums to 8, and we may assign codes of appropriate lengths such as A=0, B=10, C=11.[tested](https://code.google.com/p/lzham/wiki/statistics)\nby the author. I guessed at memory usage. Each increment of the -d option approximately\ndoubles memory usage.\n[lzhamtest v1.0](https://github.com/richgel999/lzham_codec)\n[(discussion)](http://encode.su/threads/1117-LZHAM?p=42277&viewfull=1#post42277)\nis the test code for the source code release on Jan. 25, 2015.\nTo test on note 48, it was compiled using \"cmake . ; make\" in Ubuntu.\nOption -d29 selects a 512 MB dictionary. -d26 selects 64 MB.\nDefault is -d28 (256 MB). Option -x selects extreme parsing.\n## .2024 flashzip\n\n[flashzip](http://heartofcomp.altervista.org/) 0.1\nis a free, closed source file compressor by Nania Francesco Antonio, Jan. 10, 2008.\nIt uses LZP and arithmetic coding.\n[flashzip 0.93a](http://www.encode.su/forum/attachment.php?attachmentid=385&d=1236636165)\nwas released Mar. 9, 2009.\n[flashzip](http://heartofcomp.altervista.org/FLASHZIP/home.htm) 1.1.2 was released Dec. 12, 2012. It includes a GUI that\ncalls the command line version. The command line version was tested.\nThe compression options were changed to -m0..-m3 and -mx0..-mx3, with\n-mx3 selecting maximum compression. Option -k0..-k7 select ROLZ dictionary\nsize with -k7 using 256 MB for best compression using the most memory.\n-b1024 selects a buffer size of 1024 MB for best compression but using\nthe most memory. There is a -t option for multi-threaading which defaults\nto -t1 to select a single thread. Using more threads makes compression worse.\nThe -e option creates a self extracting archive by appending the compressed\nfile to a copy of flashzip.exe, and therefore does not compress any smaller\nwhen the decompresser is included.\n## .2081 uharc\n\n[uharc 0.6b](ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc06b.zip) is a free (for noncommercial\nuse) closed source command line archiver by Uwe Herklotz, Oct. 1, 2005.\nIn maximum compression mode (-mx) it uses PPM.  In modes -m1 (fastest) to\n-m3 (best) it uses ALZ: LZ77 with arithmetic coding.  -mz uses LZP.\n-md32768 selects maximum dictionary size (uses 50 MB memory, default is -m4096).\nAdditional results for enwik8:\n## .2040 csarc\n\n[csc2](http://www.encode.su/forum/showthread.php?t=337) is a free,\nexperimental, closed source file compressor by Fu Siyuan, Apr. 18, 2009.\nIt uses LZP with order 1 modeling of literals and range coding over a 270\nsize alphabet. The program takes no options. It recognizes whether the input\nfile is compressed, and if so, decompresses it.\n[csc3](http://www.encode.su/forum/showthread.php?t=425) v.2009.08.12\nis a free file\ncompressor with source code in C by Fu Siyuan, Aug. 11, 2009. It uses LZ77. The\noption -m3 selects best and slowest compression\n(range -m1 to -m3, default -m2). -d7 selects the maximum dictionary size\n(range -d1 to -d7, default -d4). -fo turns off EXE and delta filtering\n(default unless detected by file name extension).\nThe decompresser size is based on csc3.exe, which is smaller than csc3compile2.exe,\nbut does not work on some machines. It is smaller than the zipped source code (17,247 bytes).\nTiming is similar for both versions and a version compiled with gcc 4.4 with -O2\n-s -march=pentium4 -fomit-frame-pointer.\n[csc31](http://www.encode.su/forum/attachment.php?attachmentid=1070&d=1253748905)\nwas released Sept. 23, 2009 without source code.\n[Discussion](http://www.encode.su/forum/showthread.php?p=9475#post9475).\n[csc32 a2](http://encode.dreamhosters.com/attachment.php?attachmentid=1293&d=1273375938)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?t=615),\nMay 9, 2010,\nis a rewrite of csc31. The option -m3 selects maximum compression. -d9 selects\nmaximum dictionary size. Memory usage is 528 MB for compression and 330 MB\nfor decompression.\n[csc32 final](http://encode.su/threads/615-CSC3.2-is-on-developing?p=12067#post12067),\nMar. 1, 2011, has 3 compression settings from -m1 (fastest) to -m3 (best) and dictionary sizes\nup to -d512 (512 MB) which get the best compression but use the most memory. Compression requires\nmemory in addition to the dictionary, but decompression does not. Source code is now available.\n[csarc 3.3](https://github.com/fusiyuan2010/CSC)\n[(discussion)](<http://encode.su/threads/2153-CSArc-(CSC)-3-3-A-LZ77-compressor-again>) is a free, open\nsource (public domain) archiver with a LZMA like algorithm with dedupe and\ndictionary preprocessing of text. It was released Mar. 21, 2015.\nOption are compression level -m1 to -m5, dictionary size up to -d1024m (1 GB),\n-t1 to -t8 (number of threads, default 1) and -p1 to -p4 to split large files into\n1 to 4 parts to compress in parallel (default 1). To test, I compiled from source with g++ 4.8.2.\n## .2044 packet\n\n[packet](http://heartofcomp.altervista.org/) 0.01 is a free,\nexperimental file compressor by Nania Francesco Antonio, May 11, 2008.\nIt uses LZP.  It takes no options.\n[packet](http://heartofcomp.altervista.org/PACKET/home.htm) 1.0\n[(discussion)](http://encode.su/threads/1769-Packet-archiver-1-0-released) was released Aug. 4, 2013. Options -m0..-mx9 select compression\nlevel (default -m4). Option -t2 selects 2 threads (default -t1).\n[packet 1.1](http://heartofcomp.altervista.org/PACKET/home.htm)\n[(discussion)](http://encode.su/threads/1769-Packet-archiver-1-0-released?p=35632&viewfull=1#post35632) was released Dec. 7, 2013 for 64 bit Windows. It was tested\nin Ubuntu under wine. Option -m9 (or -mx) selects maximum compression.\nDefault if -m4. -b512 selects maximum buffer size of 512 MB. Default is\n-b64. -h4 selects maximum number of buffers. Default is -h2.\n[packet 1.2](http://heartofcomp.altervista.org/packet1.2.zip) was released July 19, 2015.\n[packet 1.9](http://heartofcomp.altervista.org/packet1.9.zip)\n[(discussion)](http://encode.su/threads/1769-Packet-archiver-1-0-released?p=49552&viewfull=1#post49552) was released Aug. 19, 2016.\nOption -mx selects maximum compression time. -h8 selects 2 GB hash table memory\nfor compression (max is -h7 = 1 GB in 32 bit .exe and -h9 = 4 GB in 64 bit .exe).\n-b5 selects maximum buffer size 512 MB for both compression and decompression.\n-r (recursive) and -s (solid) have no effect for single file compression.\nThe 64 bit version was tested under Ubuntu/Wine.\n## .2088 TarsaLZP\n\n[TarsaLZP Aug 8 2007](http://asembler.republika.pl/bin/TarsaLZP.zip)\nis a free, experimental file compressor with public domain source code (FASM)\nby Piotr Tarsa.\n[Aug 10 2007](http://www.zippyshare.com/v/37878366/TarsaLZPInterim.zip.html)\nversion runs at high priority.  (CAUTION, this will make your computer unusable while running).\n[TarsaLZP 29 Jan 2012](http://encode.su/threads/1478-TarsaLZP?p=28145#post28145) is distributed as Java source and class files. It has a GUI interface.\n[TarsaLZP 18 Nov 2012](http://encode.su/threads/1603-TarsaLZP-yet-again?p=31446&viewfull=1#post31446) takes several options, but defaults were used for testing.\nIt is available as source code in Python, Java, Javascript, and C. The C version\nwas tested by compiling with MinGW gcc 4.7.0 with options \"-O3 -std=c99\" in 32 bit Vista.\n## .2090 GRZipII\n\n[GRZipII](http://magicssoft.ru/?folder=projects&page=GRZipII)\n0.2.4 is a free, open source (LGPL)\ncommand line file compressor by Grebnov Ilya, Feb. 12, 2004.  It uses BWT.\nThe -b8m option selects the maximum block size of 8 MB.\n## .2091 4x4\n\n[4x4](http://haskell.org/bz/)\n [0.2a](http://haskell.org/bz/4x4ver02a.zip)\nis a free, open source file compressor by Bulat Ziganshin,\nJune 2, 2008. It is a wrapper around GRZipII, tornado, and LZMA (7zip),\nand a subset of the FreeARC archiver.\nSource code is included in the FreeARC distribution. The program\nallows arguments to be passed to each compressor, plus 16 preset\noptions. Only the fastest and slowest preset option for each compressor\nwas tested. Options 1-7 are tornado, 8-12 are LZMA, and 1t-4t are GRZipII.\n## .2101 rzm\n\n[rzm 0.06c](http://rapidshare.com/files/96984949/rzm006c.zip.html)\n[(mirror)](http://www.geocities.com/lovepimple_mail/)\nis a free file compressor by Christian Martelock, Mar. 4, 2008.\nIt uses order-1 ROLZ as discussed\n[here](http://www.encode.su/forums/index.php?action=vthread&forum=1&topic=647&page=0).\nIt takes no options.\nMemory usage is advertised as 258 MB for compression and 130 MB for decompression.\nMeasured values (shown) are 180 MB for compression and 104 MB for decompression.\n[rzm 0.07h](http://christian.martelock.googlepages.com/index.htm) was released\nApr. 24, 2008.  Advertised memory usage is unchanged.\n## .2104 pim\n\n[pim](http://www.encode.su/) 2.01 is a free GUI archiver by\nIlia Muraviev, based on PPMd by Dmitry Shkarin, using PPM.  Version 2.01\nwas released June 14, 2007.  It has options to model color images and\n.exe files.  These make no difference on text and were turned off.\nIt was timed with a watch.\n## .2120 CTW\n\n[CTW](http://www.ele.tue.nl/ctw/) 0.1 is a free, command line file compressor with source\ncode by Erik Franken and Marcel Peeters, Nov. 13, 2002.  It uses CTW (context tree weighting),\na type of context-mixing algorithm (with single bit prediction and arithmetic coding) combining\nthe predictions of different order contexts.  Statistics are stored in a suffix tree.\n## .2139 boa\n\n[boa 0.58b](ftp://ftp.elf.stuba.sk/pub/pc/pack/boa058.zip)\nis a free, closed source command line archiver by Ian Sutton, Apr. 2, 1998.\nIt uses PPM.  The -m15 option selects maximum memory, 15 MB.\n## .2144 yzx\n\n[yzx 0.01](http://encode.dreamhosters.com/attachment.php?attachmentid=1281&d=1272928898)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?t=614) is a free,\nexperimental command line archiver by Nania Francesco Antonio, May 3, 2010. It uses \"LZKS\"\ndecribed as an LZ type algorithm. Option -b5 selects maximum memory. Option -m2 selects\nmethod 2 (default is -m1). -c8 selects number of match keys (range -c1 to -c8, default -c3).\nMemory usage is 732 MB for compression and 137 MB for decompression.\n[yzx 0.02](http://encode.dreamhosters.com/attachment.php?attachmentid=1292&d=1273221155),\nMay 7, 2010, corrects a bug in compression.\n[yzx 0.03](http://encode.dreamhosters.com/attachment.php?attachmentid=1305&d=1274441591)\nwas released May 21, 2010. The range of options is -m1..m2, -c1..c5,\n-b1..b6. Memory usage with -m2 -c5 -b6 is 404 MB for compression and\n268 MB for decompression.\n[yzx](http://heartofcomp.altervista.org/YZX/home.htm) 0.11 was released\nJan. 4, 2012. Options -m0..-m9 select compression method (fast..slow).\nOptions -b1..-b8 select ring buffer size (small..large). Options\n-h1..-h6 select search buffer size (small..large). Default is -m2 -b2 -h4.\nThere was not enough memory to test maximum compression (-m9 -b8 -h6) without\nreducing either -b or -h.\n## .2157 zstd\n\n[zstd](https://github.com/Cyan4973/zstd) is a free, open\nsource (BSD) file compressor by Yann Collet, Jan. 25, 2015.\nIt uses LZ77 and finite state entropy encoding.\nIt takes no compression options. To test, it\nwas compiled using the supplied Makefile with gcc 4.8.2 in Linux (note 48)\nand \"make -CC=gcc\" 4.8.1 in Windows (note 26).\n[zstd 0.4.0](https://github.com/Cyan4973/zstd/releases) was released Nov. 29, 2015. It features a high compression mode.\n-f means overwrite output. 20 is the compression level (only -1 to -9\nare documented).\n[zstd 0.5.1](https://github.com/Cyan4973/zstd/releases/tag/v0.5.1)\nwas released Feb. 17, 2016. The decompressor size is the\nsource for zstd_little-0.5.1.tar.gz converted to a zip -9 archive.\n[zstd 0.6.0](https://github.com/Cyan4973/zstd/releases/tag/v0.6.0) was released Apr. 12, 2016. It adds level -22 option and adds\n--ultra to allow more memory usage.\n## .2178 tornado\n\n[tornado 0.1](http://www.haskell.org/bz/tor.rar) is a free, open source file\ncompressor by Bulat Ziganshin, Apr. 16, 2007.  It uses LZ77 with arithmetic coding.\nThe -9 option selects a predefined compression profile for maximum compression.  \nThere are custom options for hash table size, hash chain length, block size, type\nof coder, and an option to force or prohibit cache matching.  Some of these options\nmight give better compression, but were not tested.\n[tornado 0.3](http://www.haskell.org/bz/tornado03.zip) has options -1\nthrough -12.  Each increment approximately doubles compression time and memory usage.\nDecompression time is fast in all cases, but memory usage is approximately 2/3 that\nof compression (for the LZ77 buffer).  -12 caused disk thrashing and was not tested\nfor enwik9.  There are several other options that were not tested.\n[tornado](http://haskell.org/bz/) [0.4a](http://haskell.org/bz/tornado04a.zip)\nwas released June 1, 2008. It includes Windows and Linux versions. There is a small\nversion (tor-small.exe) which does not include some of the advanced options.\nThe advanced options were not tested. Option -12 caused disk thrashing (2 GB memory)\nwhen enwik9 reached 80% compression, so -11 was used instead.\n[tornado 0.6](http://freearc.org/download/research/tornado06.zip),\nMar. 8, 2014, adds optimal parsing. It has 16 compression levels.\nThe default is -5. For\ntesting (note 48) it was compiled from source in Linux with g++ 4.8.1 using\nthe provided build.sh script. Windows and Linux 32 and 64 bit executables are also\nprovided.\n## .2178 LZPXj\n\n[LZPXj](http://sourceforge.net/projects/lzpx/) 1.1d\nis an experimental open source (GPL) command line file compressor by\nIlia Muraviev and Jan Ondrus, May 21, 2006.  The -m3 option selects maximum compression.\nThe -e0 option turns off the exe filter (has no effect on text).  The -r3 and -a0 options\nwere tuned experimentally on enwik7.  -r sets the rescale rate (range 1-5, default 3).\n-a0 turns off the alternate one byte matcher (default -a1 = on).\n## .2179 scmppm\n\n[scmppm](http://www.infor.uva.es/~jadiego/download.html) 0.93.3 is\na GPL open source command line compressor for XML files by James Cheney and\nJoaquín Adiego, Oct. 3, 2005, and using PPMd var. I\ncode by Dmitry Shkarin.  It works by grouping XML data by tag, then compressing\nwith ppmd (similar to XMill).  scmppm is distributed as UNIX source code only.  For this test\nit was compiled and run under WinXP using the latest version of Cygwin, g++, flex, and make as\nof May 24, 2006.  To compile I had to add the line `extern \"C\" int fileno(FILE*);`\nto lex.yy.c.\n## .2185 acb\n\n[acb](http://ctxmodel.net/files/ACB.rar)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?t=540)\nis a shareware archiver for DOS by George Buyanovsky. It achieved some popularity\nin Russia in 1997 after being described in a popular magazine there.\nacb uses a complex variant\nof LZ77 called \"associative coding\". (ACB means \"associative coding by Buyanovsky\").\nHistory is collected in a context sorted ring (like BWT) called a \"funnel of\nanalogies\". A string match is coded by the position of the longest (nearest) match in this\ndata structure. The length is coded dependent on the length of neighboring matches.\nThe result is arithmetic coded. There are 4 versions:\n\nAll versions limit file size to 64 MB but do not limit archive size. To\ntest enwik8, it was divided into 2 equal parts of 50 MB and compressed into\none archive. Archives are compressed in \"solid\" mode.\nenwik9 was divided into 16 equal parts of 62.5 MB each\n(named 01 through 16)\nand compressed to 16 separate archives. The compressor crashed (after 12 hours\nand producing 1474 MB output in 3 files) with\nan illegal interrupt when attempting to compress enwik9 into a single archive.\n## .2186 crushm\n\n[crushm](https://sites.google.com/site/ctxtree/crush-codec)\nis a free file compressor for Windows by Abhilash, July 12, 2013. It uses CM.\nIt takes no options.\n## .2190 PX\n\n[PX](http://www.geocities.com/netstore101/) v1.0 is a free command line\nfile compressor by Ilia Muraviev, Feb. 17, 2006.  It is a context mixing\ncompressor based on PAQ1 with fixed weight models.\n## .2196 DGCA\n\n[DGCA](http://www.emit.jp/dgca/dgca.html) v1.10 is a free, closed source\nGUI archiver, Aug. 8, 2006.  The installer is in Japanese but the program runs\nin several languages including English.  It was tested with default settings\nexcept for producting a self extracting archive.  This adds 189,936 bytes\nto enwik8.\n## .2200 Squeez\n\n[Squeez](http://www.speedproject.de/enu/squeez/index.html) 5.20.4600 is a commercial\n(60 day trial) GUI archiver by SpeedProject, Apr. 11, 2006.\nIt supports 13 different formats, but only\nthe native .sqx (possibly LZ77) format was tested.  The options used were 2.0 format (newest),\n32 MB dictionary (largest, actually uses 365 MB memory), Ultra compression (best),\nand all checkboxes off (including no exe or multimedia compression).  There is a SFX\noption but using UnSqueez to decompress instead gives a smaller size.\n## .2212 fpaq2\n\n[here](fpaq0x.zip).\n## .2217 TinyCM\n\n[TinyCM 0.1](http://encode.su/threads/1620-TinyCM-A-simple-CM-compressor?p=30824#post30824) is a free, open source (GPL v3) file compressor by David Werecat,\nOct. 12, 2012. It uses an order 1-2-3-6 context mixing model. It takes one\noption, a single digit \"level\" which apparently has no effect except to store\nthe value in the first byte of the archive. (I used \"9\"). Memory is the\nsame for compression and decompression. The supplied executables require\nMSVCR110.dll, which I did not have, so I recompiled the source code\nwith g++ 4.6.1 using \"gcc -O3 -march=native -s *.c -I.\" on a 2.0 GHz T3200\nunder 32 bit Vista.\n## .2226 dmc\n\n[dmc](http://www.jjj.de/crs4/dmc.c) is the original DMC\ncompressor written by Gordon V. Cormack in 1987 and described in\n\"Data Compression using Dynamic Markov Modelling\",\nby Gordon Cormack and Nigel Horspool in Computer Journal 30:6 (December 1987).\nThe algorithm is the same as described in [hook](#1998) with the\nlast 2 arguments fixed at \"2 2\".  The dmc argument \"c 1800000000\" means to\ncompress with 1.8 GB memory.  The memory size must also be given for decompression.\nThus, 10 bytes (the size of the argument) was added to the decompresser size\n(source zipped with Info-Zip 2.31 -9).\nBecause dmc compresses and decompresses\nfrom stdin to stdout, it was tested in Linux (Ubuntu\n2.6.15.27-amd64-generic), compiled in gcc 4.0.3 x86-64 as follows:\n## .2230 lza\n\n[lza](http://heartofcomp.altervista.org/) 0.01 is a free\narchiver for 32 bit Windows by Nania Francesco Antonio, May 29, 2014.\nIt uses LZ77 (based apparently on zcm).\nOption -t selects number of threads. Default is -t1. Using a greater\nnumber of threads makes compression worse by splitting the input\namong threads. -h0..-h7 selects\nhash buffer memory 8 MB to 1 GB. Default is -h2 (32 MB). -b0..-b7\nselects LZ buffer memory 8 MB to 1 GB. Default is -b3 (64 MB).\nOption combinations -b6 -h7 or -b7 -h6 or higher run out of memory.\n-m1..-m5 selects compression level (faster..better). Default is -m3.\n[lza 0.10](http://encode.su/threads/1969-LZA-archiver?p=39178&viewfull=1#post39178) was released June 29, 2014. It improves compression and speed\nand adds compression levels -mx1..-mx5 for higher compression.\nA \n[64 bit version](http://encode.su/threads/1969-LZA-archiver?p=39262#post39262) was released July 3, 2014 to support larger memory\noptions.\n[64 bit Windows version](http://encode.su/threads/1969-LZA-archiver?p=40395&viewfull=1#post40395) was released Sept. 9, 2014. The 64 bit version\nallows the hash table option up to -h9 using 4 GB memory. It was tested using\n-h8 (2 GB) and -b7 (1 GB buffer). -t1 selects 1 thread (default). -mx5 selects\nmaximum compression.\n[lza 0.61](http://encode.su/threads/1969-LZA-archiver?p=40877&viewfull=1#post40877) was released Oct. 18, 2014. It is an update to store file dates\nand empty directories. The -t option is removed so it is single threaded only.\n-h and -b have a documented max value of 7 (1 GB memory each).\n[lza 0.62](http://heartofcomp.altervista.org/lza062.zip)\nis a bug fix release, Oct. 20, 2014. Additional options -r (recurse directories),\n-s (solid mode), -v (verbose) used in testing have no effect on compression.\n[lza](http://heartofcomp.altervista.org/) [0.70b](http://heartofcomp.altervista.org/lza070b.zip)\n[(discussion)](http://encode.su/threads/1969-LZA-archiver?p=41431&viewfull=1#post41431)\nwas released Nov. 19, 2014. It uses ANS coding rather than arithmetic coding,\nbased on the public domain [ryg_rans](https://github.com/rygorous/ryg_rans)\ncoder by Fabian Giesen.\nANS extends ABC (asymmetric binary coding) to larger alphabets. ANS coding\ntheory was developed by Jarek Duda.\nMax compression level is increased to -mx9.\n[LZAwin080test](http://encode.su/threads/1969-LZA-archiver?p=42043&viewfull=1#post42043) was released Jan. 10, 2015.\n[lza 0.82b](http://heartofcomp.altervista.org/)\n[(discussion)](http://encode.su/threads/1969-LZA-archiver?p=42913&viewfull=1#post42913) was released Mar. 9, 2015. It is not compatible with\nv0.80. The 64 bit version was tested in Wine. \n## .2241 brotli\n\n[brotli](https://github.com/google/brotli) is a free, open\nsource (Apache license) file compressor by Google. It uses LZ77. It was tested\nby compiling from the Sept. 21, 2015 GitHub commit in the tools subdirectory using\nthe supplied Makefile in Ubuntu Linux with g++ 4.8.4. The -q option\nselects the compression level. The default is -q 11.\n## .2276 szip\n\n[szip](http://www.compressconsult.com/szip/) 1.12a is a free, open source\nfile compressor by Michael Schindler, Mar. 3, 2000. It uses a modified BWT\n(a Schindler transform) which sorts using a truncated string comparison to speed\nthe transform on highly redundant data. The algorithm is protected by\n[patent 6,199,064](http://www.patentstorm.us/patents/6199064/claims.html) in the U.S.\nuntil Nov. 19, 2017. The first version of szip was released on June 2, 1997.\n## .2282 balz\n\n[1.06](http://encode.su/balz/balz106.zip), May 9, 2008, has two compression\noptions, e for normal and ex for better but slower compression.  Both options use\n67 MB for compression and 48 MB for decompression.\n[1.07](http://encode.su/balz/balz107.zip)\nwas released May 14, 2008.  It uses 132 MB for compression and 95 MB for decompression.\n[1.08](http://encode.su/balz/balz108.zip)\nwas released May 20, 2008.  It uses 200 MB for compression and 126 MB for decompression.\nOnly mode ex was tested.\n[1.09](http://encode.su/balz/balz109.zip)\nwas released May 21, 2008.  It uses 128 MB for decompression.  Only mode ex was tested.\n[1.12](http://encode.su/balz/balz112.zip)\nwas released June 3, 2008.  It uses 123 MB for decompression.\n[1.13](http://encode.su/balz/balz113.zip)\nwas released June 11, 2008. It uses 127 MB for decompression.\n[balz](http://balz.sourceforge.net/) 1.15 was released as open source\non July 8, 2008. It uses 67 MB for compression and 49 MB for decompression.\n[balz 1.20](http://sourceforge.net/projects/balz/)\n[(discussion)](http://encode.su/threads/109-BALZ-An-Open-Source-ROLZ-based-compressor?p=42878&viewfull=1#post42878) was released Mar. 5, 2015. It is compatible with 1.15\nbut faster with less compression.\n## .2291 lzpm\n\n[lzpm 0.02](http://www.encode.su/downloads/lzpm002.zip) is a free, closed source\nfile compressor by Ilia Muraviev, Apr. 19, 2007.  It uses LZ77.  It takes no options.\n[lzpm 0.03](http://www.encode.su/downloads/lzpm003.zip), Apr. 28, 2007,\nuses more memory for compression (181 MB), but still uses 20 MB for decompression.\n[lzpm 0.04](http://www.encode.su/downloads/lzpm004.zip), May 4, 2007,\nuses ROLZ.  Memory usage is 83 MB for compression and 20 MB for decompression.\nThe new design uses circular hash chains for better speed on binary files,\nbut a little slower for text.\n[lzpm 0.06](http://www.encode.su/downloads/lzpm006.zip), May 19, 2007,\nimproves compression over 0.04 with the same memory usage.\n[lzpm 0.07](http://www.encode.su/downloads/lzpm007.zip), Aug. 6, 2007,\nand later versions use 280 MB for compression and 20 MB for decompression.\n[lzpm 0.08](http://www.encode.su/downloads/lzpm008.zip), Aug. 8, 2007.\n[lzpm 0.09](http://www.encode.su/downloads/lzpm009.zip), Aug. 15, 2007.\n[lzpm 0.10](http://www.encode.su/downloads/lzpm010.zip), Aug. 23, 2007.\n[lzpm](http://www.encode.su/lzpm/index.htm)\n[0.11](http://www.encode.su/lzpm/lzpm011.zip), Sept. 5, 2007,\ntakes the command 1..9 to choose the compression level (fastest...maximum).\n1 uses greedy parsing.  2..8 use 1..7 byte lookahead.  9 uses unbounded lookahead.\nAll modes use 723 MB for compression and 77 MB for decompression.\n[lzpmlite 0.11](http://lzpm.encode.su/lzpm011lite.zip), Sept. 13, 2007,\nis a \"lite\" version of lzpm, using about half as much memory and twice as fast.\nOptions range from 1..9\nwith 1 being fastest and 9 for best compression.  (3 is a good compromise).\nAll modes use 362 MB for compression and 39 MB for decompression.\n[lzpm 0.13](http://lzpm.encode.su/lzpm013.zip) was released\nDec. 1, 2007.\n[lzpm 0.14](http://lzpm.encode.su/lzpm014.zip) was released\nJan. 1, 2008.  It uses 40 MB for decompression.\n[lzpm 0.15](http://lzpm.encode.su/lzpm015.zip) was released\nJan. 16, 2008.  It uses 40 MB for decompression.\n## .2299 qazar\n\n[qazar](http://qarc.narod.ru/) 0.0pre5 is a free, closed source\ncommand line file compressor by\nDenis Kyznetsov, Jan. 31, 2006.  It uses LZP, an LZ77 variant where\nthe decompresser dynamically computes the same sequence of context\nmatches as the compressor.  The compressor uses a single bit flag\nto indicate if the pointer computed by the decompresser should be\nfollowed.  In qazar, the output symbols are arithmetic coded.\n## .2317 KuaiZip\n\n[KuaiZip](http://www.kuaizip.com/en/) 2.3.2 is a free GUI archiver\nfor Windows, Sept. 9, 2011. It uses a proprietary compression algorithm, probably\nLZMA. It takes no compression options. On the test machine\n(dual core T3200), compression uses 1.5\nthreads (75% CPU). Decompression uses one thread. Times are reported by the application.\n## .2328 qc\n\n[qc](http://qarc.narod.ru/) 0.050 is a free, closed source,\ncommand line file compressor by Denis Kyznetsov, Sept. 17, 2006.\nThe -8 option selects maximum compression (slowest and most memory).\n## .2334 ppms\n\n[ppmonstr](#1570) above.\n## .2356 dzo\n\n[dzo](http://essensolabs.com/) is a commercial GUI deduplicator and archiver\nfor Windows by Essenso Labs. A beta version (32 day free trial)\ndated Sept. 15, 2011 was tested. The trial version will compress either a single file\nor a folder. It first finds duplicate files or regions within files and produces an\nintermediate temporary file (*file.dp*) that removes the duplicates. Then it compresses\nthe temporary file using LZMA (7zip) to *file.dzo* and removes it.\nThe original files are not removed. Decompression restores a single file\nto *(dzo)file* or *folder(dzo)*, again through a temporary .dp file. Both\ncommands are activated by right-clicking on the file or folder to compress or the .dzo\nfile to decompress and selecting the command from the context menu. Times are as reported\nby the appliation. LZMA compression is multi-threaded.\n## .2428 comprox_ba\n\n[comprox_ba](http://code.google.com/p/comprox/) 20110927\n([discussion](http://encode.su/threads/1368-Some-of-my-toy-compressors)) is a free,\nexperimental, open source file compressor by Zhang Li, Sept. 27, 2011. It uses\n[BWTS](http://mattmahoney.net/dc/dce.html#Section_559)\n(BWT Scottified) with 4 MB blocks, followed by MTF (move to front), RLEZ (run length encoding\nof zeros) and arithmetic coding. BWTS is a bijective variant of BWT developed by David\nA. Scott in which the starting index is not stored. In BWTS, the input is factored into\na sequence of lexicographically non-decreasing Lyndon words, which are then context-sorted\nseparately. The starting indexes for the inverse BWTS are the beginnings of each word.\n## .2453 turtle\n\n[turtle 0.01](turtle.rar)\nis a free, experimental, closed source file compressor by\nNania Francesco Antonio, June 1, 2007.  It uses PPM.  It takes no options.\n[turtle 0.02](turtle002.rar)\nwas released June 2, 2007.  Compression is identical.\n[turtle 0.03](turtle003.rar)\nwas released June 5, 2007.  It is faster and improves compression slightly.\nThe file name is stored in the compressed file.\n[turtle 0.04](turtle004.rar)\nwas released June 8, 2007.  It recognizes several different file types.\n[turtle 0.05](turtle005.rar)\nwas released June 12, 2007.  It improves compression at the cost of time and memory.\n[turtle 0.07](turtle007.rar)\nwas released June 23, 2007.  It includes a model for audio files.\n[WinTurtle](http://heartofcomp.altervista.org/) 1.60 was\nreleased Jan. 1, 2008.\n## .2466 diz\n\n[diz](http://encode.su/threads/1583-diz-py-release?p=30027&viewfull=1#post30027)\nis a free, experimental, open source (GPL) file compressor by\nRoger Flores, Aug. 3, 2012. It is a PPMC based compressor written in Python.\nIt is distributed as source code only.\nThe program was tested as recommended by running in [pypy](http://pypy.org/)\nversion 1.9.\n## .2508 cabarc\n\n[cabarc](http://support.microsoft.com/default.aspx?scid=KB;en-us;310618) 1.00.0601\nis a command line archiver available for free download by Microsoft, Mar. 18, 1997\n(SDK released Jan. 8, 2002).  It produces .cab files, which are often used to distribute Microsoft software.\nIt is designed for very fast decompression.\nIt uses LZX, a variant of LZ77 with fixed Huffman coding, but with shorter symbols reserved for the\nthree most recent matches.  The option -m lzx:21 selects a window size of 2<sup>21</sup>\n(2 MB) for maximum compression.\nThere is a separate extraction program, \"extract\".  The actual (global) decompression time of 32 sec. includes\n15 sec. of CPU (process) time and the rest for disk I/O.\n## .2530 sr3\n\n[sr2](sr2.zip) is a free,\nopen source (GPL) file compressor by Matt Mahoney, Aug. 3, 2007.  It uses\nsymbol ranking.  It takes no options.  There are separate programs for\ncompression and decompression.\n[sr3](http://www.winturtle.netsons.org/)\n[(mirror)](sr3.zip)\nis a modification by\nNania Francesco Antonio, Oct. 28, 2007.  The context table size is increased\nfrom 4 MB to 64 MB, which effectively increases the context from order-4 to\norder-5.  This helps compression on larger files, but makes it worse for some\nsmaller files.  The program also detects file type.  For .bmp files, the order is\ndecreased.  For .wav files, the input is split into separate 1 byte wide streams\nfor each audio sample.  There is no separate compressor and decompresser program.\n## .2540 bzip2\n\n[bzip2](http://www.bzip.org/) 1.0.2 is an open source command line\nsingle file compressor by Julian Seward, released Dec. 30, 2001.\nIt uses BWT.  The -9 option selects maximum compression.\n[bzip2 1.0.3](http://gnuwin32.sourceforge.net/packages/bzip2.htm)\n(May 22, 2005) compresses very slightly larger but is faster, as shown by\nthe following table.  The decompresser\nsize is based on zipped bunzip2.exe.  This is smaller than the source\n(724,919 bytes as a zip download).\n## .2542 RH5\n\n[RH](http://encode.su/threads/1880-RH) is a free, experimental\nfile compressor by Nauful, Feb. 17, 2014. There are two versions,\nRH and RH2. RH uses order 3 ROLZ and Huffman coding, using 8 MB\nmemory. RH2 has 3 compression levels using 64 MB memory. Level c1\nuses LZP. c2 uses order 1 ROLZ with limited search. c3 uses full search.\nA literal is coded with 1 bit plus the value. A match is coded with\n1 bit to signal a match, 8 bits for the length, and 12 bits for the\nindex into the ROLZ table.\n[RH2 20Feb2014](http://encode.su/threads/1880-RH?p=36891&viewfull=1#post36891), released Feb. 27, 2014, has 5 compression levels\nc1..c5.\n[RH4_x64](http://encode.su/threads/1891-RH4-Solid-multifile-compressor?p=37239&viewfull=1#post37239), Mar. 22, 2014 is an archiver with file-level deduplication\nand compression improvements. It has 6 compression levels. There are\nseveral earlier versions without version numbers that were not tested.\n[RH5](http://encode.su/threads/1891-RH4-Solid-multifile-compressor?p=41262#post41262) was released Nov. 11, 2014. The 64 bit Windows version was tested in\nUbuntu/Wine. It has options c1..c6 to select the compression level\n(default c2), default -window:23 to select 2^23 byte window size. Larger\nwindows compress better with more memory up to 27, but above that has\nno effect. Options -hash:13 and -table:12 select the default hash\ntable sizes and index table sizes. Higher or lower values compress worse.\n-skip-checksums is not used because it has no effect on compression.\nHowever it skips a check for duplicate files when creating an archive\nfrom a directory. It would make compression worse in that case.\n## .2545 RangeCoderC\n\n[RangeCoderC v1.2](http://encode.su/attachment.php?attachmentid=1733&d=1322071572)\n[(discussion)](http://encode.su/threads/1411-Simple-Variable-Order-Rangecoder?p=27267#post27267)\nis a free, experimental open source file compressor by David Catt,\nNov. 23, 2011. The option 26 selects a simple bitwise order 26 model.\nAn order n model requires 16*2<sup>n</sup> bytes of memory.\n[RangeCoderC v1.3](http://encode.su/attachment.php?attachmentid=1734&d=1322237740),\nNov. 25, 2011, has 3 versions. The standard version is compatible with v1.2 but\nuses half as much memory. The \"double\" version uses a main model to select\namong several sub-models to improve compression at a cost in speed and memory.\nThere is also an \"indirect\" version that was not tested because there was no\n32 bit Windows version.\n[RangeCoderC v1.4](http://encode.su/attachment.php?attachmentid=1739&d=1322498413) was released Nov. 28, 2011. It has 4 versions: standard,\ndouble, indirect, and a new version, hashed, which computes a hashed context\nand gives the best compression.\n[RangeCoderC v1.5](http://encode.su/attachment.php?attachmentid=1741&d=1322594561) was released Nov. 29, 2011. It combines the 4\nmodels from v1.4 into one program and includes the model type in the archive header.\nOption c3 selects the hashed model. It gives the same size as v1.4. The other\nmodels were not tested.\n[RangeCoderC v1.6](http://encode.su/attachment.php?attachmentid=1746&d=1322766090) was released Dec. 1, 2011. It has 6 compression modes\nselected by options c0 through c5 as follows:\n[RangeCoderC v1.7 alpha](http://encode.su/attachment.php?attachmentid=1754&d=1323090055), Dec. 5, 2011, fixes the bug in c1 mode in v1.6.\nThe other 5 modes are presumably the same and were not tested.\nIt is a pre-release of version 1.7, released without source code.\n[RangeCoderC v1.7](http://encode.su/attachment.php?attachmentid=1759&d=1323436295), Dec. 9, 2011, adds two new compression modes:\n[RangeCoderC v1.8](http://encode.su/attachment.php?attachmentid=1767&d=1323796993), Dec. 13, 2011, removes two obsolete modes and adds\none mode: \"The Bitwise Adaptive Model uses probabilities instead of counts,\nwhich are adjusted nonlinearly for better compression on changing data.\nThe learning speed of the model is derived from the model order.\" The\nmodes are:\n## .2561 quad\n\n[quad](http://quad.sourceforge.net/) is a free file compressor by\nIlia Muraviev.  Only the latest version (now open source) is supported, so only that version\nappears in the main table.\n[quad 1.11HASH2](http://www.encode.su/downloads/quad-1.11HASH2.zip)\n(Apr. 5, 2007, experimental, no source code) produces the same size archives, but uses\na hash table for faster compression.\n## .2572 WinACE\n\n[WinACE](http://www.winace.com/) 2.61 is a shareware GUI/command line archiver,\nMar. 8, 2006.  It compresses in ACE and ZIP formats and decompresses\nmany others.  ACE decompresses much faster than it compresses,\nsuggesting it is based on LZ77.  The option -m5 selects maximum compression.\n-d4096 select maximum dictionary size of 4MB (default is -1024 = 1MB).\n-sfx creates a self extracting archive, which adds less space than the\nprogram itself.\n## .2589 lzsr\n\n[lzsr](http://heartofcomp.altervista.org/) 0.01 is a free file compressor\nfor Windows by Nania Francesco Antonio, Oct. 1, 2011.\nIt is described as using a \"fusion of LZ77-LZP and SR\"\nand arithmetic coding. It takes no options.\n## .2595 zling\n\n[zling](https://github.com/richox/zling)\n[(discussion)](http://encode.su/threads/1776-lightweight-ROLZ-compression-utility?p=35041&viewfull=1#post35041)\nis a free, open source (BSD\nlicense) file compressor by Zhang Li, Nov. 1, 2013. It uses order 1 ROLZ,\nbased on the order 3 ROLZ compressor zlite. It takes no options.\nThe compressor is C source code only. To test, it was compiled with\ngcc 4.8.0 -O3 for 32 bit Windows.\n[zling](https://github.com/richox/zling/archive/cristmas.zip)\n[(discussion)](http://encode.su/threads/1848-zling-updates?p=35918&viewfull=1#post35918) was\nupdated Dec. 25, 2013. It was tested in Ubuntu with gcc 4.8.1 and\nBoost_1_55_0 using the supplied Makefile.\n[zling 20140121](https://github.com/richox/zling/tree/20140121)\n[(discussion)](http://encode.su/threads/1848-zling-updates?p=36279&viewfull=1#post36279),\nJan. 21, 2014,\nhas some optimizations, and removes Boost. It was tested by compiling\nwith g++ 4.8.1 -O3 in Windows and with the supplied Makefile in Linux.\n[libzling](https://github.com/richox/libzling/tree/20140219) 20140219, Feb. 19, 2014,\nseparates the program into compression API and\na simple demo program. It was tested by building the demo using cmake\nunder Linux as recommended in the readme file.\n[libzling 20140324](https://github.com/richox/libzling/tree/20140324) was released Mar. 24, 2014. The demo\nprogram has 5 compression levels.\n[libzling 20140414](https://github.com/richox/libzling/tree/20140414) was released Apr. 14, 2014. It is\nfaster with better compression.\n[libzling 20140430-bugfix](https://github.com/richox/libzling/tree/20140430-bugfix)\n[(discussion)](http://encode.su/threads/1938-libzling-updates-20140430?p=37839&viewfull=1#post37839) was released May 4, 2014.\n[libzling 20160107](https://github.com/richox/libzling/tree/20160105) was released Jan. 5, 2016 and updated Jan. 7, 2016.\n## .2625 xpv5\n\n[xpv5](https://sites.google.com/site/ctxtree/xpv5)\nis a free Windows command line file compressor\nby Abhilash Anand, Oct. 20, 2011. It is described as using\nROLZ with an order 1 back end. It has 3 compression levels:\nc0, c1, c2. All levels use 9 MB memory for compression or\ndecompression. It is single threaded.\n## .2660 sr3c\n\n[sr3c 1.0](sr3c-1.0.zip) is a free,\nopen source (MIT license) file compressor and library by Kenneth Oksanen,\nreleased Nov. 27, 2008. It uses symbol ranking, based on ideas from SR3, but\ncompletely rewritten in C. The distribution contains a portable compression\nengine and source code for drivers for UNIX/Linux. To test, I wrote a simple driver\nfor Windows (sr3cw) and compiled it using gcc 3.4.5 -O3 -fomit-frame-pointer -march=pentiumpro\n-s and included sr3cw.exe in the distribution. The driver takes no options.\n## .2665 lzc\n\n[lzc](lzc.zip) v0.01 \nis a free, closed source file comprssor by\nNania Francesco Antonio, May 8, 2007.  It uses an LZ77 like algorithm.\nThe option 4 selects the maximum memory mode, 1 GB + 100 MB for compression and\n16 + 100 MB for decompression.  The actual memory usage indicated by Windows\nTask Manager in this mode was 360 MB for compression and 107 MB for decompression.\n[lzc 0.08](http://www.winturtle.netsons.org/) was released Nov. 15, 2007.\nIt improves BMP and WAV compression.\n## .2774 nakamichi\n\n[Nakamichi 2019-Jul-01](https://gist.githubusercontent.com/Sanmayce/33e5047d45cdcb8e7711cd7d3ed52c7f/raw/d72e7126c8fbfde07c0d727dcb353b0267b8196c/Nakamichi_Ryuugan-ditto-1TB.c) is a free, open source file compressor by Georgi Marinov,\nJuly 1, 2019. It uses LZSS. On the test machine it takes 95 days and 302 GB\nof memory to compress and 1.3 seconds and 2 GB to decompress (memory to memory).\n## .2794 crush\n\n[crush 0.01](http://encode.su/threads/1289-CRUSH-0.01-is-here!?p=25143#post25143)\nis a free, experimental file compressor by Ilia Muraviev, May 17, 2011.\nIt uses LZ77. It has 3 compression modes: cf (fast), c (medium), and cx (best).\nCompression in all modes use 143 MB memory, and decompression uses 65 MB.\n[Source code](http://sourceforge.net/projects/crush/)\n(public domain) was released on June 26, 2013. The file format consists of\n64 MiB blocks with a 4 byte header in machine dependent (LSB first for x86) order giving\nthe block size. Literal and match codes are packed LSB first and padded with trailing\n0 bits in the last byte. Codes are as follows:\n<sup>L+4</sup> + P + 1 (33..2<sup>20</sup>). A match is decoded by\ngoing back offset bytes in the output and copying the specified length to the output.\n<sup>21</sup> for strings of length 3 and 2<sup>24</sup>\nfor strings of length 4. The second table is maintained as a linked list.\nThe two rolling context hashes are computed by shifting the current hash 7 or 6\nbits left, respectively, adding the next byte, and chopping off the high bits.\nIt tests the length 3 hash first, then follows the linked list of length 4 hashes\nto find the best match\nfor up to 4, 256, or 4096 locations in the input buffer for compression options\ncf, c, and cx respectively. In addition for option cx, the compressor looks ahead\none byte and codes the current byte as a literal if starting at the next byte\nproduces a better match. A match is better if it is longer with a penalty of\nlog<sub>16</sub> offset plus one for the literal in case of looking ahead.\nThe minimum match length is 3 for offsets less than 64 KiB, otherwise 4.\n<sup>20</sup>\nlinked list pointers are saved in a rotating queue.\nAs a speed optimization for testing matches, the first and last byte\nat the current best match length are tested first, then the rest of the string.\n[crush 1.00](http://sourceforge.net/projects/crush/files/?source=navbar)\n[(discussion)](http://encode.su/threads/1289-CRUSH-0-01-is-here!?p=33701&viewfull=1#post33701)\nwas released July 1, 2013. It increases the window size from\n2<sup>20</sup> to 2<sup>21</sup>, thus increasing the minimum and maximum\nlength of an offset code by 1 bit, i.e. if L is 0 the P is 6 bits (1..64)\nand if L is in 1..15 then P is L + 5 bits (65..2<sup>21</sup>). Also, the\npenalty for coding a match offset is changed to log<sub>8</sub>(offset/16).\n[<sup>16+8</sup> bytes\n= 16 MB block size (default is 4 = 1 MB).\nThe second digit 8 selects the parsing method where 0..2 is greedy, 3..5\nis lazy, and 6..8 is optimal and uses a suffix array (libdivsufsort)\nto find matches, and higher number compress slower but better. Default is 6.\nThe third digit 0..9 (default 2) selects encoding level, where 9 is slowest\nwith best compression.]\n## .2836 xeloz\n\n[xeloz 0.3.5.3](http://encode.su/threads/1979-xeloz?p=40309&viewfull=1#post40309) is a free, open source (MIT license) file compressor\nby xezz, Sept. 7, 2014. It uses LZ77 with the following possible code lengths:\n\nOption c889 selects maximum compression. c indicates a sliding window.\nThe first digit 8 selects 2\n[xeloz 0.3.5.3a](http://encode.su/threads/1979-xeloz?p=40436&viewfull=1#post40436), Sept. 12, 2014,\nfixed a bug that caused version 0.3.5.3 to crash when decompressing files\ncompressed with uppercase option C. The option selects a fixed rather than a\nsliding window for faster compression.\n## .2839 bzp\n\n[bzp](http://heartofcomp.altervista.org/) 0.2 is a free file\narchiver by Nania Francesco Antonio, Sept. 16, 2008. It uses LZP\nand arithmetic coding. It takes no options. Earlier versions (0.0, 0.1)\nwere not tested.\n## .2843 lzwg\n\n[lzwg](https://github.com/grtamayo/LZW-Algorithm)\nis a free, experimental file compressor by Gerald. R. Tamayo, Sept. 15, 2022.\nIt uses LZW with a binary search tree. It resets the dictionary when TABLE_SIZE + 4K\ncodes are transmitted. TABLE_SIZE depends on the option used.\nOption -27 uses 13 x 2^27 bytes (1.7 GB) memory.\n## .2857 ha\n\n[ha 0.98](ftp://garbo.uwasa.fi/pc/arcers/ha098.zip) is a free\ncommand line archiver by Harry Hirvola, Jan. 7, 1993.  A later version,\n0.999b, is available for UNIX with source code and ports to DOS.  It uses order-5 PPMC\n(PPM with fixed escape probabilities for dropping to a lower order context.\nNewer PPM compressors (PPMZ, PPMII) use adaptive escape probabilities given a small context.)\nThe command a2 selects compression method HSC (default is a1 = ASC).  a21 automatically\nchooses the best method.  Time is ns/byte.\n## .2910 ulz\n\n[ulz 0.01](http://encode.dreamhosters.com/attachment.php?attachmentid=1189&d=1265054281)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?p=10967#post10967)\nis a free, experimental file compressor by Ilia Muraviev, Feb. 1, 2010. It uses\nLZ77 with bytewise encoding. The options c1 through c5 select the compression\nlevel from fastest to best. The option does not affect\nmemory usage. All levels use 43 MB for compression and 33 MB for decompression.\n[ulz 0.02](http://encode.dreamhosters.com/attachment.php?attachmentid=1190&d=1265157818)\nadds a new faster mode (c1). Options c2 through c6 are the same as c1 through c5 in ulz 0.01.\n[ulz 0.03](http://encode.su/threads/550-Ultra-fast-LZ?p=48678&viewfull=1#post48678) was released June 26, 2016.\nIt is byte aligned LZ77 similar to LZ4 but with 16 MB blocks and 256 KB window.\nIt has 3 compression levels: cf, c, cu (fast, normal, ultra). Level cu\nuses optimal parsing.\n[ulz 0.06](https://encode.su/threads/550-Ultra-fast-LZ?p=53288&viewfull=1#post53288) was released July 13, 2017. It has 9 compression levels, c1 to c9.\n## .2924 irolz\n\n[irolz](http://ezcodesample.com/rolz/rolz_article.html)\n[source code](http://ezcodesample.com/rolz/irolz.txt) is a free,\nopen source (GPL), experimental file compressor by Andrew Polar, Sept. 26, 2010.\nIt uses ROLZ. The algorithm is like LZ77 except that match offsets are coded\nby counting previous occurrences of the current context in the history buffer\nrather than as pointers. In irolz, the context is order 2. Previous occurrences\nare stored in a linked list with a maximum length of 31 (5 bit offset). Matches\nless than 4 bytes are coded as literals. Symbols (match flags, 5 bit offsets,\n8 bit lengths, and 8 bit literals) are binary arithmetic coded. Lengths and literals\nare coded in an order 2 context model. Match flags and offset counts are modeled\nwithout context.\nEach symbol and context to be predicted is mapped to 2 16-bit predictions, one\nfast adapting (learning rate 1/8) and one slow adapting (rate 1/64). The prediction\nis the average of the two.\n## .2961 lcssr\n\n[symbra](http://de.geocities.com/ocamyd/) 0.2 is a free, open source (GPL)\n[(mirror with .exe)](symbra02.zip)\nfile compressor by Frank Schwellinger, Nov. 29, 2007.  It uses symbol ranking.\nOnly source code (C++) is provided. For the test, the program was compiled\nas indicated in the source comments and tested in Windows XP (32 bit).\nThe option -c4 or -c5 selects order 4 or 5 context.  -m5 turns on suffix\nmatching with maximum buffer size, which greatly slows compression.  -p2 selects\n2 passes, which reorders the alphabet by descending frequency.  The defaults\nare -c4 -m0 -p1.\n[(mirror with .exe)](lcssr02.zip)\nis derived from symbra.  It drops the secondary symbol queue\nand instead uses a variable length context based on the length of the\nlongest match as with LZ77/LZP.  The option -b7 selects a 1152 MB buffer\nfor finding context matches.  \n## .2984 zlite\n\n[zlite](https://github.com/richox/zlite) is an open source file compressor by\nZhang Li, Aug. 20, 2013. It uses ROLZ. It was released as C source code only. To test, it\nwas compiled with MinGW gcc 4.8.0. with option -O3. zlite takes no\noptions.\n## .3062 lazy\n\n[lazy v1.00](http://mattmahoney.net/dc/lazy100.zip)\nis a free, open source file compressor by Matt Mahoney, Oct. 10, 2012.\nIt uses LZ77. It has 5 compression levels from 1 to 5. Higher levels\nare slower and use more memory to compress. However decompression speed\ndoes not change and always uses 16 MB.\n<sup>24</sup>-1 and length\n4 to 2<sup>24</sup>-1. Literals are coded as 00,N,L[N], where N is the number\nof literals to follow coded in marked binary. A marked binary number\ndiscards the leading 1, then precedes each bit by a 1 and marks the\nend with a 0 bit. For example, 5=101 would be coded as 1,0,1,1,0.\nMatches are coded as 5 bits to indicate the number of offset bits\n(where the first 2 bits are not 00) in the range 0..23, then the match\nlength as a marked binary number except for the last 2 bits, then the\nlow 2 bits of the match length are coded directly,\nand then 0 to 23 bits of the offset without the leading 1 bit.\n<sup>19</sup> buckets of 2<sup>level</sup> (2..32) pointers each,\nindexed by an order 4 context hash, maintains pointers for finding matches.\nThe longest match of length at least 4 is coded, except that if the offset\nis over 64K and the last symbol is a match, then the minimum length is 5.\n## .3085 zhuff\n\n[zhuff](http://phantasie.tonempire.net/pc-compression-f2/zhuff-fast-compression-t99.htm#155)\n[0.1](http://img49.xooimage.com/files/d/7/a/zhuff-165bce2.zip)\nis a free file compressor for Windows by Yann Collet, Dec. 13, 2009. It is described as\na combination of LZ4 and Huff0, a fast Huffman coder. LZ4 uses LZSS, an LZ77 variant\nusing flags to identify matches and literals. It requires the Microsoft runtime libraries,\nwhich are not included in the program size shown.\n[zhuff 0.8](http://fastcompression.blogspot.com/p/zhuff.html)\n[(discussion)](http://encode.su/threads/521-Zhuff-fast-compression)\nhas 3 compression levels, from -c0 (fastest) to -c2 (best). All are multithreaded, but\ndecompression at all levels and compression with -c0 is I/O bounded (about 40 seconds).\nTimes are process times for these cases, and real times for -c1 and -c2 compression.\n[zhuff](http://fastcompression.blogspot.com/p/zhuff.html) 0.95b was\nreleased Jan. 27, 2014. zhuff 0.97 beta was released Feb. 2, 2014. Both\nprograms were tested using the 64 bit Windows version under Ubuntu Wine.\nThere are also 32 bit Windows versions that produces identical compressed files.\n## .3088 lzhhf\n\n[lzhhf](https://github.com/grtamayo/lzhhf) is a free, experimental compressor\nby Gerald R. Tamayo, Sept. 2, 2022.\nLZ77 algorithm (essentially lzuf62 plus Golomb coding the match_len and then the\nliterals are dynamic Huffman coded.\nCompression memory used ~13.75MB. 12MB hash table plus 1 MB sliding\nwindow plus 0.5MB look-ahead pattern buffer.\n## .3092 slug\n\n[slug v1.1b](http://www.mytempdir.com/1309828)\n[(mirror)](http://www.geocities.com/lovepimple_mail/)\nis a free, closed source file compressor by Christian Martelock,\nApr. 26, 2007.  It uses an LZ type algorithm with\na 128K non-sliding window and Huffman coding.\nIt is designed for high speed and low memory usage.\nSystem (wall) times for enwik9: 18 (51) seconds for compression, \n14 (30) for decompression.\n[slug](http://christian.martelock.googlepages.com/index.htm) 1.27,\nMay 7, 2007, uses a ROLZ variant with a 8MB non-sliding window and semi-dynamic\nHuffman coding trees rebuilt every 4KB (more frequently near the beginning of a file).\n## .3097 ect\n\n[ect](https://github.com/fhanau/Efficient-Compression-Tool) (Efficient Compression Tool) 0.95\nis a free, open source compressor by fhanau\nthat produces smaller zip or gzip files (LZ77 deflate format) in Linux.\nTested (note 103) on Apr. 27, 2025.\nOptions tested were -9 -zip --mt-deflate to select zip at level 9. There is no upper limit on the\nlevel, but higher levels produce diminishing returns and take longer. Only compression was tested\nbecause decompression is by gzip or unzip.\n## .3098 lzuf62\n\n[lzuf](http://compgt.googlepages.com/lz77) is a free, experimental open source\nfile compressor by Gerald R. Tamayo, Apr. 15, 2009. It uses LZ77 with folded unary encoding\nof match lengths. It takes no arguments. It has a separate decompression program, lzufd.exe.\n[lzuf62](https://github.com/grtamayo/lzuf62) is an update, Sept. 2, 2022.\nImproved lzuf with optional \"sliding window\" history buffer bit sizes (WBITS = 12..20).\nCompression memory used ~13.75MB. 12MB hash table plus 1 MB sliding window plus 0.5MB\nlook-ahead pattern buffer.\n## .3098 pigz\n\n[pigz](http://zlib.net/pigz/) 2.2.3 is a free command-line file compressor for Linux,\nJan. 15, 2012. It uses the deflate (LZ77) format for compatibility with gzip, but is multi-threaded\nfor better speed at a small cost in compression ratio. -9 selects best compression.\nDecompression is single-threaded and I/O bound.\n[zlib](http://zlib.net/)\nversion 1.2.3 or higher. For this test, pigz was compiled using the supplied Makefile under Ubuntu\nLinux with g++ 4.6.1 and linked to zlib 1.2.5. Decompression was tested with unpigz, compiled\nsimilarly. It was tested on a 2.66 GHz Core i7 M620 (2 cores x 2 hyperthreads per core) as in note 48. Virtual\nmemory usage was measured with top at 115 MB for compression and 33 MB for decompression. Resident memory\nusage was 2 MB. Compression time is real time at about 350% CPU usage. Decompression is I/O bound\n(less than 100% CPU), so CPU time is reported. gzip is shown for comparison.\n[zopfli](https://code.google.com/p/zopfli/) algorithm, a very highly\noptimized and slow implementation of deflate. Decompression speed is not affected\nand is compatible with gzip. The test program was built from source code in Ubuntu\nusing the supplied Makefile with g++ 4.6.3.\n## .3102 kzip\n\n[kzip](http://advsys.net/ken/utils.htm) is a free, closed source\ncommand line compressor by Ken Silverman, compiled May 13, 2006,\nreleased May 18, 2006.  It is an optimizing compressor producing\nzip-compatible archives but with better compression.  The option /b512 sets the\nblock splitting threshold.  The default is /b256, but /b512 was found optimal\non enwik8.  /s0 (default) selects maximum compression and ranges from /s0\nto /s3.  No decompresser is included, but archives can be read with any\nprogram that reads zip files (pkzip, unzip, 7zip, WinRAR, WinACE, etc).\n\n```\nOptions      enwik8    Comp (ns/B)    enwik9\n-------    ----------  -----------  ----------\n/s0 /b0    35,029,924  2490                     (one large block)\n/s0 /b256  35,025,767  5220         310,281,906 (default, s0 = extreme mode)\n/s0 /b512  35,012,219  5410         310,248,404 (best enwik8)\n/s0 /b1024 35,016,649  4440         310,188,783 (best enwik9)\n/s1        35,028,473  5240                     (s1 = intense mode)\n/s2        42,370,689   860                     (s2 = longest run)\n/s3        63,191,700   820                     (s3 = Huffman code only)\npkzip 204  36,934,712   123                     (for comparison)\n```\n\n[uc2](http://www.nicodevries.com/nico/professional.php) (UltraCompressor II\nrevision 3 pro) is a commercial (free for noncommercial use) command line and GUI \narchiver for DOS by Nico de Vries, June 1, 1995.  It uses LZ77 and Huffman coding.\nThe -tst option selects maximum compression.\n\nuc2 includes a program for converting archives to self extracting\nprograms (uc2sea) which produced smaller files (enwik8.exe = 35,397,343 bytes,\nenwik9.exe = 312,759,499 bytes), but in this mode decompression failed for enwik9,\ntruncating the last 21 bytes of output.  uc2sea works by first extracting the\narchive and then recompressing it using a slightly different algorithm.\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp Mem\n-------           -------       ----------  -----------  -----------  -----------  ----- ----- ---\nthor 0.9a         ex            41,670,916  368,669,696     61,556 x  368,731,252     54    51 5.5\nthor 0.9a         e             45,842,692  412,096,696     61,556 x  412,157,852     44    50\nthor 0.9a         ef            55,063,944  490,400,720     61,556 x  490,461,876     45    53\n\nthor 0.94a        exx           35,696,028  315,611,168     68,922 x  315,680,090     82    32   2\n\nthor 0.95         e1            55,138,792                                            21    27\nthor 0.95         e2            45,714,740                                            21    23\nthor 0.95         e3            41,528,948                                            29    29\nthor 0.95         e4            35,795,184  314,092,324     49,925 x  314,142,249     64    34  16\nthor 0.95         e5            35,696,032  315,611,172     49,925 x  315,661,097     80    22   2\n\nthor 0.96a        e1            54,915,456  488,397,982     50,071 x  488,448,053     17    20 1.6\nthor 0.96a        e2            45,714,724  411,416,252     50,071 x  411,466,323     23    19 1.5\nthor 0.96a        e3            41,531,628  367,671,220     50,071 x  367,721,291     27    24   6\nthor 0.96a        e4            35,795,184  314,092,324     50,071 x  314,142,395     62    30  16\nthor 0.96a        e5            35,696,032  315,611,172     50,071 x  315,661,243     80    18   2\n1_OO_LL_MMM OOOOOOOO                   = 10 bit offset\n  00_LLL_MMM  OOOOOOOO OOOOOOOO          = 16 bit offset\n  010_LL_MMM  OOOOOOOO OOOOOOOO OOOOOOOO = 24 bit offset\n  011_LL_MMM                             = repeat previous offset\n```\n\n lz5 was compiled using gcc 4.8.4 with the supplied Makefile for Ubuntu.\nOption -0 through -18 selects the compression level (fastest..best).\nDefault is -0.\n \n\n```\n                Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp Mem  Alg  Note\n-------           -------       ----------  -----------  -----------  -----------  ----- ----- ---- ---  ----\nlz5 1.3.3         -0            49,358,209  433,092,957    138,210 s  433,231,167    8.7   3.9    9 LZ77  48\n                  -18           36,514,408  319,510,433    138,210 s  319,648,643  10578   3.7 1139 LZ77  48\nCompression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp Note\n-------         -------         ----------  -----------  -----------  -----------  ----- ----- ---- \ngzip 1.3.5      -9              36,445,248  322,591,995     34,408 x  322,626,403     55    22  48 (Linux)\ngzip 1.3.5      -9              36,445,248  322,591,995     38,801 x  322,630,796    101    17     (Windows)\ngzip 1.3.5                      36,518,329  323,742,882     38,801 x  323,781,683     85    19\nCompression          Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram         Options           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Note\n-------         -------         ----------  -----------  -----------  -----------  ----- ----- ---- ---- \ndoboz 0.1                       36,367,430         fail     76,471 x                 940    10       26\n                                36,367,430  322,415,409     83,591 x  322,499,000    533   3.4 1200  48\n```\n\n The Windows version 2.32 is dated June 19, 2006.\n Info-ZIP 3.00 was released July 7, 2008. Decompression was tested with\nUnZip 6.00, released Apr. 29, 2009.\n \n\n```\n                      Compression                 Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram                 Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Notes\n-------                 -------               ----------  -----------  -----------  -----------  ----- ----- ---- ---- --\nInfo-ZIP 2.31 (Linux)   -9                    36,445,373  322,592,120     57,583 x  322,649,703    104    35  0.1 LZ77\nInfo-ZIP 2.32 (DOS)     -9 (unset TZ)         36,445,333                                           178   101      LZ77 16\nInfo-ZIP 2.32 (DOS)     -9                    36,445,351                                           179            LZ77 16\nInfo-ZIP 2.32 (Win32)   -9                    36,445,474                                           183            LZ77 16\nInfo-ZIP 2.32 (Win32)   -9                    36,445,443  322,592,190     75,806 xd 322,667,996     96    13  1.2 LZ77\nInfo-ZIP 3.00 (Win32)   -9                    36,445,475  322,592,222    101,079 xd 322,693,301    114    18  1.3 LZ77 26\n```\n\n There are many programs that produce zip files.  I don't plan to test them all.\n \n\n```\n                      Compression                 Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram                 Options                 enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------                 -------               ----------  -----------  -----------  -----------  ----- ----- ---- ----\npkzip 2.0.4                                   36,934,712  327,607,376     29,184 xd 327,636,560    123    44  1.7 LZ77\npkzip 2.0.4             -ex                   36,556,552  323,403,526     29,184 xd 323,432,710    171    50  2.5 LZ77\n```\n\n Note: this is not the jar compressor from Arjsoft.\n The options -m0 through -m4 select compression level. The default,\n-m1, gives maximum compression. -m0 stores with no compression.\n-m1 through -m4 compress progressively larger but faster, with slower\ndecompression.\n \n\n```\nProgram   Options    enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg  Note\n-------   -------  ----------  -----------   ----------   -----------   ---- ----- --- ---- ----\narj 3.10  -m0     100,000,127                                             12    10   3 store 26\n          -m1      37,091,317  328,553,982     143,956 x  328,697,938    262    67   3 LZ77  26\n          -m2      37,381,391                                            224    68   3 LZ77  26\n          -m3      39,413,127                                            185    72   3 LZ77  26\n          -m4      44,157,478                                            116    91   3 LZ77  26\n```\n\n lzgt3a was added Oct. 25, 2008. It uses a 128K window size, 64K\nlookahead buffer, and improved coding.\n \n\n```\nProgram           enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg\n-------         ----------  -----------   ----------   -----------   ---- ----- --- ----\nlzgt            47,560,234                   1,989 sd                 634   234   2 LZ77\nlzgt1           43,928,072  403,385,292      2,025 sd  403,387,317   3390   865   2 LZ77\nlzgt2           57,268,099                   1,935 sd                 982   274   1 LZ77\nlzgt3           54,253,334                   1,963 sd                 889   280   1 LZ77\nlzgt3a          37,444,440  334,405,713      4,387 xd  334,410,100   1581  2886   2 LZ77\n```\n\n The most recent version was written in Visual C and ported to Windows as a\ncross compressor intended to produce self extracting archives for the\nCommodore.  By default, pucrunch appends a 276 byte header containing 6510 code to\nextract the file.  There are also standalone decompressers written in 6510\nassembler and in Z80 assembler.  I could not test in these environments, so I \nused the -d -c0 options to turn off the self extracting feature, which requires \nthe (larger) Win32 external compressor/decompresser.\n There are two additional limitations.  First, the decompresser appends a 2 byte\nheader to indicate the load address, which is required by the Commodore.  To\nmake the decompressed file bitwise identical, this must be stripped off.  Second,\nthe input file size is limited to 64,936 bytes.  The author tested a modified\nversion without a file size limit on the Calgary corpus, but this modified version\nwas not posted, so I did not use it.\n To overcome these limitations\nI wrote the following Perl scripts to compress and decompress.  The first script\ncompresses by splitting the input into blocks of 64,936 bytes, compressing them\nseparately, and appending the compressed files each with a 2 byte header to indicate\nthe block size.  The second script decompresses each block one at a time, strips\noff the 2 byte Commodore header, and appends them.  Each script takes the input\nand output files as command line arguments.  The second script is included in\nthe decompresser size.\n \n\n``` bash\n#!/usr/bin/perl\n# compress with pucrunch: perl p input output\nopen(IN,\"$ARGV[0]\")||die \"$!: $ARGV[0]\";\nopen(OUT,\">$ARGV[1]\")||die \"$!: $ARGV[1]\";\nbinmode(IN);\nbinmode(OUT);\nwhile ($n=read(IN, $s, 64936)) {\n  open(TMP1,\">tmp1\")||die \"$!: tmp1\";\n  binmode(TMP1);\n  syswrite(TMP1, $s, $n);\n  close(TMP1);\n  `pucrunch -d -c0 tmp1 tmp2`;\n  open(TMP2,\"tmp2\")||die \"$!: tmp2\";\n  binmode(TMP2);\n  $size=(stat(TMP2))[7];\n  print(\"$n -> $size\\n\");\n  $n=read(TMP2,$s,$size);\n  printf(OUT \"%c%c%s\", $size/256, $size%256, $s);\n  close(TMP2);\n}\n\n#!/usr/bin/perl\n# unpack with pucrunch: perl up input output\nopen(IN,\"$ARGV[0]\")||die \"$!: $ARGV[0]\";\nopen(OUT,\">$ARGV[1]\")||die \"$!: $ARGV[1]\";\nbinmode(IN);\nbinmode(OUT);\nwhile (($c1=getc(IN)) ne \"\") {\n  $c2=getc(IN);\n  $size=unpack(\"C\",$c1)*256+unpack(\"C\",$c2);\n  $n=read(IN, $s, $size);\n  if ($size!=$n) {die \"size=$size n=$n\\n\";}\n  open(TMP1,\">tmp1\")||die \"$!: tmp1\";\n  binmode(TMP1);\n  syswrite(TMP1, $s, $n);\n  close(TMP1);\n  `pucrunch -u tmp1 tmp2`;\n  open(TMP2,\"tmp2\")||die \"$!: tmp2\";\n  binmode(TMP2);\n  read(TMP2,$s,2);\n  read(TMP2,$s,64936);\n  printf(OUT \"%s\", $s);\n  close(TMP2);\n}\n```\n\n pucrunch suggests using -p1 and -m6 options to improve compression\nbut these do not help.\n Run times are wall times.  Using scripts, Timer 3.01 does not provide\nuseful process times, since it times Perl rather than pucrunch.\nThe decompression time (463 sec) is probably high because Windows Task Manager\nshows that pucrunch is running only a small fraction of the time, perhaps 10%.\nMost of the time is probably the overhead of file I/O and running pucrunch \n15,400 times.\n The pair of bit counts and the character count mod 3 (probably unnecessary)\nare mapped to a second table of counts to compute\nthe next-bit probability. That table is updated by incrementing the appropriate\ncount and halving both if the sum exceeds 60000. The initial mapping\nof this second table is (n0,n1) to (n0,n1) except if either of the input counts\nis 0, in which case the mapping is (0,n1) to (1,1+2^n1) or (n0,0)\nto (1+2^n0,1). The final bit prediction is n1/(n0+n1).\n The program was a submission\nto a data compresssion context for Dr. Dobbs Journal. To test, the source\ncode was compiled using make and tested in Linux. It compresses and decompresses\nfrom standard input to standard output. It takes no options.\n \n\n```\n         Compression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram   Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------   -------       ----------  -----------  -----------  -----------  ----- -----  --- ---\nlzw 0.1                 42,554,530  380,782,976     42,215 x  380,825,191   1917    27   17 LZW\nlzw 0.2                 41,960,994  367,633,910        671 s  367,634,581   3597    31   18 LZW\n```\n\n lz4opt v1.00\nis a free, closed source file compressor for 32 bit Windows by Ilia Muraviev,\nFeb. 9, 2016.\nIt is compatible with LZ4, an LZ77 compressor. Options cf, c, cb\ncompress fast, normal, and best respectively.\n \n\n```\nCompresor    Opt     enwik8      enwik9           prog     Total       Comp  Deco  Mem ALg  Note\n---------    ---   ----------  -----------      -------  -----------   ----  ----  --- ---  ----\nlz4opt 1.00  cf    50,052,286  444,844,266     48,445 x  444,892,711    5.6         18 LZ77 68\n             c     44,815,112  397,492,322     48,445 x  397,540,767   11.4         22 LZ77 68\n             cb    41,950,671  372,074,748     48,445 x  372,123,193  206     1.5  122 LZ77 68\nlz4x 1.02    c1    52,653,040  472,784,650     48,609 x  472,833,259    8.6   3.5   19 LZ77 48\n             c2    44,182,671  392,104,176     48,609 x  392,152,785   30     3.2   19 LZ77 48\n             c3    42,833,452  379,633,926     48,609 x  379,682,535   47     3.2   19 LZ77 48\n             c4    41,950,112  372,068,437     48,609 x  372,117,046  136     3.3  114 LZ77 48\n             c4    41,950,112  372,068,437     48,609 x  372,117,046   79     1.4      LZ77 68\nCompression        Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg\n-------           -------       ----------  -----------  -----------  -----------  ----- -----  --- ---\narbc2z                          38,756,037  379,054,068      6,255 sd 379,060,323   2659  2674   68 PPM2\narbc2                           38,780,256  379,093,120      6,070 sd 379,099,190   2528  2646   67 PPM2\narbc1                           48,586,591  486,892,000      6,047 sd 486,898,047   2439  2611  1.8 PPM1\narbc0                           63,501,994  644,561,590      5,988 sd 644,567,578   2459  2606  1.5 o0\nCompressed size      Decompresser  Total size   Time (ns/byte)\nProgram    Opt         enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------    ---       ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nlz4 0.2              55,092,932  488,028,718      9,556 x  488,038,274     13     7   13 LZ77 26\nlz4 0.6              55,062,753  487,772,940     42,139 x  487,815,079     14     7   13 LZ77 26\nlz4hc 0.9            44,182,558  392,102,544     43,617 x  392,146,161     65     7   14 LZ77 26\nlz4 1.2    -c0       54,303,743  481,142,522     49,128 x  481,191,650     15     6   20 LZ77 26\n           -c1       44,218,551  392,460,229     49,128 x  392,509,357     69     6   21 LZ77 26\n           -c2       42,870,164  379,999,522     49,128 x  380,048,650     91     6   20 LZ77 26\nCompression     Compressed size      Decompresser  Total size  Time (ns/byte)\nProgram      Options     enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg  Note\n-------      -------   ----------  -----------  -----------  -----------  ----- ----- ---- ---- ----\nlzss 0.01    e         48,615,051  426,009,994      44,555 x 426,054,549    193    15  625 LZSS\n             ex        38,254,303  337,565,308      44,555 x 337,609,863   9708    14  625 LZSS\nlzss 0.02    cf        50,110,565  448,712,956      48,114 x 448,761,070     22    12   17 LZSS  26\n             cf                    448,712,956      48,114 x 448,761,070      6.0       17 LZSS  63\n             c         45,093,733  399,850,630      48,114 x 399,898,744     40    11   17 LZSS  26\n             c                     399,850,630      48,114 x 399,898,744     12.5       17 LZSS  63\n             cx        42,874,387  380,192,378      48,114 x 380,240,492    265    10  145 LZSS  26\n             cx        42,874,387  380,192,378      48,114 x 380,240,492    107   2.3  145 LZSS  63\nVersion           enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg  Note\n-------         ----------  -----------   ----------  -----------    ---- ----- --- ---- ----\nBriefLZ 1.05    46,638,341  425,384,313      5,298 x  425,389,611     66    18    2 LZ77\nblzpack 1.1.0   43,300,800  390,122,722     14,907 s  390,137,629     29    15    4 LZ77  26\nblzpack 1.1.0   43,300,800  390,122,722     14,907 s  390,137,629     21   7.5    3 LZ77  48\nVersion           enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg  Note\n-------         ----------  -----------   ----------  -----------    ---- ----- --- ---- ----\nlzf 1.00  c     48,947,532  440,862,551     47,737 x  440,910,288     39    12   18 LZ77 26\n          c     48,947,532  440,862,551     47,737 x  440,910,288      8            LZ77 60\n          cx    46,318,130  416,377,741     47,737 x  416,425,478     53    11   18 LZ77 26\n          cx    46,318,130  416,377,741     47,737 x  416,425,478     14   2.3      LZ77 60\nlzf 1.01  c     48,947,532  440,862,551     47,728 x  440,910,279     39    12   18 LZ77 26\n          c     48,947,532  440,862,551     47,728 x  440,910,279      8            LZ77 60\n          cx    46,318,130  416,377,741     47,728 x  416,425,469     49    11   18 LZ77 26\n          cx    46,318,130  416,377,741     47,728 x  416,425,469     12   2.3      LZ77 60\nlzf 1.02  c     47,827,133  430,634,000     48,359 x  430,682,359     16   3.9   22 LZ77 48\n          c     47,827,133  430,634,000     48,359 x  430,682,359      7            LZ77 68\n          cx    45,198,298  406,805,983     48,359 x  406,854,342    110   3.7  151 LZ77 48\n          cx    45,198,298  406,805,983     48,359 x  406,854,342     68   2.2      LZ77 68\n```\n\n The -C8 option selects the maximum number of contexts, 2 \n\n```\n  gcc -O2 -march=pentium4 -fomit-frame-pointer -s srank.c -o srank.exe\n```\n\n Version 0.9 (Oct. 22, 2006) is a faster version (quick.exe)\nwhich handles large (64 bit) files.\n Version 1.20 (Mar. 15, 2007) is an archiver rather than a file compressor.\n Version  Version 1.30 (Aug. 14, 2007) modes 0, 1, and 2 are compatible with version 1.20,\nbut mode 3 (best compression) is new.\n Version 1.40 (Nov. 13, 2007) is an experimental version designed for better speed.\nIt has only one mode.\n \n\n```\nVersion           enwik8      enwik9      prog size     Total       Comp Decomp Mem Alg\n-------         ----------  -----------   ----------  -----------    ---- ----- --- ----\nQuickLZ 0.1     57,331,969  (fails)         45,361 x                  19    21  154 LZ77\nQuickLZ 0.9     56,900,177  507,806,141     45,086 x  507,851,227     11    11   10 LZ77\nQuickLZ 1.20    57,147,067  510,018,447     43,501 x  510,061,948     17    12    2 LZ77\nquick3 1.30b    46,378,438  410,633,262     44,202 x  410,677,464     48    12    3 LZ77\nQuickLZ 1.30 -3 46,445,704  411,493,051     47,304 x  411,540,355     49    12    2 LZ77\n             -2 51,941,357                                            23    11\n             -1 57,153,015                                            12    11\n             -0 52,803,919                                            20    16\nquickLZ 1.40    47,728,849  417,653,684     43,922 x  417,697,606     28    13   13 LZ77\nVersion   Option                                  enwik8  C/D Time   enwik9     C/D Time   Mem Note\n-------   ------------------------------------  ---------- ------- -----------  ---------  --- ----\nstz 0.7.2 -c  (LZBW2 best compression speed)    50,575,825         447,732,354   15   13    3   26\n          -c1 (LZBW3 best uncompression speed)  56,100,810         510,600,276   16   10    3   26\n          -c2 (LZBW2A best compression)         47,681,682         420,391,400   16   12    3   26\n          -c3 (LZBW3A experimental)             50,575,825         447,732,354   15   11    3   26\nstz 0.8   -c  (LZBW2 best compression speed)    50,143,263  11  11 444,061,128   16   13    3   26\n          -c1 (LZBW3 best uncompression speed)  55,670,417  16   9 506,622,114   18   12    3   26\n          -c2 (LZBW2A best compression)         47,192,312  16  11 416,524,596   14   13    3   26\n          -c3 (LZBW3A)                          54,080,795  15  11 480,696,931   18   12    3   26\n          -c4 (LZBW2B experimental)             54,080,795  13   9 480,696,931   20   13    3   26\n          -c5 (LZBW3B experimental)             54,080,795  16  12 480,696,931   19   14    3   26\n```\n\n compress 4.3d is is the Windows version of the UNIX compress\ncommand, released Jan 18, 1990.  It uses LZW and has no compression options.\n lzrw1 (Mar. 31, 1991)\nis byte-aligned LZ77 with a 12 bit offset and 4 bit length field\nallowing lengths 3-16.  Each group of 16 phrases (pointers or literals)\nis preceded by 2 flag bytes to distinguish pointers from literals.  \nMatches are found using a 4K hash table without confirmation which is \nupdated after each phrase.  It uses 16K of memory plus the input and\noutput buffers.\n lzrw1-a (June 25, 1991)\nis lzrw1 except that the length field represents values 3-18.\n lzrw2 (June 29, 1991)\nreplaces the offset with a 12 bit index into a rotating table\nof offsets, allowing the last 4K phrases (rather than 4K bytes) to\nbe reached.  The decompresser must reconstruct the phrase table\n(but not the hash table).  It uses 24K memory plus buffers.\n lzrw3 (June 30, 1991)\nreplaces the 12 bit length field with a 12 bit index into the\nhash table.  The decompresser must reconstruct the hash table.\nIt uses 16K memory plus buffers.\n lzrw3-a (July 15, 1991)\nuses a deep hash table (8 offsets per hash) with LRU replacement.\nIt uses 16K memory plus buffers.\n lzrw5 (July 17, 1991) uses LZW.  The dictionary is implemented as a tree.\nIt uses up to 384K memory plus buffers.\n There is an experimental lzrw4, but it was never fully implemented.\n All of the compression algorithms were originally implemented as\nmemory to memory compression functions in C, not as complete programs.\nI wrote a driver program which divides the input into 1 MB blocks (except lzrw5),\ncompresses them independently by calling the provided functions, and\nwriting the compressed size as a 4 byte number followed by the compressed\ndata.  However, compression could be improved by using larger blocks at\nthe cost of more memory.  For lzrw5 the block size is 64K because the program\nis not guaranteed to work correctly for larger blocks.  It did work on this\nbenchmark for a 192K block size, but not for 256K.  The distribution linked\nabove uses a 64K block size.\n \n\n```\nCompressor     enwik8      enwik9           prog     Total       Comp  Deco  Mem ALg\n-------      ----------  -----------      -------  -----------   ----  ----  --- ---\nlzrw1        59,692,493  564,053,011      3,142 s  564,056,153     24    17    2 LZ77\nlzrw1-a      59,471,657  560,457,545      4,328 x  560,461,873     23    15    2 LZ77\nlzrw2        55,360,907  511,142,568      4,420 x  511,146,988     22    16    2 LZ77\nlzrw3        52,616,827  483,918,830      4,622 x  483,923,452     21    17    2 LZ77\nlzrw3-a      48,009,194  438,253,704      4,750 x  438,258,454     38    17    2 LZ77\nlzrw5 (64K)  59,375,192  570,387,858      4,544 x  570,392,402    146    14    1 LZW\nlzrw5 (192K) 50,721,610  479,044,732                              174    14    1 LZW\n```\n\n## .3130 qbp\n\n[qbp](https://github.com/asimba/qbp) is a free, open source file compressor by Alexey Simbarsky, June 9, 2025.\nIt uses LZSS  LZSS (byte oriented, 1 byte for length + 2 bytes for offset, 4 bytes minimal match,\nsliding windows size is 64KiB )+RLE+RC32 (byte oriented, 0/1 mixed order). Source code\nis available in C, C++, and Rust. Build options (Win10, \"qbp.c\"):\n`gcc qbp.c -O3 -m64 -mtune=native -s -o ./qbp -fno-exceptions -fno-ident\n-fno-math-errno -fno-stack-protector -ffast-math -fomit-frame-pointer -fmessage-length=0 -Wall`\n## .3141 thor\n\n[thor 0.9a](http://www.maximumcompression.com/thor_09a.zip) is an experimental,\nclosed source, command line file compressor by Oscar Garcia, Mar. 19, 2006.\nIt is the fastest compressor on the [maximumcompression](http://maximumcompression.com/)\nbenchmark.  It has 3 modes: ef (fastest), e (normal) and ex (best).  However in this test it\nappears speed may be limited by disk I/O.\n[thor 0.94 alpha](http://rapidshare.com/files/27227218/THOR_094.zip.html)\n[(mirror)](http://www.encode.su/downloads/THOR_094.zip)\n[(mirror)](http://www.geocities.com/lovepimple_mail/)\nwas relesed Apr. 22, 2007.  exx is a new mode to select maximum compression.\nTimes shown are process times excluding disk I/O.  Actual times are 96 sec. to compress,\n75 sec. to decompress).\n[thor 0.95](http://rapidshare.com/files/30120842/THOR_095.zip.html)\n[(mirror)](http://www.maximumcompression.com/thor_095.zip), \nMay 8, 2007, has 5 compression options: e1 through e4 are LZP in order of increasing\ncompression; e5 is LZ77.  Note that e5 is best on enwik8 but e4 on enwik9.\n[thor 0.96a](http://rs33.rapidshare.com/files/50752881/THOR_v096.zip),\nAug. 23, 2007, works like 0.95.\n## .3148 etincelle\n\n[etincelle](http://phantasie.tonempire.net/pc-compression-f2/etincelle-fast-compressor-with-better-than-zip-compression-performance-t102.htm)\n[alpha 3](http://sd-1.archive-host.com/membres/up/182754578/Etincelle-alpha3.zip)\nis a free file compressor by Yann Collet, Mar. 26, 2010. It uses ROLZ with\nan order 1 context to reduce the offest length, followed by Huffman coding.\n## .3196 lz5\n\n[lz5](https://github.com/inikep/lz5/releases) 1.3.3 is a\nfree, open source file compressor by Przemyslaw Skibinski, Jan. 5, 2016.\nIt is a modification of lz4 by Yann Collett. It uses byte-aligned LZ77 codes\nas follows:\n## .3211 gzip124hack\n\n[gzip124hack](http://www.encode.su/downloads/gzip124hack.zip)\n[(mirror)](gzip124hack.zip)\n[(discussion)](http://encode.su/threads/890-gzip-1-2-4-hack-a-hacked-version-of-gzip?highlight=gzip124hack)\nis a modified version of gzip 1.2.4 by Ilia Muraviev, Aug. 13, 2007.\nIt uses LZ77.\nIt is a file compressor like gzip, except that it does not delete the input file.\nIt improves compression by using LZ77 lazy matching with 2 byte lookahead.\nThe compressed format is compatible with gzip.  -9 selects maximum compression.\n## .3224 doboz\n\n[doboz](https://bitbucket.org/attila_afra/doboz) 0.1 is a free,\nopen source file compressor by Attila T. Áfra, Mar. 18, 2011. It uses LZ77.\nIt is both a compression library and a simple single-threaded\nfile compressor which takes no options. To test,\nthe supplied compressor for 32 and 64 bit Windows was tested. The 32\nbit version crashed while compressing enwik9, possibly due to reading the\nwhole file into memory. The 64 bit version succeeded under Ubuntu/wine.\n## .3226 gzip\n\n[gzip](http://gnuwin32.sourceforge.net/packages/gzip.htm)\n1.3.5 is an open source single file command line compressor\nby Jean-loup Gailly and Mark Adler, Sept. 30, 2002.\nIt uses LZ77 (flate, but not compatible with zip).  \nThe -9 option selects maximum compression although its effect is small (see below).\n## .3226 Info-ZIP\n\n[Info-ZIP](http://www.info-zip.org/) 2.3.1 (Mar. 8, 2005)\nis a free, open source\narchiver for many operating systems.  It uses the standard LZ77 \"flate\" format, like\ngzip and many zip-compatible programs.  (The sizes are exactly 125 bytes larger\nthan gzip).  This test was under Linux \n(Ubuntu 2.6.15.27-amd64-generic) on a 2.2 GHz Athlon-64.\nUncompression was with UnZip 5.52 (Feb. 28, 2005), both part of the normal\nUbuntu distribution.  The -9 option selects maximum compression.\n## .3234 pkzip\n\n[pkzip](http://www.pkware.com/) 2.04e is a commercial\n(free trial) command line archiver by PKWARE Inc.\nwritten Jan 25, 1993.  It uses LZ77 (flate format).\nThe option -ex selects maximum compression.  The decompresser is pkunzip 2.04e.\nTimes are wall times.  (Timer doesn't show process times for DOS programs).\n## .3237 jar\n\n[jar](http://java.sun.com/j2se/1.5.0/docs/tooldocs/share/pack200.html) 0.98-gcc is an open\nsource command line archiver by Bryan Burns, 2002.  It uses LZ77 (zip).  It is included with Java (1.5.0_06) and\nis normally used to create .jar files for compiled Java applications and applets, but it can\nalso be used as an archiver.  It has no compression options.  \nThe cvf options creates an archive.  The M option says to not add a manifest file.\n## .3244 PeaZip\n\n[PeaZip](http://sourceforge.net/projects/peazip) 1.0 by Giorgio Tani (Nov. 6, 2006)\nis a GPL open source GUI archiver\nsupporting several common formats.  The format tested is the native format which uses zlib\n(gzip algorithm).  The \"better\" option chooses best compression (equivalent to gzip -9).\nIntegrity check (checksum) and encryption are turned off.\n## .3286 arj\n\n[arj](http://arj.sourceforge.net/) 3.10 is a free, open source\n(GPL v2) archiver by ARJ Software Russia, June 23, 2005. It is compatible\nwith the original ARJ by Robert K. Jung, which was patented\n([U.S. patent 5140321 A](http://www.google.com/patents/US5140321))\nfiled Sept. 4, 1991 and presumably expired. According to the patent,\nit uses LZ77 with flags to indicate a repeat of the last match\n(like LZX used in cabarc). Matches are found from a hash table of\nFIFO queues.\n## .3344 lzgt3a\n\n[lzgt1](http://compgt.googlepages.com/lz77)\n(click on lzgt3a.zip) is one of a group\nof free, open source, experimental file compressors by Gerald R. Tamayo, released\nJuly 17, 2008. It uses LZT (Lempel-Ziv-Tamayo) compression, a LZ77 variant\nin which the decompresser rebuilds a list of matches sorted by context match\nlength and the match length is implied or partially implied by the position\nin the list. lzgt implements LZT using a 4K sliding window, 32 byte\nlook-ahead buffer and 3 bit code length. lzgt1 is like lzgt\nbut uses a 16K sliding window and 128 byte look-ahead buffer.\nlzgt2 eliminates the code length entirely. lzgt3 is an improved version\nof lzgt2. All programs have separate decompressers (lzgtd1, etc) and are\ncompiled for DOS (and Windows).\n## .3502 pucrunch\n\n[pucrunch](http://www.cs.tut.fi/~albert/Dev/pucrunch/) is a free,\nopen source file compressor by Pasi Ojala, last updated Mar. 8, 2002.\nIt uses a combination of run length encoding (RLE) and LZ77 with Elias Gamma coding\nof the offsets and run lengths.\nThe original version was written on Mar. 14, 1997 for the Commodore series\n(Vic 20, Commodore 64, Commodore 128 and Commodore Plus 4/C16) in 6510\nassembly language, with updates on Dec. 17, 1997 and Oct. 14, 1998.\nThe 6510 is a 1 MHz, 8 bit microprocessor with 3 registers, \n16 bit (64K) address space, no cache, no pipelining, 8 bit ALU, no multiply or \nfloating point instructions, and no support for multitasking or virtual memory.  \nThe decompresser was designed to execute quickly\nin this environment with only a few hundred bytes of memory.\n## .3619 packARC\n\n[packARC v0.7RC11](https://www.dropbox.com/s/uq0nwgvr12ylut4/packARC%20v0.7RC11%20%28beta!%29%20%28GPL%29.zip)\n[(discussion)](<http://encode.su/threads/1839-packARC-v0-7RC11-GPL-release-(and-more)>)\nis a free, open source (GPL v3) archiver by Matthias Stirner, Dec. 7, 2013.\nIt incorporates packJPG (JPEG compressor), packMP3 (MP3 compressor) and\npackPNM (BMP, PPM, PGM, PBM image compressor). Other file times are compressed\nwith a simple context model and arithmetic coder. Option -sfx creates a\nself extracting archive. Option -np tells the program not to pause when done.\nFor this test, the source was compiled with MinGW g++ 4.8.0 using the supplied\nbuil_packarc.bat for 32 bit Windows.\n## .3626 urban\n\n[urban](ftp://ftp.sac.sk/pub/sac/pack/ddjcompr.zip) is an open\nsource file compressor for Unix by Urban Koistinen, Apr. 30, 1991.\nThe program is an order-2 indirect context model with bitwise arithmetic coding.\nA hash of the last two whole bytes plus the previously coded bits\nof the current byte (MSB first) are mapped to a hash table of size 710123.\nEach table element contains a count of 0s and 1s in the range 0 through 8,\nand a hash verification consisting of a second hash. When a collision\nis detected, the counts are reset to 0. Otherwise, the appropriate count is\nincremented and both are halved if either exceeds 8.\n## .3663 lzop\n\n[lzop](http://www.lzop.org/) v1.01 is a free, open source (GPL) command line\nfile compressor by Markus F.X.J. Oberhumer, Apr. 27, 2003.  A newer version, 1.02 rc1\nwas released July 25, 2005, but no Win32 executable was available for download\nas of May 29, 2006.  lzop uses LZ77.  It is designed for high speed.  -9 selects\nmaximum compression.  lzop is I/O bound.  timer 3.01 reports the decompression\nprocess time as 12 seconds.  The remaining 38 seconds is due to disk access.\n## .3676 lzw\n\n[lzw v0.1](http://www.encode.su/downloads/lzw01.zip) is a free, experimental\nfile compressor by Ilia Muraviev, Jan. 30, 2008.  It uses LZW with 16 bit\ncode words.  It takes no options.\n[lzw v0.2](http://www.encode.su/downloads/lzw02.zip) was released with\npublic domain source code for the decompresser, which zips to 671 bytes.  The file\nformat is as follows.  There is no header or trailer.\nEach 16 bit code word is in machine dependent order\n(LSB first on x86).  Codes 0-255 represent single bytes of the same value.\nCodes 256-65535 are assigned in ascending order by concatenating the decoded\nvalues of the previous two codes.  After assigning code 65535, new codes are\nassigned by replacing the oldest codes first, starting with 256.\nData is decoded into a rotating buffer of size 16 MiB (2<sup>24</sup> bytes)\nby copying a string from elsewhere in the buffer.  Neither the original nor\ncopied string crosses the buffer boundary, and they do not overlap each other.\nNo new symbol is added after decoding the first byte of the buffer.\n## .3701 MTCompressor\n\n[MTCompressor v1.0](http://encode.su/attachment.php?attachmentid=1808&d=1327074598)\n[(discussion)](http://encode.su/threads/1462-MTCompressor?p=28018#post28018)\nis a free, experimental command line compressor for Windows by David Catt,\nJan. 20, 2012. It uses an LZ77 variant similar to deflate. It is multi-threaded.\nReported time is real time running on 2 cores (note 26). Memory usage\nfluctuates during use. The peak is reported.\n## .3721 lz4x\n\n[lz4x v1.02](http://page.encode.su/#downloads) was released\nApr. 6, 2016. The options c1..c4 compress faster..better with LZ4 compatibility.\n## .3790 arbc2z\n\n[arbc2z](http://bijective.dogma.net/arbcx.zip) is a free, experimental command line\nfile compressor with source code by David A. Scott, June 23, 2006.\nIt is a bijective order-2 (PPM) arithmetic coder.  A bijective\ncoder has the property that all inputs to the decompresser are valid and produce distinct outputs.\nThe above archive also contains arbc2, which uses a different method of handling of the zero frequency problem,\narbc1 (order 1), and arbc0 (order 0), all of which are bijective.\n## .3800 lz4\n\n[lz4 v0.2](http://img28.xooimage.com/files/b/b/c/lz4-1432530.zip)\n[(website)](http://phantasie.tonempire.net/utilitaires-pc-f2/lz4-fast-compressor-for-windows-t95.htm#144)\nis a free file compressor by Yann Collet, Oct. 16, 2009.\nIt uses LZSS (an LZ77 variant\nwith flags to mark literals and matches). It takes no options.\nRun times are dominated by disk access.\n[lz4](http://fastcompression.blogspot.com/2011/01/lz4-worlds-fastest-compressor.html?spref=tw) 0.6\nwas released Dec. 12, 2010. lz4hc 0.9 (Dec. 13, 2010, same link) is a compatible\nversion with better compression. In both cases, run times are dominated by\ndisk access. Times shown are process times.\nActual times were 80+37 sec. for lz4 and 137+39 sec. for lz4hc.\nThe programs take no compression options.\n[lz4 v1.2](http://fastcompression.blogspot.com/p/lz4.html)\nwas released Oct. 10, 2011. It has 3 compression  levels (c0...c2).\nThe program automatically detects the number of cores (2, note 26) and uses the\nsame number of threads. However compression in mode c0 and all\ndecompression modes are I/O bound, using about 20% of available CPU.\nFor these modes, process time is reported. Compression modes c1 and\nc2 are real times with both cores fully utilized.\n## .3802 lzss\n\n[lzss 0.01](http://encode.su/downloads/lzss001.zip)\n(withdrawn) is a free,\nexperimental file compressor by Ilia Muravyov, Aug. 1, 2008. It uses\nLZSS, a byte aligned LZ77 variant with matches encoded with an 18 bit pointer\nand 6 bit length field, and 1 bit flags to distinguish matches\nfrom literals. It is discussed [here](http://encode.su/forum/showthread.php?t=143).\nCompression options are e (fast) or ex (smaller). The program is designed\nfor fast decompression. The program uses 625 MB for compression and 33 MB\nfor decompression.\n[lzss](http://compressme.net/)\n[0.02](http://compressme.net/lzss002.zip)\n[(discussion)](http://encode.su/threads/1875-Optimized-LZSS-compressor)\nwas released Feb. 7, 2014. Options cf, c, cx select fast, medium, and best\ncompression.\n## .3894 xdelta\n\n[xdelta](http://code.google.com/p/xdelta/) 3.0u is a free, open source command line\nfile compressor by Joshua McDonald, Oct. 12, 2008. It uses LZ77. The program is a delta\ncoder, meaning it will output the compressed difference between two files, and then\ndecompress the second file when given the first file uncompressed. It allows the first\nfile to be omitted, in which case it simply compresses. This is how the test was done.\n-9 specifies maximum compression.\n## .3901 BriefLZ\n\n[BriefLZ](http://ibsensoftware.com/download.html)\n[1.05](http://www.masm32.com/board/index.php?PHPSESSID=56c84a7413b0d35a437ae6e06fbd12e7&action=dlattach;topic=376.0;id=226)\nis a free, open source (C and MASM) file compressor by Joergen Ibsen,\nJan. 15, 2005.  It uses LZ77.  It takes no options.\nIt uses about 2 MB memory for compression and about 900 KB for decompression.\n[brieflz 1.1.0](https://github.com/jibsen/brieflz) was last\nupdated Sept. 23, 2015. To test, was compiled using the supplied\nMakefile (as blzpack) in the example subdirectory of the GitHub distribution using\ngcc 4.8.1 in Windows (note 26) and gcc 4.8.4 in Linux (note 48).\n## .3972 mtari\n\n[mtari 0.2](http://encode.su/threads/1837-FARI-Fast-Arithmetic-Compressor?p=35705&viewfull=1#post35705) is a free, open source (GPL v3) file compressor by\nDavid Werecat, Dec. 10, 2013. It is a multi-threaded bitwise order 17 context model\nwith arithmetic coding.\nTo test, it was compiled with MinGW gcc 4.8.0 with options -O2 -fopenmp.\n## .4068 lzf\n\n[lzf](http://compressme.net/) [v1.00](http://encode.narod.ru/lzf100.zip)\n[(discussion)](http://encode.su/threads/1819-LZF-Optimized-LZF-compressor?p=34984#post34984)\nis a free, experimental file compressor by Ilya Muravyov,\nOct. 29, 2013. It uses byte aligned LZ77 with a 8 KB window. Commands\nc and cx give faster or better compression, respectively.\n[lzf 1.01](http://encode.narod.ru/lzf101.zip), Oct. 29, 2013, is a performance\noptimization with no change in compresion.\n[lzf](http://compressme.net/) [1.02](http://compressme.net/lzf102.zip)\n[(discussion)](http://encode.su/threads/1819-LZF-Optimized-LZF-compressor?p=40717&viewfull=1#post40717)\nwas released Oct. 2, 2014.\n## .4092 srank\n\n[srank 1.1](http://www.cs.auckland.ac.nz/~peter-f/) is a free,\nopen source file compressor by P. M. Fenwick, originally written Sept. 5, 1996\nand last updated Apr. 10, 1997.  It uses symbol ranking, like MTF (move to front)\nin BWT, but in order 3 contexts without a BWT transform.  When a symbol is encountered\nit is encoded with 1, 3, or 4 bits according to its position in a queue of length 3, \nthen moved to the front.  Long runs of first place symbols are run length encoded\nusing 12 bits to encode the length of the length of the run.\nA miss is coded using pseudo-MTF in an order-0 context using 7 bits for \nthe first 32 symbols and 12 bits for the rest.  It is pseudo-MTF because after a\nsymbol is found it is swapped with another symbol about half way to the front,\nwith some dithering.  The algorithm is designed for speed rather than good compression.\n<sup>18</sup>.\nFor this test, the C source code was compiled with MinGW 3.4.5:\n## .4106 QuickLZ\n\n[QuickLZ](http://www.quicklz.com/) v0.1 is an open source (GPL)\ncompression library designed for high speed by Lasse Mikkel Reinhold,\nSept. 24, 2006.  Tests were performed with demo.exe.  Speed is I/O bound.\nTimes shown are process times, but wall times can be 2-4 times greater.\nOn enwik9 compression, the program reports \"file too big\".\n[1.30 beta](http://www.quicklz.com/beta.html)\n(Apr. 16, 2007) has 4 modes (0-3) with 4 separate executables.\nOnly version 3 (quick3.exe, max compression) was tested.\n## .4165 stz\n\n[stz 0.7.2](http://www.toofiles.com/fr/rawfilesdownload-documents-zip-stz.html)\nis a free, experimental file compressor by Bruno Wyttenbach, Feb. 15, 2011.\nIt uses LZ77. It has 4 compression modes as shown in the table below. Times are process\ntimes. Real times are closer to 40-45 seconds. Memory is 3.3. MB for all compression modes\nand the same for decompression. Most of the memory is for I/O buffers (2MB each). The actual algorithm\nuses 48 KB. Modes -c and -c3 compress to the same size but the archives\ndiffer by 1 byte in the header. stz.exe zip size is 40,425.\n[stz 0.8](http://www.toofiles.com/fr/rawfilesdownload-documents-zip-5313_stz.html),\nMar. 4, 2011,\nimproves compression and adds two new experimental modes. Compression and decompression process times\nin ns/byte are given below for both enwik8 and enwik9. Wall times are slower due to disk I/O.\nModes -c, -c1, and -c2 select best\ncompression speed, best uncompression speed, and best size respectively, but this appears only\nto hold for enwik8, probably because of disk I/O interference. Modes -c3, -c4, and -c5 produce identical\narchives. Additional changes are a Drag'n'drop interface, a CRC check (adds 2% to time),\nand more flexible command line interface. 5313_stz.zip size is 41,941.\n## .4246 compress\n\n## .4382 lzrw3-a\n\n[lzrw3-a](http://www.ross.net/compression/) is one of a series\nof public domain (open source) memory to memory compressors by \nRoss Williams in 1991.  The programs were\n[implemented](lzrw.zip)\nas file compressors by Matt Mahoney on Feb. 14, 2008.  The programs\nare as follows:\n## .4473 fcm1\n\n[fcm1](http://encode.su/downloads/fcm1.zip) is a free, open source file compressor\nby Ilia Muraviev, May 23, 2008.  It mixes order 0 and order 1 models and uses bitwise\narithmetic coding as in fpaq0 and paq.  The bit predictions are combined by weighted averaging,\nwith the order 1 model weighted 15/16 unless the model is in its initial state, in which\ncase the order 0 model prediction is used.  Each context is mapped to 2 16-bit counters\nin initial state 1/2.  One counter is updated by 1/8 of the prediction error and the\nother by 1/32.  The model prediction is the average of these two values.\nThe compressed file has a 4 byte header containing the file size.\n\n```\nCompressor     enwik8      enwik9           prog     Total       Comp  Deco  Mem ALg\n-------      ----------  -----------      -------  -----------   ----  ----  --- ---\nfcm1         45,402,225  447,305,681      1,116 s  447,306,797    228   261    1 CM1\n```\n\n[runcoder1](http://www.ezcodesample.com/ralpha/RunCoder1.txt)\nis a free, open source (GPL) file compressor by Andrew Polar, Mar. 30, 2009.\nIt uses an order 1 model with arithmetic coding. It takes no options.\nThe program is available as source code (C++) only. For this test\nit was compiled with MinGW g++ 3.4.2 with options `-O2 -march=pentiumpro\n-fomit-frame-pointer -s` for 32-bit Vista as noted in note 26.\n## .4598 data-shrinker\n\n[data-shrinker](http://code.google.com/p/data-shrinker/) is a free, open\nsource file compressor by Siyuan Fu, Mar. 23, 2012. It uses a LZ77 format\nsimiler to LZ4 for high speed. It takes no options. No executable was provided.\nTo test, the source code was compiled with g++ 4.5.1 -O3 -s under 32 bit Windows\nand process times measured with output to nul:\n\n```\nCompressor    Version    Opt    enwik8      enwik9          prog     Total       Comp  Deco  Mem Alg  Note\n----------    ---------  ---  ----------  -----------     -------  -----------   ----  ----  --- ---- ----\ndata-shrinker 23Mar2012       51,658,517  459,825,318     3,706 s  459,829,024     14     4    2 LZ77 26\n```\n\n[lzwc 0.3](http://encode.su/threads/1661-LZWC-A-fast-tree-based-LZW-compressor?p=31955&viewfull=1#post31955)\nis a free, open source (GPL) file compressor by David Catt, Jan. 15, 2013.\nIt uses LZW with dictionary entries coded using 2 bytes. There is also a version 0.1\nwhich produces identical compressed files but is not as fast. The program takes no options.\n\nlzwc v0.7 fixes a bug in decompression of binary files, but does not change compressed size or speed. lzwc_bitwise is a version that uses less than 16 bits to encode symbols when the dictionary is small.\n\n```\nCompressor           enwik8      enwik9          prog     Total       Comp  Deco  Mem Alg  Note\n----------         ----------  -----------     -------  -----------   ----  ----  --- ---- ----\nlzwc 0.1           46,647,318                  1,955 x                 280   290   70 LZW   26\nlzwc 0.3           46,647,318  463,892,454     3,017 x  463,895,471     85    90   71 LZW   26\nlzwc_bitwise 0.7   46,639,414  463,884,550     4,183 x  463,888,733    123   134   71 LZW   26\n```\n\n[exdupe](http://www.exdupe.com/) v0.3.3 beta is a\ndeduplicating archiver supporting full and incremental backups,\nunder development by Lasse Reinhold, Oct. 20, 2011. When the beta phase ends,\nit will be a commercial program with source code available under\nrestricted and non-permissive terms. Only 64 bit systems are supported.\nPartial source code is available for this version, although not for the\ncompression and decompression code, which is derived from QuickLZ\n(LZ77). It was tested in Linux. A later version, 0.3.6 beta,\nwas available only for 64 bit Windows on Oct. 30, 2012, and was not tested.\n\n```\nCompressor     Opt    enwik8      enwik9          prog     Total       Comp  Deco   Mem ALg  Note\n----------     ---  ----------  -----------     -------  -----------   ----  ----  ---- ---- ----\nexdupe 0.3.3        53,717,422  478,788,378  1,092,986 x 479,881,364     27     5  1000 LZ77  48\nCompressor     Opt    enwik8      enwik9          prog     Total       Comp  Deco   Mem ALg  Note\n----------     ---  ----------  -----------     -------  -----------   ----  ----  ---- ---- ----\nlzv 0.1.0           54,950,847  488,436,027     10,385 x 488,446,412      6     5     3 LZ77  62\nlzv 0.1.0           54,950,847  488,436,027     10,385 x 488,446,412     15     6     3 LZ77  26\nlzv 0.1.0           54,950,847  488,436,027     10,385 x 488,446,412      4   2.6     3 LZ77  48\n```\n\n[FastLZ](http://www.fastlz.org/) is a free, open source compression\nlibrary and file compressor by Ariya Hidayat, announced June 12, 2007 with\nno date or version number, and downloaded and tested on June 16, 2007.  \nIt uses byte-aligned LZ77.  The software was released\nas source code only (in C).  For this test it was compiled with MinGW gcc 3.4.5\nas suggested by README.TXT (plus -s to strip debugging info):\n\n```\n  gcc -march=pentium -O3 -fomit-frame-pointer -mtune=pentium 6pack.c fastlz.c -o 6pack -s\n  gcc -march=pentium -O3 -fomit-frame-pointer -mtune=pentium 6unpack.c fastlz.c -o 6unpack -s\n```\n\n6pack and 6unpack are the compressor and decompresser, respectively. They take no options. The compressed file name is stored without a path in the archive.\nsharc 0.9.10 was released Dec. 12, 2013.\n\n[sharc 0.9.11b](https://github.com/centaurean/sharc/releases/tag/sharc-0.9.11-beta), Dec. 14, 2013 has compression levels -c1 and -c2. -c0 selects\nno compression. -c1 selects dictionary encoding. -c2 selects LZP preprocessing\nfollowed by dictionary coding. The program uses the Density\n0.9.12b compression library which is now a separate component.\n\n```\nCompressor    Opt        enwik8      enwik9       prog     Total       Comp  Deco  Mem Alg  Note\n-------       ----     ----------  -----------  -------  -----------   ----  ----  --- ---  ----\nsharc 0.9.6   -c0      63,290,900  625,090,400  25,822 s 625,116,222     14    11   14 Dict  26\nsharc 0.9.6   -c1      58,612,834  554,587,996  25,822 s 554,613,818     19    15   14 Dict  26\nsharc 0.9.10  -c0      61,798,570  610,691,896  11,765 s 610,703,661     13    11    4 Dict  26\nsharc 0.9.10  -c1      57,031,766  538,757,716  11,765 s 538,769,481     14    15    5 Dict  26\nsharc 0.9.11b -c1      61,611,730  608,740,104  81,001 s 608,821,105     12     9    5 Dict  26\nsharc 0.9.11b -c2      53,175,042  494,421,068  81,001 s 494,502,069     15    14    6 LZP   26\n```\n\nalba 0.2, Feb. 6, 2014, adds extreme (e) mode. Modes c and C are unchanged.\n alba 0.5.1, Feb, 18, 2014, adds dynamic block sizing (cd).\n \n\n```\nCompressor    Opt        enwik8      enwik9       prog     Total       Comp  Deco  Mem Alg  Note\n-------       ----     ----------  -----------  -------  -----------   ----  ----  --- ---  ----\nalba 0.1      c        53,643,211  526,932,392   2,950 s 526,935,342    219    10    1 BPE   48\n              c32768   57,419,643  548,461,196   2,950 s 548,464,146    171     8    1 BPE   48\n              C        53,618,232  526,577,702   2,880 s 526,580,582    227    14    1 BPE   48\n              C32768   57,395,415  547,792,821   2,880 s 547,795,701    179    12    1 BPE   48\nalba 0.2      e        53,611,841  526,860,426   3,247 s 526,863,673    819   603    1 BPE   48\nalba 0.5.1    cd       52,728,620  515,760,096   4,870 s 515,764,966    239    10    4 BPE   48\n```\n\n[flzp v1](flzp.zip) is a free,\nopen source file compressor by Matt Mahoney, June 18, 2008.  It uses byte-oriented LZP.\nThe input is divided into blocks such that at least 33 byte values never occur, or 64KB,\nwhichever is smaller, then uses those bytes to code an end of block symbol plus match\nlengths from 2 up to the number of unused bytes - 1.  A match length is decoded by\nfinding the most recent context hash match in a 4 MB rotating buffer and outputting\nthe bytes that follow.  It uses a 1M hash table and an order 4 context hash.\nEach block begins with a 32 byte bitmap to distinguish symbols for matches from literals.\nflzp can be used as a preprocessor to a low order compressor like fpaq0 or ppmd -o3\nto improve compression and speed.\n## .5157 alba\n\n[alba 0.1](http://encode.su/threads/1874-Alba?p=36612&viewfull=1#post36612)\nis a free, open source, experimental file compressor by xezz, Feb. 4, 2014,\nupdated Feb. 5, 2014 to fix a bug in the \"C\" option.\nIt uses byte pair encoding. The option c32768 selects the maximum block size.\nThe default is 4096. It has an \"optimal\" compression mode \"C\".\nIt was tested in Linux by compiling with gcc 4.8.1 -O3.\n## .5229 lzpgt6\n\n[lzpgt](https://github.com/grtamayo/lzpgt) is a free, experimental compressor\nby Gerald R. Tamayo, Aug. 23, 2022. It uses LZP. It outputs a block of bits to flag\nwhether the next byte was predicted correctly, followed\nby a block of the missed literal bytes.\n\nlzpgt6 was released Aug. 9, 2023. It increases the prediction/guess table from 20 to 21 bits and some speed optimizations.\n\n```\nCompressor    Opt        enwik8      enwik9       prog     Total       Comp  Deco  Mem Alg  Note\n-------       ----     ----------  -----------  -------  -----------   ----  ----  --- ---  ----\nlzpgt                  56,590,342  529,409,256  24,576 x 529,433,832      7     9    2 LZP  95\nlzpgt6                 56,113,248  522,877,083  27,136 x 522,904,219      6     5    6 LZP  95\n```\n\n[snappy](http://code.google.com/p/snappy/) 1.0.1 is a free, open source (Apache)\ncompression library for Linux from Google, Mar. 25, 2011. It uses byte aligned LZ77, and is\nintended for high speed rather than good compression. Google uses snappy internally\nto compress its data structures for its search engine.\n\nThe compressed data contains tag bytes such that the low 2 bits indicate literals and matches as follows:\n\n```\n  00 = literal\n  01 = 1 byte match\n  10 = 2 byte match\n  11 = 4 byte match (not used)\n```\n\nA literal of length 1 to 60 is encoded by storing the length - 1 in the upper 6 bits. Longer literals are coded by storing 60..63 in the upper 6 bits to indicate that the length is encoded in the next 1 to 4 bytes in little-endian (LSB first) format. This is followed by the uncompressed literals.\n\nMatches of length 4 to 11 with offsets of 1 to 2047 are encoded using a 1 byte match. The match length - 4 is stored in the middle 3 bits of the tag byte. The most significant 3 bits of the offset are stored in the most significant 3 bits of the tag byte. The lower 8 bits of the offset are stored in the next byte. A match may overlap the area to be copied. Thus, the string \"abababa\" could be written using a literal \"ab\" and a match with an offset of 2 and length of 5. This would be encoded as:\n\n```\n  000001 00  (literal of length 2)\n  01100001   (literal 'a')\n  01100010   (literal 'b')\n  000 001 01 (high bits of offset, match of length 5)\n  00000010   (low 8 bits of offset)\n```\n\nMatches of length 1 to 64 with offsets of 1 to 65535 are encoded using a 2 byte match. The length - 1 is encoded in the high 6 bits of the tag byte The offset is stored in the next 2 bytes with the least significant bit first. Longer matches are encoded as a series of 64 byte matches with a final shorter match of 4 to 63. If the final part of the match is less than 4 then it is encoded as a 60 byte match plus a 4 to 7 byte match.\n\nA 4 byte match allows offsets up to 2<sup>32</sup> - 1 to be encoded\nas with a 2 byte match. The decompresser will decode them but the compressor\ndoes not produce them because the input is compressed in 32K blocks such\nthat a match does not span a block boundary.\n\nThe entire sequence of matches and literals is preceded by the\nuncompressed length up to 2<sup>32</sup> - 1 written in base 128, LSB first,\nusing 1 to 5 digits in the low 7 bits. The high bit is 1 to indicate that\nmore digits follow.\n\nCompression searches for matches by comparing a hash of the 4 current bytes with previous occurrences of the same hash earlier in the 32K block. The hash function interprets the 4 bytes as a 32 bit value, LSB first, multiplies by 0x1e35a7bd, and shifts out the low bits. The hash table size is the smallest power of 2 in the range 256 to 16384 that is at least as large as the input string. As an optimization for hard to compress data, after 32 failures to find a match, the compressor checks only every second location in the input for the next 32 tests, then every third for the next 32 tests, and so on. When it finds a match, it goes back to testing every location.\n\nAs another optimization for the x86-64 architecture, copies of 16 bytes or less are done using two 64-byte assignments rather than memcpy(). To support this, if 15 or fewer bytes remain after a match then they are encoded as literals with no further search.\n\nSnappy compresses from memory to memory rather than from file to file, so it was necessary to write a small test program (below), which was not included in the compressed size. The program loads the input into a string, compresses or decompresses it to a new string, and writes it to output. It gives the best possible compression but is not optimal for speed or memory. With this test, speed is 25 ns/byte for compression and 12 ns/byte for decompression (under 64 bit Linux). In a separate test (not shown), compressing in 32K chucks takes 9 ns/byte with very slightly larger size due to storing the size in each chunk. Decompression was not tested in this mode, but should be twice as fast. Memory usage for the test program is 2 GB to store the input and output, but actual memory usage by the library is at most 32K for the hash table.\n\nThe test program was compiled with g++ 4.4.5 -O3 in 64 bit Ubuntu Linux and linked to Snappy after running \"./configure; make\". Use -DMODE=Compress or -DMODE=Uncompress to create a compressor or decompresser respectively.\n\n```\n#define NDEBUG 1  // turn off debugging checks\n#include \"snappy.h\"\n#include <stdio.h>\nint main() {\n  std::string input, output;\n  int c;\n  while ((c=getchar())!=EOF) input+=char(c);  // read from stdin\n  snappy::MODE(input.c_str(), input.size(), &output);  // MODE = Compress or Uncompress\n  fwrite(output.c_str(), 1, output.size(), stdout);  // write to stdout\n  return 0;\n}\n```\n\nFor testing, I compiled with gcc 4.4.0 -s -O2 -march=pentiumpro\n-fomit-frame-pointer. I used the recommended compression options\n\"5000 4096 200 3\" and did not try to find a better combination.\nThe options say to use a maximum block size of 5000, a hash table\nsize of 4096 (it is recommended to be 5% to 20% smaller than the block\nsize), a maximum of 200 different byte values per block, and do not\nreplace pairs that occur less than 3 times.\n The programs were tested by compiling with g++ 4.4.0 -O2 -s -march=pentiumpro -fomit-frame-pointer\nunder Windows Vista on a 2.0 GHz T3200.\n \n\n```\nCompression         Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note\n-------         ----------  -----------  -----------  -----------  ----- -----  --- --- ----\nbpe2 v1         55,390,822  545,319,505      1,621 s  545,321,126   2785   228  0.5 Dict 26\nbpe2 v2         55,389,832  545,268,425      1,635 s  545,270,060   1257   229  0.5 Dict 26\nbpe2 v3         55,289,197  542,748,980      2,979 s  542,751,959    518   132  0.5 Dict 26\n```\n\n## .5326 kwc\n\n[kwc](http://www.metacompressor.com/download/kwc.zip)\n[(discussion)](http://encode.dreamhosters.com/showthread.php?p=10832#post10832)\nis a free GUI file compressor by sportman, Jan. 18, 2010. The input is divided into\nstrings of 6 bytes each, and each value is replaced with a dictionary code. The dictionary\nsize is not bounded, so usage increases with the size and randomness of the input.\nenwik9 uses 668 MB for compression and 333 MB for decompression.\n## .5427 bpe2\n\n[bpe2 v1](bpe2.cpp) is a free, experimental, open source (public domain)\nfile compressor by Will, Jan. 15, 2010. It uses byte pair encoding. It divides\nthe input into blocks of 8192 bytes which are compressed independently. A block\nis compressed by finding the byte pair which occurs most frequently and a byte\nvalue which never occurs in the block, and then substituing that byte value\nfor each occurrence of the pair. The byte pair and its replacement are appended\nto the block as a 3 byte header. The process is repeated until either there\nare no unused byte values left, or there is no pair that occurs at least 4 times.\nThe block is output with an additional 2 byte header to indicate its size.\n[bpe2 v2](bpe2v2.cpp), Jan. 15, 2010,\nuses a faster algorithm to find the most\nfrequent byte pair during compression.\n[bpe2 v3](bpe2v3.cpp), Feb. 12, 2010,\nhas some optimizations.\n[(discussion)](http://stackoverflow.com/questions/2093223/optimizing-byte-pair-encoding)\n## .5586 fpaq0f2\n\n[fpaq](index.html#fpaq0) is\na free, experimental command line file compressor with source code\n(in assembler) by Nikolay Petrov, Feb. 20, 2006.  It is a faster\nimplementation of fpaq0 by Matt Mahoney (Sept. 3, 2004) maintaining\narchive compatibility.  fpaq is an order-0 arithmetic coder which\nmodels independent, identically distributed (i.i.d.) characters, and is not\nintended as a general purpose compressor.  Its purpose is\nto test the efficiency of different arithmetic coding algorithms.  \nThere are several variants.\n\n```\nCompressor     enwik8      enwik9    Comp  Decomp  Author                  Date\n----------   ----------  ----------  ----  ----    --------------          ----\nfpaq0        63,391,013  641,421,110  336   351    Matt Mahoney            Sep 03 2004\nfpaq1        63,502,003               477   489    Matt Mahoney            Jan 10 2006\nfpaq0b       63,375,460               457   437    Fabio Buffoni           Jan 10 2006\nfpaq0s       63,375,457               427   417    David A. Scott          Jan 16 2006\nfpaq         63,391,013  641,421,110  255   246    Nicolay Petrov          Feb 20 2006\nfpaq0p       61,457,810  622,237,009  131   131    Ilia Muraviev           Apr 15 2007\nfpaq02       63,501,997  644,561,596 1345  1325    David Anderson          May 27 2007\nfpaqa        61,340,408  620,681,885  262   237    Matt Mahoney            Dec 15 2007\nfpaqb        61,270,458  620,278,361  264   171    Matt Mahoney            Dec 20 2007\nfpaq0m       61,389,879  621,285,504  153   135    Ilia Muraviev           Dec 20 2007\nfpaq0mw      61,271,869  618,959,309  455   457    Eugene Shelwien         Dec 21 2007\nfpaqc        61,270,455  620,278,358  252   177    Matt Mahoney            Dec 24 2007\nfpaq0pv2     61,280,398  620,379,449  116   133    Ilia Muraviev           Dec 26 2007\nfpaq0r       61,234,684  620,169,855  129   142    Alexander Rhatushnyak   Jan 09 2008\nfpaq0rs      61,202,171  619,839,546  139   138    Alexander Rhatushnyak   Jan 09 2008\nfpaq0f       58,088,230  581,053,251  265   251    Matt Mahoney            Jan 28 2008\nfpaq0f2      56,916,872  558,645,708  222   207    Matt Mahoney            Jan 30 2008\nfpaq0pv3     61,457,810  622,237,009  103   119    Nania Francesco Antonio Apr 04 2008\nfpaq0pv4     61,457,810  622,237,009   70    79    Eugene Shelwien         Apr 06 2008\nfpaq0pv4nc   61,350,834  621,169,159   64    69    Eugene Shelwien         Apr 06 2008\nfpaq0pv4nc0  61,287,662  620,506,072   68    74    Eugene Shelwien         Apr 06 2008\nfpaq0pv5     61,457,810  622,237,009   81    87    Nania Francesco Antonio Apr 06 2008\nfpaq0pv4a    61,457,810  622,237,009   70    75    Eugene Shelwien         Apr 07 2008\nfpaq0pv4anc  61,323,986  621,169,159   64    65    Eugene Shelwien         Apr 07 2008\nfpaq0pv4anc0 61,287,662  620,506,072   66    66    Eugene Shelwien         Apr 07 2008\nfpaq0pv4b1   61,287,234  620,488,244   56    60    Eugene Shelwien         Apr 18 2008\n```\n\nfpaq0 uses a 32-bit carryless arithmetic coder to code binary decisions and output one byte at a time. fpaq1 uses a 64 bit coder. fpaq0b uses a 32 bit coder but counts carries and outputs a bit at a time to achieve greater internal precision. fpaq0s improves on fpaq0b by using the compressed EOF to encode the uncompressed EOF, unlike the other models which code an extra bit for each byte to indicate the end. fpaq02 extends this idea to 64 bits. All programs except fpaq are C++ source code and compiled as follows with MinGW 3.4.2 (where %1 is the program name):\n\n```\ng++ -Wall %1.cpp -O2 -Os -march=pentiumpro -fomit-frame-pointer -s -o %1.exe\n```\n\n[fpaq0p](http://www.encode.su/downloads/fpaq0p.zip) by Ilia\nMuraviev, Apr. 15, 2007, uses an adaptive order 0 model.  Instead of keeping\na 0,1 count for each context, it keeps a probability and updates it by\nadjusting by 1/32 of the error.  This is faster because it avoids a division\ninstruction.\n\n[fpaqa](fpaqa.zip) by\nMatt Mahoney, Dec. 15, 2007, is the first implementation of Jarek Duda's\nasymmetric binary coder, described in section 3 of\n[Optimal encoding on discrete lattice with translational invariant constrains\nusing statistical algorithms](http://uk.arxiv.org/PS_cache/arxiv/pdf/0710/0710.3861v1.pdf), 2007.\n\nThe model is based on fpaq0p (adaptive order 0), but with probabilities modeled with 16 bits resolution (instead of 12) to improve compression. The source (GPL) can be compiled with -DARITH to substitute the arithmetic coder from fpaq0 and fpaq0p for the asymmetric coder.\n\nAn asymmetric coder has a single N-bit integer state variable x, as opposed to\ntwo variables (low and high) in an arithmetic coder, which allows a lookup\ntable implementation.  In fpaqa, N=10.  A bit d (0 or 1) with probability q = P(d = 1)\n(0 < q < 1, a multiple of 2<sup>-N</sup>) is coded:\n\n```\n  if d = 0 then x := ceil((x+1)/(1-q)) - 1\n  if d = 1 then x := floor(x/q)\n```\n\nTo decode, given x and q\n\n```\n  d = ceil((x+1)*q) - ceil(x*q)  (1 if fract(x*q) >= 1-q, else 0)\n  if d = 0 then x := x - ceil(x*q)\n  if d = 1 then x := ceil(x*q)\n```\n\nx is maintained in the range 2\nTo reduce the size of the coding tables, q is quantized to R=7 bits on a nonlinear scale with closer spacing near 0 and 1. The quantization is such that ln(q/(1-q)) is a multiple of 1/8 between -8 and 8.\n\nIn the source, N, R, and B are adjustable parameters up to N=12, R=7.\nLarger values improve compression at the expense of speed and memory.\nfpaqa uses 2<sup>N+R+2</sup> + 5*B/4 bytes for compression\nand 2<sup>N+R+1</sup> bytes for decompression.\n\n[fpaqb](fpaqb.zip)\n(Matt Mahoney, Dec. 17, 2007, updated to ver 2 on Dec. 20, 2007)\nis a revision of fpaqa, using the same model, but using an asymmetric\ncoder that uses direct calculations in place of lookup tables to update\nthe state.  This allows higher precision to improve compression (eliminating\na 0.03% penalty), saving memory, and allowing bytewise I/O (x in range\n2<sup>N</sup> to 2<sup>N+8</sup>-1 for N=12).  Compression\nis about the same speed as fpaqa but decompression is 28% faster.\nVer. 2 is faster but maintains archive compatibility with ver. 1.\n\n[fpaq0m](http://www.encode.su/downloads/fpaq0m.zip)\nby Ilia Muraviev, Dec. 20, 2007,\nuses arithmetic coding and 2 order 0 models averaged\ntogether, one with fast update (rate 1/16) and one slow (1/64).\n\n[fpaq0mw](http://shelwien.googlepages.com/fpaq0mw.rar)\nby Eugene Shelwien, Dec. 21, 2007, modifies fpaq0m by using a weighted\nmix of a fast (1/16) and slow (1/256) adapting order 0 model, where\nthe weight is adjusted dynamically to favor the better model.\n\n[fpaqc](fpaqc.zip)\n(Matt Mahoney, Dec. 24, 2007) is fpaqb with some optimizations\nto the asymmetric coder.\n\n[fpaq0pv2](http://www.encode.su/downloads/fpaq0pv2.zip)\n(Ilia Muraviev, Dec. 26, 2007) is a speed optimized version of fpaq0p\nwith arithmetic coding.\n\n[fpaq0r](fpaq0r.zip)\nby Alexander Rhatushnyak, Jan. 9, 2008, is an order 0 model with arithmetic\ncoding.  The model is tuned for better text compression.  When compiled\nwith -DSLOWER (fpaq0rs.exe), the arithmetic coder uses higher precision\nfor better compression with a small speed penalty.\n\n[fpaq0f](fpaq0f.zip)\nby Matt Mahoney, Jan. 28, 2008, uses an adaptive order 0 model which\nincludes the bit history (as an 8 bit state) in each context.\n(It is controversial whather this is really \"order 0\").\nIt uses arithmetic coding with 16 bit probabilities (rather than 12 bits).\n\n[fpaq0f2](fpaq0f2.zip)\nby Matt Mahoney, Jan. 30, 2008, uses a simplified bit history consisting\nof just the last 8 bits, plus some minor improvements.\n\n[fpaq0pv3](http://www.winturtle.netsons.org/fpaq0pv3.zip)\nby Nania Francesco Antonio, Apr 04, 2008, is compatible with fpaq0p but 20-30% faster.\n\n[fpaq0pv4](http://shelwien.googlepages.com/fpaq0pv4.rar)\nincluding fpaq0pv4nc and fpaq0pv4nc0, are speed optimizations by Eugene\nShelwien, Apr. 6, 2008, as discussed\n[here](http://encode.su/forums/index.php?action=vthread&forum=1&topic=671#msg9437).\nfpaq0pv4 is compatible with fpaq0p but faster.  The nc and nc0 variants dispense with the\nextra EOF flags in each byte.\n\n[fpaq0pv5](http://www.winturtle.netsons.org/fpaq0pv5.zip)\nby Nania Francesco Antonio, Apr 6, 2008, is a modification to fpaq0pv4.\n\n[fpaq0pv4a](http://shelwien.googlepages.com/fpaq0pv4A.rar)\nincluding fpaq0pv4anc and fpaq0pv4anc0 are bug fixes to fpaq0pv4 by\nEugene Shelwien, Apr. 7, 2008, as discussed above.\n\n[fpaq0pv4b](http://shelwien.googlepages.com/fpaq0pv4b.rar) by\nEugene Shelwien, Apr. 18, 2008, replaces the arithmetic coder with\nsh_v1m port (uses carries), Windows I/O, and other optimizations as discussed\n[here](http://encode.su/forums/index.php?action=vthread&forum=1&topic=679#msg9654).\nThe Intel-compiled .exe only runs on Intel machines. I tested\n[fpaq0pv4b1](http://shelwien.googlepages.com/fpaq0pv4B1.exe) which was\npatched on May 19, 2008 to run on AMD machines.\n[ghost]\n\nThe program takes 2 arguments. The first is the number of iterations. The second is the maximum string size to encode.\n\n```\nresults for enwik8:\ncommand: python ghost-compress.py enwik8 750 6\ncompressed size: 55,357,196 bytes\ncompression time: 35 hours\nmax memory usage: 65 GB\ndecompression time: 42s\nmax memory usage: 168 MB\n\nresults for enwik9:\ncommand: python ghost-compress.py enwik9 456 5\ncompressed size: 568,004,779 bytes\ncompression time: 48 hours\nmax memory usage: 88 GB\ndecompression time: 4m 05s\nmax memory usage: 2485 MB\nppp enwik9 > enwik9.ppp     (compress)\n  ppp -d enwik9.ppp > enwik9  (decompress)\n```\n\nThe original code opens both files in text mode, which does not work in Windows. For testing, I modified 3 lines of code to open the input and output files in binary mode as follows:\n\n```\n  #include <fcntl.h>  // added\n  setmode(fileno(stdout), O_BINARY);  // added\n  FILE *f = fopen(*p, \"rb\");  // changed \"r\" to \"rb\"\n```\n\nI compiled using gcc 3.4.2\nThe program uses a Windows GUI when run with no arguments. It was tested with command line arguments under Wine 1.6 in Ubuntu.\n\n```\nCompressor Opt     enwik8      enwik9           prog     Total       Comp  Deco  Cmem Dmem Alg  Note\n-------    ---   ----------  -----------      -------  -----------   ----  ----  ---- ---- ---  ----\nksc         1    79,706,130                                          3250  2790    40  265 SR   48\n            2    67,676,824                                          3730  1480    40  227 SR   48\n            3    62,570,897                                          8560  1800    59  273 SR   48\n            4    59,511,259                                         32780  6670    62  220 SR   48\n            4                580,557,413     13,507 x 580,570,920   40050  7917   155 1700 SR   48\n```\n\nlzp2 0.7c was released Oct. 10, 2009. Run times are dominated by disk access, not included below.\n\n```\nCompressor     enwik8      enwik9           prog     Total       Comp  Deco  Mem Alg  Note\n-------      ----------  -----------      -------  -----------   ----  ----  --- ---  ----\nlzp2 0.1     74,358,722  655,709,055      5,855 xd 655,714,910     11     9   15 LZP  26\nlzp2 0.7c    67,909,076  598,076,882     40,819 x  598,117,701     11     8   15 LZP  26\n```\n\nNTFS disk compression is used in Microsoft\nWindows when the \"compress files to save disk space\"\ncheckbox is checked in the folder properties dialog box. Disk compression was\nintroduced in NTFS v1.2 in mid 1995 according to\n[Wikipedia](http://en.wikipedia.org/wiki/NTFS#File_compression).\nThe compression format is called LZNT1. The algorithm is propretary. However, it was\n[reverse engineered](http://www.nf-team.org/drmad/zf/zf5/zf5_025.htm)\n(in Russian, see also [here](http://encode.su/forum/showthread.php?t=260)).\nThe algorithm is LZSS (similar to lzrw1).\nThe format consists of groups of 8 symbols each preceded by 8 flag bits packed\ninto a byte. A 0 bit indicates a literal symbol, which is decoded by copying it.\nA 1 bit indicates a 2 byte offset-length pair which is decoded by going back 'offset'\nbytes in the output and copying the next 'length'+3 bytes. An offset-length pair\nuses a variable number of bits allocated\nto the offset (from 4 to 12) depending on the position in the file, and any\nremaining bits allocated to the length of the match. A 12 bit offset would\ncorrespond to a 4 KB block on disk.\n\nI tested by copying enwik9 between folders with the compression turned on in one folder, and compared with times to copy between two folders both with compression turned off. I tried each copy twice and took the second time, which was at most 1 second faster than the first copy. I used the test machine in note 26 running Windows Vista Home Premium SP1 32 bit with 3 GB memory and a 200 GB disk between folders on the same partition. Copying between two uncompressed folders takes 41 seconds. Copying to a compressed folder takes 51 seconds, or a difference of 10 seconds. Copying from a compressed folder takes 35 seconds. I estimated 9 seconds for decompression by assuming that copying the compressed file directly would take 26 seconds based on its size of 636 MB. (This is probably wrong because the file would be cached in memory uncompressed, but the alternative is a negative time for decompression. Copying either the compressed or uncompressed file to NUL: takes 2 seconds on the second try).\n\nTimes were recorded with a watch because timer 3.01 will not time built-in commands\nlike 'copy'. Task Manager does not show any processes consuming CPU time or memory\nduring copying. However, memory use should be insignificant (under 16 KB) for\nLZSS with 4 KB blocks. Sizes are as reported by right clicking on the compressed\nfile in Explorer as 'size on disk'. The size of the decompression program is not known.\n## .6373 shindlet\n\n[shindlet](http://rapidshare.de/files/22175688/range.tgz.html)\n[(mirror)](http://board.flatassembler.net/topic.php?p=27579#27579) is\na series of 3 free command line file compressors by Piotr Tarsa.  All are\norder-0 arithmetic coders with identical models written in assembler (included).\nThe three variants are fs (frequency sorting), bt (binary tree), and\nsl (linear search).  All three produce identical sized compressed files.\nIn addition, the compressed output of bt and sl are identical.\nResults for all 3 variations are below.  Comp and Decomp show global\ntimes including disk I/O in ns/byte, with CPU (process) times in parenthesis.\nDate is the latest program timestamp in the distribution, not the release date.\n\n```\nCompressor       Date        enwik8       enwik9     prog     Total size     Comp      Decomp\n-----------  ------------  -----------  ----------  -------   -----------  ---------  ---------\nshindlet_fs  May  7, 2006  62,890,267  637,390,277  1,275 xd  637,391,552  185 (113)  123 (103)\nshindlet_bt  May 27, 2006  62,890,267  637,390,277  1,387 xd  637,391,664  163  (85)  118  (96)\nshindlet_sl  Apr 12, 2006  62,890,267  637,390,277  2,415 xd  637,392,692  166  (94)  121 (102)\n```\n\n[compact](https://minnie.tuhs.org/cgi-bin/utree.pl?file=2.11BSD/src/old/compact)\n[(man page)](https://www.freebsd.org/cgi/man.cgi?query=compact&apropos=0&sektion=0&manpath=2.10+BSD&arch=default&format=html)\nis a file compressor by Colin L. Mc Master, Feb. 28, 1979.  It was written in K&R C for\nVAX/PDP11 and SUN under Berkeley UNIX.  It uses adaptive order-0 Huffman coding.\nThe (separate) decompression program rebuilds the Huffman tree, so it need not be transmitted.\n\nNeither program takes options.  compact deletes the input file and creates an output file\nwith a .C extension.  uncompact deletes the compressed file and restores the original.\ncompact was later superceded by *compress*, which gives better compression.\n\nFor this test, compact was compiled using the provided Makefile and tested under\nUbuntu Linux.  Minor source code corrections were needed to compile under gcc.\nHowever, the decompresser size is based on the original code.  A port to Windows would\nbe possible but would require more source code changes.\n \n\n```\nCompressor                   enwik8       enwik9     prog     Total size   Comp  Deco  Mem  Alg Note\n-----------                -----------  ----------  -------   -----------  ----  ----  ---  --- ----\nTinyLZP 0.1                79,220,546  694,274,932   2,811 s  694,277,743    58    46   10  LZP  26\nTinyLZP-x86-SSE2           79,220,546  694,274,932   2,811 s  694,277,743    32    38   10  LZP  26\n```\n\n Both programs use a move-to-front algorithm with the queue position\nencoded using an interleaved Elias Gamma code. The position of the\ncurrent byte in the queue (1..256) is encoded by dropping the leading 1 bit,\npreceding each of the remaining bits with a 0 bit, then terminating with\na 1 bit. After encoding, the byte value is moved to the front of the queue.\nsmile256 also encodes EOF as 257, resulting in a file that\nis usually 1 byte larger than smile_e.\n \n\n```\nCompressor                   enwik8       enwik9     prog     Total size   Comp  Deco  Mem  Alg Note\n-----------                -----------  ----------  -------   -----------  ----  ----  ---  --- ----\nsmile_e/smile_d             71,154,788  695,562,502  207 xd   695,562,709 10517 10414  0.6  MTF  26\nsmile256                    71,154,789               256 x                11190 10840  0.6  MTF  26\n```\n\n## .6942 TinyLZP\n\n[TinyLZP](http://encode.su/threads/1619-TinyLZP-A-very-simple-LZP-compressor)\nis a free, open source (GPL v3) file compressor by David Werecat,\nOct. 12, 2012. It uses LZP and takes no options.\nThe first entry is compiled from source using \"cl /O2 tinylzp.c /I.\"\nusing Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86\nand tested on a 2.0 GHz T3200 under 32 bit Vista.\nThe second entry, TinyLZP-x86-SSE2.exe, is supplied and requires\nMSVCR110.dll (Visual Studio 2012 C++ runtime) to run.\n## .6955 smile\n\n[smile](http://256bytes.untergrund.net/demo/398) (Nov. 5, 2004)\nand [smile256](http://256bytes.untergrund.net/demo/399) (Dec. 5, 2004)\n[(discussion)](<http://encode.su/threads/1624-Win32-binary-compressor-with-smallest-native-(EXE)-footprint-not-best-ratio-question>)\nare free, open source file compressors by Andrei Frolov.\nThese programs are unique for their small executable size.\nsmile consists of two programs: a 250 byte compressor, smile_e.com\nand a 207 byte decompresser, smile_d.com. smile256 is both a compressor\nand a decompresser in 256 bytes. This includes code to parse the command\nline and open the input and output files.\nSource code is in 16 bit assembler for DOS.\nProgram size is given for the uncompressed .com files because zip\nmakes them larger.\n## .7594 barf\n\n[barf](barf.html) is a free,\nopen source file compressor by Matt Mahoney, Sept. 21, 2003.  It was written\nas a joke to debunk claims of recursive compression.  The algorithm is as\nfollows:\n\n```\nPass    enwik8      enwik9    size (zip)   enwik9+prog  Comp (wall) Decomp  Mem Alg   Filename\n----  ----------  ----------- -----------  -----------  ----------  ------- --- ----  --------\n1     76,450,126  763,918,762   983,782 s  764,902,544   315 (330)   30 (73)  4 LZ77  enwik9.x\n2     76,074,327  758,482,743   983,782 s  759,466,525   439 (462)   23 (60)  4 LZ77  enwik9.x.x\n3     76,074,326  758,482,742   983,782 s  759,466,524   488 (551)   18 (44)  4 copy  enwik9.x.x.x9v\n```\n\nA similar program, barfest.exe, compresses the million random digits file to\n1 byte, rather than the Calgary corpus.  The decompresser size is\n455,755 bytes (zipped).\n \n\n```\nhipp5819   enwik8    MB Mem  Comp (ns/byte)\n-------  ----------  ------  ----\n/o5      22,390,366  248.5  ~3710     \n/o8      20,555,951  719.5  ~4300\n```\n\n Unfortunately, the compressor will not accept truncated XML files such as this benchmark.\nIt can be made to work by appending the following 38 bytes to enwik8 or enwik9\nto create a properly formed XML file (a trailing newline is optional but was not used):\n \n\n```\n\"</text></revision></page></mediawiki>\nCompression                      Compressed size      Decompresser  Total size   Time (ns/byte)\nProgram           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem\n-------           -------                     ----------  -----------  -----------  -----------  ----- -----  ---\nxcmill 0.8        -w -P -9 -m800              26,579,004 (230,934,622)  114,764 xd (231,049,386)   616 (530)  800\nxcmill 0.9.1      -w -P -9 -m1700             26,579,004 (230,914,289)  108,845 xd (231,023,134)   711        984\n```\n\n In theory, using no compression (-N) would allow XMill to be used as a preprocessor to other\ncompressors.  However, the decompresser will not accept either enwik8 or enwik9 (with closing\ntags appended) if processed with -N (reports \"corrupt file\").\n \n\n```\nProgram    enwik8     Comp  Deco  Mem  Alg\n-------  ----------   ----  ----  ---  ---\nlzp1     56,013,656     23    20  153  LZP\nlzp2     40,350,594     80        280  LZP\nlzp3o2   33,041,439    230   270  151  LZP\n```\n\n All programs report \"malloc failed\" on enwik9.  The LZP algorithms\nuse very little memory themselves, but these implementations allocate\ninput and output buffers all at once.  This fails for enwik9 because of\nthe 2 GB process limit in Windows.\n lzp1 is both a compressor and decompresser.  To decompress, use -d as\nthe third argument.  lzp2 is a compressor only.  There is a source code\ndecompresser \"lzp2d\" but I was unsuccessful in compiling it.\nIt allows an unexplained option \"HuffType\" which I did not experiment with.\nlzp3o2 has a separate decompresser \"lzp3o2d.exe\" included in the distribution.\n \n\n```\nMay 10 2006 - benchmark began with 1 month of testing about 2 compressors per day.\nJun 10 2006 - began test data analysis.\nJun 14 2006 - updated xml-wrt 2.0 14.06.06 | ppmonstr.\nJun 17 2006 - reorganized website from 1 big page to 4 smaller pages.\nJun 19 2006 - added xml-wrt 2.0 19.06.06 (standalone LZMA mode).\nJun 20 2006 - added ocamyd 1.65 LTCB 1.0.\nJun 21 2006 - updated TC 5.0 to dev 4 (compression unchanged but faster).\nJul 19 2006 - updated TC 5.0 to dev 9, added dark 0.32b.\nJul 20 2006 - added arbc2z.\nJul 21 2006 - added TarsaLZP (July 4 2006).\nJul 22 2006 - added uda 0.300.\nJul 23 2006 - verified uda 0.300 decompression.\nJul 24 2006 - updated TC 5.0 to dev 11.\nJul 29 2006 - added CTW 0.1.\nAug 01 2006 - updated TarsaLZP (July 30 2006), added ppmvc v1.1.\nAug 06 2006 - added the Hutter Prize, renamed Large Text Compression Benchmark to Human Knowledge Compression Contest,\n              added rules for the Hutter Prize, and updated rationale to add a section on AIXI.\nAug 07 2006 - added link to paq8f, updated prize formula (Z might not decrease), and that prize committee members\n              are not elibible for prize money.  Added logo.  Minor edit to rationale.\nAug 08 2006 - the prize fund (Z) does not decrease.\nAug 11 2006 - added a lexcial and string repetition analysis to the data study.\nAug 13 2006 - typo in Rationale.\nAug 14 2006 - updated dark v0.40. Edited Rationale (AIXI, compression does not seem like AI, lossy compression).\nAug 16 2006 - raq8g and durilca 0.5(Hutter) submitted for Hutter prize, neither verified yet.\nAug 17 2006 - verified durilca 0.5(Hutter) claim.  Posted raq8g.exe for Windows.\nAug 18 2006 - verified raq8h -7 on enwik8 under Windows.  Tested paq8f -8 on enwik8 (not verified).\n              Reported raq8h -8 result (Linux).\nAug 19 2006 - updated ha, added Info-ZIP, ESP.  Clarified rules 5 and 6.\nAug 20 2006 - Removed rules and results for the Hutter prize.  These may be found on the Hutter Prize website.\n              Updated ha and Info-ZIP.\nAug 22 2006 - added paq8hp1.  Updated Info-ZIP.  Added submission times and unzipped .exe sizes for Hutter prize candidates.\nAug 23 2006 - updated paq8hp1 for enwik9 -8 (compress only).  Tuned xml-wrt|ppmonstr for enwik8 at 2 GB.  Added durilca4linux.\nAug 26 2006 - updated dark 0.46.  Fixed link to durilca4linux.  Posted enwik8.bz2 and enwik9.bz2 on the data page.\nAug 28 2006 - added paq8hp2 (enwik8, 1 GB, not checked).  Updated ppmonstr, xmlwrt|ppmonstr, slim, and ash for 2 GB memory.\nAug 29 2006 - verified paq8hp2 for enwik8 (1 GB and 2 GB).\nAug 31 2006 - added bbb.\nSep 01 2006 - updated bbb, TarsaLZP, paq8hp2 (as a preprocessor).\nSep 02 2006 - corrected error in lexical analysis table on data page (found by Szymon Grabowski).\nSep 03 2006 - added paq8hp3 -7 for enwik8 (Hutter prize candidate, verified).\nSep 05 2006 - updated paq8hp3 (enwik9 -8, not verified).\nSep 10 2006 - updated paq8hp4 (verified for enwik8), fixed links to PX and pimple.\nSep 11 2006 - updated paq8hp4 for enwik9 (compression only), added paq1 and expanded PAQ series documentation.\nSep 12 2006 - minor edits in paq8hp1, raq8g descriptions.\nSep 13 2006 - updated paq8hp2 for enwik9.\nSep 14 2006 - updated xml-wrt 3.0.\nSep 15 2006 - updated xml-wrt 3.0|ppmonstr.\nSep 20 2006 - updated paq8hp5 -7 enwik8.  Verified paq8hp4 -8 enwik9.\nSep 21 2006 - updated paq8hp5 -8 enwik8.\nSep 23 2006 - updated paq8hp5 -8 enwik9 (not verified).\nSep 24 2006 - added QuickLZ.\nSep 29 2006 - added fpaq0x, fpaq0s2.\nSep 30 2006 - clarified submission dates for paq8hp2 through paq8hp5.  Posted paq8hp2 source code.\nOct 01 2006 - updated fpaq0x1a, fpaq0s2b, tc 5.1 dev 1.\nOct 02 2006 - updated tc 5.1 dev 2.\nOct 06 2006 - posted paq8hp3 source code (now top ranked).  Added fpaq0x1b.\nOct 08 2006 - added fpaq0s3.\nOct 10 2006 - posted paq8hp4 source code (now top ranked).\nOct 12 2006 - added fpaq0s4.\nOct 13 2006 - added tc 5.1 dev 5.\nOct 15 2006 - verified paq8hp5 -8 enwik9 decompression.  Added fpaq0s5.\nOct 16 2006 - added durilca4linux_2 (now top ranked, not yet verified for enwik9).\nOct 18 2006 - updated duricla4linux_2 (-t2(11) option).\nOct 21 2006 - added fpaq2.\nOct 22 2006 - updated QuickLZ 0.9.\nOct 27 2006 - posted paq8hp5 source code (now ranked #2).\nOct 30 2006 - updated fpaq0s6.\nNov 03 2006 - mirrored enwik8.bz2 and enwik9.bz2 to mattmahoney.net/text\nNov 05 2006 - updated paq8hp6.  Linked to FV results on data page.\nNov 06 2006 - verified paq8hp6 -7 enwik9 decompression.\nNov 07 2006 - updated fastari.\nNov 10 2006 - added PeaZip.\nNov 15 2006 - added paq8j.\nNov 17 2006 - added paq8ja.\nNov 20 2006 - added fpaq3.\nNov 22 2006 - added paq8jb.\nNov 29 2006 - added paq8jc.\nDec 02 2006 - added fpaq3b.\nDec 08 2006 - added paqh8p7a (enwik8 only), posted paq8hp6 source.\nDec 10 2006 - updated paq8hp7a for enwik9 (not verified).\nDec 12 2006 - added paq8hp7.\nDec 13 2006 - updated paq8hp6 -8 enwik9.\nDec 17 2006 - posted enwik8.pmd and enwik9.pmd (PPMD var. J format).\nDec 21 2006 - added fpaq3c.\nDec 24 2006 - added quad v1.01a, tc 5.1 dev 7.\nDec 28 2006 - added fpaq3d.\nJan 01 2007 - added paq8jd (enwik8 -7).\nJan 02 2007 - updated paq8jd -8 enwik8 (not verified).\nJan 08 2007 - added hook v0.2.\nJan 11 2007 - added hook v0.3.\nJan 12 2007 - added hook v0.3a.\nJan 13 2007 - added tc 5.1dev7x.  Fixed hook.zip archive.\nJan 15 2007 - posted paq8hp7 source code.  Added hook v0.4.\nJan 17 2007 - completed dmc and Info-Zip 2.3.1.\nJan 19 2007 - added paq8hp8.\nJan 22 2007 - added hook v0.5b.\nJan 27 2007 - added chile 0.4.\nFeb 03 2007 - added ocamyd-1.66.final (merged with ocamyd LTCB)\nFeb 07 2007 - added hook v0.6.\nFeb 08 2007 - added hook v0.6b, quad v1.04a, tc 5.2 dev 2.\nFeb 09 2007 - corrected error in tc 5.2 dev 2.\nFeb 12 2007 - added ccm_extra 1.03a.\nFeb 14 2007 - added hook v0.6c.\nFeb 15 2007 - added paq8k -8 enwik8 (not verified).\nFeb 20 2007 - added paq8hp9 -7 enwik8 (verified).\nFeb 22 2007 - updated paq8hp9 -7 enwik9.\nFeb 23 2007 - added link to paq8hp9any (revised paq8hp9, not tested), added quad 1.07b, ccm 1.1.1a.\nMar 02 2007 - added ccm 1.1.2a.\nMar 06 2007 - added LZPXj 1.2h.\nMar 10 2007 - added paq8l enwik8.\nMar 11 2007 - added hook v0.7.\nMar 13 2007 - added hook v0.7b.\nMar 14 2007 - added quad 1.08.\nMar 17 2007 - added hook v0.8.\nMar 18 2007 - added hook v0.8b.\nMar 19 2007 - added hook v0.8c.\nMar 21 2007 - added hook v0.8d, FreeArc 0.36.\nMar 24 2007 - added quad 1.10.\nMar 27 2007 - added paq8hp10 -7 enwik8, posted paq8hp9 source code, added hook v0.8e, M99.\nMar 28 2007 - corrected M99 enwik8 result, updated FreeArc description, removed unsupported quad versions from main table.\nMar 31 2007 - added paq8hp10any -8 enwik8.\nApr 01 2007 - added dark 0.51, opendark.\nApr 02 2007 - updated paq8hp10any -8 enwik9 (decompression not verified), added DGCA 1.10.\nApr 05 2007 - added quad 1.11, quad 1.11HASH2, ccm 1.20a, updated FreeArc description.\nApr 06 2007 - added hook v0.9.\nApr 08 2007 - added freehook 0.2, ccm 1.20d.\nApr 09 2007 - added xmill 0.9.1 (fails), barf, quad 1.12.\nApr 10 2007 - added hook 0.9b, freehook 0.3.\nApr 19 2007 - added M99 v2.1, QuickLZ 1.20 and 1.30beta, lzpm 0.02, tornado 0.1.\nApr 22 2007 - added thor 0.94a.\nApr 23 2007 - added ccm (ccmx) 1.21.\nApr 27 2007 - added slug 1.1b.\nApr 30 2007 - added paq8hp11 -7 enwik8.  Posted paq8hp10any source code.\nMay 03 2007 - added paq8hp11any -8 enwik8, fpaq0p.\nMay 05 2007 - added lzpm 0.03 and 0.04.  Fixed misleading description of DMC algorithm in hook.\nMay 08 2007 - added lzc 0.01, hook0.9c.\nMay 09 2007 - added pucrunch, TarsaLZP May 6 2007, thor 0.95, srank 1.1.\nMay 10 2007 - added paq8hp11any -8 enwik9 (decompression not verified).\nMay 11 2007 - added lzc 0.03, updated table description (time, memory, algorithms).\nMay 14 2007 - added paq8hp12 -7 enwik8.\nMay 16 2007 - added uc2, lzc 0.04.\nMay 18 2007 - added BriefLZ 1.05.\nMay 20 2007 - added paq8hp12any -8 enwik8/9 (decompression not verified), lzpm 0.06.  Updated times in main table to process times.\nMay 21 2007 - added paq8hp12any -7/-8 enwik8 (decompression verified), 7zip 4.46a.\nMay 26 2007 - added lzc 0.05b.\nMay 29 2007 - added fpaq02.\nJun 01 2007 - added turtle 0.01.\nJun 02 2007 - added turtle 0.02.\nJun 05 2007 - added turtle 0.03.\nJun 08 2007 - added turtle 0.04.\nJun 12 2007 - posted paq8hp11any source code, added turtle 0.05.\nJun 16 2007 - added TarsaLZP ver. Jun 17 2007, FastLZ ver. Jun 12 2007, pim 2.01.\nJun 23 2007 - added turtle 0.07.\nJul 24 2007 - added lpaq1, pim 2.04b, TarsaLZP Jul 18 2007, posted paq8hp12any source code.\nJul 30 2007 - added TarsaLZP Jul 30 2007.  Updated rules to allow 1800 MB memory.\nJul 31 2007 - added pim 2.10.\nAug 03 2007 - added sr2.\nAug 07 2007 - added lzpm 0.07.  Underlined times and memory to indicate records.\nAug 08 2007 - added pimple2.\nAug 09 2007 - added lzpm 0.08, TarsaLZP Aug 8 2007.\nAug 11 2007 - added TarsaLZP Aug 10 2007.\nAug 13 2007 - added gziphack, retested gzip 1.3.5, Info-ZIP 2.32 Win32.\nAug 14 2007 - added QuickLZ 1.30, compact.\nAug 15 2007 - added lzturbo 0.01, WinTurtle 1.2.\nAug 16 2007 - added paq8fthis2 -8 enwik8, WinTurtle 1.21, lzpm 0.09.\nAug 23 2007 - added paq8n -8 enwik8, paq8osse -8 enwik8, thor 0.96a, lzpm 0.10.\nAug 24 2007 - added paq8o -8 enwik8.\nAug 29 2007 - added lzc 0.06b.\nAug 30 2007 - added HKCC-2 enwik8 decompresser, added link to paq8o ver. 2, added WinTurtle 1.30, qazar 0.0pre5.\nAug 31 2007 - added qc 0.050.\nSep 02 2007 - added HKCC-2 Sep 01 2007 version, WinRK 3.03 SFX.\nSep 06 2007 - added lzpm 0.11.\nSep 13 2007 - added lzpmlite 0.11.\nSep 14 2007 - added paq8o3 -8 enwik8.\nSep 20 2007 - added lpaq2, hook 1.0.\nSep 22 2007 - added paq8o4 v1, rings 0.1.\nSep 29 2007 - added paq8o6 -8 enwik8.\nSep 30 2007 - added lpaq3, elpaq3, lprepaq 1.2.\nOct 01 2007 - added lpaq3a, lpaq3e.\nOct 04 2007 - added lpaq4, lpaq4e.\nOct 05 2007 - added lzturbo 0.1.\nOct 16 2007 - added lpaq5, lpaq5e, withdrew HKCC-2.\nOct 20 2007 - added paq8o7 -8 enwik8.\nOct 23 2007 - added lpaq6, lpaq6e.\nOct 24 2007 - added paq8o8 -8 enwik8.\nOct 25 2007 - added lzc 0.07.\nOct 28 2007 - added rule that benchmark results will be delayed 30 days after the latest version of the program is published.\nNov 09 2007 - added lpaq7, lpaq7e*, xwrt 3.2*, sr3*.\nNov 22 2007 - added quickLZ 1.40, rings 0.2, hook 1.1, lzc 0.08*.\nNov 23 2007 - added lzpm 0.12.\nDec 03 2007 - ranked lpaq7e, xwrt 3.2, sr3, lzc 0.08.\nDec 04 2007 - added and ranked xwrt 3.2|ppmonstr J.\nDec 05 2007 - added symbra 0.2*.\nDec 11 2007 - added lpaq8*, lpaq8e*.\nDec 13 2007 - added lcssr 0.2*.\nDec 16 2007 - uploaded symbra 0.2, lcssr 0.2 mirrors, added fpaqa*, hook 1.3, lzpm 1.3, cmm1, cmm2.\nDec 17 2007 - corrected cmm1, cmm2, ranked cmm1.\nDec 18 2007 - added fpaqb*.\nDec 20 2007 - updated fpaqb v2*, added fpaq0m, bit 0.1*.\nDec 21 2007 - added lpaq1a.\nDec 24 2007 - added fpaqc*.\nDec 25 2007 - added lpq1, rings 0.3*.\nDec 26 2007 - added FreeArc 0.40-pre-4*.\nJan 09 2008 - added fpaq0r, fpaq0rs*, ranked lpaq8e, lcssr 0.2.\nJan 11 2008 - added flashzip 0.01, flashzip 0.02*, winturtle 1.60*, ccmx 1.30*.\nJan 13 2008 - added lzpm 0.14, cmm 080113*.  Updated pkzip 2.04 -ex.\nJan 17 2008 - added lzpm 0.15.\nJan 25 2008 - added fpaq0pv2, ranked FreeArc 0.40-pre-4, bit 0.1, rings 0.3, fpaq0mw.\nJan 28 2008 - added fpaq0f*.\nJan 30 2008 - added fpaq0f2*.\nJan 31 2008 - added lzw 0.1, paq9a.  Repealed 30 day wait rule and ranked pending compressors marked with *.\nFeb 04 2008 - added flashzip 0.3.\nFeb 08 2008 - added lzw 0.2, rings 1.0.\nFeb 09 2008 - added cmm3 080207.\nFeb 11 2008 - added ppp.\nFeb 12 2008 - added lzp3o2, updated ppp description.\nFeb 13 2008 - added rings 1.1, lzrw1.\nFeb 14 2008 - added lzrw1-a, lzrw2, lzrw3, lzrw3-a, lzrw5, updated lzrw1.\nFeb 17 2008 - updated lzrw1-a, lzrw2, lzrw3, lzrw3-a, lzrw5 (new .exe sizes).\nFeb 21 2008 - added durilca4linux_3.\nFeb 22 2008 - added drt|lpaq9e.\nFeb 25 2008 - added lzturbo 0.9.\nMar 04 2008 - added rings 1.2.\nMar 09 2008 - added balz 1.02, rzm 0.06c, tornado 0.3.\nMar 13 2008 - added Stuffit 12.0.0.17.\nMar 14 2008 - added cmm4 v0.0.\nApr 02 2008 - added rings 1.3.\nApr 04 2008 - added fpaq0pv3.\nApr 06 2008 - added fpaq0pv5.\nApr 14 2008 - added rings 1.4c.\nApr 15 2008 - updated rings 1.4c description.\nApr 21 2008 - added rings 1.5.\nApr 22 2008 - added durilca4linux_3 v2 (new dictionary).\nApr 28 2008 - added lpaq9f.\nMay 09 2008 - added balz 1.06.\nMay 11 2008 - added packet 0.01, slug 1.27, rzm 0.07h.\nMay 14 2008 - added balz 1.07.\nMay 18 2008 - added packet 0.02.\nMay 19 2008 - added fpaq0pv4, fpaq0pv4nc, fpaq0pv4nc0, fpaq0pv4a, fpaq0pv4anc, fpaq0pv4and0.\nMay 20 2008 - added packet 0.03b, balz 1.08, fpaq0pv4b1.\nMay 21 2008 - added balz 1.09.\nMay 22 2008 - added durilca4linux3 v3, cmm4 v0.1e.\nMay 23 2008 - updated cmm4 v0.1e description, lpaq9g, fcm1.\nJun 03 2008 - added balz 1.12.\nJun 04 2008 - added lpaq9h.\nJun 10 2008 - added paq8o8-intel -1, paq8o8z-jun7 -1.\nJun 12 2008 - added paq8o10t (enwik8 only), balz 1.13.\nJun 13 2008 - added lpaq9i.\nJun 14 2008 - added drt|ppmonstr (under lpaq9i).\nJun 17 2008 - updated paq8o8z (note 25), durilca4linux_3 v3 (2 GB).\nJun 18 2008 - added flzp v1.\nJun 19 2008 - added packet 0.90b.\nJul 17 2008 - added lzgt, lzgt1, lzgt2, lzgt3.\nJul 19 2008 - added nanozip 0.01a, balz 1.15.\nJul 20 2008 - updated nanozip 0.01a -txt, clarified method of creating zip archive of decompresser.\nJul 22 2008 - added pim 2.50, tornado 0.4a, M99 v2.2.1.\nJul 24 2008 - added 4x4 0.2a, bit 0.2b.\nJul 25 2008 - added nanozipltcb.\nJul 26 2008 - added flashzip 0.9.\nJul 28 2008 - corrected Pareto frontier.\nAug 02 2008 - added nanozip 0.03a, lzss 0.01.\nAug 18 2008 - added flashzip 0.91, lpaq9j.\nSep 05 2008 - added size vs. speed and memory graphs.\nSep 26 2008 - added bzp 0.2, ppms J.\nOct 02 2008 - added lpaq9k.\nOct 27 2008 - added nanozip 0.05a.\nOct 28 2008 - added lzgt3a.\nNov 21 2008 - added bit 0.7. Updated test computer (note 26).\nNov 27 2008 - added ppmx 0.01, sr3c.\nNov 28 2008 - added mcomp 2.00.\nDec 02 2008 - added lpaq9l, ppmx 0.02.\nDec 22 2008 - added ppmx 0.03.\nDec 29 2008 - added M1 0.2a.\nJan 02 2009 - added M1 0.3.\nJan 05 2009 - added ppmx 0.04.\nJan 09 2009 - updated link to paq8hp12any.\nJan 28 2009 - added xdelta 3.0u.\nFeb 09 2009 - added bcm 0.03.\nFeb 11 2009 - added bcm 0.04.\nFeb 21 2009 - added drt|lpaq9m.\nMar 02 2009 - added Stuffit 2009 13.0.0.19, nanozip 0.06a, NTFS (LZNT1).\nMar 05 2009 - added bcm 0.05.\nMar 06 2009 - updated bcm 0.05.\nMar 10 2009 - added flashzip 0.93a, fixed links to winturtle, flashzip, rings, hook, packet, bzp.\nMar 12 2009 - added bwmonstr 0.00.\nMar 15 2009 - added bcm 0.07.\nMar 20 2009 - added bwmonstr 0.01.\nMar 26 2009 - added flashzip 0.94, decomp8.\nApr 01 2009 - added runcoder1.\nApr 13 2009 - added lzturbo 0.94, M1 0.3b.\nApr 14 2009 - added lzuf.\nApr 16 2009 - added M1 0.3b parameter e8-m103b1-mh.\nApr 17 2009 - added lzp2.\nApr 18 2009 - added csc2.\nApr 21 2009 - added paq8p3, paq8p3 v2.\nApr 22 2009 - added decomp8b.\nApr 22 2009 - added lzbw1 0.8.\nApr 29 2009 - added hook 1.4.\nMay 08 2009 - updated opendark-A.\nMay 26 2009 - added decmprs8.\nJun 01 2009 - added bcm 0.08.\nJun 02 2009 - added reorder_v2|bcm 0.08.\nJun 05 2009 - updated reorder_v2|bcm 0.08 xlt.\nJul 14 2009 - added bwmonstr 0.02\nJul 16 2009 - updated bwmonstr 0.02 comments.\nJul 21 2009 - added durilca'kingsize\nJul 23 2009 - moved website to http://mattmahoney.net/dc/\n              added paq8px_v60_turbo, split paq from paq8hp entries, moved decompr8 series to lpaq,\n              added flashzip 0.99, updated sr3.exe to remove antivirus false alarms due to upack.\nAug 07 2009 - added packet 0.91b.\nAug 14 2009 - added csc3 v.2009.8.12, combined with csc2.\nAug 16 2009 - added and corrected rings 1.6.\nAug 26 2009 - added flashzip 0.99b4.\nSep 14 2009 - added zpaq 1.03.\nSep 15 2009 - updated zpaq 1.03 cmax3.cfg.\nSep 16 2009 - updated zpaq 1.03 cmax4.cfg, updated paq8hp12 links,\nSep 17 2009 - added rule that each compressor can only be listed once,\n              so removed xwrt|ppmonstr. Updated zpaq 1.03 with drt|cmax4.cfg (not in main table),\n              updated zpaq 1.03 cmax_enwik9.\nSep 18 2009 - updated zpaq 1.03 o0.cfg, o1.cfg, o2.cfg, drt|max_enwik9drt.cfg.\nSep 23 2009 - added csc31.\nOct 01 2009 - added zpipe 1.00 (zpaq).\nOct 07 2009 - added zpaq cbwt_j2.cfg,18.\nOct 11 2009 - added M03 v0.2a, lzp2 0.7c.\nOct 13 2009 - added bcm 0.09.\nOct 15 2009 - added zpaq v1.08 cbwt_slowmode1_1GB_block.cfg.\nOct 15 2009 - added lz4 0.2.\nOct 26 2009 - added zpaq v1.09 ocbwt_j1.cfg and corrected memory usage.\nOct 29 2009 - corrections to Pareto frontier.\nNov 12 2009 - added durilca'kingsize_4 (new dictionary).\nNov 27 2009 - added lrzip 0.40.\nNov 29 2009 - added tests for durilca'kingsize.\nNov 30 2009 - added tests for durilca'kingsize_4, added lrzip 0.42.\nDec 07 2009 - added 7zip 9.04a.\nDec 15 2009 - added zhuff 0.1, bcm 0.10.\nDec 17 2009 - added M1x2 v0.5-1.\nDec 29 2009 - updated bcm 0.10.\nJan 15 2010 - added bpe2 v1, bpe2 v2.\nJan 17 2010 - updated shindlet link.\nJan 19 2010 - added kwc.\nJan 21 2010 - added acb 2.00c.\nFeb 01 2010 - added ulz 0.01.\nFeb 06 2010 - added ulz 0.02.\nFeb 08 2010 - added m1x2 0.6.\nFeb 12 2010 - added bpe, bpe2v3.\nFeb 14 2010 - updated bpe2v3 description.\nFeb 16 2010 - updated srank link.\nFeb 19 2010 - added ppmx 0.05.\nFeb 24 2010 - added szip 1.12a, fixed typos.\nMar 01 2010 - added flashzip 0.99b8.\nMar 03 2010 - added nanozipltcb 0.08.\nMar 30 2010 - added etincelle alpha 3.\nApr 07 2010 - added bsc 1.0.0.\nApr 08 2010 - updated bsc 1.0.0.\nApr 11 2010 - added bsc 1.0.3.\nApr 23 2010 - corrections to ppmvc, ctxf.\nMay 03 2010 - added yzx 0.01, bsc 2.00, fp8_v1, plzip.\nMay 10 2010 - added csc32 a2, yzx 0.02, nanozipltcb 0.09.\nMay 21 2010 - added yzx 0.03.\nMay 27 2010 - added yzx 0.04.\nJun 06 2010 - added nanozip 0.08a.\nJun 09 2010 - updated lpaq9m.\nJun 11 2010 - updated nanozip 0.08a, cmm4 0.2b, 7ip 9.12b (note 42).\nJun 15 2010 - added bsc 2.20.\nJun 21 2010 - updated winrk 3.03, ppmonstr J.\nJun 22 2010 - added bcm 0.11.\nJun 26 2010 - updated bcm 0.11, drt (lpaq9m).\nJun 28 2010 - updated paq8hp12any (note 41), bcm link.\nJul 16 2010 - added zp 1.00.\nJul 28 2010 - added ppmx 0.06, bsc 2.26. Updated links to pimple2, ocamyd.\nAug 05 2010 - updated zp 1.00 (zpaq).\nAug 26 2010 - added lzham alpha 2.\nAug 30 2010 - added lzham alpha 3.\nSep 01 2010 - updated lzham alpha 3.\nSep 26 2010 - added irolz.\nOct 15 2010 - added st 0.51.\nNov 02 2010 - added bcm 0.12.\nDec 16 2010 - added bwtsdc v1.\nJan 06 2011 - added bsc 2.4.5.\nJan 23 2011 - added pzpaq 0.01.\nJan 24 2011 - updated pzpaq 0.01.\nJan 25 2011 - added lz4 0.6, lz4hc 0.9.\nJan 31 2011 - added xz 5.0.1.\nFeb 19 2011 - added stz 0.7.2.\nFeb 23 2011 - added ppmx 0.07.\nMar 02 2011 - added BWTmix v1.\nMar 04 2011 - added stz 0.8.\nMar 22 2011 - added csc32 final, zhuff 0.7.\nMar 23 2011 - added bsc 2.5.0.\nApr 27 2011 - added snappy 1.0.1.\nMay 17 2011 - added crush 0.01.\nMay 20 2011 - added zp 1.02.\nMay 28 2011 - updated bwtsdc description.\nJun 01 2011 - added flashzip 0.99c1. updated bcm 0.12.\nAug 29 2011 - added bsc 3.0.0.\nAug 30 2011 - corrections to bsc 3.0.0 description.\nSep 01 2011 - added enwik8.zip and enwik9.zip to textdata.html.\nSep 27 2011 - added comprox_ba 20110927, comprox_sa 20110927.\nSep 28 2011 - added dzo beta, comprox_ba 20110928, comprox_sa 20110928.\nSep 29 2011 - added comprox_ba 20110929, comprox_sa 20110929.\nSep 30 2011 - added KuaiZip 2.3.2 x86, 7zip 9.20, Info-ZIP 3.00.\nOct 02 2011 - added lzsr 0.01.\nOct 10 2011 - added comprox 0.1.1, flashzip 0.99c3.\nOct 12 2011 - added lz4 v1.2.\nOct 20 2011 - added xpv5.\nOct 31 2011 - added flashzip 0.99d1.\nNov 02 2011 - added M03 v1.1b.\nNov 05 2011 - added nanozip 0.09a. Added link to enwik8 ranking on compressionratings.com.\nNov 13 2011 - added zpaq v4.00, merged with zp.\nNov 24 2011 - added RangeCoderC v1.2.\nNov 26 2011 - added RangeCoderC v1.3.\nNov 29 2011 - added zhuff v0.8, RangeCoderC v1.4 and v1.5, link to dark.\nDec 05 2011 - added RangeCoderC v1.6, v1.7a.\nDec 09 2011 - added RangeCoderC v1.7.\nDec 13 2011 - added RangeCoderC v1.8.\nDec 17 2011 - added zcm v0.01.\nDec 23 2011 - added zcm v0.02.\nDec 31 2011 - added ppmx v0.08.\nJan 01 2012 - updated ppmx v0.08.\nJan 04 2012 - added yzx 0.11, zcm 0.03.\nJan 17 2012 - added pigz 2.2.3, updated gzip 1.3.5.\nJan 24 2012 - added MTCompressor 1.0.\nJan 26 2012 - added paq8pxd.\nJan 29 2012 - added TarsaLZP 29 Jan 2012.\nJan 30 2012 - added zcm v0.04.\nFeb 11 2012 - added paq8pxd_v2.\nFeb 17 2012 - added paq8px_v69.\nFeb 19 2012 - added zcm 0.11.\nMar 01 2012 - added fbc v1.0.\nMar 02 2012 - added fbc v1.1. Converted decmprs8, decomp8, decomp8b, all_HKCC, lpaq9* to .zpaq\nMar 05 2012 - added crook v0.1.\nMar 18 2012 - added lrzip 0.612.\nMar 22 2012 - corrected lrzip options.\nMar 23 2012 - added data-shrinker 23Mar2012.\nApr 04 2012 - added zcm 0.20b.\nApr 11 2012 - added fp8 v2, FreeArc 0.666.\nApr 19 2012 - added paq8pxd_v3.\nApr 23 2012 - added paq8pxd_v4.\nMay 02 2012 - added zcm 0.30.\nMay 15 2012 - added fp8 v3.\nMay 16 2012 - added zcm 0.40.\nMay 17 2012 - updated zcm 0.40.\nJun 02 2012 - added zcm 0.50a.\nJun 12 2012 - changed spelling \"Ratushnyak\" to \"Rhatushnyak\" due to name change.\nJun 17 2012 - added urban.\nJul 10 2012 - added bsc 3.1.0.\nAug 05 2012 - added diz.\nAug 24 2012 - added comprox 0.6.0.\nSep 01 2012 - added st 0.81.\nSep 10 2012 - added comprox 0.7.0.\nSep 11 2012 - updated comprox 0.7.0, added zcm 0.60d.\nSep 26 2012 - added comprox 0.8.0.\nSep 27 2012 - added comprox 0.8.0-bugfix1.\nOct 05 2012 - added flashzip 1.0.0.\nOct 07 2012 - added comprolz 0.1.0.\nOct 10 2012 - added lazy 1.00.\nOct 12 2012 - added TinyLZP 0.1, TinyCM 0.1.\nOct 14 2012 - updated TinyLZP 0.1, added zcm 0.70b.\nOct 18 2012 - added comprox 0.9.0, comprolz 0.2.0.\nOct 21 2012 - added smile.\nOct 22 2012 - updated smile.\nOct 23 2012 - added zpaq 6.12.\nOct 30 2012 - added exdupe 0.3.3 beta.\nNov 19 2012 - added TarsaLZP 18.nov.2012.\nNov 20 2012 - updated link to dmc.\nNov 26 2012 - added comprox 0.10.0, comprolz 0.10.0.\nDec 12 2012 - added flashzip 1.1.2.\nDec 17 2012 - added comprox 0.11.0, comprolz 0.11.0.\nDec 18 2012 - added comprox 0.11.0-bugfix1, comprolz 0.11.0-bugfix1.\nJan 15 2013 - added lzwc 0.1, lzwc 0.3, lzwc_bitwise 0.7, lzip 1.14-rc3.\nJan 17 2013 - added plzma_v3p, plzma_v3c.\nJan 18 2013 - updated plzma_v3b (not v3p), plzma_v3c.\nJan 23 2013 - added smac 1.8.\nJan 24 2013 - added zpaq 6.19.\nJan 31 2013 - added smac 1.9.\nFeb 01 2013 - added WinRAR 4.20.\nFeb 07 2013 - added smac 1.10.\nFeb 24 2013 - added smac 1.11.\nMar 11 2013 - added smac 1.12a.\nMar 15 2013 - added pigz 2.3.\nMar 25 2013 - added smac 1.13.\nApr 15 2013 - updated bwmonstr description.\nApr 20 2013 - added smac 1.14.\nApr 21 2013 - added paq8pxd_v5.\nApr 30 2013 - added WinRAR 5.00b2.\nMay 01 2013 - updated WinRAR 5.00b2.\nMay 14 2013 - added lzturbo 1.1.\nMay 15 2013 - updated lzturbo 1.1.\nMay 16 2013 - added zcm 0.80.\nMay 21 2013 - added smac 1.15.\nJun 04 2013 - added mcm 0.0.\nJun 13 2013 - added mcm 0.2.\nJun 18 2013 - added tangelo 1.0 (fp8).\nJun 22 2013 - added bcm 0.14, zcm 0.88.\nJun 26 2013 - added zpaq 6.34.\nJun 27 2013 - updated crush 0.01, added mcm 0.3.\nJun 28 2013 - updated crush 0.01 description.\nJun 30 2013 - updated bsc 3.10 description.\nJul 01 2013 - added crush 1.00.\nJul 02 2013 - updated crush 1.00.\nJul 06 2013 - added tangelo 2.0 (fp8).\nJul 08 2013 - updated tangelo 2.0.\nJul 11 2013 - added rings 2.0.\nJul 14 2013 - added bwtdisk 0.9.0.\nJul 15 2013 - added crushm.\nJul 17 2013 - added mcm 0.4.\nJul 20 2013 - added tangelo 2.1.\nJul 24 2013 - added tangelo 2.3.\nJul 31 2013 - added smac 1.16, sharc 0.9.5b.\nAug 01 2013 - updated sharc 0.9.6.\nAug 20 2013 - added packet 1.0, paq8pxd_v7, zlite.\nAug 28 2013 - added ppmz2 0.81.\nOct 14 2013 - added zpaq 6.42, zpaqd 6.32 max5.cfg.\nOct 16 2013 - added arj 3.10, zpaq 6.42 max6.cfg.\nOct 28 2013 - added lzf 1.00.\nOct 30 2013 - added lzf 1.01.\nNov 01 2013 - added zling.\nNov 04 2013 - added smac 1.17.\nNov 19 2013 - added smac 1.17a.\nDec 10 2013 - added smac 1.18, packet 1.1, packARC 0.7RC11, mtari 0.2.\nDec 11 2013 - added cm0_ext (includes cm0, cm1, bwcm).\nDec 12 2013 - added sharc 0.9.10.\nDec 13 2013 - added sharc 0.9.11b.\nDec 14 2013 - updated sharc 0.9.11b description.\nDec 19 2013 - added smac 1.19.\nDec 26 2013 - added zling Dec-25-2013.\nJan 02 2014 - added lzv 0.1.0.\nJan 08 2014 - added doboz 0.1.\nJan 17 2014 - added smac 1.20.\nJan 21 2014 - added zling Jan-21-2013.\nJan 23 2014 - added cm4_ext.\nFeb 04 2014 - added zhuff 0.95b, 0.97 beta, alba 0.1.\nFeb 05 2014 - updated alba 0.1.\nFeb 06 2014 - added alba 0.2.\nFeb 10 2014 - added lzss 0.2.\nFeb 11 2014 - updated lzss 0.2.\nFeb 17 2014 - added RH, RH2.\nFeb 18 2014 - added alba 0.5.1.\nFeb 22 2014 - added ksc.\nFeb 27 2014 - added RH2 20Feb2014.\nMar 02 2014 - added zling (libzling) 20140219.\nMar 10 2014 - added tornado 0.6.\nMar 15 2014 - added freearc 0.67a.\nMar 23 2014 - added RH4_x64 22Mar2014.\nMar 24 2014 - added libzling 20140324.\nMar 25 2014 - added ppmx 0.09, zpaq 6.50.\nApr 01 2014 - added tree 0.1.\nApr 02 2014 - updated tree 0.1.\nApr 04 2014 - updated tree 0.1.\nApr 14 2014 - added libzling 20140414.\nApr 16 2014 - added cmix v1.\nApr 28 2014 - added tree 0.3.\nApr 29 2014 - added RH4 24Apr2014.\nMay 04 2014 - added zcm 0.90.\nMay 05 2014 - added zling (libzling) 20140430-bugfix.\nMay 12 2014 - updated gzip124hack description and link.\nMay 16 2014 - added zcm 0.92.\nMay 27 2014 - added tree 0.4, tree 0.5.\nMay 29 2014 - added cmix v2.\nJun 02 2014 - added lza 0.01.\nJun 18 2014 - added paq8pxd_v8.\nJun 27 2014 - added cmix v3.\nJun 29 2014 - added paq8pxd_v10.\nJun 30 2014 - added lza 0.10.\nJul 05 2014 - added lza_x64 0.10.\nJul 06 2014 - added tree 0.9.\nJul 07 2014 - added zcm_x64 0.92.\nJul 09 2014 - updated zcm_x64 0.92.\nJul 23 2014 - added cmix v4.\nJul 27 2014 - updated st (obsolete).\nJul 31 2014 - added paq8pxd_v12.\nAug 08 2014 - added lzturbo 1.2.\nAug 11 2014 - updated lzturbo 1.2 (levels 3x).\nAug 13 2014 - added paq8pxd_v12-skbuild, cmix v5.\nAug 15 2014 - updated cmix (typo).\nAug 17 2014 - added tree v10.0, paq8pxd_v12-skbuild.\nAug 18 2014 - updated tree v0.10.\nAug 22 2014 - updated paqp8xd_v12-skbuild description.\nAug 28 2014 - added paq8pxd_v13_x64.\nSep 03 2014 - added tree v0.11, cmix v6.\nSep 07 2014 - updated tree v0.11.\nSep 08 2014 - updated tree v0.11.\nSep 09 2014 - added lza 0.51.\nSep 10 2014 - added lza_x64 0.51.\nSep 11 2014 - added xeloz 0.3.5.3.\nSep 12 2014 - added xeloz 0.3.5.3a.\nSep 14 2014 - removed st at request of author.\nSep 18 2014 - updated stuffit link.\nSep 19 2014 - added paq8pxd_v15.\nSep 23 2014 - updated paq8pxd_v15 for enwik9.\nOct 04 2014 - added tree 0.12.\nOct 06 2014 - added lzf 1.02.\nOct 13 2014 - added tree 0.13.\nOct 14 2014 - updated main table typo (libzling).\nOct 18 2014 - added lza 0.61.\nOct 20 2014 - added lza 0.62.\nOct 28 2014 - added paq8pxd_v12_biondivers1_x64.\nOct 31 2014 - added tree 0.14.\nNov 13 2014 - added rh5.\nNov 20 2014 - added lza 0.70b.\nNov 22 2014 - added tree 0.15a.\nNov 23 2014 - updated tree 0.15a.\nDec 09 2014 - added tree 0.16b.\nDec 12 2014 - updated tree 0.16b.\nDec 16 2014 - added tree 0.17.\nDec 18 2014 - updated tree 0.17.\nJan 11 2015 - added lza 0.80test.\nJan 19 2015 - added tree 0.18.\nJan 25 2015 - added zstd.\nJan 26 2015 - added lzhamtest (lzham) v1.0.\nFeb 04 2015 - added tree 0.19.\nFeb 05 2015 - added cmix v7, mcm 0.8.\nFeb 09 2015 - added pcompress 3.1.\nMar 03 2015 - added bcm 1.00, mcm 0.82.\nMar 04 2015 - updated bcm 1.00.\nMar 06 2015 - added balz 1.20.\nMar 10 2015 - added lza 0.82b.\nMar 18 2015 - added bce3.\nMar 23 2015 - added csarc 3.3.\nApr 22 2015 - added mcm 0.83.\nApr 24 2015 - added xz 5.2.1.\nApr 27 2015 - added glza 0.1 (formerly tree).\nApr 28 2015 - corrected Pareto frontier in main table.\nApr 29 2015 - corrected Pareto frontier for lzham.\nMay 13 2015 - added zcm 0.93.\nMay 27 2015 - added glza 0.2.\nMay 28 2015 - added rings 2.1 and 2.2.\nJun 08 2015 - added rings 2.5.\nJul 13 2015 - added glza 0.3.\nJul 20 2015 - added packet 1.2.\nSep 15 2015 - added cmv 00.01.00, updated nanozip 0.09a.\nSep 16 2015 - updated cmv 00.01.00, nanozip 0.09a.\nSep 23 2015 - added brotli 21 Sep 2015.\nSep 25 2015 - added brieflz 1.1.0.\nNov 11 2015 - added cmix v8.\nNov 18 2015 - added glza 0.3b.\nDec 04 2015 - added zstd 0.4.0, 0.4.2.\nDec 05 2015 - updated zstd 0.4.2.\nDec 06 2015 - added zstd_no_legacy 0.4.2.\nJan 05 2016 - added lz5 1.3.3.\nFeb 09 2016 - added libzling 20160107, lz4opt 1.00.\nFeb 18 2016 - added zstd 0.5.1, brotli 18-Feb-2016.\nFeb 20 2016 - updated brotli 18-Feb-2016.\nMar 10 2016 - added emma 0.1.3.\nMar 11 2016 - added glza 0.4.\nMar 12 2016 - updated glza 0.4.\nMar 14 2016 - added emma 0.1.4.\nMar 20 2016 - added cmv 00.01.01.\nMar 30 2016 - updated cmv 00.01.01.\nApr 08 2016 - added lz4x 1.02.\nApr 13 2016 - added zstd 0.6.0.\nApr 15 2016 - added cmix v9.\nMay 05 2016 - updated link to m1x2.\nJun 17 2016 - added cmix v10.\nJun 28 2016 - added ulz 0.03.\nJul 01 2016 - added plzip 1.5 (lzip).\nJul 07 2016 - added cmix v11.\nJul 19 2016 - added emma 0.1.12.\nAug 08 2016 - added paq8pxd_v18.\nAug 12 2016 - updated paq8pxd_v18.\nAug 23 2016 - added emma 0.1.16.\nAug 24 2016 - updated emma 0.1.16.\nAug 29 2016 - corrected emma 0.1.16 version to 0.1.6.\nSep 05 2016 - added packet 1.9.\nSep 27 2016 - added glza 0.8.\nNov 08 2016 - added cmix v12.\nApr 25 2017 - added cmix v13.\nApr 28 2017 - added emma 0.1.22.\nJun 27 2017 - added lstm-compress.\nJul 13 2017 - added ulz 0.06.\nJul 19 2017 - added paq8px_v77.\nSep 24 2017 - added emma 1.23, paq8pxd_v32, paq8px_v96.\nNov 23 2017 - added cmix v14.\nDec 14 2017 - added lstm-compress (cmix).\nJan 05 2018 - added phda9 1.0, cmve 0.2.0.\nFeb 01 2018 - moved lstm-compress to own section.\nMar 28 2018 - added phda9 1.2.\nApr 30 2018 - added phda9 1.3.\nMay 20 2018 - added cmix v15, phda9 1.4.\nAug 09 2018 - added phda9 1.5, paq8pxd_v47, glza 0.10.1, fixed cmve 0.2.0.\nOct 11 2018 - added cmix v16.\nOct 25 2018 - added phda9 1.6.\nOct 26 2018 - typo.\nFeb 22 2019 - added phda9 1.7.\nMar 27 2019 - added cmix v17.\nApr 01 2019 - added lstm-compress v3.\nMay 10 2019 - added nncp 2019-05-08.\nMay 11 2019 - updated nncp 2019-05-08.\nJul 09 2019 - added phda9 1.8.\nJul 25 2019 - replaced links to encode.ru to encode.su throughout. Added paq8pxd_v48_bwt1, paq8pxd_v61.\nAug 07 2019 - added cmix v18.\nAug 10 2019 - added nakamichi 2019-Jul-01.\nAug 12 2019 - added HP_2017_October.rar (2017 Hutter prize winner) under phda9.\nNov 19 2019 - added nncp 2019-11-16.\nMar 09 2020 - updated description of Hutter prize.\nJul 21 2020 - added tensorflow-compress v1.\nSep 09 2020 - added tensorflow-compress v2.\nDec 01 2020 - added tensorflow-compress v3.\nJan 10 2021 - added nncp v2.\nJan 12 2021 - updated tensorflow-compress.\nFeb 06 2021 - added nncp v2.1.\nApr 26 2021 - added nncp v3.\nJun 14 2021 - added starlit.\nJun 21 2021 - added cmix-hp v1.\nAug 30 2021 - added cmix-hp v2, v3, cmix v19, nncp v3.1.\nApr 26 2022 - added nanozip 0.09a mirror.\nJul 05 2022 - added paq8px_v206fix1.\nAug 14 2022 - added tensorflow-compress v4.\nSep 10 2022 - added lzuf62, lzhhf, lzpgt.\nSep 12 2022 - fixed typos in lzuf62.\nSep 15 2022 - added lzwg.\nNov 25 2022 - updated links for compact.\nDec 02 2022 - added bsc-m03 0.4.0, bsc 3.2.5.\nFeb 28 2023 - added bcm 2.03.\nJul 22 2023 - added fast-cmix-hp.\nAug 14 2023 - added lzpgt6, lzwhc.\nAug 16 2023 - updated lzwhc description.\nOct 24 2023 - added nncp v3.2.\nNov 01 2023 - added fastcmix.archive9 mirror for Hutter prize submission of fast-cmix-hp.\nNov 07 2023 - added cmix v20.\nJan 17 2024 - added fx-cmix.\nJun 04 2024 - added ghost.\nSep 17 2024 - added cmix v21.\nSep 19 2024 - updated cmix v21 (added option -t).\nOct 08 2024 - added fx2-cmix.\nNov 24 2024 - added gmix v1.\nNov 30 2024 - added fxv v1.\nDec 01 2024 - corrected fxv v1.\nApr 27 2025 - added kanzi.\nMay 01 2025 - updated kanzi.\nMay 05 2025 - added bzip3, ect 0.9.5.\nJul 03 2025 - added qbp.\nMar 17 2026 - added jax-compress.\nMar 25 2026 - added glza 0.12.\nJun 06 2026 - added stc.\nJun 10 2026 - updated stc.\nJun 22 2026 - added ppmx 0.10.\nJun 27 2026 - added cmix-lex.\nJun 30 2026 - updated cmix-lex (enwik9 decompression).\nJul 08 2026 - updated durilca_kingsize -o41.\nAug 30 2026 - added altxs 1.0.0, cmix-obias, fx2-cmix-transformer.\nAug 31 2026 - added forge-cmix, fx-deepmix.\nSep 01 2026 - updated forge-cmix v2, updated fx2-cmix, cmix-lex with compressor sizes for Hutter prize scoring.\nSep 02 2026 - updated links to fx2-cmix-transformer, fixed typos in forge-cmix.\n```\n\n This page is maintained by Matt Mahoney, mattmahoneyfl (at) gmail (dot) com.\n## .9956 arb2x\n\n[arb2x](http://bijective.dogma.net/compres11.htm) v20060602 is a\nfree, experimental command line file compressor with source code availalbe\nby David A. Scott, updated June 2, 2006.\nIt is a *bitwise* bijective order-0 arithmetic coder, best suited\nfor i.i.d. *bits*.  It takes no arguments\nexcept the input and output filenames.  The decompresser is unarb2x.exe.\n## Failed and Pending Tests\n\n## hipp\n\n[hipp](http://www.compression.ru/download/articles/cm/hipp/hipp5819.html) v0.5819\nis an experimental command line file compressor with source code available by\nBogatov Roman, Aug. 19, 2005.  It uses context mixing with ordinary and optionally sparse\n(fixed gap) contexts, using a suffix tree with path compression to store statistics.\nThe options are /m to specify the memory limit in MB (default /m2048),\n/o to specify primary context order, i.e. the depth of the suffix tree\nwith path compression (default /o256), /do to set max\ndeterministic order (actual order with path decompression) (default /do256, do >= o),\n/so to set the number of sparse contexts (default /so0).  Sparse contexts\nare useful for binary data but generally not text.  Memory usage increases\nwith the size of the file and with /o and /so (but not /do).  Also, if the\nmemory limit is exceeded then an error occurs.  Unfortunately enwik9 cannot\nbe compressed at all because initialization requires more than 800 MB.\nSome results for enwik8:\n## ppmz2\n\n[ppmz2](http://www.cbloom.com/src/ppmz.html) v0.81 is a free,\nexperimental, open source file compressor by Charles Bloom, May 9, 2004.\nIt uses PPM. It takes several compression options but only the defaults\nwere tested. Memory usage grows as the program runs.\nOn enwik9 it runs out of memory.\n## XMill\n\n[XMill](http://sourceforge.net/projects/xmill) 0.8 is an open source\ncommand line XML preprocessor/compressor by AT&T, written by Dan Suciu,\nHartmut Liefke, and Hedzer Westra in March, 2003.\nIt works by sorting by XML tags to bring similar content together, then\ncompressing with gzip, bzip2, or ppmd.  Optionally it can (in theory) output the\npreprocessed data as input to another compressor.\n[xmill 0.9.1](http://www.cs.washington.edu/homes/suciu/XMILL/xmill-0.9.1.tar.gz)\n(Mar. 15, 2004) also fails to decompress enwik9 and fails to decompress either file with -N.\n## lzp3o2\n\n[lzp3o2](http://cbloom.com/src/index_lz.html) (LZP 3 with order 2 literal\ncoding) is one of a family of open source file compressors by\nCharles Bloom, originally written in 1995.  The algorithm is described in\na [paper](http://cbloom.com/papers/lzp.html) submitted to DCC'96.\nlzp3o2 uses LZP compression with order 2 modeling of literals and arithmetic\ncoding.  The tested version of the source code\nis dated Aug. 25, 1996 and compiled for Windows Oct. 10, 1998.  The compiled\ndistribution from [here](http://cbloom.com/exe/lzpexes.zip) was tested.\n## History", "url": "https://wpnews.pro/news/fx2-cmix-transformer", "canonical_source": "https://mattmahoney.net/dc/text.html#0969", "published_at": "2026-09-08 05:50:07+00:00", "updated_at": "2026-09-08 06:02:23.603862+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing"], "entities": ["Matt Mahoney", "fx2-cmix-transformer", "Wikipedia", "Large Text Compression Benchmark"], "alternates": {"html": "https://wpnews.pro/news/fx2-cmix-transformer", "markdown": "https://wpnews.pro/news/fx2-cmix-transformer.md", "text": "https://wpnews.pro/news/fx2-cmix-transformer.txt", "jsonld": "https://wpnews.pro/news/fx2-cmix-transformer.jsonld"}}