{"slug": "gpu-lossless-compression-gpu-accelerated-lossless-data-compressors-survey", "title": "GPU-Lossless-Compression – GPU-Accelerated Lossless Data Compressors Survey", "summary": "A GitHub survey by Washington State University researcher Dingwen Tao benchmarks GPU-accelerated lossless compressors on an NVIDIA V100 GPU with 16 GB of memory, evaluating candidates for pairing with the CUDA-based lossy compressor cuSZ. The survey reports that CUDPP 2.1's cudppCompress is 2.78× slower than bzip2, with its Burrows-Wheeler and Move-to-Front/Huffman stages 2.89× and 1.34× slower respectively, while a hybrid all-core Burrows-Wheeler implementation (cuda_bzip2) achieves a 2.9× speedup over the best CPU implementation. The survey also records libbsc compressing a 3,569,598-byte test file to 159,230 bytes in 0.147 seconds (ratio 22.42) and decompressing it in 0.215 seconds, and notes CUDA 10 does not work with CUDPP while CUDA 9.1 appears to.", "body_md": "This repository suveys state-of-the-art GPU-accelerated lossless compressors. The evaluation is conductd using an NVIDIA V100 GPU (16 GB memory). Besides lossless compression survey, another purpose for this survey is to identify suitable GPU-accelerated lossless compressor for our developed CUDA-based lossy compressor (called cuSZ), which can be accessed at [https://github.com/szcompressor/cuSZ](https://github.com/szcompressor/cuSZ). Please feel free to contact me ([dingwen.tao@wsu.edu](mailto:dingwen.tao@wsu.edu)) if you find any issue or any lossless compressor that has not been covered in this survey.\n\nAbstract: CUDPP release 2.1 is a feature release that adds many new parallel primitives. We have added cudppCompress, a lossless data compression algorithm. This compression utilizes efficient parallel Burrows-Wheeler and Move-to-Front transforms (BTW) which are also exposed through the API. GPU implementation is dominated by BWT performance and is 2.78× slower than bzip2, with BWT and MTF-Huffman respectively 2.89x and 1.34x slower.\n\nCode: [http://cudpp.github.io/](http://cudpp.github.io/)\n\nPaper: [InPar’12] Patel, R.A., Zhang, Y., Mak, J., Davidson, A. and Owens, J.D., 2012. Parallel lossless data compression on the GPU (pp. 1-9). IEEE.\nSlides: [http://on-demand.gputechconf.com/gtc/2012/presentations/S0361-Lossless-Data-Compression-on-GPUs.pdf](http://on-demand.gputechconf.com/gtc/2012/presentations/S0361-Lossless-Data-Compression-on-GPUs.pdf)\n\n*Cuda 10 is not working, Cuda 9.1 seems working according to [cudpp/cudpp#185](https://github.com/cudpp/cudpp/issues/185).*\n\nAbstract: A library for lossless, block-sorting data compression. bsc is a high-performance file compressor based on lossless, block-sorting data compression algorithms. libbsc is a library based on bsc, it uses the same algorithms as bsc and enables you to compress memory blocks.\n\nCode: [https://github.com/IlyaGrebnov/libbsc](https://github.com/IlyaGrebnov/libbsc), [http://libbsc.com/](http://libbsc.com/)\n\nPaper: Burrows, M. and Wheeler, D.J., 1994. A block-sorting lossless data compression algorithm. ([https://www.hpl.hp.com/techreports/Compaq-DEC/SRC-RR-124.pdf](https://www.hpl.hp.com/techreports/Compaq-DEC/SRC-RR-124.pdf))\n\n```\n[compress]\n./bsc e /home/dtao/GPU-lossless-compression/testdata/largefile /home/dtao/GPU-lossless-compression/testdata/largefile.bsc -G\nThis is bsc, Block Sorting Compressor. Version 3.1.0. 8 July 2012.\nCopyright (c) 2009-2012 Ilya Grebnov <Ilya.Grebnov@gmail.com>.\n\n/home/dtao/GPU-lossless-compression/testdata/largefile compressed 3569598 into 159230 in 0.147 seconds.\n\n[decompress]\n./bsc d /home/dtao/GPU-lossless-compression/testdata/largefile.bsc /home/dtao/GPU-lossless-compression/testdata/largefile.bsc.out -G\nThis is bsc, Block Sorting Compressor. Version 3.1.0. 8 July 2012.\nCopyright (c) 2009-2012 Ilya Grebnov <Ilya.Grebnov@gmail.com>.\n\n/home/dtao/GPU-lossless-compression/testdata/largefile. decompressed 159230 into 3569598 in 0.215 seconds.\n```\n\n**Compression Ratio = 22.42**\n\nAbstract: bzip2-cuda is a parallel version of the BZip2 lossless data compression algorithm using NVIDIA's CUDA, with the aim of improving compression/decompression speed, leaving the compression ratio intact. bzip2 compression program is based on Burrows–Wheeler algorithm.\n\nCode: [https://github.com/bzip2-cuda/bzip2-cuda](https://github.com/bzip2-cuda/bzip2-cuda), [http://bzip2-cuda.github.io/](http://bzip2-cuda.github.io/)\n\nAbstract: We present an all-core implementation of Burrows Wheeler Compression algorithm that exploits all computing resources on a system. We develop a fast GPU BWC algorithm by extending the state-of-the-art GPU string sort to efficiently perform BWT step of BWC. Our hybrid BWC with GPU acceleration achieves a 2.9× speedup over best CPU implementation.\n\nCode: [https://github.com/aditya12agd5/cuda_bzip2](https://github.com/aditya12agd5/cuda_bzip2)\n\nPaper: [IPDPSW’15] Deshpande, A. and Narayanan, P.J., 2015, May. Fast Burrows Wheeler Compression Using All-Cores. In 2015 IEEE International Parallel and Distributed Processing Symposium Workshop (pp. 628-636). IEEE.\n\n```\n[compress]\n./bzip2 -kf -9 -n0 /home/dtao/GPU-lossless-compression/testdata/largefile\n[DEBUG] numThreads = 0\nBlock size : 900000\nnumThreads read 0\nNumber of additional CPU threads 0\n[Time thread1] 5.133659\n[Time thread2 work] 0.135336\n[Time thread2] 5.165935\nNumber of CPU threads 2\nOut of the total 4 blocks GPU did 4\ntotal compression time (with overlap) 5.166217\n\n./bzip2 -kf -9 -n1 /home/dtao/GPU-lossless-compression/testdata/largefile\n[DEBUG] numThreads = 1\nBlock size : 900000\nnumThreads read 1\nNumber of additional CPU threads 1\n[Time thread1] 2.147425\n[Time thread2 work] 0.036489\n[Time thread2] 2.183931\nNumber of CPU threads 3\nOut of the total 4 blocks GPU did 1\ntotal compression time (with overlap) 2.185086\n\n[decompress]\n./bzip2 -d  /home/dtao/GPU-lossless-compression/testdata/largefile.bz2\nTotal compression time 0.190715\n```\n\n**Compression ratio = 4.4**\n\nNote: The Burrows–Wheeler transform (BWT, also called block-sorting compression) rearranges a character string into runs of similar characters. This is useful for compression, since it tends to be easy to compress a string that has runs of repeated characters by techniques such as move-to-front transform and run-length encoding. More importantly, the transformation is reversible, without needing to store any additional data except the position of the first original character. The BWT is thus a \"free\" method of improving the efficiency of text compression algorithms, costing only some extra computation. bzip2 is a free and open-source file compression program that uses the Burrows–Wheeler algorithm.\n\nAbstract: We propose an entropy encoding based memory compression (E2MC) technique for GPUs, which is based on the well-known Huffman encoding. We study the feasibility of entropy encoding for GPUs and show that it achieves higher compression ratios than state-of-the-art GPU compression techniques. Furthermore, we address the key challenges of probability estimation, choosing an appropriate symbol length for encoding, and decompression with low latency.\n\nCode: ???\n\nPaper: [IPDPS’17] Lal, S., Lucas, J. and Juurlink, B., 2017, May. E^ 2MC: Entropy Encoding Based Memory Compression for GPUs. In 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (pp. 1119-1128). IEEE.\n\nAbstract: A Huffman decoder for processing raw (i.e. unpartitioned) Huffman encoded data on the GPU. It also includes a basic, sequential encoder.\n\nCode: [https://github.com/weissenberger/gpuhd](https://github.com/weissenberger/gpuhd)\n\nPaper: [ICPP’18] Weißenberger, André, and Bertil Schmidt. \"Massively parallel Huffman decoding on GPUs.\" In Proceedings of the 47th International Conference on Parallel Processing, pp. 1-10. 2018.\n\n```\n[compress/decompress]\n./demo 0 100\ngenerating random data.. 64527550µs\ngenerating coding tables.. 1003450µs\nencoding.. 1274812µs\nGPU buffer allocation.. 509326µs\nGPU/Host aux memory allocation.. 2601µs\nGPU memcpy HtD.. 14547µs\ndecoding.. 1520µs\nGPU memcpy DtH.. 24461µs\n```\n\nCode: [https://github.com/smadhiv/HuffmanCoding_MPI_CUDA](https://github.com/smadhiv/HuffmanCoding_MPI_CUDA)\n\nAbstract: We present an implementation of Lempel-Ziv-Storer-Szymanski (LZSS) lossless data compression algorithm by using NVIDIA GPUs CUDA Framework. Our system outperforms the serial CPU LZSS implementation by up to 18x, the parallel threaded version up to 3x and the BZIP2 program by up to 6x in terms of compression time.\n\nCode: [https://github.com/adnanozsoy/CUDA_Compression](https://github.com/adnanozsoy/CUDA_Compression)\n\nPaper: [CLUSTER’11] Ozsoy, A. and Swany, M., 2011, September. CULZSS: LZSS lossless data compression on CUDA. In 2011 IEEE International Conference on Cluster Computing (pp. 403-411). IEEE.\n\n```\n[compress]\n./main -i  /home/dtao/GPU-lossless-compression/testdata/largefile -o /home/dtao/GPU-lossless-compression/testdata/largefile.lzss\nfile size:3569598 eachblock_size:1048576 num_of_blocks:4 padding:624706\nInitializing the GPU\nAll the time took:  1.013202\nThroughput for 4 runs of 3MB is :   23.687281Mbps\n\n[decompress]\n./main -d 1 -i /home/dtao/GPU-lossless-compression/testdata/largefile.lzss -o /home/dtao/GPU-lossless-compression/testdata/largefile.lzss.out\nDoing decompression\nNum bufs 4 padding size 624706 bufsize 1048576\nInitializing the GPU\nAll the time took:  0.802943\nThroughput for  2MB is :    19.926695Mbps\n```\n\n**Compression Ratio = 1.60**\n\nCode: [https://github.com/abshkbh/cuda-lzss](https://github.com/abshkbh/cuda-lzss)\n\n```\n[GPU compress]\n./cuda_main -f /home/dtao/GPU-lossless-compression/testdata/largefile\nBefore encoding\nNumblocks = 72\nTime for CPU to GPU is 0sec and 794usec\nTime for GPU to CPU is 0sec and 2254usec\nTotal compressed output : 3092074\nOnly Kernel Time is 1sec and 122868usec\nTotal Time is 14sec and 875701usec\nAfter encoding\n```\n\n**Compression Ratio : 1.154435**\n\n```\n[CPU compress]\n./sample -c -i /home/dtao/GPU-lossless-compression/testdata/largefile -o /home/dtao/GPU-lossless-compression/testdata/largefile.lzss2\nBefore encoding\nT1 is 1557765394sec and 124395usec\nT2 is 1557765400sec and 706787usec\n```\n\n**Compression Ratio: 1.925268**", "url": "https://wpnews.pro/news/gpu-lossless-compression-gpu-accelerated-lossless-data-compressors-survey", "canonical_source": "https://github.com/dingwentao/GPU-lossless-compression", "published_at": "2026-11-21 21:21:52+00:00", "updated_at": "2026-11-21 21:44:14.540609+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools"], "entities": ["Dingwen Tao", "Washington State University", "NVIDIA V100", "cuSZ", "CUDPP", "libbsc", "bzip2-cuda", "cuda_bzip2"], "also_reported_by": [], "takeaway": "On an NVIDIA V100, CUDPP 2.1's cudppCompress runs 2.78× slower than bzip2 (BWT 2.89× and MTF-Huffman 1.34× slower), while a hybrid all-core GPU Burrows-Wheeler implementation reaches a 2.9× speedup over the best CPU version.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/gpu-lossless-compression-gpu-accelerated-lossless-data-compressors-survey", "markdown": "https://wpnews.pro/news/gpu-lossless-compression-gpu-accelerated-lossless-data-compressors-survey.md", "text": "https://wpnews.pro/news/gpu-lossless-compression-gpu-accelerated-lossless-data-compressors-survey.txt", "jsonld": "https://wpnews.pro/news/gpu-lossless-compression-gpu-accelerated-lossless-data-compressors-survey.jsonld"}}