GPU-Lossless-Compression – GPU-Accelerated Lossless Data Compressors Survey A GitHub survey by Washington State University researcher Dingwen Tao benchmarks GPU-accelerated lossless compressors on an NVIDIA V100 GPU with 16 GB of memory, evaluating candidates for pairing with the CUDA-based lossy compressor cuSZ. The survey reports that CUDPP 2.1's cudppCompress is 2.78× slower than bzip2, with its Burrows-Wheeler and Move-to-Front/Huffman stages 2.89× and 1.34× slower respectively, while a hybrid all-core Burrows-Wheeler implementation (cuda_bzip2) achieves a 2.9× speedup over the best CPU implementation. The survey also records libbsc compressing a 3,569,598-byte test file to 159,230 bytes in 0.147 seconds (ratio 22.42) and decompressing it in 0.215 seconds, and notes CUDA 10 does not work with CUDPP while CUDA 9.1 appears to. This repository suveys state-of-the-art GPU-accelerated lossless compressors. The evaluation is conductd using an NVIDIA V100 GPU 16 GB memory . Besides lossless compression survey, another purpose for this survey is to identify suitable GPU-accelerated lossless compressor for our developed CUDA-based lossy compressor called cuSZ , which can be accessed at https://github.com/szcompressor/cuSZ https://github.com/szcompressor/cuSZ . Please feel free to contact me dingwen.tao@wsu.edu mailto:dingwen.tao@wsu.edu if you find any issue or any lossless compressor that has not been covered in this survey. Abstract: CUDPP release 2.1 is a feature release that adds many new parallel primitives. We have added cudppCompress, a lossless data compression algorithm. This compression utilizes efficient parallel Burrows-Wheeler and Move-to-Front transforms BTW which are also exposed through the API. GPU implementation is dominated by BWT performance and is 2.78× slower than bzip2, with BWT and MTF-Huffman respectively 2.89x and 1.34x slower. Code: http://cudpp.github.io/ http://cudpp.github.io/ Paper: InPar’12 Patel, R.A., Zhang, Y., Mak, J., Davidson, A. and Owens, J.D., 2012. Parallel lossless data compression on the GPU pp. 1-9 . IEEE. Slides: http://on-demand.gputechconf.com/gtc/2012/presentations/S0361-Lossless-Data-Compression-on-GPUs.pdf http://on-demand.gputechconf.com/gtc/2012/presentations/S0361-Lossless-Data-Compression-on-GPUs.pdf Cuda 10 is not working, Cuda 9.1 seems working according to cudpp/cudpp 185 https://github.com/cudpp/cudpp/issues/185 . Abstract: A library for lossless, block-sorting data compression. bsc is a high-performance file compressor based on lossless, block-sorting data compression algorithms. libbsc is a library based on bsc, it uses the same algorithms as bsc and enables you to compress memory blocks. Code: https://github.com/IlyaGrebnov/libbsc https://github.com/IlyaGrebnov/libbsc , http://libbsc.com/ http://libbsc.com/ Paper: Burrows, M. and Wheeler, D.J., 1994. A block-sorting lossless data compression algorithm. https://www.hpl.hp.com/techreports/Compaq-DEC/SRC-RR-124.pdf https://www.hpl.hp.com/techreports/Compaq-DEC/SRC-RR-124.pdf compress ./bsc e /home/dtao/GPU-lossless-compression/testdata/largefile /home/dtao/GPU-lossless-compression/testdata/largefile.bsc -G This is bsc, Block Sorting Compressor. Version 3.1.0. 8 July 2012. Copyright c 2009-2012 Ilya Grebnov