ZipNN: Lossless Compression for AI Models

Moshik Hershcovitch; Andrew Wood; Leshem Choshen; Guy Girmonsky; Roy Leibovitz; Or Ozeri; Ilias Ennmouri; Michal Malka; Peter Chin; Swaminathan Sundararaman; Danny Harnik

doi:10.1109/CLOUD67622.2025.00028

ZipNN: Lossless Compression for AI Models

lossless compression tailored to neural network weight files

Moshik Hershcovitch, Andrew Wood, Leshem Choshen, Guy Girmonsky, Roy Leibovitz, Or Ozeri, Ilias Ennmouri, Michal Malka, Peter Chin, Swaminathan Sundararaman, Danny Harnik · CLOUD 2025 · 2025

In one sentence

ZipNN is a lossless compressor for neural network files that separates the highly skewed floating-point exponent bytes into their own stream and encodes them with Huffman coding only, shrinking popular BF16 models by about a third with no loss of any bit.

Abstract

With the growth of model sizes and the scale of their deployment, their sheer size burdens the infrastructure requiring more network and more storage to accommodate these. While there is a vast model compression literature deleting parts of the model weights for faster inference, we investigate a more traditional type of compression - one that represents the model in a compact form and is coupled with a decompression algorithm that returns it to its original form and size - namely lossless compression. We present ZipNN, a lossless compression tailored to neural networks. Somewhat surprisingly, we show that specific lossless compression can gain significant network and storage reduction on popular models, often saving 33% and at times reducing over 50% of the model size. We investigate the source of model compressibility and introduce specialized compression variants tailored for models that further increase the effectiveness of compression. On popular models (e.g. Llama 3) ZipNN shows space savings that are over 17% better than vanilla compression while also improving compression and decompression speeds by 62%. Using multiple workers and threads, ZipNN can achieve decompression speeds of up to 80GB/s and compression speed of up to 13GB/s. We estimate that these methods could save over an ExaByte per year of network traffic downloaded from a large model hub like Hugging Face.

Questions this paper answers

can a model file be made smaller without changing the weights at all?
do BF16 and FP32 neural network weights carry enough redundancy for lossless entropy coding to pay off?
how do I shrink the download size of a model checkpoint without quantizing or pruning it?
is it worth storing my models compressed, or are the weights too close to random to gain anything?: The compressibility of neural network weights comes almost entirely from the floating-point exponent, while the fraction and sign bits are close to random. Across four models only around 40 of the 256 possible exponent values appear, and the top 12 account for almost 99.9% of parameters.
Holds for: Measured on 1GB taken from the middle of each model, for BF16 and FP32 language models plus ResNet, where 50 exponent values appear and the top 17 dominate.; ZipNN compresses BF16 models to about 66% of their original size, with the exponent byte group reaching roughly 32.5-34.8% and the remaining bytes essentially incompressible. Llama-3.1 goes to 66.4%, Mistral to 66.3% and Bloom to 67.4%.
Holds for: BF16 models trained and left unmodified after training; measured on 1GB from the middle for large models. FP32 regular models such as Olmo and Wav2vec only reach about 83%.; ZipNN makes the case that lossless compression, not just pruning and quantization, is a lever on model storage and network cost. It argues lossless compression should be the default for traffic to and from model hubs like Hugging Face.
Holds for: As of publication in 2025; the argument rests on measurements of Hugging Face-hosted models in FP32, BF16 and FP16, and some GGUF-quantized models do not compress at all.
why are the numbers inside a trained model compressible when they look random?
which byte of an FP32 or BF16 parameter carries the redundancy that entropy coders exploit, the sign, exponent or mantissa?
how do I work out which part of a weight tensor to target for compression?
should I expect gains from compressing the mantissa bits of my checkpoints?: The compressibility of neural network weights comes almost entirely from the floating-point exponent, while the fraction and sign bits are close to random. Across four models only around 40 of the 256 possible exponent values appear, and the top 12 account for almost 99.9% of parameters.
Holds for: Measured on 1GB taken from the middle of each model, for BF16 and FP32 language models plus ResNet, where 50 exponent values appear and the top 17 dominate.; Pure repetition-removal compressors gain nothing on model weights: LZ4 and Snappy achieve zero compression savings on the tested models despite being faster than every other method.
Holds for: Tested on Llama-3.1 (BF16), Olmo-1b (FP32) and xlm-RoBERTa (FP32); model tensors are noisy and unstructured, so multi-byte repetitions are scarce.
is a compressor built for model files better than a general-purpose one on a big language model?
how do ZipNN compression ratio and throughput compare with Zstd at its default level on BF16 and FP32 checkpoints?
how do I get better than gzip- or zstd-level compression on the model files I ship?
I already zstd my checkpoints in the storage pipeline, would switching to a model-aware compressor gain me anything?: Against Zstd with default configuration, ZipNN takes Llama-3.1 BF16 from 77.7% to 66.4% compressed size while raising compression speed from 0.71 to 1.15 GB/s and decompression from 1.02 to 1.65 GB/s.
Holds for: Single thread on a single core of an Apple M1 Max with 64GB RAM, 10 runs over 1GB from the middle of the model, maximum observed standard deviation 2%.; On the clean FP32 model xlm-RoBERTa, ZipNN reaches 42.9% compressed size versus 57.4% for Zstd, with compression speed rising from 0.18 to 0.83 GB/s and decompression from 0.77 to 1.41 GB/s.
Holds for: Single thread, single core, Apple M1 Max; 1GB from the middle of the model. Speedups are largest on clean models because more of the data is compressible rather than skipped.
does looking for repeated byte patterns help when compressing model weights?
is the Lempel-Ziv stage worth keeping over Huffman-only entropy coding on floating-point model tensors?
how do I pick a compression algorithm for weight tensors when speed matters as much as ratio?
can I just use LZ4 or Snappy on my model files since they are the fastest option?: Dropping the Lempel-Ziv stage and using Huffman entropy coding alone improves both speed and compression ratio on model exponents. The repetitions LZ finds are artifacts of the skewed distribution: randomly shuffling a model's parameters changes the exponent compression ratio by at most 0.05%.
Holds for: Exponent already separated into its own stream; Huffman without exponent extraction only helps speed. An FSE encoder gains 0-2% more ratio at times over 2X the time cost.; Pure repetition-removal compressors gain nothing on model weights: LZ4 and Snappy achieve zero compression savings on the tested models despite being faster than every other method.
Holds for: Tested on Llama-3.1 (BF16), Olmo-1b (FP32) and xlm-RoBERTa (FP32); model tensors are noisy and unstructured, so multi-byte repetitions are scarce.
why do some trained models shrink much more than others when compressed losslessly?
why do checkpoints that were rounded or cast between parameter types compress below the exponent-entropy limit?
how do I predict how well a given checkpoint will compress before running a compressor over it?
my model was saved after converting from FP32, should I expect better than the usual one-third saving?: So-called clean models, which were rounded or converted between parameter types after training, compress far below the exponent-only limit: T5-base (FP32) reaches 33.7% of its original size, XLM-RoBERTa 41.8% and Clip 48.1%.
Holds for: Requires byte grouping of the fraction bytes, since the gains come from near-zero fraction byte groups; clean models lose this extra compressibility once fine-tuned again.
is decompressing a model fast enough that it does not hold up loading?
what compression and decompression throughput is reachable with multi-worker NUMA-aware chunking on model files?
how do I saturate a network link or GPU load path while decompressing model weights on the fly?
if I keep my served models compressed on disk, will startup time get worse?: ZipNN reaches up to 80GB/s decompression and up to 13GB/s compression throughput with 16 workers, each worker kept within a NUMA node and given a block size as low as 100MB. A single worker with multiple threads peaks lower, above 45GB/s for decompressing 10GB, with compression peaking around 16 threads.
Holds for: Intel Xeon Platinum 8480+ pod, 224 cores across 2 NUMA nodes, 2TB DRAM; model size matters, and 100MB inputs reach far lower throughput than 10GB ones.; Loading a 16GB Granite-3.1-8b-instruct BF16 model stored at 2/3 its original size from a clustered file system to GPU, including decompression, took approximately 3 seconds, on par with loading the uncompressed model.
Holds for: OpenShift pod with Intel Xeon Platinum 8480+, vLLM 0.7.2 with 4 workers, PVC-backed file system delivering 8GB/s per worker; a slower CPU or fewer threads would change the balance.
do gradients and optimizer state shrink as well as the model weights during training?
how do compression ratios for optimizer states and gradients compare with the model checkpoint in RoBERTa fine-tuning?
how do I cut the volume of gradient and optimizer traffic in distributed fine-tuning without lossy tricks?
is compressing the optimizer state in my training pipeline worth the CPU cost?: Gradients and optimizer states compress better than the model itself during RoBERTa fine-tuning: the model reaches about 66% compressed size, the optimizer 54% and the gradients 47%. The extra gain is concentrated in the token embeddings layer.
Holds for: BF16 RoBERTa under fine-tuning; the embedding layer of gradients and optimizers compresses better with Zstd than with Huffman, unlike the model itself.
is it cheaper to store a training checkpoint as the difference from an earlier one?
how does delta compression of training checkpoints hold up when the base checkpoint is several epochs back?
how do I store a long series of training checkpoints without keeping a full copy of each one?
I save a checkpoint every epoch, should I keep periodic full bases and deltas in between?: Delta compression of checkpoints stays well ahead of standalone compression even when the base is 5 or 10 epochs away, though it is worse than delta against the immediately preceding checkpoint.
Holds for: Self-trained ResNet18 (FP32), Amber (BF16) and Olmo (FP32) public training checkpoints, using XOR deltas; the space taken by the periodic full bases is excluded from the reported figures.; ZipNN's per-chunk choice between Huffman and Zstd always matched or beat whichever of the two was better on ResNet18 checkpoint deltas. Zstd is selected when zeros exceed 90% of a chunk or any zero run passes 3% of chunk size.
Holds for: ResNet18 (FP32) fine-tuning deltas; Huffman wins in the first two learning-rate-scheduler steps and Zstd after the third, and the thresholds were fixed by simulation rather than tuned per model.
can many fine-tuned versions of the same base model be stored cheaply together?
how much do sibling fine-tunes of one pretrained backbone compress as pairwise deltas versus standalone?
how do I cut the storage cost of hosting dozens of fine-tunes of the same backbone?
I host several task-specific fine-tunes of one base model, should I store them as deltas from each other?: Three RoBERTa variants fine-tuned on tweets for irony, offensive-language and abuse detection compress to 83.7% of original size standalone. Stored as pairwise deltas they reach 56% on average.
Holds for: Models sharing the same base checkpoint; XOR delta plus lossless compression. Requires keeping the base available to reconstruct.
what should I read first about making model files smaller without touching the weights?
which work argues for lossless compression of model repositories rather than quantization or pruning?
where do I start reading if I want to cut model hub storage and bandwidth costs losslessly?: ZipNN makes the case that lossless compression, not just pruning and quantization, is a lever on model storage and network cost. It argues lossless compression should be the default for traffic to and from model hubs like Hugging Face.
Holds for: As of publication in 2025; the argument rests on measurements of Hugging Face-hosted models in FP32, BF16 and FP16, and some GGUF-quantized models do not compress at all.; Hugging Face stated in August 2024 that it holds 1.3M models over 12PB of storage and serves 1 billion daily requests, around 6PB of network bandwidth per day. Lossless compression is estimated to save over an ExaByte of downloaded traffic per year.
Holds for: The ExaByte figure is an estimate extrapolated from the hub's reported traffic and the measured compression ratios of top-downloaded models, not a measured saving.
how much download traffic do the big model-sharing sites actually move?
what storage footprint and daily request volume does Hugging Face report, and what bandwidth could lossless compression avoid?
is the bandwidth saving from compressing model downloads big enough to justify changing a hub's pipeline?: Hugging Face stated in August 2024 that it holds 1.3M models over 12PB of storage and serves 1 billion daily requests, around 6PB of network bandwidth per day. Lossless compression is estimated to save over an ExaByte of downloaded traffic per year.
Holds for: The ExaByte figure is an estimate extrapolated from the hub's reported traffic and the measured compression ratios of top-downloaded models, not a measured saving.
does keeping a model compressed on disk make the server slower to start?
what is the end-to-end load latency for a compressed BF16 checkpoint read from a clustered file system to GPU memory?
how do I load a compressed model into GPU memory without adding latency to inference server startup?
can I store my 16GB served model compressed without paying for it at load time?: Loading a 16GB Granite-3.1-8b-instruct BF16 model stored at 2/3 its original size from a clustered file system to GPU, including decompression, took approximately 3 seconds, on par with loading the uncompressed model.
Holds for: OpenShift pod with Intel Xeon Platinum 8480+, vLLM 0.7.2 with 4 workers, PVC-backed file system delivering 8GB/s per worker; a slower CPU or fewer threads would change the balance.

Claims and scope

The compressibility of neural network weights comes almost entirely from the floating-point exponent, while the fraction and sign bits are close to random. Across four models only around 40 of the 256 possible exponent values appear, and the top 12 account for almost 99.9% of parameters. (Figure 2 and Section III-A)
Scope: Measured on 1GB taken from the middle of each model, for BF16 and FP32 language models plus ResNet, where 50 exponent values appear and the top 17 dominate.
ZipNN compresses BF16 models to about 66% of their original size, with the exponent byte group reaching roughly 32.5-34.8% and the remaining bytes essentially incompressible. Llama-3.1 goes to 66.4%, Mistral to 66.3% and Bloom to 67.4%. (Table II)
Scope: BF16 models trained and left unmodified after training; measured on 1GB from the middle for large models. FP32 regular models such as Olmo and Wav2vec only reach about 83%.
So-called clean models, which were rounded or converted between parameter types after training, compress far below the exponent-only limit: T5-base (FP32) reaches 33.7% of its original size, XLM-RoBERTa 41.8% and Clip 48.1%. (Table II)
Scope: Requires byte grouping of the fraction bytes, since the gains come from near-zero fraction byte groups; clean models lose this extra compressibility once fine-tuned again.
Against Zstd with default configuration, ZipNN takes Llama-3.1 BF16 from 77.7% to 66.4% compressed size while raising compression speed from 0.71 to 1.15 GB/s and decompression from 1.02 to 1.65 GB/s. (Table III)
Scope: Single thread on a single core of an Apple M1 Max with 64GB RAM, 10 runs over 1GB from the middle of the model, maximum observed standard deviation 2%.
On the clean FP32 model xlm-RoBERTa, ZipNN reaches 42.9% compressed size versus 57.4% for Zstd, with compression speed rising from 0.18 to 0.83 GB/s and decompression from 0.77 to 1.41 GB/s. (Table III)
Scope: Single thread, single core, Apple M1 Max; 1GB from the middle of the model. Speedups are largest on clean models because more of the data is compressible rather than skipped.
Dropping the Lempel-Ziv stage and using Huffman entropy coding alone improves both speed and compression ratio on model exponents. The repetitions LZ finds are artifacts of the skewed distribution: randomly shuffling a model's parameters changes the exponent compression ratio by at most 0.05%. (Section III-A and Figure 4)
Scope: Exponent already separated into its own stream; Huffman without exponent extraction only helps speed. An FSE encoder gains 0-2% more ratio at times over 2X the time cost.
Pure repetition-removal compressors gain nothing on model weights: LZ4 and Snappy achieve zero compression savings on the tested models despite being faster than every other method. (Section V-B)
Scope: Tested on Llama-3.1 (BF16), Olmo-1b (FP32) and xlm-RoBERTa (FP32); model tensors are noisy and unstructured, so multi-byte repetitions are scarce.
ZipNN reaches up to 80GB/s decompression and up to 13GB/s compression throughput with 16 workers, each worker kept within a NUMA node and given a block size as low as 100MB. A single worker with multiple threads peaks lower, above 45GB/s for decompressing 10GB, with compression peaking around 16 threads. (Section V-C and Figure 10)
Scope: Intel Xeon Platinum 8480+ pod, 224 cores across 2 NUMA nodes, 2TB DRAM; model size matters, and 100MB inputs reach far lower throughput than 10GB ones.
Gradients and optimizer states compress better than the model itself during RoBERTa fine-tuning: the model reaches about 66% compressed size, the optimizer 54% and the gradients 47%. The extra gain is concentrated in the token embeddings layer. (Section IV-A and Figure 7)
Scope: BF16 RoBERTa under fine-tuning; the embedding layer of gradients and optimizers compresses better with Zstd than with Huffman, unlike the model itself.
Delta compression of checkpoints stays well ahead of standalone compression even when the base is 5 or 10 epochs away, though it is worse than delta against the immediately preceding checkpoint. (Figure 9)
Scope: Self-trained ResNet18 (FP32), Amber (BF16) and Olmo (FP32) public training checkpoints, using XOR deltas; the space taken by the periodic full bases is excluded from the reported figures.
Three RoBERTa variants fine-tuned on tweets for irony, offensive-language and abuse detection compress to 83.7% of original size standalone. Stored as pairwise deltas they reach 56% on average. (Section IV-B)
Scope: Models sharing the same base checkpoint; XOR delta plus lossless compression. Requires keeping the base available to reconstruct.
ZipNN's per-chunk choice between Huffman and Zstd always matched or beat whichever of the two was better on ResNet18 checkpoint deltas. Zstd is selected when zeros exceed 90% of a chunk or any zero run passes 3% of chunk size. (Figure 8 and Section IV-B)
Scope: ResNet18 (FP32) fine-tuning deltas; Huffman wins in the first two learning-rate-scheduler steps and Zstd after the third, and the thresholds were fixed by simulation rather than tuned per model.
Loading a 16GB Granite-3.1-8b-instruct BF16 model stored at 2/3 its original size from a clustered file system to GPU, including decompression, took approximately 3 seconds, on par with loading the uncompressed model. (Section V-D2)
Scope: OpenShift pod with Intel Xeon Platinum 8480+, vLLM 0.7.2 with 4 workers, PVC-backed file system delivering 8GB/s per worker; a slower CPU or fewer threads would change the balance.
ZipNN makes the case that lossless compression, not just pruning and quantization, is a lever on model storage and network cost. It argues lossless compression should be the default for traffic to and from model hubs like Hugging Face.
Scope: As of publication in 2025; the argument rests on measurements of Hugging Face-hosted models in FP32, BF16 and FP16, and some GGUF-quantized models do not compress at all.
Hugging Face stated in August 2024 that it holds 1.3M models over 12PB of storage and serves 1 billion daily requests, around 6PB of network bandwidth per day. Lossless compression is estimated to save over an ExaByte of downloaded traffic per year. (Section II-A1 and the abstract)
Scope: The ExaByte figure is an estimate extrapolated from the hub's reported traffic and the measured compression ratios of top-downloaded models, not a measured saving.

Common misreadings

The 33% saving reported for ZipNN is not uniform across model files: it applies to BF16 models, whereas regular FP32 and FP16 models compress only to about 83-85% and so-called clean models can reach 34%.
ZipNN is not a model-compression method in the pruning, distillation or quantization sense: decompression restores the file bit-for-bit, so accuracy is unchanged and inference-time memory of the running model is not reduced.
Lossless compression does not stack reliably on top of quantization: off-the-shelf GPTQ and AWQ models still compress to 85-91%, while GGUF-quantized models in the paper's tests did not compress at all.
The 80GB/s decompression figure is a multi-worker, many-core server throughput on a 224-core Intel Xeon Platinum 8480+ pod, not the single-thread rate, which is 1.65 GB/s for Llama-3.1 on an Apple M1 Max.
Compressibility of clean models is a property of post-training rounding, not of the architecture: further fine-tuning removes the extra compressibility and the model reverts to exponent-only savings.

Terminology in this paper

exponent extraction: Rearranging a floating-point tensor so that all exponent bytes of the parameters form one contiguous compression stream, separated from the sign and fraction bits that dilute their skewed distribution.
byte grouping: Splitting the bytes of each floating-point parameter into one stream per byte position, so that fraction bytes with different degrees of near-zero structure are compressed independently.
clean model: A model whose weights underwent rounding or a parameter-type conversion after training, leaving many fraction bits zero and making it compressible beyond the exponent; further fine-tuning removes this property.
regular model: A model that was trained and left unmodified afterwards, so only its exponent bytes are compressible and the fraction and sign bits are effectively random.
compressed size (%): The percentage of the original data remaining after compression, so lower is better: a gigabyte reduced to a quarter of a gigabyte has a compressed size of 25%.
delta compression: Storing a model as the XOR difference from a similar base model, plus lossless compression of that difference, so reconstruction requires both the base and the delta.
periodic base: A scheme in which a full standalone-compressed checkpoint is stored every k checkpoints and intermediate checkpoints are kept as deltas, bounding the length of delta chains needed for recovery.

How to cite

@INPROCEEDINGS{zipnn,
  author={Hershcovitch, Moshik and Wood, Andrew and Choshen, Leshem and Girmonsky, Guy and Leibovitz, Roy and Ozeri, Or and Ennmouri, Ilias and Malka, Michal and Chin, Peter and Sundararaman, Swaminathan and Harnik, Danny},
  booktitle={2025 IEEE 18th International Conference on Cloud Computing (CLOUD)}, 
  title={ZipNN: Lossless Compression for AI Models}, 
  year={2025},
  volume={},
  number={},
  pages={186-198},
  keywords={Training;Cloud computing;Computational modeling;Redundancy;Neural networks;Telecommunication traffic;Logic gates;Artificial intelligence;Model compression;Faces;compression;lossless compression;models;AI;language models},
  doi={10.1109/CLOUD67622.2025.00028}}

References

See the full reference list in the paper.