The number 1.58 sounds like a wall. For ternary language models, which store each weight as -1, 0 or +1, it is the theoretical minimum, and Intel researchers have just walked under it. Their new format, BITCOS, packed one checkpoint down to 1.485 bits per weight. It did it without retraining the model or changing a single weight, and decoding got faster by as much as 18% on CPUs and 27% on GPUs.
The trick is in what the 1.58 figure assumes. It holds only if the three values turn up equally often. Real ternary models are full of zeros.
Why the old packing wastes space
The usual approach fits five ternary values into one eight-bit byte, or 1.6 bits each. Weights are commonly stored in blocks of 128, which leaves the last byte partly empty and pushes the real cost to 1.625 bits per weight.
Intel’s team measured 29 checkpoints from seven ternary model families and found zeros making up between 29.7% and 51.5% of the weights. The sparsest was a ternary Qwen3-1.7B made with CAT-Q post-training quantization, at 51.48% zeros.
Two streams instead of one
BITCOS stands for “BITmap and COmpacted Signs.” It splits the weights into two streams. One bit per weight records whether it is zero. A second stream holds a sign bit only for the nonzero weights. A positive or negative weight costs two bits; a zero costs one.
The arithmetic is simple. With z as the share of zeros, the cost is 2 minus z bits per weight, which falls from 1.6 bits at 40% zeros to 1.485 at 51.5%. BITCOS beats five-value packing once more than 37.5% of weights are zero, and 26 of the 29 checkpoints cleared that line. Because only the storage layout changes, unpacking returns the original values and accuracy is untouched. Intel’s own comparison is packing the same contents into a smaller box.
Speed on real hardware
The format targets token-by-token decoding at small batch sizes, where moving weights through memory is the bottleneck. Intel wrote separate unpacking kernels for AVX-512 and AVX2 CPUs and for Xe2 GPUs. On AVX-512, the kernel uses the presence bitmap as a mask and the pdep instruction to scatter sign bits across the nonzero positions. Xe2 GPUs have no such instruction, so Intel used a 2KB lookup table.
Against 2-bit kernels, BITCOS ran 10% to 18% faster on a 64-core Xeon server and 2% to 15% faster on a 24-core Core Ultra 9. On the integrated Arc 140V the gain was 9% to 22%, and on the discrete Arc Pro B70 it was 2% to 27%.
It did not win everywhere. On the eight-core Lunar Lake CPU, the fixed 2-bit kernel beat BITCOS on every model, because bandwidth was ample and unpacking became the bottleneck. BITCOS stayed ahead on the GPUs, though decoding overhead trimmed the gains.
The paper has not been peer-reviewed. All five test systems used Intel hardware, and the end-to-end benchmarks covered seven models at batch size one. Intel has yet to test the format on Nvidia, AMD or Arm hardware.
