Low-Precision Floating-Point: Why 6-Bit and 4-Bit Formats Are Gaining Ground
- Published
- Author
- Max Maxfield
- Reads
- 484
Lemieux and Overton's paper explains why 6-bit and 4-bit floating-point formats are emerging for AI, and why block scaling makes four bits workable.

Numerical representation sounds like a settled topic, but the bit layout you pick still decides speed, memory footprint, energy use and how much accuracy you keep. I recently spoke with Guy Lemieux, a professor at the University of British Columbia, and Michael Overton, a professor at New York University, about their paper The Rise of Low Precision Floating Point. Both are members of the IEEE P3109 Working Group. Their argument is that the field is moving from 32 bits down to 4, and that this shift needs a clear framework before every vendor invents its own format.

Why the number of bits is a design decision
Numerical computing underpins weather forecasting, finite element and computational fluid dynamics work, medical imaging, graphics and AI. Some workloads repeat simple operations billions of times. Others push complex calculations over huge datasets. In both cases, the way you encode a number inside the machine shapes the speed and accuracy of the result, the memory needed to hold the data and the energy spent moving and processing it.
The simplest encoding is fixed-point. You choose where the binary point sits and leave it there. With 16 bits, an 8.8 format gives eight integer bits and eight fractional bits. Shift the point and you get 12.4 or 4.12. Push it to the extremes and you get 16.0, which is an ordinary 16-bit integer, or 0.16, which covers values from 0 up to just under 1. Every one of these formats uses exactly 16 bits. Only the interpretation changes, and that interpretation sets both range and precision.
Fixed-point still earns its place where value ranges are predictable. Audio processing, digital filters and communications systems have leaned on it for years, especially in embedded code. Many inexpensive microcontrollers have no hardware floating-point unit, so fixed-point calculations run faster and use fewer resources. Motor control and industrial automation favour it when execution time has to be predictable. Fixed-point decimal arithmetic also represents money exactly to a set number of decimal places, sidestepping some binary rounding problems. And quantized neural networks often use 8-bit, 6-bit or 4-bit integers with scaling factors, which makes them a form of fixed-point arithmetic in practice.
Reading a floating-point format
Floating-point borrows the scientific notation you learned in school. The value 325.62 becomes 3.2562 × 102, and 0.0032562 becomes 3.2562 × 10-3. The digits 3.2562 form the significand, historically called the mantissa, and the exponent tells you where the decimal point lands. Normalization forces exactly one non-zero digit to the left of the point, and that is what lets the exponent do its job of floating the point around.
Binary works the same way, with two differences that matter for bit budgets. First, a normalized binary significand always starts with a 1, so you don't need to store that bit at all. It's the hidden or implicit bit, and it buys you one extra bit of precision for free. Second, the exponent needs a sign, and designers usually handle that with a bias: a fixed value is added to the real exponent before storage. That removes the need for a separate exponent sign bit and makes exponent comparison simpler in hardware.
A 16-bit layout with one sign bit, five exponent bits and ten fraction bits, which the authors describe as 1.5.10, gives you 11 bits of significand precision including the hidden bit. This is IEEE 754 half-precision, commonly called FP16. The sign bit is 0 for positive values and 1 for negative ones, and the exponent's sign is independent of the value's sign, so a number like −3.2562 × 10-3 still carries a negative exponent.
The payoff is dynamic range, meaning the ratio between the largest and smallest non-zero magnitudes you can represent. A floating-point format can cover very large and very small values without needing a huge number of bits. The price is that spacing between adjacent representable values grows with magnitude. Fixed-point gives you uniform resolution. Floating-point trades that for range. The old saying applies: there ain't no such thing as a free lunch, as Robert A. Heinlein put it in his 1966 novel The Moon Is a Harsh Mistress.

Fixed-point representations at 16 bits, with the binary point placed in different positions

A 16-bit floating-point layout showing sign, exponent and fraction fields
Why the bits get split differently
Compare three 16-bit floating-point layouts and the trade-offs become concrete. The first is a hypothetical format in which all 32 possible exponent patterns encode ordinary normalized numbers. IEEE FP16 gives up some of those patterns to represent zero, infinity, NaN (Not a Number) and subnormal numbers. Google's BF16 takes a different route, using eight exponent bits and seven fraction bits, or 1.8.7. That gives BF16 a far wider dynamic range than FP16, at the cost of precision. The three layouts use identical bit counts, and the chart makes clear how much the split matters.

The same 16 bits produce very different ranges depending on how they are divided between exponent and fraction
This is the design space the paper is really about. Each format is a bet on which error hurts more in a given workload: losing precision in each value, or running out of range entirely.
Where IEEE 754 came from
Floating-point predates any agreed standard. For decades, computer manufacturers shipped their own formats and arithmetic rules. Moving numerical software from one machine to another could be painful, and results could differ in ways that were hard to trace. That changed with IEEE 754, introduced in 1985 by the Institute of Electrical and Electronics Engineers (IEEE). It defined how numbers are represented, how operations are performed and rounded, and how special cases like infinity and invalid operations are handled.
The original standard covered 32-bit single precision (FP32) and 64-bit double precision (FP64), which became the mainstays of numerical computing. IEEE 754 has been revised since to cover new requirements, and industry groups and companies have added formats of their own. Overton's book Numerical Computing with IEEE Floating-Point Arithmetic is the reference I'd point anyone to for the details. Its use of special bit patterns for ±∞, its defined NaN behaviour and its rounding rules, including guard, round and sticky bits for correctly rounded results, are the reason floating-point results are reproducible across machines. That is still the bedrock, and the low-precision work is measured against it.
The downward trend
For decades, word sizes grew: 8-bit microprocessors gave way to 16-bit, then 32-bit, and 64-bit is now ordinary. The trend has reversed in AI. Engineers have moved to 8-bit and even 4-bit integers, and in specialized cases 1-bit values. Lemieux and Overton's paper documents the same pattern for floating-point. Deep learning pushed FP32 and FP64 aside for most of the workload, then 16-bit formats took over, followed by 8-bit, and now 6-bit and 4-bit floating-point formats are in use or under development.

Floating-point options currently in use or under development, adapted from Lemieux and Overton
The notation in that illustration follows the same convention as before: three numbers give the bits for sign, exponent and fraction. One entry needs a footnote. NVIDIA's TensorFloat-32 (TF32), introduced in 2020, has 19 effective bits in the layout shown, one sign bit, eight exponent bits and ten stored fraction bits (1.8.10). TF32 operates within a 32-bit environment, using reduced precision for certain calculations while keeping FP32 storage and accumulation. So the count of 19 is correct for the representation, even though the name suggests 32.
How four bits can do useful work
Four bits give you only 16 possible patterns, which sounds hopeless for arithmetic. The answer combines three factors: memory and bandwidth, energy, and the tolerance of neural networks for reduced precision.
First, fewer bits per value means more values fit in the same memory and more values cross the same interconnect in the same time. Arithmetic circuits working on smaller operands can also need fewer transistors and less energy. When an AI system performs billions or trillions of operations, those savings add up quickly.
Second, neural networks don't usually need the precision that scientific computing demands. If a network is deciding whether a photograph contains a chicken, an intermediate value of 0.87654321 versus 0.88 rarely changes the answer. Some calculations are far more sensitive than others, so this isn't a blanket license to throw away bits. But the tolerance is real and it's what makes aggressive formats viable.
Third, and to my mind the most interesting point, the limited range of tiny formats can be recovered with scale factors. Instead of asking each 4-bit value to cover an enormous span, you group values into blocks and attach a scaling factor to each block. Different blocks then represent different ranges. NVIDIA's NVFP4 implementation works this way: each block of 16 four-bit values carries an 8-bit scale factor, and a 32-bit scale factor applies across the whole tensor containing those blocks. The individual values still use four bits, but the scaling makes the scheme far more capable than four bits would suggest.
Reduced precision also isn't suitable everywhere, which is why modern AI accelerators tend to use mixed precision. Some operations run in low-precision formats while others keep higher precision. A common pattern multiplies two low-precision values and accumulates the results in a higher-precision format. That captures much of the performance benefit while containing the accuracy loss.
The standardization problem
Here's the risk: with so many companies building their own low-precision formats, the field could drift back to the floating-point Wild West that IEEE 754 was meant to end. Two efforts are trying to prevent that. The Open Compute Project (OCP), with contributions from AMD, Arm, Google, Intel, Meta and NVIDIA, introduced specifications for 8-bit, 6-bit and 4-bit formats in 2023. Separately, the IEEE P3109 Working Group, which includes Lemieux and Overton, is developing a broader framework for low-precision arithmetic, with formal standardization anticipated in 2027.
Bit layouts alone won't settle compatibility. Some OCP and IEEE P3109 formats share identical arrangements of sign, exponent and fraction bits but don't represent exactly the same set of numbers. Two formats that look the same on a datasheet can produce different results. The details of which bit patterns map to which values, and how special cases are handled, decide interoperability. That is precisely the lesson IEEE 754 taught the industry about FP32 and FP64, and it's being relearned at the 4-bit end.
What the paper covers and what it doesn't claim
The paper runs to 10 pages plus two pages of references. It's readable for an engineer without a numerical analysis background, yet dense enough to hold the attention of someone who lives in floating-point error analysis. The authors trace the history of low-precision formats, explain why they matter now, compare the competing designs and discuss the challenges of standardizing them.
Neither author argues that low-precision floating-point will replace everything else. Sixteen-bit and eight-bit formats are already well established in AI. How far 6-bit and 4-bit floating-point spreads beyond current uses is still an open question, and that's the honest position to take. Anyone building hardware or software around these formats should treat today's block-scaling schemes as the current state of practice, not a settled endpoint.
What to watch
My read is that the next useful question isn't how few bits a format can squeeze into, but which bits each job actually needs. Accelerator vendors will keep pushing formats down the bit count because memory bandwidth and energy per operation are expensive. The engineering work that matters is in the details: scale factor granularity, how the hidden bit and special values are handled, which operations stay in higher precision and how accumulation is done. The IEEE P3109 effort, with its anticipated 2027 standardization, is where those details will either converge or fragment.
So the future of numerical computing probably isn't about ever-more bits. It's about the right number of bits for the job. And if that job turns out to be useful floating-point arithmetic on four bits, with a few scale factors doing the heavy lifting, then WTW (What the What) is the right reaction.