Number to GPU Float Converter: FP32, FP16, BF16, FP8, and FP4 Explained

A new online converter tool demonstrates how decimal numbers are stored across GPU floating-point formats from FP64 down to FP4, revealing the precision tradeoffs critical to modern AI workloads. The tool allows users to input a number and see its representation in formats like FP32, FP16, BF16, FP8, and FP4, illustrating why approximations are necessary for memory and compute efficiency. The accompanying tutorial explains the practical implications of these formats for AI training and inference.

Number to GPU Float Converter: FP32, FP16, BF16, FP8, and FP4 Explained This converter shows how a number is stored across GPU floating-point formats, from FP64 down to FP4, and the tutorial below explains what the resulting values mean in practical AI workloads. Number to GPU Floating-Point Precision Converter Converted Values: FP64: FP32: TF32: FP16: BF16: FP8 E4M3: FP8 E5M2: FP4 E2M1: This converter shows the raw FP4 E2M1 element value. MXFP4 and NVFP4 use shared block scales, so they are not exact single-number outputs. What This Converter Shows This page does two jobs at once. First, it lets you type a number and see what happens when that number is stored in several GPU floating-point formats. Second, it gives you a beginner-friendly explanation of why those formats exist, why AI engineers care so much about them, and why the same decimal number can come back as a slightly different value once you squeeze it into fewer bits. If you have seen terms like FP16, BF16, FP8, or FP4 in GPU specs and they felt abstract, this tutorial is meant to make them concrete. The short version is that a GPU almost never stores most real-world numbers exactly. Instead, it stores an approximation that fits into a fixed number of bits. The more bits you spend, the better your approximation usually is. The fewer bits you spend, the more memory and compute you save, but the more rounding error you accept. Modern AI runs on this tradeoff. Training and inference are not just about fast matrix multiplication. They are also about deciding which parts of the workload deserve more precision and which parts can survive with less. The converter above is useful because it exposes that tradeoff in a single glance. Enter a value like 1.1 , 13.625 , 123.456 , or 0.00009 , and you can see how different formats respond. FP64 and FP32 often stay close to the original. FP16 and BF16 may move a little more. FP8 can move a lot, especially when the number needs more precision than 2 or 3 mantissa bits can carry. FP4 becomes extremely coarse, which is why practical 4-bit AI systems normally need extra scaling information around the raw 4-bit values. The rest of this tutorial builds from first principles. We will start with the basic question of how a decimal number is converted into a floating-point format. Then we will connect that idea to GPU formats used in AI and deep learning. After that, we will walk through a full FP8 example step by step. Finally, we will explain why standalone FP4 is usually not useful by itself, even though "4-bit AI" is a very real thing in production systems. What This Converter Is Doing Under the Hood FP32, TF32, FP16, and BF16 all use a common binary-float encoding flow: determine sign, compute the normalized exponent and mantissa, apply bias, round the mantissa, and then decode the packed result back into a display string. That makes these outputs directly tied to an explicit bit-budgeted representation rather than hand-wavy estimates. FP64 on this page is displayed as the JavaScript number formatted to 32 decimal places. It is effectively the high-precision reference line for comparison inside this tool. FP32 is single precision. TF32 uses the same 8-bit exponent width as FP32 but a much shorter mantissa, which mirrors its role as a throughput-oriented tensor format. FP16 cuts both range and precision compared with FP32. BF16 keeps the 8-bit exponent width of FP32 but drastically reduces mantissa bits, which is why it preserves range much better than FP16 while still being much cheaper than FP32. The FP8 modes are intentionally distinct from each other. E4M3 means 4 exponent bits and 3 mantissa bits. E5M2 means 5 exponent bits and 2 mantissa bits. That one-bit swap is not cosmetic. It changes the personality of the format. E4M3 spends more of its tiny budget on precision. E5M2 spends more on range. The converter exposes both so you can see how the same decimal number gets treated differently depending on whether the design leans toward range or precision. The FP4 result is different again. The page does not pretend that a single raw 4-bit value is a complete modern AI quantization story. Instead, it shows a coarse E2M1-style element value so you can see just how little information fits into four bits by themselves. The note under the output is important: real MXFP4 and NVFP4 workflows rely on shared scales for blocks or groups of values. Without those shared scales, a raw FP4 value is usually too crude to carry useful numeric detail on its own. A Quick Tour of the Formats on This Page FP64 is double precision. It is excellent when you want a high-precision reference, but it is rarely the workhorse for mainstream deep learning because it is expensive in both memory and throughput. Most AI engineers only touch FP64 when debugging, validating, or doing numerically sensitive scientific workloads that happen to involve GPUs. FP32 is the classic 32-bit floating-point format and remains the baseline reference for much of machine learning. It has solid range and solid precision. Historically, many frameworks used FP32 by default because it was stable and broadly supported. It is still important, especially for accumulations, optimizer states, and components that should not be aggressively quantized. TF32 is best understood as a GPU-friendly training compromise. It preserves the large exponent range of FP32 but trims precision so tensor cores can move faster. In other words, it aims to give you much of FP32's numeric comfort while relaxing the amount of detail stored in the significand. It exists because a lot of deep learning cares more about decent range plus fast tensor math than about full FP32 mantissa fidelity. FP16 reduces memory and can be very fast, but it has a narrower exponent range than FP32 or BF16. That narrower range is why naive half-precision training used to need tricks like loss scaling more often. FP16 remains useful, especially for inference or well-behaved workloads, but it is not the universal answer to all precision problems. BF16 keeps the large exponent range of FP32 while cutting precision sharply. This makes it attractive for training, because it handles large dynamic range more gracefully than FP16 while still giving major memory and throughput benefits. In modern deep learning, BF16 is often the practical default low-precision format for training because it tends to be more forgiving. FP8 pushes the savings further. With only 8 bits, you cannot have both wide range and fine precision, so vendors and standards use multiple FP8 variants. E4M3 leans toward precision. E5M2 leans toward range. FP8 is appealing because it can cut memory in half again relative to FP16/BF16, and supported hardware can do a lot of FP8 tensor math quickly. But success with FP8 depends on careful use, mixed precision, and often per-tensor or per-block scaling. FP4 is the extreme end of the spectrum here. Four bits leave almost no room for expressive numeric detail. As a raw value format, FP4 is so coarse that many different decimals collapse into the same stored result. That is why modern 4-bit AI systems do not simply say, "Every number is just a standalone FP4 element and we are done." They add structure around those 4-bit values. How GPU Float Conversion Works Why Numbers Need Conversion at All Humans usually think in decimal because we count in base 10. We write numbers like 0.1 , 3.75 , or 123.456 and assume those values are simple. Computers, however, operate in binary. Binary only has two digits: 0 and 1. That means the hardware is naturally comfortable with powers of 2, not powers of 10. A number that feels compact in decimal may need an infinite repeating representation in binary, just like 1/3 repeats forever in decimal as 0.333333.... A classic example is 0.1 . In decimal it looks exact and harmless. In binary, it repeats. So when software stores 0.1, it is already storing an approximation before AI-specific formats even enter the picture. This is not a GPU problem and it is not a bug in the converter. It is a basic fact of finite binary storage. Once you understand that every floating-point format is a limited binary approximation system, the rest of the precision story becomes much easier to reason about. There is another reason numbers need conversion: hardware cannot afford infinite storage. GPUs process huge tensors, often billions or trillions of values over time. If every value were kept at the highest precision all the time, memory traffic would explode, caches would become less effective, and throughput would fall. AI workloads are unusually sensitive to this because they involve large repeated matrix multiplies. When you shrink the data type, you do not just save disk space or VRAM. You reduce the amount of data the GPU has to move through memory buses, caches, and tensor units. That is why conversion is not an afterthought. In AI, choosing a number format is part of system design. It affects model size, training stability, inference speed, deployment cost, batch size, and even which GPU can run the workload at all. A 70B parameter model stored in a smaller format may fit on hardware that would be impossible at FP32. So the question is not only, "Can this number be represented?" The question is also, "Can it be represented well enough for the job while unlocking major gains in efficiency?" The Three Main Parts of a Floating-Point Number Most floating-point formats break a number into three conceptual parts: a sign bit, exponent bits, and mantissa bits. Some people call the mantissa the significand or fraction. The names vary, but the idea is the same. You store whether the number is positive or negative, how large or small it is in terms of powers of 2, and a short binary pattern that describes the meaningful digits. The sign is the easiest part. One bit is enough. A sign bit of 0 usually means positive, and a sign bit of 1 means negative. That lets the same format represent both +13.5 and -13.5 with the same magnitude information. The exponent controls scale. It says where the binary point moves. This is the part that makes floating-point feel like scientific notation. In decimal scientific notation, you might write 1230 as 1.23 x 10^3 . In binary floating-point, you do something similar but with powers of 2 instead of powers of 10. The exponent lets a format represent both tiny numbers and huge numbers without storing every leading or trailing zero explicitly. The mantissa carries precision. It stores the fine details once the value has been normalized. More mantissa bits mean more nearby representable values and smaller rounding steps. Fewer mantissa bits mean the format can only jump in larger increments. That is why two formats can have the same general range yet behave very differently for delicate calculations: one may have more precision, even if both can reach roughly similar magnitudes. For AI work, the most important mental model is this: exponent bits mostly buy you range, and mantissa bits mostly buy you precision. If you take bits away from the exponent, very large and very tiny values become harder to represent. If you take bits away from the mantissa, nearby values collapse together and rounding gets more aggressive. Formats like BF16 and FP8 exist because different AI tensors care about range and precision in different proportions. How a Decimal Number Becomes a Floating-Point Value The conversion pipeline is conceptually simple even though the edge cases get technical. Start with a decimal number, determine its sign, convert its magnitude into binary, normalize it, encode the exponent with a bias, round the mantissa to however many fraction bits the target format allows, and then pack everything into the target bit width. If the number is too large, it overflows. If it is too small, it may become a subnormal number or collapse all the way to zero. Let us unpack those steps in plain language. First, the converter checks the sign. That part is trivial. A negative number gets a negative sign bit. Zero is special because binary floating-point can represent both positive zero and negative zero, even though they compare as equal in most everyday code. Next, the magnitude of the number is rewritten in binary scientific notation. For example, decimal 13.625 becomes binary 1101.101 , which can be normalized as 1.101101 x 2^3 . That normalized form is important because most binary floating-point systems assume a leading 1 for normal numbers and then store only the fraction bits that come after that leading 1. After normalization, the exponent is not usually stored as a signed integer directly. Instead, formats use a bias . Suppose a format has bias 7 and the real exponent is 3. The stored exponent becomes 10. If the real exponent is -2, the stored exponent becomes 5. This trick makes it easy to represent both positive and negative exponents without needing a separate sign bit for the exponent field. Then comes rounding. This is where precision is lost. If the target format has only a few mantissa bits, most binary digits must be thrown away. The converter keeps what fits and rounds according to the target logic. Some formats in this code path use a general binary-float encoder with round-to-nearest-even behavior. The FP8 E4M3 path is simpler and rounds with the granularity that format permits. Either way, the central idea is the same: finite storage forces a choice of the nearest representable value. Finally, the stored bits can be decoded back into an ordinary number for display. That is what this tool shows you. It does not show the raw bit pattern directly. It shows the value you effectively get after quantization into the target format and then interpreting that quantized value again as a number. That is why the outputs are so useful for intuition. You can see the value drift that a real tensor element would experience in practice. Why Smaller Formats Matter So Much for AI and Deep Learning AI workloads are dominated by linear algebra on large tensors. A transformer does huge numbers of multiply-accumulate operations on activations, weights, gradients, optimizer states, and caches. Every one of those values occupies memory and must move through the hardware. When you cut the size of each element, you reduce the cost of that movement. In practice, this is often as important as the raw arithmetic speed of the GPU itself. The most obvious benefit is lower memory use . FP32 uses 4 bytes per value. FP16 and BF16 use 2 bytes. FP8 uses 1 byte. A smaller type means more weights fit in the same VRAM, more activations fit during training, and more tokens fit in key-value caches during inference. For large models, this changes which jobs are feasible on a given GPU. Sometimes the difference between not fitting and fitting comfortably is simply the choice of numeric format. The second benefit is higher throughput . Modern GPUs include specialized tensor hardware that can process lower-precision values more efficiently than full FP32 values. If the tensor cores can read smaller operands and do more work per cycle, performance increases. That is why vendor marketing repeatedly talks about TF32, FP16, BF16, FP8, and even FP4 throughput. These are not theoretical data types sitting in a standards document. They are tied directly to hardware pipelines designed to accelerate the kinds of matrix math that dominate AI. The third benefit is better system-level efficiency . Smaller formats reduce bandwidth pressure. They can improve cache behavior because more values fit in the same cache space. They can reduce interconnect traffic in distributed setups. They can lower the cost of serving models at scale. Even when arithmetic is not the bottleneck, memory movement often is. Precision reduction attacks that bottleneck directly. The fourth benefit is larger practical model capacity . This matters both for hobbyists and large organizations. A single-GPU user trying to run a local model may need lower precision simply to fit it into VRAM. A datacenter operator may already be able to fit the model, but may want smaller formats to increase batch size, improve token throughput, or reduce infrastructure cost per query. In both cases, the format decision changes what the system can do in real life. Finally, smaller formats support mixed-precision strategies . AI engineers rarely insist that every tensor use the same format. Instead, they use more precision where it matters and less where it does not. For example, accumulation may happen in a wider type than the stored inputs. Master weights may be kept in a different precision than activations. Certain sensitive layers may stay in BF16 while others move to FP8. The point is not to worship a single low-precision format. The point is to use the smallest format that keeps the workload reliable. The Compromises: What You Give Up When You Shrink the Format If smaller formats were strictly better in every way, AI would have abandoned FP32 long ago. The reason multiple formats coexist is that they solve different parts of the problem. Lower precision buys speed and efficiency, but it also increases risk. The most direct risk is rounding error. Two nearby decimal values may collapse into the same representable floating-point value once the bit budget gets too small. Rounding error is not always catastrophic. Deep learning is often surprisingly tolerant of noise. Many models still train or infer correctly when activations and weights are stored in formats that are obviously approximate. But "tolerant" is not the same as "indifferent." Some operations are sensitive to small differences. Small errors can accumulate, especially across many layers or many training steps. Sensitive reductions, normalization statistics, or outlier values can suffer if the format is too coarse. The second compromise is range . A format with too few exponent bits cannot represent very large or very small values well. Values that exceed the top of the range overflow. Values far below the bottom may underflow. Sometimes they become subnormals, which preserve a little information near zero. Sometimes they simply become zero. For AI, that matters because tensors can contain both ordinary values and outliers. If the format cannot absorb those outliers safely, training or inference quality can degrade. The third compromise is coarser spacing between values . Floating-point numbers are not evenly spaced. As magnitude grows, the gap between neighboring representable values grows too. With a tiny mantissa, those gaps become large quickly. That means the same absolute rounding step that looks small at one scale may become very noticeable at another. Beginners sometimes think that if a format can represent 1 and 2, then it should also represent plenty of values smoothly between them. That is only true if the mantissa is rich enough. The fourth compromise is that software and hardware must do more bookkeeping . Once low precision becomes aggressive, you often need scaling, calibration, loss scaling, per-channel quantization, outlier handling, or selective fallback to wider types. This does not mean low precision is bad. It means low precision becomes a systems problem, not just a format choice. The best AI stacks are built around these compromises rather than trying to ignore them. Subnormals, Zeros, Infinities, and NaNs in Plain English Floating-point discussions often become intimidating because they introduce special values early. The good news is that the concepts are simple if you treat them as practical edge cases rather than as abstract theory. Zero is the obvious one. A format needs a way to represent zero exactly. Many formats also allow both +0 and -0, which can matter in certain numerical operations even if everyday application code rarely notices. Subnormal numbers are values very close to zero that cannot be represented as normal floating-point numbers because the exponent would need to go lower than the format allows. Instead of dropping straight to zero, the format can represent a few smaller values with reduced precision. This softens underflow near zero. Wider formats have more room to do this well. Very tiny formats have much less room. Infinity appears when a computation overflows in the positive or negative direction. Rather than silently wrapping around, the format records an overflow state. This is useful because it lets later code detect that something exceeded the representable range. Some FP8 variants support infinities explicitly, while others use the extreme exponent patterns differently. NaN means "not a number." It appears when an operation does not produce a meaningful real numeric result, such as 0 divided by 0. NaN is also useful as a marker that something in the computation went wrong. The converter supports special handling for NaN and infinities where the target format allows them, which is why unusual inputs do not simply crash the display. For a beginner, the main takeaway is not memorizing every special bit pattern. The takeaway is that floating-point formats are not just a ladder of ordinary numbers. They also reserve some patterns for important edge-case meanings. As formats get smaller, the amount of room available for those edge cases shrinks, which is one reason tiny formats can feel idiosyncratic. Why BF16 Became So Important in Training BF16 deserves special attention because newcomers often assume FP16 and BF16 are almost the same. They are both 16-bit formats, but they make a different bargain. FP16 spends more of its bits on mantissa precision than BF16 does, but it has a much smaller exponent range. BF16 keeps the 8-bit exponent width of FP32, which means it can represent a much broader range of magnitudes even though its significand is short. For training, that large range matters a lot. Deep learning tensors can vary widely in magnitude. Activations, gradients, and optimizer values do not always stay in a neat narrow band. BF16 tends to be more forgiving because it is less likely to overflow or underflow when the dynamic range gets messy. That is why many modern training stacks prefer BF16 as the low-precision default when the hardware supports it. The tradeoff is that BF16 is not more precise than FP16. In fact, it usually has less fine detail in the mantissa. But in many AI workloads, keeping the range stable is more valuable than preserving a few extra bits of local precision. This is a perfect example of the broader theme of floating-point design: "better" depends on the workload. AI does not ask for one universal number format. It asks for the best tradeoff per operation. Why FP8 Is Exciting but Also Demanding FP8 is exciting because it promises another large step down in memory and another large step up in hardware throughput. Going from 16 bits to 8 bits halves storage again. On large models, that is a major systems win. It can affect the number of GPUs needed, the maximum batch size, the achievable throughput, and the economics of running the service. But FP8 is demanding because 8 bits are still extremely few. You cannot expect an 8-bit float to behave like a smaller FP16. The gaps between representable values are larger. Outliers become more dangerous. Tiny calibration mistakes matter more. That is why real FP8 training and inference flows often use scaling factors, a wider accumulation type, careful tensor-specific policies, and fallbacks for sensitive operations. Another reason FP8 needs care is that there is no single perfect FP8 layout for all tensors. E4M3 and E5M2 exist because the range-versus-precision tradeoff changes depending on the tensor's role. Some values live in a controlled range and benefit more from one extra mantissa bit. Other values can swing more wildly and need extra exponent headroom instead. Once again, mixed precision wins because it acknowledges that different tensors are different numerical environments. Step-by-Step FP8 E4M3 Example Let us do one complete worked example, because this is where the whole topic usually clicks. We will convert decimal 13.625 into FP8 E4M3, the format on this page with 4 exponent bits and 3 mantissa bits. We choose this number because it is easy to express in binary, but it still shows a real precision loss when reduced to only 3 mantissa bits. Step 1: Determine the sign. The number 13.625 is positive, so the sign bit is 0. Step 2: Convert the magnitude to binary. The whole number 13 is binary 1101 . The fractional part 0.625 is binary .101 because 0.5 + 0.125 = 0.625. Put together, 13.625 becomes 1101.101 in binary. Step 3: Normalize the binary number. Move the binary point so there is exactly one non-zero digit to the left of it: 1101.101 = 1.101101 x 2^3 . That tells us the real exponent is 3, and the significant binary digits after the leading 1 are 101101 . Step 4: Apply the exponent bias. E4M3 uses 4 exponent bits, and this implementation uses bias 7. So a real exponent of 3 is stored as 3 + 7 = 10 . Binary 10 is 1010 , which fits into the 4 exponent bits. Step 5: Round the mantissa to 3 bits. Our normalized significand is 1.101101 . E4M3 can only keep three bits after the binary point, so we would like something of the form 1.xyz . The next bits tell us how to round. Keeping only the first three fractional bits gives 1.101 , which is 1.625 in decimal. But the original value was 1.703125 after normalization. The converter rounds this to the nearest E4M3 step, which becomes mantissa bits 110 , or 1.75. Step 6: Reconstruct the represented value. Sign 0, stored exponent 10, and mantissa bits 110 decode to +1.75 x 2^3 . That equals 14. So although we started with 13.625, the nearest representable E4M3-style value in this path is 14. The rounding error is 0.375. That may sound large, but it is exactly the lesson FP8 is trying to teach. Eight bits are a tiny budget. E4M3 can preserve the rough scale of the number while sacrificing detail. For many AI tensors, that is acceptable, especially when scaling and accumulation happen in smarter ways around the raw storage format. For tasks that need finer detail on individual values, FP8 may be too aggressive. Now compare that to E5M2 mentally. E5M2 would have one extra exponent bit but one fewer mantissa bit. That means it can cover a wider range of magnitudes, but nearby representable values are even coarser because only 2 mantissa bits remain. In practice, E4M3 is often the better choice when your tensor values stay in a manageable range and you want slightly more precision. E5M2 is safer when range is the bigger concern. Examples and Practical Guidance Why FP4 Is Not Very Useful As-Is FP4 sounds exciting on paper because 4 bits are tiny. A 4-bit value uses half the storage of FP8 and one quarter the storage of FP16. If that worked cleanly for every tensor element, the efficiency gains would be enormous. The problem is that four bits are simply not much information. After you spend bits on sign and exponent, almost nothing is left for precision. The result is a value system with a very small set of representable numbers. The raw FP4 E2M1-style output on this page makes that limitation visible. Many different decimals collapse to the same quantized result because the format can only distinguish a handful of levels. If you type several nearby values into the converter, you will often see them come back as exactly the same FP4 number. That is not a bug. That is what happens when the representable set is tiny. This is why standalone FP4 is usually not useful by itself for real AI tensors. A single tensor may contain values clustered around 0.02, around 1.5, or around 120 depending on the layer, channel, or operation. A raw FP4 number cannot adapt to all of those scales at once. If the format is centered well for one region, it will be too coarse or too narrow for another. The quantization error becomes too large to ignore. Practical 4-bit systems solve this by adding shared scaling . Instead of saying, "This 4-bit code directly means the full final number," they say something closer to, "This 4-bit code is a small value inside a block, and the block also has a scale factor that tells us how to stretch these codes into the right numeric range." The scale may be per block, per group, per channel, or per tensor. That extra context is what makes 4-bit schemes usable. The note in this calculator about MXFP4 and NVFP4 is pointing to exactly that idea. Those formats are not just "take one number and convert it to a standalone 4-bit float." They are part of a larger representation strategy. The raw element value only makes sense with the shared block scale that comes with it. Without the scale, you do not really know how the 4-bit code should map back into the original numeric world. So when people say FP4 is useful for AI, the safe interpretation is usually: carefully engineered 4-bit workflows can be useful when combined with scaling, calibration, and hardware/software support. That is very different from saying that a naked 4-bit float is a drop-in replacement for FP16 or BF16. It is not. What the Converter Can Teach You in Practice The easiest way to use this page is to treat it like a laboratory. Try the same number across all formats and ask what changed. If a value stays almost identical in FP32 and BF16 but moves sharply in FP8, that tells you the number needs more local precision than 8-bit storage is giving it. If a very small number disappears in one format but survives in another, that tells you the smaller format is running out of range near zero. If a large value hits a ceiling, that tells you the exponent budget is too small. This kind of intuition is useful well beyond this single page. It helps when reading benchmark claims, GPU marketing slides, training guides, and quantization papers. A lot of confusion in AI hardware discussions comes from people treating numeric formats as if they are simply ranked from good to bad. They are not. They are tools. FP32 is not obsolete because FP8 exists. FP8 is not useless because BF16 is safer. The real question is always what tensor, what operation, what scale, and what error tolerance. Beginners also benefit from seeing that the output is often still a normal-looking decimal. The danger of low precision is not that the number becomes obviously corrupted with weird symbols. The danger is subtler. It still looks like a reasonable number, but it may be a noticeably different number than the one you started with. That is exactly the kind of silent approximation AI systems must manage. Practical Rules of Thumb for AI Workloads If you want a simple set of rules, here is a good beginner starting point. Use FP32 when you want a stable default or a reference. Use BF16 when you want a training-friendly low-precision format with strong dynamic range. Use FP16 when the workload and hardware are known to be comfortable with it, especially for inference and some mixed-precision training paths. Use FP8 when your hardware and software stack support it well and you are prepared to treat it as a managed precision strategy rather than a universal on/off switch. Be cautious with very low precision when the model contains outliers, sensitive normalization behavior, or operations where small numeric differences matter a lot. Be especially cautious if you are reducing both storage precision and accumulation precision at the same time. Many successful AI systems use low precision for storage but keep wider accumulation types to preserve stability. For FP4-style workflows, the best beginner rule is simple: never assume that a raw 4-bit element tells the whole story. Ask what the scaling strategy is. Ask whether the scale is per tensor, per block, or per channel. Ask what the accumulation type is. Ask where wider precision is still retained. Those questions matter more than the four bits alone. Try These Numbers for Aha Moments If you want the fastest way to build intuition, type the examples below into the converter and compare the outputs side by side. The goal is not to memorize every number. The goal is to notice the pattern behind the surprises: some values drift a little, some snap to cleaner neighbors, some disappear, and some reveal that tiny formats handle special cases very differently from wider ones. Pay special attention to FP32 versus TF32/BF16/FP16, then compare those with the two FP8 variants, and finally look at FP4. That order usually gives the cleanest aha moment because you can watch the number lose detail step by step instead of jumping straight from a wide format to the most aggressive one. 0.1 This is the classic reminder that the approximation problem starts before low precision even enters the picture. Standout outputs: FP64 is already 0.10000000000000000555111512312578 , FP8 E4M3 is 0.1015625 , FP8 E5M2 is 0.09375 , and FP4 becomes 0 . The aha moment here is that 0.1 looks simple in decimal but is not exact in binary. Wider formats hide the error well enough that it feels harmless. Smaller formats make the approximation much more obvious. 1.1 This is a good everyday number for seeing how nearby values collapse when mantissa bits disappear. Standout outputs: FP8 E4M3 becomes 1.125 , while FP8 E5M2 and FP4 both land on 1 . The lesson is that E4M3 and E5M2 are not just "two FP8s." E4M3 keeps a little more local detail, while E5M2 gives up that detail sooner in exchange for more exponent headroom. 13.625 This is the live version of the worked FP8 example from the tutorial above. Standout outputs: FP32, TF32, FP16, and BF16 keep 13.625 exactly here, but both FP8 formats round to 14 , and FP4 collapses all the way to 6 . The aha moment is seeing a number that behaves perfectly in several formats and then suddenly jumps once the budget gets small enough. It connects the abstract FP8 explanation to a concrete value you can reproduce instantly. 0.00009 This is the best "small number disappears" example on the page. Standout outputs: FP8 E4M3 becomes 0 , FP8 E5M2 still preserves a nearby nonzero value, 0.000091552734375 , and FP4 is also 0 . This is range, not just precision. E5M2 survives because it spends more bits on the exponent. E4M3 runs out of room near zero sooner, which is exactly why FP8 variants exist in the first place. 449 This is more educational than a giant number because it sits just past a visible cliff. Standout outputs: BF16 rounds to 448 , and both FP8 E4M3 and E5M2 also land on 448 . The aha moment is that formats often fail at boundaries, not only at absurd magnitudes. Typing 448 and then 449 back to back makes the cliff edge obvious. -0 This is the small weird one that makes many people stop and look twice. Standout outputs: FP64 displays ordinary 0 , but FP32, TF32, FP16, BF16, both FP8 formats, and FP4 all preserve -0 . The lesson is that floating-point can keep sign information even when the magnitude is zero. Most of the time you do not care, but it is part of the real behavior of the format. Infinity This shows that tiny formats do not all reserve their special bit patterns in the same way. Standout outputs: FP8 E4M3 shows NaN , FP8 E5M2 shows Infinity , and FP4 collapses to 6 . The aha moment is not "Infinity is big." It is that special values are part of the format design. One FP8 variant supports infinity directly here, while the other does not expose it the same way in this converter. NaN This is the cleanest way to see that some bit patterns are reserved for meaning, not magnitude. Standout outputs: every displayed format shows NaN . The lesson is that floating-point formats are not just lists of ordinary numbers. They also have dedicated patterns for results that are not meaningful real values, such as undefined arithmetic. Taken together, these examples teach the five recurring lessons of the whole page: decimal numbers are often approximate in binary, fewer mantissa bits create more rounding, fewer exponent bits cause earlier underflow and overflow, FP8 variants trade precision for range differently, and raw FP4 collapses aggressively unless a larger scaling scheme supports it. FAQ: GPU floating-point formats for AI What does this number to GPU float converter show? It shows the approximate numeric value you get after storing a decimal number in FP64, FP32, TF32, FP16, BF16, FP8 E4M3, FP8 E5M2, or FP4 E2M1 and then decoding that stored representation back into a readable number. Why do the outputs change when I lower precision? Lower precision means fewer exponent or mantissa bits. That forces the format to round more aggressively, which changes the stored value. The smaller the format, the fewer distinct numbers it can represent. Why does BF16 often appear in AI training guides? BF16 keeps the wide exponent range of FP32 while using only 16 bits total. That makes it more tolerant of large dynamic range than FP16, which is why it is commonly used for modern training on supported hardware. Which FP8 format is better, E4M3 or E5M2? Neither is universally better. E4M3 gives a bit more precision. E5M2 gives a bit more range. The better choice depends on whether the tensor needs more mantissa detail or more exponent headroom. Why is raw FP4 not enough for most real AI workloads? A raw 4-bit float has too few representable values. Practical 4-bit systems usually add shared scales so a small code can be interpreted relative to the local numeric range of a block or group of values. Is lower precision always faster? Not automatically. The hardware, kernels, framework support, and scaling strategy matter. In well-supported GPU paths, lower precision often improves throughput and memory efficiency, but it still has to preserve acceptable model quality. Last updated: July 16, 2026 References These references cover the floating-point format descriptions, range-versus-precision tradeoffs, TensorFloat-32 behavior, FP8 variants, and the block-scaled FP4 notes used to write the tutorial on this page. IEEE Standard for Floating-Point Arithmetic IEEE 754-2019 . IEEE, July 2019. Defines the baseline binary floating-point formats and semantics behind FP64, FP32, and FP16. https://ieeexplore.ieee.org/document/8766229 https://ieeexplore.ieee.org/document/8766229 A Study of BFLOAT16 for Deep Learning Training. Shibo Wang and Pankaj Kanwar, 2019. Useful background for why BF16 keeps FP32-like range while reducing precision. https://arxiv.org/abs/1905.12322 https://arxiv.org/abs/1905.12322 NVIDIA A100 Tensor Core GPU Architecture. NVIDIA, 2020. Primary vendor reference for TF32 positioning on Ampere tensor cores. NVIDIA Ampere whitepaper https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf FP8 Formats for Deep Learning. Paulius Micikevicius et al., 2022. Defines the E4M3 and E5M2 FP8 variants referenced throughout the page. https://arxiv.org/abs/2209.05433 https://arxiv.org/abs/2209.05433 Microscaling Data Formats for Deep Learning. Bita Darvish Rouhani et al., 2023. Defines MX formats, including MXFP4 with E2M1 elements plus shared block scale. https://arxiv.org/abs/2310.10537 https://arxiv.org/abs/2310.10537 Pretraining Large Language Models with NVFP4. NVIDIA et al., 2025. Describes NVFP4 as a block-scaled FP4 scheme rather than a standalone scalar float. https://arxiv.org/abs/2509.25149 https://arxiv.org/abs/2509.25149 FP4 All the Way: Fully Quantized Training of LLMs. Brian Chmiel et al., 2025. Supporting reference for practical FP4 training workflows and FP4 design tradeoffs. https://arxiv.org/abs/2505.19115 https://arxiv.org/abs/2505.19115 Practical FP4 Training for Large-Scale MoE Models on Hopper GPUs. Wuyue Zhang et al., 2026. Discusses software-emulated MXFP4 and its block-scaled structure. https://arxiv.org/abs/2603.02731 https://arxiv.org/abs/2603.02731 Diagnosing FP4 inference: a layer-wise and block-wise sensitivity analysis of NVFP4 and MXFP4. Musa Cim et al., 2026. Comparative analysis of NVFP4 and MXFP4 in inference workloads. https://arxiv.org/abs/2603.08747 https://arxiv.org/abs/2603.08747 Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction. Jatin Chhugani et al., 2026. Focuses on improving MXFP4 quantization fidelity. https://arxiv.org/abs/2603.08713 https://arxiv.org/abs/2603.08713 Adaptive Block-Scaled Data Types. Jack Cook et al., 2026. Uses NVFP4 as a block-scaled FP4 baseline and explores adaptive alternatives. https://arxiv.org/abs/2603.28765 https://arxiv.org/abs/2603.28765 MixFP4: Enhancing NVFP4 with Adaptive FP4/INT4 Block Representations. Jiaxiang Zou et al., 2026. Extends NVFP4 with mixed block representations. https://arxiv.org/abs/2605.31035 https://arxiv.org/abs/2605.31035