> The notation ExMm denotes a format with x exponent bits and y mantissa bits. Shouldn't that be m mantissa bits (not y) -- i.e. typo here -- or am I misunderstanding something?
You're correct yeah, 'ExMy'.
4-bit floating point FP4
31–40 of 81 posts
Re: 4-bit floating point FP4
#32This doesn't look like good a floating point format. NaNs and INFs are missing.
Re: 4-bit floating point FP4
#33When you have so few bits, does it really make sense to invent a meaning for the bit positions? Just use an index into a "palette" of pre-determined numbers.
As a bonus, any operation can be replaced with a lookup into a nxn table.
Re: 4-bit floating point FP4
#34When you have so few bits, does it really make sense to invent a meaning for the bit positions? Just use an index into a "palette" of pre-determined numbers. As a bonus, any operation can be replaced with a lookup into a nxn table.
Exactly. And pick them on the e^x curve.
Re: 4-bit floating point FP4
#35Earlier quoted context omitted.
> Even the latest CPUs have a 2:1 fp64:fp32 performance ratio Not completely - for basic operations (and ignoring byte size for things like cache hit ratios and memory bandwidth) if you look at (say Agner Fog's optimisation PDFs of instruction latency) the basic SSE/AVX latency for basic add/sub/mult/div (yes, even divides these days), the latency between float and double is almost always the same on the most recent…
> ... if you look at (say Agner Fog's optimisation PDFs of instruction latency) ... That.... doesn't seem true? At least for most architectures I looked at? While true the latency for ADDPS and ADDPD are the same latency, using the zen4 example at least, the double variant only calculates 4 fp64 values compared to the single-precision's 8 fp32. Which was my point? If each double precision instruction processes a smal…
Well, maybe not all admittedly, and I didn't look at AVX2/512, but it looks like `_mm_div_ps` and `_mm_div_pd` are identical for divide, at the 4-wide level for the basics.
Obviously, the wider you go, the more constrained you are on infrastructure and how many ports there are.
My point was more it's very often the expensive transcendentals where the performance difference is felt between f32 and f64.
Re: 4-bit floating point FP4
#36FP2 spec:
00 -> 0.0
01 -> 1.0
10 -> Inf
11 -> NaN
00 -> 0.0
01 -> 1.0
10 -> Inf
11 -> -InfRe: 4-bit floating point FP4
#37FP2 spec: 00 -> 0.0 01 -> 1.0 10 -> Inf 11 -> NaN or 00 -> 0.0 01 -> 1.0 10 -> Inf 11 -> -Inf
00 -> 0.0
01 ->-0.0
10 -> Inf
11 -> -InfRe: 4-bit floating point FP4
#38When you have so few bits, does it really make sense to invent a meaning for the bit positions? Just use an index into a "palette" of pre-determined numbers. As a bonus, any operation can be replaced with a lookup into a nxn table.
You want to make multiplication cheap, it's not just about compression
Re: 4-bit floating point FP4
#399 years ago, I shared this as an April Fools joke here on HN. It seems that life is imitating art. https://github.com/sdd/ieee754-rrp
Lowest I've used is 8 bit floats for time delays, in embedded devices.
Interesting! I have been using integers or f32 for that. What was the use case specifically? Did you write a software float for it? I remember writing a `f16` type for an IC that used that was a pain!
Re: 4-bit floating point FP4
#40When you have so few bits, does it really make sense to invent a meaning for the bit positions? Just use an index into a "palette" of pre-determined numbers. As a bonus, any operation can be replaced with a lookup into a nxn table.
You want to make multiplication cheap, it's not just about compression
Wouldn’t multiplication just be an 8 bit lookup table? a*b is just lut[a<<4+b]