121–130 of 199 posts
I think the article is very shortsighted. By 2035-40 we'll probably have memory only (RAM) computers massively available. No disks means no current OS capable of handling these computers. A change of paradigm needing new platforms and OSes. These future OSes may be 128bit, but I don't think the current ones will make it to the transition.
or better yet, design a new abstraction for not having to hard-code the limit of the pointer size but instead allow it to be extensible as more addressable space becomes a reality, instead of having to transition over and over. is this even possible? if it is, shouldn't we head in that direction?
Earlier quoted context omitted.
There's also something beautiful about seeing or creating a Kogge-Stone implementation on silicon. I know it was one of the first time I thought to myself: this is not just a straightforward pipeline, yet it all follows such a beautifully geometrical interconnect pattern. Super fast, yet very elegant to layout.
The original paper is a masterpiece to read as well, if you haven't read it. "A Parallel Algorithm for the Efficient Solution of a General Class of Recurrence Equation", by Kogge and Stone. It proves the result for _all_ associative operations (technically, a class slightly larger than associative. Kogge and Stone called this a "semi-associative" operation).
A bit sad that 1974 papers are still behind a IEEE paywall...
Edit: Just finished reading it. I have to say that the generalization of 3.2 got a bit over me, but otherwise it's pretty amazing that they could define such a generalization. Intuition for those type of problem is often to proceed one step at a time, N times.
That it is provably doable in log2(N) is great, especially since it allows for a choice of the depth/number of processors you want to use for the problem. Hopefully next time I design a latency-constrained system I remember to look at that article
Earlier quoted context omitted.
That's what I was going to chime in with - you pay for that extra address width. Binary addition and multiplication latency is super-linear with regards to operand width. Larger pointers lead to more memory use, and memory access latency is non-constant with respect to size. It might make sense for large distributed systems to move to a 128-bit architecture, but I don't see any reason for consumer devices, at least w…
> Binary addition ... is super-linear with regards to operand width No its not. That's why Kogge-Stone's carry lookahead adder was such an amazing result. O(log(n)) latency with respect to operand width with O(n) total half-adders used. It may seem like its super-linear. But the power of prefix-sums leads to a spectacular and elegant solution. Kogge-stone (and the concept of prefix-sums) is one of the most important…
The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…
An exabyte was an absolutely incomprehensible amount of memory, once. Nearly as incomprehensible as 4 gigabytes seemed, at one time. But as you note, 64 bits of addressable data can fit into a single warehouse now.
Going by the historical rate of increase, $100 would buy about a petabyte of storage in 2040. Even presuming a major slowdown, we still start running into 64 bit addressing as a practical limit, perhaps sooner than you think.
Earlier quoted context omitted.
> it's difficult to imagine what kind of new physics would let someone fit that many bytes into a machine that's practical to control with a single Linux kernel instance. I nominally agree with most of your post. But I should note that modern systems seem to be moving towards a "one pointer space" for the entire cluster. For example, 8 GPUs + 2 CPUs would share the same virtual memory space (GPU#1 may take one slice,…
It seems to me that such a memory space could be physically mapped quite large while still presenting 64-bit virtual memory addresses to the local node? How likely is it that any given node would be mapping out more than 2^64 bytes worth of virtual pages? The VM system could quite simply track the physical addresses as a pair of `u64_t`s or whatever, and present those pages as 64-bit pointers. It seems in particular…
In the Grace Hopper whitepaper, NVIDIA says that they connect multiple nodes with a fabric that allows them to creat a virtual address space across all of them.
Quoted post unavailable.
On one hand The IBM System/38 used 128 bit pointers in the 1970s, despite having a 48 bit physical address bus. These were used to manage persistent objects on disk or network with unique ids a lot like uuids. On the other hand, filling out a 64 bit address space looks tough. I struggled to find something of the same magnitude of 2^64 and I got ‘number of iron atoms in an iron filing’, From a nanotechnological point…
> filling out a 64 bit address space looks tough. I struggled to find something of the same magnitude of 2^64 and I got ‘number of iron atoms in an iron filing’ Reminded me of Jeff Bonwick's answer to the following question about his 'boiling the oceans' quip related to ZFS being a "128 bit filesystem": > 64 bits would have been plenty ... but then you can't talk out of your ass about boiling oceans then, can you? Sa…
(1/2) kT
I multiplied that by 2¹²⁸, and got 1.41×10¹⁸ J of energy. 1 ton of TNT is 4.2×10¹² J, so that is a 335 kiloton explosion worth of energy just to boot.
That's not impossible, that much heat is extracted from a nuclear reactor in a few months. If you want to go faster you need a bigger system, but a bigger system will be slower because of light speed latency.
(You do better, however, at a lower temperature, say 1° K but heat extraction gets more difficult at lower temperatures and you spend energy on refrigeration unless you wait long enough for the Universe to grow colder.)
Quoted post unavailable.
Earlier quoted context omitted.
Good luck using those to specify the data layout of a network packet.
Well, for that you'd probably use a specialisation of Integer that's bounded and can thus be represented in a machine word.