Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The road to Zettalinux

lwn.net

Post reply on HN
Print view View original

41–50 of 199 posts

Re: The road to Zettalinux

#41

Post by gwbas1c »

> Matthew Wilcox took the stage to make the point that 64 bits may turn out to be too few — and sooner than we think

Let's think critically for a moment. I grew up in the 1980s and 1990s, when we all craved more and more powerful computers. I even remember the years when each generation of video games was marketed as 8-bit, 16-bit, 32-bit, ect.

BUT: We're hitting a point where, for what we use computers for, they're powerful enough. I don't think I'll ever need to carry a 128-bit phone in my pocket, nor do I think I'll need a 128-bit web browser, nor do I think I'll need a 128-bit web server. (See other posts about how 64-bits can address massive amounts of memory.)

Will we need 128-bit computing? I'm sure someone will find a need. But let's not assume they'll need an operating system designed in the 1990s for use cases that we can't imagine today.

Re: The road to Zettalinux

#42

Post by apaprocki »

mhh__ wrote: post #23
jmillikin wrote: post #18

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

Systems for which pointers are not just integers have come and gone, sadly. Many mainframes had function pointers which were more like a struct than a pointer.

Itanium function pointers worked like that, so it could have been the new normal if IA64 wasn’t so crazy on the whole.

Re: The road to Zettalinux

#43

Post by rwmj »

dragontamer wrote: post #34
jmillikin wrote: post #18

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

> it's difficult to imagine what kind of new physics would let someone fit that many bytes into a machine that's practical to control with a single Linux kernel instance. I nominally agree with most of your post. But I should note that modern systems seem to be moving towards a "one pointer space" for the entire cluster. For example, 8 GPUs + 2 CPUs would share the same virtual memory space (GPU#1 may take one slice,…

Distributed Shared Memory is a thing, but I'm not sure how widely it is used. I found that it gives you all the coordination problems of threads in symmetric multiprocessing but at a larger scale and with much slower synchronisation.

https://en.wikipedia.org/wiki/Distributed_shared_memory

Re: The road to Zettalinux

#44

Post by cmrdporcupine »

dragontamer wrote: post #34
jmillikin wrote: post #18

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

> it's difficult to imagine what kind of new physics would let someone fit that many bytes into a machine that's practical to control with a single Linux kernel instance. I nominally agree with most of your post. But I should note that modern systems seem to be moving towards a "one pointer space" for the entire cluster. For example, 8 GPUs + 2 CPUs would share the same virtual memory space (GPU#1 may take one slice,…

It seems to me that such a memory space could be physically mapped quite large while still presenting 64-bit virtual memory addresses to the local node? How likely is it that any given node would be mapping out more than 2^64 bytes worth of virtual pages?

The VM system could quite simply track the physical addresses as a pair of `u64_t`s or whatever, and present those pages as 64-bit pointers.

It seems in particular you might want to have this anyways, because the actual costs for dealing with such external memories would have to be much higher than local memory. Optimizing access would likely involve complicated cache hierarchies.

I mean, it'd be exciting if we had need for memory space larger than 2^64 but I just find it implausible with current physics and programs? But I'm also getting old.

Re: The road to Zettalinux

#45

Post by masklinn »

mhh__ wrote: post #33
masklinn wrote: post #29

Earlier quoted context omitted.

Technically that’s still pretty common on the software side, that’s what tagged pointers are. The ObjC/Swift runtime uses that for instance, the class pointer of an object also contains the refcount and a few flags.

Pointer tagging is still spiritually an integer (i.e. mask off the bits you guarantee the tags are in via alignment) versus (say) the pointer being multiple addresses into different bits of different types of memory and tags insert other stuff here.

> Pointer tagging is still spiritually an integer

No, pointer tagging is spiritually a packed structure. That can be a simple union (as it is in ocaml IIRC) but it can be a lot more e.g. the objc “non-pointer isa” ended up with 5 flags (excluding raw isa discriminant) and two additional non-pointer members of 19 and 9 bits.

You mask things on and off to unpack the structure into component the system will accept. Nothing precludes using tagged pointers to discriminate between kinds and locations.

Re: The road to Zettalinux

#46

Post by Deukhoofd »

LinkLink wrote: post #6

For reference 2^64 = ~10^19.266 I don't think this is unreasonable at all, its unlikely that computers will largely stay the same in the coming years. I believe we'll see many changes to how things like mass addressing of data and computing resources is done. Right now our limitations in these regards are addressed by distributed computing and databases, but in a hyper-connected world there may come a time when such…

> all computers seamlessly worked together sharing computer power as needed. Even 256 bits wouldn't be out of the question then.

This sounds like it would be massively out of scope for Linux. It'd require a complete overhaul of most of its core functionality, and all of its syscalls. While not a completely infeasible idea, it sounds to me like it'd require a completely new designed kernel.

Re: The road to Zettalinux

#47

Post by lostmsu »

jmillikin wrote: post #18

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

So 64 bit address is only 1024 16TB HDDs? That number may go down quickly. There is a 100TB SSD already.

Re: The road to Zettalinux

#48

Post by EvanAnderson »

protomyth wrote: post #38
PaulHoule wrote: post #2

On one hand The IBM System/38 used 128 bit pointers in the 1970s, despite having a 48 bit physical address bus. These were used to manage persistent objects on disk or network with unique ids a lot like uuids. On the other hand, filling out a 64 bit address space looks tough. I struggled to find something of the same magnitude of 2^64 and I got ‘number of iron atoms in an iron filing’, From a nanotechnological point…

The AS/400 and iSeries also use 128-bit pointers. 128-bit would be useful for multiple pointers already in common use such as ZFS and IP6 addresses. I expect it will the last hop for a long time.

In the context of the AS/400's single-level store architecture the 128-bit pointers make a lot of sense, too.

Re: The road to Zettalinux

#49

Post by creativemonkeys »

Blikkentrekker wrote: post #3

> How would this look in the kernel? Wilcox had originally thought that, on a 128-bit system, an int should be 32 bits, long would be 64 bits, and both long long and pointer types would be 128 bits. But that runs afoul of deeply rooted assumptions in the kernel that long has the same size as the CPU's registers, and that long can also hold a pointer value. The conclusion is that long must be a 128-bit type. Can anyon…

I'm sure someone will come along and explain why I have no idea what I'm talking about, but so far my understanding is those names exist because of the difference in CPU word size. Typically "int" represents the natural word size for that CPU, which matches the register size as well, so 'int plus int' is as fast as addition can run by default, on a variety of CPUs. That's one reason chars and shorts are promoted to ints automatically in C.

Let's say you want to work with numbers and you want your program to run as fast as possible. If you specify the number of bits you want, like i32, then the compiler must make sure on 64bit CPUs, where the register holding this value has an extra 32bits available, that the extra bits are not garbage and cannot influence a subsequent operation (like signed right shift), so the compiler might be forced to insert an instruction to clear the upper 32bits, and you end up with 2 instructions for a single operation, meaning that your code now runs slower on that machine.

However, had you used 'int' in your code, the compiler would have chosen to represent those values with a 64bit data type on 64bit machines, and 32bit data type on 32bit machines, and your code would run optimally, regardless of the CPU. This of course means it's up to you to make sure that whatever values your program handles fit in 32bit data types, and sometimes that's difficult to guarantee.

If you decide to have your cake and eat it too by saying "fine, I'll just select i32 or i64 at compile time with a condition" and you add some alias, like "word" -> either i32 or i64, "half word" -> either i16 or i32, etc depending on the target CPU, then congrats, you've just reinvented 'int', 'short', 'long', et.al.

Personally, I'm finding it useful to use fixed integer sizes (e.g. int32_t) when writing and reading binary files, to be able to know how many bytes of data to read when loading the file, but once those values are read, I cast them to (int) so that the rest of the program can use the values optimally regardless of the CPU the program is running on.

Re: The road to Zettalinux

#50

Post by rektide »

cmrdporcupine wrote: post #44
dragontamer wrote: post #34

Earlier quoted context omitted.

> it's difficult to imagine what kind of new physics would let someone fit that many bytes into a machine that's practical to control with a single Linux kernel instance. I nominally agree with most of your post. But I should note that modern systems seem to be moving towards a "one pointer space" for the entire cluster. For example, 8 GPUs + 2 CPUs would share the same virtual memory space (GPU#1 may take one slice,…

It seems to me that such a memory space could be physically mapped quite large while still presenting 64-bit virtual memory addresses to the local node? How likely is it that any given node would be mapping out more than 2^64 bytes worth of virtual pages? The VM system could quite simply track the physical addresses as a pair of `u64_t`s or whatever, and present those pages as 64-bit pointers. It seems in particular…

Leaving cluster coherent address space behind - like you say - is doable. But you lose what the parent was saying:

> If everyone has the same address space, then you can share pointers / graphs between nodes and the underlying routing/ethernet software will be passing the data automatically between all systems. Its actually quite convenient.

Post reply on HN