Live data from Hacker News

The road to Zettalinux

lwn.net

51–60 of 199 posts

Re: The road to Zettalinux

#51

Earlier quoted context omitted.

> in modern languages the rule is generally to have fixed-size integers. Modern languages have unlimited size integers :-) "Modern" as in "since at least the 80s, more likely 70s".

Good luck using those to specify the data layout of a network packet.

You are supposed to stream your video data as base64 encoded xml embedded in a json array.

Re: The road to Zettalinux

#52
post #47

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

So 64 bit address is only 1024 16TB HDDs? That number may go down quickly. There is a 100TB SSD already.

1024*1024 16TB HDDs, or 1,048,576.

Re: The road to Zettalinux

#53
post #47

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

So 64 bit address is only 1024 16TB HDDs? That number may go down quickly. There is a 100TB SSD already.

1,000,000 drives at 16 TB each I think.

Kilo is 10 bits. Mega 20. Gigs 30. Tera 40. 16TB is 44 bits. 1000* is another 10 bits so 54.

Re: The road to Zettalinux

#54

Is 128 bit the limit of what we would need? We use 128 bit UUIDs. 2^256 seems to be more than the number of atoms on Earth.

The article does talk about just making the pointer type used in syscalls 256 bit wide to "give room for any surprising future needs".

The size of large networked disk arrays will grow beyond 64 bit addresses, but I don't think we will exceed 2^128 bits of storage of any size, for any practical application. Then again, there's probably people who thought the same about 32 bit addresses when we moved from 16bit to 32bit addresses.

The most likely case for "giant" pointers (more than 128 bits) will be adding more metadata into the pointer. With time we might find enough use cases that are worth it to go to 256bit pointers, with 96bit address and 160 bit metadata or something like that.

Re: The road to Zettalinux

#55
post #47

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

So 64 bit address is only 1024 16TB HDDs? That number may go down quickly. There is a 100TB SSD already.

Not quite - 1024GiB is 1TiB, so it's 1024 x 1024 x 16TiB drives

Re: The road to Zettalinux

#56
post #40

What's the average lifespan of a line of kernel code? I imagine by starting this project 12 years before its anticipated use case they can get very far just by requiring that any new code is 128-bit compatible (in addition to doing the broader infrastructure changes needed like fixing the syscall ABI)

> What's the average lifespan of a line of kernel code? There's a fun tool called "Git of Theseus" which can answer this question! You can see some graphs of Linux code on the web page: https://github.com/erikbern/git-of-theseus Named after the Ship of Theseus: https://en.wikipedia.org/wiki/Ship_of_Theseus

There're some more in the presentation article: https://erikbern.com/2016/12/05/the-half-life-of-code.html#:...

A (Linux) kernel line has half-life 6.6 years. The highest of the projects analyzed. The lowest went to Angular with half-life 0.32 years.

Re: The road to Zettalinux

#57
post #7

Earlier quoted context omitted.

Legacy, there’s lots of dumb stuff in C. As you note, in modern languages the rule is generally to have fixed-size integers. Though I think there are portability issues concerns, that world is mostly gone (it remains in some corners of computing e.g. dsps) but if you’re only using fixed-size integers what do you do when a platform doesn’t have that size? With a more flexible scheme, you have less issues there, howeve…

> in modern languages the rule is generally to have fixed-size integers. Modern languages have unlimited size integers :-) "Modern" as in "since at least the 80s, more likely 70s".

Good luck seeing your performance drop off a very sharp cliff if you start using larger numbers than your CPU can fit into a single register.

Re: The road to Zettalinux

#58
post #41

> Matthew Wilcox took the stage to make the point that 64 bits may turn out to be too few — and sooner than we think Let's think critically for a moment. I grew up in the 1980s and 1990s, when we all craved more and more powerful computers. I even remember the years when each generation of video games was marketed as 8-bit, 16-bit, 32-bit, ect. BUT: We're hitting a point where, for what we use computers for, they're…

Supercomputers? Rack-scale computing? See some of the work being done with RDMA and "far memory".

Re: The road to Zettalinux

#59

On one hand The IBM System/38 used 128 bit pointers in the 1970s, despite having a 48 bit physical address bus. These were used to manage persistent objects on disk or network with unique ids a lot like uuids. On the other hand, filling out a 64 bit address space looks tough. I struggled to find something of the same magnitude of 2^64 and I got ‘number of iron atoms in an iron filing’, From a nanotechnological point…

The AS/400 and iSeries also use 128-bit pointers. 128-bit would be useful for multiple pointers already in common use such as ZFS and IP6 addresses. I expect it will the last hop for a long time.

Those are evolved from the System/38.

Re: The road to Zettalinux

#60
post #50

Earlier quoted context omitted.

It seems to me that such a memory space could be physically mapped quite large while still presenting 64-bit virtual memory addresses to the local node? How likely is it that any given node would be mapping out more than 2^64 bytes worth of virtual pages? The VM system could quite simply track the physical addresses as a pair of `u64_t`s or whatever, and present those pages as 64-bit pointers. It seems in particular…

Leaving cluster coherent address space behind - like you say - is doable. But you lose what the parent was saying: > If everyone has the same address space, then you can share pointers / graphs between nodes and the underlying routing/ethernet software will be passing the data automatically between all systems. Its actually quite convenient.

Sounds like a disaster in terms of potential bugs.
Post reply on HN