Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The road to Zettalinux

lwn.net

Post reply on HN
Print view View original

71–80 of 199 posts

Re: The road to Zettalinux

#71

Post by p_l »

wongarsu wrote: post #61
gwbas1c wrote: post #41

> Matthew Wilcox took the stage to make the point that 64 bits may turn out to be too few — and sooner than we think Let's think critically for a moment. I grew up in the 1980s and 1990s, when we all craved more and more powerful computers. I even remember the years when each generation of video games was marketed as 8-bit, 16-bit, 32-bit, ect. BUT: We're hitting a point where, for what we use computers for, they're…

We are talking about the OS kernel that supports 4096 CPU cores. The companies who pay their engineers to do linux kernel development tend to be the same ones that have absurd needs.

And that's not enough to boot on some platforms that linux runs on (even with amd64 ISA) unless one partitions the computer into smaller cpu counts

Re: The road to Zettalinux

#72

Post by throw10920 »

rwmj wrote: post #43
dragontamer wrote: post #34

Earlier quoted context omitted.

> it's difficult to imagine what kind of new physics would let someone fit that many bytes into a machine that's practical to control with a single Linux kernel instance. I nominally agree with most of your post. But I should note that modern systems seem to be moving towards a "one pointer space" for the entire cluster. For example, 8 GPUs + 2 CPUs would share the same virtual memory space (GPU#1 may take one slice,…

Distributed Shared Memory is a thing, but I'm not sure how widely it is used. I found that it gives you all the coordination problems of threads in symmetric multiprocessing but at a larger scale and with much slower synchronisation. https://en.wikipedia.org/wiki/Distributed_shared_memory

That's what I was going to chime in with - you pay for that extra address width. Binary addition and multiplication latency is super-linear with regards to operand width. Larger pointers lead to more memory use, and memory access latency is non-constant with respect to size.

It might make sense for large distributed systems to move to a 128-bit architecture, but I don't see any reason for consumer devices, at least with current technology.

Re: The road to Zettalinux

#73

Post by jerf »

Aperocky wrote: post #60
rektide wrote: post #50

Earlier quoted context omitted.

Leaving cluster coherent address space behind - like you say - is doable. But you lose what the parent was saying: > If everyone has the same address space, then you can share pointers / graphs between nodes and the underlying routing/ethernet software will be passing the data automatically between all systems. Its actually quite convenient.

Sounds like a disaster in terms of potential bugs.

It is, but it's the same disaster of bugs we already have from multiple independent cores sharing the same memory space, not a brand new disaster or anything.

It's a disaster of latency issues too, but it's not like that's surprising anyone either, and we already have NUMA on some multi-core systems which is the same problem.

We have existing tools that can be extended in straightforward ways to deal with these issues. And it's not like there's a silver bullet here; having separate address spaces everywhere comes with its own disaster of issues. Pick your poison.

Re: The road to Zettalinux

#74

Post by dragontamer »

throw10920 wrote: post #72
rwmj wrote: post #43

Earlier quoted context omitted.

Distributed Shared Memory is a thing, but I'm not sure how widely it is used. I found that it gives you all the coordination problems of threads in symmetric multiprocessing but at a larger scale and with much slower synchronisation. https://en.wikipedia.org/wiki/Distributed_shared_memory

That's what I was going to chime in with - you pay for that extra address width. Binary addition and multiplication latency is super-linear with regards to operand width. Larger pointers lead to more memory use, and memory access latency is non-constant with respect to size. It might make sense for large distributed systems to move to a 128-bit architecture, but I don't see any reason for consumer devices, at least w…

> Binary addition ... is super-linear with regards to operand width

No its not. That's why Kogge-Stone's carry lookahead adder was such an amazing result. O(log(n)) latency with respect to operand width with O(n) total half-adders used.

It may seem like its super-linear. But the power of prefix-sums leads to a spectacular and elegant solution. Kogge-stone (and the concept of prefix-sums) is one of the most important parallel-programming / parallel-system results of the last 50 years. Dare I say it, its _THE_ most important parallel programming concept.

> multiplication latency

You could just... not implement 128-bit multiplication. Just support 128-bit pointers (aka: addition) and leave multiplication for 64-bits and below.

Re: The road to Zettalinux

#75

Post by thrown_22 »

wongarsu wrote: post #10
Blikkentrekker wrote: post #3

> How would this look in the kernel? Wilcox had originally thought that, on a 128-bit system, an int should be 32 bits, long would be 64 bits, and both long long and pointer types would be 128 bits. But that runs afoul of deeply rooted assumptions in the kernel that long has the same size as the CPU's registers, and that long can also hold a pointer value. The conclusion is that long must be a 128-bit type. Can anyon…

That's pretty much the mentioned proposal of "just use rust types", which are i16/u16 to i128/u128, plus usize/isize for pointer-sized things. The only improvement that you really need over that is to differentiate between what c calls size_t and uintptr_t: the size of the largest possible array, and the size of a pointer. On "normal" architectures they're the same, but on architectures that do pointer tagging or seg…

stdint.h has been around far longer than Rust.

I've been using those since the 00s for bit banging code where I need guarantees for where each bit goes.

Nothing quite like working with a micro processor with 12bit words to make you appreciate 2^n addresses.

Re: The road to Zettalinux

#76

Post by eterevsky »

jmillikin wrote: post #18

The section about 128-bit pointers being necessary for expanded memory sizes is unconvincing -- 64 bits provides 16 EiB (16 x 1024 x 1024 x 1024 x 1 GiB), which is the sort of address space you might need for byte-level addressing of a warehouse full of high-density HDDs. Memory sizes don't grow like they used to, and it's difficult to imagine what kind of new physics would let someone fit that many bytes into a mach…

8 EiB of data (in case we want addresses using signed integers) is around 20 metric tons of micro-SD cards one TB each (assuming they weigh 2g each). This could probably fit in a single shipping container.

Re: The road to Zettalinux

#77

Post by munro »

There was a post awhile back from NASA saying how many digits of Pi they actually need [1].

    import math

    pi = 3141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117067982148086513282306647093844609550582231725359408128481117450284102701938521105559644622948954930381964428810975665933446128475648233786783165271201909145648566923460348610454326648213393607260249141273724587006606315588174881520920962829254091715364367892590360

    sign_bits = 1
    sig_bits = math.ceil(math.log2(pi))
    exp_bits = math.floor(math.log2(sig_bits))

    assert sign_bits + sig_bits + exp_bits == 1209
I'm sure I got something wrong here, def off-by-one, but roughly it looks like it would need 1209-bit floats (2048-bit rounded up!). IDK, mildly interesting. :>

[1] https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimal...

Re: The road to Zettalinux

#78

Post by bitwize »

Pointers will get fat (CHERI and other tagged pointer schemes) well before even server users will need to byte-address into more than 2^64 bytes' worth of stuff. So we should probably be realistically aiming for 256-bit architectures...

Re: The road to Zettalinux

#79

Post by vanderZwan »

munro wrote: post #77

There was a post awhile back from NASA saying how many digits of Pi they actually need [1]. import math pi = 31415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066…

The value of Pi you mention was the one in the question, the one in the answer is:

> For JPL's highest accuracy calculations, which are for interplanetary navigation, we use 3.141592653589793. Let's look at this a little more closely to understand why we don't use more decimal places. I think we can even see that there are no physically realistic calculations scientists ever perform for which it is necessary to include nearly as many decimal points as you present.

That's sixteen digits, so a quick trip to the dev tools tels me::

    >> Math.log2(3141592653589793)
    -> 51.480417552782754
The last statement of the text I quoted is more interesting though. Although not surprising to me, given how many astronomers I know who joke that Pi equals three all the time.

Re: The road to Zettalinux

#80

Post by jerf »

gwbas1c wrote: post #41

> Matthew Wilcox took the stage to make the point that 64 bits may turn out to be too few — and sooner than we think Let's think critically for a moment. I grew up in the 1980s and 1990s, when we all craved more and more powerful computers. I even remember the years when each generation of video games was marketed as 8-bit, 16-bit, 32-bit, ect. BUT: We're hitting a point where, for what we use computers for, they're…

I think the argument for 128-bits being necessary strictly for routing memory in a single "computer" is fairly weak. We're already seeing the plateauing of memory sizes nowadays; what was exponentially in reach within just a couple of decades exponentially recedes from us as our progress goes from exponential to polynomial.

But the argument that we need more than 64 bit capability for a lot of other reasons in conjunction with memory addressability is I think very strong. A lot of very powerful and safe techniques become available if we can tag pointers with more than a bit squeezed out here and a bit squeezed out there. I could even see hard-coding the CPU to, say, look at 80 bits as an address and then us the remaining 48 for tagging of various sorts. There's precedent, and 80 bits is an awful lot of addressible memory; that's a septillion+ addressible bytes, by the time we need more than that, if we do, our future selves can deal with that. (It is good to look ahead a decade or two and make reasonable preparations, as this article does; it is hubris to start trying to look ahead 50 years or a century.)

Post reply on HN