[deleted]
David Patterson Says It’s Time for New Computer Architectures and Languages
111–120 of 204 posts
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#112Are there languages that have first-class support for representing/optimizing memory hierarchy characteristics? Optimizing C compilers, for example, may have extensions to force specific alignments: https://software.intel.com/en-us/articles/coding-for-perform... But I'm not aware of languages where e.g. declaring alignments is part of the base language. Awareness of L1, L2, L3 cache characteristics, plus NUMA nodes,…
The closest I can think of is shared memory in CUDA / local memory in OpenCL.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#113Earlier quoted context omitted.
I vaguely remember my computer arch course in college describing how everything we have today essentially suffers from the von neumann bottleneck, that even if we implemented a lambda calculus machine, it would still suffer from this bottleneck. Has this been revisited in academia lately? I am a bit out of touch, but this always interested me.
You can interpret my comment thus: I propose having lots and lots of Von Neumann bottlenecks!
You're very clever, young man, very clever. But it's bottlenecks all the way down!
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#114Earlier quoted context omitted.
I generally agree: it's about weighing trade-offs, and calling software companies "lazy" is perhaps an oversimplification. However, I do believe the speed of light and size of atoms puts an upper limit on the degree you can keep throwing hardware at the problem. We may continue to find tricks such as quantum computing, but I suspect it will get ever tougher. There's a theoretical limit to how small you squeeze data a…
> We may continue to find tricks such as quantum computing Quantum computing does not magically speed up general purpose computing tasks. We only know a handful of algorithms where quantum computers provide algorithmic complexity reductions and even that that does not mean they necessarily calculate fast in terms of IPC, they just calculate previously intractable problems in reasonable timescales. TL;DR: you won't po…
Totally glib reply, but here it is.
If I can represent screen state as quibits, and I have a whole lot of quibits, I can represent a game as a little list of player inputs.
Set up an enormous superposition, and as players hit keys, collapse the next frame. The imaginary hardware could be super quick!
TL;DR: yeah probably, but you never know.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#115Earlier quoted context omitted.
probably a dumb question but if you exposed the low level memory manipulation wouldn't you just end up with something like assembly?
I don't think it's a dumb question. And I think you're right: the closer you get to the way the CPU you're targeting likes its memory optimally managed, you're basically heading towards assembly. I have notions that there can be something 'between' assembly and, for instance, C, but that's a daydream of mine without any details worked out yet.
I’m not sure you’re actually heading down the path of assembly in terms of the language itself, but you’re definitely moving towards the architectural lock-in of assembly.
But maybe thats fine if the language can express architectural-specific decisions (like memory layout) and abstract it away in a more sane fashion than ifdefsc and the rest of the codebase can maintain the pretense of portability.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#116It's way easier to push the limits on simple things. We need languages with fewer features and clear design, running on hardware with less exotic features.
I agree in general, and while I think that is totally completely possible in languages and operating environments, I'm not so sure it is in hardware, at least not without sacrificing a lot of performance. People respected by people I trust, who have a lot more domain knowledge than I do, don't seem to think it is.
On the other hand, we can afford to sacrifice a lot of performance.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#117Are there languages that have first-class support for representing/optimizing memory hierarchy characteristics? Optimizing C compilers, for example, may have extensions to force specific alignments: https://software.intel.com/en-us/articles/coding-for-perform... But I'm not aware of languages where e.g. declaring alignments is part of the base language. Awareness of L1, L2, L3 cache characteristics, plus NUMA nodes,…
Representing memory hierarchy in a somewhat-HLL was the goal of the programming language Sequoia, presented in the paper 'Sequoia: Programming the Memory Hierarchy' [1][2] from Stanford University. Abstract: "We present Sequoia, a programming language designed to facilitate the development of memory hierarchy aware parallel programs that remain portable across modern machines featuring different memory hierarchy conf…
(Disclaimer: I work on Legion)
Sequoia has been largely superseded by its spiritual successor Legion [1], another programming system by Alex Aiken. Legion doesn't focus so much on low-level memory hierarchies, but it does a very, very good job of scaling to very large machines, and taking advantage of heterogeneous processors such as GPUs. It is also incomparably better at working with dynamic programs, whereas Sequoia needed to know basically everything about the program up front (including what machine it will run on).
If you're looking for a modern version of Sequoia that actually runs on modern hardware, I would strongly recommend looking at Legion. (Or the language component of Legion, Regent [2].)
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#118Earlier quoted context omitted.
Even SIMD, a feature of processors for over 20 years, is not handled well by compilers for automatic optimization. It seems like all code taking advantage of SIMD either drops into ASM, or uses a library from ARM/Intel to give descriptive C functions names for the underlying ASM.
I wonder why high level languages themselves don't add syntax support for wide-math instructions? I understand why loop unpacking is a little tricky, so why not let the programmer take care of it?
[deleted]
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#119> We are now a factor of 15 behind where we should be if Moore’s Law were still operative. We are in the post-Moore’s Law era.
Is this true? According to https://ourworldindata.org/technological-progress it looks like we are maybe a factor of 2 or 4 off.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#120I have had this fun idea for an odd architecture floating in my head for ages.
Lots of small processors with local work ram, cached ram, shared ram.
Each processor has a numerical id, communicates with each processor with one bit different in the id by optical link plus an additional link with to the processor complimenting all bits. They send messages and fill their caches from the pool of shared ram.
Place even parity processors on one board and odd parity processors on another board facing towards the first. Thus all processors have line of sight on their communication partners. Messages go back and forth with processors relaying messages by fixing one bit in the message address. All neighbours with a one bit better address are candidates for relaying. If any are busy, they have options to use another path.
This means the most number of hops a 2^19 core system would have to do is 9. If more than half of the bits are wrong then jump to the compliment.
So the example with half a million processors, each talking to 20 neigbours by optical links. Messages can be sent anywhere with a latency of up to HopTime*9. Filling a cache from anywhere in the shared memory pool would have a latency of twice that. If speed of light is the latency factor a 150ms latency would get you 9 hops across a 5 meter gap. Smaller is of course always going the make it better.
This is the sort of thing that would also probably need a new language. I'm not entirely sure you could come up with an appropriate language without at least a simulation of the architecture.