Well, there are some interesting new languages out there already: https://julialang.org/
In fact Moore’s law must have been the reason why Python is what it is today: “Who cares if it’s slow, tomorrow’s computers will be faster!”
David Patterson Says It’s Time for New Computer Architectures and Languages
11–20 of 204 posts
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#12It's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks. Much of machine learning is a simple repetitive computation running at low precision. Special purpose hardware can do that very well. So what's the next usefu…
It's really hard. Remember the Itanium and the Cell. You can build it, but they may not come.
GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks.
Not all CPU architectures are equally successful. Likewise, not all highly parallel architectures will be equally successful. If progress in hardware is going to be characterized by more parallelization, things like Erlang Actors are going to become more important.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#13> And Intel, he said “is trying to make all the bets,” marketing traditional CPUs for machine learning, purchasing Altera (the company that provides FPGAs to Microsoft), and buying Nervana with it specialized neural network processor (similar in approach to Google’s TPU).
Actually Intel has even wider view of the things and interest to non-(yet?)-mainstream tech, like e.g. async circuit design. They has bought Achronix, Fulcrum, Timeless Design Automation. Fulcrum in 2002 made things like "a fully asynchronous crossbar switch chip that achieves 260-Gbit/second cross-section bandwidth in a standard 180-nanometer CMOS process" - https://www.eetimes.com/document.asp?doc_id=1145012
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#14Are there languages that have first-class support for representing/optimizing memory hierarchy characteristics? Optimizing C compilers, for example, may have extensions to force specific alignments: https://software.intel.com/en-us/articles/coding-for-perform... But I'm not aware of languages where e.g. declaring alignments is part of the base language. Awareness of L1, L2, L3 cache characteristics, plus NUMA nodes,…
Representing memory hierarchy in a somewhat-HLL was the goal of the programming language Sequoia, presented in the paper 'Sequoia: Programming the Memory Hierarchy' [1][2] from Stanford University.
Abstract:
"We present Sequoia, a programming language designed to facilitate the development of memory hierarchy aware parallel programs that remain portable across modern machines featuring different memory hierarchy configurations. Sequoia abstractly exposes hierarchical memory in the programming model and provides language mechanisms to describe communication vertically through the machine and to localize computation to particular memory locations within it. We have implemented a complete programming system, including a compiler and runtime systems for Cell processor-based blade systems and distributed memory clusters, and demonstrate efficient performance running Sequoia programs on both of these platforms."
[1] DOI: 10.1109/SC.2006.55 [2] http://graphics.stanford.edu/papers/sequoia/sequoia_sc06.pdf
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#15Are there languages that have first-class support for representing/optimizing memory hierarchy characteristics? Optimizing C compilers, for example, may have extensions to force specific alignments: https://software.intel.com/en-us/articles/coding-for-perform... But I'm not aware of languages where e.g. declaring alignments is part of the base language. Awareness of L1, L2, L3 cache characteristics, plus NUMA nodes,…
C++ has alignas since C++11 https://en.cppreference.com/w/cpp/language/alignas
and since C++17 has std::hardware_destructive_interference_size and std::hardware_constructive_interference_size https://en.cppreference.com/w/cpp/thread/hardware_destructiv...
which you can use together to either make sure that 2 variables are not in the same cache line (if you are making a lock-free list for example) or to make sure at compile-time that 2 variables are close enough to share a cache-line.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#16It's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks. Much of machine learning is a simple repetitive computation running at low precision. Special purpose hardware can do that very well. So what's the next usefu…
It's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks. Not all CPU architectures are equally successful. Likewise, not all highly parallel architectures will be equally successful. If progress in hardware is going…
I vaguely remember my computer arch course in college describing how everything we have today essentially suffers from the von neumann bottleneck, that even if we implemented a lambda calculus machine, it would still suffer from this bottleneck. Has this been revisited in academia lately? I am a bit out of touch, but this always interested me.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#17The lack of obsolescence is terrible news for Dell and HP. Ran into a client this afternoon still running his seismic workflows on an old HP EliteBook 8570w with a couple big monitors, big SSD and 16 Gigs of memory.
That family was launched mid-2012!
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#18Earlier quoted context omitted.
It's really hard. Remember the Itanium and the Cell. You can build it, but they may not come. GPUs, though. Those have turned out to be very successful, they can be parallelized as much as you're willing to pay for, and they're good for some non-graphics tasks. Not all CPU architectures are equally successful. Likewise, not all highly parallel architectures will be equally successful. If progress in hardware is going…
I vaguely remember my computer arch course in college describing how everything we have today essentially suffers from the von neumann bottleneck, that even if we implemented a lambda calculus machine, it would still suffer from this bottleneck. Has this been revisited in academia lately? I am a bit out of touch, but this always interested me.
You can interpret my comment thus: I propose having lots and lots of Von Neumann bottlenecks!
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#19I'm a VC and I've seen lots of pitches lately for various tech related to new architectures, be it GPU, FPGA or RISC. I'd add cryptocurrency to list workloads with insatiable computing demands.
Massive parallel prefix and reduction operations in RAM. Start with just the MPI standard reduction ops. 1000x faster than Von Neuman bottleneck.
Re: David Patterson Says It’s Time for New Computer Architectures and Languages
#20Well, there are some interesting new languages out there already: https://julialang.org/ In fact Moore’s law must have been the reason why Python is what it is today: “Who cares if it’s slow, tomorrow’s computers will be faster!”
I think the current attitude could be summed up better as "Who care's if it's inefficient, yesterday's computers are fast enough!" I think that's true of most software, but not all -- there will always be people who care about the best performance, but we shouldn't expect them to be in the majority.