191–200 of 209 posts
Earlier quoted context omitted.
Don't worry, with LLMs, we're moving away from anything that remotely looks like "stable software" :) Also, yeah, I recall the dreaded days of cooperative multitasking between apps. Moving from Windows 3.x to Linux was a revelation.
With LLM's it is just more visible. When the age of "updates" begun, the age of stable software died.
There's a lot here that seems to misunderstand GPUs and SIMD. Note that raytracing is a very dynamic problem, where the GPU isn't sure if a ray hits a geometry or if it misses. When it hits, the ray needs to bounce, possibly multiple times. Various implementations of raytracing, recursion, dynamic parallelism or whatever. Its all there. Now the software / compilers aren't ready (outside of specialized situations like…
It's quite frustrating, and exacerbated by frequent intro-level CUDA blog posts which often just repeat what they've read.
re: raytracing, this might be crazy but, do you think we could use RT cores to accelerate control flow on the GPU? That would be hilarious!
There's a lot here that seems to misunderstand GPUs and SIMD. Note that raytracing is a very dynamic problem, where the GPU isn't sure if a ray hits a geometry or if it misses. When it hits, the ray needs to bounce, possibly multiple times. Various implementations of raytracing, recursion, dynamic parallelism or whatever. Its all there. Now the software / compilers aren't ready (outside of specialized situations like…
Having talked to many engineers using distributed compute today, they seem to think that (single-node) parallel compute haven't changed much since ~2010 or so. It's quite frustrating, and exacerbated by frequent intro-level CUDA blog posts which often just repeat what they've read. re: raytracing, this might be crazy but, do you think we could use RT cores to accelerate control flow on the GPU? That would be hilariou…
But there is seemingly a generalization here to the Raytracing software ecosystem. I dunno how much software / hardware needs to advance here, but we are at the point where Intel RT cores are passing the stack pointers / instruction pointers between shaders (!!!). Yes through specialist hardware but surely this can be generalized to something awesome in the future?
------
For now, I'm happy with stream expansion / stream compaction and looping over consume buffers and producer/append buffers.
If we had distributed operating systems and SSI kernels, your computer could use the idle cycles of other computers [that aren't on battery power]. People talk about a grid of solar houses, but we could've had personal/professional grid computing like 15 years ago. Nobody wanted to invest in it, I guess because chips kept getting faster.
SSI is an interesting idea, but the actual advantage is mostly to improve efficiency when running your distributed code on a single , or few nodes. You still have to write your code with some very real awareness of the relevant issues when running on many nodes, but now you are also free to "scale down" and be highly efficient on a single node, since your code is still "natively" written for running on that kind of s…
In the worst-case, your code is just running on the CPU you already have. If you have another node/CPU, you can schedule your whole process on that one, which frees up your current CPU for more work. If you design your app to be more scalable to more nodes/CPUs, you get more benefits. So even in the worst case, everything would just be... exactly the way it is today. But there are many cases that would be benefited, and once the platform is there, more people would take advantage of it.
There is still a massive opportunity in general parallel computing that we haven't explored. Plenty of research, but along specific kinds of use cases, and with not nearly enough investment, so the little work that got done took decades. I think we could solve all the problems and make it generally useful, which could open up a whole new avenue of computing / applications, the way more bandwidth did.
(I'm referring to consumer use-cases above, but in the server world alone, a distributed OS with simple parallel computing would transform billion-dollar markets in software, making a whole lot of complicated solutions obsolete. It might take a miracle for the code to get adopted upstream by the Linux Mafia, though)
Earlier quoted context omitted.
Having talked to many engineers using distributed compute today, they seem to think that (single-node) parallel compute haven't changed much since ~2010 or so. It's quite frustrating, and exacerbated by frequent intro-level CUDA blog posts which often just repeat what they've read. re: raytracing, this might be crazy but, do you think we could use RT cores to accelerate control flow on the GPU? That would be hilariou…
RT cores? No. Too primitive and specific. But there is seemingly a generalization here to the Raytracing software ecosystem. I dunno how much software / hardware needs to advance here, but we are at the point where Intel RT cores are passing the stack pointers / instruction pointers between shaders (!!!). Yes through specialist hardware but surely this can be generalized to something awesome in the future? ------ For…
Well, it's not that the results were good though, lol.
Earlier quoted context omitted.
What the ever loving hell, it was a perfectly reasonable idea in response to another idea. They weren't saying it should be done, and went out of the way to make it explicit that they are not claiming it would be better. It was a thought exploration, and a valid one, even if it would not pan out if carried all the way to execution at scale. Yes it was handwaving. So what? All ideas start as mere thoughts, and it is u…
It wasn't handwaving or brainstorming. Microsoft even built a research OS like this: https://www.microsoft.com/en-us/research/project/singularity... Have people really never used a higher level execution environment? The JVM and the CLR are the most popular ones. Have people never looked at their internals? Then there's the LISP machines, Erlang, Smalltalk, etc., not to mention a lot of research work on abstract mach…
They are now putting up with JITs in GPGPUs, thanks to the market pressure from folks using languages like Julia and Python, that rather keep using those languages than having to rewrite their algorithms in C or C++.
These are communities that even adopting C over Assembly, and C++ over C, has been an uphill battle, let alone something like a JIT, that is like calling for the pitchforks and torches.
By the way, one of the key languages used in the Connection Machine mentioned on the article was StarLisp.
Interesting article. Other than as an exercise, it's not clear why someone would write a massively parallel 2D renderer that needs a GPU. Modern GPUs are overkill for 2D. Now, 3D renderers, we need all the help we can get. In this context, a "renderer" is something that takes in meshes, textures, materials, transforms, and objects, and generates images. It's not an entire game development engine, such as Unreal, Unit…
Clearly the author never worked with a CM2 - I did though. The CM2 was more like a co-processor which had to be controlled by a (for that age) rather beefy SUN workstation/server. The program itself ran on the workstation which then sent the data-parallel instructions to the CM2. The CM2 was an extreme form of a MIMD design (that is why it was called data parallel). You worked with a large rectangular array (I cannot…
Earlier quoted context omitted.
Oh yes I programmed all the Amiga models, mostly in assembly level. I reprogrammed the ROMs. I also published a magazine on all the Commodore computers internals and build lots of hardware for these machines. We had the parallel Inmos Transputer systems during the heyday of the Amiga, they where much better designed than any the custom Amiga chips.
Inmos was a disaster. No application ever shipped on one. EVER. It used a serial bus to resolve the problems that should have never been problems. Clearly you never wrote code for one. Each oslink couldn't reach more than 3 feet. What a disaster that entire architecture was.