Time for an AI winter I guess.
This statement was made for experimental purposes, to understand HN biases.
Cerebras’ new monster AI chip adds 1.4T transistors
161–169 of 169 posts
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#162Any article or headline talking about transistor count is just a puff piece. The performance matters, the number of transistors never does, and I have never purchases anything based on the number of transistors.
I'm not going to buy one, but its more because I don't have several million dollars to spare.
I get that, but it's just a thing I've noticed about tech puff pieces. If the transistor count is front and center it's probably not worth reading.
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#163How do they do heat management and dissipation on such a big wafer? I can imagine some different parts heats up differently, putting a mechanical strain on wafer, and leading to cracks.
If you look at the die shots (wafer shots?), you'll notice small holes a few millimeters across spaced roughly at reticle spacing. Those are drilled holes to allow through-wafer liquid cooling. With liquid cooling not just around but through the wafer, the temperature differential is minimized.
thank you
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#164Earlier quoted context omitted.
You do realize that AI crossed human expert performance in NLP / Vision tasks ?
I'm not sure if you're assuming best intentions. If you do, it'd be nice to provide sources - to my knowledge, vision under non-optimal conditions (rain, snow, sunlight ahead) is only partially solved by resorting to sensors resistant to the disturbance. I'd be glad to learn I'm wrong.
TBH, I believe we need another AI winter to come up with something better than the AI "black boxes" we have right now.
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#165> That chip needed some 20,000 amps of current fed to it from one million copper connections to a fiberglass circuit board atop the wafer. Is the 20,000 amp number not an error?
They also say the peak power draw is 23kW. Now 23kW/20kA = 1.1V which is about what my NVIDIA 2080 GPU uses at load. So that does indeed seem to work out.
Of course 20kA is pretty insane, would be interesting to see just how they feed the power to this thing. Bond wires going to the middle of the chip? And the local regulator must be a beast, or rather lots of beasts.
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#166Earlier quoted context omitted.
Here's a video made by the author, Ian Cuttress, which goes into more detail as well. https://www.youtube.com/watch?v=FNd94_XaVlY
This is very cool. The explanation of how yield/defects was interesting: they can bypass cores with defects due to channeling and account for the statistical defects, allowing them to have a 100% yield.
This is how pretty much all large chips are designed nowadays. The perfect chips are expensive higher-capacity / faster chips, the less perfect chips are cheaper lower capacity and/or slower.
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#167Earlier quoted context omitted.
Maybe people will get to their senses and switch to Julia instead of having to waste all these time on Python bindings.
Honestly I think julia missed the boat. There was an opportunity for julia to treat the gpu as an abstract distributed node (which is a first class concept in julia) and pay respect to the reality of data gravity. But they chose to instead basically treat the gpu as a synchronous entity. I'm now bullish on elixir-nx because I think there's an outside shot they will get it right. Python is a lost cause.
I'm not sure that's necessarily the domain of a low-level package like CUDA.jl though (which I assume you're referring to). That kind of interface is more the domain of higher-level packages like https://github.com/JuliaGPU/DaggerGPU.jl and to a lesser extent https://juliagpu.github.io/KernelAbstractions.jl/stable/. Moreover, the jury is still out on whether the built-in Distributed module is an ideal abstraction for every use-case (clusters, heterogeneous compute, etc.)
WRT Nx, my biggest question is how they'll crack the problem of still needing big balls of C++ and the shims everywhere to get acceleration. Creating a compiler that generates efficient GPU or other accelerator code is a massive research project with no clear winners, never mind the challenge of reconciling the very mutation-heavy needs of GPU compute with a mostly immutable language model.
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#168Earlier quoted context omitted.
Honestly I think julia missed the boat. There was an opportunity for julia to treat the gpu as an abstract distributed node (which is a first class concept in julia) and pay respect to the reality of data gravity. But they chose to instead basically treat the gpu as a synchronous entity. I'm now bullish on elixir-nx because I think there's an outside shot they will get it right. Python is a lost cause.
I'm not sure that's necessarily the domain of a low-level package like CUDA.jl though (which I assume you're referring to). That kind of interface is more the domain of higher-level packages like https://github.com/JuliaGPU/DaggerGPU.jl and to a lesser extent https://juliagpu.github.io/KernelAbstractions.jl/stable/ . Moreover, the jury is still out on whether the built-in Distributed module is an ideal abstraction fo…
There's plenty of mutability escape hatches in the erlang vm, and basically everyone who works in the language long enough is familiar with the idea of holding on to an immutable reference to a mutable thing. Like a database connection. Or a connection to a remote microservice. Or an ETS table.
Re: Cerebras’ new monster AI chip adds 1.4T transistors
#169Earlier quoted context omitted.
I don't really agree, at least from a longer term perspective. It's early days yet, but XLA seems to be a promising intermediate representation for allowing the DL frameworks to run on a wider array of hardware without user-facing software changes. It has traction with Google, NVIDIA, IIRC Intel and maybe more (others are definitely using the same approach of compute graph splitting and subgraph scheduling, but I'm n…
Mr/mrs anonymous HN person, please put some info in your profile. You clearly have some deep knowledge of TPUs that I didn’t expect to pop up offhandedly on HN. You’re correct on all counts: dynamic tensor shapes are more or less impossible with XLA, making it more or less impossible to train a model with arbitrary image size inputs, even though the math would allow for that; the pytorch XLA work on TPUs is indeed ki…
Ah I see - I think we're pretty much on the same page in terms of timetables. Although if you include TPU, I think it's fair to say that custom accelerators are already a moderate success.
Updated my profile. I've been working on DL training platforms and distributed training benchmarking for a bit so I've gotten a nice view into the GPU/TPU battle.
Shameless plug: you should check out the open-source training platform we are building, Determined[1]. One of the goals is to take our hard-earned expertise on training infrastructure and build a tool where people don't need to have that infrastructure expertise. We don't support TPUs, partially because a lack of demand/TPU availability, and partially because our PyTorch TPU experiments were so unimpressive.
[1] GH: https://github.com/determined-ai/determined, Slack: https://join.slack.com/t/determined-community/shared_invite/...