111–120 of 182 posts
Kinda feels like the main thing google launches is waiting lists.
Gmail was a waitlist or invite only for many years. And that must’ve been their most successful product launch since search
Earlier quoted context omitted.
I think in this case they just know the demand is RED HOT and they don't have nearly the supply to go around. I don't think it's really the typical new product concerns on this one (product-market fit, are we covering use cases, are there technical problems, etc.). They know people want this and would rather have it right this second, problems and all, than wait for a slow rollout; Google just doesn't have the supply…
Then sell it for more. Just give me a price. Or let me bid on it. Or, don't announce it like it's launched until it's usable.
Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?
As far as stock prices, there was a hype cycle paired with government handouts to the people, these combined to push tech stocks to unreasonable valuations.
Earlier quoted context omitted.
TPUs do compete with GPUs for ML tasks, so yes, this is evidence that GPUs are winning. The only alternative I could imagine is that TPUs will "win" at supercomputers exclusivity aimed at inference (as opposed to training). Since TPUs excel at inference. The question is how much ML compute is used for inference as opposed to training. Not much, I guess, otherwise something like TPUs would be more popular.
Since you seem knowledgable on this topic, what is it that TPUs do differently than GPUs? Why are they better at inference?
I think the most interesting AI hardware stuff is about memristors or some type of compute-in-memory. https://arxiv.org/pdf/2303.07470.pdf https://ieeexplore.ieee.org/abstract/document/9669041 Maybe there will be something like transformers but more suited to crossbar arrays of memristors. If that actually makes sense.
Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?
Thinking about it, it’s hard to believe how fast the hype cycle moved on from crypto. Only 1-2 years ago every media person, influencer, YouTuber, tweeter etc. were talking about/selling/shilling some kind of crypto, and now all of it seems to have moved on to AGI doomsaying.
People want to rent the pricey NVIDIA DGX H100. So Google just put it in their DC, letting customers pay its full price it every ~3 months; plus they don't have to operate it, which is win (or is it ?).
they have to pay for the power and the staff to set them up/manage it
Earlier quoted context omitted.
Interestingly the "4th generation Intel Xeon Scalable processors" themselves have up to 2.45 TBps in memory bandwidth, with the 8-socket configuration, or 2 TBps with 2-socket Xeon Max and HBM. If they'd make an 8-socket Xeon Max it would have 8 TBps. Considering that the Xeon Max 9462 is $8000 vs. the H100 going for north of $40,000, that could be interesting.
The throughput these gpu's have make the price pretty competitive, but I think AMD is working on a APU in their instinct lineup. That could be pretty competitive since Nvidia is overcharging for memory and you could just use sticks instead
The physical integrity needed for extremely high bandwidth interfaces is just really tough to achieve on a DIMM-like slot without really advanced high-channel socket topologies. Those numbers listed before aren't for nothing; 2.4TBps bandwith for an 8-socket Xeon vs 2.0Tbps with a 2-socket Xeon using HBM2 is a very significant improvement in overall efficiency.
AMD should be gifting their GPUs by the dozens to the most prolific Open Source contributors if they want a piece of the cake. Their lack of access to CUDA is really harming them badly.
It's more than just that: for the money, their consumer GPUs don't compete in compute tasks (especially inference/training) and their Linux compute drivers are a pile of steaming garbage on consumer hardware. It's really interesting/depressing to watch as they've done a nice job of supplying good open source graphics drivers. They really seem to be lacking something at a leadership level in terms of understanding GPU…
I think the most interesting AI hardware stuff is about memristors or some type of compute-in-memory. https://arxiv.org/pdf/2303.07470.pdf https://ieeexplore.ieee.org/abstract/document/9669041 Maybe there will be something like transformers but more suited to crossbar arrays of memristors. If that actually makes sense.
If you’re still doing row-column access, it’s just another Von Neumann machine. If you have compute hardware within each row to perform operations on every row in parallel, it’s now just another ALU.