61–70 of 182 posts
Does this mean google just deprecated TPUs? Not surprised.
No, it is the 9,163,584th [0] indication that Google likes to pursue multiple solutions in the same space in parallel with different submarkets, risk profiles, expected payoff terms, or other dimensions.
[0] this is a conservative estimate
Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…
https://arxiv.org/abs/2304.01433 from April 4 of this year.
> I wonder why they didn't try to polish the rough edges with Pytorch, et al.
It's always funny to me when people have this blindspot - because TPUs aren't for you, they're for the ads org. Neither are PyTorch nor TF for that matter. They're more than happy to get external bug fixers but trust me those individual teams dgaf about external customers. They're not in the least bit community driven projects.
Earlier quoted context omitted.
There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better. And isn't your second para…
The word TPU isn't even mentioned here. A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.
Where are you reading that Google is going “all in” on nVidia GPUs? I don’t see that in the linked article at all.
These are clearly targeted at their cloud customers who have workloads tailored to GPUs. They’re supplying demand, as cloud providers do.
Companies can do more than thing at a time.
Earlier quoted context omitted.
There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better. And isn't your second para…
The word TPU isn't even mentioned here. A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.
TBF, there is no mention of anything remotely similar to "I am going to eat nothing but vegetables from now on".
Kinda feels like the main thing google launches is waiting lists.
Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.
For example for palm/bard this was my experience:
"Hey we have this amazing LLM!"
"Great, given you are a company can I pay you money above your costs for this service?"
"No but you can register for updates about when the wait-list will open"
They announced cool features for Google docs as well that I can't use.
Some of the things I've seen announced were maybe a year ago and still nothing. Just a wait-list or less.
I'm more interested in what normal folks are running at home. What are your builds?
Kinda feels like the main thing google launches is waiting lists.
Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.
Technical detail: > Each A3 supercomputer is packed with 4th generation Intel Xeon Scalable processors backed by 2TB of DDR5-4800 memory. But the real "brains" of the operation come from the eight Nvidia H100 "Hopper" GPUs, which have access to 3.6 TBps of bisectional bandwidth by leveraging NVLink 4.0 and NVSwitch.
Considering that the Xeon Max 9462 is $8000 vs. the H100 going for north of $40,000, that could be interesting.
Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…
Earlier quoted context omitted.
Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.
I get it, but the launches are always about what you can now do then slowly followed with "some partners can register interest". For example for palm/bard this was my experience: "Hey we have this amazing LLM!" "Great, given you are a company can I pay you money above your costs for this service?" "No but you can register for updates about when the wait-list will open " They announced cool features for Google docs as…
Not everything gets launched because sometimes they find out in that testing period that they got it wrong.