71–80 of 186 posts
Earlier quoted context omitted.
People with $3 are whales now? TPU prices are similar to other cloud resources.
Does anyone do anything useful with a $3 spend, or is it $3 X $manyManyHours?
The first specifically designed for inference? Wasn’t the original TPU inference only?
Yup. (Source: was at brain at the time.) Also holy cow that was 10 years ago already? Dang. Amusing bit: The first TPU design was based on fully connected networks; the advent of CNNs forced some design rethinking, and then the advent of RNNs (and then transformers) did it yet again. So maybe it's reasonable to say that this is the first TPU designed for inference in the world where you have both a matrix multiply un…
Certainly, RNNs are much older than TPUs?!
It looks amazing but I wish we could stop playing silly games with benchmarks. Why compare fp8 performance in ironwood to architectures which don't support fp8 in hardware? Why leave out TPUv6 in the comparison? Why compare fp64 flops in the El Capitan supercomputer to fp8 flops in the TPU pod when you know full well these are not comparable? [Edit: it turns out that El Capitan is actually faster when compared like f…
Not knowing much about special-purpose chips, I would like to understand whether chips like this would give Google a significant cost advantage over the likes of Anthropic or OpenAI when offering LLM services. Is similar technology available to Google's competitors?
No one else has access to anything similar, Amazon is just starting to scale their Trainium chip.
Its hard to be excited about hardware that will only exist in the cloud before shredding.
Can anyone comment on where efficiency gains come from these days at the arch level? I.e. not process-node improvements. Are there a few big things, many small things...? I'm curious what fruit are left hanging for fast SIMD matrix multiplication.
One big area the last two years has been algorithmic improvements feeding hardware improvements. Supercomputer folks use f64 for everything, or did. Most training was done at f32 four years ago. As algo teams have shown fp8 can be used for training and inference, hardware has updated to accommodate, yielding big gains. NB: Hobbyist, take all with a grain of salt
Can these be repurposed for other things? Encoding/decoding video? Graphics processing etc? edit: >It’s a move from responsive AI models that provide real-time information for people to interpret, to models that provide the proactive generation of insights and interpretation. This is what we call the “age of inference” where AI agents will proactively retrieve and generate data to collaboratively deliver insights and…
https://docs.jax.dev/en/latest/pallas/tpu/details.html#what-...
Its not really useful for other workloads (unless your workload looks like a bunch of matrix multiplications).
The naming of these chips (GPUs, CPUs) is kinda badass: Ironwood, Blackwell, ThreadRipper, Epyc, etc.
I always confuse Blackwell with Bakewell (tart) and my CPU is on Coffee Lake and great… now I want coffee and cake
Earlier quoted context omitted.
I believe my original sentence was accurate. I was expecting the article to provide an objective comparison between TPUs and their main competitors. If you’re suggesting that El Capitan is the primary competitor, I’m not sure I agree, but I appreciate the perspective. Perhaps I was looking for other competitors, which is why I didn’t really pay attention to El Capitan.
Andrey, this is what I'm referring to: https://news.ycombinator.com/item?id=43632709