Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our eighth generation TPUs: two chips for the agentic era

blog.google

Post reply on HN
Print view View original

221–230 of 240 posts

Re: Our eighth generation TPUs: two chips for the agentic era

#221

Post by RealityVoid »

magicalhippo wrote: post #210
WarmWash wrote: post #76

Earlier quoted context omitted.

I don't sweat sources and almost never check them. I usually prefer to manually check information after it's provided, to prevent the model from borking it's context trying to find sources that justify it's already computed output. Almost all the knowledge is already baked into the latent space of the model, so citing sources generally is a backwards process. I see it like going to the doctor and asking them to cite…

Well, I prefer it actually check datasheets so it doesn't go on a wild rabbit hunt to nowhere, since the capabilities it hallucinated for the chip in question doesn't exist.

In my experience, they all do this with dathasheets. Even if they read the actual datasheet, they misunderstand them gravely. I can't relie on them to do unusual setups or chaining stuff properly. It's true I did these attempts a couple of months ago, maybe they're better now.

Re: Our eighth generation TPUs: two chips for the agentic era

#222

Post by otterley »

what wrote: post #213
otterley wrote: post #177

Earlier quoted context omitted.

I'm sure you were kidding, but seriously, the fact that AI-produced music pretty much all sounds the same is a good indicator that AI isn't particularly creative.

It’s not about creativity. The incentive to produce drops to zero when an LLM is just going to slurp it up and regurgitate it without some form of compensation (notoriety, money, whatever).

[dead]

Re: Our eighth generation TPUs: two chips for the agentic era

#223

Post by kamranjon »

overfeed wrote: post #194
kamranjon wrote: post #88

Earlier quoted context omitted.

Consistency, new models don't behave the same on every task as their predecessors. So you end up building pipelines that rely on specific behavior, but now you find that the new model performs worse with regards to a specific task you were performing, or just behaves differently and needs prompt adjustments. They also can fundamentally change the default model settings during new releases, for example Gemini 2.5 mode…

> Consistency, new models don't behave the same on every task as their predecessors. So you end up building pipelines that rely on specific behavior If this is a deal breaker, then self-hosting is the only solution. Due to the hardware premium, all models hosted by 3rd-parties will be deprecated to make room for newer, better, and more efficient models.

Sure, but Google also leaves little to no overlap between models and often will leave models in preview mode (which many companies cannot use in production for legal reasons) - right up until the point that the previous model is deprecated.

The point is that if you want to build a platform that customers can rely on based on their own schedules of feature development, you need to support models for longer periods of time. For example, OpenAI is still offering older models like gpt4 which was released in 2023 - this gives customers plenty of time to test, experiment and eventually migrate to a newer model if it makes sense.

Re: Our eighth generation TPUs: two chips for the agentic era

#224

Post by sigmoid10 »

suttontom wrote: post #140
sigmoid10 wrote: post #13

Earlier quoted context omitted.

I'd bet that too if their management wasn't so incredibly uninspiring. Like, Apple under Cook was also pretty mild and a huge step down from Jobs, but Google feels like it fell off a cliff. If it wasn't for OpenAI releasing ChatGPT, they might still be sitting on that tech while only testing it internally. Now it drives their entire chip R&D.

Google was calling itself an "AI-first" company beginning in 2016 or 2017. They designed and built TPUs nearly a decade ago and were using transformer models in products like Google Translate but didn't make a big fuss about it, it just made the product way better. People should at least credit Sundar somewhat for this, it turned out to be quite prescient, especially the advantage of having your own chips that are sp…

AI was very different in 2016-2017 compared to what it is since ChatGPT. Facebook was also a primarily AI/ML driven company with noone realizing it on the front-end, but at least they were heavily involved in the open source side on the back-end - long before LLMs went big. In fact they enabled them to go big with things like pytorch. Google just stumbled into this. Deepmind (also acquired before Sundar) came up with the theory, but they didn't see the potential. What you call "prescience" I call luck. They did not create the demand for their own technology like e.g. Nvidia did by pushing the field ahead with full force. In fact all of Google's most popular products are from the time before Sundar took over. Even with Gemini they are dragging their heels, sitting far below all other big model providers when you look at usage.

Re: Our eighth generation TPUs: two chips for the agentic era

#226

Post by chaoz_ »

delbronski wrote: post #181
rtaylorgarlock wrote: post #120

Earlier quoted context omitted.

Do you think that Junie specifically inside the context of jetbrains IDEs and tooling matches the competitors?

I think so. From my experience Claude/codex tooling really excels at vibe coding the whole thing. You give it a folder and just say: now make it do this. And you don’t really care for the code. Junie tooling excels when you are more involved. Like, look in these two files, add this specific functionality, in this specific way. Junie is usually a lot faster and to the point. Very simple tooling , it just works for thi…

It surprises me how JetBrains managed to lose such a great market opportunity.

I don't think they ever going to be able to re-claim large chunk of developers who are now fine with thin VSCode-like + Terminal for non-JVM languages.

Perfect example of how large corp with research capacity failed to navigate their product changes.

Re: Our eighth generation TPUs: two chips for the agentic era

#227

Post by [deleted] »

magicalhippo wrote: post #210
WarmWash wrote: post #76

Earlier quoted context omitted.

I don't sweat sources and almost never check them. I usually prefer to manually check information after it's provided, to prevent the model from borking it's context trying to find sources that justify it's already computed output. Almost all the knowledge is already baked into the latent space of the model, so citing sources generally is a backwards process. I see it like going to the doctor and asking them to cite…

Well, I prefer it actually check datasheets so it doesn't go on a wild rabbit hunt to nowhere, since the capabilities it hallucinated for the chip in question doesn't exist.

[deleted]

Re: Our eighth generation TPUs: two chips for the agentic era

#229

Post by WarmWash »

svnt wrote: post #220
WarmWash wrote: post #76

Earlier quoted context omitted.

I don't sweat sources and almost never check them. I usually prefer to manually check information after it's provided, to prevent the model from borking it's context trying to find sources that justify it's already computed output. Almost all the knowledge is already baked into the latent space of the model, so citing sources generally is a backwards process. I see it like going to the doctor and asking them to cite…

Good luck all-in. But seriously what are you doing that this works? I guess if you are writing pop culture articles this might work. For anything where the output has consequences I can’t imagine finding success like this.

The latent space knowledge that the models have is stronger than the inference agent going out and trying to find information to integrate into context.

If you ask why the sky is blue, the model already has the answer. It's corrosive to your conversation to pull a bunch of unknown sources into context so the model can appease your "feels right" request.

If you don't trust the answer, your brain is still way way better at quickly scanning sources to verify the answer.

But the fact of the matter is that these models went from stumbling over "9 + 7 =" three years ago to solving erdos problems today. And benchmarks (that are so saturated we don't even both with them anymore) reveal that the models basically all have total encyclopedic knowledge of every major career field. Which also makes sense because the labs have been purposely drilling hard on building pristine datasets of all this knowledge.

I would challenge you to find one firmly established general academic question that a SOTA model gets wrong. Good luck.

Re: Our eighth generation TPUs: two chips for the agentic era

#230

Post by bigyabai »

sigmoid10 wrote: post #224
suttontom wrote: post #140

Earlier quoted context omitted.

Google was calling itself an "AI-first" company beginning in 2016 or 2017. They designed and built TPUs nearly a decade ago and were using transformer models in products like Google Translate but didn't make a big fuss about it, it just made the product way better. People should at least credit Sundar somewhat for this, it turned out to be quite prescient, especially the advantage of having your own chips that are sp…

AI was very different in 2016-2017 compared to what it is since ChatGPT. Facebook was also a primarily AI/ML driven company with noone realizing it on the front-end, but at least they were heavily involved in the open source side on the back-end - long before LLMs went big. In fact they enabled them to go big with things like pytorch. Google just stumbled into this. Deepmind (also acquired before Sundar) came up with…

This is a bizarre accounting of things. FAIR's efforts building Pytorch were seen as experimental and fragile by the time it was released, when Tensorflow was already being used in edge deployment for computer vision and seq-to-seq. Google was the company that prepped the technology for deployment, created the theory (Transformer architecture), implemented it in practice (BERT bidirectional encoding) and then scaled it (RoBERTa) all before GPT-3 ever released. Three years before Facebook released Llama.

> They did not create the demand for their own technology like e.g. Nvidia did by pushing the field ahead with full force.

They did, though. You are commenting on an eighth-generation TPU product that has been used millions of times a day for the past half-decade. It's likely that this will be the hardware providing inference for Apple's Gemini model they've selected to use with Siri. TPUs are the economically-conscious inference choice if you've already separated your training/inference workflows.

Post reply on HN