Earlier quoted context omitted.
They lost their way or HN readers are out of touch?
Google Drive losing customer data is which one? Context: https://www.theregister.com/2023/11/27/google_drive_files_di...
Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
11–20 of 28 posts
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#12Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#13Fantastic blog! In your post, you highlighted the use of Cloud TPU v5e with GKE for AI inference. How does this setup maintain high performance while managing costs, especially in high-demand scenarios like real-time data processing or live interactions?
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#14Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#15LLM tech also seems to be easily swapped between providers. So my view is that the value proposition is in the hardware designers and manufacturers (Nvidia, tsmc, Google).
Why are more people not excited by what is happening at Google? What am I missing?
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#16AI is a big deal this year. Google is one of the biggest tech companies. Yet a post about AI at Google gets... No comments at all. Back in 2005, this would have been the most talked about news of the day. Now, nobody cares what Googles up to. They lost their way.
Not my field & this could be super wrong. But my skepticism that a TPU5e launch was going to help anyone but some niche well connected other big players has been very high. I'm interested from an academic sense, and I love work like the Keras 3.0 release that makes me much more universal/portable, such that hardware like this or others can burn f-ing metal. But I don't have much trust this will actually be available at a reasonable price/with any kind of availability. https://news.ycombinator.com/item?id=38446353 https://keras.io/keras_3/
I'd love to be wrong & have this be actually available at a good price to change what is possible for people, & still interesting. Your call out is right. There's no lightning rod here, nothing to be hyoer-critical of, and few step outside the comfort zone of CUDA have anywhere near enough idea how to assess. This is an unknown on the radar for most.
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#17AFAIK Google is in the middle of massive infrastructure investment in hardware which is of similar performance to Nvidia, but they will own the whole stack. They will also be able to deploy an order of magnitude more compute than openAI to LLMs in 2024/25. LLM tech also seems to be easily swapped between providers. So my view is that the value proposition is in the hardware designers and manufacturers (Nvidia, tsmc,…
Let's not talk about friends losing all access to their accounts, forever.
I switched a whole firm off GCS when their troubleshooting by 'senior' techs was just copy and pasted boilerplate too many times. The way I set things up was cloud agnostic at the time. Not something that is possible at scale.
Can anyone else be trusted? Probably not.
Can you trust someone on the crack-pipe full of advertising dollars?
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#18Earlier quoted context omitted.
Google Drive losing customer data is which one? Context: https://www.theregister.com/2023/11/27/google_drive_files_di...
Actually if you think that Register report supports your first sentence you are perfectly demonstrating that lack of critical thinking that permeates this site.
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#19AI is a big deal this year. Google is one of the biggest tech companies. Yet a post about AI at Google gets... No comments at all. Back in 2005, this would have been the most talked about news of the day. Now, nobody cares what Googles up to. They lost their way.
Google, structurally, sucks at anything but sucking down Ad money. A good LLM allows one to basically bypass the internet for many use cases and drastically reduces ad time. As cool as LLM capabilities are the thing that delights me most about them is I can get high quality (YMMV) answers from the internet without all the godforsaken ad bullshit and wonky websites made only as mazes to keep users in ad gardens. Googl…
Re: Powering cost-efficient AI inference at scale with Cloud TPU v5e on GKE
#20AI is a big deal this year. Google is one of the biggest tech companies. Yet a post about AI at Google gets... No comments at all. Back in 2005, this would have been the most talked about news of the day. Now, nobody cares what Googles up to. They lost their way.