91–100 of 389 posts
Earlier quoted context omitted.
This may just be a classic case of Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox In short, better hardware will drive down token cost in the near-term, but will drive up the demand for tokens as it gets cheap enough for other sectors to start to use it heavily. It comes from steam engines where economists originally thought that coal demand would plummet with more efficient engines, but it actually jus…
[flagged]
Earlier quoted context omitted.
the total cost spent on tokens may go up, but i just cant imagine per token costs going up
Depends on compute capacity. If we become supply constrained on tokens, then prices will necessarily go up.
I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves. For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough. While 2 years ago nothing was useful more than 1 year long, there are many older m…
Earlier quoted context omitted.
and amazon shipping used to be free without prime, and uber used to be cheaper than taxis, and airbnb used to be cheaper than hotels. you really don't get it?
almost every pure tech commodity has gone down in price - gpus - retail computers - laptops - ~gpu~ appliances like washing machines - cloud computing i think you don't get how economy usually works in tech
I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al
Why censor yourself?
I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.
I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves. For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough. While 2 years ago nothing was useful more than 1 year long, there are many older m…
Unfortunately, there's some real physical constraints: IIRC, manufacturing a wafer takes on the order of a month, start to finish, for the physical processing.
Maybe once LLM improvements asymptote further?
I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves. For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough. While 2 years ago nothing was useful more than 1 year long, there are many older m…
Probably! But not viable yet; the chips would be about a year behind SOTA. Note the ~16 months that the article quotes as being insanely fast to get this chip to tape-out (read: start producing). We'll have to bootstrap our way there: AI is actively being used to get us closer to viable lead times for this. Unfortunately, there's some real physical constraints: IIRC, manufacturing a wafer takes on the order of a mont…
Earlier quoted context omitted.
and amazon shipping used to be free without prime, and uber used to be cheaper than taxis, and airbnb used to be cheaper than hotels. you really don't get it?
almost every pure tech commodity has gone down in price - gpus - retail computers - laptops - ~gpu~ appliances like washing machines - cloud computing i think you don't get how economy usually works in tech
Well Sam Altman finally has built a moat against Chinese open weight AI. Well done. But what will this mean for Cerebras? I remember when Tesla was building its own inference chips, and after about 2 years and billions spent, the whole effort was scuttled b/c they simply could not keep up with the iteration and R&D cycles of dedicated chip companies. I suspect the same will be the case with OpenAI vs Cerebras + Nvidi…
Hes got a press release.
The issue is, baking something to silicon requires discipline and about 2 years.
This isn't something you can just change your mind on halfway through. Trust me, I know. You need a clear vision of what you want to support, why and what bits of a chip you need to achieve that.