Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Today's Cheap AI Services Won't Last

vincentschmalbach.com

Post reply on HN
Print view View original

11–20 of 33 posts

Re: Today's Cheap AI Services Won't Last

#11

Post by antupis »

worstspotgain wrote: post #7

This was a huge concern in 2022, before open models with useful IQ levels were released. Today, it's hard to imagine prohibitive prices at the end-user level, because it's hard to envision an application that won't eventually have a "good enough" open/free on-device inference implementation. Two caveats: application-specific patents are still possible (and many torpedo patents are undoubtedly en route right now) and…

I would say RAG + llama3 is already in that good enough level. Claude and gpt4-o are best but when you need steady performance or a cheap interface on a scale is it very hard to compete against llama3.

Re: Today's Cheap AI Services Won't Last

#12

Post by dscottboggs »

londons_explore wrote: post #2

> Price increases: As VC funding dries up and companies face pressure to turn a profit, we'll likely see sharp increases in the cost of AI services. I disagree. Yes, the VC funding will dry up, but hardware and algorithmic advances will decrease running costs by equal amounts. Lack of a moat will prevent companies recouping past expenses, since any who try will be outcompeted by new market entrants who don't have tho…

Do you have some numbers to back that up or just vibes?

Re: Today's Cheap AI Services Won't Last

#14

Post by [deleted] »

londons_explore wrote: post #2

> Price increases: As VC funding dries up and companies face pressure to turn a profit, we'll likely see sharp increases in the cost of AI services. I disagree. Yes, the VC funding will dry up, but hardware and algorithmic advances will decrease running costs by equal amounts. Lack of a moat will prevent companies recouping past expenses, since any who try will be outcompeted by new market entrants who don't have tho…

[deleted]

Re: Today's Cheap AI Services Won't Last

#15

Post by hexo »

Of course. At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2). Funny that 60W old school light bulb is bad bad bad, but 400W gfx card is ok and armies of 3kW servers to produce bs responses to queries are also ok. Consuming ridiculous kilowatts to produce even more bs video is also "no problemo". Hm. Poor artists, I can see why they're furious about this.

Re: Today's Cheap AI Services Won't Last

#16

Post by cosmicradiance »

londons_explore wrote: post #2

> Price increases: As VC funding dries up and companies face pressure to turn a profit, we'll likely see sharp increases in the cost of AI services. I disagree. Yes, the VC funding will dry up, but hardware and algorithmic advances will decrease running costs by equal amounts. Lack of a moat will prevent companies recouping past expenses, since any who try will be outcompeted by new market entrants who don't have tho…

> Yes, the VC funding will dry up, but hardware and algorithmic advances will decrease running costs by equal amounts. The author said the same - "The current reliance on expensive GPU clusters may give way to more specialized, efficient AI hardware. This could help mitigate some cost pressures."

As with the cloud, prices increase once competition dries out. Even if hardware becomes cheap, the services will cost more because shrewd and greedy boards run most businesses.

Re: Today's Cheap AI Services Won't Last

#18

Post by nertirs »

londons_explore wrote: post #2

> Price increases: As VC funding dries up and companies face pressure to turn a profit, we'll likely see sharp increases in the cost of AI services. I disagree. Yes, the VC funding will dry up, but hardware and algorithmic advances will decrease running costs by equal amounts. Lack of a moat will prevent companies recouping past expenses, since any who try will be outcompeted by new market entrants who don't have tho…

I would even argue, that a lot of AI services in the future will be close to free for the public. That is because in a lot of cases, the data received from user interactions is more valuable, than the data generated by the AI service.

Re: Today's Cheap AI Services Won't Last

#20

Post by Ukv »

hexo wrote: post #15

Of course. At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2). Funny that 60W old school light bulb is bad bad bad, but 400W gfx card is ok and armies of 3kW servers to produce bs responses to queries are also ok. Consuming ridiculous kilowatts to produce even more bs video is also "no problemo". Hm. Poor artists, I can see why they're furious about this…

> At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2).

5 joules per token[0] * 200 tokens per query * 10 queries per day * 365 days per year = 1.014kWh

Which, going by the US's energy mix[1], means a year of someone's LLM usage would be about 0.4kg of CO2 - the same as a single cup of coffee[2]. Worth double-checking, since these are just napkin calculations and I could have made a large mistake.

If it is correct (within an order of magnitude or two), that seems to me a relatively small amount of energy. Obviously still expensive to provide for free to hundreds of millions of users, as almost anything would be.

[0]: https://arxiv.org/pdf/2310.03003

[1]: https://www.eia.gov/tools/faqs/faq.php?id=74&t=11

[2]: https://www.co2everything.com/co2e-of/coffee

Post reply on HN