Today's Cheap AI Services Won't Last • Hacker News Board

Re: Today's Cheap AI Services Won't Last

#31

Post by [deleted] » Fri, Jul 05, 2024, 3:39 PM UTC

Ukv wrote: post #30

leereeves wrote: post #29

Earlier quoted context omitted.

I see that the paper that reported 5J per token used LLaMA 65B, a relatively small model. ChatGPT 4 reportedly has trillions of parameters They also didn't measure power use vs context window size, and used single prompts that would be much smaller than conversations (because in a conversation with an LLM, every prompt entered sends the entire conversation to the LLM again.) Or the much larger context windows of RAG,…

> I see that the paper that reported 5J per token used LLaMA 65B, a relatively small model. ChatGPT 4 reportedly has trillions of parameters GPT-4 uses a MoE architecture, composed of many smaller models - (speculated) total parameter count is not particuarly meaningful when the vast majority of its parameters will be in submodels that aren't active for a given query. In terms of capability, LLaMA-3 is about on par w…

[deleted]

Re: Today's Cheap AI Services Won't Last

#32

Post by londons_explore » Fri, Jul 05, 2024, 5:03 PM UTC

vincent_s wrote: post #21

londons_explore wrote: post #2

> Price increases: As VC funding dries up and companies face pressure to turn a profit, we'll likely see sharp increases in the cost of AI services. I disagree. Yes, the VC funding will dry up, but hardware and algorithmic advances will decrease running costs by equal amounts. Lack of a moat will prevent companies recouping past expenses, since any who try will be outcompeted by new market entrants who don't have tho…

1. These companies have burned through billions in VC funding. Just breaking even won't be enough - they'll need significant profits to justify those investments. 2. We have at least one real-world example suggesting profitability is challenging. Microsoft is reportedly still losing money on GitHub Copilot, even after years of operation and many paying customers.

> These companies have burned through billions in VC funding.

These companies, yes. But new startups will come, spend only millions, and achieve the same thing. No moat.

Re: Today's Cheap AI Services Won't Last

#33

Post by verdverm » Fri, Jul 05, 2024, 10:58 PM UTC

merryboxingday wrote: post #13

You guys are paying for AI services? The models are literally free, just run them yourself

I pay much less by using an API, and I get faster/higher quality results, than if I paid for the graphics card(s), electricity, and cooling myself. The ROI is not there for me

Even the $20/month would take a few years to recoup the costs