Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

221–230 of 436 posts

Re: AI's Affordability Crisis

#221

Post by manapause »

sleepybrett wrote: post #163
manapause wrote: post #138

Earlier quoted context omitted.

Can confirm, my experience in “loop engineering” was “this is neat” for 45 minutes until a daily ration of tokens was evaporated. The quadratic cost trap is prohibitive to experimentation. As a localLLM evangelist, I am hopeful this will bring more attention to the joys of rolling your own sovereign AI.

Yeah, i'm hoping that gets smoother. I've been experimenting with omlx and opencode on my m5x64gb and keep running into issues w/ Qwen3.6-35B-A3B-MLX-8bit exceeding it's memory limit at the most inopportune times. Playing with 12B gemma4 (8bit) more today. Maybe I should be aiming for something targeting 48gb of memory?

It depends what your goals are and what you are using it for. This space is fluid and my answer last week would be different than my answer today! That said there’s no substitute for hard work, here are some resource to get you up to up to speed:

https://carteakey.dev/blog/local-inference/local-llm-optimiz...

https://botmonster.com/ai/self-hosted-ai-agent-frameworks-20...

Personally I find myself swapping models depending if I am engaged in “trad-development” vs building agentic probes or apps involving imagery. Tailscale the LLM to your deployments and ta-da!

Re: AI's Affordability Crisis

#222

Post by okr »

nemomarx wrote: post #112
okr wrote: post #45

Earlier quoted context omitted.

AI is a worker for me. That i pay for. Basically i am in the same game now to reduce the prizes i have to pay for my workers. Just like the employers are, that seek to reduce costs for employees, as we are simply too expensive. We need more competition among the workers. Let's introduce more chinese workforce! ;)

If you had a choice of maybe 3-4 contracting firms to hire workers from and you weren't large enough to negotiate on price I think you'd be in a pet bad spot as a business?

I would say so, yep. I just find it funny, that suddenly i am in the position to find for the cheapest option for my lovely AI workers. While usually me is the one who complains to be underpaid. I am in the same shoe as my employers now.

Re: AI's Affordability Crisis

#223

Post by rescbr »

johnvanommen wrote: post #186
sdesol wrote: post #127

Earlier quoted context omitted.

The big thing is, the western world has moved so much of the manufacturing to China and think a lot of people will not forgive Samsung and others, so I can see China owning a good portion of the supply chain.

> The big thing is, the western world has moved so much of the manufacturing to China I built my career on Solaris and it got rugpulled by Linux. That wasn’t because of software, it was because of hardware. Linux’s cost advantage existed because Sun hardware had huge margins, because their software was basically free. AI will probably be a repeat of this. Whoever can come up with the hardware solution that minimizes…

While people fly to the US to buy Macs at a lower price and bring them back in their backpacks, I guess I'll be flying to HK to buy a Chinese GPU rather sooner than later...

Re: AI's Affordability Crisis

#224

Post by trollbridge »

johnvanommen wrote: post #186
sdesol wrote: post #127

Earlier quoted context omitted.

The big thing is, the western world has moved so much of the manufacturing to China and think a lot of people will not forgive Samsung and others, so I can see China owning a good portion of the supply chain.

> The big thing is, the western world has moved so much of the manufacturing to China I built my career on Solaris and it got rugpulled by Linux. That wasn’t because of software, it was because of hardware. Linux’s cost advantage existed because Sun hardware had huge margins, because their software was basically free. AI will probably be a repeat of this. Whoever can come up with the hardware solution that minimizes…

Fortunately, Solaris skills map to Linux pretty cleanly.

Re: AI's Affordability Crisis

#225

Post by beepbooptheory »

simianwords wrote: post #118
beepbooptheory wrote: post #21

Earlier quoted context omitted.

I'm no economist but if true don't you have the opposite problem? How do you get people to need X many tokens per day such that you can sell enough to make money? Wouldn't you need an absence of competition for that to be ok?

If you are an AI bear you have multiple techniques with you - if AI costs go down you can ask how the companies will make profit and then suggest the bubble popping - if AI costs go up you can ask how people will afford it and then suggest the bubble popping - if companies actually do make profit then you can say the companies are getting too big and powerful so it’s a bad thing for consumers Essentially you have lef…

I get your point but it still like begs the question right? If you are optimistic about it all, what is the good narrative? What does it all look like? Billions upon billions of prompts to the finest models every millisecond? And then we have to scale on top of that? To like what end? How many apps do we need to code? How many questions can a single person even ask on any given day? Do I lack some imagination here?

Like what if they don't necessarily have to be super duper money making machines to legitimate how useful and nice they are for you? Is that even conceivable? What if tomorrow we all decided they are more like utilities? Would that change anything intrinsic about them for you?

Re: AI's Affordability Crisis

#226

Post by KolibriFly »

I feel like the author is jumping way too fast from "OpenAI is losing money" to "the whole AI economy is broken." A company being in the red during aggressive scaling doesn't automatically mean the unit economics don't work.

Re: AI's Affordability Crisis

#227

Post by zyuiop »

mh- wrote: post #181
dominotw wrote: post #166

Earlier quoted context omitted.

how do you have access to their financials? are you an insider? Edit: to the commenter below . It was widely reported that these companies were unprofitable 1 from last year. I am asking question to this specefic comment because they made a very specific claim about part of plan thats profitable . something only an insider would know. 1. https://www.wsj.com/tech/ai/openai-anthropic-profitability-e...

I'm curious why you didn't pose this question to the grandparent commenter, who first asserted the opposite?

The amount of capital they need to raise, despite the claimed revenue, indicates that they spend more than they gain, which is by definition unprofitable.

Re: AI's Affordability Crisis

#228

Post by dzogchen »

> To generate the $309 billion needed to service their debt, the AI industry will need to replace 46.8 million jobs, equivalent to around 27% of the current number of jobs in the US.

Lump of labour fallacy spotted.

Re: AI's Affordability Crisis

#229

Post by zyuiop »

gizzlon wrote: post #140

> Sales and Marketing: $5.73 billion .. That is, OpenAI spent 44% of their revenue on sales and marketing! Anyone know what they are spending this on? Can't remember seeing one OpenAI ad.. Is it just pr and influencers? Ads in the US?

Likely free tokens to attract customers

Re: AI's Affordability Crisis

#230

Post by KolibriFly »

thewebguyd wrote: post #114
sdesol wrote: post #90

Earlier quoted context omitted.

> Microsoft adding Deepseek support I believe this hasn't been confirmed yet but I think it speaks to a bigger problem for the AI companies which is, if you give capable developers a good reasoning LLM, they can make it work like it was a really expensive model. I believe we are 100% at the stage of good enough for the vast majority of tech companines. Fable and others will be more valuable for non-traditional tech c…

If hardware becomes the moat, the US frontier labs are screwed. We have AWS, Azure, GCP. All three have or are making inference silicon. LLMs become just another service in the public cloud's large service catalog, and open weight wins. Which makes sense to me. Selling a chatbot interface/model access to the general public was never going to be a viable long term play. You still need developers to wrap the models int…

The funniest thing would be if in a couple years LLMs just end up being another checkbox next to PostgreSQL and Kubernetes
Post reply on HN