Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Costs More Than the Engineer

tomtunguz.com

Post reply on HN
Print view View original

51–60 of 128 posts

Re: When AI Costs More Than the Engineer

#51

Post by victorbjorklund »

scrollaway wrote: post #9
avaer wrote: post #3

Earlier quoted context omitted.

I don't know, compute is compute. Arguably making complex software with LLMs isn't all that different from training a model to do a thing. You're throwing a lot of compute at the problem and hoping for a stochastic solution. The distinction will become even blurrier with time. Though I agree it might be informative to split it by industry sector.

If you’re going to include AI training in costs, you should include education as part of the costs of an engineer …

And why only education? Everything the engineered needed so far should be included. Can’t have a dev that never eaten since they were born.

Re: When AI Costs More Than the Engineer

#52

Post by croisillon »

psychoslave wrote: post #6
geon wrote: post #2

Garbage. You can't include training by the companies that develop an llm in the comparison against companies that merely use the same llm. Apples and potatoes.

Apples and potatoes are both something people will need to eat if we want to see it from the human utility perspective, and they both require some land space to be allocated for their culture (though one can of course conjugate both culture). If you want to take the DDG LLM summary at fate value, apples are lower in calories and sugar but higher in fiber compared to potatoes, which are richer in vitamins and minerals…

the saying "comparing x and y" implies that you compare something that one of them can't compete ; if people praise the softness of the skin first and foremost, comparing apples and potatoes won't lead interesting results

Re: When AI Costs More Than the Engineer

#53

Post by Certhas »

NitpickLawyer wrote: post #30
ricardobeat wrote: post #24

Open-weight models are going to completely shatter these forecasts. It takes a little more effort – right now, probably won’t be true in three months – but you can achieve the same at 1/10th of the cost.

> but you can achieve the same at 1/10th of the cost. For some tasks, sure. But not for all tasks. And for some tasks, cost per token is irrelevant if it provides real benefits that are oom compared to what you had. Local models are indeed becoming "good enough" for some tasks, but there are still tasks that they can't touch. There's a recent benchmark for kernel writing. Fable wrote a kernel that provides ~30% more…

The question is: what proportion of tasks can not be handled by GLM5.2?

How many software developers were working on code like the one you describe?

Re: When AI Costs More Than the Engineer

#54

Post by toyg »

brown_munda wrote: post #15

Mr. Mark Zuckerberg is particularly not happy about these stats. He was promised something else and he has already fired like half of the company. It is really crazy people didn't think this through.

The layoffs are irrelevant to the discourse. It's typically considered by management to be good, for mature companies, to periodically fire as many employees as they can sustain without visibly impacting operations, and then re-hire cheaper workers only where strictly necessary. This allows them to keep costs down, reduce risks of excessive worker entrenchment, and overcome the drawbacks of contingent hiring-sprees.

Excuses for these exercises will vary, AI is just the latest; but it's fundamentally just a labor-containment/efficiency-seeking strategy.

Re: When AI Costs More Than the Engineer

#55

Post by sevenzero »

MarcellusDrum wrote: post #49
sevenzero wrote: post #32

Earlier quoted context omitted.

I think its a fallacy to believe people like Zuckerberg or any other stupidly rich person aren't extremely calculative about this. I am very sure they have surrounded themselves by top tier engineers making very informed decisions while their top tier marketing teams make very calculated decisions on how its expressed to the public. The public generally is NOT in favor of AI outside of tech circles so it makes sense…

Are you really saying that Mark Zuckerberg, CEO of "Meta", can't make a massive miscalculation?

I am saying that none of this is just an "oopsie", a miscalculation, maybe, but definitely not as unexpected as it seems.

Re: When AI Costs More Than the Engineer

#56

Post by tryagainian »

littlecranky67 wrote: post #50
seasox wrote: post #33

Earlier quoted context omitted.

VC mostly, since Anthropic is not profitable.

I wonder if they ever will be. If the chinese open source models are only 3-6 months behind every major frontier model release, I can't see the business model. GLM-5.2 is supposedly on par to Opus depending on the case. And everybody and their mother can run that model in their datacenter and charge Dollars for tokens.

[dead]

Re: When AI Costs More Than the Engineer

#57

Post by gnfargbl »

NitpickLawyer wrote: post #30
ricardobeat wrote: post #24

Open-weight models are going to completely shatter these forecasts. It takes a little more effort – right now, probably won’t be true in three months – but you can achieve the same at 1/10th of the cost.

> but you can achieve the same at 1/10th of the cost. For some tasks, sure. But not for all tasks. And for some tasks, cost per token is irrelevant if it provides real benefits that are oom compared to what you had. Local models are indeed becoming "good enough" for some tasks, but there are still tasks that they can't touch. There's a recent benchmark for kernel writing. Fable wrote a kernel that provides ~30% more…

You're looking at the status quo and ignoring the trajectory. The best current open models are about as good as closed models from ~1.5 generations ago. The rate of improvement of all models is converging to zero. It follows that in a few generations, open models inferencing will be about as good as closed model inferencing.

The problem is going to become that there's no incentive for anyone to run the stupidly-expensive training phase. May God have mercy on the stock market.

Re: When AI Costs More Than the Engineer

#58

Post by glimshe »

ricardobeat wrote: post #43
NitpickLawyer wrote: post #30

Earlier quoted context omitted.

> but you can achieve the same at 1/10th of the cost. For some tasks, sure. But not for all tasks. And for some tasks, cost per token is irrelevant if it provides real benefits that are oom compared to what you had. Local models are indeed becoming "good enough" for some tasks, but there are still tasks that they can't touch. There's a recent benchmark for kernel writing. Fable wrote a kernel that provides ~30% more…

That’s true, there will always be demand for ultra-intelligent assistants, especially if they surpass what humans can achieve at similar cost. For the other 90%, the average frontier model will be good enough.

I don't disagree with you, but it's important to pay attention to where the money is. Cheap non frontier models is something that Anthropic and open AI could do too, but who's willing to pay a premium for using them? It will be like competing to sell rice, lots of demand at Rock bottom margins.

Re: When AI Costs More Than the Engineer

#59

Post by InsideOutSanta »

iLoveOncall wrote: post #7
geon wrote: post #2

Garbage. You can't include training by the companies that develop an llm in the comparison against companies that merely use the same llm. Apples and potatoes.

OpenAI and Anthropic aren't charities, so whatever cost they inccur for training will be passed down to the companies using the models. So you absolute should include it.

The problem is how it's framed:

Anthropic spends [...] about $2m of compute per employee per year against a likely all-in comp of $500k+.

The rest of the software market trails. The top 1% of companies spend $89k per engineer per year on AI

This framing makes no sense. The reason Anthropic spends so much on compute per employee is that they are building models. Anthropic employees aren't opening Claude Code and spending $2m in inference every year, so comparing it to other software companies, where AI expense is mostly inference, is completely incoherent.

Yes, the cost has to be passed down eventually, but it's not passed down to one company; it's passed down to all of Anthropic's customers, so the actual share of that money will be distributed among Anthropic's clients.

Look, I 100% agree with the idea that OpenAI and Anthropic are both unsustainable companies that have dug themselves so far into a debt hole that, most likely, the only way they'll be rescued is with government intervention, but this is still a terrible article.

Re: When AI Costs More Than the Engineer

#60

Post by tryagainian »

mazurnification wrote: post #45
niyikiza wrote: post #39

Earlier quoted context omitted.

I guess they should include tuition cost as well.

In a way in US it is - _IF_ ppl were rational economic agents and free market allocation worked student loans should reflect on the wages too.

[dead]
Post reply on HN