Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AI Costs More Than the Engineer

tomtunguz.com

Post reply on HN
Print view View original

81–90 of 128 posts

Re: When AI Costs More Than the Engineer

#81

Post by Tade0 »

gnfargbl wrote: post #29

Working regularly with AI is like managing a small team of unbelievably knowledgeable, very smart, and occasionally crashingly naïve junior developers. Because they're so knowledgeable and smart, they can get a lot done very quickly. Because they make a proportion of howling errors, you have to keep a close eye on them -- or carefully train another agent to do it for you, in which case you now have to keep a close ey…

I get the feeling that either I'm using LLMs wrong, or everyone else is.

Outside of enthusiastic use of Tab and some one-off scripts, I don't really tell it to write code. Instead I ask vague questions about the codebase and its inner workings.

Reading other people's code has always been my Achilles' heel - particularly if it's a huge project and has a lot of undocumented conventions. LLMs are brilliant at explaining this sort of stuff.

Re: When AI Costs More Than the Engineer

#82

Post by ThunderSizzle »

mazurnification wrote: post #45
niyikiza wrote: post #39

Earlier quoted context omitted.

I guess they should include tuition cost as well.

In a way in US it is - _IF_ ppl were rational economic agents and free market allocation worked student loans should reflect on the wages too.

Luckily we're not, because then your tip to a waiter/waitress would be dependent on their student loans remaining, especially considering how many expensive liberal arts majors struggle to find a sufficient career.

Re: When AI Costs More Than the Engineer

#83

Post by ThunderSizzle »

eru wrote: post #38
alexjurkiewicz wrote: post #8

Earlier quoted context omitted.

AI training uses wildly more compute than most companies, who are generally building domain specific CRUD apps. Compare AI costs per-engineer-salary-dollar, because more expensive engineers probably need more expensive AI.

> Compare AI costs per-engineer-salary-dollar, because more expensive engineers probably need more expensive AI. Let's see how this works out in the long run. For a historical analog, more expensive engineers don't use more expensive computers (by and large).

Which is probably a backwards anti-pattern companies have built.

Your most expensive engineer's time is most valuable, so if you give them standard issue which is half the speed, you are throttling the value you can get from your engineer. Not to mention the mental drain of your cursor barely being able to move due to all the bloated virtual networking systemization.

It would seem to make sense to give more valuable employees faster equipment, so that their time isn't spent toiling with the slow machine, but rather actually producing value.

Re: When AI Costs More Than the Engineer

#84

Post by NitpickLawyer »

gnfargbl wrote: post #57
NitpickLawyer wrote: post #30

Earlier quoted context omitted.

> but you can achieve the same at 1/10th of the cost. For some tasks, sure. But not for all tasks. And for some tasks, cost per token is irrelevant if it provides real benefits that are oom compared to what you had. Local models are indeed becoming "good enough" for some tasks, but there are still tasks that they can't touch. There's a recent benchmark for kernel writing. Fable wrote a kernel that provides ~30% more…

You're looking at the status quo and ignoring the trajectory. The best current open models are about as good as closed models from ~1.5 generations ago. The rate of improvement of all models is converging to zero. It follows that in a few generations, open models inferencing will be about as good as closed model inferencing. The problem is going to become that there's no incentive for anyone to run the stupidly-expen…

> The rate of improvement of all models is converging to zero.

That's so obviously not true that I don't even think it's worth the energy to even debate it. It's been said for years, yet here we are, constantly improving. People really don't get RL / the bitter lesson, do they?

> It follows that in a few generations, open models inferencing will be about as good as closed model inferencing.

Not a chance. There's hundreds of billions of dollars on one side, and oom less on the other. There's also scaling laws and information theory. No matter how good, a 30B model will not be able to be better than a 3T+ model, all things being equal.

You are mistaking models becoming "good enough" for an increasingly number of tasks, which I agree is happening, with SotA models stagnating, hitting walls etc. That will not happen for many many years to come.

Re: When AI Costs More Than the Engineer

#85

Post by NitpickLawyer »

Certhas wrote: post #53
NitpickLawyer wrote: post #30

Earlier quoted context omitted.

> but you can achieve the same at 1/10th of the cost. For some tasks, sure. But not for all tasks. And for some tasks, cost per token is irrelevant if it provides real benefits that are oom compared to what you had. Local models are indeed becoming "good enough" for some tasks, but there are still tasks that they can't touch. There's a recent benchmark for kernel writing. Fable wrote a kernel that provides ~30% more…

The question is: what proportion of tasks can not be handled by GLM5.2? How many software developers were working on code like the one you describe?

On that aspect, I agree. Smaller / open models are becoming "good enough" at an increasing number of tasks. And that's great for us, consumers. But there will always be tasks that are "worth" pursuing with better models, and cost is irrelevant for those tasks. That was the point I was trying to make.

Re: When AI Costs More Than the Engineer

#86

Post by danaris »

jonatron wrote: post #62
geon wrote: post #2

Garbage. You can't include training by the companies that develop an llm in the comparison against companies that merely use the same llm. Apples and potatoes.

Apples and oranges, or chalk and cheese. Why would you say apples and potatoes?

I read it as trying to indicate that it's even more different than apples and oranges.

Not sure it succeeds in that, but I think that's the intent.

Re: When AI Costs More Than the Engineer

#87

Post by vrganj »

yurishimo wrote: post #77
HaphazardGuess wrote: post #66

A.I suffers from the last-mile problem. It can do 90% of the work in 20 minutes but then the remaining 10% ends up taking 20 million hours to actually finish. It frustrating to the point that I sometimes want to throw the whole thing out and start from scratch.

Some people would argue that this is the best way to use AI as it exists today. Generate a POC and then if that POC makes sense, then rebuild it from scratch by hand. Maybe you can still use AI for a bit of boilerplate generation, but you should write all of the business logic and verify it by hand. Personally, I'm starting to lean more and more towards this approach. Though, I have to admit, for a well defined bug t…

I find AI useful for boilerplate stuff, very generic code like mappers etc.

For more complex stuff, I find that the best workflow is usually treating AI like a kind of stupid, but very motivated intern you're pair programming with. Nothing unsupervised and you might have to touch up/do manually the really critical parts, but it can help with a lot of the bitchwork.

Re: When AI Costs More Than the Engineer

#88

Post by vrganj »

brown_munda wrote: post #15

Mr. Mark Zuckerberg is particularly not happy about these stats. He was promised something else and he has already fired like half of the company. It is really crazy people didn't think this through.

The thing AI has been the most useful at is showing without a doubt that the emperor does in fact not have any clothes.

It's exposed the incompetence, hubris and sheer out-of-touchness of the tech leadership caste, open for everyone to see.

They're not smarter than you, they don't have any great strategic insights. They're just rich kids that happened to be at the right place at the right time and now have a cadre of sycophants blowing smoke up their ass.

Re: When AI Costs More Than the Engineer

#89

Post by fragmede »

ricardobeat wrote: post #24

Open-weight models are going to completely shatter these forecasts. It takes a little more effort – right now, probably won’t be true in three months – but you can achieve the same at 1/10th of the cost.

And 10x the headache. Money can be exchanged for goods and services, and people pay money to not have to deal with things. If you don't have the money for it, you pay for it in dealing-with-bullshit credits.

Re: When AI Costs More Than the Engineer

#90

Post by pydry »

sevenzero wrote: post #32
brown_munda wrote: post #15

Mr. Mark Zuckerberg is particularly not happy about these stats. He was promised something else and he has already fired like half of the company. It is really crazy people didn't think this through.

I think its a fallacy to believe people like Zuckerberg or any other stupidly rich person aren't extremely calculative about this. I am very sure they have surrounded themselves by top tier engineers making very informed decisions while their top tier marketing teams make very calculated decisions on how its expressed to the public. The public generally is NOT in favor of AI outside of tech circles so it makes sense…

At that level it becomes hard not to surround yourself with obsequious yes men who will instinctively agree with all your harebrained ideas about virtual reality or AI.
Post reply on HN