Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Pullback Has Officially Started

planetearthandbeyond.co

Post reply on HN
Print view View original

11–20 of 66 posts

Re: AI Pullback Has Officially Started

#11

Post by CuriouslyC »

FranzFerdiNaN wrote: post #10
CuriouslyC wrote: post #3

People roll out a complex and powerful technology without understanding the technology fully, what evals are or updating process to account for the tech, and the rollout fails, news at 11. Seriously though, "AI fucks up" is a known thing (as is humans fuck up!) and the people who are using the tech successfully account for that and build guardrails into their systems. Use version control, build automated tests (e2e/s…

If the tech only worked for coding it would be one thing. But it’s advertised as a cure for anything and everything and so people are using it for that. And you can’t build automated tests for that.

I am a big AI booster but I agree that using agents for tasks unsupervised without either rigorous oversight or strong automated constraints is a mistake.

Re: AI Pullback Has Officially Started

#12

Post by meander_water »

There's a few bits of information from the original sources that's left out:

- The METR paper surveyed just 16 developers to arrive at their conclusion. Not sure how that got past review. [0]

- The finding from the MIT report can also be viewed from a glass 5% full perspective:

> Just 5% of integrated AI pilots are extracting millions in value. > Winning startups build systems that learn from feedback (66% of executives want this), retain context (63% demand this), and customize deeply to specific workflows. They start at workflow edges with significant customization, then scale into core processes. [1]

[0] https://arxiv.org/abs/2507.09089

[1] https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...

Re: AI Pullback Has Officially Started

#13

Post by rhetocj23 »

mewpmewp2 wrote: post #4

I'm not saying AI is living up to the "hype" or "expectations" - it would largely depend on how you quantify the hype or expectations. Most rational would be to consider how much money is funneled into vs how much ROI would it have within some time range in the future, e.g. 10 years. A wise investor would look ahead 10 years, balance benefits, potential and risks. By that metric it could be too early to say if it's p…

Time is ticking and running out. You do realise future numbers have to be discounted heavily for the required rate of return given the risk, right?

This means the free cash flow to the firm OAI generates will have to be huge, given the negative cash outflows to date.

Re: AI Pullback Has Officially Started

#14

Post by rhetocj23 »

CuriouslyC wrote: post #3

People roll out a complex and powerful technology without understanding the technology fully, what evals are or updating process to account for the tech, and the rollout fails, news at 11. Seriously though, "AI fucks up" is a known thing (as is humans fuck up!) and the people who are using the tech successfully account for that and build guardrails into their systems. Use version control, build automated tests (e2e/s…

Imagine comparing a human fuck up to an AI one. Lol.

Re: AI Pullback Has Officially Started

#15

Post by Madmallard »

moomoo11 wrote: post #7

AI (LLM) is useful for coding and I use it to lookup various articles or websites and summarize. Use it where it works.. ignore the agents hype and other bullshit peddled by 19yo dropouts. Unlike the 19yo dropouts of the 2010s these guys have brain rot and I don’t trust them after having talked to such people at start up events and getting their black pill takes. They have products that don’t work and lie about numbe…

On this website on another thread there is a principal software engineer at Microsoft who wrote an essay on how agent systems are amplifying all of the employees productivity massively even on large complex tasks.

Re: AI Pullback Has Officially Started

#16

Post by rhetocj23 »

itsafarqueue wrote: post #2

AI creates the most spectacular happy path demos. It’s hard not to extrapolate to infinity when you see it.

People have a bias to want to believe something works in all cases, when it seemingly offers benefits to them. Especially when there’s a sunk investment involved.

Re: AI Pullback Has Officially Started

#17

Post by killingtime74 »

mewpmewp2 wrote: post #4

I'm not saying AI is living up to the "hype" or "expectations" - it would largely depend on how you quantify the hype or expectations. Most rational would be to consider how much money is funneled into vs how much ROI would it have within some time range in the future, e.g. 10 years. A wise investor would look ahead 10 years, balance benefits, potential and risks. By that metric it could be too early to say if it's p…

What if switching to 3 ply toilet paper (from the dreaded 1 ply) in employee bathrooms increased productivity in 5% of companies. We could also apply the same logic that these 5% could also produce 50x returns.

Re: AI Pullback Has Officially Started

#18

Post by chaboud »

I’ve been using AI coding systems for quite some time, and have worked in neural networks since the 90’s. The advancements are, frankly, almost as crazy as 90’s neural net devotees like me were claiming could be possible in the eventual future.

That said, the non-tech-executive/product-management take on AI has often been an utter failure to recognize key differences between problems and systems. I spend an inordinate amount of time framing questions in terms of promises to customers, completeness, reproducibility, and contextual complexity.

However, for someone in my role, building and ideating in innovation programs, the power of LLM assisted coding is hard to pass up. It may only get things 50% of the way there before collapsing into a spiral of sloppy overwrought code, but we often only need 30-40% fidelity to exercise an idea. Ideation is a great space for vibe coding. However, one enormous risk in these approaches is in overpromising the undeliverable. If folks don’t keep a sharp eye on the nature of the promises they’re making, they may be in for a pretty wild ride; with the last “20%” of the program taking more than 90% of the calendar time due to compression of the first “80%” and complication of the remainder.

We’re going to need to adjust. These tools are here to stay, but they’re far from taking over the whole show.

Re: AI Pullback Has Officially Started

#19

Post by FranzFerdiNaN »

Madmallard wrote: post #15
moomoo11 wrote: post #7

AI (LLM) is useful for coding and I use it to lookup various articles or websites and summarize. Use it where it works.. ignore the agents hype and other bullshit peddled by 19yo dropouts. Unlike the 19yo dropouts of the 2010s these guys have brain rot and I don’t trust them after having talked to such people at start up events and getting their black pill takes. They have products that don’t work and lie about numbe…

On this website on another thread there is a principal software engineer at Microsoft who wrote an essay on how agent systems are amplifying all of the employees productivity massively even on large complex tasks.

Now the question is whether that’s true (and thus should be objectively measurable) or if he is bullshitting because Microsoft invested so much money in it it just has to work.

Re: AI Pullback Has Officially Started

#20

Post by falcor84 »

killingtime74 wrote: post #17
mewpmewp2 wrote: post #4

I'm not saying AI is living up to the "hype" or "expectations" - it would largely depend on how you quantify the hype or expectations. Most rational would be to consider how much money is funneled into vs how much ROI would it have within some time range in the future, e.g. 10 years. A wise investor would look ahead 10 years, balance benefits, potential and risks. By that metric it could be too early to say if it's p…

What if switching to 3 ply toilet paper (from the dreaded 1 ply) in employee bathrooms increased productivity in 5% of companies. We could also apply the same logic that these 5% could also produce 50x returns.

I'm not sure what your point is, but I do absolutely think that getting proper toilet paper offers 50x returns and everyone should prioritize it.
Post reply on HN