[flagged]
Zuckerberg says AI agent development going slower than expected
241–250 of 661 posts
Re: Zuckerberg says AI agent development going slower than expected
#242Having agents is like going from walking to having a bicycle. Business executives look at this and think "at this rate of progress we'll have self-driving cars in a few years!" and start making serious plans for that world. In reality I think we're going to be riding bikes for a long time. That situation of increased individual contributor productivity makes engineers more valuable , and increases the utility of engi…
Nobody knows if we are going to "just" be riding bikes for a long time. To give time for society to adapt I hope it's the case, but we really have no idea.
It looks pretty clear LLMs don't get us there by themselves, no amount of duct tape, WD-40, etc. gets us past, say, the mathematical certainty of hallucinations.
I mean, we don't know it any more than we don't know someone won't come out with cold fusion tomorrow, but it's a fundamental breakthrough away from where we're at. This isn't some routine engineering project with a guarantee of completion if you're just willing to keep pouring the billions. That's playing the lotto, you can pour away and get flat nothing.
The only difference is they're pouring billions and praying a rabbit comes out of the hat, but it's actually not much reason to expect they're going to pull the cold-fusion level rabbit out of their hat they'd need to get us past bikes.
Re: Zuckerberg says AI agent development going slower than expected
#243[flagged]
You are saying $1.5B of AI spend would not have shown up in either of these numbers?
Please do tell us how that works, I'm sure it's fascinating and would love to know more.
Re: Zuckerberg says AI agent development going slower than expected
#244> that executives had miscalculated on the timing of the changes
Hmmm... so who is going to be thrown under the bus?
Re: Zuckerberg says AI agent development going slower than expected
#245Earlier quoted context omitted.
In my limited testing Fable is far better at obeying CLAUDE.MD than Opus is.
From what I can tell, the "established wisdom" is to get Fable to plan and Opus to implement (for cost purposes). The problem there is that Opus could ignore whatever it likes from Fable's plan.
i've yet to see a case where opus "ignores whatever it likes".
opus will definitely ignore instructions if you give it contradictory instructions, or a plan that has steps that obviously don't work with each other. but if you give it a coherent plan, it will follow it.
Re: Zuckerberg says AI agent development going slower than expected
#246I was worried this time last year that by this time this year, companies would have slashed their engineering teams down to a handful and everything would be driven by mostly autonomous agents with human guidance. But it just hasn't happened. Do I write all my code with an agent now? Yes. Can you just give an agent a desired outcome and let it work, unsupervised? Absolutely not. I can produce more code than I used to…
> Can you just give an agent a desired outcome and let it work, unsupervised? Absolutely not. Ignoring instructions - whether in AGENTS.md or my prompt - is the worst of it, and it routinely happens. It just waives things that I explicitly told it to do as part of the design. Vibe coders (in the true sense, zero oversight) claim that you just need to prompt it carefully. That's completely untrue when faced with your…
The short leash method is the way to avoid this.
Re: Zuckerberg says AI agent development going slower than expected
#247Earlier quoted context omitted.
Your context isn’t to give it orders, they just don’t work like that. Your context (AGENTS.me, skills, per-request context we are sending in for each request to bots) is to give it the info it needs in the language category it’s trained for the answers you want; you have to give it a clear instruction each prompt. Basically, when you have a long session, you can see this by saying, ok, now moving onto another thing,…
> Basically, when you have a long session, you can see this by saying, ok, now moving onto another thing, blah blah blah I try to avoid > 200k contexts, as the 1M context is where I first saw the massive decrease in reliability. And my AGENTS is really short, and I said it was ignoring decisions in the prompt.
Whenever I work on a challenging question I worry about this, because Opus will easily think for 200k tokens on the first prompt. I fear any follow up discussion is lobotomised!
Re: Zuckerberg says AI agent development going slower than expected
#248AI hype comes quickly, and goes with the wind quickly as well.
Re: Zuckerberg says AI agent development going slower than expected
#249Earlier quoted context omitted.
you might have to think the way through though and these companies are already being caught up with the huge token costs at the same time. There was an interesting comment during the cloudflare layoffs (partially driven by the fact that the company was bleeding money also because of its token costs from one estimate being 5* million$ per month (I feel so silly that I accidentally had written/meant 500 and had kentonv…
> (partially driven by the fact that the company was bleeding money also because of its token costs from one estimate being 500 million$ per month, don't quote me on that though) Cloudflare had 5000 employees (pre-layoff), so you are suggesting that every single one of them (eng, HR, legal, finance, receptionists) was using $100k tokens per month (that's $1.2M annualized, per employee), for a total of 3x gross revenu…
Sorry kentonv! , I apologize :-(
I had mistakenly written 500 million when it was around 5 million dollars so I messed up its 5 million per month[See Source], not 500 million. I wish to have a genuine discussion while you are here though because i can be wrong, I usually am and I would love to have a good faith discussion, thanks in advance!
I will try to back up a lot of it with hackernews comments from the thread when cloudflare layoffs were suggested so that I don't accidentally mis-represent anything and My suggestion wasn't a critique of cloudflare and please don't take it as such. The question was simply of the AI token costs associated.
and this was the comment that I was referencing to[0] which states the following:
> There was an recent article on X with an interesting take - it could be that companies are doing layoffs not because AI is making them more productive but because it hasn't. Their costs have gone up paying for expensive AI but haven't seen any revenue benefits to offset it.
An child comment of it talks about the coinbase layoffs which had happened around the same time[1]:
> (..) In 2023, their "Technology and Development" line item shows $1.32bn going out, and by 2025 it'd ballooned to $1.67bn. This is despite headcount actually contracting by almost a thousand people between those two statements.
Regarding this: > Let's imagine that this isn't absurd on its face. If true, then you'd expect Cloudflare's Q1 earnings to show a massive, massive net loss. In fact Cloudflare was cash flow positive in Q1.
We might be forgetting that (from my understanding, Cloudflare has never had profits) (positive annual net income) with an astronomically large P/E ratio.
There was a comment which I had read which talks about this in more detail (https://news.ycombinator.com/item?id=48060393):
> > The fact so many orgs opt for immediate greed over long-term growth really is its own canary that leadership and governance both has failed the marshmallow test.
> Why do you think it's greed? The company's stock is down and they just missed expectations on their last earnings report (unheard of in big tech in the last 2 years).
> It seems more like a traditional layoff scenario
Another comment [from the Layoff thread][2] which might summarize some things:
"Their AI costs have increased 600% but this hasn't translated into actual revenue. Also they are probably projecting AI costs to keep growing. They've done the math and at some point it is going to affect their bottom line. Reducing or limiting AI usage would be inconceivable given Cloudflare itself has invested on AI and is selling AI services. Instead they've opted for reducing about 20% of their head count."
I genuinely wish if we can have a good faith discussion about it. I appreciate cloudflare as a product myself and actively use cf tunnels, which is why I care about it as well and I wish to have a good faith discussion about it hopefully as well :-D
> The rest of your post is more qualitative, so harder to disprove, but from what I can tell, it seems equally made up.
I can be wrong, I usually am and if I am wrong, I wish to learn from it and I wish to improve as a person too!
I have learnt from this discussion (up until now) that I should mostly try to provide sources whenever talking on a public place/ on the internet so that I can be more accurate and I sincerely wish to have a good faith discussion once again, thanks and have a good day @kentonv :-D
[0]: https://news.ycombinator.com/item?id=48055149
[1]: https://news.ycombinator.com/item?id=48055413
[2]:https://news.ycombinator.com/item?id=48056124
[Source]: https://lowendtalk.com/post/quote/217055/Comment_4789235
Re: Zuckerberg says AI agent development going slower than expected
#250Earlier quoted context omitted.
> Can you just give an agent a desired outcome and let it work, unsupervised? Absolutely not. Ignoring instructions - whether in AGENTS.md or my prompt - is the worst of it, and it routinely happens. It just waives things that I explicitly told it to do as part of the design. Vibe coders (in the true sense, zero oversight) claim that you just need to prompt it carefully. That's completely untrue when faced with your…
I’m convinced the magic bullet is deterministic checks. Linters, static analyzers, etc. Whatever you can do to create deterministic gates that the LLM simply must overcome to reach a “done” state, do it. Has been making a huge difference for my team, but sister teams are so invested in writing the perfect Make No Mistakes prompt that they just can’t see it. Basically I treat it like a junior dev. We don’t get junior…
It will burn up the tokens to get through the deterministic gates, more so when n order dependencies are involved in the mix. Enough typewritters and monkeys could get it done too.