AI-first – We're just 6 months away from AGI

Hmm, interesting.

One of the problems programmers have is loading a problem into working memory. It can take an hour. An interruption, a phone call, or a meeting can mean that you have to start over (or, if not completely over, you still have to redo part of it). This is a standard programmer complaint about interruptions.

It's interesting that LLMs may have a similar issue.

I gave copilot a screenshot of my work schedule 3 times. The first time it extracted the data flawlessly and organized it the way I asked. The second time it did everything wrong until I talked it though the process in tiny steps. The 3rd time I've asked it to repeat the same steps for the new schedule and it got it wrong just like the initial second attempt. I then tried to talk it though the process step by step and it got everything wrong!? With each step it messed up things it got right the step before. It eventually asked me to upload the image again.

I suppose I could upload the image to a table to json website and provide copilot with the json but the point was to make things easier. In my mind there is nothing complicated about the structure of a table but if I ask copilot to merely extract the text from a row starting with some text it goes insane. Optical character recognition is an idea from 1870 and they had working implementations 50 years ago. I read a bunch of comments online about models getting progressively worse but my experience was over the span of a few weeks.

Would they be doctoring with the quality to make newer models look even better?

> The differences can't be that massive given that the hype already made many of these promises well before these models were ever a thing.

I'm not talking about other people's incorrect promises _and_ I mentioned a number of things in which proper usage today is different from what people were doing before.

> Basically you are just adding to it with "You just need to use the latest model, anything before that is trash".

That's not what I said. Don't put words into my mouth. I said that the older models are trash in comparison, not that they are trash. The older models require more work in prompting to get decent results.

> Ignoring that the same was said just a few months prior when those models were cutting edge.

You are conveniently ignoring the other parts I mentioned to make my claim. Here: "In addition to that we've also seen that the way you prompt (amount of use of expert language), what context you provide through instructions, and tool use make a huge difference on the outcome."

> I'm not talking about other people's incorrect promises _and_ I mentioned a number of things in which proper usage today is different from what people were doing before.

Alright? What you replied to and the context of this entire thread is about promises that have been made for a while now. In fact, we are approaching the point where we can safely talk about years of hype now. For reference, I am using the gpt-4 release as a significant ramp-up in the hype around LLMs.

> That's not what I said. Don't put words into my mouth.

That might not have been your intention. But, again, given the context of where you are replying it does read like. Even with the qualifier of "in comparison".

Like it or not, your comment does add extra qualifiers to the list.

Look, I am not saying that there isn't any progress being made here. I also agree that LLMs can be useful tools as part of a developer toolkit. What I personally don't agree with is that they can do the same job, even less so in real world scenarios. Even the latest models, including Gemini 2.5 and 03 struggle with moderately complex code base. And yes, the argument is always to let them work on small isolated bits of code. Or that if your requirements are tight enough they produce very good code. Which is entirely true, but I also envy the developers who work in structured environments where their code base is that clean and requirements that well-defined.

So, in my experience, coding with these models still sucks. Using them as interactive rubber duckies, replacements for some of the things I used to spend hours googling, debugging small snippets of code, etc. Sure, there they are very useful tools to me. But, to me, that is not coding with LLMs. That is having LLMs available as a tool whenever I need them.

Right, so now we have moved to "well yeah, but you need to use LLM agents". Do you truly not see how you are actually continuing the trend of shifting the goalpost every time someone is critical about the use of LLMs? Critical about parts that just a few short months ago had the same promise you now moved to the use of agents?

Not to mention that with each iteration the amount of tokens needed goes up substantially. Having worked with LLM APIs and their pricing there simply is no way that Cursor is breaking even on that $20 or $40 per month if everyone uses it fully. Not even close. This very much hints at the costs being hidden right now, subsidized if you will, by VC money.

Also, once you have brought junior developers up to speed and guided them they are now slightly more capable developers who can more easily on board on future projects. With LLMs you need to effectively babysit them on each project again.

And there are a lot more caveats, prerequisites and moving targets involved that make the promise and reality for many people and companies not something they can actually be met.

And again, I am not discounting that there are specific areas where people see benefits from using LLMs in agentic from. But those areas are not as ubiquitous as the hype train leads us to believe. And to start using them you need to set up a lot more in the way of infrastructure and due process as well.