To further onto this, humans could technically simulate Turing machines themselves but they don't.
Efficiency and finding the right abstractions and tradeoffs is literally identical with making new capacities and capabilities. If all of your systems were truly "run by AI", we wouldn't have systems. Systems are systems because the AI uses well developed tools and protocols that humans have put together. It's a stable capacity.
Yeah, I think the peak expectation for AI is that the user gets to be the manager of a very smart team. But we've all seen that. If the manager is clueless about the technology, the results are disappointing.
And the team (AI) is really not that smart. Just very fast at pasting and gobling stack overflow and github repos together
The biggest pet peeve is that you never know whether your request is dealt with by Opus and not Sonnet or Haiku or even something else and you still pay for Opus. You have to ask the model some difficult question and gauge which one is doing the work, but often they switch halfway the task when it goes downhill (they might say it is because of context window) but still charge you full price and you have to start the session again (them making even more money). This should be more regulated and externally audited, what exactly you are paying for.
I do wonder if "without my approval" is looser than "discuss only, do not make edits"? I'm not sure how much that finegrain thing matters in prompts nowadays tho
One problem is that agents are notoriously bad at handling ‘me’ vs ‘you’ over an extended conversation. ‘Without my approval’ far back in the context of an extended session gets fuzzy about who ‘me’ is.
This has always seemed strange to me. It will offer a suggestion, I'll make a comment about a suggestion, and it returns with, "Your suggestion..." The further distance from the suggestion, the more likely it confuses who made it.
You said what I was thinking. AI can do the things I always found mind numbing and tedious, but because I did it already early in my career, I know when to intervene because the agent is not doing it correctly. I think that is the OP's point.
I think about LLMs in software today as attempting to provide a similar role as compilers in terms of translating high-level thoughts into low-level details. The only problem is that I actually trust compilers.
I detest this compiler analogy so much: llms are non deterministic by nature, compilers should not and are not
> as an assistant that helps me make decisions, and ensures that I'm in the driver's seat This... quite describes what the current AI feels like to me. So I really don't see a problem. If AI does too much for you, you should, uh, just prompt it to do less. Everyone talks about how their AI escapes sandbox and virtualization or delete their entire repos or something. Perhaps they're using a skynet I'm not aware of. Fr…
I expect there's a probabilistic factor to it, among other things. I use the almost the same addendum as you ("do not make, commit, or push any changes without my approval") and it broadly seems to work in my day-to-day dev. However, I also set up some agents that trigger on incoming requests from my dev teams to automate things that used to be rote/manual tasks. _That_ is where I see the issues, I suspect simply bec…
I haven’t used these tools that permit LLMs to make changes on their own, but I can’t help being baffled if there’s no big red switch that changes the system prompt and disables any and all write access to the outside world.
This post ignores that verifying something works doesn't require you to fully understand it. This is easily observable in products we use everyday. We don't need to understand them to be able to verify they work correctly. The cost of verification is often cheaper than the cost of production.
It present delegation as absence of power/agency. While I think there is some kernel of truth in here, the claim "it's not empowering to hand off the details" quickly falls apart when we consider real examples. If it were true, it would mean any leader who coordinates an organization isn't empowered through delegation. We know that's false. The general claim doesn't hold at all. Somewhere in there is an argument for engineers losing something in the hand off, but it isn't clearly articulated.
This same argument could be applied to anything. Layers of abstraction exist for a reason, because at a certain point, we can only deal with so many things at once. We have to be able to delegate "the details" to others -- be that a person, a company, or an AI model. Does "not getting into the details" mean you have to understand how the GCC compiler works when you write C code? Do you need to be an expert in machine…
Your vouching for centralization of knowledge, and even understanding, which I think is a very bad idea. It’s not like the rest of us are becoming specialized at something else - knowledge, in the broad sense, is the foundation of all skills. Without it we have nothing to contribute
You are free to learn whatever you want to. I'm saying I would greatly prefer talking to an AI model that was trained by an expert or the guy who created it, or trained via an RL environment that guarantees its correctness (where possible), and not take a chance on whatever or whoever happens to be teaching the subject that day. My experiences are my own, but I have repeatedly encountered people time and time again who profess to know how something works or have a deep understanding of something and it turns out that they, in fact, do not. It turns out its very easy to convince people you are knowledgeable about something who don't know anything about the subject.
I've been vibecoding a ton for the past 9 months, built a bunch of cool little apps for myself with AI, ran experiments, built an entire SDLC on skills, did the agent orchestration harness thing, etc. In the past few weeks I've hit a wall where I'm just tired of it. Each model becomes more independent but also harder to direct in detail. They produce massive, tedious, sloppy text outputs with very little input. They'…
Similarly for me. I have a recent example too. I run a bunch of eps32 devices at home, using esphome. Pre-AI I wrote all the yaml myself and understood what’s happening and didn’t have any very bespoke hacks. Recently I asked an agent for a feature where the screen will get dimmer after some time, and eventually after a day the screen will turn off and then turn back on if I tap, which should be pretty easy in esphom…
I've been hit with the team manager trying to clear tickets on the jira board with entirely AI generated code, and what I've observed is they always make the system more complex, usually absurdly complex. The AI code never looks at things from a higher level, or reconsiders the requirements, it just layers on a load of crap until the tests pass.