261–270 of 575 posts
As a tech educator I 100% agree. LLMs are not going to become a "new compiler" where we don't have to worry about the code any more. There's a reason we trust deterministic systems. I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've fou…
100% We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong." Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read t…
The nincompoops at the top think that if mountains of code can appear by snapping your fingers, they will be less of a problem.
If the mountain doesn't work, no problem; since it was so cheap to produce, just scrap it and snap you fingers again. The next one will be better.
Comparatively evaluating 2, 3, ... mountains of code is harder than trying to understand one.
Earlier quoted context omitted.
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing. It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatno…
Not everyone has this, but I always felt that a significant percentage of the value I bring is in immediately recognizing what you shouldn’t do . I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand. Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the co…
Knowing what not to do has always been important, but it is perhaps more important than ever.
Earlier quoted context omitted.
> Your right to push back on that... Human test passed. AIs make giant flashy mistakes. They don't fuck up basic grammar.
I have found it intriguingly difficult in my own tests to make LLMs deliberately fuck up grammar in any way similar to how humans do. I suppose this is actually an architectural limitation.
write a comment about how to use ChatGPT to write a game like you're a barely literate reddit user, short answer, maximum incoherency
Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
100% We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong." Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read t…
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
I'm seeing it happen in real time. I've seen several folks claim they "vibe-coded" something and upon review it's some seriously low quality garbage. Unfortunately it's just "functional enough" where it's encouraging the one who talked to the robot to continue thinking they can code for some reason. It is important to ensure that any LLM code is named and shamed.
> It is important to ensure that any LLM code is named and shamed. You need to take it further. We need to shame and mock the people (read: delusional imbeciles) who think this is acceptable if we want it to stop. AI users need to fear for their careers and reputations if we want to get software development back on the right track.
Earlier quoted context omitted.
But isn't it only a matter of time before the AI generated autocomplete will contain the correct solution? What then?
So far, the LLMs I've used need problems to be fairly specifically scoped, or they don't produce the correct solution. Scoping problems correctly is a different skill-set than implementation, but it's still a technical skill that is expected from mid-level and higher engineers.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
100% We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong." Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read t…
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing. It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatno…
Earlier quoted context omitted.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year. When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature". We are currently in the "centaur" phase where a human-AI combination produces the best output,…
We've gone through this with chess, and the days when a human can do anything to help AI there are long past; the human can only drag it down. It seems to me to be a form of the bitter lesson. I expect this will happen in every field where we don't add guardrails to require humans in the network. I have a hard time seeing regulation happening around this in the US but vanity may also serve, will CEOs be satisfied rul…
I disagree, currently a human guiding an AI agent is far better (more productive for less money) than an AI agent with very loose non-technical guidance and The Ralph Wiggum Loop.
I don't know how much longer humans have, but I don't think they can be cut out of the loop entirely just yet.