121–130 of 287 posts
LLMs can explain the code they generate if you just ask - they never run out of patience. You can ask how it could be made faster, then ask why it did those specific things.
AI lets those with initiative shine so bright they show everyone else how it’s done.
> If you're a skilled, experienced engineer and you fear that AI will make you unemployable, adopt a more nuanced view. LLMs can't replace human engineering. I don't think LLMs were ever meant to completely replace human engineering, at least in the way we think of engineering as a craft. But the truth is the world is changing, and with LLMs/AI, the goalposts are changing, too. With massive economies of scale and hug…
I agree with your main point, and almost as if to nitpick: more and more output (specifically in the context of LLM-produced code) is paradoxical to economies of scale, precisely because LLMs make too many mistakes at scale. So then we necessarily have to limit their use to relatively simple and mundane tasks. That is, of course, still valuable in itself, but history teaches us that there _will_ be a mass movement to…
Yes, that's exactly right, it's the prisoner's dilemma. You articulated it perfectly.
Is AI perhaps an opportunity for years of paid cleanup and re-design work that those of us who have dedicated decades to learning the craft of software engineering will be able to cash in on?
If AI creates a mess, why won’t future AI be able to continue building a mess on top? And if it does hit a dead end, just regenerate a new version of the entire system in minutes. I don’t know what’s going to happen with AI coding, but it often seems to me that people are making fundamental errors when framing the problem. “But how will humans maintain AI code?” Is one such example. Why would we expect one part (code…
Because due to how complexity works, you reach a state where the expected number of breakages from modifying the code base exceeds the value from the change itself. Even if you assume the cost of labor is 0. It’s like the monkeys writing Shakespeare.
> And if it does hit a dead end, just regenerate a new version of the entire system in minutes.
If this worked it would have been done every few years at big companies. In reality, prototypes cannot take the place of a legacy system because it’s deeply integrated and depended upon. Most notably through the data it owns, but in many subtle ways as well. Simply verifying that a new system will not break is a massive undertaking. Writing meaningfully testable systems is an order of magnitude harder than implementing them.
When there’s a monetary risk of bugs (lose data, lose users, mess up core business logic etc) companies pay for the confidence to get things right. Doesn’t mean it always works, or that their priorities are right, but a payment vendor is not going to trust vibes to do a db migration.
There are still many experimental prototype domains out there, like indie games and static web sites, where throw away and start over is pretty much fine. But that’s not the entire field.
> Only humans can decrease or resist complexity.
For a simple program, maintenance is naturally entropy-increasing: you add an `if` statement for an edge case, and the total number of paths/states of your program increases, which increases entropy.
But in very large codebases, it's more fluid, and I think LLMs have the potentially to massively _reduce_ the complexity by recommending places where state or logic should be decoupled into a separate package (for example, calling a similar method in multiple places in the codebase). This is something that can be difficult to do "as a human" unless you happen to have worked in those packages recently and are cognizant of the pattern.
You know, sometimes I feel that all this discourse about AI for coding reflects the difference between software engineers and data scientists / machine learning engineers. Both often work with unclear requirements, and sometimes may face floating bugs which are hard to fix, but in most cases, SWE create software that is expected to always behave in a certain way. It is reproducible, can pass tests, and the tooling is…
In gpt2 times I used to read gpt generated text a lot, I was working on a game to guess if the text is AI generated or not, and for weeks while I was working on it I had strange dreams. It went away when I stopped consuming tokens, in gpt4 age this does not happen as I am reading hundreds of times more tokens than back then, but I think it is just more subtle.
Now I use AI to generate thousands of lines of code per day, at minimum sometimes now I just blank out when the AI doesnt spit out the code fast enough, I dont know what am I supposed to write, which libraries it is using what is the goal of this whole function etc, as it is not my code, it is foreign and I honestly dont want to be reading it at all.
This week I took the whole week off work and am just coding without AI and in few days the "blank out" is gone. Well, I did use AI to read 300 page docs of st7796s and write barebones spi driver for example, but I treat it almost as an external library, I give it the docs and example driver and it just works, but it is somewhat external to my thought process.
People argue that all fields have evolved, e.g. there are no more blacksmiths, but I argue that the machinists now are much more sophisticated than the ones in the past, pneumatic hammers allow them to work better and faster, as they use the hammer they get better understanding the material they work with, as in the machine does not take away their experience and ability to learn. I always had 2 days per week where I code without any AI, but now I think I have to change the way I code with it.
AI is for sure making me worse, and lazy. And I am not talking about the "career" here, I am talking about my ability to think.
I wrote few days ago about it: https://punkx.org/jackdoe/misery.html
> LLMs as they currently exist cannot master a theory, design, or mental construct because they don't remember beyond their context window. Only humans can can gain and retain program theory. False. > An LLM is a token predictor. It works only at the level of text. It is not capable of working at a conceptual level: it doesn't reason about ideas, diagrams, or requirements specifications. False. Anyone who have spent…
The problem with people "who have spent time in machine learning or reinforcement learning" is that they've spent no time, literally none, understanding what a concept is. There is no such thing as a higher dimensional concept, nor can they be projected into a weight space, because they aren't quantities. The concept, say, "Dog" composes with the concept, "Happy" to form "Happy Dog". The extension(Dog) is all possibl…
In practical terms, what do you think the LLM output cannot contain right now? Because the way I read it now is "LLM can't speculate". But that's trivial to disprove by asking for that happy dog on Mars speculation you have as an example - whether you want the scientific version, or child level fun, it's available and the model will give nontrivial idea connections that I could not find anywhere. (For example childlike speculation from Claude included that maybe dogs would be ok playing in spacesuits since some dogs like wearing little coats)
Similarly "And the intensions here do not compose like the extensions." is really high level. What's the actual claim with regards to LLMs?
> How often have you witnessed an LLM reduce the complexity of a piece of code? I would add that AI often makes far too clever code as well, and would defer to Kernighan's law: “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”
I think you could make similar arguments about mapping technology like Google and Apple Maps -- that using them decreases people's skills in navigating the physical world, atrophying our sense of direction and geography. And actually, that's not wrong. People really do often struggle to navigate these days if they don't have the crutch of something like Google Maps. It really has changed our relationship to the physi…
The problem is that mapping software is reliable and doesn't spit out a result of what is essentially a random number generator. You can rely on its output, the same way you can rely on a calculator. Not always, mind you, because mapping the entire globe is a massively complex task with countless caveats and edge cases, but compared to LLM output? Even with a temperature setting of 0 with the same prompt regenerated multiple times, you'll be getting vastly different output.
Also, since LLMs cover a much more broad swathe of concepts, people are going to be using these instead of their brains in a lot of situations where they really shouldn't. Even with maps, there are people out there that will drive into a lake because Google Maps told them that's where the street was, I can't even fathom the type of shit that's going to happen from people blindly trusting LLM output and supplanting all their thinking with LLM usage.
> it doesn't reason about ideas, diagrams, or requirements specifications. (...) How often have you witnessed an LLM reduce the complexity of a piece of code? > Only humans can decrease or resist complexity. It's funny how often there's a genuine concept behind posts like these, but then lots of specific claims are plainly false. This is trivial to do: ask for simpler code. I'm using that quite often to get a second…