Earlier quoted context omitted.
[flagged]
Ha, I don't know how to take that? I assure you I wrote it myself.
We are formed by environment we live in.
Comprehension debt: A ticking time bomb of LLM-generated code
151–160 of 352 posts
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#152Earlier quoted context omitted.
It's easy to avoid overly complex solutions with LLMs. First, your prompts should be direct enough to the LLM doesn't wander around producing complexity for no reason. Second, you should add rules/learning/context to always solve problems in the simplest way possible. Lastly, after generation, you can prompt the LLM to reduce the complexity of the solution.
Okay how about this situation that one of my junior devs hit recently: Coding in an obj oriented language in an enormous code base (big tech). Junior dev is making a new class and they start it off with LLM generation. LLM adds in three separate abstract classes to the inheritance structure, for a total of seven inherited classes. Each of these inherited classes ultimately comes with several required classes that are…
[deleted]
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#153[deleted]
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#154This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…
> I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up. I have some anecdotal evidence that suggests that we can accomplish far more value-add on software projects when completely away from the computer and any related technology. It's amazing how fast the code goes when you know exactly what…
Another aspect is test cases constrains the domain of possible correct codes by a lot ( a randomly picked number will never solve a quadratic equation, by having many such quadratic equations [test cases] simulatenously, we are imposing lot of constraints on the solution space). Let's say I want LLM to write a regex but by having it run on test cases, I can gain confidence. This is the thesis of Simon Willison. Once LLMs continuously learn about what a code "means" in a tight REPL internal loop, it will start to gain better understanding.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#155Earlier quoted context omitted.
It's easy to avoid overly complex solutions with LLMs. First, your prompts should be direct enough to the LLM doesn't wander around producing complexity for no reason. Second, you should add rules/learning/context to always solve problems in the simplest way possible. Lastly, after generation, you can prompt the LLM to reduce the complexity of the solution.
Okay how about this situation that one of my junior devs hit recently: Coding in an obj oriented language in an enormous code base (big tech). Junior dev is making a new class and they start it off with LLM generation. LLM adds in three separate abstract classes to the inheritance structure, for a total of seven inherited classes. Each of these inherited classes ultimately comes with several required classes that are…
[deleted]
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#156Was listening to the Dwarkesh Patel podcast recently and the guest (Agustin Lebron) [0] mentioned the book "A Deepness In The Sky" by Vernor Vinge [1].
I started reading it and a key plot point is that there is a computer system that is thousands of years old. One of the main characters has "cold sleeped" for so long that he's the only one who knows some of the hidden backdoors. That legacy knowledge is then used to great effect.
Highly recommend it for a great fictional use of institutional knowledge on a legacy codebase (and a great story overall).
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#157Earlier quoted context omitted.
It's easy to avoid overly complex solutions with LLMs. First, your prompts should be direct enough to the LLM doesn't wander around producing complexity for no reason. Second, you should add rules/learning/context to always solve problems in the simplest way possible. Lastly, after generation, you can prompt the LLM to reduce the complexity of the solution.
Okay how about this situation that one of my junior devs hit recently: Coding in an obj oriented language in an enormous code base (big tech). Junior dev is making a new class and they start it off with LLM generation. LLM adds in three separate abstract classes to the inheritance structure, for a total of seven inherited classes. Each of these inherited classes ultimately comes with several required classes that are…
True True,
I remember another example, with Linus Torvalds, who at a conference used a trivial example of simplifying functions, as to why hes good at what he does, or what makes a good lead developer in general.
It went something along the lines of.
"Well we have this starting function which clearly can solve the task at hand. Its something 99 developers would be happy with, but I can't help but see that if we just reformulate it into a do-while instead we now can omit the checks here and here, almost cutting it in half."
Now obviously it doesn't suffice as real-world example but, when scaled up, is a great view at what waste can accumulate at the macro level. I would say the ability to do this is tied to a survival instinct, one which, undoubtedly will be touted as something that'll be put in the 'next-iteration' of the model. Its not strictly something I think that can be trained to be achievable though, as in pattern matching, but its clearly not achievable yet as in your example from above.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#158This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…
I think this is part of the reason why I've had a bit more success with AI Coding than some of my colleagues. My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle. I've found even as plenty of thought leaders talk about this general approach (rapid prototyping, c…
> to rapidly build a crappy version of something so that I could better understand it, then rework it
I do this, too. And it makes me awful at generating "preliminary LOEs", because I can't tell how long something will take until I get in there and experiment a little.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#159Our collective future has "mediocrity" written all over it.
And when you think about it, LLMs are pretty much, by design, machines that look for truth in mediocrity in its etymological sense.
Re: Comprehension debt: A ticking time bomb of LLM-generated code
#160This was a pre-existing problem, even if reliance on LLMs is making it worse. Naur ( https://gwern.net/doc/cs/algorithm/1985-naur.pdf ) called it "theory building": > The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modi…
LLMs have made it better for us. The quality of code committed by the junior developers has improved.