581–590 of 699 posts
Earlier quoted context omitted.
Plot twist: this universe (planet) was created in order to reverse engineer what the prompt of the previous one was.
It's not that much of a twist, given that it was basically the plot of THGTTG.
Earlier quoted context omitted.
> Your code is unambiguously better than any LLM code if you can comment a link to the stackoverflow post you copied it from. This is not a truism. "My" code might come from an LLM and that's fine if I can be reasonably confident it works. I might try to gain that confidence by testing the code and reading it to understand what it's doing. It is also true of blog post code, regardless of how I refer to the code; if I…
Believing you know how it works and why it works is not the same as that actually being the case. If the code has no author (in that it's been plagiarised by a statistical process that introduces errors), there's nowhere to go if you realise "oops, I didn't understand that as well as I had thought!".
That's also true if I author the code myself; I can't go to anyone for help with it, so if it doesn't work then I have to figure out why.
> Believing you know how it works and why it works is not the same as that actually being the case.
My series of accidental successes producing working code is honestly starting to seem like real skill and experience at this point. Not sure what else you'd call it.
Earlier quoted context omitted.
As someone else mentioned, the process is async. But I achieve a similar effect by requiring my team to review their own PRs before they expect a senior developer to review them and approve for merging. That solves some of the problem with people thinking it's okay to fire off a huge AI slop PR and make it the reviewer's responsibility to see how much the LLM hallucinated. No, you have to look at yourself first, beca…
> requiring my team to review their own PRs before they expect a senior developer to review them I'm having a hard time imagining the alternative. Do junior developers not take any pride in their work? I want to be sure my code works before I submit it for review. It's embarrassing to me if it fails basic requirements. And as a reviewer, what I want to see more than anything is how the developer assessed that their c…
Earlier quoted context omitted.
You're not wrong, however the issue is that it's not always easy to detect if a PR includes proof that the change works. It requires that the reviewer interrupts what they're doing, switch context completely and look at the PR. If you consider that reviewer bandwidth is very limited in most projects AND that the volume of low-effort-AI-assisted PR has grown incredibly over the past year, now we have a spam problem. S…
I don't think we're talking about merely "AI-assisted" PRs here. We're talking about PRs where the submitter has not read the code, doesn't understand it, and can't be bothered to describe what they did and why. As a reviewer with limited bandwidth, I really don't see why I should spend any effort on those.
IME, "AI" PRs are categorically that kind of PR. I find, and others around me in my org have agreed, that if you actually do all that you describe, the actual net time savings of AI are often (for a mid-level dev or above) either net 0 or negative.
I personally have used the phrase "baptized the AI out of it" describing my own PRs... Where I may have initially used AI to generate a bunch of the code, looked at it and went "huh neat that actually looks pretty right, this is almost done." Then I generate unit tests. Then I fix the unit tests to not be shit. Then i find bugs in the AI-generated code. Then upon pondering the code a bit, or maybe while fixing the bugs, I find the abstractions it created are clunky, so I refactor it a bit... and by the time I'm done there's not a lot of AI left in the PR, it's all me.
Earlier quoted context omitted.
There are always unknown unknowns which a rigorous testing implementation would just hide under the rug (until they become visible on live, that is). > They had 9000+ tests. They were most probably also written by AI, there's no other (human) way. The way I see it we're putting turtles upon turtles hoping that everything will stick together, somehow.
No, those 9,000 tests are part of a legendary test suite built by real humans over the course of more than a decade: https://github.com/html5lib/html5lib-tests
It looks like the code doesn't always check whether expected errors in the testsuite match the returned errors - which is rather important to ensure one isn't just incidentally getting the expected output.
So while JustHTML looks sort of right, it'll actually do things like emit errors on perfectly valid html.
Plus, the test suite isn't actually comprehensive, so if one only writes code to pass the tests, it can fail in the real world where other parsers that actually wrote against the spec wouldn't have trouble.
For instance, the html5lib-tests only tests a small number of meta charsets and as a result, JustHTML can't handle a whole slew of valid HTML5 character encodings like windows-1250 or koi8-r - which parsers like html5lib will happily handle. There's even a unit test added by the AI that ensures koi8-r doesn't work, for some reason.
Earlier quoted context omitted.
There are always unknown unknowns which a rigorous testing implementation would just hide under the rug (until they become visible on live, that is). > They had 9000+ tests. They were most probably also written by AI, there's no other (human) way. The way I see it we're putting turtles upon turtles hoping that everything will stick together, somehow.
> They were most probably also written by AI, there's no other (human) way. Yes. They came from the existing project being ported, which was also AI-written.
Those human tests are why your browser properly renders diversely messy HTML.
Earlier quoted context omitted.
It's not a new phenomenon. Time was, people would copy-paste from blog posts with the same effect.
Always the same old tiring "this has always been possible before in some remotely similar fashion hence we should not criticise anything ever again" argument. You could intuitively think it's just a difference of degree, but it's more akin to a difference of kind. Same for a nuke vs a spear, both are weapons, no one argues they're similar enough that we can treat them the same way
At the end of the day we're not performing war by poking other people with long sticks and we're not getting the word out by sending out a carrier pigeon.
Methods and medium matters.
I know this won't be popular, however, I think the idea of differentiating a "real developer" from one who relies mostly, or even solely on an LLM is coming to an end. Right now, I fully agree relying wholly upon an LLM and failing to test it is very irresponsible. LLMs do make mistakes. They do a sloppy job at times. But give it a year. Two years. five years. It seems unreasonable to assume they will hit a plateau t…
Earlier quoted context omitted.
> added random blank lines and deleted others from code that was not even in the file they were supposed to be working in. Maybe some kind of auto-formatter?
That is my charitable interpretation, but it's always one or two changes across a module that has hundreds, maybe thousands of lines of code. I'd expect an auto-formatter to be more obvious. In any case, just looking over your own PR briefly before submitting it catches these quickly. The lack of attention to detail is the part I find more frustrating than the actual unnecessary format changes.
Earlier quoted context omitted.
You know, this is kind of a funny take at some level. Like, for any surgery, you want the doctor who has done the same operation 10 times, not the one who has 10 years of "many hat doctoring" experience. I'm not really arguing anything here, but it is interesting that we value breadth over (hopefully) depth/mastery of a specific thing in regards to what we view as "Senior" in software.
“This person is incurious” would be more apt but also more likely to apply to everyone else in the room too. Didn’t Bruce Lee famously say he fears the man who’s authored one API in ten thousand different contexts?
As analogy, I don't think you'd treat "using API XYZ 10,000 times" the same as "serving an ace in tennis, 10,000" times.