181–190 of 229 posts
Earlier quoted context omitted.
That’s why you’ve gotta test your tests. Insert bugs and ensure they fail. As the sibling comments alluded to, it’s not exclusively an AI problem since multiple people can miss the issue too. It’s wonderful that AI is an impetus for so many people to finally learn proper engineering principles though!
but who will test the tests of tests?
I don't know about all this AI stuff. How are LLMs going to stay on top of new design concepts, new languages, really anything new? Can LLMs be trained to operate "fluently" with regards to a genuinely new concept? I think LLMs are good for writing certain types of "bad code", i.e. if you're learning a new language or trying to quickly create a prototype. However to me it seems like a security risk to try to write "g…
Earlier quoted context omitted.
Modified Condition/Decision Coverage (MC/DC) is a test coverage approach that considers a chunk of code covered if: - Every branch was "visited". Plain coverage already ensures that. I would actually advocate for 100% branch coverage before 100% line coverage. - Every part (condition) of a branch clause has taken all possible values. If you have if(enabled && limit > 0), MC/DC requires you to test with enabled, !enab…
Limit I hope you don't have any string inputs, or your test is gonna take a while to run!
But yes, exhaustively testing your code is a bit exhausting ;)
The expertise in software engineering typical in these promptfondling companies shine through this blog post. Surely they know 100% code coverage is not a magical bullet because the code flow and the behavior can differ depending on the input. Just because you found a few examples which happen to hit every line of code you didn't hit every possible combination. You are living in a fool's paradise which is not a surpr…
Wouldn't a better title be "How we're forcing AI to write good code (because it's normally not that good in general, which is crazy, given how many resources it's sucking, that we need to add an extra layer on top of it and use it to get anything decent)"
Wouldn't a better title be "How we're forcing AI to write good code (because it's normally not that good in general, which is crazy, given how many resources it's sucking, that we need to add an extra layer on top of it and use it to get anything decent)"
Gentleman, the dog writes poetry and music, but it is boring, mediocre quality. Overhyped dog.
This is hallucination. Or maybe a sales pitch. If production bugs and the requirement to retain a workable code base don’t get us to write “good” code, then nothing will. And at the current state of the art, “AI” will tend to make it worse.
The first sentence is problematic: > For decades, we’ve all known what “good code” looks like. When relatively trivial concerns such as the ideal length of methods haven't achieved consensus, I doubt there can be any broadly accepted standard for software quality. There are plenty of metrics such as test coverage, but anyone with experience could tell you how easy it is to game those and that enforcing arbitrary stan…
Earlier quoted context omitted.
You do realise they can search the web? They can read documentation and api specs?
They can't think though. They can't be creative.
Of course, some of those predictions may also turn out to be true. But either way, we have abundant empirical evidence that the reasoning is not sound.