Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tao on “blue team” vs. “red team” LLMs

mathstodon.xyz

Post reply on HN
Print view View original

171–179 of 179 posts

Re: Tao on “blue team” vs. “red team” LLMs

#171

Post by HappMacDonald »

jrockway wrote: post #85
andrepd wrote: post #59

Earlier quoted context omitted.

I don't understand this. How does it slow your development if the tests being green is a necessary condition for the code being correct? Yes it slows it compared to just writing incorrect code lol, but that's not the point.

The goal of tests is not to prevent you from changing the behavior of your application. The goal is to preserve important behaviors. If you can't tell if a test is there to preserve existing happenstance behavior, or if it's there to preserve an important behavior, you're slowed way down. Every red test when you add a new feature is a blocker. If the tests are red because you broke something important, great. You sav…

This example smells a lot like "overfit" in AI training as well.

Re: Tao on “blue team” vs. “red team” LLMs

#172

Post by nyrikki »

MoreQARespect wrote: post #170
nyrikki wrote: post #130

Earlier quoted context omitted.

It isn't coupling low level implementation details, it is writing tests based on input and output of the unit under test. The expected output from a unit, given an input is not an implementation detail, unless you have some very different definition of implementation detail than I. Testing the unit under test produces the expected outputs from a set of inputs implies nothing about implementation details at all. It is…

If the "unit under test" is low level then thats coupling low level implementation details to the test. If you're vague about what constitutes a "unit" that means youre probably not thinking about this problem.

Often, even outside of software, unit testing means testing a component's (unit‘s) external behavior.

If you don't accept that concept I can see how TDD and testing in general would be challenging.

I general it is most productive when building competency with a new subject to accept the author's definitions, then adjust once you have experience.

IMHO, the sizing of components is context, language, and team dependent. But it really doesn't matter TDD is just as much about helping with other problems like action bias, and is only one part of a comprehensive testing strategy.

While how you choose to define a 'unit' will impact outcomes, TDD it self isn't dependent on a firm definition.

Re: Tao on “blue team” vs. “red team” LLMs

#173

Post by MoreQARespect »

nyrikki wrote: post #172
MoreQARespect wrote: post #170

Earlier quoted context omitted.

If the "unit under test" is low level then thats coupling low level implementation details to the test. If you're vague about what constitutes a "unit" that means youre probably not thinking about this problem.

Often, even outside of software, unit testing means testing a component's (unit‘s) external behavior. If you don't accept that concept I can see how TDD and testing in general would be challenging. I general it is most productive when building competency with a new subject to accept the author's definitions, then adjust once you have experience. IMHO, the sizing of components is context, language, and team dependent.…

>If you don't accept that concept

Nobody anywhere in the world disputes that unit tests should surround a unit.

>IMHO, the sizing of components is context, language, and team dependent. But it really doesn't matter

Yeah, thats the attitude that will trip you up.

The process you use to determine the borders which you will couple to your test - i.e what constitutes a "unit" to be tested is critically important and nonobvious.

Re: Tao on “blue team” vs. “red team” LLMs

#174

Post by thaumasiotes »

strbean wrote: post #104
groestl wrote: post #100

Earlier quoted context omitted.

> a long-standing bug was accidentally fixed In some cases (e.g. in our case) long standing bugs become part of the API that customers rely on.

It's nearly guaranteed, even if it is just because customers had to work around the bug in such a way that their flow now breaks when the bug is gone. Obligatory: https://xkcd.com/1172/

That comic doesn't show someone working around a bug in such a way that their flow breaks when the bug is gone. It shows them incorporating a bug into their workflow for purposes of causing the bug to occur.

It isn't actually possible for fixing a bug to break a workaround. The point of a workaround is that you're not doing the thing that's broken; when that thing is fixed, your flow won't be affected, because you weren't doing it anyway.

Re: Tao on “blue team” vs. “red team” LLMs

#175

Post by johnisgood »

otabdeveloper4 wrote: post #161
johnisgood wrote: post #78

Earlier quoted context omitted.

Yeah, they may do that, but people really should read the code an LLM produces. Ugh, makes me furious. No wonder LLMs have a bad rep from such users.

> people really should read the code an LLM produces Yeah, but that, like, requires that you know how to code. And wasn't the point of LLMs in the first place to let clueless people make software?

I do not know, I would hope not. The bar to entry is already too low. I do not think you will ever be able to get an LLM work flawlessly for people who do not know programming. I know how to code, and I used LLMs before. It seems to be a prerequisite to know how to code if I want useful outputs.

Re: Tao on “blue team” vs. “red team” LLMs

#176

Post by marcosdumay »

baq wrote: post #124
marcosdumay wrote: post #75

Earlier quoted context omitted.

> So I wouldn't blame people for writing specs that only match the code implementation at that time. WFT are you doing writing specs based on implementation? If you already have the implementation, what are you using the specs for? Or, if you want to apply this direct to tests, if you are already assuming the program is correct, what are you trying to test? Are you talking about rewriting applications?

Where do you work if you don’t need to reverse engineer an existing implementation? Have you written everything yourself?

Unless you are rewriting the application, you shouldn't assume that whatever behavior you find on the system is the correct one.

Even more because if you are looking into it, it's probably because it's wrong.

Re: Tao on “blue team” vs. “red team” LLMs

#177

Post by dllthomas »

thaumasiotes wrote: post #174
strbean wrote: post #104

Earlier quoted context omitted.

It's nearly guaranteed, even if it is just because customers had to work around the bug in such a way that their flow now breaks when the bug is gone. Obligatory: https://xkcd.com/1172/

That comic doesn't show someone working around a bug in such a way that their flow breaks when the bug is gone. It shows them incorporating a bug into their workflow for purposes of causing the bug to occur. It isn't actually possible for fixing a bug to break a workaround. The point of a workaround is that you're not doing the thing that's broken; when that thing is fixed, your flow won't be affected, because you we…

> It isn't actually possible for fixing a bug to break a workaround.

That's not true. For instance, if there's a bug in formatting, that might be worked around by handing the unintended formatting. But now you're (maybe) not handling the intended formatting, and a fix would break you.

Re: Tao on “blue team” vs. “red team” LLMs

#178

Post by auggierose »

cubefox wrote: post #160
auggierose wrote: post #159

Earlier quoted context omitted.

You could argue that Tao is the most representative LLM user possible, because why would you need not very intelligent people use LLMs? Just replace them with LLMs.

I assume you wouldn't want to be replaced by an LLM.

For the paid job I am currently doing, I wouldn't mind being replaced by an LLM at all. Just give me the money, no strings attached.

I mean, THAT IS WHAT TECHNOLOGY IS SUPPOSED TO BE THERE FOR. Nobody wants a job, everybody just wants to live their lives.

Re: Tao on “blue team” vs. “red team” LLMs

#179

Post by strbean »

thaumasiotes wrote: post #174
strbean wrote: post #104

Earlier quoted context omitted.

It's nearly guaranteed, even if it is just because customers had to work around the bug in such a way that their flow now breaks when the bug is gone. Obligatory: https://xkcd.com/1172/

That comic doesn't show someone working around a bug in such a way that their flow breaks when the bug is gone. It shows them incorporating a bug into their workflow for purposes of causing the bug to occur. It isn't actually possible for fixing a bug to break a workaround. The point of a workaround is that you're not doing the thing that's broken; when that thing is fixed, your flow won't be affected, because you we…

It might be as simple as a property name containing a typo, and the workaround being to expect that incorrectly spelled property name. When the typo is fixed, the code that was expecting the misspelled property breaks.

Maybe it's a bad workaround, but your users will almost certainly implement bad workarounds.

Post reply on HN