Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You can't unit test for taste

dev.karltryggvason.com

Post reply on HN
Print view View original

101–110 of 157 posts

Re: You can't unit test for taste

#101

Post by kalli »

dirkc wrote: post #99

> So with my friend Claude I set about building After this line all the references becomes *we*. I can't help but be a little disturbed by that > To begin with we downloaded ... For instance we excluded ... We also selected ... We used this as a notoriety ... I am increasingly concerned about how LLMs are anthropomorphizing and how that affects our judgement?

This was a conscious choice, addressed in a footnote in the blog post:

> This is my first time writing up a project that I worked on using an AI agent. I kept writing “we” because the project felt like a collaboration.[...] On reading it back, saying we feels like an accountability dodge, because of course I’m fully and solely responsible for any errors in this write-up or code. But just using I/me also feels dishonest, because so much of the implementation here isn’t fully mine so I feel like I’m taking too much credit for my collaboration with the machines. I figure this is a new kind of pronouns debate we’ll be having for the foreseeable future.

I think it is an interesting topic.

Re: You can't unit test for taste

#102

Post by pjmlp »

MoreQARespect wrote: post #88
pjmlp wrote: post #84

Earlier quoted context omitted.

TDD for UI effects?

snapshot test driven development again. i already wrote a similar answer in response to your other comment. it follows the definition of TDD and it works really well (with some caveats) but again some people get hung up on what their impression of TDD is (e.g. unit tests checking to see if a car object has a steering wheel or whatever...) rather than what it actually is and what about it is that actually works.

How does snapshot do "feels right" from designer point of view?

Re: You can't unit test for taste

#103

Post by pjmlp »

MoreQARespect wrote: post #86
pjmlp wrote: post #81

Earlier quoted context omitted.

Example for HLSL graphical glitch?

https://hitchdev.com/hitchstory/approach/snapshot-test-drive... set up a rendering profile and preconditions that generates a minimal snippet of images/video using a predefined GPU profile. then test for either a pixel perfect reproduction of the correct behaviour or for the properties you're looking for (if it doesnt reproduce deterministically). this is one way. i also subscribe to the view that if the type system…

Now do that interactive with feed back from design team and user testing.

Re: You can't unit test for taste

#104

Post by pjmlp »

e12e wrote: post #100
pjmlp wrote: post #82

Earlier quoted context omitted.

Now do a games engine with that approach regarding shaders and the desired visuals.

I'm not familiar with shaders and game engines, so I'm not sure what you are saying. I had a quick look at godot tests, and seems to me they cover some parts of the shaders? Anyway, I was more wondering who/how people are dogmatic about TDD, and manage to leave out one out of three core concepts from red/green/refactor ?

If I cannot write shader code without broken tests, there is a bunch of yak shaving to make testing possible in first place, and that only covers a small subset of graphics pipeline features.

It also takes zero consideration for the interactive nature of games/graphics development.

Re: You can't unit test for taste

#105

Post by dirkc »

kalli wrote: post #101
dirkc wrote: post #99

> So with my friend Claude I set about building After this line all the references becomes *we*. I can't help but be a little disturbed by that > To begin with we downloaded ... For instance we excluded ... We also selected ... We used this as a notoriety ... I am increasingly concerned about how LLMs are anthropomorphizing and how that affects our judgement?

This was a conscious choice, addressed in a footnote in the blog post: > This is my first time writing up a project that I worked on using an AI agent. I kept writing “we” because the project felt like a collaboration.[...] On reading it back, saying we feels like an accountability dodge, because of course I’m fully and solely responsible for any errors in this write-up or code. But just using I/me also feels dishone…

Thanks for pointing out the footnote, I did not get that far. And like you say, I agree it's interesting.

The footnote however does re-enforce my concern - in what other ways do we alter our behavior when it feels like we're interacting with another human?

Re: You can't unit test for taste

#106

Post by joshka »

trjordan wrote: post #4

You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…

Pattern language sites / books have existed for years.

The right approach is more work out what shared patterns are, make sure a bunch of reasonable ones are post trained into the models so that it's easy to refer to them by name (e.g. "tim pope / chris beams style commit messages", or "make invalid state unrepresentable") and then you're in a world where you can define your personal tasted through labels rather than repetition of the core arguments.

Re: You can't unit test for taste

#107

Post by kalli »

dirkc wrote: post #105
kalli wrote: post #101

Earlier quoted context omitted.

This was a conscious choice, addressed in a footnote in the blog post: > This is my first time writing up a project that I worked on using an AI agent. I kept writing “we” because the project felt like a collaboration.[...] On reading it back, saying we feels like an accountability dodge, because of course I’m fully and solely responsible for any errors in this write-up or code. But just using I/me also feels dishone…

Thanks for pointing out the footnote, I did not get that far. And like you say, I agree it's interesting. The footnote however does re-enforce my concern - in what other ways do we alter our behavior when it feels like we're interacting with another human?

That's fair, we can disagree. I don't think I'm personally anthropomorphising llms (I think my mental model of how they work is rough but fairly accurate), but at a population level it might be something to be concerned about (see all the ai-psychosis talk)

What I was getting at with the "we" in the post is more how we talk and think about work like this. I think it is different in kind to previous projects I've done where a relied on google, stack overflow and elbow grease. Programming has always been "standing on the shoulders of giants" kind of work, but doing it with agents feels different from that. Maybe it was a poor stylistic choice, but I think we need a way to talk about it in an honest way.

Re: You can't unit test for taste

#108

Post by MoreQARespect »

pjmlp wrote: post #102
MoreQARespect wrote: post #88

Earlier quoted context omitted.

snapshot test driven development again. i already wrote a similar answer in response to your other comment. it follows the definition of TDD and it works really well (with some caveats) but again some people get hung up on what their impression of TDD is (e.g. unit tests checking to see if a car object has a steering wheel or whatever...) rather than what it actually is and what about it is that actually works.

How does snapshot do "feels right" from designer point of view?

Um, show the snapshot to a designer? When it feels right, lock in the snapshot ("green") and then move on to refactor.

Or, probably more likely a group of snapshots.

Re: You can't unit test for taste

#109

Post by MoreQARespect »

pjmlp wrote: post #103
MoreQARespect wrote: post #86

Earlier quoted context omitted.

https://hitchdev.com/hitchstory/approach/snapshot-test-drive... set up a rendering profile and preconditions that generates a minimal snippet of images/video using a predefined GPU profile. then test for either a pixel perfect reproduction of the correct behaviour or for the properties you're looking for (if it doesnt reproduce deterministically). this is one way. i also subscribe to the view that if the type system…

Now do that interactive with feed back from design team and user testing.

Iterate on the design til the snapshots look the way the design team wants.

That's just an extended red where you get feedback from elsewhere.

Post reply on HN