You can't unit test for taste • Hacker News Board

Re: You can't unit test for taste

#31

Post by themgt » Thu, Jun 25, 2026, 2:26 PM UTC

timroman wrote: post #19

https://pureinference.com/insights/taste-is-the-new-skill I wrote about this a few months back. Rick Rubin is famous for this. I do think it is something that can be trained though, it just needs a lot more context. Taste builds over time through lots of unit tests, through lots of content writing, through an accumulation of product decisions. It’s hard to put it in the individual spec, but it can be teased out of 10…

This is exactly it - the ultimate skill now is to be Rick Rubin with an LLM. Not a comfortable transition as a coder.

Re: You can't unit test for taste

#32

Post by jayd16 » Thu, Jun 25, 2026, 2:27 PM UTC

chantepierre wrote: post #8

another-dave wrote: post #5

Earlier quoted context omitted.

"The effort length and instantaneous power output changes, of course." but that's what the phrase is meant to convey, right? Don't run through consumable X (energy/money/etc) like there's no tomorrow - even though there's now, we've got dozens more of those that we need to meet, so you're better off getting this one done at 75% than committing 100% to it and failing on all the others.

Yeah you're right, I hear it more like "this is a week long hike, not a sprint" as if a marathon included rest. In any length of racing there's no tomorrow. But I'm doing tongue-in-cheek pedanticness here and will stop that right now !

In a marathon, not sprinting is the rest.

Re: You can't unit test for taste

#33

Post by jpadkins » Thu, Jun 25, 2026, 2:27 PM UTC

I think another important question is can you distill taste? (another comment uses the phrase "externalize", which might mean something similar).

I think people have been trying for the written word, with some degree of success (anti-slop skills). I have been trying for visuals, and it's pretty meh. It's easy to get a multimodal LLM to follow a style guide, but a style guide doesn't capture everything that accounts for taste. And anything that is dynamic (not a screenshot test) seems really hard or really expensive.

Re: You can't unit test for taste

#34

Post by tmoertel » Thu, Jun 25, 2026, 2:29 PM UTC

trjordan wrote: post #4

You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…

> You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can.

I'm not so sure. For instance, you can write down what it means for a program to be free of XSS and other injection vulnerabilities. Now, how would you unit test for that property?

Re: You can't unit test for taste

#35

Post by ChrisMarshallNY » Thu, Jun 25, 2026, 2:30 PM UTC

> but it ended up merely in a supporting role

This has been my experience, as well, but it’s a really big support. It just needs adult supervision. I can’t understand how vibe-coded apps, actually work.

As far as “taste,” goes, I test my stuff constantly, checking for even minor “friction points,” sometimes, refactoring back to design, in order to resolve issues that many folks would ship. I’m pretty anal, and want my work to be the best experience possible.

I can’t see any LLM coming close to being able to evaluate the user experience, like I can.

Re: You can't unit test for taste

#36

Post by pydry » Thu, Jun 25, 2026, 2:31 PM UTC

trjordan wrote: post #4

You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…

I remember reading an interview with a fireman who described a time when his buddy evacuated a team because he "felt" that a floor would collapse imminently.

He couldn't articulate why but they trusted his gut and it did collapse.

A lot of software engineering relies on that kind of intuition and on a good team you can integrate it and benefit from it and avoid all manner of floor collapses.

Re: You can't unit test for taste

#37

Post by bob1029 » Thu, Jun 25, 2026, 2:34 PM UTC

pjmlp wrote: post #27

Exactly one of the reasons I never went down with all the TDD dogma of only writing code to fix broken tests. There is a reason conference talks are always about plain algorithms and data structures.

The biggest flaw I've seen with TDD is the fact that correctness does not compose upward. Every time two units come into contact, you've got an entirely new kind of unit. The tests from constituents do not cover emergent properties of the new things. You will repeat this same exercise the entire way up to the top, and the moment you come into contact with the customer (they want to change everything), the house of cards comes crumbling down and you have to start your agonizingly-slow process all over from the bottom again.

The only thing that the business seems to care about is top-down UI testing. This is also convenient because you can leave it until the very end after the customer has already seen several prototypes.

I do think TDD makes sense in isolated scopes (prove this specific custom parser works at the edges), but as the general policy for the entire product it's definitely not a viable practice. Much of the time if comes off as an ego trip to see just how cleverly we can mock something so that we can say we technically tested it.

Re: You can't unit test for taste

#38

Post by paytonjjones » Thu, Jun 25, 2026, 2:34 PM UTC

ChrisMarshallNY wrote: post #35

> but it ended up merely in a supporting role This has been my experience, as well, but it’s a really big support. It just needs adult supervision. I can’t understand how vibe-coded apps, actually work. As far as “taste,” goes, I test my stuff constantly, checking for even minor “friction points,” sometimes, refactoring back to design, in order to resolve issues that many folks would ship. I’m pretty anal, and want m…

Tools like Playwright and Maestro can already give you a small taste of what that would look like.

But overall I agree, LLMs are currently awful at being beta testers. They miss the most basic stuff that any human would immediately catch as being poor UX, and for all their visual prowess they are terrible at auditing UI.

Re: You can't unit test for taste

#39

Post by paytonjjones » Thu, Jun 25, 2026, 2:38 PM UTC

plastic-enjoyer wrote: post #29

paytonjjones wrote: post #24

Earlier quoted context omitted.

This is, in short, the big current problem with AI. LLMs are built for scale so they've given up on the kind of online learning / "long term memory" processes that would individualize them. The LLM is permanently locked to being a really cracked engineer on their first day at your company, looking at your codebase for the first time. You can scaffold a bit with .md files, but at the moment they lack the ability to do…

> LLMs are built for scale so they've given up on the kind of online learning / "long term memory" processes that would individualize them. I wonder if this is even desirable from a product perspective. You probably don't want online learning in a product that you are selling because you can't guarantee a consistent quality of the product.

You could say the same thing about employees!

And to be fair, the ability to fire employees and hire new ones is pretty important for that reason. In cases where you can't easily fire employees (e.g. unions), you encounter the very problem you're describing, and it often leads to companies preferring more consistent automations.

Re: You can't unit test for taste

#40

Post by punnerud » Thu, Jun 25, 2026, 2:40 PM UTC

trjordan wrote: post #4

You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…

If you have enough examples you can train an AI on your preferences, then use that distilled AI as a unit test. Don’t combine multiple into one AI. If they don’t agree you want it to fail so you can decide and retrain the tests.