Live data from Hacker News

TDD is dead. Long live testing.

david.heinemeierhansson.com

101–110 of 166 posts

Re: TDD is dead. Long live testing.

#101
post #49

I've been slowly coming to this realization myself lately, I thought our situation was just outside the mainstream of what most people work on, but maybe not. Our team builds data-intensive biomedical web applications (open source project for it all here: http://harvest.research.chop.edu ). Much of our UI is data-driven, so many of the bugs we encounter are at the intersection of code, config, and unique data circums…

Why moving toward more client-side JS complicates testing further?

Largely because more of the automated integration testing has to be done with a headless browser, e.g. Poltergeist.

If you have no JS in an important area of your site, you can integration test it without a headless browser. Your test process models HTTP requests as simple method calls to the web framework. (E.g. in Rails, you can simulate an HTTP request by sending the appropriate method call to Rack.) The method calls simply return the HTTP response. You can then make assertions against that response, "click links" by sending more method calls based on the links in the response, submit forms in a similar manner, etc.

But if a given part of your app depends on JS, you pretty much have to integration test in a headless browser. Given the state of the tooling, that's just not as convenient as the former approach. Headless browsers tend to be slow as molasses. There are all kinds of weird edge cases, often related to asynchronous stuff. You spend a lot of time debugging tests instead of using tests to find application bugs.

Worst of all, headless browsers still can't truly test that "the user experience is correct." That's because we haven't yet found a way to define correctness. For example, a bug resulting from the interaction of JS and CSS is definitely a bug, and it can utterly break the app. But how do you assert against that? How do define the correct visual state of the UI?

Re: TDD is dead. Long live testing.

#102

I generalized DHH's conclusion re: TDD in my blog post, "XDDs: stay healthily skeptical and don't drink the kool aid": http://www.pixelmonkey.org/2012/02/12/xdds "I list one of my skills as “thought-driven development”. This is a little tongue-in-cheek; software engineering over the last few years has developed a lot of “XDDs,” such as test-driven development, behavior-driven development, model-driven development, et…

Love that post. And I love the acronym - TDD - Thought Driven Development. Now, I can finally start telling everyone that I strictly do TDD. ;)

Re: TDD is dead. Long live testing.

#103
post #72

My biggest problem with TDD is that it tends to take people away from whiteboarding and thinking about architecture and into writing everything as if it were a recursive problem -- write test for base case, code base case, write test for next base case, code next base case, write some significant test case, code general case but often this is completely the wrong pattern for solving a problem. This may be an example…

I've always thought these two things were independent. I use whiteboarding for designing interactions and high-level architecture, then (sometimes) use TDD when I have a relatively good grasp of what a unit is supposed to do. Neither precludes the other.

Re: TDD is dead. Long live testing.

#104
post #98

Earlier quoted context omitted.

Why moving toward more client-side JS complicates testing further?

Because now you're introducing an interface when previously you had shared objects all on the server in a single codebase. For example, let's say my JavaScript client works with a "User" object delivered via REST API from the server. Let's say I change Django's User model object to modify some existing attribute. Previously, all my Python code's tests would now use this updated model object and I could simply run my…

1) See the testing pyramid article posted somewhere within this thread.

2) I never have to maintain "Mock" objects in my tests, my Mock objects came for free (I use Mockito and I have less Java Interface, I mock my real classes and I inject them in the right places).

3) Separating Django tests and JS tests shouldn't be too bad and often preferred.

4) You can test JSON payload from backend to front-end IN the back-end serialization code, speaking of this, I use JAX-RS (and Jackson/JAXB) so JSON payload is something I normally don't test since that means I'm testing the framework that is already well-tested. I normally don't test JSON payload coming from the front-end: it's JavaScript, it's the browser, I don't test an already tested thing.

But I'll give you another example of Object transformation from one form to another: I use Dozer to convert Domain model (Django model, ActiveRecord model) to Data Transfer Object (Plain old Java object). To test this, I write a Unit test that converts it and check the expected values.

5) Nobody argues end-to-end testing :)

Check PhantomJS, CasperJS, Selenium (especially WebDriver) and also Sauce Lab (We use them all). But end-to-end testing is very expensive so hence the testing pyramid.

Re: TDD is dead. Long live testing.

#105

As rhetorical gambits go, the “what other people do is religion” is getting pretty old. It’s meant to reflect on the speaker as the voice of reason among the thoughtless. Who wouldn’t want to think of themselves that way? I’m the smart one who sees through the dogma! But it doesn’t have that effect anymore – mostly it makes the speaker look someone who disrespects his readers.

As rhetorical gambits go, the “what other people do is religion” is getting pretty old.

And a bit rich coming from a high priest. He wrote an entire pretentious rant (Rails is Omakase) when someone questioned his own dogma.

Re: TDD is dead. Long live testing.

#106
I used to be dogmatic about TDD. Then I joined a team where management doesn't value unit testing that much, and so I got a lot less religious about it.

What I am missing now isn't the test-first mentality. I honestly don't think there's a quality advantage to writing the tests before the code. And the danger of ardent TDD was always investing a ton of time ironing out a unit to perfection, only to realize you missed the forest for the trees and the unit, while perfect, has no place in the overall solution.

Rather, I miss the side benefits that come with it:

- Automated tests actually get written.

- It improves development cadence when you're writing a chunk of code you don't really want to be writing (by setting mini milestones).

So now I occasionally practice TDD for those reasons.

Re: TDD is dead. Long live testing.

#107

Earlier quoted context omitted.

Why moving toward more client-side JS complicates testing further?

Largely because more of the automated integration testing has to be done with a headless browser, e.g. Poltergeist. If you have no JS in an important area of your site, you can integration test it without a headless browser. Your test process models HTTP requests as simple method calls to the web framework. (E.g. in Rails, you can simulate an HTTP request by sending the appropriate method call to Rack.) The method ca…

Yes, I've known about the headless proposition for a while.

Splitting front-end and back-end tests is desirable.

> Worst of all, headless browsers still can't truly test that "the user experience is correct."

This is the claim from the old Joel Spolsky article about automation tests but should not be the ultimate dealbreaker.

Nobody claims you should rely on automation-tests 100%. Automation-tests test functionality of your software not the look-n-feel or user-experience. You have a separate tests for that.

The problem between JS and CSS shouldn't be that many either (shouldn't be a factor that, again, becomes a dealbreaker). If you have tons of this then perhaps what's broken is the tools we use? or perhaps how we use it?

I don't test my configurations (in-code configuration, not infrastructure configuration) because configuration is one-time only. You test it manually and forget about it.

Re: TDD is dead. Long live testing.

#108
In a positive way I miss some examples of "hurting my designs" and "what that approach is doing to the integrity of your system design". Why TDD is doing that? Could you show me where do you have problems? That would be a nice piece of feedback to learn.

The post feels like a rant mixed with fallacies and a salt of contempt. I'd expect it from Zed Shaw. I'm sure Zed could do it much better.

TDD is dead and the third word in the post is "fundamentalism". Yes, nobody can say that fundamentalisms in tools are good. Every tool has its use cases. Although it doesn't justify the "TDD is dead" motto. Correct me if I'm wrong but it looks like a "Straw man" fallacy. Half of the post is dedicated to build a straw man of fundamentalism nobody can deny.

The other second half is focused in unit tests. While he talks about "Test-first units", all unit tests (written before or after) have the aforementioned issues. Unless they're not unit . Thus we're introduced to other tests types that are not unit.

@programminggeek in a previous comment talked about this, is the test pyramid (http://martinfowler.com/bliki/TestPyramid.html). In this moment the conversation is outside the TDD scope, but discussing about how good or bad are different kind of tests. Again, nobody can deny the benefits of having different test types and not only unit. IMHO is a another red herring fallacy.

So what did I get from it?

  - Fundamentalisms in tools/paradigms are bad.
  - DHH has some problems in his designs he cannot unit test.
  - Using only unit tests is bad, you need more high level tests.
  - Try capybara.

Re: TDD is dead. Long live testing.

#109

What DHH describes is called the test pyramid ( http://martinfowler.com/bliki/TestPyramid.html ), which despite the fact that DHH seems to dislike Uncle Bob, it is something Uncle Bob has demonstrated and advocated multiple times in very sensible ways. The bigger thing DHH doesn't mention or explain is that Basecamp is designed to be intentionally simple. Much of the complexity that you get in client work where you d…

I wish this post stays at the top of _ANY_ discussion about automation testing. Couple V.E.R.Y important points: 1) Testing Pyramid People, who agree or disagree, talk a lot about unit-testing yet they never talk about the other types of automation testing; as if the rest were forgotten. To make matter worse, people often haven't had the experience of managing full-blown System Test. Hint: it's very brittle, slow, an…

I would prefer this post stays as an example of how not to do a post about the testing pyramid. Martin Fowler post/article is better IMHO, and I'm sure there are much better posts.

There are some good ideas mixed with fallacies and disdain/contempt. These are not good in any discussion.

Re: TDD is dead. Long live testing.

#110

What DHH describes is called the test pyramid ( http://martinfowler.com/bliki/TestPyramid.html ), which despite the fact that DHH seems to dislike Uncle Bob, it is something Uncle Bob has demonstrated and advocated multiple times in very sensible ways. The bigger thing DHH doesn't mention or explain is that Basecamp is designed to be intentionally simple. Much of the complexity that you get in client work where you d…

Another thing I think DHH has missed here is that his experience is not particularly broad. His achievements are very impressive, but if I recall rightly, he started Rails before finishing his CS degree, and that's basically all he's done since.

One of the core assumptions of Rails is having a database. A lot of it is focused on turning HTTP requests into SQL requests, and turning SQL responses into HTML. Which is fine; that's what a lot of web stuff is, and it's fine for that. But it's not the only thing that happens in the world.

I've TDDed systems that are not locked in intimate embraces with databases, and TDD is much easier and more pleasant there. On one project we did like that, we ended up with 40k lines of test (and a similar amount of production code) that ran in a few seconds. It was the most pleasant development experience of my career, and TDD was no problem there.

Post reply on HN