Ask HN: Should you TDD a MVP? • Hacker News Board

Re: Ask HN: Should you TDD a MVP?

#31

Post by sandis » Sun, Dec 30, 2012, 9:55 PM UTC

DanielBMarkham wrote: post #16

The question here is really "what's the test?" You have to reailze the MVP _is_ the test . For a startup, customers are how you pass the test. Anything else is a red light. So in the most important way possible, as long as you have no customers, you have a test which is failing. This is important because the maintainability you're looking for in a startup is your relationship with the customer. Manage that and the re…

> "technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be zero"

That's something to quote right there.

Re: Ask HN: Should you TDD a MVP?

#32

Post by rhizome » Sun, Dec 30, 2012, 10:06 PM UTC

stakent wrote: post #27

davidw wrote: post #21

You should at least put a testing framework in place, and test some low-hanging fruit. That'll make it easier to get momentum going for more testing if things continue, and at least give you some minimal coverage.

And test at least fundamental parts of user interaction with the MVP: sign up, login/logoff, conversion funnel, what else depends of MVP. MVP may have limited set of features, but implemented ones should work. This not TDD but rather BDD without unnecessary fluff. Additionally manual testing is repetitive work. It is better to outsource it to the computer to make sure that testing is in fact done when needed - prefer…

And just as an anecdote, I built a prototype for a client that laid fallow for a couple of years while he worked on some parallel business ideas that I also built, but when it came time to return to fire up the original idea there had been major language and framework version upgrades in the meantime. I had to almost rebuild the app from scratch due to the lack of tests in the MVP that could tell me what was what. Basic tests would have cut the upgrade time in half. Not that I mind the billings, but this time and effort gets in the way of the bizdev.

Re: Ask HN: Should you TDD a MVP?

#33

Post by pbiggar » Sun, Dec 30, 2012, 10:17 PM UTC

I'm the founder of https://circleci.com, so I feel I have a little perspective here from talked to dozens of startups about this very question.

You should test a different amount based on your goals. TDD does not deliver value to your customer directly, so the only way its even usable in an MVP is helping you iterate and deliver the MVP faster.

A common mistake, and it feels like you're doing it here, is to think that TDD is a best practice that you have to do, rather than thinking about how to apply it to improve your productivity.

So how will testing help in an MVP? It allows you verify that you haven't broken anything when you write new code, and when you rewrite existing code. But because you'll constantly throw out code, spending a long time doing this will literally be wasted as you throw out the code you're testing.

Avoid writing the sort of massive extensive tests that many associate with TDD. When you unit test a function, test or two cases at most, any more that that is overkill. If it takes you 2 minutes to write a function, a function + tests should take at most an extra 30 seconds. If you can't think how to test it, don't.

So some specific advice:

- when you write a function, you're likely to test it somehow manually (refreshing the browser, at the REPL, etc). Take that manual test and put it into the test framework. It should take 10 seconds more. If it is more effort than this, don't bother.

- write MVP-type tests: "what's the minimum I can write here to viably test this"? 1-2 test cases, at most. A positive test and a negative test ideally. And then you're done, move on.

- avoid full-browser tests. Those things are a massive time sink, and nothing will change more than your front-end.

- use a CI service (self-plug: https://circleci.com). This means you don't have to run tests before you push. Just push and let the server catch it.

Re: Ask HN: Should you TDD a MVP?

#34

Post by frio » Sun, Dec 30, 2012, 10:25 PM UTC

It really depends on the degree to which you do it. I'd argue that just having one or two very basic acceptance tests is probably all you want when you're testing the waters with an MVP. The most all-encompassing test you can make; and make it from a client-side perspective (so: JS or Python or something exercising your HTTP API).

For instance, if you were making a library app: test that a logged in user can add a book to it. That's as simple as:

    class AcceptanceTest(TestCase):
        def testThatAUserCanAddABook(self):
            r = requests.post("/books/", data={"title": "My New Book"}, auth=HTTPBasicAuth('user', 'pass'))
            self.assertEquals(r.ok, True)

Then, do the absolute bare, hacky minimum to get that test passing: implement a server which simply returns 200 to that URL.

As you're working, I've found that having that singular, overarching acceptance test will speed you up massively; there's no need to arduously step through your UI, or run through various httpie commands; the test covers you as you refactor to implement your user authentication, basic API, and backing database.

So, yes: TDD can be a big time investment, and can slow down your launch if you fully break your work down into unit tests. But having simple, straightforward, all-encompassing acceptance tests can give you a great base for getting work done.

(Note that after launch, once you start writing more tests, you should eventually throw away that acceptance test, as more componentised and faster unit tests replace it :)).

Re: Ask HN: Should you TDD a MVP?

#35

Post by freshhawk » Sun, Dec 30, 2012, 10:31 PM UTC

DanielBMarkham wrote: post #16

The question here is really "what's the test?" You have to reailze the MVP _is_ the test . For a startup, customers are how you pass the test. Anything else is a red light. So in the most important way possible, as long as you have no customers, you have a test which is failing. This is important because the maintainability you're looking for in a startup is your relationship with the customer. Manage that and the re…

> Manage that and the rest takes care of itself

Not quite, you are committing yourself to throwing away your MVP and rewriting once/if you find the fit. That might be a very good idea if your MVP is extremely simple and you expect it to either fail or succeed with minimal iterations.

> Put differently, your technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be zero

Excellent point, but you probably want to factor in opportunity costs as well. You do not want to take on so much debt that it becomes impossible to succeed.

As soon as you start to iterate on the idea then the maintainability of your codebase starts to matter because that determines how quickly you can iterate on new ideas.

You do not want to get stuck doing even a partial rewrite while your competitors keep iterating their original higher quality codebase.

The overhead that testing or even full TDD add is very minimal if you have experience, so that's a tradeoff that must be made carefully. If you are thinking about learning TDD or testing while doing your MVP then that's a different story.

This is the same question as "should I use version control for my MVP" isn't it? The level of complexity for an MVP where this becomes a bad idea is different but it's the same question.

Re: Ask HN: Should you TDD a MVP?

#36

Post by vlokshin » Sun, Dec 30, 2012, 11:07 PM UTC

front end, no. It changes too much. Back end, maybe. Only for the core logic that (probably) won't change.

Basic rule: don't let work create more work if the benefit's not worth it. You probably knew that already, so just stick to it.

Re: Ask HN: Should you TDD a MVP?

#37

Post by gkarthik » Tue, Jan 01, 2013, 8:47 AM UTC

It depends. A MVP is basically used to test if your core functionality is popular among users and also to collect some useful feedback from users via a simple system.

I personally never developed a MVP for my first start up, Tune Patrol.

While, while making Tune Patrol, a social music discovery platform, my MVP would've involved just putting up a few songs and letting users play them. Like I said, this would've helped in two ways,

1. Get traction. Users know that you're up and running.

2. It helps test to see what kind of features a user might want by using a good feedback system.

Instead I went on to develop a product with a lot of functionality. My first problem was that I couldn't immediately determine if users liked my understood the functionality completely. Luckily for me turned out users understood pretty much everything and now we are developing on it.

You could also read "Lean Startup" by Eric Ries where he stresses upon the importance of a MVP and quotes a few examples from his own experiences.

Re: Ask HN: Should you TDD a MVP?

#38

Post by TheMonarch » Thu, Jan 03, 2013, 12:18 AM UTC

freshhawk wrote: post #35

DanielBMarkham wrote: post #16

The question here is really "what's the test?" You have to reailze the MVP _is_ the test . For a startup, customers are how you pass the test. Anything else is a red light. So in the most important way possible, as long as you have no customers, you have a test which is failing. This is important because the maintainability you're looking for in a startup is your relationship with the customer. Manage that and the re…

> Manage that and the rest takes care of itself Not quite, you are committing yourself to throwing away your MVP and rewriting once/if you find the fit. That might be a very good idea if your MVP is extremely simple and you expect it to either fail or succeed with minimal iterations. > Put differently, your technical debt can never exceed the economic value of your code, which in a startup is extremely likely to be z…

I agree, the assertion that you shouldn't test is absurd. For example, TDD can save you time by catching bugs for you. When Iv'e worked on apps without tests, I have to test manually each time I make a build, which gets annoying. That's time I could spend coding. I think the solution is something in the middle ground. TDD can sometimes veer off into a time wasting effort for the sake of TDD. Just TDD the complex parts, or the ones that are likely to break. Also write the tests that are easy, or quick to write.