Live data from Hacker News

Saving Agile with BDD and Cucumber

blog.fogcreek.com

41–48 of 48 posts

Re: Saving Agile with BDD and Cucumber

#41
I've been in the situation where I had a coworker who insisted on using cucumber and BDD. I would not wish that situation upon my worst enemy. It's ridiculous to think that a non developer could write a test, and it's especially ridiculous to try to parse tests as regular expressions. Good luck grepping to figure out where the implementation of the test is. It's pointless meta-work that does not increase the quality of code.

This is digital snake oil. Matt Wynne, you're making the world a worse place. Can you find something else to do?

Re: Saving Agile with BDD and Cucumber

#43

cucumber? what is this, 2013? rspec + capybara is much more enjoyable and maintainable. Maintaining a cucumber test suit is basically a nightmare. I hate writing tests for it, I hate fixing tests in it when they break, and i hate the random cucumber exceptions that cause random failures. and should you come to a point where you want to optimize the test suit performance, good luck! cucumber is good if you're not the…

Cucumber is still quite relevant. Your alternative assumes Ruby, Cucumber does not.

I personally enjoy writing Gherkin as I get a large set of feature files that serves as documentation. I don't think it's that great as a DSL or to have someone besides the step definer writing the tests. Maybe that's why you found it hard to maintain. Oftentimes I define single-use steps because my other scenarios simply don't need them. It's definitely a poor solution if you are looking at it from a programming language or code reuse perspective. Instead I use it as a way, in English, to explain features. Granted I use it almost exclusively for integration/systems tests and not unit tests or anything.

Re: Saving Agile with BDD and Cucumber

#44

I've been in the situation where I had a coworker who insisted on using cucumber and BDD. I would not wish that situation upon my worst enemy. It's ridiculous to think that a non developer could write a test, and it's especially ridiculous to try to parse tests as regular expressions. Good luck grepping to figure out where the implementation of the test is. It's pointless meta-work that does not increase the quality…

Ignoring the tooling, I don't feel it's fair to say that Wynne is "making the world a worse place". This quote in particular:

> Really the magic of BDD is in playing the game of, if we have to explain to the computer how to test the behavior that we want, we have to have figured out the behavior we want. By collaboratively doing that, by sitting down together and doing that we have to thrash out between those three groups what is it that we want so the tester and the developer and the business person are all on the same page about what is it actually going to mean for this story to be done. By the time they get to writing the code it’s a much more straight forward process. A lot of those potential bugs have been ironed out.

is incredibly important and often overlooked. It doesn't necessarily mean that a "non-developer" is writing an automated test, but that the engineering team, the QA team, and the business have discussed and reached an agreement over the acceptance tests. This implies that the test has been thought about, written, and understood by all parties including the business person which often is skipped in favor of "increasing turnaround time". Prototyping is fine if everyone agrees to it, but in a world where the expectation to ship software every sprint exists, you can't skip mutual understanding. That is a conversation and no tool is going to automate it away.

Re: Saving Agile with BDD and Cucumber

#46

I've been in the situation where I had a coworker who insisted on using cucumber and BDD. I would not wish that situation upon my worst enemy. It's ridiculous to think that a non developer could write a test, and it's especially ridiculous to try to parse tests as regular expressions. Good luck grepping to figure out where the implementation of the test is. It's pointless meta-work that does not increase the quality…

Ignoring the tooling, I don't feel it's fair to say that Wynne is "making the world a worse place". This quote in particular: > Really the magic of BDD is in playing the game of, if we have to explain to the computer how to test the behavior that we want, we have to have figured out the behavior we want. By collaboratively doing that, by sitting down together and doing that we have to thrash out between those three g…

> That is a conversation and no tool is going to automate it away.

That's my point. It's the developer's job to know what the product owner wants and write the appropriate tests for it. Cucumber brings nothing to the table.

Re: Saving Agile with BDD and Cucumber

#47
Disclaimer: I'm the creator of Cucumber.

The biggest problem with Cucumber is that most people trying it out don't understand what it is.

Cucumber is not a tool for testing software. It is a tool for testing people's understanding of how software (yet to be written) should behave.

Most bugs and delays caused by rework arise from misunderstandings, and this is the problem Cucumber aims to solve.

Cucumber is a tool that facilitates collaboration and software design (especially domain-driven design).

Here is how it works: You pop a story off your backlog and run a 20 min. meeting (Discovery Workshop) with business folks (BAs, POs, domain experts) and IT folks (developers, UX, testers if you have them).

You have a conversation about the story and come up with some concrete examples to describe the various acceptance criteria for your stories. Not in Cucumber's Gherkin language - just in plain conversational language.

For example: "The one where I upload a picture that is too big". Or: "The one where there are five taxis in range". These conversations act as catalysts to uncover subtle details where business and IT might have a different understanding.

Two things can happen at the end of this short meeting. You ask people to do a thumbs-up or thumbs-down vote on whether they understand everything that needs to be done, and whether the story is small enough. If enough people give a thumbs down, you send the story back for further analysis, maybe breaking it up into something smaller. If it's mostly thumbs-up, you're good to go.

After the 20 min. meeting you have 2-5 concrete examples that a developer (and perhaps a tester) can flesh out in more detail using Gherkin (Given-When-Then) to make it even more concrete. For example:

  Scenario: Close taxis with higher rating win 
    Given taxi A with rating 0.8 is 1400m from the customer
    And taxi B with 0.9 is 1500m from the customer
    When the customer requests a taxi
    Then taxi B should be assigned
The dev shows the example to the business person, who confirms that this is right (or wrong).

Now, the developer follows the regular TDD workflow, using the Scenario to guide the development of the core domain logic. The Cucumber scenario doesn't go through a UI using Selenium WebDriver or similar. The domain logic is implemented in such a way that external services, message queues and databases are stubbed out.

Lower level unit tests are still written, and there are far more of those than Cucumber Scenarios.

Cucumber is there to make sure you write the right code.

Unit testing tools are there to make sure you write the code right.

Using UI testing tools together with Cucumber? Please don't - or at least do it very sparingly. UI tests are expensive to maintain (the UI is more volatile than your core domain). They are slow (2-3 orders of magnitude slower than test talking directly to the domain logic). And finally - when they fail they don't tell you where the bug is.

The purpose of Cucumber is to bridge the communication gap between business and IT by providing a small set of essential scenarios to illustrate core behaviour of unwritten software. These scenarios do become regression tests, but their real value is to prevent defects by uncovering bad assumptions up-front. You end up with executable, living documentation accessible to everyone on the team. -Documentation of how the software should behave - and how it actually behaves.

Re: Saving Agile with BDD and Cucumber

#48

Disclaimer: I'm the creator of Cucumber. The biggest problem with Cucumber is that most people trying it out don't understand what it is. Cucumber is not a tool for testing software. It is a tool for testing people's understanding of how software (yet to be written) should behave. Most bugs and delays caused by rework arise from misunderstandings, and this is the problem Cucumber aims to solve. Cucumber is a tool tha…

I'll take the bait.

Cucumber is a testing tool, depending how you use it it may facilitate BDD. It may not be your intention as the author for it to be a testing tool but it is, lets look at it some more.

Gherkin (Given-When-The)

Scenario: Close taxis with higher rating win Given taxi A with rating 0.8 is 1400m from the customer And taxi B with 0.9 is 1500m from the customer When the customer requests a taxi Then taxi B should be assigned

Lets look at the Gherkin syntax. It follows a set format with forced English language. It gives a context (Given), some input data (When) and an expectation (Then). This syntax is an example based specification and lives in a plain text file generally with the file extension *.feature. We'll call the above example a feature based on the file extension, we'll also call this Gherkin format an external DSL (domain specific language).

Now on it's own this feature file is useless, it's a plain text file. Why do I need Cucumber for this it's a plain text file? Why can't I store it in a shared wiki where people can collaborative edit it and track changes? If it's a plain text file to share knowledge why do I need to use forced English with the Gherkin syntax instead of a more natural form for the intended audience? Why does it need to be text if it's demonstrating shared knowledge, maybe a comic strip may be more appropriate for the domain? Why would I need Cucumber for a team to sit down to create this collaborative knowledge?

We need to follow this strict syntax because it's an external DSL which is used by an interpreter. This interpreter parses a feature file and asserts a given input is equal to the expected output specified in the feature file Gherkin DSL. Lets think about this some more, we give some context, we provide some data, we run a computation and we assert the output, this sounds very much like an automated test. If I was to write a definition of an automated test this would be it.

This interpreter is fairly fragile. We have a miss match between the plain text feature files in our Gherkin DSL format and our test (not sure what else to call it?) execution tightly coupled with fragile regex. The implementation also promotes heavy mutation in the test (sorry, again it's not a test?) implementation which ultimately leads to fragile assertions (aka tests).

I know you say don't use UI testing tools with Cucumber, but I'll take your own example as it demonstrates mutation quite clearly https://github.com/cucumber/cucumber-jvm/tree/master/example...

The steps are stand alone

@Given("^I am on the front page$") will mutate sending the browser to a page. It doesn't give an indication if it's successful or not. The function is marked with throws InterruptedException so I can only guess it blows up if it fails, or maybe not?

Then we have in TemperatureStepdefs @When("^I enter (.+) (celcius|fahrenheit)$") which finds an element by id and sends some key events. Again how does the subsequent step know this is successful, how does this step know the previous step was successful? We don't, we assume, our test may work or we may get silent errors which cascade down.

Then you see people use things like public String currentPage = "" and update it as you progress through the workflow and assert on it. Mutation, race conditions, silent failures if you've used Cucumber for any amount of time you've been deep in these trenches.

I digress

So, if Cucumber is not about the test part why do we need the interpreter which runs a computation with given input and tests it matches the expected output. Without this part Cucumber is a set of flat text files. What do we get? flat files? Why are these better than a shared wiki, google doc, spreadsheet? They achieve the same, canonical source of knowledge.

But Cucumber promotes a conversation. No, used in an agile productive organization features will be written collaboratively and Cucumber may be a facilitator to reach this goal. Cucumber does not enforce this or ensures this happens, it's promoted but in no way is this a requirement to use Cucumber.

If you are already an efficient team delivering software you'll already be having this communication part. Cucumber isn't a tool for test so what does it give us if we are already talking and delivering? We have other better test tools (cucumber is not a test tool right?) and more efficient ways of collaboratively writing, sharing and tracking knowledge.

Cucumber is a facilitator to help organisations to start having conversations and collaboratively share knowledge. I accept this, people who are looking at ways to improve things see this as a valid usecase, it's a trojan horse to get a more agile workflow in through the backdoor. My issue is when Cucumber is used in this way it's used when you are knee deep in mud, it's a technical solution to mostly a non technical issue. Your organisation is dysfunctional, likely not delivering and a command and control structure. Your team looking at Cucumber want change but the issues lie far deeper and Cucumber will not save you. Open up a Google Doc, Wiki Page, sit down with your team and stakeholders and first talk.

So really, what is Cucumber?

As a test tool it sucks. There far better automated test tools As a shared knowledge base, it's easier to collaborate in something accessible to everyone where change is easier and can be audited. A wiki, a Google Doc, a spreadsheet, it really doesn't matter they all achieve the same goal.

As a facilitator, you have a non technical problem deeply rooted in how your organisation works. Sit down, have the talks, create the shared knowledgebase, solve the core issues then and only then look at technical facilitators.

Post reply on HN