Live data from Hacker News

Cucumber lets you write automated tests in plain language

cucumber.io

61–70 of 76 posts

Re: Cucumber lets you write automated tests in plain language

#61
post #54

Earlier quoted context omitted.

>An actual specification layer isn't any simpler than the execution layer. The point of separation of concerns isnt to keep the simple layer separate from the complex one. It's to simplify the whole thing by only addressing one concern per layer. Unit tests are often a pain in the ass to read because they are a mess of implementation and specification details. No separation. >You can't solve the problem in that way.…

Oh I understand the standard arguments. I just don't agree with them. > The point of separation of concerns isnt to keep the simple layer separate from the complex one. It's to simplify the whole thing by only addressing one concern per layer. With some caveats, I think this is just a fiction. There can be some value in having high-level tests and low-level tests, but not because it removes complexity. It can help wi…

>Oh I understand the standard arguments.

I think the problem here is that you are substituting standard arguments for my arguments.

>With some caveats, I think this is just a fiction. There can be some value in having high-level tests and low-level tests, but not because it removes complexity.

Separation of concerns isn't about having "high level" and "low level" tests. You don't have "high level" views and "low level" views in MVC. You have views and controllers and you keep them separate because otherwise you have an old-skool PHP-like mix of SQL and HTML all over the place.

That PHP style mess happens to most tests because the code that is intended to describe behavior is mushed up thoroughly with the code that attaches to the app and executes it.

If you separate the concerns, the test becomes readable and even serves as a spec.

>But this is exactly what they want you to do, and do well enough that "that's not what i meant" is not a big problem.

The problem is rarely that they didn't say what they mean. The problem is that once you've built the thing or walked them through a crystallized user story it makes them rethink what was actually needed. In many cases I've walked stakeholders through user stories and they suddenly realized no new feature was necessary.

>They certainly don't want to read cucumber tests

Nobody does, but that's because cucumber tests have abysmal syntax and UX, not because the idea of readable spectests was intrinsically bad. Throwing away spectests as a concept because cucumber is bad is like throwing away programming as a concept because COBOL is bad.

>They will tolerate rough, incremental prototypes

Of course, which is not as expensive as a fully fledged app but way more expensive to build than crystallized user stories. I prefer the cheaper option.

Re: Cucumber lets you write automated tests in plain language

#62

In my misspent youth I've tried to introduce Cucumber/Gherkin a couple of different times but it's something with a deceptive promise. The preconditions are that you have domain experts or business people interested and willing to engage in writing or reviewing these tests. Unless you have this and it's something that those people are going to sustain when the going gets tough you're just making writing tests harder…

What is it about Cucumber that is so attractive to early in career engineers (myself included)? Is it the naive belief that others, non-engineers even, will actually want to read or write your tests? It's great in theory, but like many ideologies that attract young people, political or otherwise, its promise of egalitarian test authorship rarely holds up in practice. My experience is that it ends up as an extra abstr…

I find it sad that cucumber managed to pour so much cold water on the idea of readable spectests. It's like if we gave up on programming because COBOL was bad.

Re: Cucumber lets you write automated tests in plain language

#63
There's a reason why I call BDD a Billion Dollar Disaster. I've not yet seen a Cucumber or equivalent suite not permanently broken on a mature codebase. Typically a dedicated testing team will fight tooth and nail that this is what the customer needs, even though nobody from the business side has ever looked or wants to look at the test definitions or even the output. Then of course it can't match changes to app behaviour and some poor sod has to go in periodically to fix it up after the fact.

BDD in whatever implemenation is a gigantic waste of everyone's time.

Re: Cucumber lets you write automated tests in plain language

#64

It sounds great on the label, but what you’ll end up with are single-use matchers for almost every line of your cukes, annoyance maintaining the regexes that map your cukes to the actual test implementation, and the “non-technical” people won’t be maintaining the cukes anyway. Sure, let them specify the cukes if they want, but then translate them into regular unit tests or specs and forget maintaining them. Just like…

I have ended up writing BDD scenarios as plain code comments with conventions like capital “GIVEN” etc. A simple script to scrape test files and generate a markdown file to browse.

Then you refactor your common helper functions as natural, but you’re not forced to enclose them in steps.

Re: Cucumber lets you write automated tests in plain language

#65

I posted this comment in 2013 [1] and I stand by it 12 years later: Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable. The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the…

My manager and our qa team like that they can read what we are testing step by step in plain English and reason about it the same way. If they want a new step they often request it in a similar format to the ones we have written, or directly suggest reusing a step from another test.

You dont get that for free, but i disagree with it being a 'mess' of regex...Regex for test steps is usually as simple as a basic string or number capture pattern. It's 100% a skill issue if it ends up 'messy'.

If you need something more complicated than regex you use a data source file for parameters with named columns....Which brings me nicely round to how easy it is for qa or management to look at the data being tested and again, reason about it, make suggestions, spot missing cases, etc.

Re: Cucumber lets you write automated tests in plain language

#66
post #65

I posted this comment in 2013 [1] and I stand by it 12 years later: Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable. The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the…

My manager and our qa team like that they can read what we are testing step by step in plain English and reason about it the same way. If they want a new step they often request it in a similar format to the ones we have written, or directly suggest reusing a step from another test. You dont get that for free, but i disagree with it being a 'mess' of regex...Regex for test steps is usually as simple as a basic string…

Just to pre-empt anyone saying 'but nobody will ever read them'...that absolutely isn't true in my company.

Qa read every test we write. Management reads ones of high importance, sometimes.

Re: Cucumber lets you write automated tests in plain language

#67
post #65

I posted this comment in 2013 [1] and I stand by it 12 years later: Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable. The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the…

My manager and our qa team like that they can read what we are testing step by step in plain English and reason about it the same way. If they want a new step they often request it in a similar format to the ones we have written, or directly suggest reusing a step from another test. You dont get that for free, but i disagree with it being a 'mess' of regex...Regex for test steps is usually as simple as a basic string…

Oh and one more thing. I get a bit of a 'hur durr regex bad because its regex' vibe from your comment. Regex is absolutely the right tool for the job here. The alternative would be a custom DSL.

Re: Cucumber lets you write automated tests in plain language

#68
post #66
post #65

Earlier quoted context omitted.

My manager and our qa team like that they can read what we are testing step by step in plain English and reason about it the same way. If they want a new step they often request it in a similar format to the ones we have written, or directly suggest reusing a step from another test. You dont get that for free, but i disagree with it being a 'mess' of regex...Regex for test steps is usually as simple as a basic string…

Just to pre-empt anyone saying 'but nobody will ever read them'...that absolutely isn't true in my company. Qa read every test we write. Management reads ones of high importance, sometimes.

Ps: everyone is very happy with this arrangement, quality is very important, bugs are unacceptable for our products, and auditors wouldnt pass us if they saw inadequate testing

Re: Cucumber lets you write automated tests in plain language

#69

It sounds great on the label, but what you’ll end up with are single-use matchers for almost every line of your cukes, annoyance maintaining the regexes that map your cukes to the actual test implementation, and the “non-technical” people won’t be maintaining the cukes anyway. Sure, let them specify the cukes if they want, but then translate them into regular unit tests or specs and forget maintaining them. Just like…

That's pretty much the essence of it. It's a cargo cult for scrummasters shifting a huge burden onto test maintainers for negligible benefit.

Re: Cucumber lets you write automated tests in plain language

#70

I posted this comment in 2013 [1] and I stand by it 12 years later: Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable. The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the…

Everything makes sense, except:

> The software industry has largely abandoned waterfall development

Variants of waterfall are essential for large, high-risk safety systems. There is no 1SFA dev process.

Post reply on HN