Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cucumber lets you write automated tests in plain language

cucumber.io

Post reply on HN
Print view View original

41–50 of 76 posts

Re: Cucumber lets you write automated tests in plain language

#41

Post by hakunin »

Good old cucumber. I remember a discussion over coffee with someone who swore by it back in the day (~2010). They forced customers to sit down and write these things together side by side. Kind of cringe in retrospect. Wonder what happened to them.

Business people don't really care about this stuff. Over the years I realize more and more that we engineers are naive in thinking that business side is concerned with edge cases and race conditions.

Just to give an analogy software people might get better, if you come to a lawyer because, say, you want to buy a house, you are not going to sit down with them and say "given I want to buy a house, when the seller hides water damage costing over $2000, I get to walk away from the deal". You just hope the lawyer is good and will protect you from various edge cases. You have a lot more to deal with than just closing paperwork. You probably are thinking about renovating, moving, getting inspection, etc.

Businesses are just like that with engineering. They don't want to sit down and meticulously analyze every possible edge case. They have other things to do. Especially when stakes are not that high. Most of these errors can probably be resolved with a phone call and a database edit.

I think this is probably for the best. A good engineer will make sure you're standing on a solid ground, and ask the right questions at the right time. They wouldn't need this amount of hand holding. Leave business time to focus on making deals, connections, organizing the whole operation to move forward, etc. Let them give you vague requirements, and crystallize them yourself. It's way better than a micro-managing business that thinks they know exactly how everything should be.

P.S. Also, I'm not sure why everyone is so hung up on regex = bad, it's not like switching to an AST-based language would've made anything better here. Regex is fine imo, just the entire concept isn't.

Re: Cucumber lets you write automated tests in plain language

#43

Post by thom »

I am very much for internal DSLs for functional testing, but have never seen the point of external DSLs (it feels wrong that anyone but the implementing team could or should create or maintain them). I passionately believe you should refactor and use abstractions in your test code thought, and in my experience that ends up looking quite like parameterised steps (again, in code rather than English preferably).

Re: Cucumber lets you write automated tests in plain language

#44

Post by dmix »

eloisius wrote: post #15

It sounds great on the label, but what you’ll end up with are single-use matchers for almost every line of your cukes, annoyance maintaining the regexes that map your cukes to the actual test implementation, and the “non-technical” people won’t be maintaining the cukes anyway. Sure, let them specify the cukes if they want, but then translate them into regular unit tests or specs and forget maintaining them. Just like…

> but what you’ll end up with are single-use matchers for almost every line of your cukes

Capybara has this problem as well as do other web acceptance tests. Maybe not every time a new matcher but frequent enough. Mostly because of the high variability of HTML paths. You have to design your HTML in a way (use lots of [title] tags and standard HTML) if you want to avoid it.

Re: Cucumber lets you write automated tests in plain language

#45

Post by KeenanKeenan »

I love cucumber but I never follow the pattern to (1) write the feature file, (2) generate partially implemented tests with said file, (3) finish implementing the tests + subject under test.

It's better to (1) write the test, (2) then later once all that's done, extract documentation in a human-friendly format.

Re: Cucumber lets you write automated tests in plain language

#46

Post by jonahx »

MoreQARespect wrote: post #39
blablabla123 wrote: post #37

Earlier quoted context omitted.

Nobody is going to read these test cases any way, except when troubleshooting or adding new code. But that is never any of the stakeholders It's quite funny that now with AI-based no/low code the same thing is attempted again. Even if the regexes might disappear, it's even more text, with even less structure (assuming anyone checked those prompts into git in the first place)

I use executable specs quite a lot with stakeholders. I'd never ask them to write them, but I will often write a spec/test based upon their two sentence jira and then screenshare and walk them through my interpretation to get feedback early (i.e. before ive wasted time building the wrong thing). Cucumber/gherkin is awful at this of course, and the regex thing was a terrible idea but it's not the only tool. The idea t…

> The idea that tests should be split into a specification layer and execution layer is a good one that should have taken off by now.

There is a fundamental reason it hasn't:

An actual specification layer isn't any simpler than the execution layer. That's a programmer's fallacy.

What has taken off, and is part of virtually every software project, is a loose, natural language specification, which hints at "more or less" what the stakeholders are imagining. The idea that you can close the gap between this and a complete specification in a way that all stakeholders can still digest is the fantasy of cucumber. Or any other tool that attempts it.

You can't solve the problem in that way. Because, from a high-level stakeholder's perspective, the whole point of the people below them (whether programmers or UX designers or anyone else) is to fill in those details in a way that matches their own hazy expectations, at least well enough.

Re: Cucumber lets you write automated tests in plain language

#47

Post by montroser »

Guid_NewGuid wrote: post #5

In my misspent youth I've tried to introduce Cucumber/Gherkin a couple of different times but it's something with a deceptive promise. The preconditions are that you have domain experts or business people interested and willing to engage in writing or reviewing these tests. Unless you have this and it's something that those people are going to sustain when the going gets tough you're just making writing tests harder…

Yeah, it basically just means that now you need both a business person /and/ a dev to hold their hand in order to write tests. Business people have business things to do and don't really want to spend their time writing tests anyway.

Re: Cucumber lets you write automated tests in plain language

#48

Post by didip »

This tool is awful. I hated it a decade ago and I still hate its successors in other languages.

It creates unnecessary work for something that's never to be seen by real stakeholders.

There is a new trend these days using LLM that is similar to Cucumber: Spec Driven Development using AI. You'd be left disappointed again.

Re: Cucumber lets you write automated tests in plain language

#49

Post by exe34 »

MoreQARespect wrote: post #39
blablabla123 wrote: post #37

Earlier quoted context omitted.

Nobody is going to read these test cases any way, except when troubleshooting or adding new code. But that is never any of the stakeholders It's quite funny that now with AI-based no/low code the same thing is attempted again. Even if the regexes might disappear, it's even more text, with even less structure (assuming anyone checked those prompts into git in the first place)

I use executable specs quite a lot with stakeholders. I'd never ask them to write them, but I will often write a spec/test based upon their two sentence jira and then screenshare and walk them through my interpretation to get feedback early (i.e. before ive wasted time building the wrong thing). Cucumber/gherkin is awful at this of course, and the regex thing was a terrible idea but it's not the only tool. The idea t…

Hey, could you please share what you use for writing your specs? Pure python? Or some DSL?

Re: Cucumber lets you write automated tests in plain language

#50

Post by solannou »

I'm a little bit astonished by the comments mentioning regex. I used Gerkins at work and there was no regex at all.

Once you write it, you can use the sentence for building other tests if you design it for being flexible and variable, i.e. design well enough your code.

Given I'm connected as ___ with password ___.

When I click on the element __.

Then the element ___ is present.

Then the element ___ is present.

[...]

For critical use case, it was enough. For more that's a bad idea: it take way too much maintenance cost.

Post reply on HN