Cucumber lets you write automated tests in plain language

>An actual specification layer isn't any simpler than the execution layer.

The point of separation of concerns isnt to keep the simple layer separate from the complex one. It's to simplify the whole thing by only addressing one concern per layer.

Unit tests are often a pain in the ass to read because they are a mess of implementation and specification details. No separation.

>You can't solve the problem in that way. Because, from a high-level stakeholder's perspective, the whole point of the people below them (whether programmers or UX designers or anyone else) is to fill in those details in a way that matches their own hazy expectations

If you crystallize the hazy expectations on their behalf and skip feeding it back to them then you will often find out that "thats not what i meant" after the code is complete.

Those mistakes are expensive and avoidable.

Oh I understand the standard arguments. I just don't agree with them.

> The point of separation of concerns isnt to keep the simple layer separate from the complex one. It's to simplify the whole thing by only addressing one concern per layer.

With some caveats, I think this is just a fiction. There can be some value in having high-level tests and low-level tests, but not because it removes complexity. It can help with focus and priority. Which is a problem that can be solved in many ways.

> If you crystallize the hazy expectations on their behalf and skip feeding it back to them then you will often find out that "thats not what i meant" after the code is complete.

But this is exactly what they want you to do, and do well enough that "that's not what i meant" is not a big problem. They certainly don't want to read cucumber tests as a way of ensuring you're on the same page. They will tolerate rough, incremental prototypes, per the old agile advice, and this is probably still the best way of solving the communication gap problem.

I've commented elsewhere on lack of buy-in from stakeholders as a huge pitfall.

On a technical level, cucumber is also at odds with the need for a test suite to be easily maintainable. What I mean is that each test (especially e2e tests) will want to do some setup/initialization. This is usually expressed as step definitions.

Over time, an undisciplined team may write several slightly different but effectively identical step definitions. They may also combine multiple steps into bigger steps because usually a spec writer doesn't want to exhaustively define every piece of setup, they just want to write "Given all 200 pieces of input data and mocks are magically in the right place..."

I was able to wrangle the specs into composability using Rule and Background blocks, but at that point we were just programming tests with a shitty layer over the actual code.

> Cucumber lets you write automated tests in plain language

No, it doesn't.

Or to be more precise: Cucumber lets you write your tests twice - once in something that isn't nearly as close "plain language" as it seems at first glance, and then again in a horrible mismash of regex spaghetti code that will make you wish you'd been born a thousand years before computers were invented.

I was forced to try to use it about 15 years ago and spent literally twice as much time debugging the testing framework as I did actually implementing code - velocity on actual feature development ground to a halt. I think the most hilarious part about this was that it was completely incapable of finding the most common class of problem we tended to run into at the time - cross-browser presentation issues.

One of the big problems you'll find if you try to use this trash is that nobody except a software engineer will be really able to write tests in a consistent enough way that you'll be able to implement them without a truly horrible regex soup: even if you can actually get a non-software-engineer to write cucumber tests for you (dubious), the non-engineer-types will write "given I'm logged in" for one test and then "given I have logged in" for another, and "given I gave gone through the login process" for a third. And it doesn't matter how many regexs you implement to try to cover all their variations in their language, or how clever your regexs are - the nature of natural language means there's near-infinite variations, and they'll come up with new and more annoying variations just when you start feeling confident. And that's before we even start talking about the ambiguity of natural language.

I did manage to find a decent use for cucumber-style language a couple of years ago, though - we started commenting our unit tests using this style:

  def test_something():
     """ test some thing """

     # given some prerequisite
     set_up_prerequisite()

     # when I perform some action
     perform_some_action()

     # then some condition should be true, because 
     assert some_condition is True

At first I was dubious, given my previous experience, but I quickly found this to be a very nice way to document your unit tests and make them super-readable and easy to understand at a glance. And you don't even need to install any software to start doing it. I highly recommend it!

In elixir land, the doctest is a common sight. I wouldn't say it won out over the unit test, but it's still fairly common