given that some test people might be reading these comments - is anyone aware of a configuration for test frameworks that: 1. lets users opt code out of the testing footprint (I know this is a given), but also 2. monitors opted-out code for being called a certain number of times, therefore meaning it's depended upon, therefore meaning it SHOULD be tested, and gets flagged? I feel like this would be the way to go to m…
Karate: Test Automation Made Simple
11–20 of 20 posts
Re: Karate: Test Automation Made Simple
#12given that some test people might be reading these comments - is anyone aware of a configuration for test frameworks that: 1. lets users opt code out of the testing footprint (I know this is a given), but also 2. monitors opted-out code for being called a certain number of times, therefore meaning it's depended upon, therefore meaning it SHOULD be tested, and gets flagged? I feel like this would be the way to go to m…
1. In any testing framework I've used, this is (as you said) a given. The exact mechanism you have for slicing and dicing your tests varies, but this is usually well supported IMO. 2. This feels like the job of a code coverage tool to me (and it's something I would like as well), but most coverage tools I've seen don't quite have the granularity you're looking for. In certain ecosystems you could hack something toget…
I'm a believer in "right-sized" testing, meaning balancing the cost of test creation and updating against the value the tests provide. I believe tests provide the greatest value when they are testing: * public methods * critical business logic * highly reused core logic
in other words, "code for others".
For code that's used for one specific page render or something similarly isolated, I think it's more questionable whether to bother going for 100% code coverage.
The latter category is where I'd like to "trust but verify" with devs permitting them to opt out of unit tests where they believe the code isn't critical, but have some confidence that we would catch code if it became more central and then enforce testing requirements.
Re: Karate: Test Automation Made Simple
#13Pros:
* the resulting code was relatively short
* once we got used to Karate the relatively complex tests were quick to write
* the whole framework is really flexible, you can describe complex json structures using just DSL and you can always call a Java method if you are missing something.
Cons
* some of the commenters here already noticed a lengthy README.md - well, you need to read this whole file before starting any work. The framework is reasonably well documented, but you need to ingest the documentation as a whole before attempting to write a test. You cannot just search for "how can I do ___ in Karate", I mean you can try, but you are unlikely to find an answer this way.
* entry point while short is rather steep. You need about 2-3 days before you begin to understand what's going on in the code
* the code is unreadable for bystanders. Good luck with understanding "#[] #(^SOMEVAR)" or "#[_ > 0]"
* the architecture: the whole thing runs on Maven/Java. Java interprets Karate DSL, which can contain Javascript(!) which can call Java again! Yes, you can make a chain of calls Java->DSL->JS->Java! To be fair, it makes the whole framework really flexible, but still, wtf.
Will I ever use it again? Maybe. I'd say it's specific tool for rather specific tasks. It can serve certain purposes quite well. But if you are wondering what framework you should use in your next project - there are much safer choices out there.
Re: Karate: Test Automation Made Simple
#14 Their first example for BDD is just painful to look at.
Transforming the "business" language into a "developer" language so that "there is no glue code" is missing the point of BDD.
Imagine putting that example in front of a user and asking: "Is this what you want?". Even if the end user is the developer using your API it makes little sense.
As usual, I mat be missing something here - but it seems it's basically moving the "glue code" to the Gherkin level, making using the Gherkin useless.Re: Karate: Test Automation Made Simple
#15“Made simple”. Well...I tried it and it’s by far the most complex, sprawling, cluttered, “everything under the sun” testing framework I’ve ever encountered. I kinda liked the premise, but take “simple” with a truck load of salt.
[1]: https://twitter.com/techgirl1908/status/1136645315090456576
[2]: https://hackernoon.com/the-world-needs-an-alternative-to-sel...
[3]: https://twitter.com/ptrthomas/status/1034794832571510784
[4]: https://twitter.com/ptrthomas/status/1295009968425336832
Re: Karate: Test Automation Made Simple
#16Re: Karate: Test Automation Made Simple
#17Doesn't seem to support property-based testing, which is my default go-to for automated testing, so I'll give it a miss. (I've had good experiences with QuickCheck for Haskell, Hypothesis for Python, ScalaCheck for Scala, JSVerify for Javascript)
Re: Karate: Test Automation Made Simple
#18Re: Karate: Test Automation Made Simple
#19Gherkin is supposed to be for expressing human problems in a human-readable form, that also happens to be useful for computers. Using it to specify technical details of APIs (as opposed to, I dunno, writing some code to hit those APIs) seems like a really weird choice.
Re: Karate: Test Automation Made Simple
#20Their first example for BDD is just painful to look at. Transforming the "business" language into a "developer" language so that "there is no glue code" is missing the point of BDD. Imagine putting that example in front of a user and asking: "Is this what you want?". Even if the end user is the developer using your API it makes little sense. As usual, I mat be missing something here - but it seems it's basically movi…