Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to make Selenium tests reliable, scalable, and maintainable

lucidchart.com

Post reply on HN
Print view View original

21–30 of 71 posts

Re: How to make Selenium tests reliable, scalable, and maintainable

#21

Post by technion »

stevebmark wrote: post #11

Selenium tests are inherently slow, unreliable and flappy. They have been the bane of developers for every employer I've had. Do yourself a favor and write React and test your components without a browser driver in good ol' JS with the occasional JSDom shim. It removes almost the entire need for Selenium, which should be reserved for only the faintest of smoke tests. And please, if you have to use Selenium, use headl…

Can you expand on this?

The extent of the testing I'm currently interested in is "load a page, does the JS on that page run without error"? It won't execute from a CLI and everyone I talked to pointed me at Selenium.

Re: How to make Selenium tests reliable, scalable, and maintainable

#22

Post by EdwardDiego »

karlosmid wrote: post #14

Also, by switching from Java to Ruby ecosystem is one way to improve your selenium tests. For start, use watir-webdriver and page-object gems.

Improve them how though? Speed? Reliability? If it's just a nicer API, that's all well and good, but until the key problems I face with Selenium are solved (slow and non-deterministic tests) then a nicer API to it is just rearranging deck-chairs on the Titanic.

Re: How to make Selenium tests reliable, scalable, and maintainable

#23

Post by novocaine »

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver.

Does anyone know of such a project?

Re: How to make Selenium tests reliable, scalable, and maintainable

#24

Post by rtpg »

stevebmark wrote: post #11

Selenium tests are inherently slow, unreliable and flappy. They have been the bane of developers for every employer I've had. Do yourself a favor and write React and test your components without a browser driver in good ol' JS with the occasional JSDom shim. It removes almost the entire need for Selenium, which should be reserved for only the faintest of smoke tests. And please, if you have to use Selenium, use headl…

Does it really? Does it test for whether a button you thought was present isn't actually clickable?

If you're going to write tests, I think it makes an insane amount of sense to emulate real world conditions as much as feasibly possible (making judgement calls on things that don't matter like speed of the mouse).

Re: How to make Selenium tests reliable, scalable, and maintainable

#25

Post by discodave »

stevebmark wrote: post #11

Selenium tests are inherently slow, unreliable and flappy. They have been the bane of developers for every employer I've had. Do yourself a favor and write React and test your components without a browser driver in good ol' JS with the occasional JSDom shim. It removes almost the entire need for Selenium, which should be reserved for only the faintest of smoke tests. And please, if you have to use Selenium, use headl…

I pretty much agree with your thoughts but what makes you say PhantomJS is bad software?

Re: How to make Selenium tests reliable, scalable, and maintainable

#26

Post by karlosmid »

EdwardDiego wrote: post #22
karlosmid wrote: post #14

Also, by switching from Java to Ruby ecosystem is one way to improve your selenium tests. For start, use watir-webdriver and page-object gems.

Improve them how though? Speed? Reliability? If it's just a nicer API, that's all well and good, but until the key problems I face with Selenium are solved (slow and non-deterministic tests) then a nicer API to it is just rearranging deck-chairs on the Titanic.

It seems that you try to use selenium 2, or webdriver, in order to run your unit tests. Selenium is for browser test, and by its nature it can not run in milliseconds. Its execution time is in seconds. Even when use phantomjs webdriver. It is integration testing approach because it combines execution of several javascript modules. That run in real browser. Selenium has its purpose, but fast test execution is not one of them.

Re: How to make Selenium tests reliable, scalable, and maintainable

#27

Post by logn »

patio11 wrote: post #12
stevebmark wrote: post #11

Selenium tests are inherently slow, unreliable and flappy. They have been the bane of developers for every employer I've had. Do yourself a favor and write React and test your components without a browser driver in good ol' JS with the occasional JSDom shim. It removes almost the entire need for Selenium, which should be reserved for only the faintest of smoke tests. And please, if you have to use Selenium, use headl…

I had a Rails consultancy (Makandra) recently work on a JS-heavy application that I happen to own, and they got Selenium singing on it, which had been beyond my capabilities for years. One of their tricks, which you can inspect the implementation of in their (public) utilities library [+], is using basically a vendored Firefox per project and VNCing into that Firefox to drive things around. It is thus off-screen and…

I had been using Firefox driver in Xvfb but wasn't happy with the performance/stability. So I built a Selenium driver out of Java only (using JavaFX's embedded WebKit) and used a headless JRE windowing toolkit (Monocle). My project is still a pre-release but the headless capability, Java-only system requirement, and its ajax handling might make it useful to some people currently: https://github.com/MachinePublishers/jBrowserDriver

Re: How to make Selenium tests reliable, scalable, and maintainable

#29

Post by sry_not4sale »

novocaine wrote: post #23

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver. Does anyone know of such a project?

Have you looked into BDD tools like Behat/Behave/Cucumber ?

Re: How to make Selenium tests reliable, scalable, and maintainable

#30

Post by crdoconnor »

novocaine wrote: post #23

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver. Does anyone know of such a project?

I did the exact opposite. I ripped out some robot framework tests and replaced the code with python using selenium webdriver. Works great.
Post reply on HN