Live data from Hacker News

How to make Selenium tests reliable, scalable, and maintainable

lucidchart.com

41–50 of 71 posts

Re: How to make Selenium tests reliable, scalable, and maintainable

#41

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver. Does anyone know of such a project?

Yes!! http://heliumhq.com (commercial; I'm one of the people behind it)

Re: How to make Selenium tests reliable, scalable, and maintainable

#43

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver. Does anyone know of such a project?

I did the exact opposite. I ripped out some robot framework tests and replaced the code with python using selenium webdriver. Works great.

can I ask why you decided to do this? The tests were just flakey while in robot framework world?

Re: How to make Selenium tests reliable, scalable, and maintainable

#45

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver. Does anyone know of such a project?

I'm just getting into Play Framework development, and they ship with FluentLenium, which seems to add some a more friendly API and convenience functions. Nothing too fancy, but just looking at the pure-Selenium coffee examples people have posted here shows how dramatic the effect can be.

The one downside is that the developers only seem to tag official releases once in a blue moon; despite the github repo being well updated, the last push to Maven was more than half a year ago, and so depends on a rather old version of Selenium.

Re: How to make Selenium tests reliable, scalable, and maintainable

#46

Everyone in the blogosphere (and at my own company) writing non-app-specific layers on top of selenium suggests that there is scope for a higher level framework that can be used on top of selenium. Or that the selenium api is too thin a layer over webdriver. Does anyone know of such a project?

I just write my own layers on top of Selenium (with python)

This one is a rough test automation, mostly used for filling in forms etc during development http://kopy.io/LMBKt (old one but to hand) handy to be able to open, login and fill in a form in a few seconds that by hand would take minutes.

I find that way works as the abstraction is only one level removed and I can just throw in methods that relate to that project.

Re: How to make Selenium tests reliable, scalable, and maintainable

#47
post #43

Earlier quoted context omitted.

I did the exact opposite. I ripped out some robot framework tests and replaced the code with python using selenium webdriver. Works great.

can I ask why you decided to do this? The tests were just flakey while in robot framework world?

I absolutely hated the robot framework. The DSL was just horrible to use. It had weird, unnecessary syntax quirks and gave you the minimal amount of information if something failed (wouldn't tell you which line number it failed on, for instance).

The tests were also flaky as hell but that was more to do with poor environment management. That, admittedly, was also easier to fix in python.

Re: How to make Selenium tests reliable, scalable, and maintainable

#48
post #35

Earlier quoted context omitted.

>the best reliability we managed to achieve was 90% with tests that run for 40 minutes, which is obviously not acceptable. What was actually going wrong during that 10%? I get something closer to 100% reliability, so I'm feeling a little perplexed by all of this. Do you make heavy use of sleeps?

Mostly these would be cases where the browser would, seemingly at random, end up in an unpredictable state and all proceeding test scenarios would fail because of this. (Page is white, or a completely unrelated website gets opened. We have seen lots of weird situations so far) This might be exacerbated by the fact that we use the remote Browserstack Selenium hosting service so that the tests can be executed automatic…

White page and randomly ending up on an unrelated website both sound like bugs.

Re: How to make Selenium tests reliable, scalable, and maintainable

#50
post #24

Earlier quoted context omitted.

Does it really? Does it test for whether a button you thought was present isn't actually clickable? If you're going to write tests, I think it makes an insane amount of sense to emulate real world conditions as much as feasibly possible (making judgement calls on things that don't matter like speed of the mouse).

Most Selenium tests don't test that a button is actually clickable though, they find things through the DOM and if the button is offscreen or hidden they won't realise it.

In my experience, an exception is thrown in my Selenium test if a button is hidden or not clickable and I try to click it.
Post reply on HN