Show HN: Wdio – Docker setup for WebdriverIO • Hacker News Board

Show HN: Wdio – Docker setup for WebdriverIO

#1

Post by blueimp » Sun, Mar 03, 2019, 8:45 PM UTC

Hi HN,

recently there have been a lot of discussions about Chrome's dominance on the Web leading to many websites being broken for alternative browsers.

I think one of the reasons for this is that browser automation is hard and manual testing is often limited to the browser developers themselves use.

I've distilled my best practices for browser test automation into the following project and hope it helps other developers to test their projects with more browsers:

https://github.com/blueimp/wdio

It's a Docker setup for WebdriverIO with automatic screenshots, image diffing and screen recording support for containerized versions of Chrome and Firefox.

It also includes Webdriver configurations to test an app running in Docker with Safari Desktop, Safari Mobile and Chrome Mobile via Appium and Internet Explorer and Microsoft Edge in a Windows 10 virtual machine.

Re: Show HN: Wdio – Docker setup for WebdriverIO

#2

Post by aboutruby » Mon, Mar 04, 2019, 12:01 AM UTC

I'm mostly used to Capybara / Selenium (mostly chromedriver) and have a reasonable level of Javascript/Node but I'm really not sure how I would use this project beyond the setup of having the different browsers running, am I supposed to load those config with WebdriverIO using `remote`?

Re: Show HN: Wdio – Docker setup for WebdriverIO

#3

Post by ioseph » Mon, Mar 04, 2019, 1:38 AM UTC

I may be out of the loop but one of the reasons I went with a different library to WDIO was the ability to remotely test any browser without having drivers installed on the device. Has this changed with Selenium?

Re: Show HN: Wdio – Docker setup for WebdriverIO

#4

Post by dplgk » Mon, Mar 04, 2019, 4:49 AM UTC

I've tried various automated browser tools and they all were flakey. (e.g. Randomly hangs while waiting for an element to appear when the element is already there.). Is this is a bug in my testing code or are all these types of tools actually that unreliable?

Re: Show HN: Wdio – Docker setup for WebdriverIO

#5

Post by fulafel » Mon, Mar 04, 2019, 6:53 AM UTC

The docker-comjpose file raises the question is there some handy way for versioning the used docker images in the docker-compose.yml, like pip / npm have? You would usually want to control for browser versions in this kind of testing, no?

Re: Show HN: Wdio – Docker setup for WebdriverIO

#6

Post by janpot » Mon, Mar 04, 2019, 8:19 AM UTC

wrt "Chrome's dominance in test automation", I'd like to point out that the puppeteer team is working on Firefox support.

https://www.npmjs.com/package/puppeteer-firefox https://aslushnikov.github.io/ispuppeteerfirefoxready/

Re: Show HN: Wdio – Docker setup for WebdriverIO

#7

Post by Vinnl » Mon, Mar 04, 2019, 8:41 AM UTC

I've been running WebdriverIO in Docker for a while now, as follows:

On my CI server (using GitLab CI), I run a Node Docker image, and connect the Selenium Firefox [1] or Selenium Chrome [2] Docker image to it. I then install WebdriverIO, and tell it to find Selenium on that container's hostname.

This works, but is a little bit brittle, and I've had to pin the Selenium image versions because something broke at a certain point and it didn't seem worth it to fix it yet.

Which is to say: I'd very much be in the market for using the containerised versions of Chrome and Firefox, if there were instructions for doing so in CI - the primary use case for browser automation, in my opinion.

I realise that this might not be your intended usage, but figured I'd provide this feedback just in case it is.

[1] https://hub.docker.com/r/selenium/standalone-firefox/

[2] https://hub.docker.com/r/selenium/standalone-chrome/

Re: Show HN: Wdio – Docker setup for WebdriverIO

#8

Post by blueimp » Mon, Mar 04, 2019, 9:25 AM UTC

Vinnl wrote: post #7

I've been running WebdriverIO in Docker for a while now, as follows: On my CI server (using GitLab CI), I run a Node Docker image, and connect the Selenium Firefox [1] or Selenium Chrome [2] Docker image to it. I then install WebdriverIO, and tell it to find Selenium on that container's hostname. This works, but is a little bit brittle, and I've had to pin the Selenium image versions because something broke at a cert…

Hey Vinni,

you can definitely use this project and the containerized versions of Chrome/Firefox on CI - in fact that's its primary use case.

The way this project is setup is to use the chromedriver/geckodriver servers directly, without using the selenium Java server.

My recommendation for anyone using this in a production CI system is to fork the wdio, chromedriver, geckodriver and underlying basedriver repos and set up your own Docker automated build for them.

For a given GitLab repository, you would add this project as a folder and modify the provided docker-compose.yml to replace the example app with the application files from your repository.

Please let me know if I can help you with additional instructions.

Re: Show HN: Wdio – Docker setup for WebdriverIO

#9

Post by blueimp » Mon, Mar 04, 2019, 9:37 AM UTC

janpot wrote: post #6

wrt "Chrome's dominance in test automation", I'd like to point out that the puppeteer team is working on Firefox support. https://www.npmjs.com/package/puppeteer-firefox https://aslushnikov.github.io/ispuppeteerfirefoxready/

I think puppeteer is an interesting project, but right now it's Chrome-only and therefore pretty much useless for cross-browser testing. Even with support for Firefox, it would still lack support for Safari Desktop, Safari Mobile, Internet Explorer and Microsoft Edge. It also won't allow you to run the same tests against real devices.

I think Puppeteer is likely the superior choice for any browser-automation task apart from cross-browser testing. But if you want to make sure that your website is working for your users, using a Framework that uses the standardized W3C Webdriver API is the far better choice, unless you only want to support Chrome.

Re: Show HN: Wdio – Docker setup for WebdriverIO

#10

Post by blueimp » Mon, Mar 04, 2019, 9:47 AM UTC

dplgk wrote: post #4

I've tried various automated browser tools and they all were flakey. (e.g. Randomly hangs while waiting for an element to appear when the element is already there.). Is this is a bug in my testing code or are all these types of tools actually that unreliable?

I'd say it's definitely hard to write cross-browser automated tests that are not flaky.

Some of that is due to unreliably implementations of the Webdriver API (or the previous Selenium JSON Wire protocol) in the different drivers.

Another part is that the API by nature is asynchronous, which might make it harder to reason about - although with WebdriverIO you can actually write your tests in a synchronous way, or using async/await with modern NodeJS versions.

Regarding the error you described - waiting for an element to appear when the element is already there: This might also be due to the element being outside of the viewport - e.g. the browser will not be able to click on it until you scroll there.