you might want to check out SIKULI (http://sikuli.org/) - it's like python on top of a computer vision stack, so you can write code like click([part of a screenshot]).
Ask HN: What do you use to test Frontend code?
31–40 of 47 posts
Re: Ask HN: What do you use to test Frontend code?
#32You could try with Jasmine, although it's not for this specific purpose... And I agree with simplegeek, selenium should be fine in your case. Although can't imagine how you could test animations... :)
Jasmine isn't what you're asking for, but you should use it in addition to what you're asking for.
Re: Ask HN: What do you use to test Frontend code?
#33Having dealt with the Selenium and its frustrations in the past, I tend to outsource this sort of testing if at all possible. Some things can only be truly tested with eyeballs. One tip I do have is to disable jQ animations during your test runs so you aren't having to jump through hoops in your test suite just to wait through animations - jQuery.fx.off = true will do that.
This! Also, try to mock any setTimeouts and setIntervals so that they execute callbacks synchronously (i.e, timeout = 0).
Re: Ask HN: What do you use to test Frontend code?
#34Admittedly, Selenium's standard interface leaves a lot to be desired; however, I've had great time using it in combination with a Python wrapper, where you write Selenium test cases as Python unit tests.
This. Just write the Selenium base code in the Firefox addon, and then click, export as python :D
Re: Ask HN: What do you use to test Frontend code?
#35We use SauceLabs with Selenium, Jellyfish (very fast), Capybara (very flexible), and Jasmine. Yes, Selenium may feel hard to work with at times yet the payoff is enormous when you aim to support many browser for many years.
Edit: I have a great outsourcing team that builds Selenium for me, and runs in SauceLabs. PM me if you want details.
Re: Ask HN: What do you use to test Frontend code?
#36For Javascript testing, I really like JSTestDriver for automation and Sinon for mocking/stubbing. I found bulky solutions like Selenium to be somewhat hit-or-miss and overkill for most tasks. It's very rare that with proper unit testing I really need a whole additional layer just to click around the site and make sure stuff is working as expected. Maybe some basic sanity checking, but it's really not my preferred way…
Same, I feel that if there is a good coverage from unit testing, then you should only need sanity checking. Furthermore, if you are using jquery (which itself is well testing on all the browsers), then you should have more confidence that your javascript will work (providing you're not using non-cross-browser javascript on top of jquery)
Re: Ask HN: What do you use to test Frontend code?
#37I'm not using full on unit tests for my current project. I just break the code into a few independently functioning layers, and make a single html file to test the code in each layer. I run the corresponding html file manually as I develop code in a given layer. It may be heresy to say this, but I've found that automated unit tests are more trouble than they're worth in a lot of cases.
The second thing I do is use my tool every day. I use the unstable branch on the unstable server whenever possible. Problems usually become obvious very quickly. I don't push to production until I've used the unstable server for 24 hours without incident. I find this combined with the testing practices described above to be the best compromise between reliability and getting-stuff-done-ability.
Re: Ask HN: What do you use to test Frontend code?
#38For algorithms or application logic, even rather simple cases, unit tests are an absolute must.
For user interfaces, it's a lot more effort for a lot less return. YMMV
Re: Ask HN: What do you use to test Frontend code?
#39For Javascript testing, I really like JSTestDriver for automation and Sinon for mocking/stubbing. I found bulky solutions like Selenium to be somewhat hit-or-miss and overkill for most tasks. It's very rare that with proper unit testing I really need a whole additional layer just to click around the site and make sure stuff is working as expected. Maybe some basic sanity checking, but it's really not my preferred way…
Same, I feel that if there is a good coverage from unit testing, then you should only need sanity checking. Furthermore, if you are using jquery (which itself is well testing on all the browsers), then you should have more confidence that your javascript will work (providing you're not using non-cross-browser javascript on top of jquery)
For reference, the list of tools and framework mentioned on this page can be found at http://www.romku.com/list/343858/web-testing-framework. Feel free to rate them and add any missing information.
Re: Ask HN: What do you use to test Frontend code?
#40Earlier quoted context omitted.
If you're worried about keeping it up to date, why not outsource the test runner to something like saucelabs?
I believe outside1234 is referring to the maintenance cost of keeping the tests up to date.
I'm using Selenium on a major project, and from the start refactored out common tasks. I record new tests using the Selenium IDE formerly, now the Sauce Builder plugin, then export to code and replace quite a lot with calls to our own much-simplified API (often cutting out huge sections of test code & replacing with one call).
Updating the tests even when we make significant changes to the website is must easier because of this approach.
One issue we ran into quickly... tests need to verify just one thing as much as possible, not long strings of actions that wander through the whole site. The problem is that there's considerable setup before testing that one thing -- creating users and getting them to the appropriate state, especially (worst of all when test cases require 3+ users to interact).
To address this I've set up some helper requests Selenium can make -- actions that aren't included in our production build -- to create/destroy test users of different types & in different states directly. I'm also thinking of adding a page that'll simply return a full dump of info on a given test user, so we can test that the end result of actions in the UI have resulted in the expected data.
It's all very much work in progress, but I'm convinced that with a little extra elbow grease added as pain points show up, we can keep a full suite of Selenium tests alive and useful.