Akephalos is a well-written driver that integrates soundly with the htmlunit virtual browser. However, as well-integrated as Akephalos is into both Capybara and htmlunit, it can only ever be as strong as htmlunit. As our applications became more sophisticated and large, we began to run into issues with htmlunit.
Bugs: as previously mentioned, there are bugs in htmlunit, specifically with jQuery’s live. Although all browser implementations have bugs, it’s more useful if tests experience the same bugs as actual browsers.
Compatibility: htmlunit doesn’t fully implement the feature set that modern browsers do. For example, it doesn’t fully handle DOM ranges or Ajax file uploads.
htmlunit doesn’t actually render the page, so tests that depend on CSS visibility or positioning won’t work.
These issues made us realize that virtual browsers like envjs and htmlunit are never going to keep pace with actual browsers. If we want to write applications using features from the latest and greatest browsers, we’re going to need to test with them, too.
That brought us back to the old mirage: Selenium. Selenium is well-supported, well-maintained, and works with real browsers. Integration with Ruby tests has improved considerably since the old Webrat days, and speed is better in some cases, as real browsers have been heavily optimized.
However, Selenium still has some ongoing issues that aren’t easy to solve:
Timing: because Selenium uses a real browser, it doesn’t have as much control as a virtual browser does. This leads to issues with timing, causing erratic test failures.
Overhead: real browsers come with features we don’t need, like a UI and plugins. This slows things down a bit.
Gotchas: real browsers come with strange edge cases. Using Selenium and Firefox, sometimes the tests crash because Firefox tries to update the browser. Firefox doesn’t allow more than one instance to run at once. These gotchas are inevitable, because real browsers weren’t designed to be driven programatically.
We were forced to use Selenium because we needed all the capabilities of a real browser, but all these issues kept us looking for something better.
It was possible. I’d like to introduce a headless WebKit driver for capybara: capybara-webkit.
Capybara-webkit has the following benefits:
Compatibility: uses a real rendering engine (WebKit)
Simplicity: runs headlessly, and designed to be controlled programatically
Performance: faster than htmlunit – the test suites we’ve tried it on so far yielded 15-25% speed increases
The holy grail?