Selenium Server (Grid) with Watir & Cucumber

This subject is a repeat request I’ve had from various YouTube and website visitors.  I’m often asked how to set up Selenium Grid to work with Watir and Cucumber.

The process is really rather simple, it just takes a bit of getting used to.

Audience

This is an advanced subject in web automation – mainly requiring the reader/viewer to have an understanding of:

  • Web Automation
  • Watir-Webdriver
  • Cucumber

This is not the only way to achieve this aim, as they say – there is more than one way to skin a cat.  Some prefer to use other technology stacks… this specific example goes over the concept of parallel tests using Selenium Grid, Jenkins and Watir with Cucumber.

Video & Code

Code for this project can be found at: https://github.com/continuousqa/seleniumServerSetup

This blog post corresponds to the following YouTube video:

 

Purpose

The purpose of parallel testing comes in the form of speed.  If you have 600 browser automation test scenarios, and the average test takes 5 min to complete, you have a total completion of 3000 min. (or 50 hours!)

When I worked at eHarmony, our QA architects determined a framework that was somewhat painful to use.   They sent each UAT test in serial and it took nearly 8 hours to complete!  It would be kicked off towards the end of the day and hopefully (if it didn’t crash overnight) would have results of the test by morning.

After I left eHarmony, I landed a great gig that let me pick the technology stack, as well as scope out how to solve problems with automation.  In the case of test time, I did a bit of research and figured I’d choose the Selenium Grid (aka Selenium Server) solution.  The trick was to get Watir and Cucumber to play nice with it…

When tests are run in parallel, you’re only limitation is the amount of machines you can farm jobs to.  If one had 200 virtual machines and 200 tests, that took on average 5min to complete – you would (baring any bandwidth or server issues) be able to finish all 200 tests in around 5min.  Each test would be farmed to a single VM and consumed.

In reality it would be even more efficient as Selenium Nodes can handle multiple browser sessions run in tandem.  So 50 virtual machines, each supporting 10 browser sessions is equivalent of 500 simultaneous tests it could handle.

Architecture

One problem that some have is with the concept of Selenium server and its role.  Since it’s supposed to handle parallel jobs, sometimes testers assume they can just send traffic at it and call it a day.  What really is needed are two points of parallelism.  You need a server to send simultaneous jobs out to the Selenium Server and the Selenium Server receives and manages the simultaneous jobs on its end.

Parallel_Tests Gem

Often people mention gems such as “Parallel_tests.”  There’s a variety of gems out there that promise to provide the ability to farm out parallel tests from the machine running the cucumber tests. Unfortunately, when I tried a variety of such gems 3 years ago, I found them to be lacking.

Instead, I opted for another solution…

Jenkins

My choice was to set up Jenkins in a VM.  It was the QA Jenkins.  Only QA jobs would be run here.  Jenkins is traditionally a build server for the development process.  As developers commit code, it gets compiled and deployed through the Jenkins Continuous Integration server.

However, I found it is quite useful to use for QA.  I got the idea from a release manager at eHarmony (Deepu.)  By configuring Jenkins jobs to run specific Cucumber Features, I could send multiple jobs simultaneously (multiple features) to the Selenium Server.

Jenkins also allows for some added benefits:

  • History of pass/fails
  • Reporting and Emailing of results
  • Automated test kick off when QA deployments finish

The overall architecture looks like this humble diagram:

selenium-server

The diagram depicts a relationship between three entities:

  • Cucumber and its tests
  • Jenkins
  • Selenium Server and its nodes

For this to work, we must intelligently organize test features in Cucumber.  Meaning, we don’t lump all our tests into one feature.  We break our tests up into smaller features that can be executed in parallel.  I gave some Feature and Scenario examples in the above diagram.

Set Up

Let’s start with setting up Selenium Server.  You’ll need to download the latest version of Selenium Server from:

http://www.seleniumhq.org/download/

You will need a series of VM’s and/or physical machines on your network to set up as nodes.  One VM/machine will need to be designated as your “hub” (Selenium Server.)

Selenium Server

Whichever machine is going to be your Selenium Server, you simply copy the Selenium Server JAR file to it and start it running with:

java -jar selenium-server-standalone-2.48.2.jar -role hub

If successful you should get some text in your console, stating:

2016-01-07 18:12:46.358:INFO:osjs.Server:main: Started @766ms
18:12:46.359 INFO – Nodes should register to http://192.168.45.7:4444/grid/register/

The IP will of course be unique to your local network.  The output is telling you how to set up each node, what it may not tell you, is how to find the HUB’s console in a browser… it simply is, your same IP and port, with /grid/console.

Similarly on the nodes (each VM or machine that Selenium Server will send the automation work to) will be configured like so:

java -jar selenium-server-standalone-2.48.2.jar -role node -hub http://192.168.45.7:4444/grid/register

The key difference is that the jar file’s params are modified for role “node” and we pass in -hub with the URL to the proper destination.

If all goes well you should see in both terminals that the handshake worked. Selenium’s Hub will mention a new node was registered and Selenium’s node will mention it was successful in registration with the Hub.  *For more advanced features (such as forcing a node to only run specific browser or versions) see the Selenium Server documentation.

By default it will allow all browser types to execute (IE, FF, Chrome…)

After a proper handshake and registration, your Selenium Server Hub should look like this:

Screen Shot 2016-01-08 at 12.50.28 AM

Cucumber

At this point, we’ll delve into our Cucumber project.

I like to have an environment file set up with Cucumber so that it can simply set each component at run time.  I create an env.rb file in the /support folder for this purpose.

In the env.rb file, I can create a Watir browser instance, utilizing the Selenium Hub, like so:

@browser = Watir::Browser.new(:remote, :url=>"http://192.168.45.7:4444/wd/hub", :desired_capabilities=> {browserName: browser_name,version: browser_version})

The above example makes use of some variables for browser_name and version.  You can omit that if you like (remove the desired_capabilities.)

Now, when @browser is called in a script, it will send the call to the Selenium Hub, instead of running it locally.  The hub will then farm the job to the first available node.

To test this, we could simply call some websites in a script in a Cucumber Feature/Step:

@browser.goto “http://google.com” 

When run, we’ll see the logs in the Hub and node console scroll with the results.

Scenarios in Cucumber will be created like normal (no changes with Watir or Cucumber beyond the browser set up.)

Jenkins

Download the Jenkins JAR (do not use an installer) from: https://jenkins-ci.org/

I tend to share the Selenium HUB with my Jenkins Server… Wherever you choose to run your QA Jenkins Server, simply run:

java -jar jenkins.war

Once Jenkins is up and running (usually in a few seconds), you can simply go to it in a browser:

http://localhost:8080

It is by default running on port 8080.

Adding More Executors

The executors in Jenkins are the determining factors for how many simultaneous Jobs it can manage… Let’s up it from 2, to 10:

Screen Shot 2016-01-08 at 1.00.40 AM

Under the Jenkins dropdown, pick Manage Jenkins and then Configure System.  On this screen, up the value for “# of Executors” to 10 and save.

Jobs

To create a new Job (which will equate to one of your Cucumber Features), click:

  • New Item
  • Then Choose “Freestyle” and give it a name.  Click OK.

In the job window, you’ll need to tell Jenkins what folder to use as root… by default it uses the Jenkins directory, which won’t work out well at all.  So simply click the Advanced tab under “Advanced Project Options.”

Screen Shot 2016-01-08 at 1.03.07 AM

Check, “Use Customer Workspace” and fill in the field with the path to your Cucumber project:

Screen Shot 2016-01-08 at 1.03.27 AM

Build Step

Scroll down to the Build Step portion of the job and choose:

  • Add Build Step
  • Execute Shell (on Linux or Execute Windows batch command on Windows)

Screen Shot 2016-01-08 at 1.03.38 AM

Finally, in the field here for the shell command, you’ll enter the cucumber command required to run this specific feature:

cucumber features/login.feature

Such a command would go to the root folder (set earlier on) in your Cucumber project directory, and then run Cucumber on the “features/login.feature” if it exists.

Save the Job and run it.  If it works, it will run your code, sending it to the Selenium Hub, which farms it to a node and there on that node opens the browser to run the test.

Repeat and Make More Jobs

Repeat the above steps for each of your features.  If you have specific browsers you want to pass in on the shell command (like cucumber features/login.feature BROWSER=chrome – Which I explain how to set up in other posts) you can do so.

Jenkins is a great reporting system and can be set up with portals and views… you can add your jobs to a view like: Firefox tests, or have tabs specific to Cucumber Features themselves.

Running Jobs in Parallel

Once you have your Cucumber Tests all set up in Jenkins you can try and run them simultaneously.  Simply, you can run each job one after the other and it will queue up the appropriate Feature test, sending it to the Selenium Hub, etc.

You can also, as a final step, create a Parent Job in Jenkins that runs a set of child jobs in Jenkins simultaneously.  Then one click (on one job) kicks off all Feature tests to the Selenium Grid.

There you have it, you now have all your tests running in parallel.

** If you found this article useful, please consider making a donation to keep this site alive and ongoing **

[paypal_donation_button]

Video from YouTube

Selenium Server (Grid) with Watir & Cucumber
User Rating: 0 (0 votes)