UPDATE: This is an outdated page. Please see my newer posts on Web Automation, such as the Golem ONE.

I love using Watir-webdriver to automate and validate browser elements.

When I got into a realm of having hundreds of automated browser tests synchronous tests would take too long.  In some cases it would take upwards of an hour.  At my previous job, we had 700 tests that 8 hours to finish.

However, with Selenium Grid we can parallel test.

Cucumber makes this really easy to set up by introducing features.  Each feature file in cucumber is a set of tests of similar functionality.

For example, say you have a site that does the following functions:

Login

Registration

Posting a status

Updating your profile

Each of those could be a automated test feature.  Login.feature, registration.feature, and so forth.  In each feature file you will have maybe dozens of automated tests that cover positive, negative, boundary cases.

Let’s say each feature set took on average 10min for a single browser to complete.  To run those serial, it would 40min – on just one browser.

But, if I ran the jobs to a Selenium Grid Hub (central server) and it farmed the automation tasks to Selenium Grid nodes, then the total time would be 10min.  We could add more Nodes, and farm 4 other browsers simultaneously… in essence turn a 160min job to a 10min job.

Initial Set Up

Initially you will want multiple VM’s.  If you’ll be testing IE, you’ll want these to be Windows VM’s.  One VM will just support the Hub (server), add four or five VM’s on different boxes to support your initial set of VM’s the Hub can farm jobs to.  These are the nodes.

To do this, just install Java on each VM and download the appropriate selenium Grid jar file.  It will be something like:

selenium-server-standalone-2.31.0.jar

Selenium Grid Hub

Since these VM’s may take windows updates on their own and restart from time to time, it’s a good idea to setup the Windows Scheduler with a  Task.  This task would run on this rule: “on restart run this batch file.”

You would create a batch file on the box that would contain something like this:

@echo off
“C:\[Add your Java Path Here]\java.exe” -jar “C:\[Add your Selenium Grid Jar Path]\selenium-server-standalone-2.31.0.jar” -role hub

So your windows task will point to that batch file.  In the case of the server restarting on it’s own, it will start up this hub automatically.

Selenium Grid Nodes

On each VM you want to farm tests to, you would need to have Java installed.  Also – download the same selenium server jar, you used on your hub.  For a quick test, you could just run this on the command line:

java -jar selenium-server-standalone-2.31.0.jar -role node -hub http://[the server name of your Selenium Hub]:4444/grid/register -browser browserName=chrome,maxInstances=5

In the above example, we’re setting the the role to node, pointing to the hub to register this node (using the default port of 4444 – Btw. you may need to make sure that 4444 is available and public from your Selenium Hub VM/server).  I’m also giving it an instruction to run Chrome tests here, allowing 5 simultaneous tests max.  If the browser info was omitted, it would run any browser I choose.

You would do this on each of your nodes.  The default is to allow 5 simultaneous runs of a browser, per node.  So if you had 4 nodes, that’s 20 simultaneous tests you could get per VM/node.

If you were to go to your Selenium Hub URL (i.e. http://your machine:4444/grid/console) you should see your nodes all showing up here. It will have little indicators on each node to indicate what can run here.  Maybe one node has OSX installed so it’s running Safari, another node has Windows and it’s running IE, FF, etc.

One last thing, in case of restart, you should also add a batch file like we did on the Selenium Grid server.  This will make sure that in the case of a restart on a node, when it comes back up it will auto register with the hub again.

Setting Up Cucumber

Some people like using Jenkins CI to run their web automation jobs.  That’s how we did it at eHarmony, and that’s how I initially did it at my current job.  But you don’t have to use a full CI package.  You could have a home built web application, or a set of scripts.  Ultimately, what you’re calling is a command line function, like this:

cucumber features/login.feature -set_env=Integration -browser=chrome

This tells Cucumber which test to run, which environment and which browser.  Out of the box it won’t work. You’ll need to set this up in the support file: env.rb.  Inside the features/support folder is a file called env.rb.  If you don’t have those folders, create them and the file.

This file can tell Cucumber what parameters to take on the command line.

I do something like this:

def browser_name
  (ENV['BROWSER'] ||= ‘firefox’).downcase.to_sym
end

def environment
  (ENV['ENVI'] ||= ‘int’).downcase.to_sym
end

We’re saying, I’ll take the command line param BROWSER, but I will also default to Firefox in case no browser is specified.    Same with Environment (incase you have dev, integration, staging environments.

Here’s where you further define this stuff:

Before do  |scenario|
  p “Starting #{scenario}”
  if environment == :int
    @browser = Watir::Browser.new(:remote, :url=>”http://[Your Selenium Grid Hub]:4444/wd/hub”, :desired_capabilities=> browser_name)
   #Optional: in the case of setting your default start page @browser.goto “http://[your start page of your test site]:8080″
  elsif environment == :local
    @browser = Watir::Browser.new browser_name
    @browser.goto “http://dev.env.com:8080″
  end
end

OK, so now you can pass BROWSER and ENV params on the command line.  So if you had a batch file to do:

cucumber ENV=int BROWSER=IE

It would run all IE test, to the Selenium Grid Hub.  Which would in turn send all the jobs to the IE VM’s.

Or you could do:

cucumber features/login ENV=int BROWSER=IE

which sends only the login tests, to run in IE, to the Selenium Grid Hub.

Jenkins CI

Jenkins is nice in that you could easily see the results of your jobs.  You could set up multiple tabs on Jenkins, like:

IE, Firefox, Chrome, Safri

Screen Shot 2014-01-23 at 2.31.16 PM

Each tab could have all your tests as Jenkins builds…

Like in the IE tab:

Login – IE

Registration – IE

etc.

Under the Firefox tab the same thing:

Login – FF

Registration – FF

Each of these jobs would simply run a command line of the appropriate cucumber job (as described above.)  See the Figure below, it shows the Build info area of the job where you’ll enter the cucumber command line:

Screen Shot 2014-01-23 at 2.31.53 PM

Since Cucumber is set up to handle these jobs to the nodes, when you run these jobs in jenkins it will farm them out to all your VM’s.  You could effectively turn a 4 hour test cycle into a 15min one.  The only limitation here is on the amount of VM’s you have.  The more VM’s the more simultaneous jobs you can run.

You can kick off each feature job one by one, by mousing over a job and clicking “build now” or if you like, you could build a master job, that kicks off all jobs… and just run the master.

Screen Shot 2014-01-23 at 2.32.31 PM

12 thoughts on “Selenium Grid with Watir and Cucumber Browser Automation”

  1. Hello,

    First of all I want to say thank you for the detailed article. It saved me a lot of time.
    So, my env consists of 2 vm nodes that run 4 instances of ff. But when I run the tests from the hub it only runs on one vm and on a single browser instance. So basically i’m just running my regression remotely. I do not receive an error so I am blocked.

    Any thoughts on that ?

    Andrei

    1. Hi Andrei, sorry for the late response. When you look at the hub, does it show each node? It should show 4 nodes, and for each node you have set up and configured it should show the browsers they support.

      If you see all 4 nodes listed, and you are only getting one test to one node… I think I know the problem. Consider that selenium grid doesn’t know Cucumber or the tests. So let’s say you have 5 Cucumber features… If you run feature number 1, it will send it to the first compatible node and run through each test scenario in that feature, in sequence.

      However, you run features 1, 2, 3, 4, 5 at the same time, then Selenium Grid will farm each feature to a different node that is appropriate for the test. That way you could have 10 test scenarios in each feature. Each feature being tested in parallel.

      One way to do this is either write a batch file, or use Jenkins. Jenkins can be used to kick off multiple sub jobs. So clicking to run a Jenkins job for “Test UI components” could be set up to start different sub jobs at the same time – all going to Selenium Grid. Selenium Grid will take the multiple jobs coming in and farm them to different nodes. Each node reporting back the pass/fail of cucumber to Jenkins.

  2. Hi again,
    I forgot to mension: I only setup the cucumber part without the environment method, so basically I run cucumber BROWSER=firefox

  3. hi, i was trying to use other browsers on my cucumber tests just like you showed on previous video. i am unble to run chrome, internet explorer, or even headless on my cucmber tests runing from my macintosh just like you showed. i can get the cucumber tests running for firefox, but i can not get it running for anything else. i have 340 testcases i do not want to repeat in separate test frwork, but if that is how to do it i must.

    can you share with me your knowlidge on how to set up the different browsers besides firefox, i much appreciate. please do the needful. thanks, chintan

  4. Hello, I am trying to set up testing like you have recommended in your blog. I am using cucmber, watir-webdriver, and rubymine on mac. Am able to automated tests with Firefox, however I am unable to create tests that cucumber runs Chrome and headless with phantomjs. I dont know why cucumber will not run Chrome however you state it works for you. Please assist with more clues, as I am at a logjam in my project. All my tests are in firefox. Can you share your ideas on this testing? many thanks for this helpful article! Please do the needful,
    Chintan

    1. Hi Chintan, sorry for the delay… I get a lot of spam these days and it’s hard to find the real posts and comments I get from viewers. So to your question… There are two possible solutions I can think of… First though, you shouldn’t have to worry, it is most likely an easy fix.

      Solution 1 – Webdriver/Selenium/Watir versions:
      When this happens as you describe it’s usually due to the version of webdriver/selenium installed. In your Gemfile.lock in your project you will see something like this:
      selenium-webdriver (2.39.0)
      childprocess (>= 0.2.5)
      multi_json (~> 1.0)
      rubyzip (~> 1.0)
      websocket (~> 1.0.4)
      watir-webdriver (0.6.4)
      selenium-webdriver (>= 2.18.0)

      You will want to make sure that the version in the lock file for watir-webdriver and selenium-webdriver is set to the appropriate versions… i.e. if it should be 2.43, change it in the lock file and do a new bundle install.

      Each time your browser updates to a new version, you usually need to see if the latest browser is supported by the latest selenium webdriver version… so you may need to update those values in your Gemfile.lock.

      Solution #2.
      Do you have the physical webdriver installed for Chrome? Watir-webdriver is compatible with Firefox without any work. However each other browser needs to have the selenium webdriver installed in the path of the computer (whether it’s Windows or MAC.) For example: https://sites.google.com/a/chromium.org/chromedriver/downloads

  5. Hi can we configure the above commands in JENKINS LINUX SHELL.please help me in configuring the selenium grid in jenkins.do we need to install additional RUBY gems in jenkins.please kindly help regarding this .
    I have the following gems in jenkins
    cucumber core
    selenium webdriver
    selenium-cucumber
    thanks,
    ramakrishna

    1. hmm if I understand your question correctly… you can set up your Ruby/Cucumber project with environment variables. Then you can have Jenkins run a command, like so: cucumber features/login.feature -set_env=Integration -browser=chrome which would run the login feature test, using your integration environment, with chrome. Integration can be defined to use the Selenium Grid, farming the test through it.

      Jenkins itself doesn’t access Ruby gems. You are creating an API for Jenkins to interact with.

  6. To clarify, they are showing you how to setup the grid, and kick off many jobs in a jenkins configuration. For parallel testing of a single test suite, you need to look at something like “parallel_tests” for ruby.

    1. Ah, I’m sorry… these are very old posts… I switched host providers and when I did I lost the images from the older posts. What I recommend today is to use Golem https://golem-framework.readthedocs.io/en/latest/ I should do a write up on it, but it’s more robust and works a lot better. The reporting in Golem is far better than this Jenkins solution. Jenkins was a work around to get a server to have access to run these automation jobs… this is completely solved with something like Golem and it’s not a hack like I did here in this old post. Check out Golem, I think it will work better for you.

Leave a Reply

Your email address will not be published. Required fields are marked *