Automated testing and test writing often present a common challenge - TIME. As the number of tests increases, so does the time required to run them, leading to longer wait times for test results. While more tests provide greater coverage and a better chance of catching bugs, the time investment can be frustrating for developers, who may skip tests altogether due to time constraints.

This creates a situation where tests exist, but many bugs go unnoticed. To address this issue, our team implemented a functional solution - running tests in multiple threads and remotely, using Jenkins and Selenium Grid. By distributing tests across multiple machines, we were able to speed up the testing process and reduce wait times, without disrupting our workflow.

This article offers a guide based on our experience, focusing on how to use Selenium Grid with Jenkins for parallel testing in Ruby on Rails applications.

So, let's start. We have:

• Ruby on Rails project
• Cucumber with Capybara tests
• Jenkins, customized on passing browser tests in a single thread
• Many computers on the same local network

Selenium Grid in brief

Selenium Grid is a powerful tool that allows to extend the load of your tests on multiple machines. It provides the ability to manage a number of environments from a central point, which makes it easy to run the tests with a wide combination of web browsers and operating systems. It is open source software and can be used at no charge.

Executable file of Selenium Grid is selenium_server_standalone.jar which contains all needed distributions, namely selenium hub (center of monitoring and management) and selenium node (remote browsers).

Starting Selenium hub

To start it we need to download file selenium-server.jar to our server and run the hub with the following command:

java -jar selenium-server.jar -role hub

The command is executed from a directory that contains the jar file, or run in the team added the path to the file. By default the hub runs on port 4444. If needed, you can specify the port manually by adding the optional key -port to the start-up command:

java -jar selenium-server.jar -role hub -port 4445

The server also supports other ways to start, which can be found in the manual.

After starting the hub you can view its GUI console, by navigating to <http://127.0.0.1:4445/grid/console> in a browser.

It looks like this:

Starting Selenium node

Now we need to start the nodes. In other words, nodes are the browsers which will run our tests. Enter the following command to start the nodes:

java -jar selenium-server.jar -role node 
-hub http://localhost:4445/grid/register

By default the nodes run on port 5555. It also can be changed by specifying the optional parameter -port. Keep in mind that the start command contains the key of our running hub. Thus, if we changed the port at the hub start-up it should be changed when starting a node too:

java -jar selenium-server.jar -role node 
-hub http://localhost:4445/grid/register -port 5556

The start command also has a key -role. If you put it as the node it will be launched for WebDriver (Selenium2) and Remote Control (Selenium). If you are not using Remote Control it makes sense to change the setting of this key to webdriver:

java -jar selenium-server.jar -role webdriver 
-hub http://localhost:4445/grid/register -port 5556

If you go to the Grid Console <http://127.0.0.1:4445/grid/console> you should see the following:

It means that we can run our tests in five threads for Firefox, five threads for Chrome and one thread for IE. If you do not need such set of browsers, it can be configured by the optional key -browser browserName=firefox, maxInstances=5

Since our server does not have ‘X’, they should be run simultaneously with the launch of the node or we will not be able to connect to our browsers. Our final command to run the node is:

export DISPLAY=:0.0 && xvfb-run –auto-servernum –server-num=0 
java -jar selenium-server.jar -role webdriver 
-hub http://localhost:4445/grid/register -port 5556

So what do we have?

Launched hub and node running on a single instance. But we have a local network with many working machines, and Selenium Grid provides the ability to distribute the performance of tests on other machines.

We can connect via ssh to another machine on the network, install Firefox and Chrome, copy our selenium-server.jar and run it in node mode on the machine. In our case, in addition to the server, we have launched nodes on two more remote machines. Now our Grid Console looks like this:

On the remote machines we intentionally restricted nodes - Chrome 2 pcs, Firefox 2 pcs and excluded IE, as we are not going to use it. So, we can run our tests in 9 threads for Firefox and 9 threads for Chrome.

Creating upstart services to run Selenium grid

We can use an upstart to run Selenium hub and Selenium node. First of all it is right, then it is quite conveniant. To create a Selenium hub service we perform:

nano /etc/init/selenium-hub.conf

Enter the following settings in file:

1. # Selenium Server Standalone
2. 
3. description     "run Selenium Server as hub"
4. 
5. start on runlevel [2345]
6. stop on runlevel [!2345]
7. 
8. respawn
9. 
10. exec java -jar /usr/lib/selenium-server.jar -role hub -port 4445

Now we can use the following commands for the Selenium hub.

To start service:

sudo service selenium-hub start

To stop service:

sudo service selenium-hub stop

To restart service:

sudo service selenium-hub restart

A similar config file can be created for the Selenium node.

1. # Selenium Server Standalone
2. 
3. description     "run Selenium Server as node"
4. 
5. start on runlevel [2345]
6. stop on runlevel [!2345]
7. 
8. respawn
9. 
10. exec export DISPLAY=:0.0 && java -jar selenium-server.jar 
-role webdriver -hub http://localhost:4445/grid/register -port 5556

Setting up a test environment env.rb

Then we need to specify that our tests should go to remote browsers. To do this, we should add some settings to env.rb file:

1. if ENV["SELENIUM"] == 'remote'
2.    require 'selenium-webdriver'
3.    url = 'http://127.0.0.1:4444/wd/hub' # hub address
4.
5.    browser = ENV["BROWSER"]
6.    browser = 'firefox' if browser.blank?
7.    capabilities = Selenium::WebDriver::Remote::Capabilities.send
8.    (browser.downcase.to_sym)
9.    Capybara.register_driver :remote_browser do |app|
10.   Capybara::Selenium::Driver.new(app,
11.           :browser => :remote, :url => url,
12.           :desired_capabilities => capabilities)
13.    end
14.    Capybara.current_driver = :remote_browser
15.    Capybara.javascript_driver = :remote_browser
16. end

Now, when you run the tests, you should use a command with new variable, or even two variables SELENIUM and BROWSER. Key SELENIUM is mandatory. It allows to say that we use a different environment. BROWSER parameter is optional. By default, our tests will be carried out under the control of Firefox. If we want to run them under the control of Chrome we should specify the environment variable BROWSER = chrome.

For the relative Jenkins integration with Selenium, we go to Jenkins and change our command for starting tests: SELENIUM=remote bundle exec cucumber to run tests under Firefox, SELENIUM=remote BROWSER=chrome bundle exec cucumber to run tests under Chrome.

At this stage, our tests will run on a remote browser, but only in one thread.

Installing parallel_test gem

To make our server perform the tests in multiple threads, we need to split tests into groups and exactute each group in its own dedicated process and database. For this purpose, we can use a parallel_tests gem. The installation is standard. We add gem "parallel_tests", :group => :development in Gemfile. Then go to the build settings in Jenkins. We need to create additional databases to run the tests. Namely:

1. bundle exec rake db:create            # create a test db
2. bundle exec rake db:migrate       # run migrations to the test db
3. bundle exec rake parallel:create[8]    # create 8 additional db
4. bundle exec rake parallel:prepare[8]  # prepare additional db

and we change the launch command in accordance to the gem instruction: SELENIUM=remote BROWSER=chrome bundle exec rake parallel:features[8]

It also makes sense to stop an Xvfb before starting the build, as we run it together with nodes.

Now our build will be carried out in 8 threads under the control of Chrome. When passing the test, you can open Grid Console and observe the process of parallelization.

This is more or less all about running Selenium tests for Ruby on Rails apps. We hope our readers will share our excitement about parallel automated testing.

P.S.

In our case, we have managed to reduce the time of the tests from 50 minutes in a single thread on the local machine to 8.8 minutes in 10 threads in the remote browser. And 10 threads are not the limit, if you have a large network.

While running nodes on the remote machines, we were watching CPU loading on perfoming tests - it was definitely a pleasure. Two processes can run on the same machine with no problems at all.

Special thanks to Dmitry Lyalyuevu for co-authoring.