This reference covers version: latest.

1. Getting Started

1.1. Quick Start Guide

To use Go Grid Router do the following:

  1. Install Docker to host

  2. Create configuration directory:

    $ mkdir -p /etc/grid-router/quota
  3. Create users.htpasswd file:

    $ htpasswd -bc /etc/grid-router/users.htpasswd test test-password
  4. Start Selenium standalone server on port 4445:

    $ java -jar selenium-server-standalone.jar -port 4445
    We are showing Selenium server command here because it is a well-known example. We recommend you to use Selenoid instead - it is more efficient.
  5. Create quota file (use correct browser name and version):

    $ cat /etc/grid-router/quota/test.xml
    <qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
    <browser name="firefox" defaultVersion="45.0">
        <version number="45.0">
            <region name="1">
                <host name="localhost" port="4445" count="1"/>
            </region>
        </version>
    </browser>
    </qa:browsers>
    File name should correspond to user name you added to htpasswd file. For user test we added on previous steps you should create test.xml.
  6. Start Ggr container:

    # docker run -d --name \
        ggr -v /etc/grid-router/:/etc/grid-router:ro \
        --net host aerokube/ggr:latest-release
  7. Access Ggr on port 4444 in the same way you do for Selenium Hub but using the following url:

    http://test:test-password@localhost:4444/wd/hub

2. Configuration

Ggr is using two types of configuration files:

  1. A single file to store user credentials - users file.

  2. One username.xml file (quota file) for each user storing information about available browsers.

2.1. Creating Users File

Ggr is using htpasswd files to store authentication data. Passwords are stored in encrypted form. To create such file type:

  1. Ensure you have htpasswd utility installed (e.g. from apache2-utils package on Ubuntu).

  2. Create a new users file…​

    $ htpasswd -bc /path/to/new.htpasswd username password

    ... or update an existing one:

    $ htpasswd -b /path/to/existing.htpasswd username password

2.2. Creating Quota Files

  1. Quota files define available browsers for each user. All quota files should be placed to the same directory.

  2. For user username quota file should be named username.xml.

  3. Each quota file contains the following XML:

    <qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
    <browser name="firefox" defaultVersion="45.0">
        <version number="45.0">
            <region name="1">
                <host name="host1.example.com" port="4444" count="1"/>
                <host name="host2.example.com" port="4444" count="1"/>
                ...
            </region>
            <region name="2">
                ...
            </region>
        </version>
        <version number="46.0">
            ...
        </version>
    </browser>
    <browser name="firefox" defaultVersion="45.0">
        ...
    </browser>
    ...
    </qa:browsers>

    Here we define a list of browser names, their versions and default version for each browser. Each version has one or more regions (in cloud term, i.e. data centers). Every region contains one or more hosts. Each host defined in XML should have Selenium listening on specified port. The XML namespace is needed to be fully compatible with original Java GridRouter implementation.

2.2.1. Proxying VNC Traffic

Sometimes for debugging purposes you need to see the screen of the browser where your test is being executed. The simplest way to do that is to install a VNC server to browser host. Default TCP port for VNC protocol is 5900. Ggr can proxy the VNC traffic from multiple hosts and makes it accessible on a WebSocket. For example having the running session ID you can get the VNC traffic by accessing the following endpoint:

Typical VNC traffic URL
ws://ggr-host.example.com:4444/vnc/<session-id>

Browser VNC clients like noVNC work with such URLs out of the box. By default having an XML host entry like this…​

<host name="host1.example.com" port="4444" count="1"/>

…​Ggr proxies all traffic from host1.example.com:5900. You can customize this by adding an optional vnc attribute to a host:

<host name="host1.example.com" port="4444" count="1" vnc="vnc://host1.example.com:6900"/>

Two notations are supported in vnc attribute:

  1. If the remote server has a VNC server listening on dedicated TCP port you should use vnc:// protocol and specify only host and port:

    vnc://my-vnc-host.example.com:5900
  2. If the remote server also returns VNC traffic on a web socket (which is a case for Selenoid) - then you should specify a ws:// URL without trailing <session-id> value:

    ws://my-selenoid-host.example.com/vnc

    Having this URL Ggr will append session ID and proxy VNC traffic from:

    ws://my-selenoid-host.example.com/vnc/<session-id>

2.2.2. Working with External Selenium Services

Although Ggr is mainly used for creating your own Selenium cluster you can also configure it to obtain some browsers in external Selenium services such as Saucelabs, BrowserStack or TestingBot. These services always require username and password to be specified. Credentials should be set for each browser version in respective quota file:

<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
<browser name="firefox" defaultVersion="45.0">
    <version number="45.0">
        <region name="1">
            <host name="ondemand.saucelabs.com" port="4444" count="1" username="test-user" password="my-password"/>
        </region>
    </version>
</browser>
</qa:browsers>

2.2.3. Guest Quota

By default every quota file corresponds to a user. In some cases you may need to give anonymous (guest) access to some users. This is very useful to maintain a sandbox where users can debug their tests without disturbing other tests. Guest quota is enabled using Ggr flags:

$ ./ggr -guests-allowed -guest-quota test <the-rest-of-the-flags...>

With these flags specified any browsers declared in test.xml file will be accessible without password.

2.3. Quota Reloading

  • To reload quota files just send SIGHUP to process or Docker container:

    # kill -HUP <pid>
    # docker kill -s HUP <container-id-or-name>
    Use only one of these commands depending on whether you have Docker installed.

2.4. How it Works

See the full history in our article - Selenium testing: a new hope:

2.5. Using Multiple Ggr Instances

In order to obtain reliable Selenium cluster you need to install multiple Ggr instances behind reliable load balancer such as AWS Elastic Load Balancing or Keepalived. To check Ggr instance health use /ping:

Request
$ curl -s http://example.com:4444/ping
Result
{"uptime":"2m46.854829503s","lastReloadTime":"2017-05-12 12:33:06.322038542 +0300 MSK","numRequests":42}

It returns 200 OK when Ggr operates normally. Additionally server uptime, last quota reload time and overall number of session requests from service startup are returned in JSON format.

2.5.1. Why Ggr is Stateless

Selenium uses an HTTP-based protocol. That means every action in Selenium, e.g. launching browser or taking screenshot is a separate HTTP request. When multiple instances of Ggr are handling requests behind load balancer every request can be routed to any of these instances. Here’s how it works.

Multiple Ggr Instances Working Together

ui

On the picture above there is one SLB, two instances of Ggr and one Selenium hub. Every Selenium session consists of two stages: creating new session and other requests. When you request a new session you get its identifier called session ID. You must always use this session ID in consequent requests so Selenium applies these requests to correct browser process.

For example let’s assume that new session request is routed to Ggr 1 (black arrows). This Ggr randomly chooses a hub hub1.example.com and creates a new session on it. This hub returns some session identifier marked as ID on the picture. Ggr 1 in its turn knows that session was created on hub1.example.com and extends the session ID by adding an MD5 sum S of hub hostname. This is why it returns to user new longer session with identifier S+ID just by concatenating two strings. Every Ggr instance during startup creates an in-memory map - storing host names and their MD5 sums. When consequent request for the same session S+ID arrives e.g. to Ggr 2 (red arrows) it extracts S from extended session ID and finds hub hostname in its map. Then it removes S from request session identifier and simply proxies this request to hub corresponding to S.

Because of stateless Ggr architecture you can use an unlimited number of Ggr instances behind load balancer. In order to work properly it is very important to have exactly the same XML quota files on every Ggr instance. Otherwise some Ggr instances will return 404 error when a request with unknown host S arrives.

2.5.2. Getting Host by Session ID

Sometimes you may want to get real hostname behind Ggr that is executing browser session. This can be done using session ID:

Request
$ curl -s http://test:test-password@ggr.example.com:4444/host/4355afe3f54e61eb320993066fb425133b79d8c3-e725-4f90-9e57-548b8ec3f160
Result
{"Name":"my-host.example.com","Port":4444,"Count":5,"Username":"","Password":""}

2.6. Log Files

A typical log file looks like the following:

2017/04/18 03:52:36 [12413389] [SESSION_ATTEMPTED] [my_quota] [192.168.2.3] [firefox-42.0] [firefox42-1.example.com:4444] [1]
2017/04/18 03:52:40 [12413389] [SESSION_FAILED] [my_quota] [192.168.2.3] [firefox-42.0] [firefox42-1.example.com:4444] Error forwarding the new session Request timed out waiting for a node to become available.
2017/04/18 03:52:40 [12413390] [SESSION_ATTEMPTED] [my_quota] [192.168.2.3] [firefox-42.0] [firefox42-5.example.com:4444] [2]
2017/04/18 03:52:45 [12413390] [5.86s] [SESSION_CREATED] [my_quota] [192.168.2.3] [firefox-42.0] [firefox42-5.example.com:4444] [0c500a6f-98d2-4871-acb7-637d85e1416a] [2]
....
2017/04/18 03:53:05 [SESSION_DELETED] [192.168.2.3] [firefox42-5.example.com:4444] [0c500a6f-98d2-4871-acb7-637d85e1416a]

Every line contains:

Table 1. Log entry contents
Field Example Notes

Time

2017/04/18 03:52:36

-

Request counter

[12413389]

Only present for new session requests. So far as session ID is unknown when doing attempts this counter is used to find all session attempts for each new session request.

Status

[SESSION_ATTEMPTED]

See table below for complete list of statuses.

Quota name

[my_quota]

Extracted from basic HTTP auth headers.

User IP

[192.168.2.3]

IPv4 or IPv6 address

Browser

[firefox-42.0]

Name and version. Only present for new session requests.

Hub host

[firefox42-1.example.com:4444]

Host from quota XML file

Attempt number

[1]

For SESSION_ATTEMPTED entries means current attempt number. For SESSION_CREATED entries means total number of attempts to create this session.

Session ID

[0c500a6f-98d2-4871-acb7-637d85e1416a]

As arrived from hub

Session start time

[5.86s]

-

Error

Error forwarding the new session Request timed out waiting for a node to become available.

Only present for SESSION_FAILED

The following statuses are available:

Table 2. Log entry statuses
Status Description

BAD_JSON

User request does not contain valid Selenium data

BROWSER_NOT_SET

Browser name is not present or empty string

CLIENT_DISCONNECTED

User disconnected and doing session attempts was interrupted

INVALID_URL

Session ID does not contain information about host where it was created

PROXYING_TO_VNC

Starting to proxy VNC traffic

ROUTE_NOT_FOUND

Trying to proxy session to unknown host. Usually means quota files inconsistency between multiple Ggr instances.

SESSION_ATTEMPTED

New user request for session arrived

SESSION_CREATED

A new session was created and returned to user

SESSION_DELETED

Existing session was deleted by user request

SESSION_FAILED

Session attempt on specified host failed

SESSION_NOT_CREATED

Attempts to create a new session on all hosts failed. An error was returned to user.

UNSUPPORTED_BROWSER

Requested browser name and version is not present in quota

VNC_CLIENT_DISCONNECTED

Client disconnected from VNC API

VNC_ERROR

An error occurred when trying to proxy VNC traffic

VNC_SESSION_CLOSED

Client closed VNC session

2.7. Ggr CLI Flags

The following flags are supported by ggr command:

  -graceful-period duration
    	graceful shutdown period in time.Duration format, e.g. 300s or 500ms (default 5m0s)
  -guests-allowed
    	Allow guest (unauthenticated) users to access the grid
  -guests-quota string
    	Which quota file to use for guests (default "guest")
  -listen string
    	host and port to listen to (default ":4444")
  -quotaDir string
    	quota directory (default "quota")
  -timeout duration
    	session creation timeout in time.Duration format, e.g. 300s or 500ms (default 5m0s)
  -users string
    	htpasswd auth file path (default ".htpasswd")
  -verbose
    	enable verbose mode
  -version
    	show version and exit

For example:

$ ./ggr -quotaDir /my/custom/quota/dir

When using Ggr inside Docker container these flags are passed like the following:

# docker run -d --name \
    ggr -v /etc/grid-router/:/etc/grid-router:ro \
    --net host aerokube/ggr:latest-release
    -quotaDir /my/custom/quota/dir

3. Contributing & Development

To build Ggr:

  1. Install Golang

  2. Setup $GOPATH properly

  3. Get Ggr source:

    $ go get -d github.com/aerokube/ggr
  4. Go to project directory:

    $ cd $GOPATH/src/github.com/aerokube/ggr
  5. Checkout dependencies:

    $ go get -u github.com/golang/dep/cmd/dep && dep ensure
  6. Build source:

    $ go build
  7. Run Ggr:

    $ ./ggr --help

To build Docker container type:

$ GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build
$ docker build -t ggr:latest .

3.1. Documentation

Locally can be generated with:

$ docker run --rm -v `pwd`/docs/:/documents/      \
    asciidoctor/docker-asciidoctor                \
    asciidoctor -D /documents/output/ index.adoc