Moon

Moon cluster consists of several components:

Moon 2.x has two different operation modes: single namespace mode and multiple namespaces mode.

In single namespace mode Moon itself and all launched browser pods are running in the one Kubernetes namespace. Moon 1.x was only able to work like this. This is still suitable if only one team is using Moon or you don’t need to limit browser consumption of different Moon users. By default, Moon is launched in this mode.

In multiple namespaces mode Moon is running in one namespace and browsers are launched in separate namespaces. The total number of such namespaces is unlimited. This mode is mainly needed when you want to control computing resources, browsers or network access rules (network policies) available for every team. How to enable this mode is described here.

In addition to container with browser every pod created by Moon contains one or more service images.

The following table summarizes what needs to be accessible for Moon in single namespace mode:

When running browsers in multiple namespaces required permissions differ. The following table shows Moon permissions in the namespace where it is running:

For every user namespace Moon needs the following permissions:

Moon supports a set of extension capabilities. You can pass them in your code to enable or disable some features. All these capabilities should be passed under moon:options key:

In statically-typed languages like Java or C# you should use a Map (Dictionary) to pass Moon capabilities, e.g.:

By default, all browsers in Moon are started with visible browser window. The majority of browsers nowadays support so-called "headless" mode, when browser is opening pages in the background and no window is visible to the user. Usually such mode is enabled by passing --headless flag to browser startup command in Selenium capabilities.

Moon automatically detects when browser is started in headless mode. Headless browsers do not require any graphical components like X-server or window manager, so Moon does not start such components when not needed. Because of this feature, there is no need to additionally pass enableVNC capability to show browser screen in Moon user interface.

When video recording is configured, recording a video of Selenium session is as easy as adding one capability to your test:

You can optionally add other capabilities to change recorded video name, screen size, frame rate and so on.

Running automated tests in mobile platforms is nowadays very important. Using a set of real devices connected to server via USB requires too much work to deploy and maintain. Running Android Emulators requires hardware server or virtual machines with nested virtualization enabled. Running iOS Simulators requires to have Apple hardware. Even with correct computing resources tests are slower than on desktop platforms and consume slightly more CPUs and memory per browser.

Your goal however is catching bugs and not deploying complicated browser automation infrastructure. There are a lot of cases when a bug related to mobile version of tested web application can be reproduced simply by having exactly the same screen size and User-Agent HTTP header being sent by browser. This feature is already available in Chromium-based browsers and is called Mobile Emulation.

Example capabilities to enable this functionality are shown below:

Moon comes with a preconfigured list of supported devices stored in devices set. Full list of available devices is available in Supported Mobile Devices section. In order to add your own mobile devices definitions - simply update this list.

Sometimes you may need to interact with the clipboard to check that your application copy-paste feature works. Moon has a dedicated API to interact with the clipboard:

Uploading files to browser is a built-in Selenium feature supported in the majority of Selenium clients. How to do this in different programming languages is shown below:

Often when working with a browser you may need to use additional data: browser extensions, test files to be uploaded, browser settings files (also known as browser profile) and so on. In standard Selenium every such additional file is uploaded with different code snippet. The most important thing is that anyway all such data is being uploaded as HTTP request body. Every time you send file like this your Selenium implementation needs to read all the bytes into memory and this dramatically increases memory consumption. A lot more efficient way of delivering the same functionality is packing all required files to a single archive (e.g. on your CI server). An URL to this archive is then sent as a Selenium capability, thus allowing every browser session to download it before actually launching the browser. We call an archive like this a browser context and respective capability is named just context:

When you provide context capability, Moon will download an archive and unpack it to /home/<user> directory, where <user> is name of the user configured in configuration object. Default username is just user, so default directory is /home/user/.

To create an archive with browser context:

Possible use cases of this feature include:

Your tests may need to download files with browsers. To analyze these files a common requirement is then to somehow extract downloaded files from browser containers. Moon provides an API to work with such files:

Selenium 4 and above has bidirectional functionality allowing to access advanced browser features. This works just out of the box. An example project demonstrating how to use it stored here.

Moon 1.x and Selenoid have custom /devtools/ API allowing direct access to browser using Chrome Developer Tools Protocol. For backwards compatibility this is also supported in Moon 2.x. In W3C WebDriver standard Selenium extension commands should be located under vendor prefix, so having a Selenium session ID to access this API in Moon 2 you have to use URL like this:

In some test cases you may need to override preferred browser locale. You can do this with standard Selenium capabilities. How to override locale depends on browser.

A common testing task is checking that your web application behaves as expected in different time zones. Depending on tested web application one of the following approaches can help.

Moon expects to run the majority of browsers in pods inside Kubernetes or Openshift cluster. However sometimes you may need to run Selenium tests on some external hosts: hardware servers or virtual machines. Mainly this could be needed in two situations:

To use external hosts you should have the following:

For every browser type you need to add the following to browsers set:

With such configuration Selenium session requests with be randomly load-balanced across the hosts specified in URLS environment variable. VNC feature should also work - you should be seeing remote host screen in Moon UI.

You can request one of browsers supported by Cypress (chrome, chromium, edge, electron or firefox) by specifying its name in URL. By default, Moon will use browsers/cypress-<browser-name>:latest public image.

Cypress API can change from version to version. Because of that it is recommended to make sure that Cypress version being used in your project corresponds to Cypress version in browser image. To use an image compatible with exact Cypress version - add this version as follows:

To enable video recording - simply add enableVideo parameter to URL:

You can optionally add other parameters to change recorded video name, screen size, frame rate and so on.

In addition to selecting Cypress version - you can enable additional features like changing screen resolution, passing custom test name and so on. All these optional features are set by adding parameters to URL:

Full list of supported parameters and their meaning is shown below.

Cypress provides Cypress Dashboard - an online service for storing test runs information. To send information about executed tests to this service you have to send your access key using CYPRESS_RECORD_KEY environment variable:

You can request one of browsers supported by Playwright (chrome, chromium, firefox or webkit) by specifying its name in URL. By default, Moon will use quay.io/playwright-<browser-name> repository to download images. Currently, Playwright API can change from version to version. Because of that it is recommended to make sure that Playwright client version being used in your code corresponds to Playwright server version in browser image. To use an image compatible with exact Playwright version - add this version as follows:

To enable video recording - simply add enableVideo parameter to URL:

You can optionally add other parameters to change recorded video name, screen size, frame rate and so on.

Similarly to Selenium, you can make browser pod automatically download arbitrary files as a single archive and unpack them to user directory. This feature is described in detail here. The main particularity in Playwright is that archive URL is being passed as Playwright context URL parameter and thus needs to be URL encoded.

For example to upload a browser extension:

In addition to selecting browser and its version - you can enable additional features like using headless browser versions, passing environment variables and so on. All these optional features are set by adding parameters to URL:

Full list of supported parameters and their meaning is shown below.

You can choose desired browser version by changing connection URL:

To enable video recording - simply add enableVideo parameter to URL:

You can optionally add other parameters to change recorded video name, screen size, frame rate and so on.

You can enable additional features by changing connection URL:

To list available licenses:

The output above is shown when the free license key is used. Columns meaning is as follows:

You may have multiple custom Kubernetes resources named license. In that case in order to work with Moon licenses - simply use fully qualified resource name:

To view a license key in YAML format:

To update an existing license key - simply update data field in respective license object:

When you update a license key - all changes are applied immediately. This usually also leads to graceful Moon pods restart (does not interrupt running browser sessions).

Moon 2.x supports sharing the same license key among several Kubernetes namespaces and in the majority of cases a single license key should be enough. However, in some cases you may want to use a separate Moon instance and a separate license key for some teams. To achieve this:

To delete an existing license key - simply delete respective license object:

When you delete the last license key with namespace field set to some Moon namespace, Moon will automatically fall back to the free license key with 4 parallel sessions included.

There are several ways to always have active Moon license keys:

Moon 2.x compared to Moon 1.x has no built-in authentication mechanism. This is because the recommended way to deliver authentication in Kubernetes is using available Ingress authentication features or sidecar containers delivering authentication. Moon reads username from X-Moon-Quota HTTP header being set by Ingress or sidecar container. The following sections describe possible authentication configurations.

The main configuration object in Moon is called quota. This object contains all configuration specific to one Moon user. To list available quotas:

As you can see every quota is a native Kubernetes object containing the following information:

Using object names instead of their contents allows to easily reuse the same browsers set or devices set for different quotas. To view the same list in YAML format:

To edit a quota object:

Configuration object stores various configuration options: computing resources being assigned to system Moon images, group and user identifiers to run browser pods and so on. This object corresponds to service.json configuration file in Moon 1.x. To list available configuration objects:

To view the same list in YAML format:

To edit a configuration object:

The browsers set stores browsers startup configuration. This object corresponds to browsers.json file in Moon 1.x. To list available browser sets:

To view the same list in YAML format:

To edit a browser set object:

If you were previously using Moon 1.x, you could notice that browsers set in Moon 2.x is slightly different compared to browsers.json from Moon 1.x. This is how a typical browsers.json file looks like:

For every browser type and version you had to provide an exact browser image (e.g. browsers/chrome:97.0) and this file contained configuration only for Selenium browsers. The main problem with this approach is that it requires a manual update from Moon cluster administrator every time a new browser image appears. Container images for different versions of the same browser are usually stored in the same repository with different tags, e.g.:

In Moon 2.x instead of copy-pasting the same image specification you only need to provide repository name in browsers set object:

This new configuration format means that all images for chrome browser used in Selenium tests will be downloaded from quay.io/browser/google-chrome-stable repository. Concrete image tag is determined when you request to start a new browser. For example, you pass the following Selenium capabilities:

In that case Moon will use quay.io/browser/google-chrome-stable:96.0 image. Similarly, in Cypress, Playwright and Chrome Developer Tools you are using URL path and parameters to pass the same information.

Moon loads information about available mobile devices for Mobile Emulation from devices set object. This object corresponds to devices.json file in Moon 1.x. To list available device sets:

To view the same list in YAML format:

To edit a device set object:

How to enable video recording depends on browser automation technology you are using:

To activate this solution you need to install one more Helm chart:

Like for any other Helm chart, browser-ops configuration options are stored in values.yaml file and applied like this:

Sometimes things go wrong: user can unexpectedly disconnect or browser session starts longer than needed. This can lead to overall cluster degradation because of broken browser pods occupying all available hardware. To prevent such cases Moon automatically detects and closes idle browser sessions. A session is considered idle when the delay between separate HTTP requests corresponding to a running session is bigger than configured timeout. Idle timeout may need to be increased when tested application pages are loading too slowly. To view or change idle timeout setting - take a look at configuration object:

Several rarely needed Moon command-line flags are responsible for advanced timeout settings:

Not only Moon timeout settings can cause your tests to break. A typical Moon installation looks like the following:

In addition to Moon timeouts other possible sources of timeouts exist:

Moon has reasonable defaults for resources consumed by every browser pod. Sometimes you may need to override these settings. To override resource settings globally for every browser image - use configuration object:

To update resource settings - simply edit configuration object, save and exit:

You can also override the same values for every browser type in browsers set. An example snippet can be found here.

To check service images resources requirements - simply show configuration object for your quota in YAML format:

To adjust CPU and memory consumption for each service image - simply update configuration object accordingly.

Browser automation stability and speed highly depends on how many computing resources are actually available to browser pods. Kubernetes has so-called Quality of Service (QoS) defining how many resources are assigned to pods being started. For stable browser automation we recommend always setting Guaranteed QoS class to Moon browser pods. To deliver this you have to make sure that requests and limits values for CPU and memory have equal values:

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured globally or for concrete browser types in browsers set. How to do this is described here. Also, you can override labels using labels capability.

Network policies are dedicated Kubernetes objects allowing to control network firewall rules. Using them with Moon is straightforward:

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured for concrete browser types in browsers set. How to do this is described here.

In Moon 2.x this is configured for all browser pods in configuration object. If you need to use different user and group identifiers for different Moon users, simply create several configuration objects and attach them to respective quota objects. Default values are:

This is configured globally for all browser pods in configuration object.

Built-in Moon metrics are exposed on the standard /metrics HTTP API on port 4444 (moon container of the Moon pod).

The following metrics are available:

Installing Prometheus with kube-prometheus-stack will also automatically install kube-state-metrics component. This component allows you to filter Kubernetes pods by labels, annotations, status, start time and so on. To fetch information about browser pods with some labels set, use the following Prometheus query:

Full list of available expressions can be found here.

Moon can add custom labels to started browser pods (e.g. browser automation project name, tested feature name and so on). This can be done globally in using browsers set, Selenium capabilities and so on. For example, after having a label project="MyCoolProject" on browser pods, you can filter such pods like this:

The following flags are supported:

The following flags are supported: