Moon

Moon cluster consists of several components:

Moon 2.x has two different operation modes: single namespace mode and multiple namespaces mode.

In single namespace mode Moon itself and all launched browser pods are running in the one Kubernetes namespace. Moon 1.x was only able to work like this. This is still suitable if only one team is using Moon or you don’t need to limit browser consumption of different Moon users. By default, Moon is launched in this mode.

In multiple namespaces mode Moon is running in one namespace and browsers are launched in separate namespaces. The total number of such namespaces is unlimited. This mode is mainly needed when you want to control computing resources, browsers or network access rules (network policies) available for every team. How to enable this mode is described here.

In addition to container with browser every pod created by Moon contains one or more service images.

The following table summarizes what needs to be accessible for Moon in single namespace mode:

When running browsers in multiple namespaces required permissions differ. The following table shows Moon permissions in the namespace where it is running:

For every user namespace Moon needs the following permissions:

Moon comes with a powerful user interface allowing to list and filter running browser sessions, view browser screen, launch browser sessions for manual testing and so on.

The main screen of the UI is showing the list of browser sessions that are currently running or being started. For every session you can see browser automation tool, browser name, version, test name, labels, status, duration, enabled features and so on. To delete a running session you have to click twice on the button with a trash can icon (two clicks are needed to prevent accidentally removing a running session). To create a browser session for manual testing - click on the button with browser name at the top of UI screen.

One Moon cluster can run dozens, hundreds or even thousands of browser sessions in parallel. To find sessions that belong to your project or exact build number - use filters. These controls allow to filter browser sessions by id, name and labels:

When you click on a row in the sessions list, browser screen is shown automatically if possible. For example for browser sessions with invisible browser window (so-called headless browsers) - nothing happens when you click on a row. By default, browser screen is in view-only mode. To interact with the browser - click on the button with the lock icon (🔒). Click on this button again - to switch back to view-only mode. Browser screen allows to follow automated test execution and intercept it when needed. When you launch a manual testing session - use this screen to manually execute your scenario step-by-step. All browser features like opening developer toolbar are working exactly the same as on your personal computer. To copy and paste values from your computer to Moon browser session - use Ctrl+C\Ctrl+V or Cmd+C\Cmd+V shortcuts on your keyboard.

Console is a command-line interface available on Moon UI screen and giving more power to manual testing. The following features are available:

This is how console looks like:

To open console:

Moon console works similarly to standard Unix terminal. To list all available commands type:

To show command syntax add -h or -help to command:

To view previous commands use arrow up and arrow down buttons. To clear previous commands output type:

To exit from console just close console window with mouse or type:

To list available browsers:

To list last 5 versions of chrome browser:

To list mobile emulation devices:

To launch a Chrome browser:

To launch mobile emulation:

To view browser screen using VNC:

To list available browser sessions:

To delete a running browser session:

Moon supports a set of extension capabilities. You can pass them in your code to enable or disable some features. All these capabilities should be passed under moon:options key:

In statically-typed languages like Java or C# you should use a Map (Dictionary) to pass Moon capabilities, e.g.:

By default, all browsers in Moon are started with visible browser window. The majority of browsers nowadays support so-called "headless" mode, when browser is opening pages in the background and no window is visible to the user. Usually such mode is enabled by passing --headless flag to browser startup command in Selenium capabilities.

Moon automatically detects when browser is started in headless mode. Headless browsers do not require any graphical components like X-server or window manager, so Moon does not start such components when not needed. Because of this feature, there is no need to additionally pass enableVNC capability to show browser screen in Moon user interface.

When video recording is configured, recording a video of Selenium session is as easy as adding one capability to your test:

You can optionally add other capabilities to change recorded video name, screen size, frame rate and so on.

Running automated tests in mobile platforms is nowadays very important. Using a set of real devices connected to server via USB requires too much work to deploy and maintain. Running Android Emulators requires hardware server or virtual machines with nested virtualization enabled. Running iOS Simulators requires to have Apple hardware. Even with correct computing resources tests are slower than on desktop platforms and consume slightly more CPUs and memory per browser.

Your goal however is catching bugs and not deploying complicated browser automation infrastructure. There are a lot of cases when a bug related to mobile version of tested web application can be reproduced simply by having exactly the same screen size and User-Agent HTTP header being sent by browser. This feature is already available in Chromium-based browsers and is called Mobile Emulation.

Example capabilities to enable this functionality are shown below:

Moon comes with a preconfigured list of supported devices stored in devices set. Full list of available devices is available in Supported Mobile Devices section. In order to add your own mobile devices definitions - simply update this list.

Sometimes you may need to interact with the clipboard to check that your application copy-paste feature works. Moon has a dedicated API to interact with the clipboard:

Uploading files to browser is a built-in Selenium feature supported in the majority of Selenium clients. How to do this in different programming languages is shown below:

Often when working with a browser you may need to use additional data: browser extensions, test files to be uploaded, browser settings files (also known as browser profile) and so on. In standard Selenium every such additional file is uploaded with different code snippet. The most important thing is that anyway all such data is being uploaded as HTTP request body. Every time you send file like this your Selenium implementation needs to read all the bytes into memory and this dramatically increases memory consumption. A lot more efficient way of delivering the same functionality is packing all required files to a single archive (e.g. on your CI server). An URL to this archive is then sent as a Selenium capability, thus allowing every browser session to download it before actually launching the browser. We call an archive like this a browser context and respective capability is named just context:

When you provide context capability, Moon will download an archive and unpack it to /home/<user> directory, where <user> is name of the user configured in configuration object. Default username is just user, so default directory is /home/user/.

To create an archive with browser context:

Possible use cases of this feature include:

Your tests may need to download files with browsers. To analyze these files a common requirement is then to somehow extract downloaded files from browser containers. Moon provides an API to work with such files:

Selenium 4 and above has bidirectional functionality allowing to access advanced browser features. This works just out of the box. An example project demonstrating how to use it stored here.

Moon 1.x and Selenoid have custom /devtools/ API allowing direct access to browser using Chrome Developer Tools Protocol. For backwards compatibility this is also supported in Moon 2.x. In W3C WebDriver standard Selenium extension commands should be located under vendor prefix, so having a Selenium session ID to access this API in Moon 2 you have to use URL like this:

In some test cases you may need to override preferred browser locale. You can do this with standard Selenium capabilities. How to override locale depends on browser.

A common testing task is checking that your web application behaves as expected in different time zones. Depending on tested web application one of the following approaches can help.

Moon expects to run the majority of browsers in pods inside Kubernetes or Openshift cluster. However sometimes you may need to run Selenium tests on some external hosts: hardware servers or virtual machines. Mainly this could be needed in two situations:

To use external hosts you should have the following:

For every browser type you need to add the following to browsers set:

With such configuration Selenium session requests with be randomly load-balanced across the hosts specified in URLS environment variable. VNC feature should also work - you should be seeing remote host screen in Moon UI.

In some cases you may need to configure launched browser to go through some proxy server. Selenium WebDriver protocol supports standard capabilities to configure proxy server for any browser. For example:

For some programming languages there is a wrapper object called Proxy allowing to set the same value in more type-safe way:

Very often proxy servers require to provide username and password for authentication. While the majority of Selenium implementations don’t work with such proxy servers, Moon allows to configure authentication out of the box, using the same capabilities - simply add username and password to proxy host value (username:password@host:port):

You can request one of browsers supported by Cypress (chrome, chromium, edge, electron or firefox) by specifying its name in URL. By default, Moon will use browsers/cypress-<browser-name>:latest public image.

Cypress API can change from version to version. Because of that it is recommended to make sure that Cypress version being used in your project corresponds to Cypress version in browser image. To use an image compatible with exact Cypress version - add this version as follows:

To enable video recording - simply add enableVideo parameter to URL:

You can optionally add other parameters to change recorded video name, screen size, frame rate and so on.

In addition to selecting Cypress version - you can enable additional features like changing screen resolution, passing custom test name and so on. All these optional features are set by adding parameters to URL:

Full list of supported parameters and their meaning is shown below.

Cypress provides Cypress Dashboard - an online service for storing test runs information. To send information about executed tests to this service you have to send your access key using CYPRESS_RECORD_KEY environment variable:

You can request one of browsers supported by Playwright (chrome, chromium, firefox or webkit) by specifying its name in URL. By default, Moon will use quay.io/playwright-<browser-name> repository to download images. Currently, Playwright API can change from version to version. Because of that it is recommended to make sure that Playwright client version being used in your code corresponds to Playwright server version in browser image. To use an image compatible with exact Playwright version - add this version as follows:

To enable video recording - simply add enableVideo parameter to URL:

You can optionally add other parameters to change recorded video name, screen size, frame rate and so on.

Similarly to Selenium, you can make browser pod automatically download arbitrary files as a single archive and unpack them to user directory. This feature is described in detail here. The main particularity in Playwright is that archive URL is being passed as Playwright context URL parameter and thus needs to be URL encoded.

For example to upload a browser extension:

In addition to selecting browser and its version - you can enable additional features like using headless browser versions, passing environment variables and so on. All these optional features are set by adding parameters to URL:

Full list of supported parameters and their meaning is shown below.

You can choose desired browser version by changing connection URL:

To enable video recording - simply add enableVideo parameter to URL:

You can optionally add other parameters to change recorded video name, screen size, frame rate and so on.

You can enable additional features by changing connection URL:

To list available licenses:

The output above is shown when the free license key is used. Columns meaning is as follows:

You may have multiple custom Kubernetes resources named license. In that case in order to work with Moon licenses - simply use fully qualified resource name:

To view a license key in YAML format:

To update an existing license key - simply update data field in respective license object:

When you update a license key - all changes are applied immediately. This usually also leads to graceful Moon pods restart (does not interrupt running browser sessions).

Moon 2.x supports sharing the same license key among several Kubernetes namespaces and in the majority of cases a single license key should be enough. However, in some cases you may want to use a separate Moon instance and a separate license key for some teams. To achieve this:

To delete an existing license key - simply delete respective license object:

When you delete the last license key with namespace field set to some Moon namespace, Moon will automatically fall back to the free license key with 4 parallel sessions included.

There are several ways to always have active Moon license keys:

In some cases you still may want to store license key in a Kubernetes secret and allow Moon to automatically read license key from this secret. To achieve this we provide a dedicated component called license-ops. This component is a Kubernetes job that reads license key contents from configured secret and automatically updates custom Kubernetes resource being used by Moon. To enabled license-ops you need to install one more Helm chart:

Moon 2.x compared to Moon 1.x has no built-in authentication mechanism. This is because the recommended way to deliver authentication in Kubernetes is using available Ingress authentication features or sidecar containers delivering authentication. Moon reads username from X-Moon-Quota HTTP header being set by Ingress or sidecar container. The following sections describe possible authentication configurations.

The main configuration object in Moon is called quota. This object contains all configuration specific to one Moon user. To list available quotas:

As you can see every quota is a native Kubernetes object containing the following information:

Using object names instead of their contents allows to easily reuse the same browsers set or devices set for different quotas. To view the same list in YAML format:

To edit a quota object:

Configuration object stores various configuration options: computing resources being assigned to system Moon images, group and user identifiers to run browser pods and so on. This object corresponds to service.json configuration file in Moon 1.x. To list available configuration objects:

To view the same list in YAML format:

To edit a configuration object:

The browsers set stores browsers startup configuration. This object corresponds to browsers.json file in Moon 1.x. To list available browser sets:

To view the same list in YAML format:

To edit a browser set object:

If you were previously using Moon 1.x, you could notice that browsers set in Moon 2.x is slightly different compared to browsers.json from Moon 1.x. This is how a typical browsers.json file looks like:

For every browser type and version you had to provide an exact browser image (e.g. browsers/chrome:97.0) and this file contained configuration only for Selenium browsers. The main problem with this approach is that it requires a manual update from Moon cluster administrator every time a new browser image appears. Container images for different versions of the same browser are usually stored in the same repository with different tags, e.g.:

In Moon 2.x instead of copy-pasting the same image specification you only need to provide repository name in browsers set object:

This new configuration format means that all images for chrome browser used in Selenium tests will be downloaded from quay.io/browser/google-chrome-stable repository. Concrete image tag is determined when you request to start a new browser. For example, you pass the following Selenium capabilities:

In that case Moon will use quay.io/browser/google-chrome-stable:96.0 image. Similarly, in Cypress, Playwright and Chrome Developer Tools you are using URL path and parameters to pass the same information.

Moon loads information about available mobile devices for Mobile Emulation from devices set object. This object corresponds to devices.json file in Moon 1.x. To list available device sets:

To view the same list in YAML format:

To edit a device set object:

How to enable video recording depends on browser automation technology you are using:

To activate this solution you need to install one more Helm chart:

Like for any other Helm chart, browser-ops configuration options are stored in values.yaml file and applied like this:

Sometimes things go wrong: user can unexpectedly disconnect or browser session starts longer than needed. This can lead to overall cluster degradation because of broken browser pods occupying all available hardware. To prevent such cases Moon automatically detects and closes idle browser sessions. A session is considered idle when the delay between separate HTTP requests corresponding to a running session is bigger than configured timeout. Idle timeout may need to be increased when tested application pages are loading too slowly. To view or change idle timeout setting - take a look at configuration object:

Several rarely needed Moon command-line flags are responsible for advanced timeout settings:

Not only Moon timeout settings can cause your tests to break. A typical Moon installation looks like the following:

In addition to Moon timeouts other possible sources of timeouts exist:

Moon has reasonable defaults for resources consumed by every browser pod. Sometimes you may need to override these settings. To override resource settings globally for every browser image - use configuration object:

To update resource settings - simply edit configuration object, save and exit:

You can also override the same values for every browser type in browsers set. An example snippet can be found here.

To check service images resources requirements - simply show configuration object for your quota in YAML format:

To adjust CPU and memory consumption for each service image - simply update configuration object accordingly.

Browser automation stability and speed highly depends on how many computing resources are actually available to browser pods. Kubernetes has so-called Quality of Service (QoS) defining how many resources are assigned to pods being started. For stable browser automation we recommend always setting Guaranteed QoS class to Moon browser pods. To deliver this you have to make sure that requests and limits values for CPU and memory have equal values:

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured globally or for concrete browser types in browsers set. How to do this is described here. Also, you can override labels using labels capability.

Network policies are dedicated Kubernetes objects allowing to control network firewall rules. Using them with Moon is straightforward:

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured globally or for concrete browser types in browsers set. How to do this is described here.

This is configured for concrete browser types in browsers set. How to do this is described here.

In Moon 2.x this is configured for all browser pods in configuration object. If you need to use different user and group identifiers for different Moon users, simply create several configuration objects and attach them to respective quota objects. Default values are:

This is configured globally for all browser pods in configuration object.

This is configured separately for each Moon container in configuration object. YAML syntax is exactly the same as for Kubernetes pod containers.

Built-in Moon metrics are exposed on the standard /metrics HTTP API.

The following metrics are available:

Installing Prometheus with kube-prometheus-stack will also automatically install kube-state-metrics component. This component allows you to filter Kubernetes pods by labels, annotations, status, start time and so on. To fetch information about browser pods with some labels set, use the following Prometheus query:

Full list of available expressions can be found here.

Moon can add custom labels to started browser pods (e.g. browser automation project name, tested feature name and so on). This can be done globally in using browsers set, Selenium capabilities and so on. For example, after having a label project="MyCoolProject" on browser pods, you can filter such pods like this:

The following flags are supported:

The following flags are supported:

The following flags are supported: