Running Selenium Grid with BrowserMob-Proxy in Java automated tests
BrowserMob-Proxy tool is a common way to capture and to control browser traffic in Selenium automated tests. Here in the this my post I’m showing how to integrate Selenium and BrowserMob-Proxy in standalone solution where you run your browser on the same machine where you run all other solution components.
However it turned out there is a demand for using BrowserMob-Proxy in distributed Selenium Grid topology. Here in the article I’m showing how one can integrate proxy in such environments.
How does standalone solution work
Below is the schema of the component topology when you run everything on a single host.
You run your Selenium automated test which starts the proxy process and configures your web browser to pass all its traffic through that proxy. Everything is working smoothly since there is no inter-host communication involved. All components are communicating with localhost.
There can also be optional component - for example you might be allowed to visit external network only through a corporate proxy. In such case you chain that proxy to your BrowserMob-Proxy instance.
How distributed solution is different
When you run tests with Selenium Grid you normally run your code in a different machine than your browser is running. Schematically it looks like the following:
You execute your automated tests at Host A. Your code starts proxy instance which is running at the same host. Your code sends commands to hub component which is running on different host - Host GH which in turn dispatches the commands to the appropriate nodes. For example the node for this particular run might be selected as the node on the host Host GN_1.
Each node has its own WebDriver (probably even several drivers) and the web browser is also running on that node host. You cannot start proxy on Host GN_1 from the code that is running on Host A. This is why the solution would be to configure the browser on node host to use proxy on Host A.
BrowserMob-Proxy tool exposes REST API that allows to configure it remotely. Thus another solution would be having your nodes running their own proxy instances constantly while the configuration of each particular node would be performed via that REST API from your test code. However that would involve lot more of housekeeping and configuration efforts.
How do we configure proxy in test code
In order to make such the schema work we need to pay our attention to one detail when we configure the proxy instance in our test code. Check the example below:
public void setUp() {
BrowserMobProxy proxy = new BrowserMobProxyServer();
proxy.start(12345);
ChromeOptions options = new ChromeOptions();
options.setProxy(ClientUtil.createSeleniumProxy(new InetSocketAddress("192.168.1.2", 12345)));
try {
driver = new RemoteWebDriver(new URL("http://your-hub-host:4444/wd/hub"), options);
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
Here we use new InetSocketAddress("192.168.1.2", 12345) as the parameter for ClientUtil.createSeleniumProxy(..) method where you need to change 192.168.1.2 to the IP address that is assigned to that particular network interface that is being used for communicating with your nodes (not localhost of course). Also change 12345 to the port you will be running your proxy on and your-hub-host to the host your hub is running on.
Why do I use static port here instead of letting the proxy to pick random one? We’ll talk about it in the next section.
Having this configuration will allow the nodes to configure their local browsers to use the proxy that is running on the code execution host.
What else to consider
There are few moments you need to take into account when implement the architecture like shown in the example.
-
Host that is running your test code has to be visible from your nodes. This means the IP address has to be reachable.
-
Port that is used for hosting the proxy instance has to be available by the moment you run your test
-
Port that is used for hosting the proxy instance has to be exposed (i.e. not hidden) by firewall. This is why I’m using the static port. This simplifies managing firewall rule for such port.
-
Local port that is used for communicating to proxy has to be exposed on node host and the same remote port has to be allowed for outgoing packets from your code execution host.
This seems to be it. If you still have the questions please send them to me using this form. I will amend the article according to your feedback.