Working with environment variables in testing

Environment variables - are the variables which are defined in operating system and are accessible for all the processes which are running within that system. Such variables are often used in order to transfer some data that is specific for particular environment to the executed application. For example, the variable JAVA_HOME usually stores the path to Java distribution. That path may differ on different computers, thus if some process needs to know where the tools which are the part of Java SDK bundle are stored, it just reads the value of JAVA_HOME, adds /bin to the end of the result and hence obtain the required path. By the way, all the programming languages have the toolset for working with environment variables. Lets now look at how do we treat such environment variables as testers.

Manipulating with inbound data of application

Lets assume that we’re in charge of testing the application that utilizes environment variables in its business logic. Say it would be a trading bot that expects the exchange address set in environment variable EXCHANGE_ADDRESS. If we approach to the test planning from the prospective of classical theory, we’ll prepare the set of test data, considering all the aspects of the algorithm that is used by our trade bot when it attempts to connect to the exchange (not forgetting about negative scenarios like invalid syntax of the address, or address that is behind authorization mechanism, or the address that returns to the client some error code). We will approach with that and we’ll be right. By setting again and again new values for the environment variable and restarting our trade bot, we’ll cover all the scenarios we have planned. We can do that by describing corresponding instructions (which set the values for variable and restarting trade bot) in some script file (like .bat file for Windows or shell script for *nix-like systems). But what if we want to have several bots running in parallel and running targeting different exchange addresses?

All operating systems support the mechanisms which allow the processes to summon other processes. You can check that using any process monitoring tool that your OS should have. As the result you should observe the process tree which means that all the process are in some hierarchical relationships with other processes. At the same time the context of the parent process is propagated to the child process. The context includes the environment variables as well.

env var example

For example if you open Windows command line and execute the command set, you will observe the list of some variables which are assigned with some values. Lets pick some variables to be our test subject. I have the variable named PUBLIC in my list and I’m going to use it for my experiments. In order to query the value of only that my variable we can use the command set PUBLIC. Great! We obtained its value. Lets now change the value (no panic - it will be changed for only the current process, and as we will see later - for child processes as well, but it will not impact parent processes). Execute set PUBLIC=%PUBLIC%-changed.By the way, wrapping the variable name with % is a more correct way to query the variable value in your script. You can change set PUBLIC to echo %PUBLIC% with almost the identical result. It will differ only with that set returns the name-value pair while the latter one returns just the value.

Lets now check what value does the variable have after we have changed it. Moreover we now know how to do that. We can see that it has been really changed. Lets now execute the command start cmd.exe. The command will open another command line window. Check the value of our variable in that new window. It turns out that the value is equal to what we have changed the original value to. That means the context of our original process was propagated to the child process.

Lets now get back to our first command line. Change the value of our test variable again, but this time to something different from what we set previous time. Repeat summoning of new command line window. It is easy to make sure that that our new summoned command line process have the variable value corresponding to our latest change.

Okay, so by the moment we should have three processes each handling own command line. One process is a parent and two other processes are its children. Both the children have their own PUBLIC variable storing different values. Now you can execute two trade bots each from different command lines (of course considering that we have to map approach with one my hypothetical example to another my hypothetical example), so that your test cases can be extended which parallelization aspect.

By the way, in Linux OS (as well as in Unix one) everything works using absolutely the same principles. The only commands are different. For example, if you want the variable value propagated to child process in your Linux script, you should use export command (e.g. export EXCHANGE_ADDRESS=http://blahblah.blah).

Implementing the auto-tests independent from execution environment

Except of manual testing practices, environment variables are widely used in automated testing. Often to make one’s code less coupled with execution environment, test developers delegate different configuration aspects to the people who are in charge of managing test execution environments (also often it happens that they are the same people, however it is another story). Such the approach allows to integrate tests into CI (Continuous Integration) processes more effectively where test run is only one step in the long chain which is managed by some master process.

Below I’ll list some typical configuration entities which are usually set in environment variables.

  • Home folder - a typical test produces huge number of different artifacts. Logs, screenshots, intermediate data, etc. In test code it is convenient to base on some common root folder and build all the paths in relative manner. Developers do not know where the executed test will be permitted to store data on the particular server. For the sake of security, the user on behalf of which the tests are being executed, normally have very strict privileges (including the privilege to read and write from/two particular resources). That is why the process which manages CI flow on the server can specify the folder where the tests will be permitted to store their data.

  • User name and password - at first, it is considered a bad practice to store sensitive data in your source code (including property files which are stored in VCS). Often, such data is configured on the server machines which host CI environment as corresponding environment variables.

  • Names and addresses of resources - The majority if applications interact with resources (invoke different services, file system objects, databases, etc.). Each test environment may include a lot of instances of such the resource components so that the test needs to know which resources exactly are planned or scheduled to be used for its purposes.

  • Different flags - test code might imply different modes of execution. Those modes might be regulated with the help of dedicated flags. Such flags, for example, might control the level and the depth of assertions, test types to be executed, normal response time within the environment, etc.

In general, any information that might be potentially relevant to only some particular environment/environments (but not for all the environments) - is the candidate for applying described approach.

Still have questions? Ask them here. I will try to extend the article with missed points leaning on your feedback.