Let me start with full disclosure – only a few things about Keywords and Channels are new in 4.0.4. Much of what you will see in this post came about in version 4.0.3. But I believe much of the new stuff in 4.0.3 related to channels and keywords went largely unnoticed – partially because people struggle to see why channels are anything different than environments we already had. Perhaps this post can shed a little light on that by way of an example.
Let’s say you are a tester on the JKE Money that Matters project. In your new release, JKE is adding the ability for account holders to contribute a portion of the dividends from their interest-bearing account to a charity of their choice. Research has shown that people are more likely to contribute to charities close to home so JKE is adding the ability to find nearby organizations to which the account holder can contribute. Your job will be to use a combination of artifact types in Rational Quality Manager to test this functionality across the various platforms and environments that customers might use while maximizing reuse on your test project.
There are many ways you could approach this problem. In this article, I will attempt to take an artifact-centric view from a more-or-less top-down perspective.
Yes, test cases are the fundamental units of testing, but it is amazing to me how each testing team I encounter seems to have a different definition of what a test case is. I don’t intend to get into that philosophical debate here so let’s just agree that a test case represents one scenario of activity through the application under test. For JKE, maybe the test case is “Contribute to a nearby charity“. Notice I didn’t say “Contribute to a nearby charity using my mobile phone“. You want to keep your definition of the test case as devoid of implementation details as possible. “Why would you want to do that?”, you ask. Well, the test case is a planning and design asset. If you can use the test case to describe the what and why you want to test without getting into the how you are going to test it, you can start planning and designing the test case very early. The gory details of how the test will be performed is kept for the test script. This differentiation is really important to understand if you are going to be effective using RQM. This relationship between test case and test script enables significant reuse of test assets.
Ok, back to our story… Let’s consider that JKE account holders can access their bank through their PC browser or though their mobile device. They probably have other methods as well, but let’s just focus on those two for now. RQM calls these methods “channels”. Your test team has decided you must support two channels for your testing – Desktop and Mobile.
So you want to have a single, rather generic test case of “Contribute to a nearby charity”. That is appealing from a planning standpoint, but you know there are some unique aspects of each channel. For example, the JKE mobile app will leverage the device’s ability to locate itself using GPS or WiFi to know what charities are nearby. However the desktop user will need to enter a zipcode to tell the JKE system where to look for charities. When you get down to implementing this generic test case, the details of how it is done on each channel will diverge. How do you manage that uniqueness?
As mentioned earlier, the test script defines the how. The test script is the implementation of the what defined in the test case. The one-to-many relationship of test case to test script is one of the most powerful aspects of RQM. You can have a single test case that has multiple implementations associated with it in the form of multiple test scripts. So in our example, you will need to have two test scripts – one for the Desktop channel and one for the Mobile channel… or will you? This is where Keywords come into play.
“Finally, he’s getting around to the keywords”, you say. Hopefully that preamble has provided a bit of context that will help show the value of keywords. Think of keywords as modular building blocks of scripts. Scripts can be constructed by stacking these building blocks one on top the other with a little glue in between. Furthermore, keywords can have alternate implementations that can be associated to channels. Without keywords, you would need two separate scripts for the two channels you are testing. With keywords, you can create one script with the unique portions of the script sectioned out as keywords with interchangeable parts. The whole goal of this is to continue to drive down reuse to the lowest level possible. An example will be useful to solidify these concepts, but stick with me while I introduce one more concept first.
Of course, you realize you can’t simply pick one desktop configuration and one mobile configuration, run a few tests on each and call it good. You will need to test against several combinations of desktop OS versions and desktop browsers. You need to test against several combinations of mobile OS versions and mobile form-factors. You may need to consider many combinations of properties on each channel besides OS, browser and form-factor. Mobile especially opens the flood gates to various environment parameters. So considering channels alone is not sufficient. For example, as part of this test case on the Mobile channel, you need to make sure the Find Local Participating Charity functionality works. But the mobile device has two different ways to derive its location – from GPS or from WiFi. The reason you are conducting the test (Test Case) is exactly the same. The steps required to perform the test (Test Script) are exactly the same. But for platform coverage and reporting purposes, you need to ensure you have tested on mobile devices using both location methods. This is a situation where channels are simply not sufficient to define the coverage model.
Test Environments define the specifics of the device being used for testing. In our example, you may decide to test on various combinations of iOS and Android devices configured to use GPS or Wifi locations. Each of these combinations of properties is called a Test Environment. You will want to define all the Test Environments you will test against as part of your test planning process so you can ensure you have considered and documented all the combinations you will be testing.
The InfoCenter tells you that channels are subsets of test environments. I suppose it depends on your frame of reference, but I consider them to be supersets of test environments. Test environments are more specific. Channels are more generic. For example, you may have a channel of Mobile, but your test environments might be iOS_Phone_GPS and Android_Tablet_WiFi. Both of these test environments fall under the Mobile channel.
Let’s take a look at what that might look like in Rational Quality Manager.
Enabling Channel Support
First, you need to enable Channel support in Project Properties > Execution Preferences.
The Lab Resource Properties
Next, as part of your platform coverage planning, you define the necessary lab resource and channel properties for the project. This is done by a project administrator in Manage Project Properties > Lab Resource and Channel Properties. Some of the lab resource and channel property types such as Operating System and Installed Software > Browsers will already exist if you are using one of the predefined Rational Quality Manager project templates. Form Factor and Location Source will probably not exist in your project so you will create them here. Form Factor will include values of Phone (small), Phablet (medium) and Tablet (large). Location Source will include three possible values: GPS, WiFi and None.
Define your channels based on the lab resource properties. Only use the properties that are relevant and make a difference in how you will perform the tests. These properties will be used to distinguish between channels. Clearly, the OS is a distinguishing property – mobile devices will run Android or iOS in your testing scope while desktop devices run Windows or MacOS. The location source for mobile devices will be GPS or WiFi and None for desktop devices.
The form factor will be an important property to track for platform coverage, but the JKE mobile app does not do anything unique based on screen size. So the script that you follow to test on a tablet versus a phone will be the same. This indicates to you that your channels do not need to include the form factor property in their definitions.
You create two channels: Mobile and Desktop. The Mobile channel is defined by the Operating System values of Android or iOS and Location Source properties of GPS or WiFi. The Desktop channel is defined by an Operating System value of Mac OS or Windows and a Location Source of None. (Note that this is a somewhat simplified example. You may very well need to consider various versions of each of these operating systems as well, but this article will not address that level of detail.)
|Desktop||Operating System||Mac OS
The project is now configured for your testing effort. Now you can begin creating artifacts within the project.
The Test Environments
Create a test environment with a name like iOS_Phone_GPS. In the Lab Resource Descriptions section, define the environment as a physical machine running the iOS operating system with a GPS location service on a Phone form factor.
When you save the Test Environment, you will be prompted to synchronize the environment with channels. Click Yes and you will see the Mobile channel is related to the environment.
Repeat the process to create an additional Mobile Test Environment such as Android_Tablet_WiFi.
You will also want to define a couple desktop test environments based on operating system, memory and location source, for example Windows_8GB_None and MacOS_4GB_None.
The Test Plan
Add all of your test environments to the test plan in the Test Environments section. Associating the test environments to the test plan is effectively saying that testing against these environments is “in plan”.
The Test Case
For this example, your test case will be named Contribute to Local Charities and it will be associated to the test plan.
The Test Case Execution Records
Test Case Execution Records can be generated from various places in the RQM user interface. One place is the Test Case Execution Records section of the test case. From here, use the Generate New Test Case Execution Records wizard. Select the Money that Matters test plan and Mobile channel on the left. The list of environments will be filtered down to only those related to the Mobile channel.
You would repeat the process for each Channel in your plan. In this example, you will have four Test Case Execution Records for the test case. Each TCER represents one combination of test case and test environment you wish to test against.
The Test Script
You will create a single test script for the Contribute to Local Charities test case. This test script will define the basic flow of the operations required to test the functionality. You won’t concern yourself with the unique aspects of each channel just yet.
All of the steps in this simple test will be the same for either the Desktop or Mobile channel except step 3 “Locate nearby charities”. This is where the keyword comes in.
In RQM version 4.0.4, you can create keywords directly from the action menu of the steps in an existing test script. In fact, you can multi-select several steps. The selected steps will be moved to a separate script that is associated to the keyword and the steps in the original script will be replaced with a reference to the keyword.
In the example script, you would use the action menu for step 3 and select Create New Keyword from 1 step(s).
This operation creates the Locate nearby charities keyword and creates a new test script that contains that one simple step.
The Keyword scripts for each channel
You need a unique script that implements the Locate nearby charities keyword for each channel. You create each of these scripts independently.
Associate keyword scripts to keyword
Associate these short scripts with the keyword. Each channel must have at least one script associated with it and one script must be designated as the default for that channel.
Everything comes together when you run the tests. When you run the Test Case Execution Records, you select the Channel you want to run them under. Rational Quality Manager uses that information to choose the channel-dependent keyword implementations to use during the test run.
What you now have is a configuration that uses one script to test four environments across two channels with the only item unique to the channel being the implementation of the keyword.
Consider how this would have been done without keywords. The unique steps in the procedure for the two channels would have required you to create and maintain a script for the Mobile channel and another for the Desktop channel. The one-to-many relationship between test case and test script would have allowed you to associate both scripts to the same test case, but you would need to constantly be aware of which script applies to which test environment to avoid inadvertently running the wrong script.
Admittedly you didn’t save a great deal of work in this simple example, but in real projects where you may have hundreds of test scripts that need to be applied across numerous channels, the savings in construction effort and maintenance could be substantial.