Final two videos in the Test Case Execution Records series

Dan has finished the final two videos in his series on Test Case Execution Records.  You can find them on YouTube following the links below:

Test Case Execution Records Part 5:
Generating Test Case Execution Records: Variations
Generating Test Case Execution Records: Variations examines generating test case execution records without specifying iterations, test environments, and a test plan.

Test Case Execution Records Part 6:
Running Test Case Execution Records and Generating Test Results
Running Test Case Execution Records and Generating Test Results shows how to:

  • Run tests using generated test case execution records and completing their test scripts.
  • Generate multiple test results in bulk without executing test scripts.
  • Find all execution records for a single iteration that have failed, passed, or not been run.
  • Work with test case execution records in a test plan as well as in a test case.

This series of videos is an excellent resource for anyone adopting Rational Quality Manager.

Rational Quality Manager Test Case Execution Record video series

Test Case Execution Records really are the heart and soul of Rational Quality Manager.  TCER’s are what make test case reuse possible across multiple environments and releases.  I talked a bit about them in my post on Keywords and Channels.

TCER’s are also probably the most misunderstood aspect of Rational Quality Manager.  You can get by to a point ignoring execution records, but until you get a grasp on TCER’s, you probably aren’t utilizing RQM to its potential.

Dan Chirillo – a very skilled IT Specialist in the IBM Rational services organization – has been working on a video series that very clearly explain what a Test Case Execution Record is, why it is so important, and how to use it effectively.  This is a work-in-progress, but he has four of his planned six videos out on YouTube.  I highly recommend you take the time to watch them.  Even experienced RQM users will find something useful in these videos.

Part 1: Understanding and Managing Test Case Execution Records
This video is the first in a series of videos aimed at explaining how you can use test case execution records to plan, execute, and report on testing. Part 1, Why You Need to Understand and Manage Test Case Execution Records, introduces test case execution records and explains why they are so important.

Part 2: Exploring Test Case Execution Records
Exploring Test Case Execution Records explores existing test case execution records and introduces different ways to generate them.

Part 3: Generating Test Case Execution Records for a Single Test Case
Generating Test Case Execution Records for a Single Test Case shows how to:

  • Set the iterations, owner, and test environments to generate test case execution records for.
  • Confirm that the execution records were added to the test plan and are reflected in the execution status report.
  • Determine how many tests are planned for a single iteration by filtering the test case execution records list in a test plan.

Part 4: Generating Test Case Execution Records for All Test Cases In a Test Plan
Generating Test Case Execution Records for All Test Cases In a Test Plan shows how to:

  • Select which of a test plan’s test cases to generate test case execution records for.
  • Set the iterations and test environments for the execution records.
  • Confirm that the execution records were added to the test plan and are reflected in the execution status report.
  • Determine how many tests are planned for a single iteration by filtering the data in the execution status report.

Parts 5-6, coming soon, will examine generating test case execution records without a test plan, iteration, or test environment, and show how to run test case execution records.  I’ll update this post with links when they become available.



Rational Quality Manager 4.0.4 – Keywords and Channels

Let me start with full disclosure – only a few things about Keywords and Channels are new in 4.0.4.  Much of what you will see in this post came about in version 4.0.3.  But I believe much of the new stuff in 4.0.3 related to channels and keywords went largely unnoticed – partially because people struggle to see why channels are anything different than environments we already had.  Perhaps this post can shed a little light on that by way of an example.

Let’s say you are a tester on the JKE Money that Matters project.  In your new release, JKE is adding the ability for account holders to contribute a portion of the dividends from their interest-bearing account to a charity of their choice. Research has shown that people are more likely to contribute to charities close to home so JKE is adding the ability to find nearby organizations to which the account holder can contribute. Your job will be to use a combination of artifact types in Rational Quality Manager to test this functionality across the various platforms and environments that customers might use while maximizing reuse on your test project.

There are many ways you could approach this problem. In this article, I will attempt to take an artifact-centric view from a more-or-less top-down perspective.

Test Cases

Yes, test cases are the fundamental units of testing, but it is amazing to me how each testing team I encounter seems to have a different definition of what a test case is. I don’t intend to get into that philosophical debate here so let’s just agree that a test case represents one scenario of activity through the application under test. For JKE, maybe the test case is “Contribute to a nearby charity“. Notice I didn’t say “Contribute to a nearby charity using my mobile phone“. You want to keep your definition of the test case as devoid of implementation details as possible. “Why would you want to do that?”, you ask. Well, the test case is a planning and design asset. If you can use the test case to describe the what and why you want to test without getting into the how you are going to test it, you can start planning and designing the test case very early. The gory details of how the test will be performed is kept for the test script. This differentiation is really important to understand if you are going to be effective using RQM. This relationship between test case and test script enables significant reuse of test assets.


Ok, back to our story… Let’s consider that JKE account holders can access their bank through their PC browser or though their mobile device. They probably have other methods as well, but let’s just focus on those two for now. RQM calls these methods “channels”. Your test team has decided you must support two channels for your testing – Desktop and Mobile.

So you want to have a single, rather generic test case of “Contribute to a nearby charity”. That is appealing from a planning standpoint, but you know there are some unique aspects of each channel. For example, the JKE mobile app will leverage the device’s ability to locate itself using GPS or WiFi to know what charities are nearby. However the desktop user will need to enter a zipcode to tell the JKE system where to look for charities. When you get down to implementing this generic test case, the details of how it is done on each channel will diverge. How do you manage that uniqueness?

Test Scripts

As mentioned earlier, the test script defines the how. The test script is the implementation of the what defined in the test case. The one-to-many relationship of test case to test script is one of the most powerful aspects of RQM. You can have a single test case that has multiple implementations associated with it in the form of multiple test scripts. So in our example, you will need to have two test scripts – one for the Desktop channel and one for the Mobile channel… or will you? This is where Keywords come into play.


“Finally, he’s getting around to the keywords”, you say.  Hopefully that preamble has provided a bit of context that will help show the value of keywords.  Think of keywords as modular building blocks of scripts. Scripts can be constructed by stacking these building blocks one on top the other with a little glue in between. Furthermore, keywords can have alternate implementations that can be associated to channels. Without keywords, you would need two separate scripts for the two channels you are testing. With keywords, you can create one script with the unique portions of the script sectioned out as keywords with interchangeable parts. The whole goal of this is to continue to drive down reuse to the lowest level possible. An example will be useful to solidify these concepts, but stick with me while I introduce one more concept first.

Test Environments

Of course, you realize you can’t simply pick one desktop configuration and one mobile configuration, run a few tests on each and call it good. You will need to test against several combinations of desktop OS versions and desktop browsers. You need to test against several combinations of mobile OS versions and mobile form-factors. You may need to consider many combinations of properties on each channel besides OS, browser and form-factor. Mobile especially opens the flood gates to various environment parameters. So considering channels alone is not sufficient. For example, as part of this test case on the Mobile channel, you need to make sure the Find Local Participating Charity functionality works. But the mobile device has two different ways to derive its location – from GPS or from WiFi. The reason you are conducting the test (Test Case) is exactly the same. The steps required to perform the test (Test Script) are exactly the same. But for platform coverage and reporting purposes, you need to ensure you have tested on mobile devices using both location methods. This is a situation where channels are simply not sufficient to define the coverage model.

Test Environments define the specifics of the device being used for testing. In our example, you may decide to test on various combinations of iOS and Android devices configured to use GPS or Wifi locations. Each of these combinations of properties is called a Test Environment. You will want to define all the Test Environments you will test against as part of your test planning process so you can ensure you have considered and documented all the combinations you will be testing.

The InfoCenter tells you that channels are subsets of test environments. I suppose it depends on your frame of reference, but I consider them to be supersets of test environments. Test environments are more specific. Channels are more generic. For example, you may have a channel of Mobile, but your test environments might be iOS_Phone_GPS and Android_Tablet_WiFi. Both of these test environments fall under the Mobile channel.

The Example

Let’s take a look at what that might look like in Rational Quality Manager.

Enabling Channel Support

First, you need to enable Channel support in Project Properties > Execution Preferences.


The Lab Resource Properties

Next, as part of your platform coverage planning, you define the necessary lab resource and channel properties for the project. This is done by a project administrator in Manage Project Properties > Lab Resource and Channel Properties. Some of the lab resource and channel property types such as Operating System and Installed Software > Browsers will already exist if you are using one of the predefined Rational Quality Manager project templates. Form Factor and Location Source will probably not exist in your project so you will create them here. Form Factor will include values of Phone (small), Phablet (medium) and Tablet (large). Location Source will include three possible values: GPS, WiFi and None.


The Channels

Define your channels based on the lab resource properties. Only use the properties that are relevant and make a difference in how you will perform the tests. These properties will be used to distinguish between channels. Clearly, the OS is a distinguishing property – mobile devices will run Android or iOS in your testing scope while desktop devices run Windows or MacOS. The location source for mobile devices will be GPS or WiFi and None for desktop devices.

The form factor will be an important property to track for platform coverage, but the JKE mobile app does not do anything unique based on screen size. So the script that you follow to test on a tablet versus a phone will be the same. This indicates to you that your channels do not need to include the form factor property in their definitions.

You create two channels: Mobile and Desktop. The Mobile channel is defined by the Operating System values of Android or iOS and Location Source properties of GPS or WiFi. The Desktop channel is defined by an Operating System value of Mac OS or Windows and a Location Source of None. (Note that this is a somewhat simplified example. You may very well need to consider various versions of each of these operating systems as well, but this article will not address that level of detail.)

Channel Property Valid values
Mobile Operating System Android
Location Source GPS
Desktop Operating System Mac OS
Location Source None


The project is now configured for your testing effort. Now you can begin creating artifacts within the project.

The Test Environments

Create a test environment with a name like iOS_Phone_GPS. In the Lab Resource Descriptions section, define the environment as a physical machine running the iOS operating system with a GPS location service on a Phone form factor.


When you save the Test Environment, you will be prompted to synchronize the environment with channels. Click Yes and you will see the Mobile channel is related to the environment.

Related Channel

Repeat the process to create an additional Mobile Test Environment such as Android_Tablet_WiFi.


You will also want to define a couple desktop test environments based on operating system, memory and location source, for example Windows_8GB_None and MacOS_4GB_None.



The Test Plan

Add all of your test environments to the test plan in the Test Environments section. Associating the test environments to the test plan is effectively saying that testing against these environments is “in plan”.


The Test Case

For this example, your test case will be named Contribute to Local Charities and it will be associated to the test plan.


The Test Case Execution Records

Test Case Execution Records can be generated from various places in the RQM user interface. One place is the Test Case Execution Records section of the test case. From here, use the Generate New Test Case Execution Records wizard. Select the Money that Matters test plan and Mobile channel on the left. The list of environments will be filtered down to only those related to the Mobile channel.


You would repeat the process for each Channel in your plan. In this example, you will have four Test Case Execution Records for the test case. Each TCER represents one combination of test case and test environment you wish to test against.


The Test Script

You will create a single test script for the Contribute to Local Charities test case.  This test script will define the basic flow of the operations required to test the functionality.  You won’t concern yourself with the unique aspects of each channel just yet.


The Keyword

All of the steps in this simple test will be the same for either the Desktop or Mobile channel except step 3 “Locate nearby charities”. This is where the keyword comes in.

In RQM version 4.0.4, you can create keywords directly from the action menu of the steps in an existing test script. In fact, you can multi-select several steps. The selected steps will be moved to a separate script that is associated to the keyword and the steps in the original script will be replaced with a reference to the keyword.

In the example script, you would use the action menu for step 3 and select Create New Keyword from 1 step(s).


This operation creates the Locate nearby charities keyword and creates a new test script that contains that one simple step.

The Keyword scripts for each channel

You need a unique script that implements the Locate nearby charities keyword for each channel.  You create each of these scripts independently.

Desktop version of Locate Nearby Charities Script

Mobile version of Locate Nearby Charities Script

Associate keyword scripts to keyword

Associate these short scripts with the keyword.  Each channel must have at least one script associated with it and one script must be designated as the default for that channel.



Everything comes together when you run the tests.  When you run the Test Case Execution Records, you select the Channel you want to run them under.  Rational Quality Manager uses that information to choose the channel-dependent keyword implementations to use during the test run.



What you now have is a configuration that uses one script to test four environments across two channels with the only item unique to the channel being the implementation of the keyword.


Consider how this would have been done without keywords. The unique steps in the procedure for the two channels would have required you to create and maintain a script for the Mobile channel and another for the Desktop channel. The one-to-many relationship between test case and test script would have allowed you to associate both scripts to the same test case, but you would need to constantly be aware of which script applies to which test environment to avoid inadvertently running the wrong script.

Admittedly you didn’t save a great deal of work in this simple example, but in real projects where you may have hundreds of test scripts that need to be applied across numerous channels, the savings in construction effort and maintenance could be substantial.

Rational Quality Manager 4.0.4 – Manual Test Editor

Rational Quality Manager’s manual test editor got a bit of a makeover in version 4.0.4.  Some updates are pretty obvious, some are not so obvious.

Obvious – Stack Top to Bottom


Since time began, RQM manual tests have consisted of a Step column, a Description column, an Expected Results column and a Mystery column.  The Mystery column really contains icons for attachments, comments and assisted data, but you really don’t know that until you add the first one of those.

4.0.3 manual test row

Optionally, you could use a toolbar icon to toggle on and off an additional column that contains the full text of your comment.

4.0.3 comment column

Why that toggle tool was implemented as two checkboxes instead of a simple toggle button is a mystery.  Perhaps RQM Dev got a really good deal on checkboxes or something.


Now in 4.0.4, you not only have the same “Stack Left to Right” orientation, but you have the option of a “Stack Top to Bottom” orientation which puts the Expected Results below the Description column.  The Mystery column remains on the right side, but when you hover in it, three tool icons appear for attachments, comments and assisted data.

4.0.4 stacking

This will probably be a matter of personal preference, but this orientation gives you more horizontal space to enter and read your description and expected results.

Not So Obvious – No More Comments Column

Comments now appear directly in the mystery column so there is no more toggling on and off of the comment column.  Darn, I guess we will never know why they used checkboxes instead of a toggle.


Not So Obvious – Column Headers in “Stack Right to Left”

In the traditional Right to Left orientation, the Description and Expected Results headers are now stuck to the toolbar so they don’t scroll away.

4.0.4 headers

Obvious – Maximize Steps Display

You have a bunch of steps to enter into a manual test.  That’s all you will be doing for the next 20 minutes.  You don’t need to see the Formal Review section.  You don’t need to see the State, Originator or Owner.  You just want as much space as possible to edit your test.

Presenting the Maximize Steps Display icon:

4.0.4 Maximize

The steps editor now consumes everything except the Related Information area on the right side, if you have that turned on.

Obvious – Multiple Step Selection

Using Control-Click, you can select multiple steps in a test.

Not So Obvious – What to do then

One thing you can do with multiple test steps is copy them to the step clipboard.

4.0.4 Copy multiple steps

The big change here is the removal of the step clipboard window.  This is a big improvement as you can copy/paste steps without losing screen real estate to a window that really didn’t add that much value.  Instead of dragging and dropping steps to and from the clipboard window, you just use the copy/paste in the step menus.

Obvious – On Demand views

The Test Data, Execution Variables and Keyword view windows are all “floating” and can be resized.  The old fixed-sized windows really didn’t lend themselves to scaling to large projects.

4.0.4 floating windows

Not So Obvious – Compact View

Before 4.0.4, the Compact view setting hid all graphics from the description and expected results columns for all steps except the step you have selected in an effort to compress the information into less screen real estate.  It was moderately effective.

In 4.0.4, the Compact view does much the same thing in Stack Left to Right mode.  In Stack Top to Bottom mode, it also hides the Expected Results cell.  One handy addition is that each step gets a “twistie” that can be used to expand that step individually.  So you can see the graphics for the step you are working plus the one before, for example.

4.0.4 compact

Obvious – Keyword Support

Keyword support is maturing in RQM.  Keywords will be a major part of our Multi-Channel testing initiative going forward.  The new keyword support in 4.0.4 deserves its own blog post so I will save that for another day.

Rational Quality Manager 4.0.4 – First Look

I’ll be the first to admit I have been pretty slow on the uptake of CLM 4.0.4.  I have been kept pretty busy with a lot of test automation opportunities and been focusing my time on Rational Test Workbench lately.  I got as far as installing CLM 4.0.4 the other day and then got pulled away again.  Well I finally got a few minutes to browse the New and Noteworthy page today and to tinker around a bit in the tool.  Wow, I’m a bit biased, I will admit, but Rational Quality Manager came out on top in this release!

I will try to blog about some of the new features over the next week or so, but I wanted to start with a gem I found buried in the details:

Displaying short names of links instead of numbers in table views

This all comes down to usability.  One of the most powerful capabilities of the Rational Collaborative Lifecycle Management system is the ability to link artifacts one to another.  With these links, you can know which test cases validate which requirements.  You can see which development plans are tested by which test plans.  There are a multitude of places where these links show up, particularly in tables when you choose to view Traceability.  Let’s take a test case list in a test plan, for example.

4.0.3 test cases

This traceability has huge benefits.  And as long as the relationships have been one-to-one or somewhere close, you are OK.  But when you start to scale up to a one-to-many relationship, our table displays start to break down.  Now to be fair, it’s not easy to cram a lot of relationship information into a table such as the one above.  But prior to 4.0.4, if you had more than one link per cell, we would just list a stream of links with a one-based index.  You had no idea just looking at them as to what the links pointed to.

4.0.3 test cases multiple requirements

Oh sure, you could hover over each one and find the details in the hover window or you could click on the test case and go to the Requirements section to see the list, but that takes time and effort.

Now here is the same table display in 4.0.4.

4.0.4 test case multiple requirements

The difference may be subtle at first, but the requirement links are no longer just a one-based index.  Those numbers are the “webID” numbers – the unique numbers associated to every artifact in the repository.  Everyone gets attached to their favorite requirements by number.  Or you may get to know a pesky defect by its number rather than its name.  Now you can pick them out of the crowd easily.  The links consume very little more space than before, but they provide so much more value.