My Internet of Things and MobileFirst adventure – Part 8: Build the mobile app

This is part of a series of posts related to My Internet of Things and MobileFirst adventure.  An index to all posts can be found at the end of the first post.

Finally, we come to creating the mobile app that will tie this all together.  This is going to involve a lot of steps so I will cover the basic outline of what I did without getting into every line of code.  You can download all the source files for this project (including the Node-RED flows) from IBM Bluemix DevOps Services if you want to see the details.

But at a high level, here’s what needs to be done:

  1. Add the IBM Mobile Push service to the Bluemix application.  This will manage the push notifications with APNS (Apple Push Notification Service).
  2. Add nodes to the Bluemix Node-RED flow to initiate a push request when it receives the visitorAlert event.  This is how Bluemix will tell the mobile app that the doorbell was pressed.
  3. Create a MobileFirst project.  This will house an adapter that will retrieve the file from Cloudant as well as generate the SDK library the iOS app will use.
  4. Create a new Xcode project to house the native iOS app
  5. Add the MobileFirst SDK to the Xcode project.
  6. Add the Bluemix SDK to the project.
  7. Write code

Add the IBM Mobile Push service to the Bluemix application

  1. Open the Bluemix application.
  2. Click ENABLE APP FOR MOBILE and then YES and then CONTINUE.
  3. Click ADD A SERVICE OR API  and select Push from the Mobile category, then click USE and then RESTAGE.  I should point out that there is a Push iOS 8 service as well.  I chose not to use it and stuck with the older Push service.  Part of the reason was that Push iOS 8 depends on the newer Advanced Mobile Access and I already knew using it would require customizing my Bluemix Node-RED instance to support it.  I took the path of least resistance, but at some point, I will need to adopt AMA to keep up with the times.
  4. At the top of the application’s overview page, you will now see the app’s registration credentials.  You will need these later to tell the mobile app how to connect to Bluemix.

Configuring Apple Push is not for the faint of heart.  First of all, it requires you to have an Apple Developer license.  You can buy a personal license for $99 per year if you don’t have access to a corporate license.  Once you have a license, you must use the Apple Developer Portal to create a Device ID, an App ID and a Provisioning Profile as well as an SSL certificate that must be used by any application that wants to request APNS to send a push notification.  That process is out of scope for this blog, but you can read about it in the iOS Developer Library.

Once you have your SSL key, you will need to register it with the IBM Bluemix Push Service.

  1. Go to your Bluemix application and click the Push service.
  2. Select the Service Mode Sandbox or Production, depending on if you have a Development or Production SSL key from Apple.

    Bluemix Push Dashboard

    Bluemix Push Dashboard

  3. Click EDIT under APNS.
  4. Upload your p12 SSL key and enter the password for it.

It is possible to use the IBM Push dashboard to send out test notifications.  This can be helpful in debugging the process.

Bluemix Push Notification tab

Bluemix Push Notification tab

Add nodes to Bluemix Node-RED to initiate Push notifications

Node-RED on Bluemix includes nodes for IBM Push.  However, I was not able to get them to work as expected.  So I chose to go about this through brute force with HTTP nodes.  This has the advantage of demonstrating more of the details of how the Bluemix Push SDKs work, but I should really revisit this someday.

  1. Open your Bluemix Node-Red flow editor.
  2. Add a function node.  Its input should be connected to the output of the visitor Alert Event IBM IoT In node.  Name it send notification.
    1. Add the following code.  Yes, I am cheating.  the picture filename is not a URL, but I got lazy and decided to use the URL element to pass it.
      var newmsg = {};
      newmsg.headers = { 
          "IBM-Application-Secret" : "<your-secret>" };
      newmsg.payload = {
          "message" : {
              "alert"    : "Someone is at the door!  " +
                     "Would you like to see who it is?",
              "url"      : msg.payload.pictureFilename
          },
          "settings" : {
              "apns"   : {
                  "sound" : "doorbell/Doorbell.mp3"
              }
          },
          "target" : {
              "platforms" : ['A']
              }
      };
        
      return newmsg;
      
    2. Replace <your-secret> with the value of your application secret code from the application’s overview page.
  3. Add an http request function node (note – this is not an http input nor http output node).  Its input should be connected to the output of the send notification function node.
    1. Set its method to POST.
    2. Set the URL to https://mobile.ng.bluemix.net:443/push/v1/apps/<your-appId>/messages where <your-appId> is the application appId found on the overview page.
    3. Set the return type to UTF-8 String.
  4. Add debug nodes if you want.

    Final Bluemix Node-RED flow

    Final Bluemix Node-RED flow

Create a MobileFirst project

I’m going to leverage the IBM MobileFirst Platform Foundation for this mobile application.  This is, of course, not necessary to use Bluemix services.  You can access Bluemix services directly from an iOS native app.  I’m not going to demonstrate them in this tutorial, but the MobileFirst Foundation provides a long list of capabilities that make developing applications easier and more secure.  What it does mean is that there will be a server-side component called an adapter that will do the actual Cloudant communication.  The iOS app will use the MobileFirst SDK to invoke routines on the adapter.

To create the MobileFirst project, you need to have either the MobileFirst Studio extension to Eclipse or the MobileFirst command line interface installed.  I’m going to be using the command line interface here.

  1. Create a new project from the command line with ‘mfp create DoorbellIOSNative‘.
  2. cd DoorbellIOSNative
  3. Start the mfp Liberty server with ‘mfp start‘.
  4. Create an adapter that will retrieve the Cloudant data with ‘mfp add adapter CloudantAdapter --type http‘.
    1. Implement the adapter logic.  The simplest way to do that is to copy the CloudantAdapter.xml and CloudantAdapter-impl.js files from this MobileFirst tutorial.
    2. Be sure to edit the domain, username, and password values in the xml file so they reflect the values from your Bluemix overview page.
  5. Add a new iOS native API using $ mfp add api APIiOS -e ios.  There are two configuration files you need to edit in the project’s /apps/APIiOS folder.  Use this MobileFirst tutorial as a guide:
    1. The worklight.plist file contains several settings that your iOS app will need to know in order to connect to the MobileFirst server which hosts the adapter.
    2. The application-descriptor.xml file defines the application name, bundleID and version for the MobileFirst server.
  6. Once you have finished configuring these files, deploy the project to the MobileFirst server with ‘mfp push‘.

Create an Xcode project

The Xcode project will contain your iOS native app source code.  Create a blank project and create an app with a UIImageView to contain the picture.  If you are not that comfortable coding, you can download my project from IBM Bluemix DevOps Services.  Developing the app UI is not really the focus here, but rather showing how you configure the project to access the MobileFirst adapter and IBM Push from Bluemix.

Add the MobileFirst SDK to the Xcode project

When you created the MobileFirst project above, two things were created that you will need to add to your Xcode project – the worklight.plist file and the WorklightAPI folder.  You also need to add several frameworks to the Xcode project to get access to the libraries you will need.  The IBM MobileFirst Platform Foundation Knowledge Center does a good job of explaining how to do this.

Add the Bluemix SDK to the project

The SDKs you need for IBM Push are included with a bundle of SDKs for Bluemix.  You can download them from the IBM Bluemix Docs.  The only tricky part here is that these libraries are Objective-C libraries.  Since I coded my iOS app in Swift, I had to create an Objective-C Bridging Header.  This is a really easy way to expose Objective-C code to Swift applications.

Write code

There are multiple places in the AppDelegate and ViewController where I needed to add custom code.  The two main functions I needed to deal with were configuring the app to receive and handle push notifications and then to use the MobileFirst adapter to retrieve the image.

Push notification configuration

didFinishLaunchingWithOptions

This is sort of the main routine for an iOS application.  It is invoked when the app launches.  Here is where you need to have the app register for push notifications with the following code:

        // ========================================
        //  Register for push notifications
        // ========================================
        // Check to see if this is an iOS 8 device.
        let iOS8 = floor(NSFoundationVersionNumber) > floor(NSFoundationVersionNumber_iOS_7_1)
        if iOS8 {
            // Register for push in iOS 8
            let settings = UIUserNotificationSettings(forTypes: 
                UIUserNotificationType.Alert | 
                UIUserNotificationType.Badge | 
                UIUserNotificationType.Sound, categories: nil)
            UIApplication.sharedApplication().registerUserNotificationSettings(settings)
            UIApplication.sharedApplication().registerForRemoteNotifications()
        } else {
            // Register for push in iOS 7
            UIApplication.sharedApplication().registerForRemoteNotificationTypes(
                UIRemoteNotificationType.Badge | 
                UIRemoteNotificationType.Sound | 
                UIRemoteNotificationType.Alert)
        }

Notice that the way you register for push changed with iOS 8 so the code here first determines the OS level, then does what it needs to do.

didRegisterForRemoteNotificationsWithDeviceToken

This method is invoked by the framework when the app has successfully registered with APNS.  Here is where you want to initialize the Bluemix and Push SDKs.  You would replace the placeholders with the route, appId, and appSecret from your Bluemix application.

        // Initialize the connection to Bluemix services
        IBMBluemix.initializeWithApplicationId(
            "<your-app-id>",
            andApplicationSecret: "<your-app-secret>",
            andApplicationRoute: "<your-app-route>")
        
        pushService = IBMPush.initializeService()
        if (pushService != nil)  {
            var push = pushService!
            push.registerDevice("testalias",
                withConsumerId: "testconsumerId",
                withDeviceToken: self.myToken).continueWithBlock{task in
            
                if(task.error() != nil) {
                    println("IBM Push Registration Failure...")
                    println(task.error().description)
                    
                } else {
                    println("IBM Push Registration Success...")
                }
                return nil
            }
        } else {
            println("Push service is nil")
        }

didReceiveRemoteNotification

This method is invoked by the framework whenever a push notification is received.  The payload of the push notification is passed in in the userInfo object.  In a series of ‘if let’ statements, I extract the bits of information I need.  Then I determine if the app was in the background or inactive.  If so, that means the user already saw the notice, read it and chose to invoke it by tapping on it.  In that case, go straight to the code that gets the picture from Cloudant and loads it into the UIImageView.  If the app was in the foreground, create a message alert and only load the image if the user taps OK.

        if let aps = userInfo["aps"] as? NSDictionary {
            if let alert = aps["alert"] as? NSDictionary {
                if let fileName = userInfo["URL"] as? NSString {
                    if let message = alert["body"] as? NSString {
                        if let sound = aps["sound"] as? NSString {
                            if (application.applicationState == UIApplicationState.Inactive ||
                                application.applicationState == UIApplicationState.Background) {
                                    println("I was asleep!")
                                    self.getPicture(fileName)
                                    
                            } else {
                                var noticeAlert = UIAlertController(
                                    title: "Doorbell",
                                    message: message as String,
                                    preferredStyle: UIAlertControllerStyle.Alert)
                                
                                noticeAlert.addAction(UIAlertAction(
                                    title: "Ok",
                                    style: .Default,
                                    handler: { (action: UIAlertAction!) in
                                        println("User wants to see who's at the door")
                                        println(fileName)
                                        self.getPicture(fileName)
                                }))
                                noticeAlert.addAction(UIAlertAction(
                                    title: "Cancel",
                                    style: .Default,
                                    handler: { (action: UIAlertAction!) in
                                        println("Handle Cancel Logic here")
                                }))
                                
                                let fileURL:NSURL = NSBundle.mainBundle()
                                    .URLForResource(
                                        "Doorbell", 
                                        withExtension: "mp3")!
                                
                                var error: NSError?
                                self.avPlayer = AVAudioPlayer(
                                    contentsOfURL: fileURL, 
                                    error: &error)
                                if avPlayer == nil {
                                    if let e = error {
                                        println(e.localizedDescription)
                                    }
                                }
                                
                                self.avPlayer?.play()
                                
                                // Display the dialog
                                self.window?.rootViewController?
                                    .presentViewController(
                                        noticeAlert, 
                                        animated: true, 
                                        completion: nil)   
                            }   
                        }
                    }
                }
            }
        }

Retrieve image from Cloudant

didFinishLaunchingWithOptions

When the app first starts up, connect to the MobileFirst Platform Server.  I’m leaving out some details here, but basically you need to call wlConnectWithDelegate.  That method takes a listener parameter, but that listener is really trivial in my case.

        let connectListener = MyConnectListener()
        WLClient.sharedInstance().wlConnectWithDelegate(connectListener)

getpicture

This routine, called when a user chooses to see the picture through the push notifications, will use the MobileFirst adapter procedures to search, then retrieve the image from Cloudant.

        // Search Cloudant for the document with the given fileName
        let searchRequest = WLResourceRequest(
            URL: NSURL(string: "/adapters/CloudantAdapter/search"), 
            method: WLHttpMethodGet)
        let queryValue = "['pictures','ddoc','pictures',10,true,'fileName:\"" + 
            (fileName as String) + "\"']"
        searchRequest.setQueryParameterValue(
            queryValue,
            forName: "params")
        searchRequest.sendWithCompletionHandler { 
           (WLResponse response, NSError error) -> Void in
            if(error != nil){
                println("Invocation failure. ")
                println(error.description)
            }
            else if(response != nil){
                let jsonResponse = response.responseJSON
                if let rows = jsonResponse["rows"] as AnyObject? as? NSArray {
                    if (rows.count > 0) {
                        if let row = rows[0] as AnyObject? as? Dictionary<String,AnyObject> {
                            if let doc = row["doc"] as AnyObject? as? Dictionary<String,AnyObject> {
                                if let payload = doc["payload"] as AnyObject? as? Dictionary<String,AnyObject> {
                                    var base64String : String = payload["value"] as! String
                                    
                                    // Strip off prefix since UIImage doesn't seem to want it
                                    let prefixIndex = base64String.rangeOfString("base64,")?.endIndex
                                    base64String = base64String.substringWithRange(
                                        Range<String.Index>(
                                            start: prefixIndex!, 
                                            end: base64String.endIndex))
                                    // Strip out any newline (\n) characters
                                    base64String = base64String.stringByReplacingOccurrencesOfString(
                                        "\n", 
                                        withString: "")
                                    
                                    // Convert to NSData
                                    let imageData = NSData(
                                        base64EncodedString: base64String, 
                                        options: NSDataBase64DecodingOptions.IgnoreUnknownCharacters)
                                    
                                    // Create an image from the data
                                    let image = UIImage(data: imageData!)
                                    
                                    // Stuff it into the imageView of the ViewController
                                    AppDelegate.vc!.updatePicture(image!)
                                    
                                }
                            }
                        }
                    }
                }
            }
        }

Testing

With all this in place, I build the app in Xcode and deploy it to my iPhone (remember, it must be a physical device because APNS doesn’t work with a simulator).  My app screen looks like this:

Doorbell App - Initial Screen

Doorbell App – Initial Screen

I walk up to my door and press the doorbell button.  The Node-RED flow on the Raspberry Pi takes a picture, stores it on the local file system and sends an MQTT message to the IBM Internet of Things Foundation broker.  The Node-RED flow running on Bluemix in the cloud receives the message and tells the Raspberry Pi to upload the picture to the Bluemix Cloudant database.  At the same time, Node-RED uses the IBM Push service, also running on Bluemix, to send a remote notification to the app on my phone.  The app receives the push notification and presents an alert to me.

Doorbell App - Push Notification

Doorbell App – Push Notification

I tap Ok to see who is at the door.  The iOS app uses the IBM MobileFirst SDK to invoke adapter routines to first search for, then retrieve, the picture from the Bluemix Cloudant database.  It then pops that picture onto my mobile screen so I can see who is ringing my doorbell, even if I am hundreds of miles from home and can’t do a thing about it.

Doorbell App - Someone at the Door

Doorbell App – Someone at the Door

Conclusion

Well, that about does it.  This has been quite an adventure.  Clearly this isn’t an app you would put into production.  I have over-engineered several areas and it is far more complicated than a doorbell needs to be.  But we did get a chance to explore several technologies including:

  • Setting up a Raspberry Pi with an external button input and a camera
  • Node-RED visual editor for controlling and interacting with the Internet of Things
  • Bluemix applications and services including Cloudant Databases, Node.js runtimes and mobile Push.
  • Registering and app with Apple Push Notification Service.
  • IBM MobileFirst Platform Foundation for enterprise security and access to Systems of Record such as Cloudant databases
  • Using Xcode to create a native iOS application that receives push notifications from IBM Bluemix Push and retrieves data from Bluemix Cloudant datastores.

What’s next?  Well, IBM just released MobileFirst Platform Foundation version 7.1 which supports deploying the MobileFirst server in a container in Bluemix – built on Docker technology.  Maybe I will see if I can eliminate my local laptop entirely and have everything except the Raspberry Pi in the cloud!

Advertisements

Part 5: Set up the Bluemix application environment

This is part of a series of posts related to My Internet of Things and MobileFirst adventure.  An index to all posts can be found at the end of the first post.

Today I am going to configure my Bluemix environment that will talk to my Raspberry Pi.  I already have a Bluemix account, but you can go to https://ibm.biz/IBM-Bluemix and get your own free account.

Create a Bluemix application

Bluemix provides a number of boilerplates.  Boilerplates are packages of services already bundled together.  This is usually the easiest way to get started.

  1. Click Catalog and then MobileFirst Services Starter to create a new application based on the boilerplate.
  2. Enter a name for your app.  It has to be unique, so consider including your initials or something in the name.  Click Create.
  3. Bluemix will start cranking through the creation process.  First your application will be staged, then created.  Once it says your app is running, the creation process is done.

Add the IBM Internet of Things Foundation service

IBM Bluemix provides a service called Internet of Things Foundation.  This is sort of a registry service and a message broker all in one.  You can register your device in the Internet of Things, then communicate with it through IoT nodes in Node-RED.

  1. From your Bluemix App’s Overview page, click Add a service or API.  Search or scroll to the bottom and select Internet of Things Foundation.
  2. Your app will need to be restaged to add the service.  Wait until App Health indicates your application is running again.

Add the Raspberry Pi to the Internet of Things Foundation

  1. Once the app is running again, click the Internet of Things Foundation tile.
  2. Click Launch Dashboard.
  3. From the Overview tab, click Add a device.
  4. Create a device type.  This can be anything you want, say myPi.
  5. You can optionally choose the attributes you want to assigned for each device of this type.  You can skip this for now if you want.  If you choose to add attributes, you will be asked to supply defaults on the next screen.  You can also add additional metadata in JSON format.
  6. After entering all that information about the device type, you will return to the Add Device wizard.  The only bit of information here that is absolutely required is the Device ID.  That is the device’s MAC address.  To find it, go back to your Raspberry Pi command line and type ifconfig eth0.  The output will look something like this:

    ifconfig output

    ifconfig output

  7. Type the string of characters following the HWaddr (leaving out the colons) into the Bluemix IoT dialog.  In my example, it would look like this:

    IoT MAC Address entry

    ifconfig output

  8. Click through the rest of the screens until you get to the page displaying Your Device Credentials.  You will be shown a box with the Authentication Token.  Copy this down immediately!  You will not be given a second chance to see it!

Configuring the IBM Push Notifications service

Next, I’ll configure the IBM Push Notifications service.  This is one of those cases where in reality, I came back and did this later, but since we are in Bluemix now, let’s set it up here.

The IBM Push Notifications service is, in a sense, a notification broker.  Applications that want to sent push notifications to devices (in my case, the Raspberry Pi), will send message requests to the IBM Push Notifications service via HTTP REST commands.  The IBM Push Notifications service will, in turn, dispatch the request to the notification services for the appropriate platform.  In my case, I an going to be writing the app on iOS, so the IBM Push Notifications service will invoke the APNS (Apple Push Notification Service), which will send a remote notification to my device.  In order to receive the notification, the device app software will need to register with the IBM Push Notifications service.

Configuring Apple Push is not for the faint of heart.  First of all, it requires you to have an Apple Developer license.  You can buy a personal license for $99 per year if you don’t have access to a corporate license.  Once you have a license, you must use the Apple Developer Portal to create a Device ID, an App ID and a Provisioning Profile as well as an SSL certificate that must be used by any application that wants to request APNS to send a push notification.  That process is out of scope for this blog, but you can read about it in the iOS Developer Library.

Once you have your SSL key, you will need to register it with the IBM Bluemix Push Service.

    1. Go to your Bluemix application Overview and click the IBM Push Notifications service.
    2. Click Setup Push.
Bluemix Push Dashboard

Bluemix Push Dashboard

  1. Click Choose file under Apple Push Certificate.  (Note that the process would be similar for Google Cloud Messaging for Android devices.)
  2. Upload your p12 SSL key and enter the password for it.

It is possible to use the IBM Push Notifications dashboard to send out test notifications.  This can be helpful in debugging the process.

That takes care of configuring the Bluemix services in the cloud.  You have two primary services:

Internet of Things Foundation

  • Acts as a broker for MQTT messages between IoT Applications and Devices
  • Maintains registration information for all your Devices

IBM Push Notifications

  • Acts as a broker for remote notifications between requesting applications and the vendor cloud messaging services.
  • Provides a manual Push testing interface

That takes care of the system setup tasks.  In the next post, I will start implementing my solution and configure my Raspberry Pi Node-RED flows to send Push Notifications.  In a later post, I will enable my mobile application to request a picture from the Raspberry Pi and the Raspberry Pi to return the picture using the IoT Foundation.

Part 2: Raspberry Pi software

This is part of a series of posts related to My Internet of Things and MobileFirst adventure.  An index to all posts can be found at the end of the first post.

With the hardware all set up, I was ready to dive into the software on the Raspberry Pi.  I quickly learned that there is a bunch of software available for the Raspberry Pi.  Most of it is really easy to install with ‘apt-get’ command and npm.  I also discovered that the Raspberry Pi Documentation is really helpful.  Here are a few things I started with:

VNC

I knew I wanted to run the GUI from my MacBook and not require an external monitor.  I followed the instructions in the Raspberry Pi Documentation pretty much exactly as written.

I already had VNC Viewer on my MacBook.  I connected to the Pi and had GUI on my MacBook in no time.

Git

Git is a very popular open source version control system.  I could already see I was going to need Git on the Pi in order to get some code I would need, so I installed with

sudo apt-get install git-core

wiringPi

But, I still don’t have a way to monitor my pushbutton or control my LED circuits.  After a little Googling, I came across wiringPi, an access package for the GPIO interface.  It also comes with a simple command line interface, so I would have a way to test out my circuits.  I installed wiringPi using the instructions on the wiringPi install page.

Now, I could go to my command prompt and run gpio commands.  The pin numbering used by wiringPi is not obvious.  There is a whole history there but I’m not going to get into that.  I could read the state of the pushbutton using

gpio read 1

I could control the LED with the following:

gpio mode 6 out
gpio write 6 1
gpio write 6 0

Eclipse

Yes, you can install Eclipse on the Raspberry Pi!  My architecture did not include a Java program written in the Eclipse IDE, but I tried it anyway.  Again, installation is pretty easy with the package manager:

sudo apt-get install eclipse

There are a couple prerequisites, though.

JDK 7

It is best to get the full Oracle Java 7 JDK using the Raspberry Pi Documentation.

Pi4J

Pi4J is an API that gives you simple access to the GPIO pins on the board.  Install it with

curl -s get.pi4j.com | sudo bash

I followed Ian Bull’s tutorial with a few minor modifications to create a simple Java program that monitors the pushbutton and controls the LED.  But to get it to run, I ran into another issue – the underlying wiringPi must be run as root.

Configure Eclipse to run programs as root

There is a really clever Stack Overflow on how to do this.

Node-RED

Ok, now that I was able to interface with the hardware using the command line and a Java program, I was ready to move on to bigger and (hopefully) better things.  I wanted to use Node-RED.  What is Node-RED?  I lifted this description from an IBM page:

Node-RED provides a browser-based UI for creating flows of events and deploying them to its light-weight runtime. With built in node.js, it can be run at the edge of the network or in the cloud. The node package manager (npm) ecosystem can be used to easily extend  the palette of nodes available, enabling connections to new devices and services.

A textual description doesn’t really do it justice.  You really need to see a flow to understand how powerful it can be, so let’s get it installed, but beware – there be dragons here.

Installing Node.js

I Googled around a bit and instead of following the instructions on nodered.org, I jumped into a tutorial I found.  Unfortunately, that tutorial was outdated and lead me down a dead end.  As of this writing, Node-RED does not run on the very latest Pi-compatible version of Node.js.  I would definitely recommend you follow the instructions specific to the Raspberry Pi version you have from the Node-RED documentation.  I won’t repeat those instructions here.

Installing additional Nodes

Node-RED can be extended easily by installing additional nodes.  The best place to look is the Node-RED Library.  I installed node-red-contrib-gpio, but here is where another dragon appeared.  Being relatively unfamiliar with Node.js, I overlooked one very important line in the instructions: “From inside your node-red directory“.  In the case of a local Node.js instance, that means the hidden .node-red directory in my home directory on the Pi.  Until I figured that out, the nodes just wouldn’t show up in Node-RED.  So, the installation instructions should be:

cd ~/.node-red
npm install node-red-contrib-gpio

In the next post, I will look at how I created my first Node-RED flow on the Raspberry Pi before I stared interfacing it to the Internet of Things.

Domains, Environment and Projects – Administration

This is a follow-up to my previous post Domains, Environments and Projects – making sense of it all.

Now that we understand the purpose of Domains and their related artifacts in the Rational Test Control Panel, let’s take a look at how we can make use of them.

Visibility

Again, as described in the KnowledgeCenter, A domain represents a logical grouping of related systems that are part of a real business project and it is the basic unit of management within IBM Rational Test Virtualization Server.  In the last post, I mapped each organizational division to a domain.  This will enable JKE to organize which stubs are published and visible in which domains.  That will help us make only relevant stubs visible to the various teams.

Security Access

Domain-level security is disabled by default.  As an administrator, go into the Administration > Security tab of Rational Test Control Panel and enable it.

Enable Domain-level security

Enable Domain-level security

Once enabled, any user that is not specifically added to a domain will not have access to anything in that domain.

No domains for unauthorized user

No domains for unauthorized user

Users are added to domains in the Administration > Domains and environments area by selecting the domain and then the Users sub-menu and clicking the green add icon.  Users can be given three levels of access: Domain administrator, User and API User.

Add a user to a domain

Add a user to a domain

Now that user will see any available domains in the domain selection box at the upper-right of RTCP.

Domains available to a user appear in the domain selection list

Domains available to a user appear in the domain selection list

With this in place, you not only provide visibility constraints, but also access constraints.  But, access constraints apply not only to users, but to proxies and agents as well.  Since we have not yet configured any proxies or agents for domain-level security, they cannot register to domains or environments within those domains.  If our domain user looks at the Environments tab (was VIE tab prior to v8.6), he will see this message:

No agents registered for this domain

No agents registered for this environment

This brings us to our third use for domains – controlling access to services.

Proxy/Agent access control

Proxy and Agent access control is implemented with security tokens.  An administrator creates a security token which is used by the proxy or agent to authenticate to RTCP.  The security token’s permissions determine which domains the proxy or agent can register with.  The security token is associated to a user when it is created.  The proxy or agent therefore gets the access of the user who corresponds to the security token being presented.

As an administrator user, create a token in the Administration > Security tab.  Select the user to which the security token will be bound and enter a description for the token.

Create user security token

Create user security token

Now provide the security token to the proxy or agent configuration settings.  For proxies, this is done by pasting the security token string into the security-token attribute of the server element of the registration.xml file for the proxy.

Adding the security token to the proxy registration.xml file

Adding the security token to the proxy registration.xml file

For agents, this is done by pasting the security token string into the security-token attribute of the rtcpURL element of the Agent.config file for the agent.

Adding the security token to the Agent.config file

Adding the security token to the Agent.config file

Restart any proxies or agents you have modified.  The proxies and agents started using this security token will be visible in all domains and environments, but they can only be used in domains where the user associated with the security key has permissions to work.

Proxy/Agent visibility

A proxy or agent’s visibility can be restricted as well.  This is done by the domains element in the proxy’s registration.xml or the agent’s Agent.config file.  For example, to cause my proxy and agent to only be visible to the Mobile Banking domain, I would add the following:

Restricting proxies or stubs to a specific domain

Restricting proxies or stubs to a specific domain

You can further restrict a proxy or agent’s access to only given environments within a domain.

Restricting proxies or stubs to a specific environment within a domain

Restricting proxies or stubs to a specific environment within a domain

Wrapping it all up

Now this becomes really powerful.  So consider again our multiple JKE teams all trying to build client applications that leverage the same payment calculator service.  Each team wants to use a stub for the payment calculator service, but they each want their own variant of that service.  The domain architecture of Rational Test Control Panel and the
ability to control which domains the proxies and agents can register into would enable each team to have their own personal proxy/agent pair visible only in their domain.  The members of each team would have permissions to their own domain.  Each team would publish and run their own Payments Calculator stub to their own domain.  Now, without changing the endpoints of any of the client applications, each team’s proxy will route requests from their client to their stub.  There are all sorts of options where multiple domains could share proxies and agents, thereby enabling them to use a common stub.  This could be part of a strategy to use incremental integration testing at an enterprise level.

Domains with separate and shared proxies

Domains with separate and shared proxies

Domains, Environments and Projects – making sense of it all

I was recently supporting an opportunity that involved multiple test teams supporting multiple products that all needed to virtualize common dependent services. It’s not that difficult to visualize how to create, publish and use virtual services in a small, well-contained project, but when you start looking at this from an enterprise perspective, it can turn into a large hairball quickly.

A fictional example

The Auto Loan application in the Personal Banking division of JKE Bank had a need for computing loan payments. It’s a pretty simple operation – you provide the loan amount, term of the loan and interest rate. In return, you get a value for the monthly payment amount. So the Personal Banking Auto Loan team encapsulated that capability in a web service, published it to the JKE world and built the Auto Loan application to leverage it.

Pretty soon, the Personal Banking division software architects decided it didn’t make sense to duplicate this functionality across all the applications in their portfolio, so they decreed that all Personal Banking apps that need to calculate payments must use the common Payment Calculator service. The two applications delivered by the Personal Banking team leveraged a common, consistent payment calculator.

At an executive golf outing, the VP of Personal Banking was bragging to the other VPs how efficient and cost effective his software team was as demonstrated by the common payments calculator service. Not to be outdone, the VP of Corporate Banking wanted in on this action and demanded that his peer share the payment calculator with his group. Just after another round of beers and just before a brawl broke out on the 12th hole, they agreed to allow the VP of Core Services to take ownership of the Payment Calculator service and deliver it as a common service across JKE. By the end of the round, The VPs of Billing, On-Line Banking and Mobile Banking were also all committed to leveraging this new, awesome common service.

Back in the office the following week, everyone realized the Payment Calculator was designed for auto loans with a maximum of five years so enhancements needed to be added to support longer loan terms. The Auto Loan team had to redo a bunch of integration testing to verify the changes didn’t break the Auto Loan application – which they did – and had to coordinate getting the regressions fixed – which they did. The Corporate Banking team required the addition of provisions for semi-annual payments instead of monthly payments. The Billing team wanted the service to return the total amount of interest that would be paid across the term of the loan. The On-Line team wanted the ability to deduct down payment values. The Mobile team wanted all values to be converted to local currencies. Each of the respective test teams realized they each had Dev, Test, Performance and UAT environments for each application. They just shook their heads asking “How are we ever going to test our applications while all these changes are going into the service we depend on?”

Projects and Environments dependent on a common service

Projects and Environments dependent on a common service

Hopefully you now see the problem. Today’s software systems are interconnected webs of loosely coupled applications; yet stable versions are required to complete integration testing. We have established how stubs or virtualized services can help, but how do you coordinate all this across so many teams and applications?

 Rational Integration Tester and Rational Test Control Panel Concepts

Let’s take a quick look at several Rational Integration Tester concepts that help us manage this challenge.

The Project

I can’t think of any other word in IT that is more overloaded and overused that “Project”, but nonetheless, it is a fundamental concept of Rational Integration Tester. The project is a physical disk location that contains files. From the Rational Integration Tester Workbench user’s perspective, the project is the container that holds all of your development artifacts including tests, stubs, datasets and other things. Let’s leave it at that – the Rational Integration Tester project is the disk location where you store the development artifacts related to the tests and stubs you are creating with the Workbench.

There is no Project concept in Rational Test Control Panel. You typically have a small number (maybe one) of Rational Test Control Panel instances, even in a large deployment where you may have many RIT projects. So the relationship between RIT project and RTCP instance is a many-to-one relationship.

The Environment

Every software project – err – “undertaking” must go through multiple stages in its journey to production delivery such as Dev, Test, Performance, UAT and Prod. There is hardware and software devoted to testing in each of these stages which is generally called an Environment. So although we are talking about one system, the implementation may vary considerably between, say, a Dev environment on very simple systems like Tomcat to HA WAS deployment in UAT or Prod. Rational Integration Tester captures the physical details in its own Environment artifact in the Rational Integration Tester project. The reason for this is that we need to know the implementation details of each environment to know how to apply tests and/or stubs to those environments.

The RIT environment maps directly to the environment in RTCP. Stubs, for example, can be created once in RIT and then applied to any or all environments. You can think of it this way: When a stub is deployed into RTCP, it carries along with it the environment details such as hostnames, IP addresses and port numbers. Stubs that have been deployed into RTCP are specific to the environment(s) into which it is deployed.

The Domain

The domain is a bit more of an obscure concept. A domain is an Rational Test Control Panel artifact that represents a logical grouping of related systems that are part of a real business project. It may be served by one or more IBM Rational Integration Tester projects, depending on how large or small the domain is, and how many Rational Integration Tester users are working on it. A domain can contain one or more environments.

The domain is much more loosely coupled to Rational Integration Tester. There is no domain artifact in RIT – you simply choose the domain into which you want to publish your stubs when you publish to RTCP.

Fictional example meets Rational Integration Tester

So let’s look at how our fictional JKE organization might apply RIT and RTCP to support their efforts to virtualize their common Payment Calculator. Keep in mind this is not the only way to implement a solution but one way.

Each of these application teams wants to create a stub that will represent the “to be” status of the Payment Calculator service that implements the new functionality they require. The Personal Banking Home Loans application team needs to virtualize the service supporting longer term loans. The Corporate Banking teams need to virtualize a service calculating semi-annual payments. The Billing team wants to virtualize the Total Interest Paid response value and so on.

So each team creates their own Rational Integration Tester project and implements their own stubs within their separate project. They each define their individual Environments within their projects.

Each of the application teams is mapped to a Domain in Rational Test Control Panel. This enables logical separation between the teams. The structure in the Environments page of RTCP will look something like this:

Domains and Environments in RTCP

Domains and Environments in RTCP

There is one area we could debate heavily on this proposal and that is the decision to have each team maintain their own separate stub in their own separate RIT projects. You may believe it would be better to have a single stub within a single project. I can definitely see why that would appear to be desirable. Rational Integration Tester does provide SCM integrations for version control. However, my perception is that these teams are all working on very different tasks in parallel. SCM is good for keeping track of sequential versions of artifacts like stubs but it isn’t so good at keeping track of parallel development efforts. RIT’s SCM integration definitely has no capabilities for merging changes together later. So my reasoning for keeping separate projects is based on my perception that these fictional teams want to keep their distance from each other and will for the foreseeable future. Again, this point could be debated and there are many other factors to consider.

So we have a scenario to build upon. In my next blog, I will look at how you can control permissions and access to domains and environments for both people and services such as Agents and Proxies.

What’s New?

I’m finally recovering enough from last week’s IBM Innovate conference to jot down a few notes on exciting stuff I saw in the testing space at the show.

Rational Test Workbench Mobile Test Edition V8.5.1

Support for testing mobile applications – native, hybrid and web – has been in Rational Test Workbench for a while now.  For those organizations that are looking for a complete package that includes mobile UI test automation, performance testing, service virtualization, integration testing and even UI test automation for desktop applications, Rational Test Workbench provides it all in one package.  RTW has been a compelling offering for enterprise clients for that reason.

However, if you only need a hammer, you don’t want to buy the whole tool chest. So as of now, you can buy just the mobile UI test automation capability in a lower cost offering.  This not only gives customers what they want, but it also makes the Rational offering more price competitive for mobile test automation.

Rational Test Workbench V8.6

There are some really exciting updates to the service virtualization and integration testing capabilities in Rational Test Workbench as well.  I would categorize the improvements in the following:

  • SAP support improvements
  • Support for MQ Telemetry Transport (MQTT)
  • Topology Discovery
  • Mainframe support enhancments
  • Other stuff

SAP support improvements

SAP support has been enhanced to include support for SAP XI 3.0 and SAP PI 7.31 and above for both integration testing and service virtualization.

Support for MQ Telemetry Transport (MQTT)

MQTT is at the heart of the Internet of Things.  It is a lightweight TCP protocol that is quite prevalent in smart appliances, vehicles and, well, “things”.  The Dev team had a pretty clever demo in the Expo Center where they could inject MQTT messages into a a traffic simulation and verify the expected behavior of other vehicles.  For example, send a message from one vehicle indicating it had stalled, blocking a street and verify other vehicles rerouted to avoid the incident.  Smart vehicles and self-driving cars are not just futuristic fiction anymore.  Testing these systems to ensure safety and reliability will require advanced testing techniques and tools.

Topology Discovery

Rational Integration Tester (RIT) can be a very powerful tool in the hands of a tester.  But what if the tester is just handed a complex system and told “test this”?  It is quite common when working with a client on a proof of concept that we end up discovering component relationships that the tester didn’t even know existed.  That “discovery” process can be extremely time consuming.  With Topology Discovery, you let Rational Integration Tester do the work for you.  In discovery mode, you tell RIT to monitor the system to determine which components are accessed by your component under test.  RIT will monitor the interactions and build a visual model for you, including capturing the message schemas and formats.  This can really help get you started much faster when faced with testing a new or unknown application.

Mainframe support enhancements

Ok, I will be the first to admit I am not a mainframe guy.  A lot of the new stuff I have seen is alphabet soup to me.  But for those of you that can make sense of it, RIT now supports executing tests against CICS programs directly using IPIC without the need for the CICS transaction gateway.  There have also been enhancements to System Z synchronization of WAS to create the IMS and CICS models.

Other stuff

  • The stub editor and message editors have seen some really nice UI enhancements making it easier to edit and manage stubs and test messages.
  • Enhanced TIBCO support.
  • Enhanced ability to publish and start stubs through automation in DevOps scenarios.

I also expect to hear about more advances in the near future as well and will pass them along when I do.

 

Sneak Peek – Mobile Quality. First Impressions Count

Javier Bastante of Keynote Systems and I will be presenting a session called “Mobile Quality. First Impressions Count” next week at IBM Innovate.  We are going to talk about a number of things, but one area we will address in particular is how you can combine the automated testing capabilities of IBM Rational Test Workbench with the Keynote DeviceAnywhere mobile device cloud to really increase your platform coverage and decrease your time to test.  We plan to demo this live.  Test Workbench is on a Windows machine hosted on the SoftLayer cloud.  The devices we will use to test on are hosted on the DeviceAnywhere cloud.

I have full confidence in both of these solutions.  However, I have been burned by poor hotel conference room network connections before.  To alleviate that risk and reduce my stress level, I prerecorded a shortened version of the demo we will show.  Just in case.  Who knows. Better safe than sorry.  So, I have a backup plan.

While I had this nice video on my laptop anyway, I thought, “Why not share it with our audience viewing at home?”  So I get a backup plan and you get a sneak peek at the demo we will show at the conference.

And then I thought, “Maybe someone that had not planned on coming to our session will see the video on YouTube, become interested and decide to attend our session.”  So I get a backup plan, you get a sneak peak and I get a shameless plug for my session and might possibly be able to drive up attendance.  Hmmm, seems like I am getting the better end of this deal, but I hope you find the video informative anyway.  Oh, and don’t be afraid to drop in on our session.  Tuesday, 11:15AM.  Dolphin-Northern A4.