Part 7: Requesting and Receiving a Picture

This is part of a series of posts related to My Internet of Things and MobileFirst adventure.  An index to all posts can be found at the end of the first post.

The next step is to see if we can make the iOS app request a picture from the Raspberry Pi and have it respond with one.

I stared by creating a simple iOS app that had a button to start the request process and a UIImageView to display the picture.  The UI eventually evolved a bit, but this gave me a place to start and keep it simple.

MQTT Mobile Client software

There is no iOS SDK specific to the IBM IoT Foundation.  There are, however, several MQTT pods out on cocoapods.org that could do the job.  MQTT is the protocol underlying the messaging in the IoT Foundation.  I chose to use MQTTClient.  I followed the simple instructions to install using cocoapods.  MQTTClient provides interfaces at various levels.  The simplest way to use MQTTClient is through MQTTSessionManager.

IBM IoT Foundation conventions

The IoT Foundation does layer on some extensions that are supported by conventions you must observe when using MQTT. The full IoT Foundation documentation is very helpful.

First of all, the IoT Foundation considers there to be two types of “things” in the Internet of Things: devices and applications.

Devices:

  • A device can be anything that has a connection to the internet and has data it wants to get into the cloud.
  • A device is not able to directly interact with other devices.
  • Devices are able to accept commands from applications.
  • Devices uniquely identify themselves to the IoT Foundation with an authentication token that will only be accepted for that device.
  • Devices must be registered before they can connect to the IoT Foundation.

Applications:

  • An application is anything that has a connection to the internet and wants to interact with data from devices and/or control the behaviour of those devices in some manner.
  • Applications identify themselves to the IoT Foundation with an API key and a unique application ID.
  • Applications do not need to be registered before they can connect to the IoT Foundation, however they must present a valid API key that has previously been registered.

For some reason, it took me a while to wrap my head around this.  I wanted to consider my iPhone a “device”, but in fact, it would be an “application”. For one reason, it is going to be sending commands to get information from the Pi.  Secondly, note that you don’t register applications in advance – they just provide a unique key.  This would be important if I were to do into production with my doorbell.  I can’t register every possible mobile phone in advance.

Commands:

  • Commands are the mechanism by which applications can communicate with devices. Only applications can send commands, which must be issued to specific devices.

Events:

  • Events are the mechanism by which devices publish data to the Internet of Things Foundation.

So my iOS app will be an application that will send commands and subscribe to events that will be published by my Raspberry Pi which is a device.  Got it.

Here are some other MQTT concepts and their manifestation in IoT Foundation.

Connection parameters

  • MQTT host:  The host of the IoT broker is org_id.messaging.internetofthings.ibmcloud.com where ord_id is the org assigned to your IoT Foundation service when it was created.  You can find it from the Bluemix dashboard for your application.
  • MQTT client identifier:  This is an identifier the application must provide when it connects to the IoTF.  it must be unique.  The convention is a:org_id:app_id, where ord_id is the same as above, and app_id is something unique for each app instance.  Since there will only be one app instance at a time for my simple app, I just used the Bluemix AppID for this.
  • MQTT username: Use the IoT Foundation API Key generated when you created your IoT Foundation service.
  • MQTT password: Use the IoT Foundation Authentication Token

Topic names

The IoT Foundation uses a strict convention for MQTT topics that map to device types, device IDs, commands, and events.  Reference the documentation for all the details, but here are the topic strings I used for the command I would send from the application and the event I would subscribe to:

Connecting to IoT Foundation from the Application

Again, sparing some of the details, here’s my connect() method:

Receiving the picture

First step is to send the takePicture command to the Raspberry Pi through the IoT Foundation.  The actual content of the message is irrelevant in this case.  The operation on the Pi will be invoked just by receiving the command.

The picture will come back from the Raspberry Pi as a Bas64 encoded string as the data in the pictureTaken event.  This wasn’t as easy as I expected.  Turns out that MQTT has a limit to the size of the messages it will transmit.  Who knew?  Well, I didn’t anyway.  So, as you will see shortly, I break the picture up into 3kb chunks and send them back in sequence.  So on the application side, I created a completion handler that would piece the picture back together again.  The completion handler gets called each time a packet event arrives.

The MQTTSessionManagerDelegate routine handleMessage gets invoked each time a packet arrives, which then invokes the callback.

The callback is smart enough to know when it receives the final packet and then completes the reconstruction of the image and eventually displays the picture in the UIImageView.

Servicing the command from the Raspberry Pi

There are Node-RED nodes available to communicate directly to the IoT Foundation from within Node-RED.  Follow the instructions for node-red-contrib-scx-ibmiotapp to set it up in the Raspberry Pi.

With the ibmiot nodes in place, create the following flow:

Node-RED flow that takes a picture

Some details:

  • takePicture  This is the IoT node that receives the takePicture command from the application.  Configure it with the requested credentials.  The device type and device id must match those you created for the Raspberry Pi in the IoT Foundation dashboard.  The command is “takePicture”.
    takePictureNode
  • Take Picture  This function node is pretty much the same as in an earlier blog post.  It invokes the camera through JavaScript in the Node.js environment.  It sends on a message containing the picture’s filename and timestamp.

  • delay 2 s  The camera is asynchronous so you need to give it a little time to take the picture and save it to a file before processing.
  • Base64 encode  This is an exec node which runs an operating system command.  The command value is “base64 -w0” and the msg.payload value is checked.  This means an OS command base64 will run against the filename provided by the Take Picture node.  “-w0” keeps base64 from injecting newline characters in the string.  The Base64 string is sent to the next node as the payload.
    Base64Node
  • splitIntoPackets  This function node slices the payload into an array of messages containing 3k chunks of the data.  The date, picture name, number of packets and packet index are all added to each message as well.  The array becomes a series of messages sent by the node.  The node also has an output that sends the picture name and one that sends the total number of packets.  These are just for debugging purposes.

  • pictureTaken  This ibmiot output node sends the device event “pictureTaken” for each packet.  Use the same values for authentication, device type and device id that you used for the ibmiot in node.
    PictureTakenNode

Where are we?

Ok, we covered a lot of ground in this post.  We talked a bit about MQTT and how IoT Foundation fits with it.  We looked at how the Swift code will use the MQTTClient library to send commands and receive picture packets and reassemble them.  We also looked at how the Raspberry Pi will receive the IoT command through the IoT input node, use the Raspicam Node module to take a picture, use an OS command to convert it to Base64, use a function node to packetize it, then use the IoT output node to send the packets back to the phone as events.

I didn’t show it here, but I added some code to have the iOS app invoke this whole process in response to the user’s choice from the Push notification.  So I accomplished what I had set out to do.  The Pi sends a push notification to the phone to let the user know there is someone ringing the doorbell.  The user can then choose to see a picture of the visitor by tapping Picture in the push notification.  The app will then send an MQTT command to the Pi through the IBM IoT Foundation.  The Pi then takes a picture and sends it back to the phone which displays it.

Next

I decided to go for some bonus points.  What if the user could actually watch video of the visitor?  Next post.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s