Part 8: Watching video from the Pi on the Phone

This is part of a series of posts related to My Internet of Things and MobileFirst adventure.  An index to all posts can be found at the end of the first post.

Let me say right off that there are dozens of alternatives available if you want to stream video from a Raspberry Pi to another device.  They all seem to have their pros and cons.  Some are really inefficient.  Others require a lot of processing and an additional app server.  Still others are very proprietary and only work on a very limited set of devices or platforms.  So, feel free to try your own approach.

I decided to go with the VLC media player.  I have known VLC as a good media player for a long time, but I had no idea it can do so many more things – in particular, act as a video streamer.

So here’s the basic plan:

  1. Configure the Raspicam to record video to a fifo file.
  2. Configure VLC on the Raspberry Pi to stream that fifo content to a URL
  3. Connect a VLC player object embedded in the iPhone app to the URL stream.

Install VLC on the Raspberry Pi

You can install the Debian image on the Raspberry Pi:

sudo apt-get install vlc

That was easy.

Add the VLC SDK to the iOS app project

There’s a Pod for that.  In fact there are multiple.  I went with MobileVLCKit.

Create the Raspberry Pi Node-RED flows

Let’s start on the Raspberry Pi side and deal with sending the command from the app later.  Let me warn you, this gets a little crazy.  There are several steps here that I probably could have done more simply in a shell script.  But I tried to stay in Node-RED as much as I could, which ended up complicating some things.  Here we go.

Start Stream Flow

Create the following flow:

 

 

I warned you.  Let’s go through the nodes:

  • startStream  Receive the IoT Foundation startStream command from the application.
  • mkfifo  Creates a fifo file on the filesystem.  This is an ‘exec’ node, which just runs the given OS command.  It returns stdout as the payload of the message, but in this case, I don’t really care about the contents of stdout.  mkfifo
  • start raspivid  Invokes the shell command “/home/pi/bin/startraspivid.sh”.  This shell script starts up raspivid recording 640×640 video at 25 frames per second.  This seemed to produce decent looking video yet kept the bandwidth requirements down.  These numbers could definitely be tweaked if needed.  raspivid is streaming to the fifo file named vidstream.  Yes, the rest of the command looks odd.  The issue was that I needed the node to launch raspivid and let it continue to run on another process yet return execution to the next node.  I tried a number of approaches and the only one that seemed to work was to background it (‘&’), but I found I also had to redirect stdout and stderr (‘ > raspividout.txt 2>&1’) or execution wouldn’t return to the node.

  • start vlc  This is similar to the raspivid node.  It invokes a bash shell script that starts up VLC (‘cvlc’).  VLC will take the vidstream fifo file as input and produce a video stream on port 8554.  I had to do the same dance with backgrounding and redirecting output as I did with the raspivid script.
    VLC has a seemingly infinite number of command line switches and arguments.  I won’t claim to have figured all this out myself.  I relied heavily on the VLC documentation and a couple blogs on the Raspberry Pi forum.

  • get IP Address  This function node figures out the IP address of the Raspberry Pi.  I need this because the iPhone app will need to know where to connect the VLC player to view the stream.  Now this brings up a significant limitation to the VLC approach – the Raspberry Pi must be visible to the mobile device on the network.  Unless you are willing to go through all the steps necessary to get your Raspberry Pi a public IP address and deal with all the security issues that will arise, that’s a pretty big limitation.  I decided I can live with the limitation that video only works while my phone is on my local Wifi network with my Raspberry Pi.

  • streamStarted  An ibmiot output node that publishes the device event streamStarted with a payload that includes the IP address of the Raspberry Pi.

Stop Stream Flow

The Stop Stream flow is invoked by receiving a stopStream IoT Foundation command from the mobile app.  It simply kills the Raspian processes for vlc and raspivid and then publishes a device event acknowledging that it did.

stopStreamFlow

Stop Stream Node-RED Flow

    • stopStream  ibmiot input node that receives the command stopStream.
    • Stop Stream  An exec node that runs the script stopstream.sh.

    • format Message  My mobile app doesn’t actually do anything with this, but I created this node to parse out the process identifiers (PIDs) of the vlc and raspivid processes that were killed and include them in the payload of the streamStopped event.

  • streamStopped  An ibmiot output node that publishes the device event streamStopped with a payload that includes the PIDs of the killed processes.

Add functionality into iOS app

Again, I won’t go into all the details here, but I want to hit the high points.  What needs to be done in the app is

  1. Add a UIView object to a view to act as the ‘drawable’ for the mediaPlayer
  2. Add a VLCMediaPlayer object.
  3. Send the startStream command through IoT Foundation
  4. Receive the IP address and have the mediaPlayer attempt to connect to it.

Add a UIView object

I decided to create a tabbed view controller, making the first tab the Picture view controller and the second the Video view controller. The Video view controler just has a UIView on the storyboard with an IBOutlet in the view controller code.

Add a VLCMediaPlayer object

With the MobileVLCKit pod added to my Xcode workspace, this is as simple as

In the view controller’s viewDidLoad method, I configure the mediaPlayer.

Send the startStream command

In the view’s viewWillAppear method, I send the startStream command.  This is really the same as sending the takePicture command. I do provide a completion handler, but this time, the handler will only be invoked once – when the streamStarted event is published by the Raspberry Pi.

Receive the IP address and connect the MediaPlayer

The callback gets invoked when the streamStarted event is published.  The payload of the streamStarted event is the IP address of the Raspberry Pi, so all that is left is to set the mediaPlayer’s media value and tell it to play.

Remember, though, that as we discussed earlier, the mediaPlayer will not be able to connect to the stream if the Pi is not visible to the mobile device. If that is the case, the mediaPlayer object throws up an alert. I could have handled that in my code with a delegate, but I decided to just let the mediaPlayer deliver the bad news.

IMG_0673

Summary

There are a lot of ways to skin this cat.  I chose to use VLC because it was relatively easy (compared to other methods I considered), but it does have the significant limitation that my Raspberry Pi has to be network-ly visible to my iPhone.  But nonetheless, while I am in the same Wifi region, it is pretty cool that when I hear the doorbell ring, I don’t even need to get up out of my recliner to see if it is a siding salesman.  The world of WALL-E is fast approaching.

So next I went a little crazy.  I got a new Apple Watch and thought I would see if I could build it into this environment in the next post.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s