David's Blog

projects from my internship at Progress Software

Running a Startup Script on a BeagleBone Black

For my Roll Call project I built a while ago I mentioned that I was able to run my local node server off a BeagleBone Black, with a startup script to run the app to make setting up and restarting easier. Here’s how I did it.

First, plug your BeagleBone into your computer, then go to the getting started page on the BeagleBone website and follow all the instructions there. You may need to install some drivers. Once the BeagleBone is working, follow the “Click here to launch” link on that page (I can’t link it here since it’s specific to your IP address). At the top of the page, there should be a green header with a checkmark that verifies your BB is configured correctly. Scroll halfway down the page and launch the Cloud9 IDE on the BB.

Now, copy and save your project onto the BB. I saved mine in the default cloud9 directory inside a folder called LocalHue. Once this is done, navigate to the root directory, then cd /lib/systemd/system. This is where the startup scripts for the OS are stored. In this directory, create a new .service file (I named mine hue-boot.service). Here’s my code for the service file:

Screen Shot 2014-09-08 at 12.04.03 PM

The working directory line is where your project is stored. The ExecStart line is the command to run inside that directory (“node app.js” in my case). Change those lines to fit your project as necessary. Next, save the file and you’re done! If you restart your BB the app should run automatically.

 

Provisioning an iOS App for Ad-Hoc Testing

So you’ve written a hot new iOS app and tested it on your personal device, and everything seems to be going smoothly so far. But now, before you take the plunge and submit it to the App Store, you want to share it around with a few friends to get their feedback. After googling around for a few minutes, no painfully easy method for doing this jumps out at you. Unfortunately, Apple’s provisioning process is pretty complicated, but I’ll explain how I did it for my project to hopefully make the process smoother for you.

First, you’ll need to talk to all those friends and ask them for their device’s UDID. The easiest way to do this is to plug your device into a computer and open up iTunes. Navigate to the ‘summary’ page for your device. Directly to the right of the phone image should be a line that begins with ‘Serial Number’. Click on Serial Number and the info will toggle to the UDID. It should look something like this:

Screen Shot 2014-08-20 at 1.39.28 PM

You’re not gonna get my UDID that easily!

Once you have the UDID, you’ll need to register it under your approved devices list on your developer account. Go to your account here, then click on ‘Devices’. Click the plus button to add the new device, using the UDID you collected from earlier.

Now you’re ready to create an ad-hoc distributing profile, which is basically an authorization that allows you to run an app on an approved device. Go back to your apple developer account, then navigate to Provisioning Profiles. Click the + button to add a new profile, select ‘ad hoc’, then navigate through the other menus to create your profile. Make sure all the devices you just registered are checked when you get to the device authorization page. Once the provision is created, download & double-click to install it. Then, open Xcode, navigate Xcode → Preferences → Accounts, click on your account and hit View Details, then click the little refresh button in the lower left corner. The provisioning profile you just created should show up.

Now you need to generate a build of your app that your users can run on their devices. Still in Xcode, scroll through the list of available run targets in the upper left corner until you select either “iOS device” or your plugged in device (basically, the option that isn’t a simulator). Navigate to Product → Archive. A window showing your archives should pop up.

Screen Shot 2014-08-21 at 4.23.19 PM

Select the archive you just made, then click Export → Save for Ad-Hoc Development. Navigate through the menus, which will result in your creating a .ipa file. This is an iOS Package file, used to distribute your app.

Your friends can install this package on their device by plugging their device into a computer running iTunes, then double-clicking on the .ipa file. In iTunes, click on the Apps button that shows up next to the device details. Your app should be somewhere in the list of apps on the left side of the screen. Click on your app, hit install, then hit sync. The app should now be available to run on the device!

If you get stuck anywhere in this tutorial, Apple has published their own App Distribution Guide. I found it pretty difficult to follow, but it’s more thorough than my quick tutorial here so it could well cover some case I glossed over.

Displaying OpenEdge Data in an AngularJS Web App

For my next project I was tasked with displaying data from a backend OpenEdge service on a modern web app built on AngularJS. As is often the case, Node.js is the perfect middleman to bridge the gap between backend and frontend. OpenEdge data can be accessed from Node using a Javascript Data Object (JSDO). I initialize the JSDO within a server-side instance of Node and then populate it with data from an OpenEdge temp-table using the JSDO.fill() method. In just a few lines of code the JSDO can be converted into an array of javascript objects, each object containing several key:value pairs representing various properties of each entry in the database (in this case I had a list of companies, with data fields like street address, city, state, and zip code for each company).

 

Once the data is in this array form we are back into a more typical use case of Node and Angular within the MEAN stack. I make the data accessible from Node through a RESTful API using Express. Angular accesses the data by sending a GET request to Node, which responds with a string form of the array. Angular receives this response and formats the data in an HTML table using the ng-repeat directive to iterate through the array. Note that by default, client-side Javascript can only send requests to endpoints on the same server as a security measure, so I had to enable Cross Origin Resource Sharing (CORS) on both the client and server side of the project in order for Angular to make an HTTP request to the remote Node server. With this in place, our stack is complete:

Screen Shot 2014-08-19 at 4.35.55 PM

Project Architecture.

One of the goals of this project was to host everything on Modulus, so I had to find some way to serve static web content using Node. Luckily, there’s a Node package exactly for this purpose (it seems there always is). Called Serverfy, you can simply run this package in your directory of static web content and it automatically re-formats everything into the Node application format. In a nutshell, it utilizes the node-static module to format the server correctly, then responds to each GET request for page data by serving the entire HTML file as a response. My (very simple) data table is pictured below, but once you have the data it’s easy to dress it up with a nice UI and display it in a visually appealing format to the end user.

Screen Shot 2014-08-19 at 3.52.09 PM

Data Table in Angular

 

 

Full Hue Control Outside Of The Local Network

As I mentioned in my last post, the Philips Hue system has the design limitation that it can only be controlled while connected to the same network as the bridge. Personally I turn off my phone’s wifi while on-the-go to save battery, and I don’t want to have to turn wifi on each time I come home just to control my lights. Or maybe I’ve just left the house and forgotten to turn the lights out, but am not quite in wifi range anymore to control them from my phone. Whatever the reason, there’s no point in saddling the Hue service with this network limitation when the technology exists for an easy fix.

My last project already demonstrates that it is possible to control the Hue from outside the local network. However, it was only performing two very specific actions with the bulbs: changing color to red/green and blinking the lights a few times. For this project I set out for full control of the lights: changing their color, brightness, and saturation, as well as being able to blink the lights or switch them to ‘gradient mode’, where the light slowly cycles through its color spectrum. This is pretty much the extent of the Hue API, and I wanted to control it all from one simple app.  

The architecture for this project is pretty much identical to the last project. The iOS app sends instructions to the Modulus Node server, which updates its Mongo database. Every few seconds, the local Node server, hosted on the same network as the bridge, pings the Modulus server, which queries the database and returns its contents. If the database has been updated since the local Node server last checked, it will parse those updated contents and send the appropriate update instructions to the Hue bridge. The Hue bridge then updates the lightbulbs. Here’s the architecture diagram:

The architecture diagram for the project.

The architecture diagram for the project.

As there wasn’t too much architectural innovation to figure out for this project, I decided to polish it a bit and streamline a few elements of the system. I stopped hosting the local Node server on my laptop, since I would have to restart the server each morning when i brought my laptop in. Instead, the server is now hosted on a BeagleBone Black, a pocket-sized microcomputer running the Angstrom Linux distribution. The BeagleBone is nice since it supports Node out of the box, but you could switch it out for a Raspberry Pi or Arduino and achieve the same result with a bit more legwork (node.js on Arduino likely requires a bit of hacking). To the BeagleBone I added a system daemon startup service that runs the service upon booting, which means the server can now run 24/7 with no human involvement.

IMG_0011

The BeagleBone in action

In addition, I’ve added a security layer to the project so some random person reading over my code couldn’t just bombard my server with http requests and cause the lights to freak out. Now, each request is authenticated with the server before being processed. Right now I’m just using a simple app key to authenticate requests since the system is built for use on one single Hue bridge, but if I were to expand the server to support multiple users and multiple bridges I would likely use passport.js to authenticate multiple users.

One other thing of note is that I’m running both the Roll Call and Hue Controller apps in one single Node.js server hosted on Modulus. Because Node is non-blocking, the two programs can coexist side-by-side and the server can handle simultaneous calls to both processes without any problems. Of course, when multiple requests are being sent rapidly to the Hue bulbs they might appear to be acting strangely to the user, but if I were using separate bulbs for each project you would never know the two were sharing a server. 

To finish, here’s a screenshot of the app in action:

IMG_0010

 

Roll Call v2: Adding Philips Hue Lights

This next project incorporates Philips Hue smart lightbulbs into my previous Roll Call project. The bulbs are set up in a row with nametags on them for the members of our office. Now, when a user enters or exits the building, their corresponding lightbulb will blink for a few seconds and change color (green for entering, red for exiting). So instead of having to open a website to check who is in the building, I can simply glance at the panel of lightbulbs. In addition, the blinking is an attention-grabbing way to notify me whenever someone enters or exits the building. Here is what the setup looks like:

Normally the lights would be red or green for each user, I turned them off for this picture since the lights were washing out everything.

Normally the lights would be red or green for each user, I turned them off for this picture since the lights were washing out everything.

Now to explain the technology that makes this possible. The Philips Hue lights communicate through Zigbee, a low-power radio transmission protocol. The lights are controlled through a central hub called the bridge, which plugs directly into a wireless router. Zigbee is a mesh network, so any signal transmitted by the bridge spreads through the lightbulb network until it reaches its target. Thus, bulbs can receive signals from the bridge even if they aren’t directly in range with it. Hue users can talk to the bridge through mobile or web apps.

Philips provides a fully-featured API for controlling the lighbulbs through the bridge, allowing you to manipulate the color, brightness, and saturation of individual lightbulbs, as well as supporting special events like color gradients or light blinking. While this API is powerful, it has the downside of requiring any application utilizing it to be on the same network as the bridge. When I started this project, I wanted to communicate directly from my Modulus-hosted Node server to the bridge, but because of the Hue architecture this isn’t possible. As an added challenge, I was working on this project at the Progress office, which has a private enterprise network with a firewall that blocks TCP connections, preventing the bridge from being controlled through external Hue servers (another feature of the product).

Thus, if I wanted to manipulate the lights from outside the network, I would need some sort of go-between that resided within the network in order to talk to the bulbs, but was also capable of spanning the firewall to communicate with external servers. I decided to use a locally-hosted instance of Node to bridge this gap. This is able to reach beyond the firewall as it relies on HTTP requests, which the firewall allows. With this decision in place, the architecture for our project looks like this:

The architecture diagram for the project.

The architecture diagram for the project.

Let’s dissect this a bit. When the user walks into the building, the iPhone in their pocket detects the Bluetooth signals transmitted from the Gimbal beacons positioned around the office. This triggers the phone to open the Beacon Receiver app in the background, which sends a POST request to the Modulus server with updated position/time data for the user. The Modulus server stores this information in a Mongo database.

While this is going on, the locally hosted Node server has been sending a GET request to the Modulus server every few seconds, asking for user location data. Each time this occurs, the Modulus server queries the database and responds to the GET request with this data. (I use the Cron package in Node to perform this repeated action). Since the database has been updated, the local server is now receiving information that differs from its local data. The server has to correct for this difference by updating the light in the Hue network corresponding to this user. It sends a PUT request to the bridge telling it to update the color and blink for a few seconds (I couldn’t do this from the Modulus server since it’s not on the same network as the bridge, hence the need for a local server). Once the bridge receives this information, it sends it back out to the Zigbee network, where it eventually reaches the lightbulb and causes its color to change.

Sounds complicated, doesn’t it? Surprisingly, the whole process only takes a few seconds. The limiting factor is how often you set the local Node server to ping the Modulus server. I’ve set the local server to ping the Modulus server every second, so it takes about a second for the lightbulbs to change. But if I pinged more frequently the response time would shrink accordingly.

I can definitely see the market for a set of lightbulbs that can easily be controlled through HTTP requests. At the scale of my project, I could’ve achieved the same result with an Arduino and a set of LEDs, but in a real household the Philips bulbs look much more presentable and their API packs a good deal of power. I was a little frustrated that I couldn’t control the bulbs from outside the network, as it seems like this should be possible with the current hardware setup given that the bridge plugs into the router itself. I get the sense that the Hue just “isn’t quite there” yet, but it certainly provides a taste of what is to come in the home automation industry. I’m looking forward to doing more projects like these as the technology gets more powerful and streamlined.

Bonus low-quality GIF of the project in action, Eduardo’s light is blinking red indicating that he has just left the building:

blinking

The Roll Call App: Integrating Beacons with Node.js and MongoDB

For this project I wanted to extend the functionality of the beacon explorer app described in my last blog post, creating something practical that could be used within the office. I decided on a “roll call” style project to see which people are in the office at any given time. With so many people here working from home or out on business trips, it’s nice to see who is actually around when trying to make lunch or meeting plans. Before I launch into explaining the project, here is a link to the website.

To build this application I needed some way of receiving and storing data from multiple devices. Node.js is well suited to this task as its non-blocking event loop makes handling multiple communications at the same time a breeze (given the current size of our office this would never really be an issue, but it makes scaling really easy if I ever wanted to use the project in a larger environment). Progress recently acquired the Node application hosting service Modulus, so I decided to give them a try for hosting the web server. Modulus has built in MongoDB support, so it made sense to use Mongo to store the information.

With my technologies in place, next I needed to figure out how to make them communicate with each other. As my Node application is an HTTP server, I need to send data to it using POST requests and respond to GET requests asking for data. The Express framework greatly simplifies all web interactions within Node, so I chose to run that on my Node server to handle all the requests from iOS and web browsers. iOS in turn can send POST requests using the NSURLSession class. For interacting with MongoDB, another Node framework called Mongoose comes in handy. Mongoose simplifies all the connecting and data organizing difficulties of working with a database. Here’s a quick diagram of the full setup:

Screen Shot 2014-07-15 at 2.38.46 PM

So what does this project actually do? All our office members have the Roll Call iOS app installed on their phones. Upon startup the app prompts for the user’s name, which is cached in the iPhone’s memory using NSUserDefaults, a class for storing simple data between user sessions. That’s the extent of user interaction with the app; it never has to be opened again. Since this app utilizes Apple’s Location Services, iOS searches for the office beacon region even when the app isn’t running. When iOS detects nearby beacons, it passively opens the app in the background for about ~10 seconds. In this time, the app sends a POST request to the Node server with 3 pieces of information: the name of the user, the timestamp of the request, and a boolean representing whether the user entered or left the building.

The request containing this information is processed by the Express framework in Node. Once the data is extracted, Node converts it into the JSON format utilized by Mongo and passes it on to the database. If this is the first database entry for a particular name, a new user is created. Otherwise, the user’s data entries are updated to reflect their latest actions. Finally, this data needs to be displayed somewhere. Modulus provides a client-facing URL for each hosted application, which I’ve populated with the MongoDB content through Node. When the user types in the project URL, it sends a GET request to the Node server, which in turn pulls the user table and action log from MongoDB and formats it with html tags for the browser. The website can be viewed here, and looks like this:

Screen Shot 2014-07-15 at 3.52.48 PM

(this data was “faked” by me walking in and out of the building a couple times, in a real application of the project I would add a password to protect privacy)

Nothing fancy, but a simple and clean way to keep track of who’s around. This project could easily be extended to a larger office by simply placing more beacons around the office. Or a warehouse manager could use it to keep track of employees within a large space, even dividing the warehouse into regions based on different beacon major and minor IDs in order to pinpoint an employee’s location within a warehouse. Software models could be run on this data to improve efficiency within the warehouse. There are many potential ways to expand this project into a full commercial product.

Finally, I just want to say that I think that the coolest aspect of this project is how the user never needs to open the iOS app for the project to function, other than at the very beginning to input their name. Apple has implemented some powerful functionality into their Core Location service, and developers can take advantage of this to implement useful and relevant location-specific projects without ever having to bother the user. I would imagine in the future that more and more apps will shy away from dependence on foreground use and instead lean towards the background monitoring + notification system used here, as it provides a simpler and more streamlined user experience.

I’ll post a Github link to the project soon.

My First Beacon App

My first project with Progress was to write an iOS app that interacted with Gimbal Beacons. Gimbal Beacons are small, battery powered devices that transmit a bluetooth signal to the region around them. One might associate Bluetooth with the clunky phone accessories that required a painful pairing system to operate, but that is an older technology. This new technology, called Bluetooth Low Energy (BLE), relies on a pairingless transmission-reception system that uses far less power and is much simpler to implement. Each beacon sends out a signal with 3 pieces of information: a general ID called the UUID that is used to define a large group of beacons, and two more specific IDs called the major and minor IDs used to distinguish smaller groups of beacons. These signals are received by a BLE-capable device such as a smartphone, which can then provide location-specific information to the user based on their positioning relative to the beacons. The beacons are small and their batteries last a long time due to the simplicity of the information they are transmitting. For this project I used beacons from Gimbal (key for scale):

Image

 A common example of a use for this technology is in a department store, where the user would load up the store’s app and receive things like discount offers or product information as they navigated through the different departments. It could also be used at train stations to track the number of zones a passenger crossed and charge them according, eliminating the need to carry an additional card and swipe on/off (thought this one up while struggling to find my Clipper card on the train into work this morning). 

For my first project I just wanted to build a dirt-simple app to explore the basic capabilities of iBeacon – iOS interaction to have a base to build on for future projects. I’ll walk you through the app I built and it’s key features. Here is what it looks like upon startup:

Image

Not too exciting. The app is looking for the region defined by our Gimbal Beacons. A region is just a bunch of beacons that have the same UUID. In our department store example, all the San Francisco Macy’s beacons would have one UUID while the San Jose Macy’s beacons would have another  The beauty of the app is that it doesn’t need to be running (not even in the background) to detect the beacons. Instead, it utilizes Apple’s Location Services protocol, which constantly checks if the region is nearby. If Location Services detects that the phone has entered the region, the app is then initialized and can notify the user that they have entered the region. If a user turns their phone on in the morning and drives to the store, the app will open up all on its own, greatly increasing the number of times it gets used. In addition, battery consumption is greatly decreased as the app isn’t constantly looking for beacons when it doesn’t need to be.

When the region entry notification pops up, the user can simply tap on it to open the app:

Image

Image

Now we finally have some content. The status bar at the top tells you where the app is in the beacon location process. Here it has already found beacons, so it simply displays the number of visible beacons. If it didn’t see beacons, it would instead say either “ranging” or “monitoring”. This is an important distinction. Monitoring is the process of looking for a beacon region and is handled by Apple’s Location Services. Thus, monitoring works when the app is in the background or not even running at all. Monitoring takes very little battery life. Ranging, by contrast, is the process of actively looking for beacons within a region. Ranging can only occur when the app is in the foreground, and it uses a lot more battery life. Our app passively monitors until it finds the beacon region, then notifies the user to open the app, where it starts actively ranging for beacons. Thus, the app is efficient as possible and doesn’t waste battery life when it doesn’t need to.

Now on to the list below. Each element in the list corresponds to a beacon, showing all three of its identifying features (major, minor, UUID). In addition, it shows the approximate distance of the phone from the beacon. This metric isn’t very accurate as it uses strength of signal to calculate distance. Since signals will degrade at varying rates depending on what material is between the beacon and the phone, the measurement never quite lines up with the actual distance, but it is useful for distinguishing rough proximity to an area.

That’s the gist of the app for now! My next project is to integrate the beacon technology with Node.js and Rollbase to enable the user to interact with location-specific data from a database. In our department store example, we could store different coupons in the database and update them weekly, which would then get pushed to the user as they entered the store. Or the train company could store zone pricing and customer payment information in the database for the app to use to accurately charge the user. Location-based interactions can simplify many experiences between user and other entities, and the beacon technology offers a great improvement in accuracy over the GPS technology currently used in phones. I’m excited to see what uses this technology will be put to in the upcoming months.

Some technical details: This app was written for iOS 7.1 and uses the CoreLocation framework to achieve the background region monitoring.