Thursday, September 30, 2010

Robot tracking experiments

As an organizer of MIT's 6.270 Autonomous Robot Design Competition I've been working on an improved vision-based system for tracking the contestants' robots on the playing field.

(Sidenote: I competed in 6.270 last January and at some point I'll write a whole post or two about my experience. To sum it up, I had an awesome time competing, which is why I'm now an organizer of the competition)

The basic concept is that we wirelessly feed each robots its coordinates throughout the round and act like GPS to help the robots navigate. However, this isn't as easy as it sounds.

Our approach is to mount an overhead camera facing down at the playing field and then analyze the video to find special "fiducial" patterns on the robots. This isn't too hard in a controlled environment, but it gets tricky when the system has to ignore other objects on the field or when pieces of a robot go flying after it slams into a wall (which happens quite often during testing!).

The fiducial patterns I came up with look like this:

Essentially it's a white square with one white corner, and then a few bits of information in the center.

To track these, each frame of video is run through a square detection algorithm. Then for each of these potential objects, each of the 4 corners are checked to find the reference corner - now the software can tell where there are objects and their orientations.

Finally, it checks the inside of each square to see which sections are white and which are black - these are essentially bits that we can use to encode the team number in order to distinguish multiple robots.

So far I put together a simple proof of concept that can track and identify a bunch of these fiduccial markers in real time:

You can see how it has ignored all of the distractions placed in the view and labeled each of the markers it finds. It places a large red dot in the corner that has the reference mark so you can see the perceived orientation.

The software uses the OpenCV library for video capture and processing. The C code is available at github under the MIT license: It should be updated moderately often as I get this system fully fleshed out in time for this January's competition.

Hopefully this was an interesting peek at what's going on behind the scenes to get the competition ready for January. If you go to MIT you should definitely consider competing, or if you live in the area you should definitely check out the final competition at the end of January!

Friday, September 3, 2010

Using a GP2D02 IR distance sensor with Arduino

Just wanted to share an Arduino library I made over a year ago which I just realized might come in handy for others playing with the popular GP2D02 distance sensor.

The GP2D02 is a neat digital distance sensor made by Sharp which uses reflected IR light to measure distances from about 3" to 3' (I think it's been discontinued now though). The fact that it has a digital output is convenient when using a microcontroller, although since the Arduino has analog inputs you might also consider using the analog variants like the GP2D12 or GP2Y0A02 instead.

To install the libary, you need to download and place the GP2D02 directory into the Arduino libraries directory and then restart the Arduino IDE.

Once you've installed the library you can set up a sensor like this:

#include <GP2D02.h>
  o|       |---Black--------|Gnd
   | GP2D02|---White--->|---|Pin 2 (note the interfacing diode)
  O|       |---Red----------|+5V
   |_______|---Yellow ------|Pin 7

const int INPUT_DATA_PIN = 7;
const int OUTPUT_CLOCK_PIN = 2;


Then you can get the distance from it like this:

byte distance =;

And that's it!

(Note that the shift-in function in the library was adopted from and modified in order to account for the timing requirements of the GP2D02 sensor)

Wednesday, August 11, 2010

Playing with webkit notifications

My brother was working on a simple desktop app that needed popup notifications and as we were brainstorming ideas, my fondness for web-apps prompted me to look into the webkit notification API (and try to convince him to make the app into a web-app).

I've written up a short tutorial for the webkit notification API below. The API is currently only supported by Google Chrome, but it's pretty cool and hopefully will be adopted by other browsers. Basically it lets you pop-up a small notification (after requesting permission) as an always-on-top window separate from the browser window. It looks like this:
and it shows up in the upper right corner on Macs or the lower right corner in Windows.

For the impatient, here's a quick demo: (remember, only works in Chrome)

From this point on, I'm going to make the (terrible) assumption that all of our app's visitors are using Chrome so we don't have to worry about checking browser support or degrading gracefully.

So how do you show a basic notification? First you need to request permission from a user with window.webkitNotifications.requestPermission();This will cause the browser to ask the user for permission:

There's one very important caveat about requesting permission: the page can only request permission when a user initiates some action on a page, e.g. clicking on a button. This means you can't have your page request permission when it loads.

So we'll make a button for requesting permission like this: (assume jQuery is already included)
<button id="notifSettings">Change notification settings</button>
$("#notifSettings").click( function(){

Which looks like:

(In my opinion, this is the biggest shortcoming of the API that may prevent its widespread adoption in web-apps - it makes no sense that the user has to prompt the app to ask him for permission by clicking something. The way it should work is that the browser automatically asks the user for permission the first time an app tries to display a notification, and that permission preference is saved. If the user denies notifications, then the app can only request permission again by having the user click a "notifications settings" button, or if a notification is to be displayed as the result of a user action, e.g. if a button's click handler tries to display a notification.)

The requestPermission(callback) function also optionally accepts a callback function which is called after the user chooses to allow or deny notifications.

The next part of the API that's important is the function window.webkitNotifications.checkPermission();. This allows you to check if notifications are allowed - it returns a non-zero integer if they are NOT allowed. So if notifications are really vital to your app, you might prod the user to enable them with code like this:
$(document).ready( function(){
if (window.webkitNotifications.checkPermission() != 0){
alert("Please enable webkit notifications!");

This will use an old-fashioned alert to tell the viewer to turn on notifications if they're not allowed when the page loads. (Please don't actually do this on your site - it's really annoying)

Let's get to displaying notifications!
We'll use the function window.webkitNotifications.createNotification(icon, title, body). This function call returns a notification object which we then show by calling its .show() method. That object also allows us to setup callbacks - I'll demonstrate this in a bit.

Here's a little demo:

The source code is simple:

<button id="notify" onclick="return false;">Notify me!</button>
<button id="notifSettings2" onclick="return false;">Change notification settings</button>
<script type="text/javascript">
$("#notifSettings2").click( function(){
$("#notify").click( function(){
if (window.webkitNotifications.checkPermission() != 0){
alert("Oops! Please enable notifications by clicking the 'Change notification settings' button!");
} else {
window.webkitNotifications.createNotification("", "Notification title", "Notification text!").show();

You can set up callbacks by setting the ondisplay, onclose, or onerror properties of the notification. For example, we can make a really annoying "non-dismissible" notification that reopens when the user click "Dismiss":

In practice you wouldn't want to do something so annoying, but the ondisplay and onclose callbacks can be used to integrate the notifications into your web app better and let the app know when the notifications are closed.


<button id="startMadness" onclick="return false;">Start the madness!</button>
<button id="stopMadness" onclick="return false;">Stop the madness!</button>
<button id="notifSettings3" onclick="return false;">Change notification settings</button>
<script type="text/javascript">
$("#notifSettings3").click( function(){
var notification = null;
var notify = function(){
if (window.webkitNotifications.checkPermission() != 0){
alert("Oops! Please enable notifications by clicking the 'Change notification settings' button!");
} else {
notification = window.webkitNotifications.createNotification("", "Notification title", "Try dismissing me!");
notification.onclose = notify; //make the onclose event trigger the notify function again - madness!;
$("#startMadness").click( notify );
$("#stopMadness").click( function() {
notification.onclose=null; //clear the onclose event handler
notification.cancel(); //close the notification

The last thing I'll demonstrate is an HTML notification, which allows you to specify another webpage to load in the notification window instead of just plaintext. Use the window.webkitNotifications.createHTMLNotification(url) function.


<button id="htmlNotif" onclick="return false;">HTML Notification!</button>
<button id="notifSettings4" onclick="return false;">Change notification settings</button>
<script type="text/javascript">
$("#notifSettings4").click( function(){
$("#htmlNotif").click( function(){
if (window.webkitNotifications.checkPermission() != 0){
alert("Oops! Please enable notifications by clicking the 'Change notification settings' button!");
} else {

There are limitations to what the page loaded in the notification window can do. For example, it's not possible to access the HTML5 localStorage from javascript within the notification. Also, it's not possible to type into form textboxes in the notification window, which is unfortunate.

So that's the basics of webkit notifications, hopefully you can put them to good use in your own web-apps!

For more info, check out or

Monday, July 19, 2010

My first instructable!

I'm a huge fan of DIY (in case you couldn't tell from this blog...), so naturally I love Instructables - a site with user-submitted how-to/DIY guides. There I saw a really neat "word clock" which is a clock that spells out the time in words rather than numbers, and thought it would make a cool present for my girlfriend at the time.

I improvised on the design and came up with a neat glossy black finish, packaged inside an Ikea shadowbox. It turned out looking really cool:

In the spirit of giving back, I wrote my first instructable so you can build your own clock just like this!

RFID door

--- A demo of how the RFID door mechanism works. Normally the RFID reader is facing outward so you can wave your ID on the outside of the door to unlock it.

Last semester I was lucky enough to get an order through during Sparkfun's Free Day - $100 of free electronics - awesome! Among the things I got for free was the ID-12 RFID reader, a cute little RFID module that outputs the tag ID over a 9600 baud serial line. Naturally I needed to put it to good use, so when I was bored one night (avoiding psets) I set out to make my door unlock/open via RFID.

The biggest obstacle was the mechanical actuation of unlocking the door. My door doesn't have a deadbolt, which means the knob itself has the locking mechanism built in. Ideally I would have installed an electric strike, but unfortunately the cutout in the steel door frame was too small to install one. I also briefly considered sticking some sort of motor inside the door frame that would push the latch open (since the interlock mechanism that normally prevents this wasn't working), but that would require too much torque in a small space to be practical.

Since I couldn't figure out a way to easily release the latch itself, I decided to go for the brute force approach instead - hook up a motor to turn the actual doorknob from the inside. At first this too seemed nearly impossible; there was no good way to mechanically grip the knob, and even if you could it still requires a huge amount of torque to rotate. Of course the obvious solution, as always, was duct tape! One of my friends pointed out that the knob only has to turn about 45 degrees, so I could just wrap duct tape around the knob giving me a kind of "rope" to pull.

Next I needed a high-torque motor to act as a winch and pull down on the duct tape "rope." Conveniently I had a bunch of LEGO left over from competing in MIT's 6.270 Autonomous Robot Design Competition, so I made a nice gearbox out of lego and attached this to the duct tape. The gearbox let me use a cheap high-rpm DC motor to power the "winch" instead of having to buy an expensive high-torque motor. I mounted the gearbox/winch to the door with 3 strips of Command adhesive (one of the greatest inventions ever), which theoretically provide ~9lb of holding force.

This design also had the nice feature that the knob is still fully functional even if everything fails, so it's impossible to get trapped in my room :)

To control the motor, I hooked up an ATMega328 (Arduino) with a TA7291P motor driver. However, I needed a way for the Arduino to know how far to turn the knob and how far to turn it back afterward. I added a simple microswitch to the gearbox that could count revolutions (see picture above). The Arduino just has to turn on the motor and wait for a certain number of revolution "clicks" in order to turn the perfect amount every time. This worked great, but since I planned to use this door opener every day, I made sure to design for failure. I programmed a timeout in case something failed and the switch wasn't activated, which would leave the motor forever trying to turn the knob and eventually burn it out or blow out some other component. As it turned out, this was absolutely necessary, since the LEGO gearbox tended to jam up over time (the gears would "wander" back and forth on the axle).

Next I hooked up the ID-12 RFID reader. This was extremely simple since the package just needs 5V and ground and outputs TTL serial. Instead of hooking the serial output directly to the Arduino, I decided to feed it to my room automation server for authentication. When you swipe an ID card, the tag ID goes to the computer for verification, and if it passes the computer sends a signal to the Arduino telling it to unlock the door. This made it easier for me to update access-control since I didn't need to reprogram the Arduino all the time, and it allowed me to integrate it into my room automation system and provide a web interface so I can unlock my door from anywhere (in case I need to let someone in when I'm not around).

--- The electronics - main components: ATMega328 microcontroller, FTDI USB serial interface, TA7291P motor driver, and ID-12 RFID reader.

It's amazing how productive you can be in one night if you really don't want to do school work :)

Wednesday, June 23, 2010

Room automation

[This is my 2nd "catching up" post - I started this project in summer 2009 and am just getting around to documenting it]

After finishing my 4 channel light dimmer, I wanted to create a comprehensive room automation system, inspired by fellow MIT student Zack Anderson's MIDAS project.

Having just been introduced to the world of Arduino, I chose to use an Arduino as the central microcontroller. In order to make the automation more useful, I hooked the Arduino up my server (a Dell PowerEdge 2500 deskside server I picked up for free on Reuse - MIT's recycling/reuse mailing list) so that I could integrate a web interface and more sophisticated control logic. More on this later.

The first step to the automation system hardware was getting all of my lights computer-controlled. I piggy-backed my 4-channel light dimmer to the Arduino's serial pins such that messages from my computer would go to both the light-controller and room-automation Arduinos. Next, I needed a way to turn my room's built-in fluorescent lights on and off. I decided that replacing the wall switch with a relay would probably get me in trouble with my dorm's facilities manager (not to mention the risk of electric shock since I couldn't access the circuit breaker), so I was constrained to physically flipping the switch externally. I built a servo mount that screws into the switch plate and holds a servo next to the switch - the servo's arms are then able to flip the switch on or off. Pretty simple. Unfortunately, this covered up most of the switch, making it hard to manually turn on the lights, so I attached a GP2D02 IR distance sensor so that I could wave my hand in front of it and the servo would flip the switch. Cool, now I've got a computer-controllable, hands-free light switch.

Next I wanted to motorize my window shade. I have a theory that I wake up more pleasantly and in a better mood if sunlight is slowly let into the room in the morning rather than using a jarring alarm clock. I made a trip to American Science and Surplus
(an awesome surplus store with all sorts of electronics junk that you don't need) to browse for a decent motor to use. I found a neat motor+gearbox that spun at about 100rpm (a little slow, but not bad) and had a built-in microswitch for counting revolutions for $5. I hooked it up to the arduino with a TA7291P motor driver, and wrote a simple program that would rotate the motor to a specific position. To my dismay, the whole gearbox assembly was fairly loud when spinning at full speed, which was a problem if I wanted to peacefully wake up by opening the shade. I tried PWM, but that caused the motor to "whine" and even then I couldn't get the motor to go very slowly before it would just stall. Luckily, the TA7291P has built-in voltage control, so I hooked up a transistor which allowed the Arduino to switch the reference voltage and be able to drive the motor at a low voltage. By driving the motor at a lower voltage, I achieved a slower speed than possible with just PWM, while also avoiding the "whine." At the low speed, the window shade takes about 10-15 minutes to fully open, which is perfect for a gentle awakening.


[Funny anecdote: I stopped by Microsoft's booth at the career fair and happened to mention my room automation - the recruiter chuckled and let me know that one of their normal interview questions is how to design a remote controlled window shade.]

--- The main control board. Notable components from left to right: filter cap (since servo is on 5V supply), 5V regulator, transistor for switching window shade motor speed, TA7291P motor driver, and at the very end (but not visible) an Arduino USB board.

I guess now would be a good time to mention how my computer fits into the picture. One of my big requirements for this system was the ability to control my room over the web from my phone or iPod Touch. There's an Ethernet shield for the Arduino, but since I already had a server running in my room 24/7, I just hooked the Arduino up to that via USB. I wrote a backend server in Java that handled the serial interface to the Arduino and exposed a socket interface to the rest of the computer. The web frontend was made with PHP, which opens a socket to the Java backend in order to send commands to the Arduino. This allowed me to turn on/off my room light, dim 4 other lights, and control my window shade all from a simple web interface. I also wrote a "wakeup" cron script that opens the shade slowly every morning.

Now that I've got the basic pieces in place, I'd like to abstract the idea into a simpler all-encompassing system. I envision being able to select moods or activities rather than setting the brightnesses of each light, and I'd like to integrate music and movie control as well. By next year I want to be able to click a single button on my iPod touch, and watch as the lights dim, the window shade closes, and a movie starts playing.

Saturday, May 1, 2010

Catching up on projects - 4-channel light dimmer

I haven't posted anything in a while, which means I've gotten behind on documenting the projects I've been working on. So to start catching up, today's project is the 4-channel computer controlled light dimmer I built a summer ago. Basically it allows me to dim 4 incandescent lights independently simply by sending commands over a serial interface.

So how does it work? Well, to actually dim lights running at 110V AC, it's not as simple as putting a resistor or two in series - this would dissipate tons of power (if you can even find a resistor big enough) and would be extremely inefficient. The trick is to essentially turn the light on and off really quickly. Then you can adjust the ratio of on-time to off-time (duty cycle) to adjust the brightness.

This is basically pulse width modulation (PWM), but there's a catch - since the lights are already running at 60Hz, any old PWM frequency won't work. For example, if you tried to arbitrarily use PWM at a frequency near 60Hz you would likely notice a beat frequency in your light brightness. Sometimes the PWM on-cycle would occur when the AC voltage is near its peak, but a little bit later the on-cycle might occur when the AC voltage is near zero leaving the lights dark (even though they're supposed to be on). So the obvious solution is to base the pulse width modulation off the AC line frequency.

To do this, the controller needs to be able to detect the phase of AC line - the easiest way is with a zero-crossing detector. In my circuit the 110V AC is stepped down to ~9V AC, this is then rectified (but not filtered) to produce a nice 9V positive voltage signal that is essentially the "absolute value" of the AC version. Using this, it's very easy to detect when the AC line voltage is crossing zero - just hook up that rectified signal to a microcontroller pin (through an optocoupler to protect it).

--- What the rectified AC signal looks like, with the zero-crossing input overlaid - notice the "blips" near the zeroes.

The microcontroller (I used an ATMega328 - Arduino) has an interrupt handler that fires when the AC line crosses zero. When this occurs, it waits for a certain amount of time before putting the output pin high. The amount of time that it waits is calculated from the desired brightness. For example, 100% brightness means the microcontroller should not wait at all, while 50% brightness means it should wait ~4.16ms before setting the output high (At 60Hz, the line crosses zero every (1/2)*(1/60Hz)=8.333ms, so 50% brightness means the light should be off for half that time).

All that's left is actually switching the lights on and off. My controller uses a triac triggered by an opto-diac - this provides optical isolation which is preferable to just using a 5V trigger triac by itself.

Once I had everything all wired up, I stuffed it inside an electrical box - since the box is metal it helps shield some of the RF interference generated from switching the load and more importantly provides a grounded casing for safety.

--- The beautiful mess inside the electrical box. Note the fuse in the bottom left corner for safety. You can also see the ATMega328 in the upper-right.

--- The completed light dimmer. I added LEDs next to each outlet so you can see which channels are supposed to be on.