Thursday, September 30, 2010

Robot tracking experiments

As an organizer of MIT's 6.270 Autonomous Robot Design Competition I've been working on an improved vision-based system for tracking the contestants' robots on the playing field.

(Sidenote: I competed in 6.270 last January and at some point I'll write a whole post or two about my experience. To sum it up, I had an awesome time competing, which is why I'm now an organizer of the competition)

The basic concept is that we wirelessly feed each robots its coordinates throughout the round and act like GPS to help the robots navigate. However, this isn't as easy as it sounds.

Our approach is to mount an overhead camera facing down at the playing field and then analyze the video to find special "fiducial" patterns on the robots. This isn't too hard in a controlled environment, but it gets tricky when the system has to ignore other objects on the field or when pieces of a robot go flying after it slams into a wall (which happens quite often during testing!).

The fiducial patterns I came up with look like this:

Essentially it's a white square with one white corner, and then a few bits of information in the center.

To track these, each frame of video is run through a square detection algorithm. Then for each of these potential objects, each of the 4 corners are checked to find the reference corner - now the software can tell where there are objects and their orientations.

Finally, it checks the inside of each square to see which sections are white and which are black - these are essentially bits that we can use to encode the team number in order to distinguish multiple robots.

So far I put together a simple proof of concept that can track and identify a bunch of these fiduccial markers in real time:

You can see how it has ignored all of the distractions placed in the view and labeled each of the markers it finds. It places a large red dot in the corner that has the reference mark so you can see the perceived orientation.

The software uses the OpenCV library for video capture and processing. The C code is available at github under the MIT license: It should be updated moderately often as I get this system fully fleshed out in time for this January's competition.

Hopefully this was an interesting peek at what's going on behind the scenes to get the competition ready for January. If you go to MIT you should definitely consider competing, or if you live in the area you should definitely check out the final competition at the end of January!

Friday, September 3, 2010

Using a GP2D02 IR distance sensor with Arduino

Just wanted to share an Arduino library I made over a year ago which I just realized might come in handy for others playing with the popular GP2D02 distance sensor.

The GP2D02 is a neat digital distance sensor made by Sharp which uses reflected IR light to measure distances from about 3" to 3' (I think it's been discontinued now though). The fact that it has a digital output is convenient when using a microcontroller, although since the Arduino has analog inputs you might also consider using the analog variants like the GP2D12 or GP2Y0A02 instead.

To install the libary, you need to download and place the GP2D02 directory into the Arduino libraries directory and then restart the Arduino IDE.

Once you've installed the library you can set up a sensor like this:

#include <GP2D02.h>
  o|       |---Black--------|Gnd
   | GP2D02|---White--->|---|Pin 2 (note the interfacing diode)
  O|       |---Red----------|+5V
   |_______|---Yellow ------|Pin 7

const int INPUT_DATA_PIN = 7;
const int OUTPUT_CLOCK_PIN = 2;


Then you can get the distance from it like this:

byte distance =;

And that's it!

(Note that the shift-in function in the library was adopted from and modified in order to account for the timing requirements of the GP2D02 sensor)