Multi-Touch Videos

Last semester I built a Multi-Touch display and a corresponding framework using .NET and XNA. I have much to say on the subject, but just wanted to post a couple of videos today.

This first demo application is called SurfaceSimon. It’s a simple implementation of the classic ’80s memory game Simon. While the game itself is really single-touch (unless I implement a multi-player version) it demonstrates the basic functionality of the library.

The next video is a bit more involved, and demonstrates hit-testing (the button presses) and point tracking (labeling a touch point from one frame to the next). This application is called SurfaceCommand and heads in the direction of a real-time-strategy game interface.

3 Comments so far

  1. Seth on March 29th, 2008

    Nice work! Once you get some of the responsiveness issues worked out, it’ll be that much better.

    Did you write the tracking library also? What are you using for the finger tracking?

  2. Paul on March 29th, 2008

    Hi Seth,

    Thanks for the comment, and thanks for your efforts at Yes, I wrote the tracking library as well. The only thing I didn’t create is the image processing and webcam capture. I’m using a package called AForge.NET ( for some grayscale and threshold filters and DirectShow.NET ( for grabbing images from the webcam. The library provides blob hit testing (testing if a blob is within a specified rectangle) and frame-to-frame tracking. The tracking works by labeling each newly detected blob with a GUID and testing if those blobs exist from one frame to the next. I do that with simple intersection testing. If you look closely at the videos you’ll see red rectangles outlining the bounding box of each blob, and debug information at the top of each rectangle. That debug information is optional of course, and maybe I’ll post a video that details the frame-to-frame labeling aspect and how blob merging works. I’d originally looked at the Touchlib library, but wanted a native .NET implementation. I’ll be posting the software, under an open-source license, soon.


  3. […] the initial work on the Multi-Touch table I had issues with coupling between the acrylic and your touch points. You […]

Leave a reply