Archive for July, 2008

Multi-touch Mouse Emulator

I’ve added a mouse emulator to the Bespoke Multi-Touch Framework. The emulator is UI independent and passes mouse messages to the operating system. I’ve included a GUI that runs in the Windows systray and allows the emulator to be started/paused/stopped and for various settings to be adjusted. The gestures are:

Single point
Press and hold – Mouse move
Press briefly – Left click

Two points
Two brief touches next to each other. Move the mouse and double click
Press and hold right point and tap to the left. Left click
Press and hold left point and tap to the right. Right click
Press and hold left point and drag right point vertically– Mouse wheel
Press and hold bottom point and drag top point horizontally – Alt-tab (not actually a mouse command, but seemed useful)

Below are a couple of demonstration videos. The first is just on the Windows desktop, and the second video is a brief game of Starcraft.

 

The Bespoke Multi-Touch Framework

I’m pleased to announce the initial release of the Bespoke Multi-Touch Framework. You can download the release from here. There are a two packages available: source+pre-compiled-binaries; and source-only.

6 sample applications are included:

  • HelloWorld (the canonical intro app and almost one-liner for interacting with the API – outputs to a text console)
  • CalibrationDemo
  • InkDemo (Pen-style interaction on a multi-touch surface)
  • ParticleTrails (presents smoke-like particle effects at each interaction point)
  • SurfaceSimon (2D XNA version of the classic 80s memory game – demonstrates multi-touch hit testing)
  • SurfaceCommand (3D XNA application demonstrating a simple real-time strategy (RTS) style multi-touch interface)

The samples (except for HelloWorld) employ the XNA presentation layer and share a common set of key-commands, such as:

V – Toggle display modes (there are 5 display modes, 4 debug + 1 normal)
C – Calibrate
Esc – Exit the app

This release is admittedly light on the documentation. I’m actively working on the doc and will be updating it soon. I’ve also left out WinForms and WPF sample applications, but these presentation layers should work without issue. The release is labeled 4.0 (instead of 1.0 for a typical intro release) because I’ve been working on this for awhile and have done a few internal releases. Aside from these few disclaimers, the code should at least compile cleanly and hook to a DirectShow capable video camera. Be certain to configure the Bespoke.MultiTouch.Framework.config file to match your multi-touch surface.

I hope you find the framework useful. If you have any questions please let me know.

Paul

Multi-Touch Montage

Below is a video montage demonstrating many of the features of the Bespoke Multi-Touch Framework.

Multi-Touch Ink Demo

I’ve recently added Ink support to the Bespoke Multi-Touch Library. This allows for single-touch, pen-based/writing-style interaction through a multi-touch surface. This video demonstrates the collection of ink stroke data for training and classifying arbitrary symbols.

Multi-Touch Calibration & Particle trail Demo

Below are a couple of videos that demonstration some functionality of the Bespoke Multi-Touch Framework.

The calibration system compensates for differences in alignment between the camera, surface, and projector. This algorithm takes a series of sample points to calculate offset vectors and interpolates the remaining screen coordinates. The system generates a lookup table for quick (no run-time calculation) conversion between camera and screen points, and the calibration table can be serialized for automatic loading.

 

The next video is a demo app that produces 3D particles at every multi-touch interaction point. This is an extremely simple application, and is included in the framework package — coming soon!