The Adventures of the Original Hamster

Most of you have probably seen Jeff Han’s multitouch screen, the Microsoft Surface, and all these other multitouch devices that are popping up everywhere. What’s the first thing that comes to your mind when you see those videos? If you are anything like me you’ll probably want three of them, preferably yesterday. Alas, I don’t have extra 10,000 lying around to buy a MS Surface, so I decided to build my own.

Fortunately, the excellent NUI Group forum is filled with people who have the same intentions and are willing to share advices. There are dozens of blogs describing how to build a DIY multitouch screen. I’ve been planning on building one ever since I saw the Reactable in action a few years ago. Now that I have the time and motivation to actually do this, I have started the building process and I will try to document the steps here on the blog. I will also write how-to pdfs both in English and Finnish and release them once the table is finished.

I don’t want to go into too much detail about how different multitouch surfaces work, but I’ll just quickly go over the science behind the technique I’m going to use. Basically I’m going to build a multitouch table that will use the FTIR method that Jeff Han also uses in his screen.

Source of the image

So basically, you have an acrylic screen that is lighted up with infrared LEDs from the side. Once you put your finger on the acrylic, it will reflect the IR light downwards and your finger would look really bright below the acrylic if you could see IR light. So, you will need something that IS able to see infrared light. For this purpose you use a video camera that has been modified to pass through infrared light and block visible light. Why not just use normal light then, you might ask? Because you want to also project an image on the screen and the light from the projector would disturb the finger tracking.What your camera sees is pretty much like this video (after some image processing to make the blobs brighter)

OK, you have a bunch of bright blobs where someone touches the screen/table. How do those blobs then turn into a cool multitouch screen? You need some software that detects the blobs and then does something cool with the information or sends it along to some other software that might do something even cooler. The weapon of choice for many people is the Touchlib library, but it only works on Windows or Linux. So us Mac users need to use Boot Camp, Parallels etc. or do the tracking with some different software altogether. Unfortunately, there aren’t many options. ReacTIVision 1.4 is pretty much the only free, easily available software for this. It is mainly used for fiducial tracking and I’ve used it in some of my projects. Version 1.4 does plain finger tracking also, but it’s pretty slow when using USB cameras. I will cover the software side some other day in more detail and show you some other options that you might have.

The tracking software uses the TUIO protocol to send the information to Flash, Processing, Quartz Composer, Max/MSP or any other programming environment that understands OSC. Then it’s just up to you what to do with the tracking data.

This is a really superficial explanation on how this all works. I will go on to details within the next days and weeks when I start to document the building process my own multitouch screen. There is tons of information on the Internet about all of this if you are interested. The NUI Group and Google are your friends.

Tomorrow, Multitouch Table Post #2 - Acrylic and the LEDs.

Here’s some documentation of a project we finished this spring at the university. It’s a project for a course in interface design.

Our group (Aku Meriläinen, Danilo Mascarenhas, Marjukka Parkkinen and Matti Niinimäki) decided to create a multi-user interface that would allow a group of people to work together or compete in creating some sort of a moving collage. At some point in the planning process the idea of communism and propaganda came to surface, so we decided to go with that idea. We experimented with different input devices and visual styles, but eventually this is what came out of it.

I’d like you to meet The Interactive Propaganda Generator. It’s designed to have two different sides or “teams” - the communists and the capitalists - that you can control with different input devices to produce a collage of the two sides shooting different items and symbols at each other. The scene consists of three layers: a changeable background, two characters (one for each side) and the propaganda-tubes. Here are the controls explained:

Control 1 - ReacTIVision fiducial tracking

• The characters are selected and moved around on the screen by using paper dolls on a glass table
• You can change the bodies and heads of the characters.
• Included are all your favourite capitalist and communist icons from Ronald McDonald to Che Guevara
• The paper dolls have fiducial symbols attached to their backside and the camera under the table tracks the ID and the position of the symbol
• Tracking is done with ReacTIVision software

Control 2 - PlayStation2 Dance Pad

• You can move the capitalist propaganda-tube around the screen with the dance pad
• One button turns the tube on/off
• Two buttons for changing the ammunition of the tube (missiles, bibles, money)
• Two buttons for changing the background image
• One button for activating the speech option for the capitalist character

Control 3 - PlayStation2 Dual Shock Controller

• Some controls as the dance pad, but for the communist side

Control 4 - Audio Input

• A microphone picks up the voices of the users and when the speech option is activated (from the dance pad or dual shock controller) the characters on screen will open and close their mouths based on the sound input

Here’s the hardware setup. Click the image for details.
All programming done with Quartz Composer. I could release the .qtz file but it’s not really useful to anyone because it depends on a very specific setup. Email me if you’re interested in details.

And finally some video. Unfortunately, we don’t have any better footage of the system in action but hopefully you’ll get some idea.