toshism

Matthieu Aubry, Julien Rouvière and Xavier Maurice have developed a drum system that uses two cameras to track drum sticks in real-time. This allows the player to drum away on virtual drums, the computer analyzes what the player is hitting in the virtual world and plays the corresponding sounds.

“Video acquisition of the 2 cameras simultaneously, at a frequency of 30Hz. At this step, image data are huge: 640 * 480 * 3 bytes * 2 cameras * 30Hz = 55Mb/s transferred on the firewire bus! Because we have the 2d positions of the sticks on the screen, and because we know the position of the cameras in 3 dimensions, we can determine via geometrical algorithms the 3d positions of the sticks. Using a positioning algorithm, we determine if the stick is hitting a drum element. We then compute the sound localization and the volume, depending on the speed.”

(more…)