Beat Tapper / DiMaß

DiMaß¹, pronounced dee-MAHS², is a technique for Direct Manipulation Audio Scrubbing and Skimming. It allows you to quickly search, skim, scrub and slide through audio with continuous audio feedback using a constant pitch time-stretching algorithm.

DiMaß is currently implemented as part of Beat Tapper using the Semantic Time Framework. Beat Tapper is a tool for tagging audio files with beat metadata for Mac OS X. It runs on any Mac (Intel or PPC) with OS 10.4.5 (Tiger) or higher. Any QuickTime-compatible movie with audio can be imported into Beat Tapper.

Here is an example beat metadata file for an excerpt of Blue Danube Waltz by Johann Strauss. The corresponding audio file can be obtained from Wikipedia. Wikipedia offers the excerpt in Ogg Vorbis (.ogg) format; you must first convert it to a QuickTime-compatible format before it can be imported into Beat Tapper. I used Audacity to convert the ogg file to a wav file, which can then be imported into Beat Tapper. Alternatively, you can create your own beat file using the Beat Tapper interface.

To use DiMaß, audio time stretching must be enabled: open up the "Audio Controls" panel using from the "View" menu (shortcut: ⌘K). Select either "Stretchy (Phase Vocoder)", or "Pitchy".

¹ The name is a pun on the German word das Maß, which translates to "quantity of measure". Die Maß, which is pronounced the same way as DiMaß, is a southern German term for "a litre of beer" (think Oktoberfest).
² The "ß" is the German sharp "S", an abbreviation for two S's, not the Greek letter beta.

Beat Tapper v1.5

Released on March 2, 2007.

Download (281 kB)

Related Literature

E. Lee, H. Kiel, J. Borchers. Scrolling Through Time: Improving Interfaces for Searching and Navigating Continuous Audio Timelines. Technical Report AIB-2006-17, RWTH Aachen, December 2006.

E. Lee, J. Borchers. DiMaß: A Technique for Audio Scrubbing and Skimming using Direct Manipulation. Proceedings of the AMCMM 2006 Audio and Music Computing for Multimedia Workshop (in conjunction with ACM Multimedia 2006), pp. 107-114, October 2006.

E. Lee, J. Borchers. DiMaß: Audio Scrubbing and Skimming with Continuous, High-Fidelity Feedback. WWDC 2006 Apple Worldwide Developers Conference (Scientific Poster Session), August 2006.

Personal Orchestra Lite

Personal Orchestra Lite (or POlite) is a simple proof-of-concept conducting application created using the Semantic Time Framework. Using POlite, you can load in your favourite orchestral music and conduct it using the mouse! The tempo of the music will react to your movements -- the faster you move, the faster the orchestra will play, and when you stop, the orchestra stops as well. Moreover, the orchestra will synchronize their beat to your beat -- the system tracks the vertical movement of your "baton" and looks for the beats in the lower turning points of your gestures.

POlite currently works with audio only (optional video support coming soon). The binary release comes with a sample movie bundle, which is an excerpt from a recording of Wolfgang Amadeus Mozart's Symphony No. 40 in G Minor, performed by the Fulda Symphonic Orchestra. The original file, distributed under the EFF Open Audio License Version 1.0, can be found on Johannes Henkel's homepage. New movie bundles can be created by duplicating the bundle structure and editing the Info.plist file.

Before you can conduct, you must first choose a device to use as the baton. Choose this device by opening up the pop-up menu as shown below.

The application will then grab all events from that device for the baton (you should see the black dot move together with the baton). If you are on a computer with only one mouse-like input device (e.g., an iMac), and you have selected it for the baton, you can still quit out of the application by pressing ⌘Q.

POlite v0.1

Released on March 12, 2007.
Sample movie bundle included with this release.

Download (4.5 MB)

Related Literature

E. Lee, H. Kiel, S. Dedenbach, I. Gruell, T. Karrer, M. Wolf, J. Borchers. iSymphony: An Adaptive Interactive Orchestral Conducting System for Conducting Digital Audio and Video Streams. Extended Abstracts of CHI 2006 Conference on Human Factors in Computing Systems, pp. 259-262, April 2006.

E. Lee, T. Marrin Nakra, J. Borchers. You're the Conductor: A Realistic Interactive Conducting System for Children. Proceedings of the NIME 2004 International Conference on New Interfaces for Musical Expression, pp. 68-73, June 2004.

J. Borchers, E.Lee, W. Samminger, M. Mühlhäuser. Personal Orchestra: A real-time audio/video system for interactive conducting. ACM Multimedia Systems Journal, 9(5):456-465, March 2004.

iRhyMe

Coming soon...