Posted on

Transcribing drums into sheet music on the fly

As we were experimenting with our motion capture drumming system, we had the idea of testing it out as the controller for a rhythm game. Games like Rock Band and Guitar Hero show the notes to be played as coloured symbols on a vertically scrolling pane. Our idea was to instead use actual music notation to represent the notes. In other words, we wanted to recreate the experience of sight reading a drum transcription and hopefully make a didactic tool in the process.

The following video shows a rough prototype of our game where the top staff line shows what should be played, the bottom line shows what is being played and each hit is scored based on its accuracy. Notice how the music notation is generated on the fly and in realtime based on what the drummer is doing.

Playing around with this system led to a revelation, the idea of adapting live transcription into a tool that drummers could use to edit music notation in the simplest way imaginable. Unlike other editors, ours would work by letting the user place notes on a virtual timeline and, from that, the notation would be entirely generated automatically. After two years of development we will be releasing the result, Aered, on this website very soon. How it works and what sets it apart from existing software will be the topics of the next batch of posts.

Posted on

Not so transportable air drums

The computer graphics lab where Richard and I used to work had a motion capture studio. These are used to record the movement of actors for the purpose of animating computer generated characters you see in films or video games. The lab’s system is of the optical tracking kind, made by Vicon. It uses a dozen high speed infra-red cameras to film the studio from different angles, and then triangulates the position of markers worn by the actors to reconstruct their movements over time.

I had been getting good results tracking the motion of some gherkin pincers live at 120 Hertz for a 3D modeling interface project. This is when Richard came up with the idea of trying to use the system to do “air drumming”.

We drilled a few holes in Richard’s drumsticks to attach some markers and configured the system to recognize them in real time. After a bit of experimenting, we got this proof of concept working well enough for Richard to drum like he would on his real kit:

When we get closer to the release of Aerodrums early next year, we will explain how we cut down on the few suitcases of Vicon cameras and routers to fit the result in a backpack.
In the meanwhile, we will be talking about Aered, a sheet music editor designed specifically for drummers.

Posted on

High-priced, loud and bulky

Terry Bozzio's Kit

I started playing drums when I was about 9 years old. I didn’t have a drumkit at first, only sticks, so I used to play on whatever household surface was nearest to me. I think it was only after enough of the furniture was pitted by drumstick marks that my parents decided to use their holiday savings on buying me my first drumkit, a brand new Premier Fusion 5-piece.

My neighbours were less tolerating of the racket I made with it than my parents. After enough complaints, we finally came to an arrangement that permitted me to play between 4 and 6pm every day. I remember getting up on a Saturday morning and waiting all day long for 4pm to roll around. If my neighbours cars were gone I used to sneak a few minutes of play.

In my early teens I started playing in my first band with friends from school. We would practice every weekend in the bass player’s home. His parents had better plans for their real estate than to host my drumkit, and I didn’t want to part with it during the week anyway, so I relied on my father to provide lifts back and forth. I must have wasted at least an hour every band practice setting up and tearing down my kit.

When I began college, by the time I got home in the evening, everyone else would be returning home from work expecting a few hours to relax. The neighbours would be putting their baby to sleep, etc. It was always too late to start making noise and, for lack of opportunity, I ended up not drumming at all for weeks at a time.

But enough with my life story, in the next post we’ll be talking about the ideas that led to Aerodrums, and about its remarkably non-portable ancestor.