Audio Analysis and BPM Calculation in a Spreadsheet

Last week I showed the beginnings of my audio analysis program. This week it’s time to talk about the goals. My final goal is to do BPM calculation on various music sources. It should be fairly fast, but there’s no need for a realtime readout. I did some Google-ing, but couldn’t find any freely available Java implementation. So I ended up reinventing the wheel.

I did however look for a little help. Yov408’s explanation on GameDev is an exceptionally well tutorial and introduction into beat detection. If you read the article, you’ll notice that the first thing you need is the energy of the music file. So I added an energy calculating filter to my architecture.

Post Office
Creative Commons License photo credit: wenzday01

See the Google Docs spreadsheet with a visual representation.

Afterward, I wanted to get some insight into the algorithm, so I took some random samples and put them in a spreadsheet. My test song is one with a very very clear beat, so if my beat detection algorithm works on anything, this will be it.

I did have to expand the algorithm a little, to calculate the actual BPM. The document only describes beat detection, but once you got that far, BPM calculation is fairly trivial (just count the beats and divide by the time).

If you open the spreadsheet, you’ll notice that I got lucky with my first sample. Immediately I calculated a pretty good BPM of the song (it is about 128 BPM), however, when I tried another sample of the same song, the result was completely wrong. But you might notice that if I did not use the adaptive algorithm to calculate the threshold value C.

See the Google Docs spreadsheet with a visual representation.

I tried to implement that, and although the theory sounds good, the results were even worse. I’m pretty sure I need to go over it one more time to figure out the best values for the constants (I have a different input range then the article). But it’s a start.

Next week, I’ll try to tune that adaptive algorithm and hopefully publish my code. If you have some experience in BPM calculation, I’d love to hear what algorithm you used. Because there are as many theories out there as there are people calculating BPM counts.

Image credit

Hans Hans and The Gang, The Lost Vikings in Their Teens

Games based on cartoons. It seems like a match made in heaven. Yet only very few games manage to pull it off. Leaving Loki’s Lockup does. It’s based on an up-and-coming Nickelodeon cartoon called Hans Hans, the Biking Viking.

Upon playing the game, I immediately thought of The Lost Vikings. It’s an old DOS (and other platforms) puzzler game that gives you a team of Vikings that need to work together to obtain their goal. They each have their speciality.

the_lost_vikings

Just as The Lost Vikings, Leaving Loki’s Lockup involves Vikings working together. They are both nicely animated, funny and inventive.

Leaving Loki’s Lockup does fall a bit short in the control department. They aren’t very fluid and it takes a bit getting used to. But once you know the quirks, this is another free casual game that’s very enjoyable.

Another great game found via JayIsGames

Audio Analysis Architecture Based On JLayer

As mentioned last week, my next project revolves around audio analysis. The first step is acquiring data and for that,  I had already found the perfect Java solution. JLayer makes it easy to obtain data, but a sound file contains very large amounts of it. This post goes into a basic architecture to tame that data and get it into a form that can be processed.

downloadDownload the NetBeans example project

There are a few ways to get the wave data out of an mp3 file using JLayer. You could use the MP3SPI driver to play data through the JavaSound API and capture the bytes. But there’s no reason to make it that complicated.

JLayer can stream data to its own AudioDevice class. This is a callback class that has hooks for opening and closing a device, which we don’t need. The important hook is the one that sends the bytes to the device. This is where you can capture the raw stream. Most audio analysis, however, doesn’t use the raw stream, but averages the data to reduce the amount of data to process.

So that is how I ended up with my layered architecture:

  1. JLayer itself decodes the mp3 file and offers the byte stream to …
  2. … the audio device. This layer does only very basic processing. In the current example project you’ll find a device that averages the samples. I could add Fast Fourier Transform in the future.
  3. The final layer are the actual brains. I’ve called them “processor”. This is where the magic will happen.

That’s all there is to the current version of my project. For now it’s just a zip file, but if there is interest, I might put it on some public VCS.

The main advantage of the current architecture are:

  • There’s only one dependency to outside projects: JLayer. This has advantages for portability.
  • Splitting the basic processing that every analysis project needs anyway into its own layer, frees the actual processing from doing actual processing.

downloadDownload the NetBeans example project

(Image credit)

5 Free Audio Analysis Tools

Now that my previous project is finished, it’s time to start the next. In an entirely different direction, I will now tackle the field of audio analysis. My first step was to take a look at what’s already out there and I must admit, it was overwhelming. There are a lot of very specialized tools for just one task and there are extremely general tools that throw in everything and the kitchen sink.

Here’s a summary of the most interesting program I found and which I will focus on a little more in the future. You’ll notice that most of the audio tools you can find for free are toolkits with only basic GUI frontends. I didn’t really find anything particularly user friendly.

  1. Vamp seems to be the audio analysis plugin framework that has the most traction within the open source community. It is fairly complicated to get everything configured the way you want, but there are many existing plugins available, so it’s probably worth your time.
  2. Just as Vamp above, Marsyas is a research driven framework for audio analysis and more. It seems to have more industry support, but I guess most of that is closed source. The list of links on their project page is long and intruiging, and definitely worth a little more research.
  3. SoundRuler is one of many tools that offers a view on the waveform, fast Fourier transform (FFT) and some more. What makes this noteworthy is that it is geared at beginners (me) and has a bunch of manuals. However, upon installing I didn’t really find it very userfriendly, but that might be only a first wrong impression. I will go through some of the documentation in the next week.
  4. WaveSurfer is a similar program, but it seems to be more humble in goals and therefor a little easier to interpret and work with.
  5. I’m not sure if the CLAM project will suite my needs, but my interest was tickled by their graphical way of configuring the tools. Certainly something I’ll be writing a little more about later on.

Overdubbing
Creative Commons License photo credit: lazha

So far I haven’t really found anything that I’m really happy with. I have tried all of the above, except CLAM and then another 5 to 10 that didn’t make the list. The only result is, that I can’t help but feel lost. Those tools dump boatloads of information to my screen with very little context or explanation.

I can understand the raw waveform plot, I can even interpret the FFT plots, but all the other stuff is just voodoo. I have the feeling all those tools were programmed by true audio geeks, with an extremely deep knowledge of waveforms. So if you are one of them, you’ll love them all. But what I need is a tutorial or course in sound analysis and sound properties. How to interpret all the little lines and numbers.

Image credit (check it out, cool stuff)

GTA Chinatown Wars, the Story and End

Since I have completed the storyline, this might be my final entry on GTA Chinatown Wars. The game only shows me a 48% completion, so I still got a lot of stuff to do. I don’t think I’ll be going for 100, but I might take on a few additional missions. According to the in-game counter, I’ve put over 10 hours in the game, so I got my money’s worth. However, there are still things I’d like to discuss.

gta_cw_main_characterI haven’t talked about the story yet, because I didn’t think it was worth mentioning. It starts of pretty so-so, but in the end things come together nicely and I guess it’s an acceptable excuse to have a big blowout fight. For a game it’s pretty ok story. It might even make a movie worth watching, but it wouldn’t work as a book. As far as storytelling is concerned, game makers are stuck between a rock and a hard place. On one side, you don’t want to spend too much time in cutscenes to show off your story, on the other hand, it’s difficult to tell a story if you only have action sequences. Ever since adventure games and full motion video games went out of style, we only have games like Half-life left who walk this fine line pretty nicely

gta_cw_gunfightIf you complete the story missions, you will undoubtely for ever remember the last three missions. The game ends in some seriously intensive gunfights. If you don’t like the gunplay in Chinatown Wars at all, I’m affraid you will be a bit bummed by the end. But by the time you make it this far into the game, you’ll probably have a pretty good grip of how to handle the situation. I thoroughly enjoyed the final missions. Althouh I had to replay them quite a few times. The key to victory: take it slow, don’t rush.

After you complete the story, the “Guardian Lions” mission is unlocked. Once completed, you can upload the mission data to the Rockstar Social Club for a bonus. Although it’s a small thing, I really love this online integration (and it cuts back on piracy, so it’s a win-win)

Anyway, if you still haven’t got the game, and aren’t convinced by now, I don’t think I will ever do so, but still: go buy and play the game!

Open Softwear, a Free E-book to Get Started With Wearable Computing

Now is the time, if you ever wanted to get into wearable computing, such jackets that control your iPod, trousers that measure the distance you’ve run, and other general coolness like that. The guys at softwear.cc have compiled an e-book to get you started immediately.

This beautifully illustrated and completely free e-book will give you insight into the materials you need, the circuits to build and the software that will perform the tricks.  Thanks to Make: for the pointer.

Exporting Models From Blender to Flash Sandy 3D, Experiments in 3D

Last week, reader rafbaf asked how to use Blender models in Sandy 3D. There are a few options, for instance Sandy 3D can import the Collada format that Blender exports. But there is also an easier way. A Python script for Blender allows you to export models straight to ActionScript code. However, as you will see in this post, it’s not as straightforward as it seems.

downloadExample FlashDevelop project.

There’s a reason this post is titled “experiments”. It is actually an unfinished work, but I wanted to give you a look at what I’ve already been able to create. This is how far I got for now:

(Either JavaScript is not active or you are using an old version of Adobe Flash Player. Please install the newest Flash Player.)

  1. First, I got an A-Wing from the blendermodels archive. Before I got the first render result shown in the picture above, I had to fix the normals of most of the meshes in there (in Blender: go into edit mode, select vertices and ctrl-n)
  2. Next, get the export script from Dennis’s page and install it. Take care to get the correct directory, it’s not that easy to find. If you have one of the latest versions of Blender, you should look in your user folder. For Windows that’s “C:Documents and Settings<username>Application DataBlender FoundationBlender.blenderscripts”.
  3. If you haven’t already done so, you also need to have Python installed. Python is necessary to execute the script.
  4. Once you got this out of the way, you will need to restart Blender. Check the file > export menu and verify that the ActionScript option is available. If it’s not, you probably placed the AS3Export.py file in the wrong directory.
  5. Select the object you want to export and click the export to ActionScript menu item. It’s probably a good idea to enter a package name, as this will keep your code cleaner.

Now you should have one or more .as files: one for every mesh and texture in your model. This model doesn’t come with textures, so there are none.

Next we are going to use those files in a FlashDevelop project. The first step is to create a new project in FlashDevelop and put the Sandy 3D library inside the src folder. Like described in previous blog posts.

Now inside the src directory, you should create a directory with the same name as the package you specified in step 5 above. If you forgot the package name or are not sure, you can open one of the files that was created and check the first line.

That’s pretty much it: add the meshes as Shape3D objects to your scene, just like you would add a cube or other shape:

1
scene.root.addChild( new Mesh() );

(you might want to add an appearance too)

You’ll notice I’m skipping a few steps here, but they are all very basic. If you download the example project and read through my previous blog posts, everything should be clear (feel free to comment if it isn’t).

You’ll notice a few issues/problems (correct me if I’m wrong):

  • The 3D object I choose, consists of several meshes. Each with their own texture (material in Sandy3D terminology). The export plugin doesn’t really help you on this point. You need to create every mesh and add them individually to the scene.
  • There seems to be an issue with the normal calculations. I did correct them in Blender as you can see in the small rendering at the top of my post, but still there seem to be missing triangles in the Sandy 3D version.
  • It looks like there is an issue with a bounding box or face culling. I still need to investigate this thoroughly.

To be continued…

downloadExample FlashDevelop project.

Tweet-Tweet: Backup Your Twitter Account Into Your WordPress Database

One thing that has been bothering me with social services is the difficulty of creating a backup. What would happen if Twitter suddenly dissapeared, or even worse, what would you do without your Delicious bookmarks? It has happened a few times before, services like Geocities and Stick-it were closed almost over night (I’m exagerating, but if you are on holidays in the wrong period, you might come back to see your data gone)

Backups of your local system are important, but so are backups of remote systems. Although people tend to thrust third parties a lot more, most of them don’t offer any kind of data guarantee. Certainly if you’re not paying for the service, there’s very little you can do if your data is gone.

Twitter is certainly a bad offender for me. Although I haven’t been tweeting that much lately, I do try to share interesting insights and valueable sites, so I don’t want to see those go. So when I saw “Tweet Tweet”, a WordPress plugin to backup your tweets, I was immediately charmed. I noticed it hasn’t been updates in a while, but I think it is the only plugin that actually stores a backup copy of your Tweets.

As an added extra, you can integrate the tweets in your blog. Which I’ll probably do at some point, when I get around to updating the layout.

Have fun and don’t forget to backup.

Image credit.

Grand Theft Auto Chinatown Wars FAQ on Squidoo

I’ve been continuing my pursuit for GTA Chinatown Wars completion. The game has really drawn me in and it gets better every time I play. I also tend to loose more and more time, which is a good thing. If you happen to have the game, but have put it aside, I must urge: give it another try. The controls take getting used to, but it will be so rewarding.

GTA Chinatown Wars FAQ lens.

To extract at least something useful out of the whole experience, I’ve begun to document some of the questions I encountered and the answers I found (or didn’t, for some). I’ve wanted to try out Squidoo for quite some time, so here is my GTA Chinatown Wars FAQ lens. It doesn’t have the usual walkthrough you can find on pretty much any gaming site. This FAQ has questions that aren’t answered anywhere else, but that might be important for you to fully enjoy the game. Feel free to add extra questions.

And don’t forget to add me to your friends list if you also have the game. My Nintendo DS Friend Code code is 335201600798.