Unity creates a game engine for 3D game creation targeted at PC, Mac, Wii, web and iPhone. In scoring a couple of recent iPhone games in Unity, I needed a means of adding interactive music; usually I would employ fmod or wWise for this sort of thing in a console or standalone desktop game, but the footprint issues are such that it’s not workable for iPhone, and those features are overkill for what I’ve needed.

It is possible to integrate fmod with Unity, by the way; it’s just not very practical on the iPhone. The startup time alone appears to be increased by 6-10 seconds on a 3Gs (and Unity already has a 7-12 second startup time on the iPhone as it is). Cool interactive audio is not worth making an iPhone user wait 20 seconds or more for the game to start!

So I wrote a simple engine for managing interactive music in Unity specifically for iPhone game development. It’s below, along with a simple demo app that exercises some of its capabilities. I’m placing it under a Creative Commons license, so feel free to grab it if you deem it useful in your project. I will also continue to iterate over it and improve it with further Unity games. I’ve zipped this up in an example project, but you’ll find the example audio files are missing; just add your own if you want to run it.

How It Works

You can use much of the Music Manager by simply dragging the script onto an object (such as the Main Camera, which will already have an AudioListener) and then dragging-and-dropping music files into it in the Unity editor.

Here’s a screenshot of the properties it exposes in the Unity editor:

This engine employs the concept of musical buckets called “Sequences” which the audio programmer batches together using metadata: the developer provides a name for a sequence of music, such as “level1.danger” or “victory” and then the engine manages playing layers based on that metadata at runtime. Various looping, fading, playing outros, and layering of loops in a sequence and between sequences is supported and can change based on player actions.

I also added an extension of the basic MusicManager which simplifies the process of adding music in a few very common sequence categories, such as menu music, action music, ambient sound, overlaid layer effects, and victory and defeat music. This means that if you’re building a typical game, you don’t need to name these sequences, and you will instead only need to name your own custom sequences.

The audio clips within a sequence can be played randomly, or in the order in which they are listed. So if you have a number of clips in a sequence that, for example, ratchet up tension as a player moves deeper into a level or closer to some goal, you can move from one clip to the next based on player events (events might be fired when the player acquires or uses objects, enters certain physical areas of the level, solves a puzzle or defeats an enemy).

On the other hand, if you have a looser sort of game, you can add multiple clips to a sequence and have the Music Manager play them randomly, creating a random arrangement of music each time the player plays the game level. You can choose to loop the individual clips or the entire sequence.

You can also play layers of sound on top of clips, but on the iPhone be aware that only one compressed audio file can play at a time, so these layers should be uncompressed and must therefore be very short in duration.

These layers can be positional, if you add them as mono files. Compressed stereo files cannot be positional in Unity on iPhone (the sound will come from everywhere, not from one source).

There’s a corner case where this positional/mono issue becomes important to the music you use: if you have source music in your game, such as a jukebox or radio that is playing music that should be heard relative to your player’s character, you’ll need to make sure that source music is added to a Sequence as a mono file or else you won’t be able to pin it to a specific position, and you will need a custom Sequence — that is, add the file via a new script using a new Sequence name — so you can then move the Audio Source around to the correct 3D location. Unity will handle 3D positioning automatically based on where you place the Audio Source and where your Audio Listener is located (the Audio Listener is attached to the Main Camera by default, and there is only one per scene).

Example App

Below is an example app to illustrate, using some music I composed for an upcoming iPhone title called “Revolt.”

Note that the Unity web player will NOT function in Safari on Snow Leopard, since that version of Safari is 64-bit. On Mac, you’ll either need to run this in Firefox or turn off Safari’s 64-bit support.

API

Unity exposes API’s in C# and in Javascript; I’ve written these two files in C#, but a Javascript project can include and use them. There are only two source files needed; simply drag them into your project Assets, then drag one of them — usually the LevelMusicManager — onto an object that should handle playback of audio (often the Main Camera).

To programatically add a file to a sequence, you’d invoke something like the following in your script:

// Let the MusicManager be assigned in the editor:
public LevelMusicManager musicManager;


// Let your array of audio clips be populated in the editor:
public AudioClip[] myCustomMusicFiles;


// In your Start() method, add the files to the manager:
musicManager.AddAudioClip(myCustomMusicFiles[0], "my.custom.tag");


To play sequence of files, you’d execute a command such as this:

musicManager.Play("my.custom.tag");


Further info about the API and how properties such as looping are handled can be found in the header of the MusicManager.cs file.

AI Influence

Geeking out a little bit…

The Music Manager is modeled after event-based AI (Artificial Intelligence) techniques, such as that used for enemies trying to find, follow, and attack a player. In the AI world, a state machine tracks the current state of an enemy, player, or other object, and based on state changes it executes certain patterns of behavior, which may change the object’s state further, causing another pattern of behavior to occur. These updates can occur at varying intervals, depending on rules established in the behaviors themselves, so they’re not necessarily pinned to frame-based updates.

This isn’t a giant switch or if/else loop; in many AI systems, the behaviors themselves are modeled as objects which can be plugged in and subsequently removed without adding giant streams of conditional logic to the object itself. This architectural strategy has the advantage of allowing developers to add new behaviors without needing to update the objects that execute those behaviors. There’s a very low tax to adding new behaviors to existing objects, even when those behaviors are very complex.

Similarly, the Music Manager considers a musical sequence as an object with AI. When its state changes, it updates its behavior; the “behavior” is the playing of a new clip. That clip may have several properties, in that it may be played at various locations, with varying DSP such as reverb or sound occlusion characteristics, EQ, fading and looping, etc. These properties can be added without complicating the core sequence and metadata logic.

Put simply: The intention is to make the system modular and very easy to update, which is critical since no audio middleware does 100% of the job needed on a given title, and adding the last 10-20% of custom functionality is often a beast to implement.

Notes

A few problems specific to Unity on iPhone should be noted:

• Unity, unlike fmod, has no audio codecs itself, so it cannot interpret ogg (or other) files in software — it is limited to the iPhone hardware. This imposes a critical limitation: The iPhone can play only one compressed audio file at a time. For this reason, the overlaid effects must be uncompressed, and should be very short and bounced at the lowest reasonable bitrate.
• Also because of this limitation, it’s best to bake ambient sound design and music into the same few ambient loops if you expect to play both types of sound at the same time, as the iPhone cannot mix ambient tracks and music tracks in real time if both are compressed — which they should be, considering the footprint limitation of the phone, and considering that Apple will not distribute any app over 10MB in footprint over the air (meaning while the phone is not connected to computer), which significantly impacts sales.
• Unity also has some timing issues with audio loops, so you can’t be as clever with loops as you can on other platforms (no looping in the middle of phrases, or it will be subtly noticeable due the timing hiccup that can occur). You may also find it helpful to hide the restart of loops beneath sound design; this is an old hack, but useful considering the audio loop timing issues in Unity’s iPhone product.

Bugs, Suggestions, Future Work

As I continue to work with Unity on iPhone, I will continue to update the functionality of this Music Manager. However, it’s likely that I will not use this for non-iPhone projects; for those, I will still rely on audio middleware packages, where it makes sense to do so.

Download

Here’s an archive containing the Unity project that consists of the MusicManager plus the example scene and example script, minus the audio clips which are used by the example (you can add your own audio files if you wish to run the example scene).

No warranty is expressed or implied, use this at your own risk. You may use it in commercial or non-commercial projects and update it however you like without any need for attribution. However, if you do improve the code, please drop me a line so that I can include your improvements.

unityMusicManager.zip