Playing sounds and music on iPhone, which API to use?

Member
Posts: 129
Joined: 2009.03
Post: #1
Hello,

Just looking for a little advice, re: playing music and sounds on iPhone. Smile

I'm porting a game, and the existing audio assets are either .wav's or .mod's (for the music). I will probably be converting the .mods to .mp3 or some other format, because apparently, .mods take up a lot of cpu per frame (for mixing channels I think), and I want to minimize the % of cpu used per frame for audio.

I'm wondering if I will need to change the format of my .wav sound effects too? I was reading in the apple developer docs, to use .caf format, would that be faster?

Lastly, taking in to account that I want to minimize cpu usage for audio, which API / framework should I be using to play sound effects, and also music?

Many Thanks,
Quote this message in a reply
Moderator
Posts: 704
Joined: 2002.04
Post: #2
I'm nowhere near being an audio coding guru, but I'm using AVAudioPlayer in several projects for playing music; whilst you'll probably need to code a wrapper for anything beyond basic playback, it is very easy to use - which with my non-guru staus is very welcome Rasp - and if you store your music as .mp3, .aac, or a couple of others (Apple's developer documentation has a table) decoding is assisted by dedicated hardware, taking some of the strain off of the CPU.

Hope that helps Grin

Mark Bishop
--
Student and freelance OS X & iOS developer
Quote this message in a reply
Member
Posts: 129
Joined: 2009.03
Post: #3
Thanks Mark, yep, that kinda confirms what other people have been saying too, and sounds like just the ticket to play back music. Smile

Cheers,
Quote this message in a reply
Moderator
Posts: 3,579
Joined: 2003.06
Post: #4
Yah, AVAudioPlayer for music because that can use hardware decompression.

You can also use Tremor for Ogg/Vorbis, but that does cost some CPU, although it isn't ridiculous if you have some to spare.

For sound effects (short sounds), the best bet is OpenAL.
Quote this message in a reply
Member
Posts: 43
Joined: 2009.10
Post: #5
I agree with what everyone else said:

Music: AVAudioPlayer
Sound Effects: OpenAL

Specifically you should look into the Sound Manager wrapper for all of them. It works pretty nicely.

http://www.71squared.com/2009/05/iphone-...d-manager/
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #6
Like everyone said: OpenAL+AVAudioPlayer - but that's the easy part...

The tricky bits are understanding audio sessions and what format is suitable for your music, since you only get 1 hardware decoded stream at a time (and some other app might already be using it). The "Audio Development Tips for iPhone" video in the Advanced Videos series on iTunes U lays it out nicely (Apple login required).

Also, if you want to save yourself some grief just target OS 3.1 and above - the 2.x series has some serious bugs and inconsistencies.

@alerus I'm not sure I'd recommend that Sound Manager code - last time I looked it used a somewhat naive OpenAL implementation (and was incomplete to boot). Should be OK for short one-off sounds but I wouldn't trust it for anything moderately more complicated.
Quote this message in a reply
Member
Posts: 43
Joined: 2009.10
Post: #7
Frank C. Wrote:@alerus I'm not sure I'd recommend that Sound Manager code - last time I looked it used a somewhat naive OpenAL implementation (and was incomplete to boot). Should be OK for short one-off sounds but I wouldn't trust it for anything moderately more complicated.

Do you have any other recommendations? Its been fine in my preliminary tests, but I also haven't given it a big run through. The only problem I found was a minor bug where it would free the sound memory if you hard stopped a sound, but that was pretty simple to fix by removing the left over code.
Quote this message in a reply
Moderator
Posts: 3,579
Joined: 2003.06
Post: #8
Hmph... For some reason I read the posts as referring to the old SoundEngine code Wacko

I haven't checked out that Sound Manager code myself. I do know you can put together your own full OpenAL management code in under 500 lines though.
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #9
Yes definitely stay away from the old "SoundEngine" - sadly there's probably thousands of apps still using that...

I don't really have any recommendations - I stopped looking and rolled my own long ago (a plain C API for my sanity). Most iPhone audio code I see online has fundamental flaws, like sounds that play themselves, or one monolithic "play" function. These designs really limit what you can do, but still might be OK in some situations.

When looking for (or writing) a sound API for games it's best to separate the playback mechanism/speaker from the actual audio waveform/effect. Then to play a sound you basically say: speaker->Play(effect). Ideally each speaker should play exactly 1 sound at a time and you set attributes like gain, position, pitch, looping, etc. on the speakers instead of the audio (and at any time too, even during playback). Since speakers don't care what audio they play, and since each one only plays 1 sound at a time you get some basic but powerful functionality. E.g. Multiple speakers can play the same sound simultaneously, like the same explosion sound at different positions/pitches. Or a speaker can string together a series of sounds that cut each other off - a lift playing a "moving" loop then a "stopped" sound when it comes to rest, or a character's voice playing a taunt then saying "ouch!" when they get hit, stopping the (perhaps incomplete) taunt.
Quote this message in a reply
Member
Posts: 43
Joined: 2009.10
Post: #10
I've yet to really dive into OpenAl, so I could be wrong, but my impression from the Sound Manager wrapper I linked to is that it does separate sounds from the audio. For instance, if you want to play an effect, you can instantiate multiple instances of the effect playing at once by using different sound sources with different pitches, locations, etc., all from the same buffer. Or, a sound source can be overridden by specifying the same source id as a previously used source.

Is there a reason the source id is not appropriate for what you're describing?


EDIT: On further evaluation it appears that the stop control is less powerful than the play control. The stop control stops all sources playing a specified buffer. Though it should be a pretty simple modification to add a method that allows a person to do selective source stopping, rather than all source buffer stopping.
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #11
alerus Wrote:I've yet to really dive into OpenAl, so I could be wrong, but my impression from the Sound Manager wrapper I linked to is that it does separate sounds from the audio. For instance, if you want to play an effect, you can instantiate multiple instances of the effect playing at once by using different sound sources with different pitches, locations, etc., all from the same buffer. Or, a sound source can be overridden by specifying the same source id as a previously used source.
I might be looking at a different version of that code but I don't see a way to override or ask for a specific a source - it just grabs the next available source or overrides the oldest one (?) You can play the same buffer more than once, but you have little control other that play it and forget it. E.g. you can't move it once it's playing, or change pitch over time, etc. and if the manager decides to replace a playing source it just does it - there's no concept of priority.

Now, I don't want to knock that code too much - it's plenty good for a number of situations. If all your sound effects are short and the last effect always takes priority then that approach will be fine. My game Pinch 'n Pop! basically handled audio that way before I got around to writing a more robust engine.
Quote this message in a reply
Member
Posts: 43
Joined: 2009.10
Post: #12
Frank C. Wrote:I might be looking at a different version of that code but I don't see a way to override or ask for a specific a source - it just grabs the next available source or overrides the oldest one (?) You can play the same buffer more than once, but you have little control other that play it and forget it. E.g. you can't move it once it's playing, or change pitch over time, etc. and if the manager decides to replace a playing source it just does it - there's no concept of priority.

Now, I don't want to knock that code too much - it's plenty good for a number of situations. If all your sound effects are short and the last effect always takes priority then that approach will be fine. My game Pinch 'n Pop! basically handled audio that way before I got around to writing a more robust engine.

I have no personal problems with criticism on this stuff. I visit this board to learn and help others learn, and that doesn't happen without the criticism. So I certainly don't mind even if you knock it a lot Smile

As for the source control, I believe you're supposed to be able to with this method (in the version I'm looking at)

Code:
- (NSUInteger)playSoundWithKey:(NSString*)aSoundKey gain:(float)aGain pitch:(float)aPitch location:(CGPoint)aLocation shouldLoop:(BOOL)aLoop sourceID:(NSUInteger)aSourceID


So essentially what you would do if you wanted source control is you pass SourceID a value other than -1, and it will use that source. When -1 is passed it does default to the next available source.

Some caveats: as I noted previously there is no source controlled "stop" method predefined--stopping will stop all sources playing the specified buffer. Though this is a very simple method addition to provide source controlled stopping.

The other caveat is in the play method above, if you specify a source, and that source is still playing anything, it returns 0 and doesn't do anything. Though this could be fixed by changingthe behavior to stop the source, and rebinding to the new buffer, which can be done in a couple lines of code.


So the criticisms you have given I believe are technically valid, though could be fixed with the minor changes I suggested above. Would you agree, or do you think there is something else its missing?
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #13
Ya I must've been looking at a different version of that code (didn't see any sourceID) though managing AL sources manually kinda defeats the purpose of a high-level wrapper Wink

alerus Wrote:So the criticisms you have given I believe are technically valid, though could be fixed with the minor changes I suggested above. Would you agree, or do you think there is something else its missing?

Shoehorning in new features can only get you so far with the current design, but If it does everything you need then I don't see a problem using it. My main issues with that code remain the same - read my second post again!
Quote this message in a reply
Member
Posts: 43
Joined: 2009.10
Post: #14
I was looking over your second post again and have a clarification question. When you say "speaker" I was assuming this was analogous to and openAL source. Is this correct? If so, wouldn't specifying the source id, be the same, and therefore just as complex, as specifying which speaker should play a sound? If I'm understanding that correct as well, it seems the only functionally different feature is that missing (when incorporating the minor changes I suggested) would be on the fly pitch/position change, rather than setting it at the start of each effect play. Not that such a thing is unimportant, I just want to make sure I'm understanding everything so that if go on to make a more advanced system later myself, I'll know the kinds of things I should consider.
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #15
A speaker is a virtualized AL source. It lets you access similar properties (position, gain, pitch, etc.) but it shields you from having to deal with AL sources directly.

E.g. my speaker implementation uses a pool of 16 AL sources. When you ask a speaker to play a sound it first checks if it already owns a source in the pool and stops it if it's playing, then it asks the source pool manager for a free source. The pool manager returns either the first idle source, or if they're all playing it returns a source of lesser or equal priority. If all else fails it returns nil and the speaker simply doesn't play. This is similar to what that Sound Manager code does but the priority distinction is important.

You've argued that keeping track of source Ids is the same as keeping track of speaker objects - this is true, but keep in mind that you can create hundreds of speaker objects, and even play them all at once, but you can't (or at least really really shouldn't) do that with OpenAL sources. In my example above you'd only get 16 actual noises at a time, but at least you won't slow to a crawl and/or crash.

So, in a game where you know you'll only ever have 8 or 16 sound sources then using AL sources directly should be fine, but if you have something like a shooter with dozens of enemies on screen at once constantly dying and respawning, it probably makes sense to use a virtualized system where each entity has it's own abstract speaker objects.
Quote this message in a reply
Post Reply