[Suggestion] Extending audio plugin

My suggestions would be quite easy to implement, as the web audio api already offers all of it:

1) The ability to mix/submix.

With web audio one can do amazing things using effects and filters. But, when you have a set of sounds that should act in a context, it is very painful to edit every single sound. Imagine the steps of a player, the 'swish' sound when he swings his sword and the shot sound from his second weapon, a crossbow. All of them should sound within a cetain context, for example a cave he's exploring, or a forest, where he gets ambushed.

Currently we would need to create the very same effects for all of the sounds and change all of them seperately depending on the player's location. But with a submix, I could create a new audio tag (e.g. "player sounds") and route all of the player's sounds to there, while only needing to change the effects applied to "player sounds". This gets even more intersting when taking into account the enemies and their sounds. A submix "enemy sounds" could be created and a "ambience" mix, where both "enemy sounds" and "player sounds" are routed to. Now I'd only need to apply/change effects of "ambience", instead of hundreds of single audio tags (which also reduces CPU load).

Web audio already supports this. Every audio node accepts input from multiple outputs. There is also javascript example code in the api's documentation.

2) Splitting/merging channels

Currently an audio file is treated as a whole. With the analyser node we can access peak and rms information, but only as a sum of all channels. It would help much if individual channels could be accessed, for example to distinguish the peaks from the left and the right channel of a stereo file. After the analysing they would be merged back into one audio stream.

Web audio already offers this. It's the channel splitter node and the channel merger node.

3) Extending the looping feature

By adding loop start and end we could have some very nice additions to the current implementation. For example a file with a nice intro before the main theme. Loop start would then be set to right after the intro, resulting in playing an intro followed by an indefinitely looping main theme.

Web audio already offers this as attributes (loopStart, loopEnd) of the AudioBufferSourceNode. This should only be offered for sounds of course, as only those are buffered in memory, while music is streaming.

4) Recording feature

I may be the only one wishing this, but I add it to this wish list nevertheless. An option to record the final output to disk. This could be realized with the ScriptProcessorNode and its AudioProcessingEvent.

Ashley

Let me again point out that all the features are already there. They would "only" needed to be made accessible in C2. So the workload would be quite low compared to features that would have to be programmed first.

I agree with these points especially the loop start and end triggers.

My additions to this would be to be able to control positional sound ranges and doppler values for individual sounds (or tags).

Sometimes you want only some sounds effected by doppler and you may want some sounds to have larger ranges than others. Currently you can only have 1 range and enable doppler for ALL sounds which kind of makes their current functionality super limited.

Excellent post(s)!

+1 to all of the above!

I also wish that some visual, "behavior" like additions were made to the audio plugins. An Interface and Functions that are easy to access, link and change their parameters, like controlling volume, time scale (which translates as tempo+pitch stretching), and all of the advanced audio effects. All these can be coded with events and variables, but most of the time it's difficult and the more sounds you have, the more complicated it gets. I guess something like Families and Containers combined, with all of the above functionality accessed thru the object type properties panel.

I hope I make any sense   <img src="smileys/smiley9.gif" border="0" align="middle" />

#3 for sure. I've always missed that from working with other engines and it's a small pain working around it.

Bump! Those suggestions are amazing.

Isn't possible to mix audio files using web audio API with current audio plugin?

I'm basically talking about the layers for sound effect.

Is it possible to do a simple voice recording app for mobile? I'm looking for a solution, preferably using the open source Opus codec to have small file size. Then sending and retrieving the audio files from a server.

The audio plugin is very complicated because it covers four audio APIs (HTML5 audio, web audio, phonegap audio, and an old appmobi audio API that probably doesn't need to be there any more). There are still major platforms without web audio support (mainly IE, old Android browsers) so don't forget all these features need to have some reasonable fallback when running with HTML5 audio. My point is you are trying to make it sound easy to implement but it's not quite that straightforward. Particularly with #3 (extended looping), we actually still play music tracks with HTML5 audio routed through the web audio API due to its ability to stream which reduces latency and memory use, but it doesn't support anything other than looping the entire track as far as I am aware (and sadly on most browsers not even a seamless loop). It's not simple to just switch music to the web audio API, because we used to do that, and on some devices (particularly Android) it can take 30sec+ just to decode the entire audio file, then it has to hold the entire track in memory (30mb+ per track), and if you free it to try to save memory, you have to do the 30sec+ decode again next time you want to play it.

One day I'd love to make a clean break and rewrite the audio plugin to use Web Audio exclusively (with just HTML5 audio for music streaming) and add a bunch of new features, but I don't think it's a good idea to do that until browsers with web audio support have something like 95%+ share. Until then adding features to the audio plugin can be pretty awkward.

Ashley

Just out of curiousity,,.how many of those audio api's support audio seeking and getting the current audio position? I've coded extended looping into my own games and I've only needed those two things.

Ashley

I hope you've seen, that my post is one year old. At that time I thought, the web audio feature would mean web audio exclusive. Also note, I've put "only" in quotation marks to tell you that I'm aware that it isn't that easy