The default sample rate of web audio context is 48000. However, most people can only hear 20-20000Hz and almost no one can hear over 30000Hz, why it is set to 48000?
var a = new AudioContext();
var samplerate = a.sampleRate;
alert(samplerate); // 48000
The fact that 20kHz is approximately the highest frequency humans can hear is exactly the reason why values above 40kHz are so commonly used by audio hardware.
To represent a signal digitally you have to sample it with at least twice the frequency. https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem
By default an AudioContext uses the sampleRate of the audio hardware which is commonly 44.1kHz (thanks to the CD format) or 48kHz these days.
Related
I created a batch of sounds assembled with this tool:
AudioSprite
https://github.com/tonistiigi/audiosprite
The output is generally used for JS libraries, such as Howler, Zynga Jukebox, or SoundJS - but I wanted to see if it's possible to implement in AS3.
I started creating a Sound player that can load, parse and play the sounds based on the JSON and MP3 file this tool generates.
So far so good! ... except for loops.
Now, the big question is - is there a way to play a Sound-loop seamlessly given that all music & sounds coexist in the same MP3 file, and it has a start & end range to play and stop it?
Example of how the sounds are placed in the file:
mygame_sounds.mp3 = [BUZZ + LASER + BOING ... + TRACKLOOP]
I'm looking for a solution that does not involve using the SAMPLE_DATA Event (given it eats up a lot of CPU usage). If there's no way around it, please explain why.
So far I've had mild success using flash.utils.Timer objects triggered after a given AudioSprite's duration, but it's not consistent.
To stop / dispose of a non-looping sound, I rely on a Master Timer (running at very short intervals) and that seems to "cut" the sample appropriately. But I already tried using this Master Timer to play a looped-sound over and over - same latency issues.
Is there any method to predict / measure how much latency is to be expected by the time the sound completes one pass?
In SoundJS we could not find a way to allow smooth looping of audio sprites in AS3 and went with a timer. We found Web Audio was the only api that allowed smooth looping, and therefore recommended staying away from audiosprites for sounds that needed to loop smoothly if any other plugin might be used.
Hope that helps.
The reason of why you can't get smooth loops of a track retrieved from a larger audio file is that you cannot check sound position faster than once per SWF frame, which length depends on stage.frameRate and total processing time of your application and is generally varied. So, if your looping sounds lasts say 5.123 seconds (I don't care how many samples, just that its length does not make a full number of frames regardless of stage.frameRate), your sound will attempt to play for either 5.125 seconds (205 frames at 40 fps, IMO best bet for this particular sound), 5.133 seconds (154 frames at 30 fps) or some weird number of frames if the SWF would experience lag. The excess milliseconds cannot be totally controlled by any means due to AS3/Flash engine optimization. So, consider not using audio sprites and shift into audio packs (several audio files in an SWF, or one sound in an MP3).
Although I'm still working on the perfect solution, this is the best I could come up with:
Load the JSON file / ByteArray.
Parse the JSON file to obtain each sprites' ID, start and end times.
Load the MP3 file / ByteArray (requires loadCompressedDataFromByteArray()) into a master Sound object.
Once loaded, check if any sprites are marked as "loops".
Create separate Sound objects for the above loops, and extract the portion from the master Sound via loadPCMFromByteArray() with some "magic-numbers" (details below).
To play a one-shot sound, call the master Sound's play(sprite.start * 1000) (depending on the format, usually the JSON's start values are in seconds, needs to be in milliseconds).
To play a seamless-loop sound, call the individual Sound object's (created in step #5) play(0, 9999) method.
I won't go too deep in details on how to stop the sounds (SoundChannel.stop(), bam!), but I'll explain the "magic-numbers" mentioned above. See this snippet:
var goldenOffset:UInt = (64 << 5);
var goldenDuration:UInt = (64 << 2);
var sampleRate:UInt = 44100;
for (id in loops) {
var sprite:AudioSpriteItem = _mapSprites.get(id);
var loop:Sound = _mapLoops.get(id);
var sampleBytes = new ByteArray();
var samplesTotal:UInt = cast(sprite.duration * sampleRate + goldenDuration);
var samplesStart:UInt = cast(sprite.start * sampleRate + goldenOffset);
_sound.extract(sampleBytes, samplesTotal, samplesStart);
sampleBytes.position = 0;
loop.loadPCMFromByteArray(sampleBytes, samplesTotal, "float", true);
}
Quite honestly, these magic goldenOffset and goldenDuration values were just found via Trial-and-Error. I could get close to a seamless loop without them by just calculating the start and duration with the sampleRate (assuming 44100 by default), but each endings had a bit of a hiccup to it.
After several adjustments, those couple "64 left bit-shifted" values made the loops sound smoother.
I posted the Haxe project on github (compiled SWC also available in /bin folder) if you wish to try it / read through the code.
FLAudioSprite
Github page: https://github.com/bigp/FLAudioSprite
SWF Demo (Download): bit.ly/FLAudioSpriteSWFDemo
I have an AS3 music player built into an app that I'm putting together. It works perfectly with almost every file I've used, but there is one file that it stops early on. The file is roughly 56 seconds long, the player stops at about 44 seconds. I'm using trace to show the length, and for every other song the length is correct. In this case, trace shows roughly 44 seconds instead of 56. Here's the code I use to load the file:
length = 0;
request = new URLRequest(fileAddress);
track = new Sound();
track.load(request);
track.addEventListener(Event.COMPLETE, TrackLoaded);
And here's the TrackLoaded function:
private function TrackLoaded(e:Event):void{
length = track.length;
if (playWhenLoaded == true){
trackChannel = track.play(0);
trackChannel.addEventListener(Event.SOUND_COMPLETE, TrackFinishedPlaying);
playWhenLoaded = false;
}
Works perfectly with every other file. What am I missing?
Are you willing to host this 56 sec MP3 somewhere for download & analysis? Or if you can yourself check the header info via a Hexeditor. I suspect either of two things:
1) Header has incorrect time length embedded and Flash takes that as final duration and stops there. After all why read anymore remaining bytes? they could be just metadata not audio samples Besides what encoder would lie about true duration? So its accepted as final duration even if your ears know that its incorrect.
2) MP3 samplerate /bitrate issue: Consider what sample rate is this one problem MP3? Check the sample rate against of a working MP3. Also are these various found sounds or you made each yourself? I ask to confirm you put the same settings for each file yet this one does not work?
In anycase I think you could solve this particular MP3 by re-encoding it. Maybe save as WAV or AIFF first then take that new uncompressed audio and convert back to a MP3 with samplerate of 44100 khz + Stereo sound + Constant Bitrate (avoid Variable B.Rate like hell if you dont want issues)
Checking and fixing either of the above should get you are correctly parsed MP3. Hope it helps
I have a desktop application which streams raw PCM data to my browser over a websocket connection. The stream looks like this ...\\x00\\x00\\x02\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\xff\\xff\\xff\\xff\\....
The question is simple: can I play such a stream in HTML with the Web Audio API / WebRTC / ...?
Any suggestions are very welcome!
code edit
This code plays noise, randomly generated:
function myPCMSource() {
return Math.random() * 2 - 3;
}
var audioContext;
try {
window.AudioContext = window.AudioContext || window.webkitAudioContext;
audioContext = new AudioContext();
} catch(e) {
alert('Web Audio API is not supported in this browser');
}
var bufferSize = 4096;
var myPCMProcessingNode = audioContext.createScriptProcessor(bufferSize, 1, 1);
myPCMProcessingNode.onaudioprocess = function(e) {
var output = e.outputBuffer.getChannelData(0);
for (var i = 0; i < bufferSize; i++) {
output[i] = myPCMSource();
}
}
So changing the myPCMSource() to the websocket stream input, should make it work somehow. But it doesn't. I don't get any errors, but the API is not playing any sound nor noise.
Use a ScriptProcessorNode, but be aware that if there is too much load on the main thread (the thread that runs your javascript, draws the screen, etc.), it will glitch.
Also, your PCM stream is probably in int16, and the Web Audio API works in terms of float32. Convert it like so:
output_float[i] = (input_int16[i] / 32767);
that is, go from a [0; 65535] range to a [-1.0; 1.0] range.
EDIT
I was using output_float[i] = (input_int16[i] / 32767 - 1);, this article shows that you should use output_float[i] = (input_int16[i] / 32767);. Now it's working fine!
Just for the record, the ScriptProcessorNode is deprecated. See the MDN article for details. The feature was replaced by AudioWorklets and the AudioWorkletNode interface.
In short, a ScriptProcessorNode runs outside of the browser's internal audio thread, which creates at least on frame (128 samples) of latency. Worse, the ScriptProcessorNode often fails to respond quickly enough, if the thread is busy, so will just randomly drop the ball every so often.
Worklets are basically task-specific workers that run in one of the browsers internal threads (paint, layout, audio etc). Audio worklets run in the audio thread, and implement the guts of custom audio nodes, which are then exposed through the WebAudio API as normal.
Note: You are also able to run WebAssembly inside worklets to handle the processing.
The solution provided above is still useful, as the basic idea holds, but it would ideally use an audio worklet.
I am looking to take the audio input from the browser and stream it to multiple listeners. The intended use is for music, so the quality must mp3 standard or thereabouts.
I have attempted two ways, both yielding unsuccessful results:
WebRTC
Streaming audio directly between browsers works fine, but the audio quality seems to be non-customisable though what I have seen. (I have seen that it is using the Opus audio codec, but seems to not expose any controls).
Does anyone have any insight into how to increase the audio quality in WebRTC streams?
Websockets
The issue is the transportation from the browser to the server. The PCM audio data I can acquiring via the method below has proven too large to repeatedly stream to the server via websockets. The stream works perfectly in high speed internet environments, but on slower wifi it is un-usable.
var context = new webkitAudioContext()
navigator.webkitGetUserMedia({audio:true}, gotStream)
function gotStream (stream)
{
var source = context.createMediaStreamSource(stream)
var proc = context.createScriptProcessor(2048, 2, 2)
source.connect(proc)
proc.connect(context.destination)
proc.onaudioprocess = function(event)
{
var audio_data = event.inputBuffer.getChannelData(0)|| new Float32Array(2048)
console.log(audio_data)
// send audio_data to server
}
}
So the main question is, is there any way to compress the PCM data in order to make it easier to stream to the server? Or perhaps there is an easier way to go about this?
There are lots of ways to compress PCM data, sure, but realistically, your best bet is to get WebRTC to work properly. WebRTC is designed to do this - adaptively stream media - although you don't define what you mean by "multiple" listeners (there's a huge difference between 3 listeners and 300,000 simultaneous listeners).
There are several possible ways of resampling and/or compressing your data, none of them native though. I resampled the data to 8Khz Mono (your mileage may vary) with the xaudio.js lib from the speex.js environment. You could also compress the stream using speex, though that is used usually for audio only. In your case, I would probably send the stream to a server, compress it there and stream it to your audience. I really don't believe a simple browser to be good enough to serve data to a huge audience.
WebRTC seems to default to one mono channel around 42 kb/s, it seems to be primarily designed for voice.
You can disable the audio processing features using constraints to get a more consistent input from the browser using:
navigator.mediaDevices.getUserMedia({
audio: {
autoGainControl: false,
channelCount: 2,
echoCancellation: false,
latency: 0,
noiseSuppression: false,
sampleRate: 48000,
sampleSize: 16,
volume: 1.0
}
});
Then you also should set stereo and maxaveragebitrate params on the SDP:
let answer = await peer.conn.createAnswer(offerOptions);
answer.sdp = answer.sdp.replace('useinbandfec=1', 'useinbandfec=1; stereo=1; maxaveragebitrate=510000');
await peer.conn.setLocalDescription(answer);
This should output a string which looks like this:
a=fmtp:111 minptime=10;useinbandfec=1; stereo=1; maxaveragebitrate=510000
This could increase the bitrate up to 520kb/s for stereo, which is 260kps per channel. Actual bitrate depends on the speed of your network and strength of your signal tho.
As the title says, if I have an audio node that emits sound and I connect it to two separate GainNodes, which in turn are connected to the Audio Context destination, the sound plays at double speed / double pitch (as if half samples are sent to one gain node and half samples to the other, and the time is halved as well).
I have created an handy jsfiddle here, just drag your sound files in the black rectangle canvas and listen.
// audioContext: Web Audio context
// decoded: decoded audioBuffer
// gainNode1, gainNode2: gain nodes
var bSrc = audioContext.createBufferSource();
bSrc.connect (gainNode1);
bSrc.connect (gainNode2);
gainNode1.connect (audioContext.destination);
gainNode2.connect (audioContext.destination);
bSrc.buffer = decoded;
bSrc.loop = false;
// You'll hear two double-speed buffers playing at unison
bSrc.start(0);
Is that by design? What I would like is to exactly "duplicate" the sound (that will be sent to two different routes, the fiddle is just a proof-of-concept for a bigger project).
Edit:
I tested this on Chrome Version 24.0.1312.56 / Ubuntu 12.10 and the behaviour is present.
The behaviour is also present on Chrome Version 24.0.1312.68 / Ubuntu 12.10
On Chrome Version 24.0.1312.57 / Mac OSX, the Audio API works well and this behaviour is not present.
Could it be a Linux-only issue?
Sounds like a Linux implementation issue. It works for me in Chrome on OS X.