How can I play .spx file by html5 ? - html5-audio

from html5 spec, it seem support spx:
http://dev.w3.org/html5/spec-preview/the-source-element.html
Using:
But from my trying, it can't play in both Firefox 17 and Chrome, could you help ?

I have found that speex.js on GitHub (https://github.com/jpemartins/speex.js) can solve your problem. With speex.js you can demux & decode speex file (*.spx or *.ogg) to wav on fly, which is supported by both Chrome/Firefox and many other modern browsers HTML5 ready.
In your html include thoese *.js file under https://github.com/jpemartins/speex.js/tree/master/public/js/lib
<script src="bitstring.js"></script>
<script src="pcmdata.min.js"></script>
<script src="speex.js"></script>
function below will do the trick to convert spx to wav codec:
/**
* #param bufSpx ArrayBuffer (Uint8Array) holding content of speex file (*.spx or *.ogg)
*/
function decodeFile(bufSpx) {
var stream, samples, st;
var ogg, header, err;
ogg = new Ogg(bufSpx, {file: true});
ogg.demux();
stream = ogg.bitstream();
header = Speex.parseHeader(ogg.frames[0]);
console.log(header);
comment = new SpeexComment(ogg.frames[1]);
console.log(comment.data);
st = new Speex({
quality: 8,
mode: header.mode,
rate: header.rate
});
samples = st.decode(stream, ogg.segments);
var waveData = PCMData.encode({
sampleRate: header.rate,
channelCount: header.nb_channels,
bytesPerSample: 2,
data: samples
});
// array buffer holding audio data in wav codec
var bufWav = Speex.util.str2ab(waveData);
// convert to a blob object
var blob = new Blob([bufWav], {type: "audio/wav"});
// return a "blob://" url which can be used as a href anywhere
return URL.createObjectURL(blob);
}

The spec says:
The type attribute gives the type of the media resource, to help the
user agent determine if it can play this media resource before
fetching it.
The spec itself does not specify any audio or video formats to be supported and support is up to individual browsers.
... and no browser supports .spx as far as I know.

Related

All MIME types supported by MediaRecorder in Firefox and Chrome?

Where can I find a list of all MIME types that are supported by Firefox or Chrome? All examples I've seen so far using video/webm only.
I've not seen any sort of comprehensive list yet for Firefox but I have managed to find something (via a post on the MediaRecorder API from Google's web updates section) that links to this test set that seems to shed some light on things.
Essentially, it looks like the following are (at time of writing) accepted MIME types for video/audio in Chrome:
video/webm
video/webm;codecs=vp8
video/webm;codecs=vp9
video/webm;codecs=vp8.0
video/webm;codecs=vp9.0
video/webm;codecs=h264
video/webm;codecs=H264
video/webm;codecs=avc1
video/webm;codecs=vp8,opus
video/WEBM;codecs=VP8,OPUS
video/webm;codecs=vp9,opus
video/webm;codecs=vp8,vp9,opus
video/webm;codecs=h264,opus
video/webm;codecs=h264,vp9,opus
video/x-matroska;codecs=avc1
audio/webm
audio/webm;codecs=opus
(EDITED 2019-02-10: Updated to include brianchirls' link find)
I made this small function in my utils.js to get the best supported codec, with support for multiple possible naming variations (example : firefox support video/webm;codecs:vp9 but not video/webm;codecs=vp9)
You can reorder videoTypes, audioTypes and codecs arrays by priority, depending on your needs, so you'll always fall on the next supported type.
EDIT: Add support for audio, fixed mimetype duplicates
function getSupportedMimeTypes(media, types, codecs) {
const isSupported = MediaRecorder.isTypeSupported;
const supported = [];
types.forEach((type) => {
const mimeType = `${media}/${type}`;
codecs.forEach((codec) => [
`${mimeType};codecs=${codec}`,
`${mimeType};codecs=${codec.toUpperCase()}`,
// /!\ false positive /!\
// `${mimeType};codecs:${codec}`,
// `${mimeType};codecs:${codec.toUpperCase()}`
].forEach(variation => {
if(isSupported(variation))
supported.push(variation);
}));
if (isSupported(mimeType))
supported.push(mimeType);
});
return supported;
};
// Usage ------------------
const videoTypes = ["webm", "ogg", "mp4", "x-matroska"];
const audioTypes = ["webm", "ogg", "mp3", "x-matroska"];
const codecs = ["should-not-be-supported","vp9", "vp9.0", "vp8", "vp8.0", "avc1", "av1", "h265", "h.265", "h264", "h.264", "opus", "pcm", "aac", "mpeg", "mp4a"];
const supportedVideos = getSupportedMimeTypes("video", videoTypes, codecs);
const supportedAudios = getSupportedMimeTypes("audio", audioTypes, codecs);
console.log('-- Top supported Video : ', supportedVideos[0])
console.log('-- Top supported Audio : ', supportedAudios[0])
console.log('-- All supported Videos : ', supportedVideos)
console.log('-- All supported Audios : ', supportedAudios)
For Firefox, the accepted mimetypes can be found in MediaRecorder.cpp and confirmed using MediaRecorder.isTypeSupported(...)
Example:
21:31:27.189 MediaRecorder.isTypeSupported('video/webm;codecs=vp8')
21:31:27.135 true
21:31:41.598 MediaRecorder.isTypeSupported('video/webm;codecs=vp8.0')
21:31:41.544 true
21:32:10.477 MediaRecorder.isTypeSupported('video/webm;codecs=vp9')
21:32:10.431 false
21:31:50.534 MediaRecorder.isTypeSupported('audio/ogg;codecs=opus')
21:31:50.479 true
21:31:59.198 MediaRecorder.isTypeSupported('audio/webm')
21:31:59.143 false
MediaRecorder support for common audio codecs:
MediaRecorder.isTypeSupported('audio/webm;codecs=opus'); // true on chrome, true on firefox => SO OPUS IT IS!
MediaRecorder.isTypeSupported('audio/ogg;codecs=opus'); // false on chrome, true on firefox
MediaRecorder.isTypeSupported('audio/webm;codecs=vorbis'); // false on chrome, false on firefox
MediaRecorder.isTypeSupported('audio/ogg;codecs=vorbis'); // false on chrome, false on firefox
Firefox used Vorbis for audio recording in the 1st implementations but it
moved to Opus since.
So OPUS it is!
This may prove of interest:
MediaRecorder is currently experimental on Safari (August 2020.)
caniuse Opus
caniuse MediaRecorder
Based on #MillenniumFennec's answer (+ audio + removing duplicates + some other improvements):
function getAllSupportedMimeTypes(...mediaTypes) {
if (!mediaTypes.length) mediaTypes.push(...['video', 'audio'])
const FILE_EXTENSIONS = ['webm', 'ogg', 'mp4', 'x-matroska']
const CODECS = ['vp9', 'vp9.0', 'vp8', 'vp8.0', 'avc1', 'av1', 'h265', 'h.265', 'h264', 'h.264', 'opus']
return [...new Set(
FILE_EXTENSIONS.flatMap(ext =>
CODECS.flatMap(codec =>
mediaTypes.flatMap(mediaType => [
`${mediaType}/${ext};codecs:${codec}`,
`${mediaType}/${ext};codecs=${codec}`,
`${mediaType}/${ext};codecs:${codec.toUpperCase()}`,
`${mediaType}/${ext};codecs=${codec.toUpperCase()}`,
`${mediaType}/${ext}`,
]),
),
),
)].filter(variation => MediaRecorder.isTypeSupported(variation))
}
console.log(getAllSupportedMimeTypes('video', 'audio'))
Sorry, can't add comments; but thought it important to note:
Implementation of recording raw samples via ScriptProcessor or audioWorklet is flawed for a number of reasons, one here - mainly because it connects you to an output node, and clock 'correction' happens before you see the data.
So lack of audio/wav or other raw format really kills.
But just maybe.... seems 'audio/webm;codecs=pcm' is supported in chrome.
ISTYPESUPPORTED
Building on the previous answers (thanks #Fennec), I have created a jsfiddle to list all the supported types: https://jsfiddle.net/luiru72/rfhLcu26/5/. I also added a non-existent codec ("notatall").
Among the results of this script, if you call it from Firefox, you will find:
video/webm;codecs:vp9.0
video/webm;codecs=vp8
video/webm;codecs:vp8
video/webm;codecs:notatall
Note that you will not find "video/webm;codecs=vp9.0", and you will not find "video/webm;codecs=notatall" either.
This is because isTypeSupported on Firefox is able to understand the request "video/webm;codecs=vp9.0" or "video/webm;codecs=notatall" and it responds that it is not supported; but it is not able to understand the request "video/webm;codecs:vp9.0" or "video/webm;codecs:notatall", so isTypeSupported on Firefox (as of version 92.0, 2021-09-14) responds that it is supported.
MEDIARECORDER
I have created another jsfiddle to experiment with MediaRecorder: https://jsfiddle.net/luiru72/b9q4nsdv/42/
If you try to create a MediaRecorder on Firefox using the wrong syntax "video/webm;codecs:vp9,opus" or "video/webm;codecs:notatall,opus", you do not get an error, you just get a video encoded in VP8 and Opus. If you open the file using a tool like MediaInfo https://sourceforge.net/projects/mediainfo/, you realize that it is encoded in VP8,Opus.
If you specify "video/webm;codecs=vp8", you get an error because vp8 cannot encode audio. You need to specify both: "video/webm;codecs=vp8,opus", or you can just rely on defaults, specifying only the container format "video/webm". In this way, you now get the file encoded in VP8,Opus, but the actual video and audio encoder defaults could change over time, so if you want to be sure that VP8 and Opus are used, you need to specify them.
Key take away points:
you should use the syntax: video/webm;codecs=vp8, not video/webm;codecs:vp8
when creating a MediaRecorder, you should take extra-care: for example, on Firefox, video/webm;codecs=vp8 is supported, but when creating a MediaRecorder you should use "video/webm" or "video/webm;codecs=vp8,opus
if you specify an incorrect syntax, for example video/webm;codecs:vp9,opus in Firefox, you do not get an error, you just get a file that is encoded in VP8,opus. You only realize that it is in a different format from the one intended if you open it with a program like MediaInfo that is able to show you the codecs that have been used
I found a solution today which involves using
var canRecordVp9 = MediaRecorder.isTypeSupported('video/webm;codecs=vp9');
to differentiate between Chrome(and Opera) and Firefox, and then do
if (canRecordVp9)
{
mediaRecorder = new MediaRecorder(stream, {mimeType : 'video/webm;codecs=vp9'});
} else
{
mediaRecorder = new MediaRecorder(stream);
}
to construct the MediaRecorder accordingly.
Then, when grabbing the blob:
if (canRecordVp9)
{
blob = new Blob([myArrayBuffer], { "type" : "video/webm;codecs=vp9" });
} else
{
blob = new Blob([myArrayBuffer], { "type" : "video/webm" });
}
and finally, use the FileReader to get the blob as a dataUrl:
`
var reader = new FileReader();
reader.onload = function(event)
{
var blobDataUrl = event.target.result;
}
reader.readAsDataURL(blob);`
I then save the blobDataUrl as a webm file, and videos recorded in Chrome work fine in Firefox, and vice-versa.

Play audio base64 string in Internet Explorer

How to play audio with the base64-string in Internet Explorer?
Code
audio = $("#sound1")[0];
audio.src = "data:audio/wav;base64," + reader.result;
// reader.result - строка UklGRhIqBQBXQVZFZm10IBIAAAAHA...
//audio.src = "somerecord.wav";
audio.load();
Works correctly in Chrome, but not in IE.
Another way is create file and play it in IE. But how to do that without store file on disk?
If you are looking for an IE alone fix, the below code might help.
window.navigator.msSaveOrOpenBlob(recordedBlob, fileName, { type: recordingType });
where recordedBlob is the blob URL (Using the base64-string you can create one blob URL, a sample for the same is available here),
fileName is the name of the output file &
recordingType could be a JSON value like "audio/mp3"

HTML5 <audio> poor choice for LIVE streaming?

As discussed in a previous question, I have built a prototype (using MVC Web API, NAudio and NAudio.Lame) that is streaming live low quality audio after converting it to mp3. The source stream is PCM: 8K, 16-bit, mono and I'm making use of html5's audio tag.
On both Chrome and IE11 there is a 15-34 second delay (high-latency) before audio is heard from the browser which, I'm told, is unacceptable for our end users. Ideally the latency would be no more than 5 seconds. The delay occurs even when using the preload="none" attribute within my audio tag.
Looking more closely at the issue, it appears as though both browsers will not start playing audio until they have received ~32K of audio data. With that in mind, I can affect the delay by changing Lame's MP3 'bitrate' setting. However, if I reduce the delay (by sending more data to the browser for the same length of audio), I will introduce audio drop-outs later.
Examples:
If I use Lame's V0 encoding the delay is nearly 34 seconds which requires almost 0.5 MB of source audio.
If I use Lame's ABR_32 encoding, I can reduce the delay to 10-15 seconds but I will experience pauses and drop-outs throughout the listening session.
Questions:
Any ideas how I can minimize the start-up delay (latency)?
Should I continue investigating various Lame 'presets' in hopes of picking the "right" one?
Could it be that MP3 is not the best format for live streaming?
Would switching to Ogg/Vorbis (or Ogg/OPUS) help?
Do we need to abandon HTML5's audio tag and use Flash or a java applet?
Thanks.
You can not reduce the delay, since you have no control on the browser code and buffering size. HTML5 specification does not enforce any constraint, so I don't see any reason why it would improve.
You can however implement a solution with webaudio API (it's quite simple), where you handle streaming yourself.
If you can split your MP3's chunk in fixed size (so that each MP3 chunks size is known beforehand, or at least, at receive time), then you can have a live streaming in 20 lines of code. The chunk size will be your latency.
The key is to use AudioContext::decodeAudioData.
// Fix up prefixing
window.AudioContext = window.AudioContext || window.webkitAudioContext;
var context = new AudioContext();
var offset = 0;
var byteOffset = 0;
var minDecodeSize = 16384; // This is your chunk size
var request = new XMLHttpRequest();
request.onprogress = function(evt)
{
if (request.response)
{
var size = request.response.length - byteOffset;
if (size < minDecodeSize) return;
// In Chrome, XHR stream mode gives text, not ArrayBuffer.
// If in Firefox, you can get an ArrayBuffer as is
var buf;
if (request.response instanceof ArrayBuffer)
buf = request.response;
else
{
ab = new ArrayBuffer(size);
buf = new Uint8Array(ab);
for (var i = 0; i < size; i++)
buf[i] = request.response.charCodeAt(i + byteOffset) & 0xff;
}
byteOffset = request.response.length;
context.decodeAudioData(ab, function(buffer) {
playSound(buffer);
}, onError);
}
};
request.open('GET', url, true);
request.responseType = expectedType; // 'stream' in chrome, 'moz-chunked-arraybuffer' in firefox, 'ms-stream' in IE
request.overrideMimeType('text/plain; charset=x-user-defined');
request.send(null);
function playSound(buffer) {
var source = context.createBufferSource(); // creates a sound source
source.buffer = buffer; // tell the source which sound to play
source.connect(context.destination); // connect the source to the context's destination (the speakers)
source.start(offset); // play the source now
// note: on older systems, may have to use deprecated noteOn(time);
offset += buffer.duration;
}

In search for a library that knows to detect support for specific Audio Video format files

I`m building a mobile-web app, and I am implementing Video & Audio tags. Apparently not all devices know to run all file formats. Modernizr knows to return to me the Codec, but how can I know if my file has this specific codec?
I can identify the file extension or mim-type, but than I`ll need to compare the codec with the file extension, and maintain an array with this data, and this seems like a sisyphic task.
What do you guys say? Any one knows on a good library that knows provide us with this information? Or maybe my approach is wrong and theres a better way to achive that.
Please not that Firefox OS for example display a msg to the user that specific file format isn't supported by the OS, but he doesn`t provide this msg when dealing with Audio files. Which means that I need to provide this feedback to the user, and I also prefer to do it in a customised visual way to my application.
An example:
I got video.mp4. Modernizr returns Codec of H264. this two pieces of information don't relate.
I cannot place fallbacks for my video to run in all video formats available. The app is a browser of a Cloud Files Hosting (like dropbox) and if the user uploaded a file which cannot run on FirefoxOS, than he must see understand why its file don`t run, and for that I need this library, instead of managing this compression by myself.
Some references:
Mozile - Supported media formats
Dive Into HTML5 - detect video formats
Thank you.
If you have access to the mime type you could simply use the audio or video's canPlayType() method:
canPlay = (function(){
var a = document.createElement('audio'),
v = document.createElement('video');
function canPlayAudio(type){
var mime = 'audio/';
if (a && a.canPlayType) {
if (type.indexOf(mime) === -1) type = mime+type;
return !!a.canPlayType(type) || !!a.canPlayType(type.replace(/^audio\/(x-)?(.+)$/, mime+'x-$2'))
} return false
}
function canPlayVideo(type){
var mime = 'video/';
if (v && v.canPlayType) {
if (type.indexOf(mime) === -1) type = mime+type;
return !!v.canPlayType(type) || !!v.canPlayType(type.replace(/^video\/(x-)?(.+)$/, mime+'x-$2'))
} return false
}
return {
audio: canPlayAudio,
video: canPlayVideo
}
})();
Then you can perform the tests like this (note: including the "audio/" or "video/" part is optional):
canPlay.audio('audio/mpeg') // true
canPlay.audio('m4a') // true
canPlay.audio('wav') // true
canPlay.audio('flac') // false
canPlay.audio('audio/ogg') // true
canPlay.video('video/mpeg') // false
canPlay.video('video/mp4') // true
canPlay.video('m4v') // true
canPlay.video('video/webm') // true
canPlay.video('avi') // false

Play audio stream with Audio API

I am working with the HTML5 audio api to play sound. This works fine with regular mp3 files but when using a sound stream such as http://95.173.167.24:8009, it fails to play.
Here is the code i'm using:
if('webkitAudioContext' in window) {
var myAudioContext = new webkitAudioContext();
}
request = new XMLHttpRequest();
request.open('GET', 'http://95.173.167.24:8009', true);
request.responseType = 'arraybuffer';
request.addEventListener('load', bufferSound, false);
request.send();
function bufferSound(event) {
var request = event.target;
var source = myAudioContext.createBufferSource();
source.buffer = myAudioContext.createBuffer(request.response, false);
source.connect(myAudioContext.destination);
source.noteOn(0);
}
Can anyone point me in the right direction on this?
Any help is appreciated.
Thanks
The problem is likely that SHOUTcast is detecting your User-Agent string as a browser. It looks for any string with Mozilla in it, and says "Oh, that's a browser! Send them the admin panel."
You need to force the usage of the audio stream. Fortunately, this is easily done by adding a semicolon at the end of your URL:
http://95.173.167.24:8009/;
Note that the User-Agent string in your logs will be MPEG OVERRIDE.
This will work for most browsers. Some browsers may still not like the HTTP-like resopnses that come from SHOUTcast, but this will at least get you started.