Is there any way to take the system's sound as an input for the Web Audio API?
For example take a user's spotify music that is playing as an input and analyze it to do stuff with it?
There is a specification which almost does what you want. It's the counterpart of the Media Capture and Streams spec (aka getUserMedia()).
navigator.mediaDevices.getDisplayMedia();
The spec is called Screen Capture and as the name implies it is meant for capturing video and not audio. Capturing audio is optional and not yet supported by any browser if I recall correctly.
There is currently an open issue discussing audio-only support. Maybe it's a good idea to share your use case there.
Related
I'm trying to figure out if web based audio streaming sites use the Web Audio API for playback or if they rely on the audio element or something else.
Since the user of an audio streaming service typically doesn't need more functionality than starting and stopping the audio, then I guess that the audio element is enough. If a VU-meter is required then I would guess the Web Audio API would be used since it has an built in analyser node. But since IE doesn't support the API then I suppose you'd rather use the audio element and reach the IE users than using fancy extras such as an VU-meter.
I've been looking at the source code for Spotifys web player, Grooveshark, BBC radio and the Polish public radio but I find neither audio elements or use of the Web Audio API. I did find that the Swedish public radio (sr.se) makes use of the audio element though.
I'm not asking for anyone to go through the JavaScript source code for me, but rather if someone who is familiar with the subject could point me in the right direction.
I don't know of any internet radio services playing back their streams with the Web Audio API currently, but I wouldn't be surprised to find one. I've been working on one myself using Audiocog's excellent Aurora.js library, which enables codecs in-browser that wouldn't normally be available, by decoding the audio with JavaScript. However, for compatibility reasons as you have pointed out, this would be considered a bit experimental today.
Most internet radio stations use progressive HTTP streaming (SHOUTcast/Icecast style) which can be played back within an <audio> element or Flash. This works well but can be hard to get right, especially if you use SHOUTcast servers as they are not quite 100% compatible with HTTP, hurting browser support in some versions of Firefox and a lot of mobile browsers. I ended up writing my own server called AudioPump Server to get better browser and mobile browser support with HTTP progressive.
Depending on your Flash code and ActionScript version available, you might also have to deal with memory leaks in creative ways, since by default Flash will keep all of your stream data in memory indefinitely as it was never built to stream over HTTP. Many use RTMP with Flash (with Wowza or similar on the server), which Flash was built to stream with to get around this problem.
iOS supports HLS which is basically a collection of static files served by an HTTP server. The encoder writes a chunk of the stream to each file as the encoding occurs, and the client just downloads them and plays them back seamlessly. The benefit here is that the client can choose a bitrate to stream and, raising quality up and down as network conditions change. This also means that you can completely switch networks (say from WiFi to 3G) and still maintain the stream since chunks are downloaded independently and statelessly. Android "supports" HLS, but it is buggy. Safari is the only browser currently supporting HLS.
Compatibility detection is not something you need to solve yourself. There are many players, such as jPlayer and JW Player which wrangle HTML5 audio support detection, codec support detection, and provide a common API between playback for HTML5 audio and Flash. They also provide an optional UI if you want to get up and running quickly.
Finally, most stations do offer a link to allow you to play the stream in your own media player. This is done by linking to a playlist file (usually M3U or PLS) which is downloaded and often immediately opened (as configured by the user and their browser). The player software loads this playlist and then connects directly to the streaming server to begin playback. On Android, you simply link to the stream URL. It will detect the Content-Type response header, disconnect, and open its configured media player for playback. These days you have to hunt to find these direct links, but they are out there.
If you ever want to know what a station is using without digging around in their compiled and minified source code, simply use a tool like Fiddler or Wireshark and watch the traffic. You will find that it is very straightforward under the hood.
We use Web Audio for streaming via Aurora.js using a protocol very similar to HTTP Live Streaming. We did this because we wanted the same streaming backend to serve iPhone, Android and the web.
It was all a very long and painful process that took over 6 months of effort, but now that its all finished, its all good.
Have a look at http://radioflote.com and feel free to shoot questions or clarifications regarding anything. Go ahead and disassemble the code if you want to. Not a problem.
How does one go about streaming video in HTML5 ? I can go with using a single browser of the latest version if I have to.I need to be able to start playing from any location of the movie even if the entire video has not been loaded by the browser.
WebRTC ?
I've already seen this question and no one has answered.
Does not allow the viewer to skip to the middle of a video in any
browser. They must watch the video straight through start to finish,
which is not ideal.
This is the main point for streaming.
Currently if you want to use pure HTML5 and work cross-browser you are limited to progressive streaming with the <video> element.
While that still does allow the user to skip ahead via the scrubber or programatically by setting the .currentTime there will still be some buffering while the browser re-loads enough content to be comfortable playing smoothly.
Solutions like Smooth Streaming, HLS do not work across browsers today so you would require a Flash or Silverlight plugin, though with MPEG-DASH being recognized by the W3C there is some hope for the future as samples like this demonstrate http://dash-mse-test.appspot.com/release-notes.html
For today however if you want to stick with an HTML5 solution and you have source in a format the works with the browser then you should be fine
Chrome has implemented the Media Source API in the mean time. Hoping the rest will follow.
http://www.w3.org/TR/media-source/
Abstract
This specification extends HTMLMediaElement to allow JavaScript to generate media streams for playback. Allowing JavaScript to generate streams facilitates a variety of use cases like adaptive streaming and time shifting live streams.
I would like to clarify certain things with what I found and raise certain questions with things that I dont know,
Capturing cam/mic through browser could be done through getusermedia();
Is there anything for i devices? because getusermedia() doesn't seem to work in i devices
How could I trap actual audio from web browser application (eg. if I play an audio file and forward it 2mins, I would like to capture the actual audio stream from the html5 player so that I hold the actual audio data)
You need to use Flash, if you are not going to support mobile devices. One best solution is to use wami-recorder.
From their website:
The Problem
As of this writing, most browsers still do not support WebRTC's getUserMedia(), which promises to give web developers microphone access via Javascript. This project achieves the next best thing for browsers that support Flash. Using the WAMI recorder, you can collect audio on your server without installing any proprietary media server software.
The Solution
The WAMI recorder uses a light-weight Flash app to ship audio from client to server via a standard HTTP POST. Apart from the security settings to allow microphone access, the entire interface can be constructed in HTML and Javascript.
Hope this helps.
When you play audio using the audio element in Chrome you get annoying clicks and cracks. At least under my 64bit Linux installation, even after I formatted and installed a new Fedora version. (Firefox and Opera are fine, even IE9 in a VirtualBox Windows 7.)
But demos using the Web Audio API instead of the audio element have perfect sound. So I was wondering if I could use the Web Audio API like the audio element? But there are some things you seem not to be able to do with this API. Or am I missing something? The things I couldn't find where:
starting to play a file before it is completely loaded
getting buffer progress updates (depends on the previous point)
getting play progress updates
seeking
Is there a way to do this with the Web Audio API?
This is where I would use it: http://tinyurl.com/magnatune-player
I think you should still use <audio> for streaming at the very least. You can treat it as a MediaElementAudioSourceNode in web audio if you'd like:
var mediaSourceNode = context.createMediaElementSource(audioElement);
AFAIK, there is no way to stream web audio directly. In fact, the web audio api suggests that you don't:
4.9. The AudioBuffer Interface
This interface represents a memory-resident audio asset (for one-shot sounds and other short audio clips). Its format is non-interleaved IEEE 32-bit linear PCM with a nominal range of -1 -> +1. It can contain one or more channels. Typically, it would be expected that the length of the PCM data would be fairly short (usually somewhat less than a minute). For longer sounds, such as music soundtracks, streaming should be used with the audio element and MediaElementAudioSourceNode.
Unless you used a MediaElementAudioSourceNode (which I would assume suffer from the same issues you're having since it's just using an <audio> tag) AFAIK the answers to your questions are:
starting to play a file before it is completely loaded: No.
getting buffer progress updates (depends on the previous point): Possibly (You could check for progress events on the XHR)
getting play progress updates: No.
seeking: No.
In the meantime Chrome fixed the audio playback issues. So I don't need any workarounds anymore.
I'm trying to build a html5 app, part of it requires the recording of audios whose length should be up to a minute and then encode it into 64 based, so I did quite a bit searching and didn't find a good answer.
Are there any ways to record an audio with Html5, Javascript or maybe local APIs on a html5 app, especially on an IOS device?
Html5 doesn't yet have a widely accepted method of recording audio. If you are targeting non-iOS, you can use flash, or any number of javascript wrappers that call out to flash( eg. http://www.sajithmr.me/jrecorder-jquery I think soundmanager2 will also work, but I'm not sure). For iOS, you still need to write an app. :(
Short answer is no for now. Because audio not yet implemented to HTML5 browsers yet.
Method is working but you can't capture any audio.
Well here is a detailed answer for you: Recording html5 audio
This may be too late for a response, but having said that, both Chrome and Firefox now fully support getUserMedia and you can use it along with the AudioContext interface if needed, to capture audio directly from the browser.
The following gitHub project records audio and saves it in MP3 format directly in the browser using just HTML5 and JS.
The audio recording is saved in base64 and can also be directly listened to from the browser after the recording is made.
The project can be found here:
https://github.com/nusofthq/Recordmp3js
and is an extension of RecorderJS that also uses libmp3lame.js.
If you wish you can read more details about the actual implementation:
http://nusofthq.com/blog/recording-mp3-using-only-html5-and-javascript-recordmp3-js/