I am using an AnalyserNode to analyse audio provided by a MediaElementAudioSourceNode. Is there a way to sample and analyse the audio that will play x seconds into the future?
No. Unless you want to just slap a delay on the audio output, so you will hear the audio at the right time synced with your "future analysis". But the media player controls, for example, will have that x-second lag.
Related
I'd like to know how YouTube plays long-form videos so quickly, with seeking, on mobile.
This is an example video: https://www.youtube.com/watch?v=eyU3bRy2x44
I can load it just fine on mobile within 5-15 seconds and I can even seek through it.
Are they using HLS? Or are they using any other streaming technology? Are they using MP4 with highly optimized MOOV Atoms placed at the front of the files?
I'd like to know because I want to serve up long-form videos on one of my websites, and they take forever to load even if served from a CDN.
Thank you in advance!
Your videos should not really take a long time to load even with 'normal' HTTP streaming if the CDN is doing its job properly.
One possible problem might be the quality/bit rate of your videos - if they are only available in high quality or high bit rate then this will definitely cause a delay in initial playback.
Many (most?) YouTube videos now will support multiple bit rates, which allows the client device select the bit rate that is most appropriate for the current network conditions. This technique is called adaptive bit rate streaming, as you likely are aware given the reference to HLS above.
MPEG DASH, as Aquary mentions, is an adaptive bit rate streaming format. It is designed to be an open standard - Apple's HLS, Microsofts's Smooth streaming and Adobe Dynamic Streaming are the other main adaptive bit rate formats.
For videos that support adaptive bit rate streaming the client will usually start up at a low or medium bit rate to ensure quick start up and then 'step up' to the highest bit rate the network will support once the video is playing. This helps fast startup. When you jump to the middle of a video the same approach is used to 'start' again from the point you have selected.
You can quite often see this if you look closely at a video when it starts up - the playback quality will improve after a short while as the video steps up through the bit rates to higher quality streams.
YouTube uses MPEG-DASH in HTML5 on the devices that are capable of that. This allows seeking through the media and start from the moment which you select.
Traditional progressive download (AKA pseudo-streaming) is not a good option in case of long videos because by default, media players try to download entire video even though you may stop the playback. Seeking is also supported in PD but your video should be prepared for that and your media server needs to be able to process seek requests properly.
I have web application(similar to Karaoke) where user can record his voice over instrumental.
After recording user plays back recording. Here I play instrumental in <audio> tag and voice using web audio api. To sync both audios on play/pause I calculate time like this
pausedAt = Date.now() - startedAt;
startedAt = Date.now() - pausedAt;
This works fine. Issue is when user uses slider on audio tag to move forward/backward. I am thinking of solution like this
Use ontimeupdate event, stop the voice and then use startAt(currentTime) where currentTime is of instrumental playing in audio tag.
Since there is no seekTo function in api, I have to stop and then start audio. Is there any better solution for this?
Second issue I face is seeking on audio tag is not smooth. If I arbitrarily clicks on progress bar sometimes it doesn't work. When I saw network tab in developer tool window I saw something like shown in image. It sends out some 600 requests and some 86 MB data downloaded whereas file size is less than 10 MB.
You really should use the Web Audio API to do this. will never give you precise synchronization, and it will rely on streaming to seek - which is going to result in extra downloading, as you saw. Just load the song via XHR and decodeAudioData, and provide your own playback controls.
I have a very fast connection and it takes about 2-3 seconds before the song actually starts playing. It's a relatively average 128kbps MP3 size (3mb-4mb). I have set preload="auto" but that didn't help much. Is there a way to just start playing the audio right away and continue to buffer it (sort of like YouTube does)?
Here is an example that I am currently working on. It's going to play an audio simultaneously on all connected clients. So if you have 2+ laptops, you can try it out. All computers must be connected before you start playing the audio. (double click on a song to start playing).
Running video and audio without complete buffering is called smooth / adaptive streaming. It can be achieved in players like silverlight and flash.
What it actually does is to create chunks of files and let the user play file chunk by chunk. Since you are downloading chunks, it will not require whole file to download.
Well, I am not giving you the full fledged answer since I haven't studied much but I am giving you the exact idea of how it works.
I had same issue but with HTML5 Video.. I overcame it with using Smooth streaming media Azure..
Here is a tutorial of the same : http://www.wrapcode.com/featured/windows-azure-media-services-mp4-to-smooth-streaming/
I will keep you updated once I find something useful :-)
If you use preload=none, then you have no buffer at the beginning but it will buffer your content "on the fly"
I have an Icecast server which streamsmy contet, and when I use pause and play, it buffers my content, even with preload=none.
Do not use preload=auto. It will take some time to start.
Check out this space shooter demo.
The HTML5 audio is perfect on Chrome 18 and Firefox 10. There is no lag in playing sounds and each sample plays perfectly. The last time I tried to play sounds using HTML5 audio and JavaScript I couldn't get a sound to play more than once.
What sorcery is Scirra doing to make this so perfect?
I'm the developer of Construct 2, so I hope I'm sufficiently qualified to answer your question :)
HTML5 audio is indeed a mess, so I've gone to considerable lengths to try and make it bulletproof in Construct 2. Here's an outline of what I've done:
Use the Web Audio API
HTML5 audio appears designed for streaming music, so a HTML5 Audio object is kind of a heavyweight object. Playing 10 sounds a second with it like Space Blaster does can easily seize up the browser. On the other hand, the Web Audio API is a high-performance audio engine with routing, effects, and lightweight sound playback. It's perfect for games. Audio buffers and audio playback are separated, so you can have one data buffer and efficiently play it many times simultaneously, whereas some browsers are so buggy if you play a HTML5 sound a few times it re-downloads it each time! Since it was actually designed for games and such, you can happily play back tonnes of sound for ages and it will still hum along nicely. It can also use HTML5 audio as a sound source, although I only use HTML5 audio for things the user has designated as music tracks (since that's where you'd prefer to have streaming - typically everything else in the Web Audio API is fully downloaded before playing).
The Web Audio API is supported in Chrome, has also made it in to iOS 6+ (although it's muted until you try to play some audio in a touch event), Firefox are working on support, and it should be coming soon to Chrome for Android. So on these platforms audio will be significantly more reliable.
More info on HTML5Rocks and the proposed spec - you'll have to use the spec as the documentation for now, there's not much else out there.
Other browsers: implement an audio recycling system
The Web Audio API isn't yet supported everywhere, notably IE, which means you still need to crowbar HTML5 audio in to something that might work for games for backwards compatibility. The way to do this is to recycle audio objects.
The player's laser in Space Blaster fires 10 times a second - and that's not including any other sound effects! As I mentioned earlier, Audio is kind of a heavyweight object, so if you're doing new Audio() 10+ times a second, lo and behold, the browser eventually dies and audio starts glitching up. However, you can drastically reduce the number of Audio objects created by recycling them.
Basically, for each sound effect, keep a cache of every Audio object you've created with that sound as a source. Then, when playing a new sound, search the cache for any sound effects which have finished playing (the ended property will be true). If you find one, rewind it back to the beginning (currentTime = 0) and play() again. Otherwise, create a new Audio() object in the cache.
Since the player's laser sound effect is short, instead of creating 600 Audio objects a minute, there will just be 3 or 4 that it keeps cycling round. Some browsers unfortunately will still download it 4 times (Safari did this last I checked!) or have high latency the first time each sound buffer is played, but eventually the browser catches up since the same buffers are always being reused. So basically sound might be a bit weird for a few moments, then it clears up. We also use the HTML5 app cache so next time you play everything loads from disk, so subsequent plays should perform well immediately.
That's basically it. It's still a little dodgy on the first play with HTML5 audio, but every time after that should be fairly solid providing the browser has a sane audio implementation. There are a number of ways to try to clone Audio objects, but I've found that rewinding existing Audios works best.
There's no SoundManager or any Flash/plugin-based fallbacks at all since we make a point of being pure HTML5.
We also support audio APIs provided by PhoneGap and appMobi for mobile, since HTML5 audio on mobile isn't even worth trying. That makes a total of four audio APIs our audio engine wraps, and yes, it does look like a frankenstein mess, but it works.
That's it. I suppose our competitors will read this, but who cares when there's SO rep to be had???!!!1111
Can I click on some time and eventhough the video is not downlaoded till that time and the video starts playing from the time I clicked?
If yes, what would be a better option for viewing streamed video- HTML5 or flash player like JW player? WHich one will have less lag?
HTML5 browsers generally don't support this, however I believe the video tag in Safari will work with the Apple HTTP Streaming format for this functionality. For something that works across all browsers, Flash Player can do it using either RTMP streaming or HTTP Streaming (either with our without OSMF to support this). Probably the easiest place to start is with OSMF, which supports both RTMP and HTTP streaming of video.
I have tested this with Firefox and Firebug and while you can "seek" to a point in the video without having to watch video preceding the point you wish to seek, this still causes the entire video file to download.
Indeed, each time you seek in Firefox 4, the entire video downloads.
This may change and improve as HTML 5 video implementations become more mature and may differ based on the actual browser being used.
More information on the test here:
http://www.stevefenton.co.uk/Content/Blog/Date/201106/Blog/HTML-5-Video-In-Real-Life/