I'm building a large flash site (AS3) that uses huge FLVs as transition videos from section to section. The FLVs are 1280x800 and are being scaled to 1680x1050 (much of which is not displayed to users with smaller screens), and are around 5-8 seconds apiece. I'm encoding the videos using On2's hi-def codec, VP6-S, and playback is pretty good with native FLV players, Perian-equipped Quicktime, and simple proof-of-concept FLV playback apps built in AS3.
The problem I'm having is that in the context of the actual site, playback isn't as smooth; the framerate isn't quite as good as it should be, and more problematically, there's occasional jerkiness and dropped frames (sometimes pausing the video for as long as a quarter of a second or so). My guess is that this is being caused by garbage collection in the Flash player, which happens nondeterministically and is therefore hard to test and control for.
I'm using a single instance of FLVPlayback to play the videos; I originally was using NetStream objects and so forth directly but switched to FLVPlayback for this reason. Has anyone experienced this sort of jerkiness with FLVPlayback (or more generally, with hi-def Flash video)? Am I right about GC being the culprit here, and if so, is there any way to prevent it during playback of these system-intensive transitions?
Jerkiness in FLV/F4V playback is hardly the result of garbage collection - the video is actually kept in memory until it is fully unloaded, so there's no garbage collection taking place at all (unless you've done something unorthodox like placing the video on a container with cacheAsBitmap set to true).
Switching to F4Vs if you don't need specific FLV features (like the link posted by daidai suggests - disclaimer, it's my blog) would help, but overall, you also have to be aware of anything that's impacting compositing of the video on screen. Because a big size video forces everything that's overlapping it to be re-rendered, any small object can have a big impact on performance.
Do you have things on top of the video? Try getting rid of them, if possible, or setting cacheAsBitmap to true (if they're not animated). This is specially good for complex vector drawings or text (bitmaps/images are much faster to draw). If they have different blending modes, you're sort of screwed - can you use the normal blending mode instead?
Do you have anything below the video? If so, get rid of them while the video is playing (just set visibility to false).
To check on unnecessary renderings, somehow pause the video, right-click, turn on 'show redrawn regions' (debug player). Do you see the red square drawing continuously? Then there's something happening there on the background, better to get rid of it.
Do you have anything happening on the background via onEnterFrame or timers/intervals? Try pausing it.
Is your video 30fps, and a CG rendering (eg, can use motion blur)? If yes, try rendering at 24fps instead, or something between - still good quality, but much less data to decode and draw.
Can you use smaller videos? Even if you're still rendering them at the same bigger size, less pixel data to decode and render helps immensely, and the impact in quality can be pretty small.
That pretty much sums it all. Getting rid of everything else on screen is usually your #1 priority though.
Have you tried encoding the videos in H.264 instead? Flash player supports them for some time now, it may end up working better. That's what most people doing HD video in Flash use (eg Youtube, Vimeo, etc).
Just as a test, try plaing a 1080p HD video on youtube and see if that pauses at all for you.
Check out http://zehfernando.com/2010/benchmarking-video-playback-performance-in-flash/
Related
I have video that will be divided into 4 videos.
First the player will stream a lower resolution of the original video, then the user can zoom into the video to see more details, I need the player to stream one of the 4 videos - that's higher in resolution- based on where the user zoomed in.
How can I make that using VideoJS or any other video player ?
After searching, this is the answer ...
For zooming into the video, you can follow this tutorial: Zooming and rotating for video in HTML5 and CSS3
For switch streaming of videos in the same player, you can make that by changing the source on html5 video tag and make some calculations to know where the user zoomed in and hence change the source video.
As there is no response yet let me analyse the problem. This is by no means meant as a full answer, but other people will probably be able to answer parts of the problem:
First the player will stream a lower resolution of the original video,
This means you will need to create/use a video stream. There are plenty of plugins you can use for videostreaming, and depends on what you want. You can consider writing it yourself using for example C#'s System.IO objects and transforming the video in bytes(And putting it back together) The resolution would be easiest reached by just having a seperate video file for this step of the proces. (a lower resolution one used for streaming only)
then the user can zoom into the video to see more details, I need the player to stream one of the 4 videos - that's higher in resolution- based on where the user zoomed in.
So you need to trigger a zoom effect. This means you would need to detect zoom. This would be possible with Javascript in a webbrowser, if you want a browser based application. When that zoom is triggered you can retrieve what position the mouse is on the screen/in the div or on some sort of overlay. Depending on this position you could show another stream.
How can I make that using VideoJS or any other video player ?
Basically these steps above is how i would start looking into this specific case. Considering your VideoJS as a suggestion i assume this is browser based. This would probably mean using Javascript libraries, maybe combined with a server side language.
Thats as far as i can go. Maybe someone can pick up specific parts of the thing i wrote and help you a step further.
Have a nice day!
On iOS, and I think Safari, we have Apples HTTP Live streaming, which can take a playlist of video files in a certain format, buffer them one by one, and play them seamlessly back to back.
Is there any way to achieve this in other browsers? I can of course detect the video finishing, and then start buffering and playing the next one, but I want to remove the buffering by downloading subsequent videos as each earlier video finishes. I do not want to hack this by stacking up video elements and switching between them. There are various serious consequences to this method.
Perhaps if there is a way to use javascript to download the files into in-memory buffers and somehow point the video element to play from these buffers?
I've seen swapping in video elements for each segment tried, and it's never smooth.
There is an HTML spec in the works that defines exactly the process you're describing, concatenating media segments into a buffer that can be fed into a media element.
http://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html
This isn't available in browsers today but is working its way into Chrome/Safari. Even when it is, it will take a good amount of work to implement this process.
Your best bet if you need it now would be to use a Flash player that can handle HLS, like OSMF, JWplayer, and soon Video.js.
In html5, you can preload videos, preload the meta information, or not preload at all. The only problem, of course, will be with Internet Explorer which will download everything or nothing at all (and not even show the controls).
This is all controllable with javascript, too. If you want to play them back-to-back automatically, though, this would need to be done with javascript or some third-party tool. Reading where the file is playing is accessible through the API.
I'm having some serious problems with flash at the moment. I have a swf loaded that uses Stage3D, displaying 3D content. When I overlay a video player, on full screen size, it will cause massive lag/jitter in the video content. The video content is loaded locally so there is no streaming to take into account. This only happens on some machines but considering the project this cannot be neglected.
I have since recoded my project to remove any form of loading i was doing with a Loader and just directly put the file path into the netstream and have then attached the netstream to StageVideo. But the problem still persists and I am at my wits end. I even set Stage3D to invisible during video playback.
It's a complete horror to debug because it's simply not a problem on my machine, but out of 10 people 4 of them have reported this problem and we can't release our product like this.
Any help to resolve this would be greatly valued and appreciated.
Regards,
Kevin
Well as StageVideo is using GPU like Stage3D does. So I guess those two stages are interrupting each other processing on GPU (especially on older video cards).
So my answer is to try to stop that video while you are working on Stage3D.
Check out this space shooter demo.
The HTML5 audio is perfect on Chrome 18 and Firefox 10. There is no lag in playing sounds and each sample plays perfectly. The last time I tried to play sounds using HTML5 audio and JavaScript I couldn't get a sound to play more than once.
What sorcery is Scirra doing to make this so perfect?
I'm the developer of Construct 2, so I hope I'm sufficiently qualified to answer your question :)
HTML5 audio is indeed a mess, so I've gone to considerable lengths to try and make it bulletproof in Construct 2. Here's an outline of what I've done:
Use the Web Audio API
HTML5 audio appears designed for streaming music, so a HTML5 Audio object is kind of a heavyweight object. Playing 10 sounds a second with it like Space Blaster does can easily seize up the browser. On the other hand, the Web Audio API is a high-performance audio engine with routing, effects, and lightweight sound playback. It's perfect for games. Audio buffers and audio playback are separated, so you can have one data buffer and efficiently play it many times simultaneously, whereas some browsers are so buggy if you play a HTML5 sound a few times it re-downloads it each time! Since it was actually designed for games and such, you can happily play back tonnes of sound for ages and it will still hum along nicely. It can also use HTML5 audio as a sound source, although I only use HTML5 audio for things the user has designated as music tracks (since that's where you'd prefer to have streaming - typically everything else in the Web Audio API is fully downloaded before playing).
The Web Audio API is supported in Chrome, has also made it in to iOS 6+ (although it's muted until you try to play some audio in a touch event), Firefox are working on support, and it should be coming soon to Chrome for Android. So on these platforms audio will be significantly more reliable.
More info on HTML5Rocks and the proposed spec - you'll have to use the spec as the documentation for now, there's not much else out there.
Other browsers: implement an audio recycling system
The Web Audio API isn't yet supported everywhere, notably IE, which means you still need to crowbar HTML5 audio in to something that might work for games for backwards compatibility. The way to do this is to recycle audio objects.
The player's laser in Space Blaster fires 10 times a second - and that's not including any other sound effects! As I mentioned earlier, Audio is kind of a heavyweight object, so if you're doing new Audio() 10+ times a second, lo and behold, the browser eventually dies and audio starts glitching up. However, you can drastically reduce the number of Audio objects created by recycling them.
Basically, for each sound effect, keep a cache of every Audio object you've created with that sound as a source. Then, when playing a new sound, search the cache for any sound effects which have finished playing (the ended property will be true). If you find one, rewind it back to the beginning (currentTime = 0) and play() again. Otherwise, create a new Audio() object in the cache.
Since the player's laser sound effect is short, instead of creating 600 Audio objects a minute, there will just be 3 or 4 that it keeps cycling round. Some browsers unfortunately will still download it 4 times (Safari did this last I checked!) or have high latency the first time each sound buffer is played, but eventually the browser catches up since the same buffers are always being reused. So basically sound might be a bit weird for a few moments, then it clears up. We also use the HTML5 app cache so next time you play everything loads from disk, so subsequent plays should perform well immediately.
That's basically it. It's still a little dodgy on the first play with HTML5 audio, but every time after that should be fairly solid providing the browser has a sane audio implementation. There are a number of ways to try to clone Audio objects, but I've found that rewinding existing Audios works best.
There's no SoundManager or any Flash/plugin-based fallbacks at all since we make a point of being pure HTML5.
We also support audio APIs provided by PhoneGap and appMobi for mobile, since HTML5 audio on mobile isn't even worth trying. That makes a total of four audio APIs our audio engine wraps, and yes, it does look like a frankenstein mess, but it works.
That's it. I suppose our competitors will read this, but who cares when there's SO rep to be had???!!!1111
I have pages where i need to play dozens of small audio file when the user clicks on things. Responsiveness is very important.
I'm thinking of using one for each file, and preloading the audio files. Is this the a reasonible approach?
Thanks.
What I experience using SoundManager2 (audio Javascript lib) is that Chrome nor Firefox have no issues loading and playing multiple (100+) sounds through their Html5 capabilities
(Firefox must play OGG though)
With IE9 it's a different story. Looks like it has a limit to load and play no more than 40 sounds. :-(
As, the game we develop requires constantly to have 50+ sounds played within 1 minute period, we have to fallback to Flash for playing sounds on IE9.. luckily SM2 does it too
I also confirm this behaviour with html5 mode using jPlayer. I'm only able to create 40 instances of jPlayer. Each can preload and play sound that it defines.
41st and following instances will fail with an error on IE9/Windows7
Error: "Media URL could not be loaded"
It's reasonable, and probably the correct solution. I recently wrote a demo application (http://www.soundscribe.com) that makes heavy use of individual (and simultaneous) audio clips in HTML5. IE9 and FF3/4 handle it well. Chrome has some issues that seem to be specifically related to simultaneous playback (which probably won't apply to your app). The biggest block I hit was in IE9, which seems to have a mysterious limit on the number of audio objects that can exist at once. The max is about 40, after which IE9 will silently fail to download the file. FF and Chrome both try to support an unlimited number.
The alternative approach of putting all the audio in a single file and changing the offset to play is a bad choice for several reasons. It's much more complicated to code, you need to keep up with additional metadata (where does the clip start, how long is it), and it's likely to work slightly different between browsers. And the worst part, there's really no way to know when your clip is fully loaded. You can only tell when the clip "can play through", which is determined by the browser based on the size of the audio file and the current download rate. This means that even after the browser reports the audio clip is ready, you may not be able to play a clip somewhere near the end.
It seems like a reasonable approach. However you need to consider a couple of things.
Each sound clip will need to be held in memory. While this is will not matter for most cases, users with a lot of tabs open, multiple programs open or old computers may get slow down on their computer. Especially if the sound files are large.
From a usability point of view, if I hear a sound every time a click a button on the site, I'll leave immediately