Stream from canvas.captureStream fails to play when send over webrtc - google-chrome

I am working on a conferencing app using webrtc.
To reduce bandwidth I draw the webcam streams on canvas and resize the canvas based on the number of the active webcams in the room. So far so good and it was (is) working until chrome updated to version 104 on android devices ...
The stream from canvas plays fine on the source device but when send to other peers over webrtc the stream gets stuck in loading status ...
This only happens in some devices and not all of them ...
after some debugging and testing I figured it out that its because of the canvas size !
Since I am resizing the canvas, it does not have a standard video size and I think that where it fails ...
I tried setting the canvas size multiples of 32 and 64, some sizes works with 32 and some with 64 but not all ...
This is only happening in Android Chrome version 104+, I tried looking at the changelogs for this release to see what has changed, but could not find anything. So anyone knows what change could have cause this and how to fix it ?
I have also enabled simulcast for the streams a,d when its disabled most of the sizes works but still not all of them !

Related

HTML5 audio with HLS streaming with DVR timeshift on Samsung Internet

i am trying to make a web based streaming player (http://srv.deutschlandradio.de/themes/streamplayer/index.html) work for Samsung Internet (Samsung's Android stock browser), without success.
The stream plays, but I am unable to control it's position. It seems that Samsung's browser supports basic HLS streaming, but no seeking (nor live playing) of streams with DVR (DVR functionality allows viewers to pause, play, and rewind live streams for a certain time).
For example, audioElement.currentTime always returns 0, and changing it's value, it switches back to 0 every time.
This works fine on other browsers (as documented here: https://www.w3schools.com/tags/av_prop_currenttime.asp).
Is there any documentation on streaming support with DVR window for Samsung Internet?
I am asking Samsung directly at the same time, but as Peter O'Shaughnessy suggests I am posting it here. I still hope I am not the only one who tried to implement this on StackOverflow... Maybe there are some workarounds? MediaSourceExtensions are not supported either, as it seems.
Thanks!
The samsung browser, known as "Samsung Internet", does not seem to support MediaSource Extensions, see this thread for more details: https://github.com/SamsungInternet/support/issues/5
It behaves exactly as chrome without MediaSource extensions.

Determining current rendition for HTML5 HLS streams

I've got an HTML5 <video> element whose source is a .m3u8 (HLS stream)
I have an M3U8 with three different renditions: 640x360, 960x540, and 1280x720
On Desktops I have a Flash Player for playing the video, so the HTML5 fallback is only intended for mobile (iOS and Android) - I am doing all of my testing on an iPad and, once it's working, I will try it out on Android and hope everything works the same.
My goal is to, at any point in time, figure out what rendition the video element is playing. The rendition is subject to change as the user's bandwidth changes.
I tried using the .videoHeight property, but it always returns 480 regardless of the rendition being downloaded - which is particularly odd because 480 isn't even an option.
Does anyone know how I can figure out the rendition being downloaded?
Cleaning up some old questions that never received answers:
Unfortunately this one is just not possible. The HTMl5 video spec and HTML5 video implementations in most browsers are intended to abstract away all of the underlying magic involved in playing videos. You give it a source, it plays. Everything else is completely hidden and you have no access. No access to metadata channels, no access to audio channels, no access to bitrate and resolution information,...
At best I developed a solution to guess which resolution was playing. Every 10 seconds a 1 MB file was loaded over AJAX. I measured the speed at which this downloaded to guess at their current bandwidth. I know that QuickTime will only play a rendition if you have double the required bandwidth. So if the 960x540 rendition requires 1400 kbit/s then it won't play unless you have 2800 kbit/s bandwidth.
It's not very good (and wastes 6 MB of bandwidth per minute) but it's better than nothing.

Two H.264 mp4 videos: One plays in Chrome, one doesn't

I have two different videos, both (as far as I know) generally captured in the same manner, that I'm trying to play using an HTML5 video tag in Chrome. Both videos open and play perfectly in VLC, so I don't think there's any issue with a corrupted file, and both are mp4's with an H.264 format, using YUV color space. However, when I try to play one in Chrome (Version 21.0.1180.89) it gives me a grayed-out play button, while the other works perfectly. For reference, my OS is Ubuntu 10.10, although I've seen the same problem in newer versions of the OS. This is whether I'm loading the video into the HTML5 tag, or navigating directly to the URL where the video is being stored. I'm somewhat at a loss here, does anyone know what direction I should go to find what the significant differences are between the two videos?
Edit:
This one works: https://dl.dropbox.com/u/100841270/1_G101_20120914_0139PM_Course_101.mp4
This one does not: https://dl.dropbox.com/u/100841270/1_G101_20120914_1156AM_Course_101.mp4
Update:
It appears to have nothing to do with OS, since I've seen the same problem in both Windows and Linux. Chrome 22 beta in Ubuntu didn't seem to work either.
We had this problem and found that encoding the files in accordance with iPhone's webview's standards created files that played fine in Chrome. Chrome and iPhone webview share the same render engine, and it appears they have similar HTML5 video requirements.
Not all H.264 encoded Mp4 files are supported by Chrome and slight differences in the encoding process can produce videos that do not work. Even if the EXACT same encoding settings were used, H.264 is a variable bit-rate encoder, so different videos may exceed bitrate limits.
The encoding settings that were successful for us were:
Only use the H.264 Baseline Profile Level 3.0
Resolution below 640 x 480 and framerate up to 30 fps
B frames are not supported in the Baseline profile.
bitrate limit of 900kb.
Here is the reference we used to arrive at those settings. Likely not all of these are required for Chrome, but we stuck to these rules and found that all videos worked on both platforms. Further research could likely determine the exact setting that is/was causing Chrome to not play the video.
I am running Windows XP, and chrome doesn't like the second one either.
My best guess of the cause is that, the working one is only 6.4 MB, but the other one is about 21.7 MB. Chrome might just be refusing to directly play a video that big. Have you tried having YouTube host it, and embedding their player into your site? That may solve the problem. (If you are worried about random strangers watching the videos, why did you post them here? Why would anybody even want to watch them? Though, you can make videos private on YouTube, in case these are just two videos that demonstrate the same problem you are facing with the real videos.)
That may also be compounded by a different problem that exists with both videos, manifested when I try to use Windows' built-in player. Both videos appear distorted when I use my computer's video player, stretched like 300% horizontally.
Are there other videos you have that fail in exactly the same way? Since these are only test videos for the real thing, if this is the only video with that problem, I would not say that it really is a problem unless it recurs. The dysfunctional video may have just run into that one-in-a-million chance that it has just the right contents for it to not work.

Stage3D causes fullscreen (Stage)Video to lag/jitter

I'm having some serious problems with flash at the moment. I have a swf loaded that uses Stage3D, displaying 3D content. When I overlay a video player, on full screen size, it will cause massive lag/jitter in the video content. The video content is loaded locally so there is no streaming to take into account. This only happens on some machines but considering the project this cannot be neglected.
I have since recoded my project to remove any form of loading i was doing with a Loader and just directly put the file path into the netstream and have then attached the netstream to StageVideo. But the problem still persists and I am at my wits end. I even set Stage3D to invisible during video playback.
It's a complete horror to debug because it's simply not a problem on my machine, but out of 10 people 4 of them have reported this problem and we can't release our product like this.
Any help to resolve this would be greatly valued and appreciated.
Regards,
Kevin
Well as StageVideo is using GPU like Stage3D does. So I guess those two stages are interrupting each other processing on GPU (especially on older video cards).
So my answer is to try to stop that video while you are working on Stage3D.

How to accurately measure HTML5 Browser Framerates (FPS)?

What is the most accurate way to measure framerates, i.e. FPS, in modern HTML5 browsers? I'm specifically interested in FPS for Canvas animations.
http://weblogs.mozillazine.org/roc/archives/2010/11/measuring_fps.html will tell you that trying to measure framerate by counting how often your setTimeout runs is not accurate. The browser can run your Timeout callback multiple times between screen paints.
Turns out Mozilla has a window.mozPaintCount https://developer.mozilla.org/en/DOM/window.mozPaintCount available, which should provide an accurate FPS. However, this only works for Mozilla.
There's an open issue for Chrome for something similar: http://code.google.com/p/chromium/issues/detail?id=65348
A manual way to check for hardware accelerated FPS in Chrome is to grab the Chrome Beta channel (as of posting date) and go to about:flags and turn on FPS Counter. However, on a Mac, acceleration only turns on when using WebGL. So, no way to check FPS for Canvas on Chrome for Mac.
What are other strategies for accurately measuring HTML5 FPS?
Thanks!
Please check:
https://github.com/mrdoob/stats.js - it's the best FPS monitor I know. It also gives you some stats about mem/cpu usage (you have to run your browser with special parameter to expose that data), but may also suffer from the inaccuracy you described.
https://github.com/pcwalton/firefox-framerate-monitor
Also, in new chrome builds (probably canary stream) there should be an option for displaying FPS in about:flags.