I am using janus to build my webrtc SFU server. I need the chrome browser to send the video resolution from a start to a fixed value and remain unchanged during the transfer. Where should I set it?
I tried setting the degradationPreference in the js code, but it didn't work, the resolution will still change, it seems that chrome does not support this parameter.
var senderList = config.pc.getSenders();
var sender = config.pc.getSenders().find(function(s) {return s.track.kind == "video"});
if(sender) {
var parameters = sender.getParameters();
parameters.degradationPreference = "maintain-resolution";
sender.setParameters(parameters);
}
Image1 with varying resolution
Image2 with varying resolution
I looked at frameHeightSend/frameWidthSend in chrome://webrtc-internals, hoping it will keep the same value from the start, but now it grows slowly at startup and will fluctuate during subsequent transfers.
I found a message that sets the constant resolution in IOS, which is set when the screen is shared, and whether there are similar settings in chrome.
Related
I am only using the desktop Application, no mobile.
I am experimenting with letting the user set the screen resolution during run time. I give him the Display Modes available and he applies one. This part actually works. The problem occurs when i save this mode and try to set this display mode the next time they launch the game.
I am using preferences to store the mode the user selected. I am unable to access preferences before the Create method in my Game class, or in the DesktopLauncher Object, where you normally set up the config file and pass it into the application. So my DesktopLauncher looks like this.
val config = Lwjgl3ApplicationConfiguration()
config.setFullscreenMode(Lwjgl3ApplicationConfiguration.getDisplayMode())
Lwjgl3Application(MainGame(), config)
I use the current screen resolution on the creation of the application. Then in my Create method in my MainGame class i get the mode they set from preferences and i set it like so...
override fun create() {
var modes = Gdx.graphics.displayModes.toList()
val mode = Gdx.graphics.displayMode
val preference: Preferences = Gdx.app.getPreferences("screenPreference")
val screenWidth = preference.getInteger("width", mode.width)
val screenHeight = preference.getInteger("height", mode.height)
val refreshRate = preference.getInteger("refreshRate", mode.refreshRate)
modes = modes.filter { it.width == screenWidth }
modes = modes.filter { it.height == screenHeight }
modes = modes.filter { it.refreshRate == refreshRate }
if (modes.isNotEmpty()) {
Gdx.graphics.setFullscreenMode(modes[0])
}
....
}
To summarize i get the list of modes, i pull from preferences what was set last, and i filter the list according to what was in preferences. This should leave one item left in the list and i apply it. If for some reason the list is empty, then i don't set it, or there is no preference set i just apply the current mode again.
This is where the weird stuff happens. I have checked all the numbers when creating my screens and cameras, and they are all correct. I do receive the correct resolution, but the application doesn't render correctly. Below are a couple examples of what happens.
In the first image you see the bounds of the application to the screen. My application only renders in the bottom corner, and the rest is black. What happened to achieve this effect is i started the application with a smaller resolution than my native resolution, so 1280x1024, then in my create method i set the application full screen mode to 1920x1080 before building the rest of my application. I have checked my cameras and my viewports, and they all have the resolution 1920x1080, but the image is not filling the entire screen.
And a second.
This one is what happens when i reverse the settings. So i start at native resolution 1920x1080, and in my create method i set it to 1280x1024, again before creating the rest of my application. This gives me black bars on both sides of the image like id expect, but the application is HUGE, and only a portion of it fits in the window, the rest goes out of bounds, as depicted by the dotted lines.
It will remain like this the entire time, unless i change the resolution while the application is running, it will then correct itself for the rest of the applications life.
I am confounded by this effect i am getting, and am looking for an answer as to why, or how to fix it.
I found the issue that was causing the image to render incorrectly. I was setting the display mode in the create() function in my main game class. This function is not run on the rendering thread, and you do not want to use Gdx.graphics on anything other than the rendering thread, as described in the libgdx wiki https://github.com/libgdx/libgdx/wiki/Threading
There is a function where you can pass in a lambda to be run on the rendering thread.
Gdx.app.postRunnable {
Gdx.graphics.setFullscreenMode(Gdx.graphics.getDisplayMode(modes[0]))
}
After passing that into postRunnable the game renders correctly on launch.
I'm facing a CORS issue that is driving me insane. Allow me to share an example URL:
http://www.jungledragon.com/image/19905/mature_female_eastern_forktail.html/zoom
As the issue can only be reproduced once per page, here is a list of other images:
http://www.jungledragon.com/all/recent
From that overview, you can open any photo page. Next, from that photo page click the image once more to launch it fullscreen, as that is where the issue lies.
Now allow me to explain the setup, and the problem. The site itself is hosted on a Linux server within my control. The site is at www.jungledragon.com. The images, however, are stored at Amazon S3, where the image bucket has an alias of media.jungledragon.com.
The basic situation is simple:
<div id="slideshow-image-container">
<div class="slideshow-image-wrapper">
<img src="http://media.jungledragon.com/images/1755/19907_large.jpg?AWSAccessKeyId=05GMT0V3GWVNE7GGM1R2&Expires=1409788810&Signature=QH26XDrVuhyr1Qimd7IOBsnui5s%3D" id="19907" class="img-slideshow img-sec wide" data-constrained="true" data-maxheight="2056" crossorigin="anonymous">
</div>
</div>
As you can see, I'm just using the normal 'html' way of loading an image. The image URL is signed and can time out, but that shouldn't be relevant. It is my understanding that CORS does not apply to this situation, since loading images from an external domain this way has been supported for decades. The image is not loaded using javascript, after all.
Just to be sure though, the crossorigin attribute is set in HTML. Furthermore, as a way of testing, I have set a very liberal CORS policy on the image bucket:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Authorization</AllowedHeader>
</CORSRule>
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Content-Type</AllowedHeader>
<AllowedHeader>x-amz-acl</AllowedHeader>
<AllowedHeader>origin</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Now, the situation gets a bit more complicated. The fullscreen image viewer is supposed to get a background color that is the dominant/average color of the actual image on screen. That color is calculated using canvas, yet it is only calculated once. The first time it is calculated for that image, the result is communicated to the back-end using an ajax call and then stored forever. Subsequent visits to the image will not run the calculation logic again, it will simply set the background color of the body element and all is good.
Here is the logic that does the calculation:
<script>
$( document ).ready(function() {
<?php if (!$bigimage['dominantcolor']) { ?>
$('#<?= $bigimage['image_id'] ?>').load(function(){
var rgb = getAverageRGB(document.getElementById('<?= $bigimage['image_id'] ?>'));
document.body.style.backgroundColor = 'rgb('+rgb.r+','+rgb.g+','+rgb.b+')';
if (rgb!==false) {
$.get(basepath + "image/<?= $bigimage['image_id'] ?>/setcolor/" + rgb.r + "-" + rgb.g + "-" + rgb.b);
}
});
<?php } ?>
});
Yes, I'm mixing in back-end code with front-end code. The above code says that if we do not yet know the dominant color in the scene, calculate it. The load function is used because at document ready, the actual image from the normal html may not have been loaded completely. Next, if the dominant color is not known yet, and the image is loaded, we trigger the function that calculates the dominant color. Here it is:
function getAverageRGB(imgEl) {
var blockSize = 5, // only visit every 5 pixels
defaultRGB = {r:0,g:0,b:0}, // for non-supporting envs
canvas = document.createElement('canvas'),
context = canvas.getContext && canvas.getContext('2d'),
data, width, height,
i = -4,
length,
rgb = {r:0,g:0,b:0},
count = 0;
if (!context) {
return defaultRGB;
}
height = canvas.height = imgEl.naturalHeight || imgEl.offsetHeight || imgEl.height;
width = canvas.width = imgEl.naturalWidth || imgEl.offsetWidth || imgEl.width;
imgEl.crossOrigin = "anonymous";
context.drawImage(imgEl, 0, 0);
try {
data = context.getImageData(0, 0, width, height);
} catch(e) {
/* security error, img on diff domain */
return false;
}
length = data.data.length;
while ( (i += blockSize * 4) < length ) {
++count;
rgb.r += data.data[i];
rgb.g += data.data[i+1];
rgb.b += data.data[i+2];
}
// ~~ used to floor values
rgb.r = ~~(rgb.r/count);
rgb.g = ~~(rgb.g/count);
rgb.b = ~~(rgb.b/count);
return rgb;
}
The following line is CORS-relevant:
data = context.getImageData(0, 0, width, height);
Although I believe I have set up CORS correctly, I can live with this code failing in some browsers. It seems to work fine in Firefox and IE11, for example. If it fails, I would expect it to fail calculating the dominant color. However, something far worse is happening in highly specific cases: the image is not shown alltogether.
My thinking is that my 'classic' loading of the image via img src tags should have nothing to do with this script working or failing, in all cases at least the image should just load, irrespective of the canvas trick.
Here are the situations I discovered where the image does not load alltogether, which I consider a major issue:
On iOS7 on iPhone 5, the first load works fine. The calculation may fail but the image loads. Refreshing the page often breaks the image. 3rd and 4th tries then continue to succeed, and so on.
Worse, at work in Chrome 36 the image does not load alltogether. I say at work, since at home it is not an issue. Possibly a proxy makes the difference. I can refresh all I want, for images that do not have the calculation ran yet, it keeps failing.
The natural thing to do then is to debug it using Chrome's inspector. Guess what? With the inspector open, it always succeeds. The image will always load and the CORS request headers and responses look perfectly fine. This leaves me with virtually no way to debug this. I can tell though that when opening the inspector when the image does not load does give me the "CORS error" in the console, from the previous request I made. Refreshing with the inspector open will then make that go away.
From reading other questions I've learned that cache may be an influence, yet more likely the issue lies in the origin header not sent by the browser. I believe the issue may be in that direction, yet I fail to understand this:
How it influences my "normal" loading of the image using img tags
How it is only an issue behind a proxy (supposedly) in Chrome, and only when the inspector windows is closed
How it works so unreliably and inconsistently in Safari on iOS
As said, I can live with only some browsers succeeding with the canvas part, but I can't live with the image not being normally loaded in any case. That part should just work.
I realize the situation is incredibly hard for you to debug, but I hope my explanation triggers some much-needed help.
Update: I've discovered that when I remove crossorigin="anonymous" from the img tag, the image will load correctly in the specific scenarios I mentioned. However, the consequence of that move is that the color calculation will no longer work in Chrome, not at home and not at work. It continues to work in Firefox though. I'm investigating what to do next.
I managed to solve the issue myself. I still cannot fully explain cause and effect here, but this is what I did:
I removed crossorigin="anonymous" from the html's img element. This will at least make sure that the image is always loaded.
The color calculation part I solved by basically rewriting its logic:
var imgSrc = $('#<?= $bigimage['image_id'] ?>').attr('src');
var cacheBurstPrefix = imgSrc.indexOf("?") > -1 ? '&' : '?';
imgSrc += cacheBurstPrefix + 'ts=' + new Date().getTime();
var imagePreloader = new Image();
imagePreloader.crossOrigin = "Anonymous";
imagePreloader.src = imgSrc;
$(imagePreloader).imagesLoaded(function() {
var rgb = getAverageRGB(imagePreloader);
document.body.style.backgroundColor = 'rgb('+rgb.r+','+rgb.g+','+rgb.b+')';
if (rgb!==false) {
$.get(basepath + "image/<?= $bigimage['image_id'] ?>/setcolor/" + rgb.r + "-" + rgb.g + "-" + rgb.b);
}
});
Instead of reusing the img element from the html, I'm creating a new in-memory image element. Using a cache bursting technique I'm making sure it is freshly loaded. Next, I'm using imagesLoaded (a 3rd party plugin) to detect the event of this in-memory image being loaded, which is far more reliable than jQuery's load() event.
I've tested extensively and can confirm that in no case does normal image loading ever break again. It works in every browser and proxy situation. As an added bonus, the color calculation part now seems to work in far more browsers, including several mobile browsers.
Although I am still not confident on the root cause, after much frustration I'm very happy with the new situation.
I've built an AIR application with flash/as3 that has a webcam display on the stage. While building the app, and in all my tests everything looks and works just dandy, but when I publish for AIR the image gets stretched. The bounds of the image seem to stay the same, but the actual cam output is what's distorted. Has anyone come into this problem before?
I should add, this is a desktop app, which is permanently installed on one machine, so device compatibility should not be an issue.
this is the camera setup:
var cam:Camera = Camera.getCamera();
cam.setMode(280,380,20);
var video:Video = new Video(380,380);
this is where i first call the camera...
video.attachCamera(cam);
video.x = 355;
video.scaleX = -1;
video.y = -100;
addChildAt(video, 0);
the reason for the odd sizing, is that it sits behind a frame, that changes positions throughout the interactive.
Not necessarily the answer you are looking for, but you should keep this in mind:
You are asking the camera to capture at the resolution of 280 x 380, which is not a standard 4:3 aspect ratio.
When you call cam.setMode(280,380,20); the docs say that Flash will try to set the cameras resolution to your specifications, and if the camera does not support that resolution it will try to find one that matches. So you may or may not be getting this actual resolution.
setMode() has a fourth parameter, which can disable this functionality. Read the docs on that so you understand the implications :)
Then you display the video in a Video object that is 380x380. So I would expect the image to be stretched in the horizontal direction (b/c the original source is only 280).
It's not clear why this behaves differently: are you saying that running the debug version of the app works, but when you export the release build and run that it looks funky?
Finally, what is scaleX = -1 doing? I recall this as some sort of nifty trick I used in the past... but it's purpose here is escaping me :)
Yep, source code would be cool. Btw, i suggest you, as soon as you get the video streaming running, to set by hand the video.width and video.height property.
This will force the cam to display at the correct size.
I am pre-loading some images and then using them in a lightbox. The problem I have is that although the images are loading, they aren't being displayed by the browser.
This issue is specific to Chrome. It has persisted through Chrome 8 - 10, and I've been trying on and off to fix it all this time and have got nowhere.
I have read these similar questions,
Chrome not displaying images though assets are being delivered to browser
2 Minor Crossbrowser CSS Issues. Background images not displaying in Google Chrome?
JavaScript preloaded images are getting reloaded
Which all detail similar behaviour but in Chrome for Mac. Whereas this is happening in Windows.
All other browsers seem to be fine.
If you have Firefox and Chrome open, load the page in Firefox, and then in Chrome, the images appear.
Once you have manually loaded the images, using the Webkit webdev toolbar thingy, they always show up
All the links the images and such are fine and working
Clearing everything from Chrome doesn't seem to make any difference (cache, history, etc)
If anyone has any ideas it would be fantastically helpfull, as I'm literally all out of options here.
PS, Apologies if there are late replies, I'm off on holiday for a week tomorrow! :D
Update
Here is the javascript function which is preloading the images.
var preloaded = new Array();
function preload_images() {
for (var i = 0; i < arguments.length; i++){
document.write('<');
document.write('img src=\"'+arguments[i]+'\" style=\"display:none;\">');
};
};
Update
I'm still having issues with this, and I've removed the whole preloading images function. Perhaps delivering a style sheet via document.write() isn't the best way?
Chrome might not be preloading them as it's writing to the DOM with no display, so it might be intelligent enough to realise it doesn't need to be rendered. Try this instead:
var preloaded = new Array();
function preload_images(){
for (var x = 0; x < preload_images.arguments.length; x++)
{
preloaded[x] = new Image();
preloaded[x].src = preload_images.arguments[x];
}
}
The Javascript Image object has a lot of useful functions as well you might find useful:
http://www.javascriptkit.com/jsref/image.shtml
onabort()
Code is executed when user aborts the
downloading of the image.
onerror()
Code is executed when an error occurs
with the loading of the image (ie: not
found). Example(s)
onload()
Code is executed when the image
successfully and completely downloads.
And then you also have the complete property which true/false tells you if the image has fully (pre)loaded.
It turns out that Chrome takes into account the HTTP Caching and discards any preloaded images immediately after preload if the Caching is incorrectly set to expire.
In my case I am generating the images dynamically and by default the response was sent to the browser with immediate expiration.
To fix it I had to set the following below:
Response.Cache.SetExpires(DateTime.Now.AddYears(1));
Response.Cache.SetCacheability(HttpCacheability.Public);
return File(jpegStream, "image/jpeg");
Background: I'm currently working on an intranet site that makes use of the MochaUI library (working from the virtual desktop demo). I'm using Mootools 1.2.4 and MochaUI 0.9.7. The windows that are opened in my "virtual desktop" implementation load their content via iframes. Some of the loaded pages are pretty hefty in terms of css and scripting, so it's important that Window objects are adequately garbage collected when the user closes a window. This is ostensibly taken care of by the library (it does do a fair job when using Firefox).
Update
The originally posted question had become overly long from subsequent edits/updates. The title wasn't accurate anymore, so I changed that as well. Also, see my answer below for a partial solution.
Here are the essential points:
Chrome goofs up like so:
Chrome fails to free up memory allocated for MochaUI window objects when they're closed. Instead, Chrome's memory usage freezes (literally) at the level reached after the window has finished loading its iframe content, setting a lower bound on memory usage until the page is refreshed.
Memory used by the process continues to increase with subsequent window openings/closings. Eventually, some type of cap is reached, and the memory usage stops climbing as steeply/starts to oscillate instead of jump up dramatically.
This problem is most apparent when the windows in question are loading fairly hefty (memory-wise) iframe content. The window I'm using for all testing purposes loads a 580 kb page (uncached) in its iframe.
Strangely enough, the expected garbage collection does take place, when
the browser is subsequently minimized
another tab is opened in the same browser window
a Memory Timeline is being recorded in Developer Tools. (comedy option)
Does this behavior suggest any possible approaches to solving #1?
I'm not sure if this was tested in Windows, but if so keep in mind that whenever you minimize a window in windows it moves all data to the pagefile. When opening the window again it won't move the memory blocks back unless the program tries to access them, and thus any garbage stays in the pagefile but isn't actually collected.
If you'd automate it, it would not only slow the program down it would also not help with any memory issues.
see the following url for a bit more info
https://micksmix.wordpress.com/2010/01/08/why-does-task-manager-show-an-applications-memory-usage-drop-after-minimizing-it-to-the-the-taskbar/
Update
The following changes to the MochaUI closingJobs function are a big improvement over what I previously posted here. The main change is now the iframe's onunload event is manually called by changing the src property, instead of being fired when the windowEl.destroy method removes the iframe from the DOM. (got the idea from here).
If you want to use this code, just delete the existing closingJobs function and copy paste this code in its place. It should work with both 0.9.7 and 0.9.8 MochaUI, and either Mootools 1.2.4 or 1.3.
closingJobs: function(windowEl){
windowEl.setStyle('visibility', 'hidden');
var instances = MUI.Windows.instances;
var instance_id = windowEl.id
var cleanup_delay = 50;
/*
Reset canvases with width/height = 0.
This pretty reliably frees a few hundred Kb of
memory in chrome.
*/
instances[instance_id].canvasControlsEl.width = 0;
instances[instance_id].canvasControlsEl.height = 0;
instances[instance_id].canvasEl.width = 0;
instances[instance_id].canvasEl.height = 0;
if(instances[instance_id].options.loadMethod == 'iframe')
{
/*
The following line determines how long to delay the execution of
the windowEl.destroy function. The line below gives 10 milliseconds
per DOM element in the iframe's document.
You could probably do just as well with a hard-coded value.
*/
cleanup_delay = instances[instance_id].iframeEl.contentDocument.getElementsByTagName("*").length * 10;
/*
Set the Browser property in the iframe's window to Internet Explorer.
This causes Mootools to run its purge function, which iterates over
all the iframe document's DOM elements, removing events/attributes etc.
Assuming you have mootools included in the iframe content.
*/
if(instances[instance_id].iframeEl.contentDocument.defaultView.MooTools)
{
if(instances[instance_id].iframeEl.contentDocument.defaultView.MooTools.version.contains("1.3"))
instances[instance_id].iframeEl.contentDocument.defaultView.Browser.ie = true;
else
instances[instance_id].iframeEl.contentDocument.defaultView.Browser.Engine.trident = true;
}
instances[instance_id].iframeEl.src = "javascript:false";
}
MUI.cleanWindow.delay(cleanup_delay, null, windowEl);
},
cleanWindow: function(windowEl)
{
var instances = MUI.Windows.instances;
var instance_id = windowEl.id
if (Browser.ie){
windowEl.dispose();
}
else {
windowEl.destroy();
}
instances[instance_id].fireEvent('onCloseComplete');
/*
Changed - Only execute getWindowWithHighestZindex() and focusWindow()
functions if there will actually be open windows after the
current one closes.
*/
if (instances[instance_id].options.type != 'notification' && instances.__count__ > 1){
var newFocus = MUI.getWindowWithHighestZindex();
MUI.focusWindow(newFocus);
}
if (this.loadingWorkspace) this.windowUnload();
if (MUI.Dock && $(MUI.options.dock) && instances[instance_id].options.type == 'window'){
var currentButton = $(instances[instance_id].options.id + '_dockTab');
if (currentButton != null){
MUI.Dock.dockSortables.removeItems(currentButton).destroy();
currentButton = null; //Is this necessary?
}
MUI.Desktop.setDesktopSize();
}
//Changed - moved this to the end of the function.
delete instances[instance_id];
}