I'm seeing the following behavior in a WebKit-based browser on embedded device: after there is an error drawing to the canvas (I'm not sure yet exactly about the nature of the error, but some times it looks like as if it failed to draw because the drawing was too slow...), the canvas itself becomes stale. I.e. no more drawing commands will affect the image (the error is caught in JavaScript code).
As right of now I'm not able to fix the error - how would I cause the canvas to refresh, as in recover after drawing error?
You can try to force re-initialization with something like this:
try {
//to catch the error you recevie
} catch(err) {
var tempHeight = canvas.height;
canvas.height = 0;
canvas.height = tempHeight;
}
(some would say just set canvas.height = canvas.height (or width) but this doesn't work in all browsers).
Another approach is to detach the canvas element itself, create a new element and attach that to the original parent with a attachChild.
However, these are hacks and hopefully the bug you see will be removed in next update.
Related
I'm working on a D3 application and have bumped into a strange zooming issue that occurs in Chrome but not Firefox (these are the only browsers I've tested my code in). I've boiled the problem down to the code snippet below.
Basically, in certain conditions, the zoom handler I have defined does not get called when I scroll over my canvas element (using either my mouse's scroll button or two fingers on my laptop's trackpad). However, clicking and dragging on the canvas to try to pan it does successfully invoke the zoom handler.
Details of the issue:
Zooming works as expected as long as the "click" event handler does not get called. Once it's called, scroll zooming will not work. And it has something to do with invoking ctx.getImageData in the click event handler.
If I omit the .call at the end of the canvas selection that invokes clearRect, the zooming issue occurs even if you did not trigger a "click" event
The issue occurs in other versions of D3 besides v5. I've tried v6 and v7.
The Chrome version I'm using is 92.0.4515.159
There are a couple ways I know of to resolve the issue:
The issue is resolved if the canvas element's opacity is not set to 0.
It can also be resolved by calling window.addEventListener('wheel', () => {})
From what I recall, another solution is defining the zoom behavior on a parent element of the canvas instead.
The code below creates a canvas element in the top left corner (it's not visible since its opacity is 0). Zooming on the element will print a "zooming" message to the console. Clicking on the canvas will print "click". You'll find that with the code as is, if you click on the element and then try to scroll zoom, "zooming" will not get printed. You can also run the code in JSFiddle: https://jsfiddle.net/yr3jdvhw/37/
I'd like to know what's causing this issue. As I mentioned, I could just set the canvas's opacity to a non-zero value to avoid the problem. But I'd really like to know the root cause.
<html>
<head>
<script src="https://d3js.org/d3.v5.min.js"></script>
</head>
<body>
<canvas class="my-canvas"></canvas>
<script>
const myCanvas = d3.select(".my-canvas")
.style('opacity', 0)
.on('click', function () {
console.log("click")
let ctx = this.getContext('2d')
const pixel = ctx.getImageData(0, 0, 1, 1)
})
.call(d3.zoom().on('zoom', () => {
console.log("zooming")
}))
.call(function (s) {
let ctx = s.node().getContext('2d')
ctx.clearRect(0, 0, s.node().width, s.node().height)
})
</script>
</body>
</html>
This is a Chrome bug, I just opened a new issue about it, let's hope this will get fixed soon.
From my investigations, the root issues are that they do handle wheel events at the compositor level and that when your canvas gets deaccelerated, it doesn't take the good path in the compositor anymore and thus isn't seen as a "wheel event handler region" anymore.
Calling getImageData() currently deaccelerates your canvas, it goes from the GPU to the CPU and stays there, which is why calling this method causes the issue. Similarly, until you perform any drawing operation, the canvas isn't moved to the GPU yet, and thus here too the bug reproduces.
Note that hopefully in a few versions { willReadFrequently } will be available without a flag, and that at this moment most canvases will always be accelerated.
Chrome auto-rotates any image from a file input drawn to a canvas based on it's exif data. This is great, but iOS doesn't do the same. Is there a way to prevent this behavior in so I can just transform the image myself. With a fix I wrote it works in iOS, disabling that fix works on Android ... would rather disable/enable then play the browser identifying game.
I've tried setting the style of the image to image-orientation: none; .... but that didn't do anything. Still rotated it.
Edit: I detected this by looking to see if the 'imageOrientation' on the style object was undefined or an empty string on a newly create img tag. Maybe not a perfect test, but it worked for my situations I tested. Not sure on how future proof it is.
This should be future proof:
// returns a promise that resolves to true if the browser automatically
// rotates images based on exif data and false otherwise
function browserAutoRotates () {
return new Promise((resolve, reject) => {
// load an image with exif rotation and see if the browser rotates it
const image = new Image();
image.onload = () => {
resolve(image.naturalWidth === 1);
};
image.onerror = reject;
// this jpeg is 2x1 with orientation=6 so it should rotate to 1x2
image.src = 'data:image/jpeg;base64,/9j/4QBiRXhpZgAATU0AKgAAAAgABQESAAMAAAABAAYAAAEaAAUAAAABAAAASgEbAAUAAAABAAAAUgEoAAMAAAABAAIAAAITAAMAAAABAAEAAAAAAAAAAABIAAAAAQAAAEgAAAAB/9sAQwAEAwMEAwMEBAMEBQQEBQYKBwYGBgYNCQoICg8NEBAPDQ8OERMYFBESFxIODxUcFRcZGRsbGxAUHR8dGh8YGhsa/9sAQwEEBQUGBQYMBwcMGhEPERoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoa/8IAEQgAAQACAwERAAIRAQMRAf/EABQAAQAAAAAAAAAAAAAAAAAAAAf/xAAUAQEAAAAAAAAAAAAAAAAAAAAA/9oADAMBAAIQAxAAAAF/P//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAQUCf//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQMBAT8Bf//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQIBAT8Bf//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEABj8Cf//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAT8hf//aAAwDAQACAAMAAAAQH//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQMBAT8Qf//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQIBAT8Qf//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAT8Qf//Z';
});
}
The only way to really find out for sure if the browser rotates based on exif data: Load up an image with exif ratation and see how it comes out.
This is due to an update in Chrome 81 that now has and respects the 'image-orientation' property. https://developer.mozilla.org/en-US/docs/Web/CSS/image-orientation
Chrome now defaults all images to 'from-image' meaning it will read the EXIF data to determine the rotation data of the image. Below is basically what I did to detect if the browser supports functionality like this since future versions of iOS and other browsers expect to do this also.
function browserImageRotationSupport(){
let imgTag = document.createElement('img');
return imgTag.style.imageOrientation !== undefined;
}
I was able to use this test to differentiate the browsers:
if (CSS.supports("image-orientation", "from-image")) {
...
}
const iOS = !!navigator.platform && /iPad|iPhone|iPod/.test(navigator.platform);
I use this snippet to check if it is IOS and only rotate the canvas ctx if it is IOS. I think older versions of android don't auto-rotate the image because I still have bugreports coming in from android users.
Setting the CSS on the canvas element as opposed to the img will fix this if you're drawing to a canvas that is part of the DOM.
canvas {
image-orientation: none;
}
As of writing the element has to be in the DOM because it uses the computed style. That only exists in a DOM context. You can read more in the issue on the Chromium tracker.
https://bugs.chromium.org/p/chromium/issues/detail?id=158753
Okay, I know that canvas.toDataUrl() will produce an image in png format. However, when I try to get the image from http://threejs.org/examples/#webgl_lines_sphere. All I see is a black image on chrome. To replicate the steps -
1) Open dev console and select the canvas element.
2) canvas = $0
3) context = canvas.getContext('webgl', {preserveDrawingBuffer: true})
4) img = canvas.toDataUrl()
5) document.write('<img src="'+img+'"/>')
The image is blank.
However, I tried with a different canvas at link http://threejs.org/examples/#canvas_geometry_cube. Please do the following steps to replicate.
1) Open dev console and select the canvas element.
2) canvas = $0
3) context = canvas.getContext('2d', {preserveDrawingBuffer: true})
4) img = canvas.toDataUrl()
5) document.write('<img src="'+img+'"/>')
This gave the expected result. Why is there a difference and how can this be avoided to retrieve first image too?
I was also getting a solid black image.
My code previously was:
this.renderer = new THREE.WebGLRenderer({premultipliedAlpha: false});
I have changed the parameter in the THREE.WebGLRenderer to:
this.renderer = new THREE.WebGLRenderer({preserveDrawingBuffer: true});
I am getting an image on taking a snapshot.
Hope it helps.
This is because the first example (see sources line 103) does use a THREE.WebGLRenderer creator, while the second one (see sources line 92) uses a THREE.CanvasRenderer.
Some notes :
There is no preserveDrawingBuffer contextAttribute in the context2d API, only in the WebGL one.
You can create only one context per canvas element.
You can't set the preserveDrawingBuffer flag after context's creation
With three.js you can simply call renderer.domElement.toDataURL() (you'll need to go to the iframe target to be able to call from the dev tools).
Another solution, (and better than preserveDrawingBuffer flag) is to call canvas.toDataURL() in the rendering loop itself, before the browser takes controls again.
I'm facing a CORS issue that is driving me insane. Allow me to share an example URL:
http://www.jungledragon.com/image/19905/mature_female_eastern_forktail.html/zoom
As the issue can only be reproduced once per page, here is a list of other images:
http://www.jungledragon.com/all/recent
From that overview, you can open any photo page. Next, from that photo page click the image once more to launch it fullscreen, as that is where the issue lies.
Now allow me to explain the setup, and the problem. The site itself is hosted on a Linux server within my control. The site is at www.jungledragon.com. The images, however, are stored at Amazon S3, where the image bucket has an alias of media.jungledragon.com.
The basic situation is simple:
<div id="slideshow-image-container">
<div class="slideshow-image-wrapper">
<img src="http://media.jungledragon.com/images/1755/19907_large.jpg?AWSAccessKeyId=05GMT0V3GWVNE7GGM1R2&Expires=1409788810&Signature=QH26XDrVuhyr1Qimd7IOBsnui5s%3D" id="19907" class="img-slideshow img-sec wide" data-constrained="true" data-maxheight="2056" crossorigin="anonymous">
</div>
</div>
As you can see, I'm just using the normal 'html' way of loading an image. The image URL is signed and can time out, but that shouldn't be relevant. It is my understanding that CORS does not apply to this situation, since loading images from an external domain this way has been supported for decades. The image is not loaded using javascript, after all.
Just to be sure though, the crossorigin attribute is set in HTML. Furthermore, as a way of testing, I have set a very liberal CORS policy on the image bucket:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Authorization</AllowedHeader>
</CORSRule>
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Content-Type</AllowedHeader>
<AllowedHeader>x-amz-acl</AllowedHeader>
<AllowedHeader>origin</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Now, the situation gets a bit more complicated. The fullscreen image viewer is supposed to get a background color that is the dominant/average color of the actual image on screen. That color is calculated using canvas, yet it is only calculated once. The first time it is calculated for that image, the result is communicated to the back-end using an ajax call and then stored forever. Subsequent visits to the image will not run the calculation logic again, it will simply set the background color of the body element and all is good.
Here is the logic that does the calculation:
<script>
$( document ).ready(function() {
<?php if (!$bigimage['dominantcolor']) { ?>
$('#<?= $bigimage['image_id'] ?>').load(function(){
var rgb = getAverageRGB(document.getElementById('<?= $bigimage['image_id'] ?>'));
document.body.style.backgroundColor = 'rgb('+rgb.r+','+rgb.g+','+rgb.b+')';
if (rgb!==false) {
$.get(basepath + "image/<?= $bigimage['image_id'] ?>/setcolor/" + rgb.r + "-" + rgb.g + "-" + rgb.b);
}
});
<?php } ?>
});
Yes, I'm mixing in back-end code with front-end code. The above code says that if we do not yet know the dominant color in the scene, calculate it. The load function is used because at document ready, the actual image from the normal html may not have been loaded completely. Next, if the dominant color is not known yet, and the image is loaded, we trigger the function that calculates the dominant color. Here it is:
function getAverageRGB(imgEl) {
var blockSize = 5, // only visit every 5 pixels
defaultRGB = {r:0,g:0,b:0}, // for non-supporting envs
canvas = document.createElement('canvas'),
context = canvas.getContext && canvas.getContext('2d'),
data, width, height,
i = -4,
length,
rgb = {r:0,g:0,b:0},
count = 0;
if (!context) {
return defaultRGB;
}
height = canvas.height = imgEl.naturalHeight || imgEl.offsetHeight || imgEl.height;
width = canvas.width = imgEl.naturalWidth || imgEl.offsetWidth || imgEl.width;
imgEl.crossOrigin = "anonymous";
context.drawImage(imgEl, 0, 0);
try {
data = context.getImageData(0, 0, width, height);
} catch(e) {
/* security error, img on diff domain */
return false;
}
length = data.data.length;
while ( (i += blockSize * 4) < length ) {
++count;
rgb.r += data.data[i];
rgb.g += data.data[i+1];
rgb.b += data.data[i+2];
}
// ~~ used to floor values
rgb.r = ~~(rgb.r/count);
rgb.g = ~~(rgb.g/count);
rgb.b = ~~(rgb.b/count);
return rgb;
}
The following line is CORS-relevant:
data = context.getImageData(0, 0, width, height);
Although I believe I have set up CORS correctly, I can live with this code failing in some browsers. It seems to work fine in Firefox and IE11, for example. If it fails, I would expect it to fail calculating the dominant color. However, something far worse is happening in highly specific cases: the image is not shown alltogether.
My thinking is that my 'classic' loading of the image via img src tags should have nothing to do with this script working or failing, in all cases at least the image should just load, irrespective of the canvas trick.
Here are the situations I discovered where the image does not load alltogether, which I consider a major issue:
On iOS7 on iPhone 5, the first load works fine. The calculation may fail but the image loads. Refreshing the page often breaks the image. 3rd and 4th tries then continue to succeed, and so on.
Worse, at work in Chrome 36 the image does not load alltogether. I say at work, since at home it is not an issue. Possibly a proxy makes the difference. I can refresh all I want, for images that do not have the calculation ran yet, it keeps failing.
The natural thing to do then is to debug it using Chrome's inspector. Guess what? With the inspector open, it always succeeds. The image will always load and the CORS request headers and responses look perfectly fine. This leaves me with virtually no way to debug this. I can tell though that when opening the inspector when the image does not load does give me the "CORS error" in the console, from the previous request I made. Refreshing with the inspector open will then make that go away.
From reading other questions I've learned that cache may be an influence, yet more likely the issue lies in the origin header not sent by the browser. I believe the issue may be in that direction, yet I fail to understand this:
How it influences my "normal" loading of the image using img tags
How it is only an issue behind a proxy (supposedly) in Chrome, and only when the inspector windows is closed
How it works so unreliably and inconsistently in Safari on iOS
As said, I can live with only some browsers succeeding with the canvas part, but I can't live with the image not being normally loaded in any case. That part should just work.
I realize the situation is incredibly hard for you to debug, but I hope my explanation triggers some much-needed help.
Update: I've discovered that when I remove crossorigin="anonymous" from the img tag, the image will load correctly in the specific scenarios I mentioned. However, the consequence of that move is that the color calculation will no longer work in Chrome, not at home and not at work. It continues to work in Firefox though. I'm investigating what to do next.
I managed to solve the issue myself. I still cannot fully explain cause and effect here, but this is what I did:
I removed crossorigin="anonymous" from the html's img element. This will at least make sure that the image is always loaded.
The color calculation part I solved by basically rewriting its logic:
var imgSrc = $('#<?= $bigimage['image_id'] ?>').attr('src');
var cacheBurstPrefix = imgSrc.indexOf("?") > -1 ? '&' : '?';
imgSrc += cacheBurstPrefix + 'ts=' + new Date().getTime();
var imagePreloader = new Image();
imagePreloader.crossOrigin = "Anonymous";
imagePreloader.src = imgSrc;
$(imagePreloader).imagesLoaded(function() {
var rgb = getAverageRGB(imagePreloader);
document.body.style.backgroundColor = 'rgb('+rgb.r+','+rgb.g+','+rgb.b+')';
if (rgb!==false) {
$.get(basepath + "image/<?= $bigimage['image_id'] ?>/setcolor/" + rgb.r + "-" + rgb.g + "-" + rgb.b);
}
});
Instead of reusing the img element from the html, I'm creating a new in-memory image element. Using a cache bursting technique I'm making sure it is freshly loaded. Next, I'm using imagesLoaded (a 3rd party plugin) to detect the event of this in-memory image being loaded, which is far more reliable than jQuery's load() event.
I've tested extensively and can confirm that in no case does normal image loading ever break again. It works in every browser and proxy situation. As an added bonus, the color calculation part now seems to work in far more browsers, including several mobile browsers.
Although I am still not confident on the root cause, after much frustration I'm very happy with the new situation.
if I draw to the canvas a lot in quick succession, e.g. a context.fillRect in a loop, browsers seem to wait until the loop has finished before any of the drawing is displayed (possibly via double-buffering)
Is there any way to force the browser to update the display, either explicitly or implicitly after each draw operation?
It is not really because of any double-buffering that you don't see the results, but rather because JavaScript in the web browser is single-threaded. If you similarly create a single loop in JavaScript that repeatedly does something like myDiv.style.top = parseInt(myDiv.style.top) + 1 +"px"; you will see that nothing will visibly change in the browser—even over many seconds—until your JavaScript has finished executing.
To draw changes and see the results on the screen, you need to use setInterval or setTimeout to yield control back to the browser but ask to run code at some point in the future.
For example, to draw a new random, randomly-colored rectangle on the canvas 15 times a second:
var canvas = document.getElementsByTagName('canvas')[0];
var ctx = canvas.getContext('2d');
setInterval(function(){
ctx.clearRect(0,0,canvas.width,canvas.height);
var r=Math.random()*255, g=Math.random()*255, b=Math.random()*255;
ctx.fillStyle = 'rgb('+r+','+g+','+b+')';
var w=Math.random()*canvas.width, h=Math.random()*canvas.height;
var x=Math.random()*(canvas.width-w), y=Math.random()*(canvas.height-h);
ctx.fillRect(x,y,w,h);
},1000/15);
It is better to use window.requestAnimationFrame() for better browser behaviour.