Html5 canvas and video - html

How can I make a canvas for the video to fit perfectly in the area
See picture

...It can't be done with native Html5 Canvas transformations.
You are trying to transform the display Canvas content (== your video) into a non-parallelogram -- which is not possible with native Canvas transformations. The Canvas can only be reshaped into parallelograms.
But...
You can skew the canvas to approximate your desired display.
The gold parallelogram in this image can be done with native Canvas transformations
Example code an a Demo:
var canvas=document.getElementById("canvas");
var ctx=canvas.getContext("2d");
var cw=canvas.width;
var ch=canvas.height;
var angle=-Math.PI*.06;
var img=new Image();
img.onload=start;
img.src="https://dl.dropboxusercontent.com/u/139992952/multple/billboard1.jpg";
function start(){
cw=canvas.width=img.width;
ch=canvas.height=img.height;
ctx.drawImage(img,0,0);
ctx.transform(1,Math.tan(angle),0,1,0,0);
ctx.strokeStyle='gold';
ctx.lineWidth=4;
ctx.strokeRect(333,135,275,150);
}
<canvas id="canvas" width=300 height=300></canvas>

Drawing process
Prepare the background image offline by removing the red area making an alpha channel out of it instead, then use it like this:
Draw the video to canvas with transformation applied matching closely the shape plus a little overlap
Draw the background (or parts of it after first draw) on top of the video. This will mask the edges of the video that doesn't fit perfectly.
Transformation
Offline process:
Plot the points of the (now) alpha area so you have the coordinates, either a triangle or a quadrilateral. Note: canvas only support affine transformations (2D/parallelograms) so triangle has to be used. For CSS you can use 3D transforms with basis in a quadrilateral shape.
Normalize the values
In action:
Scale the canvas and the normalized values to the desired size
Apply transform, draw video, draw background as overlay on top of video
And since most video is max 30 frames per second you can also throttle the drawing loop to half to leave more resources for other things (use requestAnimationFrame() with a toggle flag).
Alternative approach:
Prepare a video with everything setup as you want. It should compress well since the surrounding areas don't change (use a long key-frame interval), and in this case the background doesn't contain much details due to low depth-of-field which also helps.

Related

Building an web based image annotation tool - saving annotations to localStorage

I am building a web application for annotating images. The work flow is as follows:
Select a project - using : action = list all sub-projects
Click on a sub-project : action = fetch all the images within-sub project
Display the images as a horizontal scrollable thumbnail gallery
Onclick image thumbnail from the gallery, display the larger image for annotation.
I am using canvas to display larger image. I have used another canvas as a layer to the first one, and I am able to draw rectangles using mouse over regions of interest. I am saving it locally. However, when I move on to the next image, the rectangle also gets carried to the next image.
My question is, instead of using just one layer, do I have to dynamically create as many canvas layers as I have in the annotation dataset. I am not sure because in each sub project I have around 8000-9000 images. Though I wont be annotating on all of them, still creating as many canvases as layers doesn't really sound good for me.
The following is the code:
HTML Canvas
<div class="body"> <!-- Canvas to display images begins -->
<canvas id="iriscanvas" width=700px height=700px style="position:absolute;margin:50px 0 0 0;z-index:1"></canvas>
<canvas id="regncanvas" onclick="draw(this, event)" width=700px height=700px style="position:absolute;margin:50px 0 0 0;z-index:2"></canvas>
</div> <!-- Canvas to display images ends -->
Step 4 given above: OnClick display thumbnail
function clickedImage(clicked_id) {
var clickedImg = document.getElementById(clicked_id).src;
var clickedImg = clickedImg.replace(/^.*[\\\/]/, '');
localStorage.setItem("clickedImg", clickedImg);
var canvas = document.getElementById("iriscanvas");
var ctx = canvas.getContext("2d");
var thumbNails = document.getElementById("loaded_img_panel");
var pic = new Image();
pic.onload = function() {
ctx.drawImage(pic, 0,0)
}
thumbNails.addEventListener('click', function(event) {
pic.src = event.target.src;
});
}
Draw rectangles on second layer of canvas
window.onload=function(){
c=document.getElementById("regncanvas");
if (c) initCanvas(c);
};
function initCanvas(canvas){
// Load last canvas
loadLastCanvas(canvas);
}
function draw(canvas, event){
// Draw at random place
ctx=c.getContext("2d");
ctx.fillStyle="#ff0000";
ctx.beginPath();
ctx.fillRect (250*Math.random()+1, 220*Math.random()+1, 40, 30);
ctx.closePath();
ctx.fill();
// Save canvas
saveCanvas(canvas);
}
function saveCanvas(c){
localStorage['lastImgURI']=c.toDataURL("image/png");
}
function loadLastCanvas(c){
if (!localStorage['lastImgURI']) return;
img = new Image();
img.onload = function() {
ctx=c.getContext("2d");
ctx.drawImage(img, 0, 0, img.width, img.height);
};
img.src= localStorage['lastImgURI'];
}
Can someone guide me please?
The following is a screen grab of my application:
I have developed OCLAVI which is an image annotation tool with loads of features. It's still in beta but just after 3 weeks of release, it is gaining attraction quickly.
I have few advises for you.
HTML Canvas follow draw and forget strategy and every time redrawing the image is not a good idea. Be it 10 images or 10k, you should have one canvas for drawing the image and one canvas for drawing the shapes. Image canvas need be touched only when the image changes. Different shapes can share the same canvas.
You should integrate a data storage. Local storage is clearly not a good option to store this amount of data (especially if you have a team member who also would be annotating on the same image dataset.)
Isolate the code to a separate-separate file according to the shape. It will be very handy when you will think of adding support for Circle, Polygon, Cuboidal, Point interactions. Trust me following OOPs concepts will relive you from a lot of pain.
In terms of complexity
zooming with coordinates is easy
move with coordinates is of medium level difficulty
but you need to think with pen and paper to implement move on a zoomed image canvas (P.S. take care of the canvas flickering when the image moves
). How much the image can move in each direction also need to be calculated.
Take care of the image to canvas dimension ratio because at the end you need to have the coordinates scaled down to image level.
If your canvas size vs image size ratio is 1:1 then your job is simplified.
But this won't happen always because some images might be very small or very large to directly fit in window screen and you need to scale up and down accordingly.
The complexity increases if you like to use percentage width and height for canvas and your other team member annotating the image has a different screen size. So he drawing something will look something else on your screen.

Reverse Clipping in Canvas

I want to clip html5 canvas so that I can achieve drawing result as per following image.
I want to achieve clip path such that all drawing will be performed only in black area.
Method 1
Assuming the white areas are transparent and the black is non-transparent then you can use composite mode:
ctx.globalCompositeOperation = 'source-in';
... draw graphics on top - only solid color will be affected ...
ctx.globalCompositeOperation = 'source-over'; // reset to default mode
A demo fiddle for this here
Method 2
Another simpler approach is to simply fill the canvas with the graphics you need then use clearRect() for those areas you want transparent.
This operation is fairly fast so there shouldn't be any flashing (and in case you can trigger this operation with a single requestAnimationFrame call).
A demo fiddle with clearRect
A demo fiddle with clearRect + requestAnimationFrame
Just note that calling rAF makes the code asynchronous but the purpose of using it is that your draw operations are synchronized within a frame update so flicker will be removed (if you should for some reason get problem with that).
Method 3
Create rectangle regions surrounding the area you want to preserve by a series of call to rect(). The set that as a clipping mask by using clip().
This techniques works best if the non-clipped areas are in a certain order or else you would have to define a lot of regions.
Remember to translate canvas 0.5 pixel first and only use integer values for the rectangles.
Method 4
Parse the pixel buffer manually to fill in pixels in the areas fulfilling the requirements, for example non-transparent pixels only.
Just be aware of that this is probably the slowest approach, it's affected by CORS restrictions (in case you draw an external image onto the canvas first) and it's more tedious if you want to fill in shapes, images, gradients and so forth which in case you would probably prefer an off-screen canvas to copy from.
There are other ways using different composite modes and drawing order to achieve the same result but I leave it with this as it should cover most scenarios.
You can use layering to fill your need:
make a copy of your image with all black made transparent
draw the original image on the canvas
draw your desired shapes
draw the transparent image on top
A Demo: http://jsfiddle.net/m1erickson/dFRUf/
This function creates a temporary canvas with the color-range you specify made transparent:
function makeImageTransparentByColor(image,r1,r2,g1,g2,b1,b2){
// create a temporary canvas and draw the image on the canvas
var bk=document.createElement("canvas");
var bkCtx=bk.getContext("2d");
bk.width=image.width;
bk.height=image.height
bkCtx.drawImage(image,0,0);
// get the pixel array from the canvas
var imgData=bkCtx.getImageData(0,0,bk.width,bk.height);
var data=imgData.data;
// loop through each pixel and make every pixel transparent
// that is between r1-r2, g1-g2 and b1-b2
for(var i=0;i<data.length;i+=4){
var r=data[i];
var g=data[i+1];
var b=data[i+2]
if(r>=r1 && r<=r2 && g>=g1 && g<=g2 && b>=b1 && b<=b2){
data[i]=0;
data[i+1]=0;
data[i+2]=0;
data[i+3]=0;
}
}
// put the modified pixels back on the canvas
bkCtx.putImageData(imgData,0,0);
// return the canvas with transparent image
return(bk);
}

Canvas gradient performance

I am currently programming a little game using canvas. For the game I need some kind of fog which hides the most part of the map and only a small area around the player should be visible. Therfor I use a second canvas to overlay the one where the game takes place and fill it with a gradient (from transparent to black):
function drawFog(){
fogc.clearRect(0,0,700,600);
// Create gradient
var grd=fogc.createRadialGradient(player.getPosX(),player.getPosY(),0,player.getPosX(),player.getPosY(),100);
grd.addColorStop(0,"rgba(50,50,50,0)");
grd.addColorStop(1,"black");
// Fill with gradient
fogc.fillStyle=grd;
fogc.fillRect(0,0,700,600);
}
Unfortunatly this is causing huge perfomance problems since it will be redrawn for every frame.
I wanted to ask if there might be a better solution to achieve the same effect with a better performance.
Cache the gradient to an off-screen canvas then draw in the canvas with drawImage() instead:
Create an off-screen canvas the size of the fog
Draw in the gradient
Use off-screen canvas as an image when you need the fog.
This way the processing creating and calculating the gradient is eliminated. Drawing an image is basically a copy operation (there is a little bit more, but performance is very good).
function createFog(player) {
// Create off-screen canvas and gradient
var fogCanvas = document.createElement('canvas'),
ctx = fogCanvas.getContext('2d'),
grd = fogc.createRadialGradient(player.getPosX(),
player.getPosY(),
0,player.getPosX(),
player.getPosY(),100);
fogCanvas.width = 700;
fogCanvas.height = 700;
grd.addColorStop(0,"rgba(50,50,50,0)");
grd.addColorStop(1,"black");
// Fill with gradient
ctx.fillStyle = grd;
ctx.fillRect(0,0,700,600);
return fogCanvas;
}
Now you can simply draw in the canvas returned from the above function instead of creating the gradient every time:
var fog = createFog(player);
ctx.drawImage(fog, x, y);

Fitting Or Warping Text (or an image) Into A Custom Envelope Outline Using HTML5 Canvas

Let's say that I use some HTML5 markup:
<canvas id="e" width="400" height="200"></canvas>
<script>
var canvas = document.getElementById("e");
var context = canvas.getContext("2d");
context.fillStyle = "red";
context.font = "bold 72px JerseyLetters";
context.fillText("Bulls", 50, 100);
</script>
To make some cool text like this:
Then I decide I want these letters to fit into an envelope that looks like this:
Hoping to get something like this:
How would I go about (1) defining an envelope like the one above and then (2) putting the text in the envelope using HTML5 Canvas to get the result?
I am open to either something that directly places text in the envelope or a solution that first creates an image and then fits the image in an envelope.
Thanks!
EDIT
I added the tags "webgl" and "three.js" to this question on the advice of #markE. I will research those two packages in the mean time as well. I'm very new to .
webGL way:
Do it as a image-processing with pixel-shader.
Render text with 2d canvas, bind webGL texture with buffer and fill texture with canvas image (rendered text). Have prepared envelope that actually maps the area that envelope holds and also every pixel play role of the UV coordinate from the first image. Running that as pixel shader, you have image-to-be-squeezed and envelope (uvs) you'll output final image. That way, it's completely font and text independent. You could even probably make one image-processing step more so you could load any envelope shape and process it on spot, so it becomes font, text and envelope-shape independent.
I'm not sure how well did I explain this.
Hope this helps, though.
SVG provides these sort of text transforms. See http://tavmjong.free.fr/SVG/TEXT_PATH/TextPath.html
EDIT: This link appears to be converting the text to actual SVG. Probably not going to be helpful for you, sorry.

How to get rid of feathered edges on HTML5 Canvas strokes?

Currently, when using the HTML5 canvas element, stroked paths have slightly feathered edges.
Here is some example code I am using:
this.context.lineJoin = "round";
this.context.lineTo(x1, y1);
this.context.lineTo(x2, y2);
this.context.closePath();
this.context.stroke();
I was wondering if there was a way to create lines without slightly feathered edges.
When drawing lines, canvas automatically antialiases them, which is what you describe as feathered edges.
To avoid antialiasing, you will need to draw the lines manually using putImageData directly. (see MDN for canvas pixel manipulation)
A suitable algorithm for this is Bresenham's line algorithm which is quite easy to implement for JS/canvas.
Canvas uses subpixel accuracy.
Add 0.5 to your coorxinates. 0.0 is the border between pixels and thus line falls on two image data pixels.