`GetThumbnailAsync` transparency issues - windows-runtime

I know there are a number of threads concerning "zomg png transparency gone when I create a thumbnail on UWP" but this is a bit more nuanced than that.
I realize the output of GetThumbnailAsync is a bitmap, generally without transparency support. I know that by default the background color used for transparent images converted to a bitmap under UWP is the background color of the application (which is a different problem altogether), but what I discovered is that when calling GetThumbnailAsync against icons with existing 32-bit transparency, this transparency is preserved.
However when GetThumbnailAsync is called against a PNG with transparency, the background is coerced to black. This rather sucks when the only non-transparent color in the PNG is black, because you just end up with an all-black blob.
Heres' a screenshot with 2 Image elements rendered, the source for each being a call to GetThumbnailAsync(). The first icon was called against an exe with a 32-bit icon w/ transparency, the second was against a 32-bit PNG w/ transparency:
This demonstrates that transparency is actually supported in the resulting bitmap images created by GetThumbnailAsync() and that the background color can actually be "transparent" and not "your application color." Yet for some reason the PNG thumbnails always lose their transparency. Is there a fix for this?

I think this boils down to a "bug" in Windows 10, it would seem that transparency data is lost in general. Here's what happens when I drag the source file in Windows Explorer, which is a PNG w/ transparency and a black-only foreground:
The solution I just coded up (so pardon any oversights!) is to bypass GetThumbnailAsync() for transparent PNGs and generate my own PNG thumbnail that preserves the transparency:
public static class ThumbnailExtensions
{
public static async Task<IRandomAccessStream> GetTransparentThumbnailAsync(this StorageFile file, ThumbnailMode mode, uint requestedSize, ThumbnailOptions options = ThumbnailOptions.None)
{
if (file.FileType.ToLower() == ".png")
{
//try to create a scaled-down version of the PNG with transparency
using (var fstream = await file.OpenAsync(FileAccessMode.Read))
{
var thumbnail = new InMemoryRandomAccessStream();
var decoder = await BitmapDecoder.CreateAsync(BitmapDecoder.PngDecoderId, fstream);
var encoder = await BitmapEncoder.CreateForTranscodingAsync(thumbnail, decoder);
encoder.BitmapTransform.ScaledHeight = requestedSize;
encoder.BitmapTransform.ScaledWidth = requestedSize;
encoder.BitmapTransform.InterpolationMode = BitmapInterpolationMode.Fant;
await encoder.FlushAsync();
await thumbnail.FlushAsync();
return thumbnail;
}
}
return await file.GetThumbnailAsync(mode, requestedSize, options);
}
}
The result isn't the prettiest (even InterpolationMode.Fant sucks; but there are Lanczos implementations for C# floating around somewhere), but it is transparent:
The code above would need to be extended to support preserving the aspect ratio and padding or cropping images that are not 1:1 to fit.

Related

Drawing photo from file to canvas rotates in Chrome based on EXIF

Chrome auto-rotates any image from a file input drawn to a canvas based on it's exif data. This is great, but iOS doesn't do the same. Is there a way to prevent this behavior in so I can just transform the image myself. With a fix I wrote it works in iOS, disabling that fix works on Android ... would rather disable/enable then play the browser identifying game.
I've tried setting the style of the image to image-orientation: none; .... but that didn't do anything. Still rotated it.
Edit: I detected this by looking to see if the 'imageOrientation' on the style object was undefined or an empty string on a newly create img tag. Maybe not a perfect test, but it worked for my situations I tested. Not sure on how future proof it is.
This should be future proof:
// returns a promise that resolves to true if the browser automatically
// rotates images based on exif data and false otherwise
function browserAutoRotates () {
return new Promise((resolve, reject) => {
// load an image with exif rotation and see if the browser rotates it
const image = new Image();
image.onload = () => {
resolve(image.naturalWidth === 1);
};
image.onerror = reject;
// this jpeg is 2x1 with orientation=6 so it should rotate to 1x2
image.src = 'data:image/jpeg;base64,/9j/4QBiRXhpZgAATU0AKgAAAAgABQESAAMAAAABAAYAAAEaAAUAAAABAAAASgEbAAUAAAABAAAAUgEoAAMAAAABAAIAAAITAAMAAAABAAEAAAAAAAAAAABIAAAAAQAAAEgAAAAB/9sAQwAEAwMEAwMEBAMEBQQEBQYKBwYGBgYNCQoICg8NEBAPDQ8OERMYFBESFxIODxUcFRcZGRsbGxAUHR8dGh8YGhsa/9sAQwEEBQUGBQYMBwcMGhEPERoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoa/8IAEQgAAQACAwERAAIRAQMRAf/EABQAAQAAAAAAAAAAAAAAAAAAAAf/xAAUAQEAAAAAAAAAAAAAAAAAAAAA/9oADAMBAAIQAxAAAAF/P//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAQUCf//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQMBAT8Bf//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQIBAT8Bf//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEABj8Cf//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAT8hf//aAAwDAQACAAMAAAAQH//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQMBAT8Qf//EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQIBAT8Qf//EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAT8Qf//Z';
});
}
The only way to really find out for sure if the browser rotates based on exif data: Load up an image with exif ratation and see how it comes out.
This is due to an update in Chrome 81 that now has and respects the 'image-orientation' property. https://developer.mozilla.org/en-US/docs/Web/CSS/image-orientation
Chrome now defaults all images to 'from-image' meaning it will read the EXIF data to determine the rotation data of the image. Below is basically what I did to detect if the browser supports functionality like this since future versions of iOS and other browsers expect to do this also.
function browserImageRotationSupport(){
let imgTag = document.createElement('img');
return imgTag.style.imageOrientation !== undefined;
}
I was able to use this test to differentiate the browsers:
if (CSS.supports("image-orientation", "from-image")) {
...
}
const iOS = !!navigator.platform && /iPad|iPhone|iPod/.test(navigator.platform);
I use this snippet to check if it is IOS and only rotate the canvas ctx if it is IOS. I think older versions of android don't auto-rotate the image because I still have bugreports coming in from android users.
Setting the CSS on the canvas element as opposed to the img will fix this if you're drawing to a canvas that is part of the DOM.
canvas {
image-orientation: none;
}
As of writing the element has to be in the DOM because it uses the computed style. That only exists in a DOM context. You can read more in the issue on the Chromium tracker.
https://bugs.chromium.org/p/chromium/issues/detail?id=158753

Storing the result of blended images in AS3.0

I am making an interactive animation which background images are masked by a shape captured in a camera. In the every frames, camera changes images into black and white images and then it's used to mask background images.
In the code, "now" is the images captured by camera, "P1" is the back ground image.
After I masked by using blend mode multiply, I wish to store the result image which is masked and use for other things.
I don't understand when I use blendmode function, how the result is defined well.
Can I capture, copy or store the result image?
var P1:MovieClip = new p1();
var mskimg:MovieClip = new maskimage();
var bitmap_obj:Bitmap = new Bitmap(now);
addChild(P1);
P1.blendMode = BlendMode.LAYER;
addChild(bitmap_obj);
bitmap_obj.blendMode = BlendMode.MULTIPLY;
You can draw a masked object if you put everything that you intend to be drawn into a container, then call BitmapData.draw() to draw the complete layered object to some BitmapData. Then, you can use that bitmap data to do whatever you want, Graphics.beginBitmapFill(), FileReference.save(), whatever. The operation is however expensive, so it'll be better if you trigger the drawing by responding to a user action, and not do that every frame by default, unless required by your application's logic.

HTML5 Image quality after scaling bitmap is bad

I am creating my first game in HTML 5 for mobile.
I'm just using the library "Easeljs-0.7.0", nothing PhoneGap.
I made my images to canvas 2024x1024 pixels, then each image by setting the scaleX and scaleY. However when the scale is less than 0.5 makes all images with poor quality.
How to solve this?
var test = new createjs.Bitmap("test.png");
test.scaleX = 0.23456;
test.scaleY = 0.23456;
APP.stage.addChild(test);
APP.stage.update();
Unfortunately this is how Canvas renders the bitmap, and its not something that can be controlled by JavaScript. It is possible to get some results in various browsers using the `context.imageSmoothingEnabled' property, but currently it requires vendor prefixes. Check out this thread:
Canvas Image Smoothing
To do this with EaselJS, you need to get the canvas context, which is not currently available without manually accessing it:
var context = myStage.canvas.getContext("2d");
context.webkitImageSmoothingEnabled = context.mozImageSmoothingEnabled = true;

How do you change the background color of a live tile at runtime?

I'm trying to change the background color of a live tiles on my Windows 8 application at runtime. Is this possible? Does anybody have a code snippet?
Background color is specified in app manifest. Thus it does not seems to be possible.
The same can be emulated by sending custom time at runtime with appropriate background image (for a color). This is quirky way, but my little mind can think of this approach only.
Suppose you have already fixed the problem but I will post this info here, might be useful if someone come accross this thread.
For Primary Tile the answer from Bruno Lemos is correct.
For Seconday Tiles I would use TileUpdateManager.CreateTileUpdaterForSecondaryTile(Tile2_ID).
Using TileNotifications and TileUpdaterManager works instantly to me.
Using the Tile templates you will have to modify some XML code from the template, but you could also download NotificationsExtensions NuGet package and do it this way:
var tile2 = TileContentFactory.CreateTileSquare150x150PeekImageAndText01();
tile2.Branding = TileBranding.Name;
tile2.Image.Src = "ms-appx:///assets/Logo-transparent.png"; //Useful to have the same logo image with transparent background and other colors
tile2.TextHeading.Text = "Heading";
tile2.TextBody1.Text = "String1";
tile2.TextBody2.Text = "String2";
tile2.TextBody3.Text = "String3";
var doc = new XmlDocument();
doc.LoadXml(tile2.ToString());
var updater = TileUpdateManager.CreateTileUpdaterForSecondaryTile(tile2_id);
updater.EnableNotificationQueueForSquare150x150(true); //enables up to 5 different tile *animations* for 150 square icon, can be enabled for other sizes too
updater.Update(new TileNotification(doc) { Tag = "1" });
The code above takes into account the SecondaryTile has been created previously.
You can see the different templates for icons here: https://msdn.microsoft.com/en-us/library/windows/apps/xaml/windows.ui.notifications.tiletemplatetype.aspx
You only have to change the line TileContentFactory.CreateNAMEOFTEMPLATE and fill the different string/image fields.
Info about EnableNotificationQueue to have more than 1 animation in the tile can be found here: https://msdn.microsoft.com/en-us/library/windows/apps/hh781199.aspx
For Primary / Default Tile, you can do what #Tilak said: create tile images notifications and use TileUpdateManager.CreateTileUpdaterForApplication().Update to update it.
For Secondary Tiles, do the following:
In the app manifest, set the background color to "transparent"
In the app manifest, use images with transparent background
Now you can do that at runtime:
var tile = new SecondaryTile("YOUR_TILE_ID");
tile.VisualElements.BackgroundColor = Colors.Red;
await tile.UpdateAsync();
The problem is: The background is not updating immediately and I don't
know why. But you can see the effect after sign out / sign in, for
example. (tested using windows 10)
So this answer is incomplete, but hopefully can give insights to people coming here from google, like me.
#brunolemos

How to capture an image of an HTML element, and maintain transparency?

I'm working on a page that will allow a webmaster to add styles to their twitter feed. Several of these styles use transparency in their display. I want to create a list of images for them to choose from. Currently I am taking screenshots on a checked background such as this:
But that isn't really what I want.
Is their some method of capturing an image of an HTML element, and maintaining the transparency?
EDIT: I'm just digging into this, so I'm coming across new topics, such as HTML5 Canvas, and -moz-element. Is it possible to set a canvas background to the html element using -moz-element, then extract the image data from the canvas? I'm going to try this unless someone who's 'been there done that' heads me off.
EDIT: The -moz-element and canvas was a deadend. -moz-element will set the item as a background, but will not allow you to save background image. And canvas doesn't save its background, even when the background is a normal image.
It requires a little bit of work, but it is doable, as long as it's HTML you're laying out. You can't recover the transparency of markup in pages you've downloaded without saving those pages and editing them locally. By rendering the HTML elements multiple times, on different background colors, the opacity can be derived using an image editor. You're going to need a decent image editor, I use GIMP.
Render the elements you want to save three times, on a black, a white and a neutral gray background(#888).
Using a screen capture program, capture those displays and crop them to the exact same areas.
In GIMP open those images as layers and order them black, white and gray, top to bottom.
Set the top, black layer to difference mode. This will give a grayscale difference between the black and white layers.
Merge down the top layer. This will leave us with two layers. The gray background layer and the grayscale difference. Invert the colors of the difference layer, and copy it to the clipboard.
Add a layer mask to the gray background layer and paste the clipboard into the layer mask.
Delete the grayscale layer and apply the layer mask on the gray background layer. That should leave one layer with opacity similar to the original.
The opacity is off by a bit, but if we duplicate the layer and merge it with itself, it's right in the ballpark.
It's probably not pixel perfect, but it is proof of concept. Opacity of HTML markup can be captured.
Using Puppeteer makes this much easier to do. It runs a web-page in-memory.
Start a local fileserver - python -m SimpleHTTPServer 8080
Then this script should do the trick:
const puppeteer = require('puppeteer')
;(async () => {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto('http://localhost:8080/index.html', {
waitUntil: 'networkidle0'
})
const elements = await page.$('body')
await page.evaluate(() => (document.body.style.background = 'transparent'))
await elements.screenshot({ path: 'myImg.png', omitBackground: true })
await browser.close()
})()
the docs for .screenshot() are here.
What you'd need is a web browser that can render into an image buffer in memory instead of the screen.
My guess is that all browsers can do this (that should be part of the renderers) but I'm not aware of any browser where you can access this function, at least not from JavaScript.
If you download the WebKit sources, there should be test cases which do something like that :-/
No, there's no software that will allow you to take screenshots and preserve the transparency of individual visual elements as a transparent spot in the image, because that's not how a screenshot works - screenshots are WYSIWYG, by definition, all elements in your screenshot will always have a non-transparent background.
I think your best bet here is to recreate the desired portion as an image, where you can control the transparency normally. It's not the best solution, but if you're doing this a lot with the same kinds of things, it will be much faster for you rather than cropping/editing screenshots.