Limit rendered image size in icepdf - icepdf

While rendering a bunch of PDFs to images, icepdf seemingly randomly bombs out with an OutOfMemoryError. Trying to track this down I find two things:
Close to the OOM it rendered an A0 page or similarly large document pages
With eclipse memory analyzer I find 1/2GB images in memory.
This suggests to limit the output image size to something managable. I wonder what the easiest way is to do this?
I looked at icepdf's Page object but there it is strongly recommended to just always use Page.BOUNDARY_CROPBOX and other uses seem not to be documented in the Javadoc.
How can I limit the output image size of Document.getPageImage or what other measure could I use to prevent the OOM (other than just increasing the Xmx, which I can't). Reduction of image quality is an option. But it should apply only to "oversize" images, not to all.
I tried already to use a predefined image using Document.paintPage(), but this was not sufficient.
Debug finally allowed me to zoom in on a document that is problematic. I get a log like:
2016-12-09T14:23:35Z DEBUG class org.icepdf.core.pobjects.Document 1 MEMFREE: 712484296 of 838860800
2016-12-09T14:23:35Z DEBUG class org.icepdf.core.pobjects.Document 1 LOADING: ..../F1-2.pdf
2016-12-09T14:23:37Z WARN class org.icepdf.core.pobjects.graphics.ScaledImageReference 1 Error loading image: 9 0 R Image stream= {Type=XObject, Length=8 0 R, Filter=FlateDecode, ColorSpace=DeviceGray, Decode=[1, 0], Height=18676, Width=13248, Subtype=Image, BitsPerComponent=1, Name=Im1} 9 0 R
so this would be Height=18676, Width=13248 which is really huge.
I guess that the OOM happens already during loading of the image, so later scaling does not help. Also it seems that the property org.icepdf.core.imageReference=scaled does not hit early enough.
For me it would be fine to just ignore oversized images like this. Any chance?

Image loading is by far the most memory expensive memory task when decoding PDF content. At this time there isn't an esasy way to turn off image loading for really large image however I'll give you a few code hints if you want to implement this your self.
The ImageReferenceFactory.java class is the factory behind the system property org.icepdf.core.imageReference, you'll see that the default for getImageReferenced() is ImageStreamReference. You can create a new ImageReference type like this:
public static org.icepdf.core.pobjects.graphics.ImageReference
getImageReference(ImageStream imageStream, Resources resources, GraphicsState graphicsState,
Integer imageIndex, Page page) {
switch (scaleType) {
case SCALED:
return new ScaledImageReference(imageStream, graphicsState, resources, imageIndex, page);
case SMOOTH_SCALED:
return new SmoothScaledImageReference(imageStream, graphicsState, resources, imageIndex, page);
case MIP_MAP:
return new MipMappedImageReference(imageStream, graphicsState, resources, imageIndex, page);
case SKIP_LARGE:
return new SkipLargeImageReference(imageStream, graphicsState, resources, imageIndex, page);
default:
return new ImageStreamReference(imageStream, graphicsState, resources, imageIndex, page);
}
}
Next you can extend the class ImageStreamReference with your new SkipLargeImageReference class. Then override the call() method as follows and it will skip the loading of any image over the defined MAX_SIZE .
public BufferedImage call() {
BufferedImage image = null;
if (imageStream.getWidth() < MAX_SIZE && imageStream.getHeight() < MAX_SIZE){
long start = System.nanoTime();
try {
image = imageStream.getImage(graphicsState, resources);
} catch (Throwable e) {
logger.log(Level.WARNING, "Error loading image: " + imageStream.getPObjectReference() +
" " + imageStream.toString(), e);
}
long end = System.nanoTime();
notifyImagePageEvents((end - start));
return image;
}
return null;
}
On a side note: To minimize the the amount of memory needed to decode an image make sure you are using org.icepdf.core.imageReference=default as this will decode the image only once. org.icepdf.core.imageReference=scaled will actually decode the image at full size and then do the scale which can create a very large memory spike. We are experimenting with NIO's direct ByteBuffers which looks promising to moving the decode memory usage off the heap, so hopefully this will get better in the future.

Related

Can i load data referenced by a Web Component dynamically, with caching?

I'm currently learning Web Components and I wonder if it is possible to have a Component load its own data dynamically, similar to how <img> does from its src attribute, i.e. something like this:
<my-fancy-thingy src='/stuff.json'></my-fancy-thingy>
Obviously this functionality would be useful if stuff.json could be rather large, so it should also be possible to make use of the browser's caching mechanism so the referenced file doesn't get reloaded every time we request the page, unless changed.
Can this be done?
Sure, take inspiration from <load-file> See Dev.to Post
/*
defining the <load-file> Web Component,
yes! the documenation is longer than the code
License: https://unlicense.org/
*/
customElements.define("load-file", class extends HTMLElement {
// declare default connectedCallback as sync so await can be used
async connectedCallback(
// attach a shadowRoot if none exists (prevents displaying error when moving Nodes)
// declare as parameter to save 4 Bytes: 'let '
shadowRoot = this.shadowRoot || this.attachShadow({mode:"open"})
) {
// load SVG file from src="" async, parse to text, add to shadowRoot.innerHTML
shadowRoot.innerHTML = await (await fetch(this.getAttribute("src"))).text()
// append optional <tag [shadowRoot]> Elements from inside <load-svg> after parsed <svg>
shadowRoot.append(...this.querySelectorAll("[shadowRoot]"))
// if "replaceWith" attribute
// then replace <load-svg> with loaded content <load-svg>
// childNodes instead of children to include #textNodes also
this.hasAttribute("replaceWith") && this.replaceWith(...shadowRoot.childNodes)
}
})
Change .text() to .json() and it parses JSON files
Caching can be done by storing the String in localStorage (but a 5MB limit total, I think):
https://en.wikipedia.org/wiki/Web_storage
https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage
You need to come up with "data has changed" strategy; as the Client has no clue when data actually was changed. Maybe an extra semaphore file/endpoint that provides info if the (large) JSON file was changed.
This works like a charm
export class MonElement extends HTMLElement {
constructor(){
super();
this.attachShadow({mode:'open'});
(...)
this.shadowRoot.appendChild(atemplate);
}
connectedCallback(){...}
static get observedAttributes(){
return ['src'];
}
attributeChangedCallback(nameattr,oldval,newval)
{
if (nameattr==='src') {
this[nameattr]=newval;
here do the fetch for the src value which is newval then update what you got in the innerdom
}
(...)

Flash Builder (FlexPrintJob & PrintDataGrid) Chrome Shockwave error

I have an mx:application using the Flex 4.6.0 SDK and I’m having some issues with the FlexPrintJob in Chrome only. The FlexPrintJob worked fine in chrome, up until maybe a couple weeks ago (I made no changes to the code) and now I’ve started experiencing “Shockwave Crashes”.
While printing I’m using:
first title page template
middle section template to handle a DataGrid. This is using the PrintDataGrid and loops through the dataProvider to see if the data will fit on one page, if not it will create another.
Terms and conditions last page template
Problem: I’ve narrowed it down to this, I’m getting the Shockwave Crash error in chrome when the data (for the middle section) exceeds one page and tries to create another. This just started happening, I’m guessing with a chrome update…Sorry if I left something out and my description is lacking detail. I can add more clarification if needed.
Any ideas what’s going on?
Thanks!
--moe
public function doPrint(): void {
// Create a FlexPrintJob instance.
var printJob: FlexPrintJob = new FlexPrintJob();
// Start the print job.
if (printJob.start()) {
// Create a FormPrintView control as a child of the application.
var thePrintView: FormPrintView = new FormPrintView();
addElement(thePrintView);
// Set the print view properties.
thePrintView.width = printJob.pageWidth;
thePrintView.height = printJob.pageHeight;
thePrintView.horizontalAlign = "center";
// Set the data provider of the FormPrintView component's DataGrid to be the data provider of the displayed DataGrid.
thePrintView.summaryGrid.dataProvider = summaryGrid.dataProvider;
// Create a single-page image.
thePrintView.showPage("single");
// If the print image's DataGrid can hold all the data provider's rows, add the page to the print job.
if (!thePrintView.summaryGrid.validNextPage) {
printJob.printAsBitmap = false;
printJob.addObject(UIComponent(mainPagePrint), FlexPrintJobScaleType.MATCH_WIDTH);
printJob.addObject(thePrintView);
printJob.addObject(UIComponent(terms), FlexPrintJobScaleType.MATCH_WIDTH);
}
// Otherwise, the job requires multiple pages.
else {
// Create the first page and add it to the print job.
thePrintView.showPage("first");
printJob.printAsBitmap = false;
printJob.addObject(UIComponent(mainPagePrint), FlexPrintJobScaleType.MATCH_WIDTH);
printJob.addObject(thePrintView);
thePrintView.pageNumber++;
// Loop through the following code until all pages are queued.
while (true) {
// Move the next page of data to the top of the PrintDataGrid.
thePrintView.summaryGrid.nextPage();
printJob.printAsBitmap = false;
// Try creating a last page.
thePrintView.showPage("last");
// If the page holds the remaining data, or if the last page was completely filled by the last grid data, queue it for printing.
// Test if there is data for another PrintDataGrid page.
if (!thePrintView.summaryGrid.validNextPage) {
// This is the last page; queue it and exit the print loop.
printJob.addObject(thePrintView);
printJob.addObject(UIComponent(terms), FlexPrintJobScaleType.MATCH_WIDTH);
break;
} else // This is not the last page. Queue a middle page.
{
thePrintView.showPage("middle");
printJob.addObject(thePrintView);
thePrintView.pageNumber++;
}
}
}
// All pages are queued; remove the FormPrintView control to free memory.
removeElement(thePrintView);
}
// Send the job to the printer.
printJob.send();
}
I ended up taking things apart one-by-one and figured out what it was - the issue was is the FormPrintView. I had some unneeded properties in my PrintDataGrid that I think were originally copied from my main datagrid in my application.
I'm not sure why it was working before and just starting acting crashing now, but either way I shouldn't have had some of those properties there in the first place.
thanks!
--moe
<mx:PrintDataGrid id="summaryGrid" width="100%" height="100%" sizeToPage="true"
alternatingItemColors="[#f7f7f7, #ffffff]" alpha="0.8"
borderStyle="solid" borderThickness="1" color="#646262"
creationComplete="summaryGrid_creationCompleteHandler(event)" fontSize="9"
headerColors="[#ffffff, #e8e8e8]" headerHeight="25" paddingTop="5"
rowHeight="55" textAlign="center" wordWrap="true" paddingRight="-1" >

Yii2 - Image upload and resizing,ajax upload support extension

Is there any good image uploading and resize extension for yii2; I don't want to use kartik because since I had a problem I've not gotten any help to understand where the problem is, same situation with Illustrated behavior so I am stack in my project.
What I want is multiple image uploading,ajax support(even for old browser if not to turn to normal file input), image resizing keeping good quality,allowing one image to be saved in different sizes and Preview the file when selected from client side(not obliged).
Usually I use image magick direcly.
Check if these two functions can be useful for you:
public static function generateImagesScaledAndCropped($inputFile, $outputFile, $params)
{
$imageMagickConvert = \Yii::$app->params['imagick.convert'];
$cmd = sprintf("%s %s -resize %dx%d^ -gravity Center -crop %dx%d+0+0 %s", $imageMagickConvert, $inputFile, $params['edge'], $params['edge'], $params['edge'], $params['edge'], $outputFile);
exec($cmd);
}
public static function generateImagesScaledByWidth($inputFile, $outputFile, $params)
{
$imageMagickConvert = \Yii::$app->params['imagick.convert'];
$cmd = sprintf("%s %s -resize %d %s", $imageMagickConvert, $inputFile, $params['width'], $outputFile);
exec($cmd);
}
Params are:
<?php
return [
'imagick.convert' => '/usr/bin/convert',
'imagick.composite' => '/usr/bin/composite',
];
I use Imagine as abstract layer on Imagine library which
uses populars php libraries to work with images
http://www.yiiframework.com/doc-2.0/ext-imagine-index.html

LibTiff.NET append mode bug?

I've started using LibTiff.NET for writing tiff IPTC tags lately and discovered strange behavior on some files that i have here. I'm using sample code that ships with LibTiff.NET binaries, and it works fine with most of the images, but some files are having image data corruption after these lines:
class Program
{
private const TiffTag TIFFTAG_GDAL_METADATA = (TiffTag)42112;
private static Tiff.TiffExtendProc m_parentExtender;
public static void TagExtender(Tiff tif)
{
TiffFieldInfo[] tiffFieldInfo =
{
new TiffFieldInfo(TIFFTAG_GDAL_METADATA, -1, -1, TiffType.ASCII,
FieldBit.Custom, true, false, "GDALMetadata"),
};
tif.MergeFieldInfo(tiffFieldInfo, tiffFieldInfo.Length);
if (m_parentExtender != null)
m_parentExtender(tif);
}
public static void Main(string[] args)
{
// Register the extender callback
// It's a good idea to keep track of the previous tag extender (if any) so that we can call it
// from our extender allowing a chain of customizations to take effect.
m_parentExtender = Tiff.SetTagExtender(TagExtender);
string destFile = #"d:\00000641(tiffed).tif";
File.Copy(#"d:\00000641.tif", destFile);
//Console.WriteLine("Hello World!");
// TODO: Implement Functionality Here
using (Tiff image = Tiff.Open(destFile, "a"))
{
// we should rewind to first directory (first image) because of append mode
image.SetDirectory(0);
// set the custom tag
string value = "<GDALMetadata>\n<Item name=\"IMG_GUID\">" +
"817C0168-0688-45CD-B799-CF8C4DE9AB2B</Item>\n<Item" +
" name=\"LAYER_TYPE\" sample=\"0\">athematic</Item>\n</GDALMetadata>";
image.SetField(TIFFTAG_GDAL_METADATA, value);
// rewrites directory saving new tag
image.CheckpointDirectory();
}
// restore previous tag extender
Tiff.SetTagExtender(m_parentExtender);
Console.Write("Press any key to continue . . . ");
Console.ReadKey(true);
}
}
After opening i see mostly blank white image or multiple black and white lines instead of text that have been written there (i don't need to read\write tags to produce this behavior). I noticed this happens when image already has a custom tag (console window alerts about it) or one of tags have got 'bad value' (console window in this case says 'vsetfield:%pathToTiffFile%: bad value 0 for "%TagName%" tag').
Original image: http://dl.dropbox.com/u/1476402/00000641.tif
Image after LibTiff.NET: http://dl.dropbox.com/u/1476402/00000641%28tiffed%29.tif
I would be grateful for any help provided.
You probably should not use CheckpointDirectory method for files opened in append mode. Try using RewriteDirectory method instead.
It will rewrite the directory, but instead of place it at it's old
location (as WriteDirectory() would) it will place them at the end of
the file, correcting the pointer from the preceeding directory or file
header to point to it's new location. This is particularly important
in cases where the size of the directory and pointed to data has
grown, so it won’t fit in the space available at the old location.
Note that this will result in the loss of the previously used
directory space.

saving google map to image from a browser component window inside a c# application

I wanted to save the google map into an image from a webpage.
while i was searching for that i got this program.
http://www.codres.de/downloads/gms.exe[^]
besides other alternatives like print screen i wanted to use a program or map api which can save a specified dimension of google map instead of the screen.
i have used browser component in c# for http access and for displaying certain webpages.
I want to know whether there are options to capture the browser screen to image using any c# functionality or even the browser component would have given such options. just a guess.
i would like to have answers, suggestions on how to capture the map with custom dimension and zoom size to an image.
I used this to get captcha Image from the current page, so you can use similar code just amend the imageID to point to the google map image and use this solution for zooming.
public string newsavefunction(WebBrowser webBrowser1)
{
IHTMLDocument2 doc = (IHTMLDocument2)webBrowser1.Document.DomDocument;
IHTMLControlRange imgRange = (IHTMLControlRange)((HTMLBody)doc.body).createControlRange();
string imagename = string.Empty;
try
{
foreach (IHTMLImgElement img in doc.images)
{
imgRange.add((IHTMLControlElement)img);
imgRange.execCommand("Copy", false, null);
using (Bitmap bmp = (Bitmap)Clipboard.GetDataObject().GetData(DataFormats.Bitmap))
{
bmp.Save(#"F:\captchaimages\captchapic.jpg");
}
imagename = img.nameProp;
break;
}
}
catch (System.Exception exp)
{ }
return imagename;
}