So, using PyQt5's QWebEngineView and the .setHTML and .setContent methods have a 2 MB size limitation. When googling for solutions around this, I found two methods:
Use SimpleHTTPServer to serve the file. This however gets nuked by a firewall employed in the company.
Use File Urls and point to local files. This however is a rather bad solution, as the HTML contains confidential data and I can't leave it on the harddrive, under any circumstance.
The best solution I currently see is to use file urls, and get rid of the file on program exit/when loadCompleted reports it is done, whichever comes first.
This is however not a great solution and I wanted to ask if there is a solution I'm overlooking that would be better?
Why don't you load/link most of the content through a custom url scheme handler?
webEngineView->page()->profile()->installUrlSchemeHandler("app", new UrlSchemeHandler(e));
class UrlSchemeHandler : public QWebEngineUrlSchemeHandler
{ Q_OBJECT
public:
void requestStarted(QWebEngineUrlRequestJob *request) {
QUrl url = request->requestUrl();
QString filePath = url.path().mid(1);
// get the data for this url
QByteArray data = ..
//
if (!data.isEmpty())
{
QMimeDatabase db;
QString contentType = db.mimeTypeForFileNameAndData(filePath,data).name();
QBuffer *buffer = new QBuffer();
buffer->open(QIODevice::WriteOnly);
buffer->write(data);
buffer->close();
connect(request, SIGNAL(destroyed()), buffer, SLOT(deleteLater()));
request->reply(contentType.toUtf8(), buffer);
} else {
request->fail(QWebEngineUrlRequestJob::UrlNotFound);
}
}
};
you can then load a website by webEngineView->load(new QUrl("app://start.html"));
All relative pathes from inside will also be forwarded to your UrlSchemeHandler..
And rember to add the respective includes
#include <QWebEngineUrlRequestJob>
#include <QWebEngineUrlSchemeHandler>
#include <QBuffer>
One way you can go around this is to use requests and QWebEnginePage's method runJavaScript:
web_engine = QWebEngineView()
web_page = web_engine.page()
web_page.setHtml('')
url = 'https://youtube.com'
page_content = requests.get(url).text
# document.write writes a string of text to a document stream
# https://developer.mozilla.org/en-US/docs/Web/API/Document/write
# And backtick symbol(``) is for multiline strings
web_page.runJavaScript('document.write(`{}`);'.format(page_content))
Related
I have a small GUI that I use to load/save json configuration files, the most important parameters are in the gui below:
![conf]
The problem I have been trying to solve is that I am not able to create an object inside a QTextEdit and am not sure why despite I am following official documentation on how to do that.
Below a snippet of code both for the load and save button.
Also for the sake of brevity I only kept how I did the spinbox and, of course, the textedit:
void SettingsForm::on_loadBtn_clicked()
{
// Opening file dialog....
if(listDocksConfig.isEmpty())
{
QMessageBox::information(this, tr("Message"), tr("Please Open Configuration"));
}
else
{
QJsonDocument doc;
QJsonObject obj;
QByteArray data_json;
QFile input(listDocksConfig);
if(input.open(QIODevice::ReadOnly | QIODevice::Text))
{
data_json = input.readAll();
doc = doc.fromJson(data_json);
obj = doc.object();
const double xposValue = obj["X Pos"].toDouble();
QTextEdit textEdit = QTextEdit::setText(obj["comments"]); // <- Error Here
ui->doubleSpinBox_XPos->setValue(xposValue);
ui->textEdit->setText(textEdit); // <- Error Here
}
else
{
// do something
}
}
}
void SettingsForm::on_saveBtn_clicked()
{
// saving configuration with file dialog....
if(listDocksConfig.isEmpty())
{
// do something...
}
else
{
const double xposValue = ui->doubleSpinBox_XPos->value();
QTextEdit textEdit = ui->textEdit->setPlainText(); // <- Error Here
QJsonDocument doc;
QJsonObject obj;
obj["X Pos"] = xposValue;
obj["comments"] = textEdit.toString(); // <- Error Here
doc.setObject(obj);
QByteArray data_json = doc.toJson();
QFile output(listDocksConfig);
}
}
What I have done so far:
I consulted the official documentation on how to solve this problem, but could not figure out why that was not working. I also went ahead and try to use an alternative such as setText but still no luck.
I came across this source which I used as guidance for my example and solved almost all problems but the QTextEdit one.
This additional post was useful but still couldn't solve the problem.
Thanks for pointing to the right direction for solving this problem.
this line is wrong!!
QTextEdit textEdit = ui->textEdit->setPlainText();
setPlainText() needs const QString &text as parameter
you cant do that, read the official doc here
the method is void, ie. it returns nothing so you can not use void to init a QTextEdit object
update:
you already have a textEdit in the layout, so no reason to redefine one...
you can do:
ui->textEdit->setPlainText(obj["comments"].toString());
I have a public (anyone with the link can view) file on my Google Drive and I want to use the content of it in my Android app.
From what I could gather so far, I need the fileID, the OAuth token and the client ID - these I already got. But I can't figure out what is the exact methodology of authorising the app or fetching the file.
EDIT:
Simply reading it using file.readAsLines didn't work:
final file = new File(dogListTxt);
Future<List<String>> dogLinks = file.readAsLines();
return dogLinks;
The dogLinks variable isn't filled with any data, but I get no error messages.
The other method I tried was following this example but this is a web based application with explicit authorization request (and for some reason I was never able to import the dart:html library).
The best solution would be if it could be done seamlessly, as I would store the content in a List at the application launch, and re-read on manual refresh button press.
I found several old solutions here, but the methods described in those doesn't seem to work anymore (from 4-5 years ago).
Is there a good step-by-step tutorial about integrating the Drive API in a flutter application written in dart?
I had quite a bit of trouble with this, it seems much harder than it should be. Also this is for TXT files only. You need to use files.export() for other files.
First you need to get a list fo files.
ga.FileList textFileList = await drive.files.list(q: "'root' in parents");
Then you need to get those files based on ID (This is for TXT Files)
ga.Media response = await drive.files.get(filedId, downloadOptions: ga.DownloadOptions.FullMedia);
Next is the messy part, you need to convert your Media object stream into a File and then read the text from it. ( #Google, please make this easier.)
List<int> dataStore = [];
response.stream.listen((data) {
print("DataReceived: ${data.length}");
dataStore.insertAll(dataStore.length, data);
}, onDone: () async {
Directory tempDir = await getTemporaryDirectory(); //Get temp folder using Path Provider
String tempPath = tempDir.path; //Get path to that location
File file = File('$tempPath/test'); //Create a dummy file
await file.writeAsBytes(dataStore); //Write to that file from the datastore you created from the Media stream
String content = file.readAsStringSync(); // Read String from the file
print(content); //Finally you have your text
print("Task Done");
}, onError: (error) {
print("Some Error");
});
There currently is no good step-by-step tutorial, but using https://developers.google.com/drive/api/v3/manage-downloads as a reference guide for what methods to use in Dart/Flutter via https://pub.dev/packages/googleapis: to download or read the contents of a Google Drive file, you should be using googleapis/Drive v3, or specifically, the methods from the FilesResourceApi class.
drive.files.export(), if this is a Google document
/// Exports a Google Doc to the requested MIME type and returns the exported content. Please note that the exported content is limited to 10MB.
drive.files.get(), if this something else, a non-Gdoc file
/// Gets a file's metadata or content by ID.
Simplified example:
var drive = new DriveApi(http_client);
drive.files.get(fileId).then((file) {
// returns file
});
However, what I discovered was that this Dart-GoogleAPIs library seemed to be missing a method equivalent to executeMediaAndDownloadTo(outputStream). In the original Google Drive API v3, this method adds the alt=media URL parameter to the underlying HTTP request. Otherwise, you'll get the error, which is what I saw:
403, message: Export requires alt=media to download the exported
content.
And I wasn't able to find another way to insert that URL parameter into the current request (maybe someone else knows?). So as an alternative, you'll have to resort to implementing your own Dart API to do the same thing, as hinted by what this OP did over here https://github.com/dart-lang/googleapis/issues/78: CustomDriveApi
So you'll either:
do it through Dart with your own HttpClient implementation and try to closely follow the REST flow from Dart-GoogleAPIs, but remembering to include the alt=media
or implement and integrate your own native-Android/iOS code and use the original SDK's convenient executeMediaAndDownloadTo(outputStream)
(note, I didn't test googleapis/Drive v2, but a quick examination of the same methods looks like they are missing the same thing)
I wrote this function to get file content of a file using its file id. This is the simplest method I found to do it.
Future<String> _getFileContent(String fileId) async {
var response = await driveApi.files.get(fileId, downloadOptions: DownloadOptions.fullMedia);
if (response is! Media) throw Exception("invalid response");
return await utf8.decodeStream(response.stream);
}
Example usage:
// save file to app data folder with 150 "hello world"s
var content = utf8.encode("hello world" * 150);
driveApi.files
.create(File(name: fileName, parents: [appDataFolder]),
uploadMedia: Media(Stream.value(content), content.length))
.then((value) {
Log().i("finished uploading file ${value.id}");
var id = value.id;
if (id != null) {
// after successful upload, read the recently uploaded file content
_getFileContent(id).then((value) => Log().i("got content is $value"));
}
});
I'm trying to get some file information about a file the user select with the FileOpenPicker, but all the information like the path and name are empty. When I try to view the object in a breakpoint I got the following message:
file = 0x03489cd4 <Information not available, no symbols loaded for shell32.dll>
I use the following code for calling the FileOpenPicker and handeling the file
#include "pch.h"
#include "LocalFilePicker.h"
using namespace concurrency;
using namespace Platform;
using namespace Windows::Storage;
using namespace Windows::Storage::Pickers;
const int LocalFilePicker::AUDIO = 0;
const int LocalFilePicker::VIDEO = 1;
const int LocalFilePicker::IMAGES = 2;
LocalFilePicker::LocalFilePicker()
{
_init();
}
void LocalFilePicker::_init()
{
_openPicker = ref new FileOpenPicker();
_openPicker->ViewMode = PickerViewMode::Thumbnail;
}
void LocalFilePicker::askFile(int categorie)
{
switch (categorie)
{
case 0:
break;
case 1:
_openPicker->SuggestedStartLocation = PickerLocationId::VideosLibrary;
_openPicker->FileTypeFilter->Append(".mp4");
break;
case 2:
break;
default:
break;
}
create_task(_openPicker->PickSingleFileAsync()).then([this](StorageFile^ file)
{
if (file)
{
int n = 0;
wchar_t buf[1024];
_snwprintf_s(buf, 1024, _TRUNCATE, L"Test: '%s'\n", file->Path);
OutputDebugString(buf);
}
else
{
OutputDebugString(L"canceled");
}
});
}
Can anybody see whats wrong with the code or some problems with settings for the app why it isn't work as expected.
First an explanation why you are having trouble debugging, this is going to happen a lot more when you write WinRT programs. First, do make sure that you have the correct debugging engine enabled. Tools + Options, Debugging, General. Ensure that the "Use Managed Compatibility Mode" is turned off.
You can now inspect the "file" option, it should resemble this:
Hard to interpret of course. What you are looking at is a proxy. It is a COM term, a wrapper for COM objects that are not thread-safe or live in another process or machine. The proxy implementation lives in shell32.dll, thus the confuzzling diagnostic message. You can't see the actual object at all, accessing its properties requires calling proxy methods. Something that the debugger is not capable of doing, a proxy marshals the call from one thread to another, that other thread is frozen while the debugger break is active.
That makes you pretty blind, in tough cases you may want to write a littler helper code to store the property in a local variable. Like:
auto path = file->Path;
No trouble inspecting or watching that one. You should now have confidence that there's nothing wrong with file and you get a perfectly good path. Note how writing const wchar_t* path = file->Path; gets you a loud complaint from the compiler.
Which helps you find the bug, you can't pass a Platform::String to a printf() style function. Just like you can't with, say, std::wstring. You need to use an accessor function to convert it. Fix:
_snwprintf_s(buf, 1024, _TRUNCATE,
L"Test: '%s'\n",
file->Path->Data());
I have a bunch (over 1000) HTML files with just simple text. It's just a combination of text within a <table>. It's an internal batch of documents, not for web production.
The job we have is to convert them into JPEG files using Photoshop and the old copy paste method. It's tedious.
Is there a way you would do this process to make it more efficient/easier/simple?
I thought about trying to convert the HTML into Excel and then mail merging it into Word to print as JGEG. But I can't find (and rightly so) anything to convert HTML to XLSX.
Thoughts? Or is this just a manual job?
Here's a little something I created to convert a single html file to jpeg. It's not pretty (to say the least), but it works fine with a table larger than my screen. Put it inside a windows forms project. You can add more checks and call this program in a loop, or refactor it to work on multiple html files.
Ideas and techniques taken from -
Finding the needed size - http://social.msdn.microsoft.com/Forums/ie/en-US/f6f0c641-43bd-44cc-8be0-12b40fbc4c43/webbrowser-object-use-to-find-the-width-of-a-web-page
Creating the graphics - http://cplus.about.com/od/learnc/a/How-To-Save-Web-Page-Screen-Grab-csharp.htm
A table for example - copy-paste enlarged version of http://www.w3schools.com/html/html_tables.asp
static class Program
{
static WebBrowser webBrowser = new WebBrowser();
private static string m_fileName;
[STAThread]
static void Main(string[] args)
{
if (args.Length != 1)
{
MessageBox.Show("Usage: [fileName]");
return;
}
m_fileName = args[0];
webBrowser.DocumentCompleted += (a, b) => webBrowser_DocumentCompleted();
webBrowser.ScrollBarsEnabled = false; // Don't want them rendered
webBrowser.Navigate(new Uri(m_fileName));
Application.Run();
}
static void webBrowser_DocumentCompleted()
{
// Get the needed size of the control
webBrowser.Width = webBrowser.Document.Body.ScrollRectangle.Width + webBrowser.Margin.Horizontal;
webBrowser.Height = webBrowser.Document.Body.ScrollRectangle.Height + webBrowser.Margin.Vertical;
// Create the graphics and save the image
using (var graphics = webBrowser.CreateGraphics())
{
var bitmap = new Bitmap(webBrowser.Size.Width, webBrowser.Size.Height, graphics);
webBrowser.DrawToBitmap(bitmap, webBrowser.ClientRectangle);
string newFileName = Path.ChangeExtension(m_fileName, ".jpg");
bitmap.Save(newFileName, ImageFormat.Jpeg);
}
// Shamefully exit the application
Application.ExitThread();
}
}
You can load all files in one page and use this lib html2canvas to covert.
You can running in the background use nodejs with node-canvas or make it a desk app with node-webkit
In case anyone was looking for answer that works, I ended up using a program called Prince: https://www.princexml.com
It works amazingly, and just have to target the HTML with CSS or JS to make it match your output!
I'm attempting to create a PDF file from an HTML file. After looking around a little I've found: wkhtmltopdf to be perfect. I need to call this .exe from the ASP.NET server. I've attempted:
Process p = new Process();
p.StartInfo.UseShellExecute = false;
p.StartInfo.FileName = HttpContext.Current.Server.MapPath("wkhtmltopdf.exe");
p.StartInfo.Arguments = "TestPDF.htm TestPDF.pdf";
p.Start();
p.WaitForExit();
With no success of any files being created on the server. Can anyone give me a pointer in the right direction? I put the wkhtmltopdf.exe file at the top level directory of the site. Is there anywhere else it should be held?
Edit: If anyone has better solutions to dynamically create pdf files from html, please let me know.
Update:
My answer below, creates the pdf file on the disk. I then streamed that file to the users browser as a download. Consider using something like Hath's answer below to get wkhtml2pdf to output to a stream instead and then send that directly to the user - that will bypass lots of issues with file permissions etc.
My original answer:
Make sure you've specified an output path for the PDF that is writeable by the ASP.NET process of IIS running on your server (usually NETWORK_SERVICE I think).
Mine looks like this (and it works):
/// <summary>
/// Convert Html page at a given URL to a PDF file using open-source tool wkhtml2pdf
/// </summary>
/// <param name="Url"></param>
/// <param name="outputFilename"></param>
/// <returns></returns>
public static bool HtmlToPdf(string Url, string outputFilename)
{
// assemble destination PDF file name
string filename = ConfigurationManager.AppSettings["ExportFilePath"] + "\\" + outputFilename + ".pdf";
// get proj no for header
Project project = new Project(int.Parse(outputFilename));
var p = new System.Diagnostics.Process();
p.StartInfo.FileName = ConfigurationManager.AppSettings["HtmlToPdfExePath"];
string switches = "--print-media-type ";
switches += "--margin-top 4mm --margin-bottom 4mm --margin-right 0mm --margin-left 0mm ";
switches += "--page-size A4 ";
switches += "--no-background ";
switches += "--redirect-delay 100";
p.StartInfo.Arguments = switches + " " + Url + " " + filename;
p.StartInfo.UseShellExecute = false; // needs to be false in order to redirect output
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.RedirectStandardError = true;
p.StartInfo.RedirectStandardInput = true; // redirect all 3, as it should be all 3 or none
p.StartInfo.WorkingDirectory = StripFilenameFromFullPath(p.StartInfo.FileName);
p.Start();
// read the output here...
string output = p.StandardOutput.ReadToEnd();
// ...then wait n milliseconds for exit (as after exit, it can't read the output)
p.WaitForExit(60000);
// read the exit code, close process
int returnCode = p.ExitCode;
p.Close();
// if 0 or 2, it worked (not sure about other values, I want a better way to confirm this)
return (returnCode == 0 || returnCode == 2);
}
I had the same problem when i tried using msmq with a windows service but it was very slow for some reason. (the process part).
This is what finally worked:
private void DoDownload()
{
var url = Request.Url.GetLeftPart(UriPartial.Authority) + "/CPCDownload.aspx?IsPDF=False?UserID=" + this.CurrentUser.UserID.ToString();
var file = WKHtmlToPdf(url);
if (file != null)
{
Response.ContentType = "Application/pdf";
Response.BinaryWrite(file);
Response.End();
}
}
public byte[] WKHtmlToPdf(string url)
{
var fileName = " - ";
var wkhtmlDir = "C:\\Program Files\\wkhtmltopdf\\";
var wkhtml = "C:\\Program Files\\wkhtmltopdf\\wkhtmltopdf.exe";
var p = new Process();
p.StartInfo.CreateNoWindow = true;
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.RedirectStandardError = true;
p.StartInfo.RedirectStandardInput = true;
p.StartInfo.UseShellExecute = false;
p.StartInfo.FileName = wkhtml;
p.StartInfo.WorkingDirectory = wkhtmlDir;
string switches = "";
switches += "--print-media-type ";
switches += "--margin-top 10mm --margin-bottom 10mm --margin-right 10mm --margin-left 10mm ";
switches += "--page-size Letter ";
p.StartInfo.Arguments = switches + " " + url + " " + fileName;
p.Start();
//read output
byte[] buffer = new byte[32768];
byte[] file;
using(var ms = new MemoryStream())
{
while(true)
{
int read = p.StandardOutput.BaseStream.Read(buffer, 0,buffer.Length);
if(read <=0)
{
break;
}
ms.Write(buffer, 0, read);
}
file = ms.ToArray();
}
// wait or exit
p.WaitForExit(60000);
// read the exit code, close process
int returnCode = p.ExitCode;
p.Close();
return returnCode == 0 ? file : null;
}
Thanks Graham Ambrose and everyone else.
OK, so this is an old question, but an excellent one. And since I did not find a good answer, I made my own :) Also, I've posted this super simple project to GitHub.
Here is some sample code:
var pdfData = HtmlToXConverter.ConvertToPdf("<h1>SOO COOL!</h1>");
Here are some key points:
No P/Invoke
No creating of a new process
No file-system (all in RAM)
Native .NET DLL with intellisense, etc.
Ability to generate a PDF or PNG (HtmlToXConverter.ConvertToPng)
Check out the C# wrapper library (using P/Invoke) for the wkhtmltopdf library: https://github.com/pruiz/WkHtmlToXSharp
There are many reason why this is generally a bad idea. How are you going to control the executables that get spawned off but end up living on in memory if there is a crash? What about denial-of-service attacks, or if something malicious gets into TestPDF.htm?
My understanding is that the ASP.NET user account will not have the rights to logon locally. It also needs to have the correct file permissions to access the executable and to write to the file system. You need to edit the local security policy and let the ASP.NET user account (maybe ASPNET) logon locally (it may be in the deny list by default). Then you need to edit the permissions on the NTFS filesystem for the other files. If you are in a shared hosting environment it may be impossible to apply the configuration you need.
The best way to use an external executable like this is to queue jobs from the ASP.NET code and have some sort of service monitor the queue. If you do this you will protect yourself from all sorts of bad things happening. The maintenance issues with changing the user account are not worth the effort in my opinion, and whilst setting up a service or scheduled job is a pain, its just a better design. The ASP.NET page should poll a result queue for the output and you can present the user with a wait page. This is acceptable in most cases.
You can tell wkhtmltopdf to send it's output to sout by specifying "-" as the output file.
You can then read the output from the process into the response stream and avoid the permissions issues with writing to the file system.
My take on this with 2018 stuff.
I am using async. I am streaming to and from wkhtmltopdf. I created a new StreamWriter because wkhtmltopdf is expecting utf-8 by default but it is set to something else when the process starts.
I didn't include a lot of arguments since those varies from user to user. You can add what you need using additionalArgs.
I removed p.WaitForExit(...) since I wasn't handling if it fails and it would hang anyway on await tStandardOutput. If timeout is needed, then you would have to call Wait(...) on the different tasks with a cancellationtoken or timeout and handle accordingly.
public async Task<byte[]> GeneratePdf(string html, string additionalArgs)
{
ProcessStartInfo psi = new ProcessStartInfo
{
FileName = #"C:\Program Files\wkhtmltopdf\wkhtmltopdf.exe",
UseShellExecute = false,
CreateNoWindow = true,
RedirectStandardInput = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
Arguments = "-q -n " + additionalArgs + " - -";
};
using (var p = Process.Start(psi))
using (var pdfSream = new MemoryStream())
using (var utf8Writer = new StreamWriter(p.StandardInput.BaseStream,
Encoding.UTF8))
{
await utf8Writer.WriteAsync(html);
utf8Writer.Close();
var tStdOut = p.StandardOutput.BaseStream.CopyToAsync(pdfSream);
var tStdError = p.StandardError.ReadToEndAsync();
await tStandardOutput;
string errors = await tStandardError;
if (!string.IsNullOrEmpty(errors)) { /* deal/log with errors */ }
return pdfSream.ToArray();
}
}
Things I haven't included in there but could be useful if you have images, css or other stuff that wkhtmltopdf will have to load when rendering the html page:
you can pass the authentication cookie using --cookie
in the header of the html page, you can set the base tag with href pointing to the server and wkhtmltopdf will use that if need be
Thanks for the question / answer / all the comments above. I came upon this when I was writing my own C# wrapper for WKHTMLtoPDF and it answered a couple of the problems I had. I ended up writing about this in a blog post - which also contains my wrapper (you'll no doubt see the "inspiration" from the entries above seeping into my code...)
Making PDFs from HTML in C# using WKHTMLtoPDF
Thanks again guys!
The ASP .Net process probably doesn't have write access to the directory.
Try telling it to write to %TEMP%, and see if it works.
Also, make your ASP .Net page echo the process's stdout and stderr, and check for error messages.
Generally return code =0 is coming if the pdf file is created properly and correctly.If it's not created then the value is in -ve range.
using System;
using System.Diagnostics;
using System.Web;
public partial class pdftest : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
}
private void fn_test()
{
try
{
string url = HttpContext.Current.Request.Url.AbsoluteUri;
Response.Write(url);
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.FileName =
#"C:\PROGRA~1\WKHTML~1\wkhtmltopdf.exe";//"wkhtmltopdf.exe";
startInfo.Arguments = url + #" C:\test"
+ Guid.NewGuid().ToString() + ".pdf";
Process.Start(startInfo);
}
catch (Exception ex)
{
string xx = ex.Message.ToString();
Response.Write("<br>" + xx);
}
}
protected void btn_test_Click(object sender, EventArgs e)
{
fn_test();
}
}