Apify pupeteer, don't wait for complete load of page - puppeteer

I'm having a problem with page which is taking too long to load (some scripts).
If i set handlePageTimeoutSecs or navigationTimeoutSecs parametres the request fails and goes back to the queue, is it possible to set to stop loading and parse data regardless of the complete of the page? Thank you for your help

Related

Node/Express server hangs intermittently but automatically continues after a pause

Unfortunately I cannot put the code here because despite all kinds of debugging I cannot point to the problem to present the problematic segment of the code. If I try to load a HTML which in turn has other requests (js,css,png, etc.), it loads all except one... The server/browser hangs for some time and then after about a minute or so does actually load! If I try to test the server with individual manual requests of these very URLs in the HTML file, it works fine.
While tying to load the HTML file, Chrome Network tab shows "pending..." of one request or sometimes two. But ultimately all the URLs requested are served. Thats what bugs me...
I tried to set http.globalAgent.maxSockets to 100 as suggested here as the HTML file would make more than 8 requests for the different js, css, etc. This did not help either.
I have reached a deadend. Any help would be appreciated.

Node.js/Express 4 - How to update a page to show progress in workflow

i built a small page to track a workflow with multiple steps. The page contains a form with submit button to start the flow. Each step sends back a status to the page. The page html is rerendered with res.render() then and sent to the server with res.send(html).
My problem now is, that when i use res.send(), the next status update gives a failure, because the response process is finished before.
Using res.write() instead crashes the page for whatever reason. I looks like the page is appended a second time (from a buffer??), instead of overwriting it.
Any ideas how to solve that??
Moe

How to delay MainPage, so it starts after data is loaded?

I have an application that stores data in isolated storage. I have a function to read the data (called in Apps.xaml.cs):
await ReadData();
However, before the data read, OnNavigatedTo is called in MainPage. At that point the buffer holding the data is still null.
How can I prevent MainPage.OnNavigateTo() from being as long as ReadData is not complete?
I tried syncing Apps & MainPage by setting a semaphore in Apps, and WaitOne() in the MainPage constructor. This hang the app on the WaitOne.
Any idea how this timing problem can be solved?
You can't delay the loading of the main page - as if the loading cycle is not executed within a timely fashion the app will be closed by the OS.
The best way around this is to create an interim splash page, which will handle the downloading of data whilst showing UI to the user. Once the data has downloaded you can then redirect to your main application page.

How long is the delay before the thumbnail url is available on a new file?

I'm inserting a new file, and using the returned File object to store a thumbnail. Intermittently, getThumbnail() returns null for .pdf files.
I'm guessing that the explanation is the thumbnail is generated asynchronously and there are times when the processing is incomplete before the insert() call returns with anincomplete File object.
Is there any way I can make this behave more deterministically?
Alternatively, anybody know if the subsequent processing of the thumbnail constitutes a "change" that would be returned by a get changes call?
AFAIK yes the thumbnails are calculated asynchronously. The delay can be different based on server loads, file type and file size but in my testing the thumbs for PDF were available very shortly after the file has been created.
Probably at this point the best you could do is try a subsequent request and keep trying until you get a thumbnail but don't forget to use exponential back-off not to overload the server and kill your quota in some case.
I don't think that when the thumbnail is ready this counts as a change in the changes feed in that case.

Trigger a web page refresh

I am working on an android application that will show an html page that contains only some text on a tablet device. The device will be on and showing this page for long periods of time(several hours). The text on this page will get changed from time to time.
To change the text on the page I've made a separate second page that contains a form to enter the new strings into and a submit button that uses ASP to generate a new version of the first page and save it over top of the original copy. This is set up and working great, but it means that I have to refresh the page very frequently in order to ensure I am always showing the latest message.
I am looking for a way that I could trigger a refresh only when a new message is saved. That way I will not have to refresh the page every minute but the new message will still get shown in a timely manner.
No dice, HTTP is built as a stateless, pull-only (ignoring file uploads) protocol. The server can't push data to the client, the client has to actually poll the server for new information.
However, you can minimize the overhead of this by using an AJAX call with JSON as the transport protocol instead of generating entire web pages and update your page on the client side. The overhead should be minimal for almost any application.
If you were just a web-app, I would suggest looking into the various Comet frameworks.
http://www.google.com/search?q=comet+framework
But, since you have an Android shell around it, you can make a Socket connection back to your server and have the server signal when it's time to refresh. It's essentially the same, but you don't need to code up the push in JavaScript if you're more comfortable in Java.