Does page.goto(url, {waitUntil: 'domcontentloaded'}) actually wait until Javascript has finished manipulating the DOM? Like for instance on a SPA? If not, how do I find out when the DOM is ready?
To wait for 'networkidle2' seems rather inefficient...
Thx!
Related
I am working on html2canvas with vue based app
html2canvas(document.querySelector("#interactivecanvas")).then(canvas => {
//document.body.appendChild(canvas)
let image_data = canvas.toDataURL("image/png");
image_data = image_data.replace('data:image/png;base64,', '');
});
The code above is working, however when the code is executing the vue app will get stuck which I can't manipulate other UI components unless the html2canvas finishes the process. Additionally I always have to wait a while for the process to complete and then upload the data to server, which is too tedious and time-consuming.
How would I bypass the problem, like using another worker to process it??
Moreover, I am wondering is it possibl to process it in the background with a queue of tasks because I don't want to manually repeating the process of waiting and updating?
I am using pyppeteer to trigger headless chrome and perform some actions. But first I want all the elements of the web page to load completely. The official documentation of pyppeteer suggests a waitUntil parameter which comes with more than 1 parameters.
My doubt is do i have to pass all the parameters or any one in particular is sufficient? Please suggest if following snippet helps in my case?
await page.goto(url, {'waitUntil' : ['load', 'domcontentloaded', 'networkidle0', 'networkidle2']})
No, you don't have to pass all possible options to 'waitUntil'. You can pick either of them, or more options at the same time if you like, but if you are:
not deailing with a single-page app,
not interested in all network connections (like 3rd party trackings for example)
then you are good to go with: 'domcontentloaded' to wait for all the elements to be rendered on the page.
await page.goto(url, {'waitUntil' : 'domcontentloaded'})
The options in details:
load: when load event is fired.
domcontentloaded: when the DOMContentLoaded event is fired.
networkidle0: when there are no more than 0 network connections
for at least 500 ms.
networkidle2: when there are no more than 2 network connections
for at least 500 ms.
[source]
Note: of course it is true for the NodeJs puppeteer library as well, they work the same way in terms of waitUntil.
Is there a way to start the performance profiling programmatically in Chrome?
I want to run a performance test of my web app several times to get a better estimate of the FPS but manually starting the performance profiling in Chrome is tricky because I'd have to manually align the frame models. (I am using this technique to extract the frames)
CMD + Shift + E reloads the page and immediately starts the profiling, which alleviates the alignment problem but it only runs for 3 seconds as explained here. So this doesn't work.
Ideally, I'd like to click on a button to start my test script and also starts the profiling. Is there a way to achieve that?
in case you're still interested, or someone else may find it helpful, there's an easy way to achieve this using Puppeteer's tracing class.
Puppeteer uses Chrome DevTools Protocol's Tracing Domain under the hood, and writes a JSON file to your system that can be loaded in the dev tools performance panel.
To get a profile trace of your page's loading time you can implement the following:
const puppeteer = require('puppeteer');
(async () => {
// launch puppeteer browser in headful mode
browser = await puppeteer.launch({
headless: false,
devtools: true
});
// start a page instance in the browser
page = await browser.newPage();
// start the profiling, with a path to the out file and screenshots collected
await page.tracing.start({
path: `tests/logs/trace-${new Date().getTime()}.json`,
screenshots: true
});
// go to the page
await page.goto('http://localhost:8080');
// wait for as long as you want
await page.waitFor(4000);
// or you can wait for an element to appear with:
// await page.waitForSelector('some-css-selector');
// stop the tracing
await page.tracing.stop();
// close the browser
await browser.close();
})();
Of course, you'll have to install Puppeteer first (npm i puppeteer). If you don't want to use Puppeteer you can interact with Chrome DevTools Protocol's API directly (see link above). I didn't investigate that option very much since Puppeteer delivers a high level and easy to use API over CDP's API. You can also interact directly with CDP via Puppeteer's CDPSession API.
Hope this helps. Good luck!
You can use the chrome devtools protocol and use any driver library from here https://github.com/ChromeDevTools/awesome-chrome-devtools#protocol-driver-libraries to programmatically create a profile.
Use this method - https://chromedevtools.github.io/devtools-protocol/tot/Profiler#method-start to start a profile.
I'm running Chrome in headless mode with Puppeteer, and I discovered that if an URL I load contain a javascript code like:
while (true) {console.log('crash')}
The page will load forever even though I have timeout set in place and waitUntil defined:
await page.goto('http://...', {waitUntil: ['load', 'documentloaded', 'networkidle0'], 'timeout': timeout})
How can I ensure that no JS (or any other kind of) abuse don't stuck my code?
We are using puppeteer to run automated tests on hundreds of websites and URLs. Some of those websites are very slow and run into a timeout. That is often the case because there is an ad that does not finish loading. So increasing the timeout is not an option.
Is there a way to get the currently rendered HTML (DOM) at the moment the timeout is happening? page.content() is only returning a promise that is still pending.
You might be able to use something like evaluate, which injects a custom JavaScript function to execute. However, if the thread is truly "locked" then it'll likely run into the same issue.
const body = await page.evaluate(() => document.documentElement.outerHTML);
You might also need to be a little more flexible on how you orchestrate the script by catching goto timeouts and then trying the above.