I'm navigating with Puppeteer around a React website.
Two sample lines of code:
await page.waitForSelector('a.btn-lg[data-target="#loginModal"]');
await page.click('a.btn-lg[data-target="#loginModal"]');
With a sufficient slowMo value, the effects are consistent - the button gets clicked every time.
However, without slowMo, sometimes the button does get clicked, and sometimes it doesn't (a window wired to it doesn't open).
It happens for a lot of elements, not just this one button in particular.
I just started using Puppeteer, and it looks like I'm either misusing the library, or the website somehow screws up my efforts.
Please tell me why sometimes the effects of clicking are visible and sometimes not, and how to remedy it.
UPDATE:
Code such as this does not work either.
await page.evaluate(() => (document.querySelector('span.pum-close') as any).click());
await page.$$eval('span.pum-close', elements =>
elements[0].click()
);
Related
I have a small automation code that works on a website, using puppeteer, chrome headless.
The code is very simple.
locate an element.
click on it.
the thing is that the click events are SUPER slow, I assume it should take less than a few ms.
but it takes 90-400ms to perform a single click event.
there is no need to scroll because all the elements appear on the screen.
here is a piece of code:
logger.verbose(`before clicking on row`);
// get by link text
// search for any element with this class and text
let workElement = await page.$x(`//span[contains(#class,"tr-class") and contains(text(),"${jobSearchId}")]`);
//we don't want to click on the element itself because it makes things more difficult
let row = await (await workElement.getProperty("parentNode")).getProperty("parentNode");
await row.click();
logger.verbose(`row clicked`);
//click on the button
logger.verbose(`button click before`);
await page.click(".searchJobDescription");
logger.verbose(`button clicked`);
0|main | 2022-08-17 13:09:05.283 - verbose: before clicking on row
0|main | 2022-08-17 13:09:05.685 - verbose: row clicked
0|main | 2022-08-17 13:09:05.685 - verbose: button click before
0|main | 2022-08-17 13:09:05.776 - verbose: button clicked
clicking on the row takes 402ms
and clicking the button takes 91ms
even a human can perform these actions much faster.
can anyone help me understand how to speed up these actions?
puppeteer is very slow and I don't know why.
So...
The solution is super weird and seems not related to the problem, but I have tested it again and again, and I'm 100% sure that this is the cause (or part of a bigger bug I can't see).
I have built a cookies file that stores the page cookies, when I am injecting the cookies back to the page, everything slows down, I am using the next method for it.
async function injectCookies() {
const cookiesList = getCookiesFromJarAsList();
await page.setCookie(...cookiesList);
}
when I am removing the call for injecting the cookies, puppeteer runs fast again. (~30ms per click)
async function injectCookies() {
const cookiesList = getCookiesFromJarAsList();
// removing this line makes Puppeteer great again
// await page.setCookie(...cookiesList);
}
I am not even sure that this is a puppeteer issue.
it seems like Chrome/Chromium problem because when I debugged the CDP, chrome itself returned very slow answers for the command "mouseMove" (which is part of Puppeteer click method).
not sure what's going on, so if someone smart along the way will be able to solve it, it'll be great.
For me the problem been solved and I wasted more then a week in debugging it, so I'm leaving it this way.
if someone can reproduce it, and open a bug for puppeteer/chromium guys, I am sending him very good Karma.
A similar question may have been asked, but despite spending a couple of hours I could not find a satisfying answer.
I would like to call Frame.click(). At the time of calling it - I want to make sure that no navigation is pending. I also don't know if the element I am about to click on will result in navigation (it's passed in dynamically).
I tried https://www.npmjs.com/package/pending-xhr-puppeteer, but that seems to no longer be supported. (still uses Request instead of HTTPRequest)
Right now I am resorting to page.WaitForNetworkIdle(). Is this the best I can do?
P.S. It seems page.WaitForNavigation used to return right away if no navigation was taking place (on a year old version of the code). Since I updated to 12.1 it started waiting for the timeout and then throwing an exception if no navigation started within that timeframe.
I think you're confusing navigation and requests/responses. Navigation can be finished but requests/responses can still happened (eg: .aspx requests).
To make sure the navigation has happened, you could wrap it in a promise via Promise.all() method.
await Promise.all([
page.click(`#submit`),
page.waitForNavigation(),
]);
Additionally you could await page.content(); to make sure the full HTML contents of the page, including the doctype.
In the case of an ajax request you can use page.waitForResponse():
await Promise.all([
page.click(`#submit`),
page.waitForResponse(response => response.status() === 200),
]);
I have a button nested in this video player that I am trying to simply click. Any idea how do i select?
await page.waitForSelector('.icon-play');
await page.click('.icon-play');
await page.waitFor(6000);
}
<a tabindex="-1" href="#" role="button" class="icon-play comp largePlayBtn largePlayBtnBorder" aria-label="Play clip" data-order="1" data-plugin-name="largePlayBtn" style="display: block;"></a>
I had the same problem a while back. A nested link inside a image.
This is due to the fact that the element needs to be visually clickable. You can perform a simple javascript click action using the HTMLElement.click() to bypass the puppeteer click action.
page.$eval(`HTMLElementSelector`, element =>
element.click()
);
As you're clicking a link, which implies navigation you would want to englobe that inside a promise. Something like that should do the trick.
await Promise.all([
page.$eval(`HTMLElementSelector`, element =>
element.click()
),
await page.waitForNavigation(),
]);
this answer is to make a clear understanding of why the puppeteer's click doesn't work sometimes.
Puppeteer's API has different semantics from the native browser API.
Puppeteer's page.click() seems like a straightforward wrapper on the browser's native HTMLElement.click(), but it actually operates quite differently.
working of page.click()
when we click using page.click() instead of invoking the click event handler directly on the element as the native HTMLElement.click() does
Puppeteer scrolls the element into view
moves the mouse onto the element
presses one of a few mouse buttons
optionally triggers a delay
then releases the mouse button
You can also trigger multiple clicks. In other words, Puppeteer performs a click like a human would.
That's why when we click page.click() it clicks at (x,y) position of the screen as a human would. So, sometimes we don't see expected results as it doesn't show any error and it shouldn't, we think that page.click() is not working.
So, the easiest solution is to use page.evaluate() and click with native browser API.
Solution
await page.evaluate(() => {
document.querySelector('selector').click();
});
Below is a screenshot of a webpage that I am trying to write a Cypress test for.
As you can see, I have managed to write "teladoc" into the input box, but I now need to click the dropdown menu to navigate to a different page.
I am not able to get the ID, etc. of the dropdown menu.
When I try to inspect the dropdown menu, the page reloads and the dropdown disappears.
Does someone know how I can inspect this? I tried through the cypress explorer too, but it reloads in that beforehand too.
Since the locator is not there, You can play around with Keypress. The idea is when you have typed and there is the suggested list, you can first press the down key and then enter key. This might result in flaky tests as you don't know how much time does it take for the suggested item to appear.
cy.get(selector).type('{downarrow}{enter}')
You can also directly use contains. If this works this way the tests won't be flaky. You can play around with the timeout value.
cy.contains('Teladoc Health Inc - United States', {timeout: 5000})
.should('be.visible')
.click()
Open the browser console and type this
setTimeout(() => { debugger; }, 5000)
I downloaded the latest version of Puppeteer a couple weeks ago, so I'm new with it. The first thing I noticed is that
await this.page.waitForNavigation();
does not seem to work. If I run in not headless mode and debug, I can see the waitForNavigation() returns as soon as navigation starts, not finishes. Who cares when navigation starts? You can't do anything until navigation is complete.
How can I be sure a page is ready? Right now I have had to fill my code with lots of
await this.page.waitFor(SomeDelayMs);
Generally speaking, you're better off using:
await page.waitForSelector('your_selector')
That will cause puppeteer to wait until a specific selector is available before continuing execution.
You can also use something like this if you're dealing with something that only shows up once clicked:
await page.waitForSelector('your_selector', {visible: True})