I have a small automation code that works on a website, using puppeteer, chrome headless.
The code is very simple.
locate an element.
click on it.
the thing is that the click events are SUPER slow, I assume it should take less than a few ms.
but it takes 90-400ms to perform a single click event.
there is no need to scroll because all the elements appear on the screen.
here is a piece of code:
logger.verbose(`before clicking on row`);
// get by link text
// search for any element with this class and text
let workElement = await page.$x(`//span[contains(#class,"tr-class") and contains(text(),"${jobSearchId}")]`);
//we don't want to click on the element itself because it makes things more difficult
let row = await (await workElement.getProperty("parentNode")).getProperty("parentNode");
await row.click();
logger.verbose(`row clicked`);
//click on the button
logger.verbose(`button click before`);
await page.click(".searchJobDescription");
logger.verbose(`button clicked`);
0|main | 2022-08-17 13:09:05.283 - verbose: before clicking on row
0|main | 2022-08-17 13:09:05.685 - verbose: row clicked
0|main | 2022-08-17 13:09:05.685 - verbose: button click before
0|main | 2022-08-17 13:09:05.776 - verbose: button clicked
clicking on the row takes 402ms
and clicking the button takes 91ms
even a human can perform these actions much faster.
can anyone help me understand how to speed up these actions?
puppeteer is very slow and I don't know why.
So...
The solution is super weird and seems not related to the problem, but I have tested it again and again, and I'm 100% sure that this is the cause (or part of a bigger bug I can't see).
I have built a cookies file that stores the page cookies, when I am injecting the cookies back to the page, everything slows down, I am using the next method for it.
async function injectCookies() {
const cookiesList = getCookiesFromJarAsList();
await page.setCookie(...cookiesList);
}
when I am removing the call for injecting the cookies, puppeteer runs fast again. (~30ms per click)
async function injectCookies() {
const cookiesList = getCookiesFromJarAsList();
// removing this line makes Puppeteer great again
// await page.setCookie(...cookiesList);
}
I am not even sure that this is a puppeteer issue.
it seems like Chrome/Chromium problem because when I debugged the CDP, chrome itself returned very slow answers for the command "mouseMove" (which is part of Puppeteer click method).
not sure what's going on, so if someone smart along the way will be able to solve it, it'll be great.
For me the problem been solved and I wasted more then a week in debugging it, so I'm leaving it this way.
if someone can reproduce it, and open a bug for puppeteer/chromium guys, I am sending him very good Karma.
Related
hi im using laravel 8 and my software is pos ..
everything working so good but i have big problem ..
thats sometimes the cashier can refresh the page when the customer is gone without save the invoice
..
so what i did is this ..
document.addEventListener('keydown', (e) => {
e = e || window.event;
if(e.keyCode == 116)
{
var is_admin = $("#is_admin").val();
if(is_admin != 1)
{
e.preventDefault();
// this code here will not allow f5 to work
}
}
});
but the cashiers goes to the address bar and hit enter and like that he refresh the page
also sometimes they hit the refresh button beside url bar
so i start chrome in kisok mode in full screen
but the problem thats he can move the mouse to the top of the browser and the url bar will show again and he can do refresh page ..
so the solution for my problem is there any way to set password in chrome when refresh the page or close the chrome or is there any way to start chrome without close bar and url bar in kisok mode
thanks ..
Based on my research I didn't find anything related to setting a password in chrome for closing, instead, I have other solutions that may help you by considering this closing/refreshing issue happens accidentally.
#1st Solution - Closure Extension
https://chrome.google.com/webstore/detail/closure/jjagagcgljmlnihcilbpbfcglnopepjb
a very simple extension that works by locking the current browser tab. Click the toolbar icon or right-click on a page and select “Confirm Closure”. The favicon for the website in the current tab will turn into a padlock.
if the cashier clicked on the refresh button or the closing button a confirmation popup will show up.
#2nd Solution - Disable Close Button
checkout these 5 software that claims to prevent accidental closing of software by disabling the close button
https://www.raymond.cc/blog/prevent-program-closing-disabling-close-button/
#3rd Solution - Saving Draft.
The last solution I have is a workaround you can make by using
window.addEventListener('beforeunload', function (e) {
// saving current invoice in localstorage to be retrieved later
});
// check this answer
// https://stackoverflow.com/questions/13443503/run-javascript-code-on-window-close-or-page-refresh
beforeunload event, so you can save a draft of the current invoice in locale storage before closing the window, but you are should be very aware of how to manage these drafts, when to retrieve them, and when to clean them.
Also, you can use service workers if you choose this kind of solution.
Again, this all about if the cashier accidentally makes this behavior, which I think he must be aware of what he is doing, so you are making your validations as you can to prevent such behavior and make your system as robust as possible, I encourage you to think of this problem in a technical way then you should take the 3rd solution, and for the client just offer him the other 2 solutions which I see they will work well, otherwise, if the client wants to make something wrong in purpose then it will be his responsibility.
I'm navigating with Puppeteer around a React website.
Two sample lines of code:
await page.waitForSelector('a.btn-lg[data-target="#loginModal"]');
await page.click('a.btn-lg[data-target="#loginModal"]');
With a sufficient slowMo value, the effects are consistent - the button gets clicked every time.
However, without slowMo, sometimes the button does get clicked, and sometimes it doesn't (a window wired to it doesn't open).
It happens for a lot of elements, not just this one button in particular.
I just started using Puppeteer, and it looks like I'm either misusing the library, or the website somehow screws up my efforts.
Please tell me why sometimes the effects of clicking are visible and sometimes not, and how to remedy it.
UPDATE:
Code such as this does not work either.
await page.evaluate(() => (document.querySelector('span.pum-close') as any).click());
await page.$$eval('span.pum-close', elements =>
elements[0].click()
);
I downloaded the latest version of Puppeteer a couple weeks ago, so I'm new with it. The first thing I noticed is that
await this.page.waitForNavigation();
does not seem to work. If I run in not headless mode and debug, I can see the waitForNavigation() returns as soon as navigation starts, not finishes. Who cares when navigation starts? You can't do anything until navigation is complete.
How can I be sure a page is ready? Right now I have had to fill my code with lots of
await this.page.waitFor(SomeDelayMs);
Generally speaking, you're better off using:
await page.waitForSelector('your_selector')
That will cause puppeteer to wait until a specific selector is available before continuing execution.
You can also use something like this if you're dealing with something that only shows up once clicked:
await page.waitForSelector('your_selector', {visible: True})
open a new tab in Chrome (opens http://google.com for example)
open a testpage.htm containing history.pushState({},"test","#lightbox");
which changes the url to testpage.htm#lightbox
if you hit the back button of your chrome the url is changed to http://google.com, which is two states back, not one
firefox and msie10 work both good, so it's a chrome issue
how can i fix this? is there a workaround?
thank you in advance
(feel free to give my question a better title and feel free to correct my english)
notes:
on step 2 you can also use window.location.hash = "#lightbox" but does the same issue
on step 3 you can simulate the back button from within your code, using history.back() and in this very case the url is switched to the correct one testpage.htm, so this is related only to the back button of Chrome's gui
I also tried
window.addEventListener("popstate", function(ev){ ev.preventDefault(); });
window.addEventListener("hashchange", function(ev){ ev.preventDefault(); });
without success :(
Update 2: does the same using History.js
I am starting my internship on a Home Server able to control mutliple domotics equipments from a web page. The global idea is that based on a click on a button, a certain script is spawned on the server and controls a microcontroller.
My tutor built a simple website he gave me, using AJAX to always stay on 1 page, and brings the menus according to user actions (they are hidden if not used, brought back to front if used).
I have set up an apache server which I configured to execute CGI scripts, and it works.
To always stay on one page, I used the '204 No Content' return trick, so that the server's answer to the page is 'I don't have anything to say, just stay on this page'.
But the one problem I have is that the CGI is launched only once. If I click the button for the first time it works, afterwards nothing happens.
I tried using the SSI (shtml) to use the in a button code instead of using a FORM with GET method but the script still won't execute twice.
I might be using the wrong tools. Should I keep going with CGIs ? Or is there something else (like AJAX, jquery) that actually is designed to do what I want ?
Thanks for having read.
EDIT : I have found a way around it (it's always when I'm desperate after looking for days for an answer that I go to forums and then find myself a nice solution in the next hour .... )
I used a basic link, and for some reason it has a different behaviour than using a button. Whatever. My interrogations on the technologies used still stand though :)
EDIT2 : My solution is crappy, for some reason the script is also called at page refresh (or when the page loads for the first time). It's strange, because since it's in the it should only be spawned when I click on it ...
Familiarize yourself with jQuery and its AJAX API. You can't make it not load a new page unless you use AJAX. Here is an example of what an AJAX call looks like:
$.ajax({
url: 'http://server.domain.com/cgi-bin/myfile.cgi',
data: {
x: 1,
today: '20110504',
user: 'Joe'
}
}).success(function(data, status, xhr) {
if (data)
alert(data);
});
That is for jQuery 1.5 or higher. You can run that whenever a certain button is clicked like this:
HTML for the button:
<input type="button" id="doThis"/>
JavaScript:
$(function() {
$('#doThis').click(function() {
//put AJAX sample shown above, here
});
});