For some reason I'm not able to click on an element that appears on a screen with puppeteer js.
Here is the code:
const getAllElements = await page.$$('._1Nk0C');
for (let [i, link] of getAllElements.entries()) {
try {
await link.click();
await sleep.sleep(4);
await link.click('._1NHYN _3d86A Ddtb4');
} catch (e) {
console.error(e);
}
}
Here I find all elements with '._1Nk0C'
It then clicks on the element which as it enlarge in forefront. await link.click();
I then try to click the button on screen. I can confirm this is on the screen.
await link.click('._1NHYN _3d86A Ddtb4');
Nothing happens. It doesn't error out just doesn't click on element. Am I missing something?
elementHandle.click([options]) does not accept a selector as an argument. If you're trying to click on an element in the page based on its selector try:
await link.click();
await sleep.sleep(4);
await page.click(selector);
Related
My Html code has Button-tags that have same id "hoge".
If you get the selector from the Chrome Dev Tool, it will be the same for both "#hoge".
<html>
<body>
<button id="hoge">Hoge</button>
<div class="shadow">
#shadow-root (open)
<button id="hoge">Hoge</button>
</div>
</body>
</html>
I want to get element of button-tag in shadow dom with puppeteer.
But, my javascript code gets element of 1st button.
const element = page.waitForSelector("pierce/#hoge");
This is not what I want.
I'm guessing it's because you didn't specify a unique selector, but i don't know what is unique selector for puppeteer.
If you know how to solve this problem, please let me know.
Long story short
I work with puppeteer a lot and wanted this knowlegde to be in my bag. One way to select a shadow Element is by accessing the parent DOM Node's shadowRoot property. The answer is based on this article.
Accessing Shadow Root property
For your html example this does the trick:
const button = document.querySelector('.shadow').shadowRoot.querySelector('#hoge')
waiting
Waiting though is a little more complicated but can be acquired using page.waitForFunction().
Working Sandbox
I wrote this full working sandbox example on how to wait for a certain shadowRoot element.
index.html (located in same directory as app.js)
<html>
<head>
<script>
// attach shadowRoot after 6 seconds for emulating waiting..
setTimeout(() => {
const btn = document.getElementById('hoge')
const container = document.getElementsByClassName('shadow')[0]
const shadowRoot = container.attachShadow({
mode: 'open'
})
shadowRoot.innerHTML = `<button id="hoge" onClick="doStuff()">hoge2</button>`
console.log('attached!.')
}, 6000)
function doStuff() {
alert('shadow button clicked!')
}
</script>
</head>
<body>
<button id="hoge">Hoge</button>
<div class="shadow">
</div>
</body>
</html>
app.js (located in same directory as index.html)
var express = require('express')
var { join } = require('path')
var puppeteer = require('puppeteer')
//utility..
const wait = (seconds) => {
console.log('waiting', seconds, 'seconds')
return new Promise((res, rej) => {
setTimeout(res, seconds * 1000)
})
}
const runPuppeteer = async() => {
const browser = await puppeteer.launch({
defaultViewport: null,
headless: false
})
const page = await browser.newPage()
await page.goto('http://127.0.0.1:5000')
await wait(3)
console.log('page opened..')
// only execute this function within a page context!.
// for example in page.evaluate() OR page.waitForFunction etc.
// don't forget to pass the selector args to the page context function!
const selectShadowElement = (containerSelector, elementSelector) => {
try {
// get the container
const container = document.querySelector(containerSelector)
// Here's the important part, select the shadow by the parentnode of the
// actual shadow root and search within the shadowroot which is like another DOM!,
return container.shadowRoot.querySelector(elementSelector)
} catch (err) {
return null
}
}
console.log('waiting for shadow elemetn now.')
const containerSelector = '.shadow'
const elementSelector = '#hoge'
const result = await page.waitForFunction(selectShadowElement, { timeout: 15 * 1000 }, containerSelector, elementSelector)
if (!result) {
console.error('Shadow element not found..')
return
}
// since waiting succeeded we can get the elemtn now.
const element = await page.evaluateHandle(selectShadowElement, containerSelector, elementSelector)
try {
// click the element.
await element.click()
console.log('clicked')
} catch (err) {
console.log('failed to click..')
}
await wait(10)
}
var app = express()
app.get('/', (req, res) => {
res.sendFile(join(__dirname, 'index.html'))
})
app.listen(5000, '127.0.0.1', () => {
console.log('listening!')
runPuppeteer()
})
Start example
$ npm i express puppeteer
$ node app.js
Make sure to use headless:false option to see what's happening.
The application does this:
start a small express server only serving index.html on /
open puppeteer after server has started and wait for the shadow root element to appear.
Once it appeared, it gets clicked and an alert() is shown. => success!
Browser Support
Tested with chrome.
Cheers ' ^^
How can I make a function that waits until a certain CSS selector loads in puppeteer?
I want to refresh the page over and over until the '.product-form__add-to-cart' is present, then I want it to continue on with the code.
The corresponding puppeteer method is page.waitForSelector.
Example:
await page.waitForSelector('.product-form__add-to-cart')
But if you need to reload the page to get the desired element (maybe because the site you are visiting can look different between visits) than you can just check if the element is present in the DOM after the page is rendered and if not: then you can try to page.reload it.
await page.goto(url, { waitUntil: 'domcontentloaded' })
const selectorExists = await page.$('.product-form__add-to-cart')
if (selectorExists !== null) {
// do something with the element
} else {
await page.reload({ waitUntil: 'domcontentloaded' })
}
You can do it in a loop as well, where you break if the selector appears, but be careful: if it never shows up you will end up in an endless loop! Set a maximum number of tries.
Edit
If you are sure about running it in a loop until the selector appears: then you can try with a while loop:
await page.goto(url, { waitUntil: 'domcontentloaded' })
let selectorExists = await page.$('.product-form__add-to-cart')
while (selectorExists === null) {
await page.reload({ waitUntil: 'domcontentloaded' })
selectorExists = await page.$('.product-form__add-to-cart')
}
// if condition meets the script will go on
As I said earlier: you can end up with an endless loop like this, so be careful.
I have the following element in html.
<a title="Download photo" href="https://example.com/photos/GXqvtQh1N9A/download?force=true" rel="nofollow" download="" target="_blank" class="_1QwHQ _1l4Hh _1CBrG _1zIyn xLon9 _1Tfeo _2L6Ut _2Xklx"><svg class="Apljk _11dQc" version="1.1" viewBox="0 0 32 32" width="32" height="32" aria-hidden="false"></a>
From the console when Chromium is open.
I can query it like so:
document.querySelector('a[title="Download photo"]');
I can create a reference to it:
var link = document.querySelector('a[title="Download photo"]');
I then can click on it like so:
link.click();
I try the same exact thing in Puppeteer.js in code. Same page.
for (const handle of getAllElements) {
try {
await handle.click();
const downloadButton = await page.$('a[title="Download photo"]');
downloadButton.click();
await sleep.sleep(2000);
} catch (e) {
console.error(e);
}
}
The initial handle.click() works and it opens me to the page I'm discussing here.
But then downloadButton.click() doesn't function.
I've also tried page.click(downloadButton).
I've also tried:
const downloadButton = await page.$('a[title="Download photo"]');
await downloadButton.click();
To ensure I'm working with the same page I visually do it while the page is on the screen.
Any ideas what's gong on?
As you mentioned it opens a layer on top each time you click on the image. Also, a[title="Download photo"] needs to be relative to the handle not page. Here is the working code:
for (const handle of getAllElements) {
await handle.click();
await handle.$eval('a[title="Download photo"]', el => el.click());
//allow download
await page._client.send('Page.setDownloadBehavior', {
behavior: 'allow',
downloadPath: './'
});
await new Promise(resolve => setTimeout(resolve, 2000));
//click on X to close the layer
await page.click('._1NHYN');
}
I am using puppeteer to scrape a page, there is a simple form for searching on the page
I enter a search text in one input, the others are empty, however, when i click on the search, the others input are not empty, what' s wrong with it ?
await page.goto('http://www.dollmedia-btp.com/annuaire/', {waitUntil: 'domcontentloaded'});
await page.type("input#input-recherche-activite", "Maçonnerie");
await page.type("input#input-recherche-raison-sociale", "");
await page.type("input#input-recherche-ville","");
await page.type("input#input-recherche-tel", "");
I don' t know what to do, thanks for your helps
You have to clear the fields, the type method with an empty string does not remove the input's value.
Custom function:
async function clear(page, selector) {
await page.evaluate(selector => {
document.querySelector(selector).value = "";
}, selector);
}
Usage:
await clear(page,"input#input-recherche-raison-sociale");
Or without creating function:
await page.$eval('#input-recherche-raison-sociale', el => el.value = '');
I am having issue with puppeteer.
I want to delete the added item to the form. For example, I have a form and added some fake data ("example"). I want to delete this "example", it doesn't matter whatever position it located. I just only want to delete this "example".
So, it means, puppeteer adds it and will delete in the next step.
I have tried:
// fake data
const metadatatest = {
text: 'example,
}
describe('Should be navigate through details', () => {
it('can navigate through detail', async () => {
// this adds fake data successfully
await page.waitForSelector('[data-testid="appCard"]')
await page.click('[data-testid="appCardDetails"]')
await page.waitForSelector('[data-testid="overviewSectionMetadataForm"]')
await page.click('[data-testid="overviewSectionMetadataEditButton"]')
//await page.$eval('[data-testid="metadataInput"]', el => el.value = 'example')
await page.type('[data-testid="metadataInput"]', metadatatest.text)
await page.waitForSelector('[data-testid="metadataInput"]')
await Promise.all([
page.click('[data-testid="overviewSectionMetadataEditButton"]'),
]);
// I want to delete this
})
})
I have also tried using
await page.keyboard.press('Backspace')
await page.keyboard.press('Clear')
await page.keyboard.press('Delete')
but no luck.
any help please!
So what you're asking is about clearing text from an input field, am I reading that correctly? Puppeteer doesn't have a built in method for that but I have found a workaround which will do it for you.
First, you need to click 3 times on the input field you wish to clear. This acts as a select all action for all text entered in that element:
await page.click(selector, { clickCount: 3 });
Now you can use your previous attempt to clear the text:
await page.keyboard.press('Backspace');
Update 1:
Your final code for clearing and then entering the text you want into the input field should look something like this:
await page.click('[data-testid="metadataInput"]', { clickCount: 3 });
await page.keyboard.press('Backspace');
await page.type('[data-testid="metadataInput"]', metadatatest.text);