I have an iFrame with several items and i want to update a record.
const frame = await page.frames()[1];
const P1_CUSTOM_NAME = await frame.$('#P1_CUSTOM_NAME');
await P1_CUSTOM_NAME.type('MyFavoritCustomer', {delay: 20});
This will not overwrite the field P1_CUSTOM_NAME. Unfortunately it appends the value 'MyFavoritCustomer'.
Any suggestions how i can clear an item value?
Try a triple click to focus and select all.
const frame = await page.frames()[1];
const P1_CUSTOM_NAME = await frame.$('#P1_CUSTOM_NAME');
await P1_CUSTOM_NAME.click({clickCount: 3});
await P1_CUSTOM_NAME.type('MyFavoritCustomer', {delay: 20});
Related
I have code that will login to a page, navigate to a list of messages, get the first message, and delete it. I need to be able to get the list of messages and delete each in turn. When I try to do that, I run into problems.
The site is rendered as plain html until the delete button is clicked. At this point, an iframe opens with the delete confirmation inside of it. If the confirmation is clicked, it returns me to the list of messages.
This is working until the iframe pops up. The existing code doesn't find the selector in the iframe. The code does work when it is not in a loop, though. So how can I interact with the iframe in the loop?
TimeoutError: Waiting for selector .navigation-footer button failed: Waiting failed: 30000ms exceeded
const messageList = await page.$$(".message-list tr");
for (message of messageList) {
//get first message
await page.click(".message-list tr");
//wait for the message to load
await page.waitForSelector(".circle-cross");
//get time and message text
const msgTime = await page.$eval("time", el => el.getAttribute("dateTime"));
const paragraphs = await page.evaluate(() => {
let paraElements = document.querySelectorAll(".bubble p");
//array literal
const paraList = [...paraElements];
//gets the innerText of each element
return paraList.map((el, index) => el.innerText);
});
//get author name
await page.waitForSelector(".user p a")
let authorLink = await page.$(".user p a")
let authorName = await authorLink.evaluate(el => el.innerText.trim());
//append message to messages.txt
const stream = fs.createWriteStream("messages.txt", { flags: 'a' });
stream.write(authorName + "\n");
stream.write(msgTime + "\n");
paragraphs.forEach((item, index) => {
stream.write(item + "\n");
});
stream.end();
//delete the message
await page.click(".circle-cross");
//handle the iframe verification
const elementHandle = await page.waitForSelector("iframe.fancybox-iframe");
const frame = await elementHandle.contentFrame();
const button = await frame.waitForSelector(".navigation-footer button");
await frame.click(".navigation-footer button");
}
UPDATE: I did get this working, eventually, by substituting in:
await button.evaluate(el => el.click());
instead of
await frame.click(".navigation-footer button")
Simply put the code you have in the forEach loop, however you will have to add async on this line:
messageList.forEach(async (el) => {
Thus your ending result should then be:
//get a list of messages
const messageList = await page.$$(".message-list tr");
messageList.forEach(async (el) => {
//get the first message
await page.click(".message-list tr");
//wait for the message to load
await page.waitForSelector(".circle-cross");
//delete the message
await page.click(".circle-cross");
//handle the iframe verification
const elementHandle = await page.waitForSelector("iframe");
const = await elementHandle.contentFrame();
await frame.waitForSelector(".navigation-footer button");
await frame.click(".navigation-footer button");
}
I'm wondering if it's possible within puppeteer to access a FileSystemDirectoryHandle (from the File System Access API). I would like to pass in a directory path via puppeteer as though the user had selected a directory via window.showDirectoryPicker(). On my client page I use the File System Access API to write a series of png files taken from a canvas element, like:
const directoryHandle = await window.showDirectoryPicker();
for (let frame = 0; frame < totalFrames; frame++){
const fileHandle = await directoryHandle.getFileHandle(`${frame}.png`, { create: true });
const writable = await fileHandle.createWritable();
updateCanvas(); // <--- update the contents of my canvas element
const blob = await new Promise((resolve) => canvas.toBlob(resolve, 'image/png'));
await writable.write(blob);
await writable.close();
}
On the puppeteer side, I want to mimic that behavior with something like:
const page = await browser.newPage();
await page.goto("localhost:3333/canvasRenderer.html");
// --- this part doesn't seem to exist ---
const [dirChooser] = await Promise.all([
page.waitForDirectoryChooser(),
page.click('#choose-directory'),
]);
await dirChooser.accept(['save/frames/here']);
//--------------------------------------
but waitForDirectoryChooser() doesn't exist.
I'd really appreciate any ideas or insights on how I might accomplish this!
Is it possible to calculate the TTFB with Puppeteer?
I couldn't find anything in their docs.
I currently have this code:
const browser = await puppeteer.launch(launchOptions);
const page = await browser.newPage();
const response = await page.goto(url);
const { status } = response;
This might do what you want:
let start = new Date()
page.once('response', () => console.log(new Date() - start))
await page.goto(url)
This is how I solved it:
const browser = await puppeteer.launch(launchOptions);
const page = await browser.newPage();
await page.goto(url);
const navigationTimingJson = await page.evaluate(() =>
JSON.stringify(performance.getEntriesByType("navigation"))
);
const [navigationTiming] = JSON.parse(navigationTimingJson)
const TTFB = navigationTiming.responseStart - navigationTiming.requestStart;
You can try using page.metrics()
More info here: https://pptr.dev/#?product=Puppeteer&version=v13.1.3&show=api-pagemetrics
Recently I am learning puppeteer using their docs and try to scrape some information.
First approach
First I collect a list of url from the mainpage. Second I create a new tab and go those url iterately and collect some data. I doubt when I enter the loop the new tab didn't work as I expect and freezed without giving any data. Eventually I got a error TimeoutError: Navigation timeout of 30000 ms exceeded. Is there any better approach?
(async () => {
const browser = await puppeteer.launch({ headless: true });
const mainpage = await browser.newPage();
console.log('goto main page'.green);
await mainpage.goto(mainURL);
console.log('collecting some url'.green);
const URLS = await mainpage.evaluate(() =>
Array.from(
document.querySelectorAll('.result-actions a'),
(element) => element.href
)
);
if (typeof URLS[0] === 'string') console.log('OK'.green);
console.log('collecting finished'.green);
const newTab= await browser.newPage();
console.log('create new tab'.green);
var data = [];
for (let i = 0, n = URLS.length; i < n; i++) {
//console.log(URLS[i]);
// use this new tab to collect some data then close this tab
// continue this process
await newTab.waitForNavigation();
await newTab.goto(URLS[i]);
await newTab.waitForSelector('.profile-phone-column span a');
console.log('Go each url using new tab'.green);
// collecting data
data.push(collected_data);
// close this tab
await collectNamePage.close();
console.log(data);
}
await mainpage.close();
await browser.close();
console.log('closing browser'.green);
})();
Second approach
This time I want to skip the part where I collect those data using a new tab. Hence I collect my urls using page.$$() and try to iterating using for...of over urls and collect my data using elementHandle.$(selector) but this approach also failed.
I am getting frustrated. Am I doing it wrong way or I didn't understand their documentation?
In your script, you do not need newTab.waitForNavigation(); at all. Usually, this is used when the navigation is caused by some event. When you just use .goto(), the page loading is waited automatically.
Even if you need waitForNavigation(), you usually do not await it before the navigation triggered, otherwise you just get the timeout. You await it with navigation trigger together:
await Promise.all([element.click(), page.waitForNavigation()]);
So try to just delete await newTab.waitForNavigation();.
Also, do not close the new tab in the loop, delete it after the loop.
Edited script:
const puppeteer = require('puppeteer');
const mainURL = 'https://www.psychologytoday.com/us/therapists/illinois/';
(async () => {
const browser = await puppeteer.launch({ headless: false });
const mainpage = await browser.newPage();
console.log('goto main page');
await mainpage.goto(mainURL);
console.log('collecting urls');
const URLS = await mainpage.evaluate(() =>
Array.from(
document.querySelectorAll('.result-actions a'),
(element) => element.href
)
);
if (typeof URLS[0] === 'string') console.log('OK');
console.log('collection finished');
const collectNamePage = await browser.newPage();
console.log('create new tab');
var data = [];
for (let i = 0, totalUrls = URLS.length; i < totalUrls; i++) {
console.log(URLS[i]);
await collectNamePage.goto(URLS[i]);
await collectNamePage.waitForSelector('.profile-phone-column span a');
console.log('create new tab and go there');
// collecting data
const [name, phone] = await collectNamePage.evaluate(
() => [
document.querySelector('.profile-middle .name-title-column h1').innerText,
document.querySelector('.profile-phone-column span a').innerText
]
);
data.push({ name, phone });
}
console.log(data);
await collectNamePage.close();
await mainpage.close();
await browser.close();
console.log('closing browser');
})();
In the following example how do I wait for the pop up window to finish loading?
After clikcing the google icon you get a pop up window to login to gmail, when I try to interact
with the second page it is undefined (as I don't know how to wait for it to fully load.
Any advice?
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless: false});
page = await browser.newPage();
await page.goto("https://www.example.com/signin");
await page.waitForSelector(".Icon-google");
await page.click(".Icon-google");
const pages = await browser.pages();
console.log(pages[2].url());
})();
You can wait for a new target to be created.
const browser = await puppeteer.launch({headless: false});
page = await browser.newPage();
await page.goto("https://app.testim.io/#/signin");
await page.waitForSelector(".Icon-google");
const nav = new Promise(res => browser.on('targetcreated', res))
await page.click(".Icon-google");
await nav
const pages = await browser.pages();
console.log(pages.length);//number of pages increases !
console.log(pages.map(page => page.url()));
P.S. first I tried page.waitForNavigation() but it didn't work, probably because it's a popup.
const [newPage] = await Promise.all([
new Promise((resolve) => page.once('popup', resolve)),
page.click('something.that-will-open-the-popup')
]);
await newPage.waitForSelector('.page-is-loaded')