I am using Cheerio to parse the html and then running the following:
+$("#price1").text()|| +$("#price2").text() || undefined
Basically it tries to get the prices if the first fails and finally returns undefined if all fails.
How can I do the same thing in Puppeteer? I am not sure how to use page.evaluate to execute this chain of if-else commands. (I am also using Typescript as well.)
It seems the analog would be something like this:
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch();
try {
const [page] = await browser.pages();
await page.goto('https://example.org/'); // your URL here
const price = await page.evaluate(() => {
const element = document.querySelector('#price1') ||
document.querySelector('#price2');
if (element) return Number(element.innerText);
return undefined;
});
console.log(price);
} catch(err) { console.error(err); } finally { await browser.close(); }
Related
I have a very simple Puppeteer script that uses exposeFunction() to run something inside headless Chrome.
(async function(){
var log = console.log.bind(console),
puppeteer = require('puppeteer');
const browser = await puppeteer.launch();
const page = await browser.newPage();
var functionToInject = function(){
return window.navigator.appName;
}
await page.exposeFunction('functionToInject', functionToInject);
var data = await page.evaluate(async function(){
console.log('woo I run inside a browser')
return await functionToInject();
});
console.log(data);
await browser.close();
})()
This fails with:
ReferenceError: window is not defined
Which refers to the injected function. How can I access window inside the headless Chrome?
I know I can do evaluate() instead, but this doesn't work with a function I pass dynamically:
(async function(){
var log = console.log.bind(console),
puppeteer = require('puppeteer');
const browser = await puppeteer.launch();
const page = await browser.newPage();
var data = await page.evaluate(async function(){
console.log('woo I run inside a browser')
return window.navigator.appName;
});
console.log(data);
await browser.close();
})()
evaluate the function
You can pass the dynamic script using evaluate.
(async function(){
var puppeteer = require('puppeteer');
const browser = await puppeteer.launch();
const page = await browser.newPage();
var functionToInject = function(){
return window.navigator.appName;
}
var data = await page.evaluate(functionToInject); // <-- Just pass the function
console.log(data); // outputs: Netscape
await browser.close();
})()
addScriptTag and readFileSync
You can save the function to a seperate file and use the function using addScriptTag.
await page.addScriptTag({path: 'my-script.js'});
or evaluate with readFileSync.
await page.evaluate(fs.readFileSync(filePath, 'utf8'));
or, pass a parameterized funciton as a string to page.evaluate.
await page.evaluate(new Function('foo', 'console.log(foo);'), {foo: 'bar'});
Make a new function dynamically
How about making it into a runnable function :D ?
function runnable(fn) {
return new Function("arguments", `return ${fn.toString()}(arguments)`);
}
The above will create a new function with provided arguments. We can pass any function we want.
Such as the following function with window, along with arguments,
function functionToInject() {
return window.location.href;
};
works flawlessly with promises too,
function functionToInject() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(window.location.href);
}, 5000);
});
}
and with arguments,
async function functionToInject(someargs) {
return someargs; // {bar: 'foo'}
};
Call the desired function with evaluate,
var data = await page.evaluate(runnable(functionToInject), {bar: "foo"});
console.log(data); // shows the location
exposeFunction() isn't the right tool for this job.
From the Puppeteer docs
page.exposeFunction(name, puppeteerFunction)
puppeteerFunction Callback function which will be called in Puppeteer's context.
'In puppeteer's context' is a little vague, but check out the docs for evaluate():
page.evaluateHandle(pageFunction, ...args)
pageFunction Function to be evaluated in the page context
exposeFunction() doesn't expose a function to run inside the page, but exposes a function to be be run in node to be called from the page.
I have to use evaluate():
You problem could be related to the fact that page.exposeFunction() will make your function return a Promise (requiring the use of async and await). This happens because your function will not be running inside your browser, but inside your nodejs application and its results are being send back and forth into/to the browser code. This is why you function passed to page.exposeFunction() is now returning a promise instead of the actual result. And it explains why the window function is not defined, because your function is running inside nodejs (not your browser) and inside nodejs there is no window definition available.
Related questions:
exposeFunction() does not work after goto()
exposed function queryseldtcor not working in puppeteer
How to use evaluateOnNewDocument and exposeFunction?
exposeFunction remains in memory?
Puppeteer: pass variable in .evaluate()
Puppeteer evaluate function
allow to pass a parameterized funciton as a string to page.evaluate
Functions bound with page.exposeFunction() produce unhandled promise rejections
How to pass a function in Puppeteers .evaluate() method?
How can I dynamically inject functions to evaluate using Puppeteer?
So, I have a front end in React.js or Ember.js - not sure. I'm just trying to automate some testing for it. When looking at the HTML in the Chrome Dev Tools, I see
<label class="MuiFormLabel-root-16 MuiInputLabel-root-5 MuiInputLabel-formControl-10 MuiInputLabel-animated-13 MuiInputLabel-outlined-15" data-shrink="false">Username</label>
This is set in an iframe (which isn't too important for this issue). I'm trying to get the ElementHandler using the puppeteer function
frame.$(selector)
How do I get the selector given
Username
I've tried a few things, but with little success.
If I understand correctly, you need to find an element by its text content. If so, these are at least two ways:
const puppeteer = require('puppeteer');
(async function main() {
try {
const browser = await puppeteer.launch(); // { headless: false, defaultViewport: null, args: ['--lang=en'] }
const [page] = await browser.pages();
await page.goto('https://example.org/');
const element1 = await page.evaluateHandle(() => {
return [...document.querySelectorAll('h1')].find(h1 => h1.innerText === 'Example Domain');
});
console.log(await (await element1.getProperty('outerHTML')).jsonValue());
const [element2] = await page.$x('//h1[text()="Example Domain"]');
console.log(await (await element2.getProperty('outerHTML')).jsonValue());
await browser.close();
} catch (err) {
console.error(err);
}
})();
My solution was to place HTML data attributes in the front end code so that I can easily select them.
I learned that puppeteer can add a function on the page's window object like this:
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.exposeFunction('md5', text =>
crypto.createHash('md5').update(text).digest('hex')
);
await page.evaluate(async () => {
// use window.md5 to compute hashes
const myString = 'PUPPETEER';
const myHash = await window.md5(myString);
console.log(`md5 of ${myString} is ${myHash}`);
});
So I'm wondering is there any way that Cypress can attach functions to window function like puppeteer does?
Can someone explain why this code isn't working. I have a console log before I run page.evaluate() which logs what I expect, but the console log inside page.evaluate never runs.
const puppeteer = require('puppeteer');
(async () => {
try {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.example.com');
page.on('response', async response => {
const url = response.url();
if (url.includes('something')) {
console.log('this code runs');
await page.evaluate(() => {
console.log("this code doesn't run");
});
}
});
} catch (err) {
console.log(err);
}
})();
Console log doesn't work in page.evaluate()
https://github.com/GoogleChrome/puppeteer/issues/1944
Try to use this code for display console.log from evaluate
page.on('console', msg => {
for (let i = 0; i < msg.args().length; ++i)
console.log(`${i}: ${msg.args()[i]}`);
});
page.evaluate(() => console.log('hello', 5, {foo: 'bar'}));
https://pptr.dev/#?product=Puppeteer&version=v1.20.0&show=api-event-console
The code inside page.evaluate is run in the browser context, so the console.log works, but inside the Chrome console and not the Puppeteer one.
To display the logs of the Chrome context inside the Puppeteer console, you can set dumpio to true in the arguments when launching a browser using Puppeteer:
const browser = await puppeteer.launch({
dumpio: true
})
Console.log works but in the browser context. I'm guessing here that you are trying to see the log in the CLI. If you want to see the log set headless to false and then see the log in the browser console.
What I'm trying to do here is loop through Storybook stories so I can perform visual regression testing on them:
const puppeteer = require('puppeteer');
const { toMatchImageSnapshot } = require('jest-image-snapshot');
expect.extend({ toMatchImageSnapshot });
test('no visual regression for button', async () => {
const selector = 'a[href*="?selectedKind=Buttons&selectedStory="]';
const browser = await puppeteer.launch({headless:false, slowMo: 350});
const page = await browser.newPage();
await page.goto('http://localhost:8080');
const storyLinks = await page.evaluate(() => {
const stories = Array.from(document.querySelectorAll('a[href*="?selectedKind=Buttons&selectedStory="]'));
const links = stories.map(story => story.href);
return links;
});
await storyLinks.forEach( (storyLink) => {
page.goto(storyLink).then(async (res,rej) => {
const screen = await page.screenshot();
return await expect(screen).toMatchImageSnapshot();
});
});
await browser.close();
});
One problem is that I get this because of the await broswer.close() that isn't waiting for everything to finish:
Protocol error (Page.navigate): Target closed.
at Session._onClosed (../../node_modules/puppeteer/lib/Connection.js:209:23)
at Connection._onClose (../../node_modules/puppeteer/lib/Connection.js:116:15)
at Connection.dispose (../../node_modules/puppeteer/lib/Connection.js:121:10)
at Browser.close (../../node_modules/puppeteer/lib/Browser.js:60:22)
at Object.<anonymous>.test (__tests__/visual.spec.js:24:16)
at <anonymous>
This happens for each storyLink except the first.
If I comment out the await browser.close() line, the screenshots are being taken, but not in the expected wait. Instead of each story having one screenshot, the last story is being screenshotted for the amount of stories. For example, I've got 4 stories in total, but I will have 4 screenshots of the last story instead of one for each story.
I don't understand why this behaviour is happening. The storyLinks returned from the page.evaluate funtions are correct, but then everything breaks and I've got no idea why.
Any ideas?
forEach is not good for async-await. Use for..of instead,
for (let storyLink of storyLinks) {
await page.goto(storyLink)
const screen = await page.screenshot();
await expect(screen).toMatchImageSnapshot();
};