I would like to control a headless chrome instance using puppeteer, taking snapshots and clicking on various page elements, while capturing a har file. Is this possible? I have looked at the API but haven't found anything useful.
There is no HAR generator helper in Puppeteer. But you can use chrome-har to generate HAR file.
const fs = require('fs');
const { promisify } = require('util');
const puppeteer = require('puppeteer');
const { harFromMessages } = require('chrome-har');
// list of events for converting to HAR
const events = [];
// event types to observe
const observe = [
'Page.loadEventFired',
'Page.domContentEventFired',
'Page.frameStartedLoading',
'Page.frameAttached',
'Network.requestWillBeSent',
'Network.requestServedFromCache',
'Network.dataReceived',
'Network.responseReceived',
'Network.resourceChangedPriority',
'Network.loadingFinished',
'Network.loadingFailed',
];
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
// register events listeners
const client = await page.target().createCDPSession();
await client.send('Page.enable');
await client.send('Network.enable');
observe.forEach(method => {
client.on(method, params => {
events.push({ method, params });
});
});
// perform tests
await page.goto('https://en.wikipedia.org');
page.click('#n-help > a');
await page.waitForNavigation({ waitUntil: 'networkidle2' });
await browser.close();
// convert events to HAR file
const har = harFromMessages(events);
await promisify(fs.writeFile)('en.wikipedia.org.har', JSON.stringify(har));
})();
Here you can find an article about this solution.
Solution proposed by #Everettss is the only option (so far), but is not as good as HAR saved in browser. Look at this, in both cases I generated HAR for google.com page. At top you have HAR generated by puppeteer-har (which is using chrome-har). Too little requests here, no metrics for main document, strangely different timing.
Puppeteer is not a perfect option for HAR files. Therefore I am suggesting to use https://github.com/cyrus-and/chrome-har-capturer
Related
I'm wondering if it's possible within puppeteer to access a FileSystemDirectoryHandle (from the File System Access API). I would like to pass in a directory path via puppeteer as though the user had selected a directory via window.showDirectoryPicker(). On my client page I use the File System Access API to write a series of png files taken from a canvas element, like:
const directoryHandle = await window.showDirectoryPicker();
for (let frame = 0; frame < totalFrames; frame++){
const fileHandle = await directoryHandle.getFileHandle(`${frame}.png`, { create: true });
const writable = await fileHandle.createWritable();
updateCanvas(); // <--- update the contents of my canvas element
const blob = await new Promise((resolve) => canvas.toBlob(resolve, 'image/png'));
await writable.write(blob);
await writable.close();
}
On the puppeteer side, I want to mimic that behavior with something like:
const page = await browser.newPage();
await page.goto("localhost:3333/canvasRenderer.html");
// --- this part doesn't seem to exist ---
const [dirChooser] = await Promise.all([
page.waitForDirectoryChooser(),
page.click('#choose-directory'),
]);
await dirChooser.accept(['save/frames/here']);
//--------------------------------------
but waitForDirectoryChooser() doesn't exist.
I'd really appreciate any ideas or insights on how I might accomplish this!
I've previously been getting results from bing using Puppeteer, but I'm now receiving the below error.
This comes from the following code, which previously worked.
(async () => {
const puppeteer = require('puppeteer');
const browser = await puppeteer.launch({headless: false})
const page = await browser.newPage();
await page.goto("https://www.bing.com/search?q=example+searchterm", { waitUntil: 'networkidle2'})
})();
When I do the same search in my browser, I get thousands of pages of results. Any ideas what's going on here?
There are some intranet sites that log me out more often than I like and I tried using Python, PowerShell and Javascript. The last one worked but it was based on launching those sites in child/popup windows which caused the focus to keep returning to those pages. I then looked at Puppeteer to do this using a headless Edge session.
I am not a developer - just know how to write short scripts
I am open to any Windows-based solution as I am not allowed to use Linux
The following is what I tried with Puppeteer (against MS Edge (chromium)):
const puppeteer = require("puppeteer-core");
const edgePaths = require("edge-paths");
const EDGE_PATH = edgePaths.getEdgePath();
async function loadTabs(){
const browser = await puppeteer.launch({executablePath: EDGE_PATH});
const url1 = await browser.newPage();
const url2 = await browser.newPage();
await url1.goto("https://url1");
await url2.goto("https://url2");
await delay(3000);
await browser.close();
}
function delay(time){
return new Promise(function(resolve){
setTimeout(resolve, time)
});
}
loadTabs();
The script is cobbled together from what I found about Puppeteer and JavaScript but I make no claim to this being even set up correctly.
Thanks!
I have tried multiple examples to get the splitwise login but unable to get it working.
Although, I'm quite new to puppeteer but login felt a simple usecase for understanding puppeteer.
const puppeteer = require('puppeteer')
const screenshot = 'login.png';
(async () => {
const browser = await puppeteer.launch({headless: false})
const page = await browser.newPage()
await page.goto("https://www.splitwise.com/login", {
waitUntil: 'networkidle2'
});
await page.type('#user_session_email', 'atest')
await page.type('#user_session_password', 'test')
await page.click('[name="commit"]')
await page.waitForNavigation()
browser.close()
console.log('See screenshot: ' + screenshot)
})()
Unfortunately, the page has two forms with identical ids (but different classes) and these forms have inputs with identical ids as well. You just need more specific selectors:
await page.type('form.form-stacked #user_session_email', 'atest')
await page.type('form.form-stacked #user_session_password', 'test')
await page.click('form.form-stacked [name="commit"]')
This does not seem to be a puppeteer issue.
It seems that javascript code in page is actively blocking triggered events somehow.
Are you able to set these values using regular javascript in the console?
I'm developing an app using Angular js and i'm doing a a questionnaire component which has a lot of nested pages to develop, so instead of doing changes and then spends a lot of time to return to the same page and see the results - i was advised to use Puppeteer to simulate to automate to page navigation ... I've found many examples of Puppeteer that outputs a png file - but none that's simply open a URL...
Here is what i've tried so far...
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('http://cnn.com');
// await page.screenshot({path: 'example.png'});
//await browser.close();
})();
As you can see in my example - it didn't opened a new page...