Does Carlo emit a 'console' event like puppeteer?
Not directly but you can get the underlying puppeteer page which does, like this:
const app = await carlo.launch(...)
const window = app.mainWindow();
const page = window.page_;
page.on('console', () => {
...
})
Related
I want to get meta data in head tag using puppeteer.
but I got a lot of empty objects.
here is my code:
const description = await page.$eval("head > meta[name='description']", (e) => e.content);
console.log(description);
const title = await page.$eval('head > title', (e) => e.textContent);
console.log(title);
const headList = await page.$$eval('head', (e) => console.log(e))
console.log(headList) // {}
I want to get list like [meta, meta, meta, meta] <- like this.
In chrome browser console, I got a list using
const a = document.querySelectAll('meta');
Its code is work for me. but I got some issues when the puppeteer launch its browser on docker container environment.
My OS is mac os M1.
So, I am shooting this problem so on.
const list = await page.evaludate(() => Array.from(document.querySelectorAll('meta'), (e) => { return e. content }))
My Html code has Button-tags that have same id "hoge".
If you get the selector from the Chrome Dev Tool, it will be the same for both "#hoge".
<html>
<body>
<button id="hoge">Hoge</button>
<div class="shadow">
#shadow-root (open)
<button id="hoge">Hoge</button>
</div>
</body>
</html>
I want to get element of button-tag in shadow dom with puppeteer.
But, my javascript code gets element of 1st button.
const element = page.waitForSelector("pierce/#hoge");
This is not what I want.
I'm guessing it's because you didn't specify a unique selector, but i don't know what is unique selector for puppeteer.
If you know how to solve this problem, please let me know.
Long story short
I work with puppeteer a lot and wanted this knowlegde to be in my bag. One way to select a shadow Element is by accessing the parent DOM Node's shadowRoot property. The answer is based on this article.
Accessing Shadow Root property
For your html example this does the trick:
const button = document.querySelector('.shadow').shadowRoot.querySelector('#hoge')
waiting
Waiting though is a little more complicated but can be acquired using page.waitForFunction().
Working Sandbox
I wrote this full working sandbox example on how to wait for a certain shadowRoot element.
index.html (located in same directory as app.js)
<html>
<head>
<script>
// attach shadowRoot after 6 seconds for emulating waiting..
setTimeout(() => {
const btn = document.getElementById('hoge')
const container = document.getElementsByClassName('shadow')[0]
const shadowRoot = container.attachShadow({
mode: 'open'
})
shadowRoot.innerHTML = `<button id="hoge" onClick="doStuff()">hoge2</button>`
console.log('attached!.')
}, 6000)
function doStuff() {
alert('shadow button clicked!')
}
</script>
</head>
<body>
<button id="hoge">Hoge</button>
<div class="shadow">
</div>
</body>
</html>
app.js (located in same directory as index.html)
var express = require('express')
var { join } = require('path')
var puppeteer = require('puppeteer')
//utility..
const wait = (seconds) => {
console.log('waiting', seconds, 'seconds')
return new Promise((res, rej) => {
setTimeout(res, seconds * 1000)
})
}
const runPuppeteer = async() => {
const browser = await puppeteer.launch({
defaultViewport: null,
headless: false
})
const page = await browser.newPage()
await page.goto('http://127.0.0.1:5000')
await wait(3)
console.log('page opened..')
// only execute this function within a page context!.
// for example in page.evaluate() OR page.waitForFunction etc.
// don't forget to pass the selector args to the page context function!
const selectShadowElement = (containerSelector, elementSelector) => {
try {
// get the container
const container = document.querySelector(containerSelector)
// Here's the important part, select the shadow by the parentnode of the
// actual shadow root and search within the shadowroot which is like another DOM!,
return container.shadowRoot.querySelector(elementSelector)
} catch (err) {
return null
}
}
console.log('waiting for shadow elemetn now.')
const containerSelector = '.shadow'
const elementSelector = '#hoge'
const result = await page.waitForFunction(selectShadowElement, { timeout: 15 * 1000 }, containerSelector, elementSelector)
if (!result) {
console.error('Shadow element not found..')
return
}
// since waiting succeeded we can get the elemtn now.
const element = await page.evaluateHandle(selectShadowElement, containerSelector, elementSelector)
try {
// click the element.
await element.click()
console.log('clicked')
} catch (err) {
console.log('failed to click..')
}
await wait(10)
}
var app = express()
app.get('/', (req, res) => {
res.sendFile(join(__dirname, 'index.html'))
})
app.listen(5000, '127.0.0.1', () => {
console.log('listening!')
runPuppeteer()
})
Start example
$ npm i express puppeteer
$ node app.js
Make sure to use headless:false option to see what's happening.
The application does this:
start a small express server only serving index.html on /
open puppeteer after server has started and wait for the shadow root element to appear.
Once it appeared, it gets clicked and an alert() is shown. => success!
Browser Support
Tested with chrome.
Cheers ' ^^
I'm working on an extension that intercepts network requests using the chrome.devtools.network API.
The extension is written with React.
Currently, the interception is done on the devTools panel, so the interception started only when the user enter my extension.
I can move the interception logic to the devTools but I wonder how to pass the data to the panel dynamically, so the React component on the panel will be updated with the new data and can show it.
Any suggestion?
I've solved it by doing something similar to this answer like this:
devTools.tsx:
chrome.devtools.panels.create('Apex Network', "", 'panel.html', function (extensionPanel) {
var _window; // Going to hold the reference to panel.html's `window`
let reqs = [];
const requestHandler = request => {
reqs.push(request);
};
chrome.devtools.network.onRequestFinished.addListener(requestHandler)
extensionPanel.onShown.addListener(function tmp(panelWindow) {
extensionPanel.onShown.removeListener(tmp); // Run once only
chrome.devtools.network.onRequestFinished.removeListener(requestHandler)
_window = panelWindow;
_window.INITIAL_DATA = reqs;
}) })
Panel.tsx
declare global {
interface Window { INITIAL_DATA: any; }
}
window.INITIAL_DATA = window.INITIAL_DATA || {};
const Panel = () => {
const requestHandler = (request) => {...}
useEffect(()=>{
window.INITIAL_DATA?.forEach(requestHandler)
window.INITIAL_DATA = [];
chrome.devtools.network.onRequestFinished.addListener(requestHandler)
});
}
So in the end, it's done by doing that:
When the devtools is open, the devtools register to the network requests and store them in an array.
When the devtools extension is open, the devtools unregister for the network requests and the requests is passed to the panel and the devtools stop capturing requests
The panel handle the request that captured at the devtools
The panel register to the network requests
Thanks
I need to execute a script in every Window object created in Chrome – that is:
tabs opened through puppeteer
links opened by click()ing links in puppeteer
all the popups (e.g. window.open or "_blank")
all the iframes contained in the above
it must be executed without me evaluating it explicitly for that particular Window object...
I checked Chrome's documentation and what I should be using is Page.addScriptToEvaluateOnNewDocument.
However, it doesn't look to be possible to use through puppeteer.
Any idea? Thanks.
This searches for a target in all browser contexts.
An example of finding a target for a page opened
via window.open() or popups:
await page.evaluate(() => window.open('https://www.example.com/'))
const newWindowTarget = await browser.waitForTarget(async target => {
await page.evaluate(() => {
runTheScriptYouLike()
console.log('Hello StackOverflow!')
})
})
via browser.pages() or tabs
This script run evaluation of a script in the second tab:
const pageTab2 = (await browser.pages())[1]
const runScriptOnTab2 = await pageTab2.evaluate(() => {
runTheScriptYouLike()
console.log('Hello StackOverflow!')
})
via page.frames() or iframes
An example of getting eval from an iframe element:
const frame = page.frames().find(frame => frame.name() === 'myframe')
const result = await frame.evaluate(() => {
return Promise.resolve(8 * 7);
});
console.log(result); // prints "56"
Hope this may help you
I am using the chrome puppeteer library directly to run browser integration tests. I have a few tests written now in individual files. Is there a way run them in parallel? What is the best way to achieve this?
To run puppeteer instances in parallel you can check out this library I wrote: puppeteer-cluster
It helps to run different puppeteer tasks in parallel in multiple browsers, contexts or pages and takes care of errors and browser crashes. Here is a minimal example:
const { Cluster } = require('puppeteer-cluster');
(async () => {
const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_CONTEXT, // use one browser per worker
maxConcurrency: 4, // cluster with four workers
});
// Define a task to be executed for your data
cluster.task(async ({ page, data: url }) => {
await page.goto(url);
const screen = await page.screenshot();
// ...
});
// Queue URLs
cluster.queue('http://www.google.com/');
cluster.queue('http://www.wikipedia.org/');
// ...
// Wait for cluster to idle and close it
await cluster.idle();
await cluster.close();
})();
You can also queue your functions directly like this:
const cluster = await Cluster.launch(...);
cluster.queue(async ({ page }) => {
await page.goto('http://www.wikipedia.org');
await page.screenshot({path: 'wikipedia.png'});
});
cluster.queue(async ({ page }) => {
await page.goto('https://www.google.com/');
const pageTitle = await page.evaluate(() => document.title);
// ...
});
cluster.queue(async ({ page }) => {
await page.goto('https://www.example.com/');
// ...
});
// My tests contain about 30 pages I want to test in parallel
const aBunchOfUrls = [
{
desc: 'Name of test #1',
url: SOME_URL,
},
{
desc: 'Name of test #2',
url: ANOTHER_URL,
},
// ... snip ...
];
const browserPromise = puppeteer.launch();
// These test pass! And rather quickly. Slowest link is the backend server.
// They're running concurrently, generating a new page within the same browser instance
describe('Generate about 20 parallel page tests', () => {
aBunchOfUrls.forEach((testObj, idx) => {
it.concurrent(testObj.desc, async () => {
const browser = await browserPromise;
const page = await browser.newPage();
await page.goto(testObj.url, { waitUntil: 'networkidle' });
await page.waitForSelector('#content');
// assert things..
});
});
});
from https://github.com/GoogleChrome/puppeteer/issues/474
written by https://github.com/quicksnap
How I achieved this was to create your suite(s) of tests in individual files, as you have done already. Then create a testSuiteRunner.js file (or whatever you wish to call it) and set it up as follows:
require('path/to/test/suite/1);
require('path/to/test/suite/2);
require('path/to/test/suite/3);
...
Import all of your suite(s) using require statements, like above, (no need to give them const variable names or anything like that) and then you can use node ./path/to/testSuiteRunner.js to execute all of your suites in parallel. Simplest solution I could come up with for this one!
I think the best idea would be to use a test runner like Jest that can manage that for you. At least I do that this way. Please keep in mind the machine might blow up if you'd run too many Chrome instances at the same time so it's safe to limit the number of parallel tests to 2 or so.
Unfortunately, it isn't clearly described in the official documentation of how Jest parallels the tests. You might find this https://github.com/facebook/jest/issues/6957 useful.
Libraries as puppeteer-cluster are great but remember first of all you want to parallel your tests, not puppeteer's tasks.