The problem that I have is that it doesn't recognize that puppeteer.use i a function. I took these lines of code const StealthPlugin = require('puppeteer-extra-plugin-stealth') puppeteer.use(StealthPlugin()) const AdblockerPlugin = require('puppeteer-extra-plugin-adblocker') puppeteer.use(AdblockerPlugin({ blockTrackers: true })) from puppeteer extra and downloaded and followed all the instructions that I needed but it still doesn't work. The problem that I had from the beginning and which I was hoping to solve was that I can't run my programm headless: true. And sadly it didn't work entirely.
This is the error that I'm getting when I'm trying to run the program:
`Uncaught TypeError TypeError: puppeteer.use is not a function
at <anonymous> (undefined:3:11)
at Module._compile (undefined:1159:14)
at Module._extensions..js (undefined:1213:10)
at Module.load (undefined:1037:32)
at Module._load (undefined:878:12)
at executeUserEntryPoint (undefined:81:12)
at <anonymous> (undefined:23:47)`
this is my code:
`const puppeteer = require('puppeteer');
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
const AdblockerPlugin = require('puppeteer-extra-plugin-adblocker')
puppeteer.use(AdblockerPlugin({ blockTrackers: true }))
const product_url = "https://www.zalando.se/nike-sportswear-air-flight-lite-mid-hoega-sneakers- whiteblack-ni112n02z-a11.html"
const cart = "https://www.zalando.se/cart"
async function givePage(){
const browser = await puppeteer.launch({headless: true})
const page = await browser.newPage();
return page;
}
async function addToCart(page){
// going to website
await page.goto(product_url)
// clicking "handla"
await page.waitForSelector("button[class='DJxzzA u9KIT8 uEg2FS U_OhzR ZkIJC- Vn-7c- FCIprz heWLCX JIgPn9 LyRfpJ pxpHHp Md_Vex NN8L-8 GTG2H9 MfX1a0 WCjo-q EKabf7 aX2-iv r9BRio mo6ZnF PLvOOB']");
await page.click("button[class='DJxzzA u9KIT8 uEg2FS U_OhzR ZkIJC- Vn-7c- FCIprz heWLCX JIgPn9 LyRfpJ pxpHHp Md_Vex NN8L-8 GTG2H9 MfX1a0 WCjo-q EKabf7 aX2-iv r9BRio mo6ZnF PLvOOB']", elem => elem.click);
// clicking "OK" to cookies
await page.waitForSelector("button[class='uc-btn uc-btn-primary']");
await page.click("button[class='uc-btn uc-btn-primary']");
// clicking "size EU 41"
await page.evaluate(() => document.getElementsByClassName('_6G4BGa _0Qm8W1 _7Cm1F9 FxZV-M IvnZ13 Pb4Ja8 ibou8b JT3_zV ZkIJC- Md_Vex JCuRr_ na6fBM _0xLoFW FCIprz pVrzNP KRmOLG NuVH8Q')[4].click());
console.log("körs")
await new Promise(r => setTimeout(r, 1000));
// going to "cart"
await page.goto(cart)
// clicking "gå till checkout"
await page.waitForSelector("button[class='z-1-button z-coast-base-primary-accessible z-coast- base__sticky-sumary__cart__button-checkout z-1-button--primary z-1-button--button']");
await page.click("button[class='z-1-button z-coast-base-primary-accessible z-coast-base__sticky- sumary__cart__button-checkout z-1-button--primary z-1-button--button']", elem => elem.click);
await new Promise(r => setTimeout(r, 1000));
// logging into gmail
const email = await page.waitForSelector("[id='login.email']");
await email.type('marc.ehadsadwa#gmail.com');
await new Promise(r => setTimeout(r, 500));
// typing in password
const password = await page.waitForSelector("[id='login.secret']");
await password.type('Bradasdwad');
await new Promise(r => setTimeout(r, 1000));
// clicking "log in"
await page.evaluate(() => document.getElementsByClassName('DJxzzA u9KIT8 uEg2FS U_OhzR ZkIJC- Vn-7c- FCIprz heWLCX JIgPn9 LyRfpJ pxpHHp Md_Vex NN8L-8 GTG2H9 MfX1a0 WCjo-q EKabf7 aX2-iv r9BRio mo6ZnF E6Km4r') [0].click());
await new Promise(r => setTimeout(r, 1000));
// clicking "hämta hos postombud"
}
async function checkout(){
var page = await givePage();
await addToCart(page);
}
checkout();`
I've tried like solving the problem but when I do another one shows up like that the program don't know what web browser it's supposed to go into.
This is my package.json:
package name: (marce)
version: (1.0.0)
description:
git repository:
keywords:
author:
license: (ISC)
About to write to C:\Users\marce\package.json:
{
"name": "marce",
"version": "1.0.0",
"main": "index.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"puppeteer": "^19.6.0",
"puppeteer-extra": "^3.3.4",
"puppeteer-extra-plugin-adblocker": "^2.13.5",
"puppeteer-extra-plugin-stealth": "^2.11.1"
},
"devDependencies": {},
"description": ""
}
Puppeteer itself does not have a use function. You'll have to use puppeteer-extra instead.
Adjust your code in the following way:
Replace:
const puppeteer = require('puppeteer');
With:
const puppeteer = require('puppeteer-extra');
Related
So my problem is that I can't run puppeteer headlessly and I've done many things to try and fix it and I've tried going in to the link and I've tried to solve it myself and I just can't do it myself. I've just recently started coding and I'm been on this problem for weeks so if someone would be able to help me would it be amazing.
This is my code:
async function startBrowser() {
const browser = await puppeteer.launch({executablePath: '/path/to/chromium', headless: true,
ignoreDefaultArgs: ['--disable-extensions']});
const page = await browser.newPage();
return page;
}
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
const AdblockerPlugin = require('puppeteer-extra-plugin-adblocker')
puppeteer.use(AdblockerPlugin({ blockTrackers: true }))
const product_url = "https://www.zalando.se/nike-sportswear-air-flight-lite-mid-hoega-
sneakers-whiteblack-ni112n02z-a11.html"
const cart = "https://www.zalando.se/cart"
async function addToCart(page){
// going to website
await page.goto(product_url)
// clicking "handla"
await page.waitForSelector("button[class='DJxzzA u9KIT8 uEg2FS U_OhzR ZkIJC- Vn-7c- FCIprz
heWLCX JIgPn9 LyRfpJ pxpHHp Md_Vex NN8L-8 GTG2H9 MfX1a0 WCjo-q EKabf7 aX2-iv r9BRio mo6ZnF
PLvOOB']");
await page.click("button[class='DJxzzA u9KIT8 uEg2FS U_OhzR ZkIJC- Vn-7c- FCIprz heWLCX JIgPn9
LyRfpJ pxpHHp Md_Vex NN8L-8 GTG2H9 MfX1a0 WCjo-q EKabf7 aX2-iv r9BRio mo6ZnF PLvOOB']", elem
=> elem.click);
// clicking "OK" to cookies
await page.waitForSelector("button[class='uc-btn uc-btn-primary']");
await page.click("button[class='uc-btn uc-btn-primary']");
// clicking "size EU 41"
await page.evaluate(() => document.getElementsByClassName('_6G4BGa _0Qm8W1 _7Cm1F9 FxZV-M
IvnZ13 Pb4Ja8 ibou8b JT3_zV ZkIJC- Md_Vex JCuRr_ na6fBM _0xLoFW FCIprz pVrzNP KRmOLG NuVH8Q')
[4].click());
console.log("körs")
await new Promise(r => setTimeout(r, 1000));
// going to "cart"
await page.goto(cart)
// clicking "gå till checkout"
await page.waitForSelector("button[class='z-1-button z-coast-base-primary-accessible z-coast-
base__sticky-sumary__cart__button-checkout z-1-button--primary z-1-button--button']");
await page.click("button[class='z-1-button z-coast-base-primary-accessible z-coast-
base__sticky-sumary__cart__button-checkout z-1-button--primary z-1-button--button']", elem =>
elem.click);
await new Promise(r => setTimeout(r, 1000));
// logging into gmail
const email = await page.waitForSelector("[id='login.email']");
await email.type('marc.ehadsadwa#gmail.com');
await new Promise(r => setTimeout(r, 500));
// typing in password
const password = await page.waitForSelector("[id='login.secret']");
await password.type('Bradasdwad');
await new Promise(r => setTimeout(r, 1000));
// clicking "log in"
await page.evaluate(() => document.getElementsByClassName('DJxzzA u9KIT8 uEg2FS U_OhzR ZkIJC-
Vn-7c- FCIprz heWLCX JIgPn9 LyRfpJ pxpHHp Md_Vex NN8L-8 GTG2H9 MfX1a0 WCjo-q EKabf7 aX2-iv
r9BRio mo6ZnF E6Km4r')[0].click());
await new Promise(r => setTimeout(r, 1000));
// clicking "hämta hos postombud"
}
async function checkout(){
var page = await startBrowser();
await addToCart(page);
}
checkout();
This is the error I'm getting:
Uncaught Error Error: Failed to launch the browser process! spawn /path/to/chromium ENOENT
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
at onClose (undefined:299:20)
at <anonymous> (undefined:293:24)
at emit (undefined:513:28)
at ChildProcess._handle.onexit (undefined:289:12)
at onErrorNT (undefined:476:16)
at processTicksAndRejections (undefined:82:21)
Any existing sample on how to use puppeteer with nordVpn ?
I tried that:
page = await browser.newPage();
await useProxy(page, `socks5://login:password}#fr806.nordvpn.com:1080`);
I also tried:
'--proxy-server=socks5://login:password#fr806.nordvpn.com:1080'
This script works, you need to change the user/pass to yours... these are not your Nord user/pass... you need to get the service/api ones from in your account settings. Change the server to whatever one you need to use.
#!/usr/bin/env node
// Screengrab generator
// outputs a JSON object with a base64 encoded image of the screengrab
// eg;
const puppeteer = require('puppeteer');
let conf = new Object();
conf.url = "https://www.telegraph.co.uk";
// VPN
conf.vpnUser = conf.vpnUSer || 'USERNAME';
conf.vpnPass = conf.vpnPass || 'PASSWORD';
conf.vpnServer = conf.vpnServer || "https://uk1785.nordvpn.com:89";
(async() => {
const browser = await puppeteer.launch({
headless: true,
args: [
'--disable-dev-shm-usage',
'--proxy-server='+conf.vpnServer
]
});
try {
const page = await browser.newPage();
await page.authenticate({
username: conf.vpnUser,
password: conf.vpnPass,
});
await page.goto(conf.url, { waitUntil: 'networkidle2' });
} catch (error) {
console.error(error);
} finally {
await browser.close();
}
})();
I'm trying to submit a login form, but all I get is a timeout after 30 seconds.
My code is rather simple and I can't find anything wrong:
const puppeteer = require('puppeteer');
const creds = {
user: "1234",
password: "1234"
};
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({width: 1280, height: 800});
await page.goto('https://shop2.idena.de/NewShop/');
await page.type('input[name="FORM_LOGIN"]', creds.user);
await page.type('input[name="FORM_PASSWD"]', creds.password);
await Promise.all([
page.click('button[name="FORM_TYPE"]'),
page.waitForNavigation()
]);
await page.screenshot({path: 'example.png', fullPage: true});
await browser.close();
})();
Any ideas what's going wrong here?
Change the order of the promises a bit, it could be possible, the navigation happens super fast and the waitForNavigation is just waiting for nothing. Or maybe your website loads very slow after clicking the login button.
await Promise.all([
page.waitForNavigation({timeout: 60000}),
page.click('button[name="FORM_TYPE"]'),
]);
If I use your example with headful option, I get this dialog that prevents the page from loading:
So this addition can help (not sure if some dialog emerges with correct credentials):
const puppeteer = require('puppeteer');
const creds = {
user: "1234",
password: "1234"
};
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({width: 1280, height: 800});
await page.goto('https://shop2.idena.de/NewShop/');
await page.type('input[name="FORM_LOGIN"]', creds.user);
await page.type('input[name="FORM_PASSWD"]', creds.password);
page.on('dialog', async dialog => {
console.log(dialog.message());
await dialog.accept();
});
await Promise.all([
page.click('button[name="FORM_TYPE"]'),
page.waitForNavigation()
]);
await page.screenshot({path: 'example.png', fullPage: true});
await browser.close();
})();
EDIT: I'm updating my answer since more infomration has been provided in the original question.
The problem is that there's a dialog you need to confirm/dismiss:
Perhaps you didn't see it because the script was too fast. I recommend debugging puppeteer scripts with headless set to false and slowMo to some number greater than 0:
const browser = await puppeteer.launch({ headless: false, slowMo: 200 });
Then you need to get rid of the dialog:
page.on('dialog', async (dialog) => {
await dialog.accept();
});
The whole script that now passes:
const puppeteer = require('puppeteer');
const creds = {
user: "1234",
password: "1234"
};
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({width: 1280, height: 800});
await page.goto('https://shop2.idena.de/NewShop/');
await page.type('input[name="FORM_LOGIN"]', creds.user);
await page.type('input[name="FORM_PASSWD"]', creds.password);
page.on('dialog', async (dialog) => {
await dialog.accept();
});
await Promise.all([
page.click('button[name="FORM_TYPE"]'),
page.waitForNavigation()
]);
await page.screenshot({path: 'example.png', fullPage: true});
await browser.close();
})();
Tried below code but getting an error.
Error: net::ERR_SSL_VERSION_OR_CIPHER_MISMATCH at
https://www.xxxxxxsolutions.com/
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({ignoreHTTPSErrors: true, acceptInsecureCerts: true, args: ['--proxy-bypass-list=*', '--disable-gpu', '--disable-dev-shm-usage', '--disable-setuid-sandbox', '--no-first-run', '--no-sandbox', '--no-zygote', '--single-process', '--ignore-certificate-errors', '--ignore-certificate-errors-spki-list', '--enable-features=NetworkService']});
const page = await browser.newPage();
try {
await page.goto('https://www.xxxxxxxsolutions.com/', {waitUntil: 'networkidle2', timeout: 59000});
const cookies = await page._client.send('Network.getAllCookies');
JSON.stringify(cookies, null, 4);
} catch (e) {
console.log(e);
}
await browser.close();
})();
#mujuonly, this is version related issue. Please try the same code above 1.16.0 or latest version 2.0. It's working fine.
I'm new to puppeteer and node, trying to use a proxy with puppeteer in order to collect requests & responses, hopefully also websocket communication, but so far couldn't get anything to work..
I'm trying the following code:
const puppeteer = require('puppeteer');
const httpProxy = require('http-proxy');
const url = require('url');
let runProxy = async ()=> {
// raise a proxy and start collecting req.url/response.statusCode
};
let run = async () => {
await runProxy();
const browser = await puppeteer.launch({
headless: false,
args: ['--start-fullscreen',
'--proxy-server=localhost:8096']
});
page = await browser.newPage();
await page.setViewport({ width: 1920, height: 1080 });
await page.goto('http://www.google.com',
{waitUntil: 'networkidle2', timeout: 120000});
};
run();
I've tried some variation from https://github.com/nodejitsu/node-http-proxy but nothing seems to work for me, some guidance is at need, thanks
try this, use https-proxy-agent or http-proxy-agent to proxy request for per page:
import {Job, Launcher, OnStart, PuppeteerUtil, PuppeteerWorkerFactory} from "../..";
import {Page} from "puppeteer";
class TestTask {
#OnStart({
urls: [
"https://www.google.com",
"https://www.baidu.com",
"https://www.bilibili.com",
],
workerFactory: PuppeteerWorkerFactory
})
async onStart(page: Page, job: Job) {
await PuppeteerUtil.defaultViewPort(page);
await PuppeteerUtil.useProxy(page, "http://127.0.0.1:2007");
await page.goto(job.url);
console.log(await page.evaluate(() => document.title));
}
}
#Launcher({
workplace: __dirname + "/workplace",
tasks: [
TestTask
],
workerFactorys: [
new PuppeteerWorkerFactory({
headless: false,
devtools: true
})
]
})
class App {}