I am trying to use puppeteer to render a simple web component using this code:
async function ssr(url) {
const start = Date.now();
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setJavaScriptEnabled(true);
await page.goto(url, { waitUntil: 'networkidle0' });
await page.waitForSelector('#hello');
const html = await page.content();
await browser.close();
const ttRenderMs = Date.now() - start;
console.info(`Headless rendered page in: ${ttRenderMs}ms`);
return { html, ttRenderMs };
}
This is what it tries to load:
<!DOCTYPE html>
<html>
<head>
<title>Lit + SSR</title>
</head>
<body>
<script type="module">
import { LitElement, html } from 'lit-element';
export class HelloElement extends LitElement {
static get properties() {
return {
name: { type: String }
};
}
render() {
return html`
Hello, <b>${this.name}</b>!
`;
}
}
customElements.define('x-hello', HelloElement);
</script>
<x-hello id="hello" name="Ninja"></x-hello>
</body>
</html>
While puppeteer is able to render the page correctly (verified by page.screenshot), page.content or page.evaluate(() => document.documentElement.innerHTML) both do not return the shadow DOM and therefore is useless.
How do I get puppeteer to return me the markup that includes the shadow root?
Related
I am using Vue3.js. I need to open a PDF in a new window, then open the print dialog but this only works on MS Edge and Google Chrome, but does not work on Firefox or Safari.
const getBlobPdf = async (url: string) => {
const response = await axios.get(url, { responseType: "blob" });
return new Blob([response.data], { type: "application/pdf" });
};
const printPdf = async ({ url }) => {
const blob = await getBlobPdf(url);
window.open(URL.createObjectURL(blob), "_blank")?.print();
};
I had tried window.focus(); window.print(); but didn't work.
I found a way to make it work on Firefox and Safari. But I don't know if there is another way without using setTimeout()
const printPdf = async ({ url }) => {
const blob = await getBlobPdf(url);
const printWin = window.open(URL.createObjectURL(blob), "_blank");
if (printWin) {
setTimeout(() => {
printWin.print();
}, 200);
}
};
I am trying to make it log the data found from inspect>network>preview but right now it logs inspect>network>headers.
Here is what I have:
const puppeteer = require("puppeteer");
const url =
"https://www.google.com/";
async function StartScraping() {
await puppeteer
.launch({
headless: false,
})
.then(async (browser) => {
const page = await browser.newPage();
await page.setViewport({
width: 1500,
height: 800,
});
page.on("response", async (response) => {
if (response.url().includes("Text")) {
console.log(await response);
}
});
await page.goto(url, {
waitUntil: "load",
timeout: 0,
});
});
}
StartScraping();
It depends how you want it formatted. More information can be found here: https://github.com/puppeteer/puppeteer/blob/9ef4153f6e3548ac3fd2ac75b4570343e53e3a0a/docs/api.md#class-response
I've modified your code a bit to where I think you would want the response:
page.on("response", async (response) => {
if (response.url().includes("Text")) {
console.log(await response.text());
}
});
I am using nextjs. I need to render custom landing pages according to their specific url. I am able to render all the details from the database of that particular URL except for the object which contains the details for the page. The pages has been built with the help of grapesjs.
Following is the data in db:
Following is code for rendering the list of the pages:
index.js
import Link from "next/link"
export const getStaticProps = async () => {
const res = await fetch("http://localhost:5000/api/webpage/");
const data = await res.json();
return {
props: {
data,
}
};
};
const blog = ({ data }) => {
return (
<div>
{data?.map((currentElement) => {
return (
<div key={currentElement.id} className="ssr-styles">
<h3>
{/* {currentElement._id} */}
<Link href={`/blog/${currentElement.url}`}>
{currentElement.name}
</Link>
</h3>
</div>
);
})}
</div>
);
};
export default blog;
Following is the code where the page is actually rendering:
[pageno].js=>
export const getStaticPaths = async () => {
const res = await fetch("http://localhost:5000/api/webpage/");
const data = await res.json();
const paths = data.map((currentElement) => {
return {
params: { pageno: currentElement.url.toString() }
};
});
return {
paths,
fallback: false
};
};
export const getStaticProps = async (context) => {
const url = context.params.pageno;
const res = await fetch(`http://localhost:5000/api/webpage/url/${url}`);
const data = await res.json();
return {
props: {
data
}
};
};
export const Details = ({ data }) => {
return (
<>
<div key={data.url} className="ssr-styles">
{data._id}
<h3>{data.name}</h3>
</div>
</>
);
};
export default Details;
how do I render the html inside the object content so as to get a proper webpage?
you can try using html-react-parser library. it converts an HTML string to React elements.
I'm trying to make a basic multiplayer game with Socket.IO, p5.js and NodeJS, hosting it on Replit.
I have a basic httpServer with socket.io, and it serves the HTML, CSS and JavaScript files fine. But when I put the <link> tag in the HTML to load the CSS, the CSS loads fine (I can see it in the Sources tab in the Chrome DevTools) but it doesn't actually apply to the HTML.
The code is here, but I'll put it here as well.
index.js The main NodeJS file
const { readFileSync } = require('fs');
const { createServer } = require('http');
const { Server } = require('socket.io');
const httpServer = createServer((req, res) => {
const r = /^\/?(index\.(html|css|js))?$/i;
if (!r.test(req.url))
{
res.writeHead(404);
res.end('Not found');
return;
}
const m = req.url.match(r);
// reload the file every time
const content = readFileSync(__dirname + '/public/' + (m[1] || 'index.html'));
const length = Buffer.byteLength(content);
res.writeHead(200, {
'Content-Type': 'text/html',
'Content-Length': length,
});
res.end(content);
});
const io = new Server(httpServer, {
// Socket.IO options
});
let players = {};
io.on('connection', (socket) => {
players[socket.id] = {
id: socket.id,
x: 0,
y: 0
};
socket.on('disconnect', (reason) => {
delete players[socket.id];
});
});
io.on('data', data => {
players[data.id].x += data.x;
players[data.id].y += data.y;
});
setInterval(() => {
io.sockets.emit('data', players);
}, 1000 / 60);
httpServer.listen(6001);
public/index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Multiplayer Online IO Game 2</title>
<link rel="stylesheet" href="index.css">
<script src="/socket.io/socket.io.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.4.0/p5.min.js" integrity="sha512-N4kV7GkNv7QR7RX9YF/olywyIgIwNvfEe2nZtfyj73HdjCUkAfOBDbcuJ/cTaN04JKRnw1YG1wnUyNKMsNgg3g==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<script src="/index.js" defer></script>
</head>
<body>
</body>
</html>
public/index.css
body
{
margin: 0px;
overflow: hidden;
}
canvas
{
display: none;
}
The canvas' display: none was to see if the CSS actually did anything, but it doesn't.
public/index.js The client JavaScript
let ID = null;
let players = {};
const socket = io({
// Socket.IO options
});
socket.on('connect', () => {
ID = socket.id;
});
socket.on('connect_error', (err) => {
alert(`There was an error connecting to the WebSocket server:\n${err.message}`);
});
socket.on('data', (data) => {
players = data;
});
function setup()
{
createCanvas(windowWidth, windowHeight);
}
function draw()
{
background(255);
fill(0);
for (const id of Object.keys(players))
{
const player = players[id];
circle(player.x, player.y, 10);
}
}
Your server is using the content type text/html for all responses regardless of the type of file being returned. Some web browsers are strict about content-types and will not process a stylesheet if it has the wrong content type. Here's a example fix:
const httpServer = createServer((req, res) => {
const r = /^\/?(index\.(html|css|js))?$/i;
if (!r.test(req.url))
{
res.writeHead(404);
res.end('Not found');
return;
}
const m = req.url.match(r);
// reload the file every time
const content = readFileSync(__dirname + '/public/' + (m[1] || 'index.html'));
const length = Buffer.byteLength(content);
res.writeHead(200, {
// Determine the content type based on the file extension
'Content-Type': m[2] ? getContentType(m[2]) : 'text/html',
'Content-Length': length,
});
res.end(content);
});
function getContentType(ext) {
switch (ext.toLowerCase()) {
case 'html':
return 'text/html';
case 'css':
return 'text/css';
case 'js':
return 'text/javascript';
default:
return 'application/octet-stream';
}
}
You might want to consider using a more full-featured HTTP server such as express instead of rolling your own.
I have two spans with text inside it like this.
<span class="date">today</span>
<span class="date">tomorrow</span>
I want to get those texts and console.log() it on my terminal. How can I do that?
Thank you.
const puppeteer = require('puppeteer');
const html = `
<!doctype html>
<html>
<head><meta charset='UTF-8'><title>Test</title></head>
<body>
<span class="date">today</span>
<span class="date">tomorrow</span>
</body>
</html>`;
(async function main() {
try {
const browser = await puppeteer.launch();
const [page] = await browser.pages();
await page.goto(`data:text/html,${html}`);
const texts = await page.evaluate(() => Array.from(
document.querySelectorAll('span.date'),
span => span.innerText
));
console.log(texts); // [ 'today', 'tomorrow' ]
await browser.close();
} catch (err) {
console.error(err);
}
})();