Running Puppeteer with headless: false & Firebase Functions - google-cloud-functions

I'm trying to use Puppeteer + Firebase Functions while using the headless: false option
await puppeteer.launch({
headless: false
});
It works well locally. I use the headless: false so it won't trigger a bot check, which work well.
When I deploy it to Firebase Functions it doesn't seem to be able to launch a browser:
Error: Failed to launch the browser process!
[15:32:0209/162803.759118:ERROR:file_path_watcher_linux.cc(78)] Failed to read /proc/sys/fs/inotify/max_user_watches
[15:15:0209/162804.834741:ERROR:ozone_platform_x11.cc(234)] Missing X server or $DISPLAY
[15:15:0209/162804.834794:ERROR:env.cc(225)] The platform failed to initialize. Exiting.
Is there some sort of a solution to run it as a Firebase Function?

I’ll start with the cases that combined Cloud functions and Puppeteer. Here is an old question that discusses the same log error. I think this is the first place to look for a solution, as it provides links to a couple of Github Issues reporting similar error logs. In this other Github issue, a user recommended code to start Puppeteer in Cloud functions. This example can help you as well.
I’ll leave these two guides that can walk you through. This old medium article and this more updated version from logrocket.
Other question has several workarounds; unfortunately, none of them include Cloud Functions, just in case this is caused purely by Puppeteer. And here is how to run Headful Mode.
However, if you want to avoid being detected by the bot, there is also a discussion about this that could give you some hints.
There is no exact solution for this, as it seems there are several causes and workarounds. I would advise you to open a new question once you have tried some alternatives already provided by the community, and see if there's a more specific solution.

Related

Is WebGPU something that can actually be tested right now?

I have a hobby project that could use some gpu compute, I figured it would also be a great time to test out the new WebGPU API. So far I tried looking at various WebGPU demos on latest versions of both chrome canary and firefox dev edition, both having webgpu flag enabled beforehand but nothing works. In console I get various errors about some vertex buffer or some context not existing, big piles of red.
So.. I was under the impression that I could actually check it out? Does this work for anyone? Is it broken rn?
Possibly relevant system information: Windows 10, Nvidia M3000M
Can you show me the error message?
If it's your own example, I guess you may not have configured how to introduce it .wgsl file.
webpack4: rules: [{ test: /\.wgsl$/, use: "raw-loader" }]
webpack5: rules: [{ test: /\.wgsl$/, type: "asset/source" }]
vite: import xxx from './xx.wgsl?raw
I've been toying with WebGPU in Chrome and it seems to be possible to test without major issues.
It does seem that a number of demos out there are outdated, using an older API or syntax for WSGL, but most of the time I'm just getting warnings and not errors. Those warnings are quite informative and have been straightforward to fix too (eg: replacing [[location(0)]] with #location(0)).
As a tip, I've been using it with TypeScript and combining it with #webgpu/types makes the development way easier.
Here an answer to the main question, if webgpu can be tested yet:
As #Rob mentioned as well, caniuse.com/webgpu shows that
no browser has it activated by default yet
but you CAN TEST IT by activating it as experimental feature, there just won't be many users able to see your work if you were to publish the site.
(august 2022)
As of January 2023 you can enable WebGPU in Canary builds of Chrome if you enable the feature through about://flags.
It's also possible to view some examples (e.g. the ones here) in shipping versions of Chrome. You can request a site token for the origin trial, which will make the feature available for your site--however the site says "The origin trial is expected to end in Chrome 109 (Feb 1, 2023)."
Beta and Nightly builds of Firefox have an implementation of WebGPU that's not as fully featured.
As other answers suggest though, at time of writing it's still pretty experimental and there are very few good demos beyond spinning cubes and some basic particle systems.

The Folium library can't generate maps

Since yesterday afternoon, I have not succeeded in using Folium to generate maps, even the most basic display function can not be completed, the network connection is normal, it is likely that there is a problem with the call of js.
I tried switching the network environment and trying to change computers, but it didn't work.
Failed to load resource: net::ERR_CERT_DATE_INVALID
leaflet.awesome-markers.js:17 Uncaught ReferenceError: L is not defined
at leaflet.awesome-markers.js:17
at leaflet.awesome-markers.js:122
leaflet.css:1 Failed to load resource: net::ERR_CERT_DATE_INVALID
map.html:39 Uncaught ReferenceError: L is not defined
at map.html:39
#This is the code for the most basic function I've tried.
import folium
m = folium.Map(location=[29.488869,106.571034],
zoom_start=16,
control_scale=True,
width='50%')
m.save('map.html')
I hope to generate map pages
I dont think there is any problem in your jupyter notebook/Python IDLE. Check with your browser. If you’ve determined that the ERR_CERT_DATE_INVALID is caused by an issue on your computer, try these steps to resolve it:
First things first, check the Date and Time set on your computer, if
these are wrong it probably explains how you got the
ERR_CERT_DATE_INVALID error.
Sometimes fixing this error is as simple as shutting down your
browser and then restarting it. Other times a system reboot may work.
However, there are a couple of instances where you’ll need to do a
little more work to set things straight.
Check your connection, if you’re connected to public WiFi or some
other public network there’s a chance that your browser is right and
you don’t actually have a secure connection. If that’s the case, stop
browsing and resume when you’re on a more secure setup.
Scan your computer with a trusted antivirus software, you may have
malware of some sort that is causing the issue. Unfortunately we
can’t provide you with info on how to fix every last piece of
malware, but if your antivirus can’t, someone on the internet
probably knows.
Disable any third-party plugins you have running on Chrome. Sometimes
these can cause unwanted problems.
Clear your browser cache on Chrome. Click the menu icon, open History
and select “clear browsing data.”
Delete and then re-install Chrome. Sometimes this helps.
There are a few other crazy fixes like bringing down your firewall or
modifying network settings. However I would not suggest this one since your PC then become susceptible to virus.

How to disable webRTC in Chromium?

I was trying to disable webRTC in Chromium for the past few hours unsuccessfully. I would like to solve it without using extensions for headless: true by either modifying the right files in puppeteer-bundled chromium or with launch arguments if that is possible? So far I was unable to pass the test at browserleaks.com/webrtc
Is there an easy way I can implement this every time I use puppeteer in my projects?
Edit: When headless:false this can be solved by loading in local extensions. However there seem to not be simple solution equivalent to adding arguments when headless:true.
If you are online concerned about the IP leak, you can use --force-webrtc-ip-handling-policy=default_public_interface_only on the command line.
If you are writing a Chromium based application, you need to set the policy in content::RendererPreferences webrtc_ip_handling_policy

Allow multiple instances of Google Chrome?

Disclaimer: This may be a better question on SuperUser, but my use
case matches SO.
I'm trying to develop an Angular app on my local computer; I'm not yet ready to set up a webserver or anything like that. I'm loading some JSON configuration files from the same directory, and I'm running into Origin Policy issues (which was expected).
I know that programs like gChat can run multiple instances with the /mutex flag (I think that's it, it's been a while since I tried that). Is there any such provision for Google Chrome? Basically, I'd like to run Chrome as my main browser for everything I do, and then open a separate instance with lowered web security for testing purposes.
Thanks!
You can bring up a development web server, serving data from the current working directory, using:
python -mSimpleHTTPServer
This doesn't directly answer your question, I know. But hopefully it's even better than a direct answer. :)
I found a solution to this - although Chrome won't let you run multiple instances, you can run Chrome and Chrome Canary side-by-side simultaneously. So now, Chrome is my main browser, and Chrome Canary (with the --disable-web-security) flag is my testing environment. So far, I haven't had any issues with discrepancies between the browsers.

Building web server using chrome.socket API

I've built a web server using Chrome Packaged Apps. The problem I see repeatedly is that chrome.socket.accept() and chrome.socket.write() don't invoke their callback functions. It usually works more or less reliably if request rate is less than 1 request per seconds. If I go above that, then I start seeing errors or missing callbacks.
I did similar tests with sample "webserver" app build by Google (https://github.com/GoogleChrome/chrome-app-samples/tree/master/webserver). It has the same problem. It usually takes less than 100 requests before web server stops responding. The easiest way to reproduce the problem is to use Chrome browser as a client and hold F5 key for few seconds.
It would be desirable to have a sample app that demonstrates how to build reliable web server using chrome.socket. So far I tried several different workarounds for monitoring the situation from the app itself and restarting socket when socket stops working, but it's not easy because there is no reliable way to check the status of the connection or status of the last operation when callback is not fired. I tried to use getInfo() method, but it always returns "connected=true" regardless of the situation.
I saw this on Windows 7 and Chrome OS (Chromebook).
Just an update on this. According to this the issue is now fixed.
There are still other problems with the sample web server application. I noticed that I could make the sample app lock up by holding down Ctrl-R in the browser. I wrote a more robust one that you can use here: https://github.com/kzahel/web-server-chrome