Problem with content encoding and connecting with DevTools to chrome headless - google-chrome

I am starting a headless chrome instance like that, e.g.
/opt/google/chrome/chrome --headless --disable-gpu --remote-debugging-port=9222 https://www.chromestatus
and want to connect with DevTools to this instance by entering localhost:9222 into a running browser. However, what I see is just some garbled stuff like this here "��TQo�0~ϯ���$���li51����manri̜8�������m::�...", probably because the server process at localhost:9222 sends the response gzipped, but does not include a content-encoding in the response headers (at least, that's my theory why this happens...)
Any idea how I can fix this? I haven't found any option to control this behavior (e.g. to tell the headless chrome instance to not zip the response data...)
BTW: I see the same whether I enter the url in chrome, chromium or firefox or use wget, and also whether I use chrome (Google Chrome 84.0.4147.105) or chromium doesn't make a difference...

Related

Chrome access/request log

I'm looking for a way to output requests and their response codes from Chrome. Basically I would like what is in the Network tab in developer tools in a log file or accessible from outside of Chrome.
I tried starting Chrome with --enable-logging --v=1 but I cannot find any traces of accessed urls.
Is there a way I can get that logging?

Keep Chrome running in headless mode

I want to use Chrome browser in headless mode to produce images (PNG, JPEG) out of SVG graphic. Code works in normal interactive mode, but I have problems to use it in headless mode.
My main problem is that headless Chrome exits before drawing of HTML page is completed. As I understand, if I start Chrome with following arguments:
chromium --headless http://myserver.org
It exits together with document.onload event. But at this moment not all data fetched from the server (I using XMLHttpRequest) and therefore drawing is not complete.
I found workaround if I start chrome with debugging port enabled like:
chromium --headless --remote-debugging-port=7777 http://myserver.org
But this is not that I want, especially when I do not have privileges to open http ports on the node. Is there possibility to let Chrome running longer with other flags? I check a lot of them, but did not found appropriate one. Or is there any other methods to postpone exit of the headless Chrome?
You could try this answer https://stackoverflow.com/a/46424041/4830701
Copy pasted here for reference
Use the binary /opt/google/chrome/chrome directly not google-chrome which points to bash script /usr/bin/google-chrome.
Taken from comments in
https://developers.google.com/web/updates/2017/04/headless-chrome#screenshots

IntelliJ IDEA and Google Chrome fail to connect for JavaScript debugging

JetBrains IntelliJ IDEA sometimes loses its own JetBrains IDE Support plugin in Chrome. Upon debugging in IDEA, Chrome complains that:
hmhgeddbohgjknpmjagkdomcpobmllji is blocked
Requests to the server have been blocked by an extension
and stays on this page:
chrome-extension://hmhgeddbohgjknpmjagkdomcpobmllji/loading.html?__QUERY__
Where hmhgeddbohgjknpmjagkdomcpobmllji is JetBrains' plugin ID.
Instead of redirecting to the target page of the form:
http://localhost:63342/__PROJECT__/__PATH__/index.html
Restarting Google Chrome or disabling-and-enabling IDE Support plugin helps. IDEA "finds" Chrome again and Debug workflow works as intended.
Another possible problem is a failed authorization.
To debug a URL like this:
http://localhost:63342/__PROJECT__/__PATH__/index.html
Open it in the browser before starting to debug. It'll give 404 Not Found. Then switch back to IDEA and it will offer you to Copy authorization URL to clipboard:
http://localhost:63342/__PROJECT__/__PATH__/index.html?_ijt=__KEY__
Open this URL.
After this manual authentication is complete, IntelliJ IDEA should be able to debug index.html in Chrome.
IDEA always opens index.html for debugging in the browser window where the latest authentication happened. E.g., in a Chrome Incognito window, instead of a Chrome "normal" window, if you've just authorized IDEA in Incognito.
Going incognito may require an additional setup:
https://intellij-support.jetbrains.com/hc/en-us/community/posts/206201319-launching-chrome-incognito-in-intellij-12

Headless Chrome is not redirecting

We're using headless Chrome for an application testing. We have a page that refuses to redirect in headless mode but redirects perfectly fine when not running headless. The response header contains the URL to redirect to, and a 302 status code is returned.
What would cause Chrome to behave differently when running in headless mode? Is there any logging that would help me to determine why the redirect isn't working as expected?
Edit: Found some more info. It looks like I'm unable to redirect to sites with bad certs. How can I deal with ERR_CERT_AUTHORITY_INVALID when running Chrome in headless mode?
I was having the same issue and as you pointed out it happens with sites that have bad certificates. Passing the --ignore-certificate-errors flag didn't work at the time of the original post as there was a bug.
Some options to pass to have chrome-headless send more debugging info:
--log-level=0 --enable-logging --v=1 bug tracker for chrome/chromium headless not accepting the --ignore-certificate-errors flag:
bugs.chromium.org/p/chromium/issues/detail?id=721739 – karlos Dec 7
'17 at 20:15
For anyone reading now the bug was patched recently here: https://chromium.googlesource.com/chromium/src.git/+/c8f0691b18dc5d941d5b6b3c67a483da02400670
And the --ignore-certificate-errors flag should allow redirects to occur when using headless chrome for sites with bad certificates.
as to date seems like --enable-features=NetworkService solves the issue of self signed certificated in headless mode

selenium + chromedriver hangs at start, because chrome is connectting google server first

I use selenium to drive chrome to get some url, but it hangs for about 15 more seconds.
I found that chrome will try to connect www.googleapis.com, clients2.google.com or www.google.com:443. in my environment, google server is not accessible, so it hangs until the connects to google server all failed. this is disgusting. how can i fix it?
UPDATE:
I finally found a perfect solution, you can write a chrome extension and override newtab with:
"chrome_url_overrides":
{
"newtab": "newtab.html"
}
then set chrome start up page to newtab.
Becasue chrome will automatically disable extentions that is not uploaded to chrome extension store. You'd better use chromium instead.
Why not put www.googleapis.com in /etc/hoses (C:\Windows\System32\Drivers\etc\hosts on windows) and point to 127.0.0.1 so that it fails immediately?
I think this will fix the problem.
I try everything to disable chrome from connecting its servers but failed.
But I found a small trick to solve this problem:go to setting page and you can set a url when chrome starts. I set it to http://127.0.0.1:62333. Just a invalid url that failed immediately. this cannot disable chrome from connection its servers but the selenium does not hang anymore.