Easier way to download chrome driver - google-chrome

I have a set of tests that downloads the latest Google Chrome from https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb and then parses the contents on https://chromedriver.chromium.org/downloads to figure out the right chrome driver version to download. The Chrome version eg: 102.0.5005.115, does not always match the chrome driver version, eg: 102.0.5005.61.
Currently, I parse the contents of https://chromedriver.chromium.org/downloads and try to figure out the right version. This is brittle as the HTML contents may change.
#!/bin/sh
GOOGLE_CHROME_VERSION="$(google-chrome --version | cut -f 3 -d ' ')"
# https://www.chromium.org/developers/version-numbers/
GOOGLE_CHROME_VERSION_MAJOR=$(echo "${GOOGLE_CHROME_VERSION}" | cut -f 1 -d'.')
echo "GOOGLE_CHROME_VERSION_MAJOR => [${GOOGLE_CHROME_VERSION_MAJOR}]"
# https://chromedriver.chromium.org/downloads
GOOGLE_CHROME_DRIVER_LINE=$(curl https://chromedriver.chromium.org/downloads | grep "If you are using Chrome version ${GOOGLE_CHROME_VERSION_MAJOR}," | head -1)
GOOGLE_CHROME_DRIVER_VERSION=$(echo "${GOOGLE_CHROME_DRIVER_LINE}" | rev | cut -f 1 -d' ' | rev)
echo "GOOGLE_CHROME_DRIVER_VERSION => [${GOOGLE_CHROME_DRIVER_VERSION}]"
wget https://chromedriver.storage.googleapis.com/${GOOGLE_CHROME_DRIVER_VERSION}/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
rm chromedriver_linux64.zip
chmod a+x chromedriver
mv chromedriver /usr/local/bin
Is there a simpler way to download a chrome driver for a specific version of google chrome?

https://chromedriver.chromium.org/downloads/version-selection
I hope this helps. Apparently, it supports some automated URL resolving:
First, find out which version of Chrome you are using. Let's say you have Chrome 72.0.3626.81.
Take the Chrome version number, remove the last part, and append the result to URL "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_". For example, with Chrome version 72.0.3626.81, you'd get a URL "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_72.0.3626".
Use the URL created in the last step to retrieve a small file containing the version of ChromeDriver to use. For example, the above URL will get your a file containing "72.0.3626.69". (The actual number may change in the future, of course.)
Use the version number retrieved from the previous step to construct the URL to download ChromeDriver. With version 72.0.3626.69, the URL would be "https://chromedriver.storage.googleapis.com/index.html?path=72.0.3626.69/".
After the initial download, it is recommended that you occasionally go through the above process again to see if there are any bug fix releases.

Related

Headless Chrome crashes when converting large HTML file to PDF on Heroku server

I’m trying to use Chrome (version 78.0.3904.108) from the Heroku buildpack for Google Chrome in headless mode to create a PDF of an HTML file on a Heroku “hobby” dyno.
It works fine for small files. But when I try to convert a 428 KiB HTML file it crashes after a few seconds. The files all contain a lot of images that are also referenced through file:// URLs. I used the same Chrome version to convert the large file on my computer, and it worked without a problem.
The invocation goes as follows:
~ $ $HOME/.apt/opt/google/chrome/chrome \
> --headless \
> --no-sandbox \
> --disable-gpu \
> --print-to-pdf="/tmp/test.pdf" \
> file:///tmp/large-file.html
[1127/175233.676324:ERROR:broker_posix.cc(46)] Received unexpected number of handles
[1127/175233.694403:ERROR:print_render_frame_helper.cc(1785)] Printing failed.
[1127/175233.695067:ERROR:headless_shell.cc(562)] Print to PDF failed
The ulimit is unlimited, there is space left in /tmp, and /tmp is writeable.
Does Chrome start a lot of processes or threads so that it exceeds these limits? Or is ulimit uninformative for some container-related reason? Or what else could be the culprit?
Update: It works with a standard-2x dyno but not with standard-1x and below. So whatever the limitation is, it must be one that is different between those two dyno types.

How to disable Google Chrome extension autoupdate

How do I disable Google Chrome extension autoupdate?
Solutions I've found for this:
1. Disabling a concrete extension update
That's what I wanted!
You can do this by editing the extension's manifest.json file:
On Windows: C:\Users\<USERNAME>\AppData\Local\Google\Chrome\User Data\Default\Extensions\<EXTENSION-ID>\<VERSION>\manifest.json (find out the extension's ID by enabling Developer Mode in the extension settings page)
On MacOS: Open /Users/USERNAME/Library/Application Support/Google/Chrome/Default/Extensions/EXTENSION-ID/VERSION/manifest.json in a text editor.
On Ubuntu for Chromium: ${HOME}/.config/chromium/Default/Preferences
In this file, set the "update_url" property to something invalid like "https://localhost" for example. For the given url, it makes auto-updating that extension as simply impossible.
Source: https://productforums.google.com/d/msg/chrome/l3zOZeO-5-M/Y7VaR0KCWNIJ
2. Disabling all Google Chrome extension updates
For any OS: Just type chrome://plugins/ at address bar and turn Google Update plugin off. Source: How to disable Google Chrome auto update?
For Windows OS: Set Registry values:
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Update]
"AutoUpdateCheckPeriodMinutes"=dword:00000000
"UpdateDefault"=dword:00000000
Source: Making Google Chrome leave itself alone
If the chrome extension is on Github (which many if not most of them are), you can simply:
(1.) clone the Github repo,
(2.) reset the head to the version that you want, and
(3.) enable Developer Mode at chrome://extensions/
(4.) select the "Load unpacked" option from chrome://extensions/, and then select the folder enclosing the source code for the extension.
I recently used this technique to downgrade my version of Reddit Link Opener, which no longer supports users who have opted out of using that site's redesign. This worked for me on MacOS, but should work on all platforms.
If the extension is loaded as an unpacked extension (in the manner described above), it will NOT auto-update to a newer version.
Disabling update for a specific extension:
This can be achieved with the system policies, (more details here)
For Linux :
Get the installed extensions list (IDs), this can be found with ls -l ~/.config/google-chrome/Default/Extensions or chrome://extensions
Create the necessary directory if not present mkdir -p /etc/opt/chrome/policies/managed (with root)
Create the needed file policies file touch /etc/opt/chrome/policies/managed/google-chrome.json
Edit that file with the code bellow
open the page chrome://policy/ and reload the policies
{
"ExtensionSettings": {
"ghijklmnopabcdefghijklmnopabcdef": {
"update_url": "https://127.0.0.1/update_url",
"override_update_url": true
},
"YOUR-EXTENSION-ID-LIKE-THE-PREVIOUS-EXAMPLE": {
"update_url": "https://127.0.0.1/update_url",
"override_update_url": true
}
}
}
Note: this can not be applied widely to all extensions in a single rule and also for each newly installed extension the file need to be updated
Hi all those solitions for me have one disadvantage is that all extensions have no updates, I needed to stop only for one extension in this case and wanted al the other to keep making updates.
I think I found the solutuion for windows
Go to
C:\Users\YOUR_NAME_HERE\AppData\Local\Google\Chrome\User Data\Default\Extensions\YOUR_FOLDER APP HERE\
In that folder app click in properties and select read only an aplly that to all subfolders and files... for now for me solved the problem !!!
Regards xichas
this is a complementary answer to the accepted one https://stackoverflow.com/a/27657703/1422630 , allowing disable all at once on chromium
this is also only for linux (may be run on windows thru cygwin tho, not tested..)
this script will
backup the prefs file,
modify it,
if didnt succeed will output "FAILED"
show the differences using meld if installed
#!/bin/bash
set -ue
strPref="$HOME/.config/chromium/Default/Preferences"
cat "$strPref" |egrep "\"update_url[^,]*," -o |sort -u
read -p "existing unique urls above..." -n 1
strBkp="${strPref}.`date +"%Y%m%d%H%M%S"`.bkp"
if cp -v "$strPref" "$strBkp";then
strUpdUrl="clients2.google.com/service/update2/crx" #change this if needed #TODO should match any URL...
sed -i -r "s#(update_url\":\"https{,1}://)(${strUpdUrl})#\1127.0.0.1#g" "$strPref"
if grep "$strUpdUrl" "$strPref";then echo FAILED >&2;exit 1;fi
cmdDiff=colordiff
if which meld;then cmdDiff=meld;fi
#$cmdDiff <(cat "$strPref" |egrep "\"update_url[^,]*," -o) <(cat "$strBkp" |egrep "\"update_url[^,]*," -o)
$cmdDiff <(cat "$strPref" |sed -r 's#","#",\n"#g') <(cat "$strBkp" |sed -r 's#","#",\n"#g')
fi
tested on chromium: Version 63.0.3239.84 (Official Build) Built on Ubuntu , running on Ubuntu 16.04 (64-bit)
obs.: that script also works for google-chrome, just change the preferences file path
After updating Google Chrome to v60, no solution found on the Internet has helped me
So i just blocked IP addresses, used for updating, by doing following steps:
Opened Chrome with blank browser tab
Waited, until extension
autoupdate begins, by looking on to network tab in Resource
Monitor
Wrote out all the IP addresses with high download rate. My IP address list was:
64.233.161.94
64.233.161.102
64.233.163.95
74.125.238.132
108.177.14.138
173.194.73.132
173.194.222.102
216.58.209.110
216.58.209.97
173.194.222.99
173.194.32.227
173.194.113.172
173.194.32.224
195.216.237.77
74.125.232.170
143.215.130.61
74.125.238.147
173.194.122.137
173.194.44.66
173.194.44.67
173.194.44.95
173.194.122.136
74.125.232.183
74.125.232.171
Created outbound rule for chrome.exe in Windows Firewall and added listed IP addresses to blocklist
After I enabled this rule, chrome was unable to update my extensions.
Just (re)install the extension via Load unpacked.
Let's suppose "Roboform Password Manager" extension version 8.6.5.5 dropped some important functionality, so you want to keep version 8.6.2.2 installed.
Go to chrome://extensions/
Enable Developer mode
Get the required version of the plugin:
If Chrome still got the version you need:
Utilize Pack extension button on the plugin details page.
Just copy the extension folder, e.g. C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default\Extensions\pnlccmojcmeohlpggmfnbbiapkmbliob. The extension id is visible in the url bar, on the plugin details page, e.g. chrome://extensions/?id=pnlccmojcmeohlpggmfnbbiapkmbliob.
If the version you need was overwritten already:
Get appropriate ".crx" from some extensions archive
Look for ".crx" in "C:\Program Files\..." (applications/installers sometimes bundle original ".crx" versions, unaffected by any updates)
Unzip (e.g. with 7-zip) your ".crx" (or paste the extension folder contents) to a non-temporary folder - you would have to keep those files in place until you uninstall the extension.
Click Load unpacked, select that folder.
If you just drag&drop the ".crx" file, Chrome extension details would show Source=Chrome Web Store, and it would get updated as soon as you click Update extensions now. But for an unpacked extension, you get a special "Unpacked extension" overlay icon, Source=Unpacked extension and it won't get updated.
Just tested on Chrome 79.0.3945.88 (64-bit), Windows.
Now, Chrome shows "Disable developer mode extensions" popup on each startup. Personally, I just manually dismiss them each time. I do not re-start Chrome too often.

How to convert bash file to a binary executable

I created a binary executable from bash script on linux server through SHC. The binary created works fine on linux machines, but through mistake on Mac. How could I convert my bash file to binary executable that is able to run everywhere(ubuntu, CentOS, Mac, Cygwin)?
shc -v -r -T -f ir16fetcher.sh
mv ir16fetcher.sh.x ir16fetcher
Shebang of my bash script
#!/bin/bash
On Linux machines
./ir16installer
USAGE : ir16fetcher <servername/ip address> [the n th latest build - optional. Default 1]
EXAMPLE: ir16fetcher jagger 2
EXAMPLE: ir16fetcher 167.116.6.155
REQUIRE: Please make sure conf file in installation folder ~/IRinstall/ir16 & ~/IRinstall/irmanager
On my Mac
./ir16installer
-bash: ./ir16installer: cannot execute binary file
I think it's not gonna work
"The compiled binary will still be dependent on the shell
specified in the first line of the shell code (i.e.
#!/bin/sh), thus shc does not create completely independent
binaries."
From http://www.datsi.fi.upm.es/~frosal/sources/shc.html
You will have to do this for every architecture and operating system you need to support. In any case, there doesn't really seem to be any benefits of using this method for distribution. It adds dependencies and complicates delivery, and I'm pretty sure whatever obfuscation the "shc" compiler implements is easily reversed.
if the goal here is to "hide" your source code, and then have the "hidden" copy of the code be executable on the Unix OSes you listed, then, encryption is really your only option.
I say this because encryption tools are available on every base Unix install. For your purposes, this is a very good thing as you wont have to download or configure anything additional. They're just there, as part of the natural installation of the OS. One of such tools is called openssl.
To Encrypt your file/script with openssl:
echo precious-content | openssl aes-128-cbc -a -salt -k mypassword
U2FsdGVkX1+K6tvItr9eEI4yC4nZPK8b6o4fc0DR/Vzh7HqpE96se8Fu/BhM314z
To Decrypt your file/script with openssl:
echo U2FsdGVkX1+K6tvItr9eEI4yC4nZPK8b6o4fc0DR/Vzh7HqpE96se8Fu/BhM314z | openssl aes-128-cbc -a -d -salt -k mypassword
precious-content
Now, to get openssl to do what you want it to do automatically without having to spend hours of your own time figuring out a way, you can paste your script to a site like www.EnScryption.com. This site will generate an "executable" version of your code for you, which you can then run on any Mac, Ubuntu, RedHat, CentOS box.

Open URL in Chrome & save its source code using Command prompt

I am having a hard time to find how to save the page as html or .txt using command line in Chrome Browser,
This is what I've done so far,
C:\Users\Cipher\AppData\Local\Google\Chrome\Application>chrome.exe --new-window
http://google.com
This command will open a new window of Chrome browser and visit google.com but i couldn't be able to figure our how can i save google.com as html or as txt file ,
is there anyway to do so using command prompt ?
You cannot perform the task you describe manually, but you can perform it using WebDriver automation.
Chrome can be remote controlled using an API called WebDriver (part of Selenium 2 automating suite). WebDrive has bindings for various programming languages, including e.g. JavaScript and Python.
Here is example code for Python (not tested):
from selenium import webdriver
driver = webdriver.Chrome('/path/to/chromedriver') # Optional argument, if not specified will search path.
driver.get('http://www.google.com/');
html = driver.page_source
f = open("myhtml", "wt")
f.write(html)
f.close()
Orignal example
Do you really need to open Google Chrome? You can get the page source using Wget (available for UNIX systems or for Windows in this post on SuperUser). Once installed, just use the following command:
wget http://google.com -O yourfilename.html
And this should be all :) I don't think there's a way to tell Chrome to download the HTML from the command line though :(
UPDATE: There's a repo on GitHub called chrome-cli that allows the user to control Chrome from the command line. Downside is that it only works on Mac OS X.
I created a small script to do perform exactly this task: https://github.com/abiyani/automate-save-page-as . See the demo gif in the README.
It automates the keyboard actions you would otherwise perform to save the page manually (literally sends those key signals to OS). As a side effect of it being used in another project of mine, it's been tested on various linux flavors: Ubuntu, Mint, Fedora, etc - and works fine on all of them. It probably won't work (at least without modifications) on Mac, and certainly not on Windows.
This should work :
cd c:\Program Files (x86)\Google\Chrome\Application
c:\Program Files (x86)\Google\Chrome\Application>chrome.exe --headless --dump-dom --enable-logging --disable-gpu https://www.google.com >c:\yourpath\yourfile.html

How do I create binary patches?

What's the best way to go about making a patch for a binary file?
I want it to be simple for users to apply (a simple patch application would be nice). Running diff on the file just gives Binary files [...] differ.
Check out bsdiff and bspatch (website, manpage, paper, GitHub fork).
To install this tool:
Windows: Download and extract this package. You will also need a copy of bzip2.exe in PATH; download that from the "Binaries" link here.
macOS: Install Homebrew and use it to install bsdiff.
Linux: Use your package manager to install bsdiff.
Courgette, by the Google Chrome team, looks like most efficient tool for binary patching executables.
To quote their data:
Here are the sizes for the recent 190.1 -> 190.4 update on the developer channel:
Full update: 10,385,920 bytes
bsdiff update: 704,512 bytes
Courgette update: 78,848 bytes
Here are instructions to build it. Here is a Windows binary from 2018 courtesy of Mehrdad.
xdelta (website, GitHub) is another option. It seems to be more recent, but otherwise I have no idea how it compares to other tools like bsdiff.
Usage:
Creating a patch: xdelta -e -s old_file new_file delta_file
Applying a patch: xdelta -d -s old_file delta_file decoded_new_file
Installation:
Windows: Download the official binaries.
Chocolatey: choco install xdelta3
Homebrew: brew install xdelta
Linux: Available as xdelta or xdelta3 in your package manager.
Modern port: Very useful .NET port for bsdiff/bspatch:
https://github.com/LogosBible/bsdiff.net
My personal choice.
I tested it, and it was the only one of all links. Out of the box I was able to compile it (with Visual Studio, e.g., Visual Studio 2013). (The C++ source elsewhere is a bit outdated and needs at least a bit polishing and is only 32 bit which sets real memory (diff source size) limits. This is a port of this C++ code bsdiff and even tests if the patch results are identical to original code.)
Further idea: With .NET 4.5 you could even get rid of the #Zip library, which is a dependency here.
I haven't measured if it is slightly slower than the C++ code, but it worked fine for me, (bsdiff: 90 MB file in 1-2 minutes), and time-critical for me is only the bspatch, not the bsdiff.
I am not really sure, if the whole memory of a x64 machine is used, but I assume it. The x64 capable build ("Any CPU") works at least. I tried with a 100 MB file.
-
Besides: The cited Google project 'Courgette' may be the best choice if your main target are executable files. But it is work to build it (for Windows measures, at least), and for binary files it is also using pure bsdiff/bspatch, as far as I have understood the documentation.
For small, simple patches, it's easiest just to tell diff to treat the files as text with the -a (or --text) option. As far as I understand, more complicated binary diffs are only useful for reducing the size of patches.
$ man diff | grep -B1 "as text"
-a, --text
treat all files as text
$ diff old new
Binary files old and new differ
$ diff -a old new > old.patch
$ patch < old.patch old
patching file old
$ diff old new
$
If the files are the same size and the patch just modifies a few bytes, you can use xxd, which is commonly installed with the OS. The following converts each file to a hex representation with one byte per line, then diffs the files to create a compact patch, then applies the patch.
$ xxd -c1 old > old.hex
$ xxd -c1 new > new.hex
$ diff -u old.hex new.hex | grep "^+" | grep -v "^++" | sed "s/^+//" > old.hexpatch
$ xxd -c1 -r old.hexpatch old
$ diff old new
$
This is a simpler, cleaner, better version suggested by bmaupin that uses process substitution instead of intermediate files, diff, and grep:
$ comm -13 <(xxd -c1 old) <(xxd -c1 new) > old.hexpatch
$ xxd -c1 -r old.hexpatch old
$ diff old new
$
Here the comm -13 removes lines that appear only in the first input as well as lines that appear in both inputs, leaving only the lines exclusive to the second input.
HDiffPatch can run on Windows, macOS, Linux, and Android.
It supports diffs between binary files or directories;
Creating a patch: hdiffz [-m|-s-64] [-c-lzma2] old_path new_path out_delta_file
Applying a patch: hpatchz old_path delta_file out_new_path
Install:
Download from last release, or download the source code & make;
Jojos Binary Diff is another good binary diff algorithm;
diff and git-diff can handle binary files by treating them as text with -a.
With git-diff you can also use --binary which produces ASCII encodings of binary files, suitable for pasting into an email for example.
https://github.com/reproteq/DiffPatchWpf
DiffPatchWpf
DiffPatchWpf simple binary patch maker tool.
Compare two binary files and save the differences between them in new file patch.txt
Apply the patch in another binary fast and easy.
Now you can apply the differences in another binary quickly and easily.
example:
1- Load file Aori.bin
2- Load file Amod.bin
3- Compare and save Aori-patch.txt
4- Load file Bori.bin
5- Load patch Aori-patch.txt
6- Apply patch and save file Bori-patched.bin
alt tag
https://youtu.be/EpyuF4t5MWk
Microsoft Visual Studio Community 2019
Versión 16.7.7
.NETFramework,Version=v4.7.2
Tested in windows 10x64bits
Assuming you know the structure of the file you could use a C / C++ program to modify it byte by byte:
http://msdn.microsoft.com/en-us/library/c565h7xx(VS.71).aspx
Just read in the old file, and write out a new one modified as you like.
Don't forget to include a file format version number in the file so you know how to read any given version of the file format.