Open URL in Chrome & save its source code using Command prompt - google-chrome

I am having a hard time to find how to save the page as html or .txt using command line in Chrome Browser,
This is what I've done so far,
C:\Users\Cipher\AppData\Local\Google\Chrome\Application>chrome.exe --new-window
http://google.com
This command will open a new window of Chrome browser and visit google.com but i couldn't be able to figure our how can i save google.com as html or as txt file ,
is there anyway to do so using command prompt ?

You cannot perform the task you describe manually, but you can perform it using WebDriver automation.
Chrome can be remote controlled using an API called WebDriver (part of Selenium 2 automating suite). WebDrive has bindings for various programming languages, including e.g. JavaScript and Python.
Here is example code for Python (not tested):
from selenium import webdriver
driver = webdriver.Chrome('/path/to/chromedriver') # Optional argument, if not specified will search path.
driver.get('http://www.google.com/');
html = driver.page_source
f = open("myhtml", "wt")
f.write(html)
f.close()
Orignal example

Do you really need to open Google Chrome? You can get the page source using Wget (available for UNIX systems or for Windows in this post on SuperUser). Once installed, just use the following command:
wget http://google.com -O yourfilename.html
And this should be all :) I don't think there's a way to tell Chrome to download the HTML from the command line though :(
UPDATE: There's a repo on GitHub called chrome-cli that allows the user to control Chrome from the command line. Downside is that it only works on Mac OS X.

I created a small script to do perform exactly this task: https://github.com/abiyani/automate-save-page-as . See the demo gif in the README.
It automates the keyboard actions you would otherwise perform to save the page manually (literally sends those key signals to OS). As a side effect of it being used in another project of mine, it's been tested on various linux flavors: Ubuntu, Mint, Fedora, etc - and works fine on all of them. It probably won't work (at least without modifications) on Mac, and certainly not on Windows.

This should work :
cd c:\Program Files (x86)\Google\Chrome\Application
c:\Program Files (x86)\Google\Chrome\Application>chrome.exe --headless --dump-dom --enable-logging --disable-gpu https://www.google.com >c:\yourpath\yourfile.html

Related

Alternative to "Web Server for Chrome"?

I need help with an alternative to "Web Server for Chrome" extension provided by Chrome. I don't even want to download chrome let alone using it so is there any alternative to run HTMl and JS files on a web server? I want something very simple which does not require extra coding or effort. please and thank you.
You should install a server software for this purpose like XAMPP server. No need to download a specific browser as long as you have the server software.
Maybe you can do the following (if you have python installed ):
Open command prompt
Type cd your_path_to_folder_in_which_file_is_located
Type the command : python -m http.server
Hit enter
Open browser and enter localhost:8000
Done! to cancel the server , hit CTRL+C
If you do not have python then you can install Web Server for Chrome on any other browser . Edge , firefox .Simply go into the chrome web store and search the same . You should be able to add it the usual way.

Is there an option to enable json-loading from HDD in Chrome or Internet Explorer?

Is there an option to enable Chrome or the Internet Explorer to load (via ajax-get) an JSON file from the local harddisk?
Firefox does so by default. Internet Explorer says 'premission denied' at the line
request.open(method, url, true);
There are no simple cleaner way you can do this, you have to do some server-side coding.
I suggest you node.js this is very handy for this type of IO operations.
It looks very difficult for first time but it is very handy.
Download and install nodejs.
create a simple project to handle this.
In the directory you want files to get (e.x. D:\myfiles), create one file server.js, content of that js file would be following
var connect = require('connect');
var serveStatic = require('serve-static');
connect().use(serveStatic("./")).listen(105);
open command promp from and get to this directory (D:\myfiles)
Run these two commands npm install connect and npm install serve-static (up to this, is one time process)
Run node server.js - That's all your server is ready and running
call http://localhost:105/<file name> and you can get any files within that directory.
For chrome you can use following command line flags, taken from following question
--allow-file-access-from-files
--disable-web-security
As noted in that question you need to restart your all chrome browsers.
For internet explorer, trick is to use ActiveX version of the XMLHttpRequest object.
See following link.
https://social.msdn.microsoft.com/Forums/ie/en-US/9ae077e0-a7b9-433f-835f-2643aa1a7e09/ie11previewcannot-read-local-files-with-ajax-method-of-jquery?forum=iewebdevelopment

Open google chrome (all users installation) in kiosk mode from terminal

I execute this in order to open Google chrome in kiosk mode:
"C:\Users\Javier\AppData\Local\Google\Chrome\Application\chrome.exe" --user-data-dir=$(mktemp -d) --kiosk "url"
but, what I need to modify if google chrome is installed in Program Files(x86)? If I execute:
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe --user-data-dir=$(mktemp -d) --kiosk "url"
I get an error in --user-data-dir=$(mktemp -d) sentence.
Thanks in advance.
Javier
I think there may be no elegant solution at this point. "mktemp -d" is a linux/unix command and I can't find it, or an equivalent, on windows. It may have worked at one time (I stumbled on mktemp on a microsoft development page), but on 8.1 it didn't. The error I was getting said it had to do with permissions, but that wasn't the case.
If someone could get commands in Cygwin working at the windows command prompt then maybe the above would work. Another option may be to create something for PowerShell that does the same and plug that in there.
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --user-data-dir=c:\somedir --kiosk "some.url.here" does work though. It works from a shortcut, so it should also work from the command line, but you do lose the random temp directory.
For my case, I just wanted some websites/web-apps to default full screen (F11) from a shortcut and I'd just Alt-F4 to close, and Alt-Tab between them and the desktop. As it is, I'm thinking I'll have to make a separate data-dir for each one I want to do.
I was using the command chrome.exe --user-data-dir=$(mktemp -d) --kiosk http://someUrl/ and it was working well for over a week until I started testing my program with a limited privilege user today. I noticed that chrome process itself did not start after calling process.start() method. There were no entries in event logs to see what went wrong. After looking at your solution, I changed my program to generate a temp folder using C# command pass it as an argument and it started working. Thanks for the help. This is the simplified version of the code I use.
DirectoryInfo tempDirectory = Directory.CreateDirectory(Path.Combine(Path.GetTempPath(), "Chrome" + DateTime.Now.ToString("ddMMHHmmss")));
Process process = new Process();
process.StartInfo.FileName = GetBrowserPathFromRegistry();
process.StartInfo.Arguments = String.Format("--user-data-dir={0} --kiosk {1}", tempDirectory.FullName, this.uri);
process.Start();

Launch Portable Google Chrome with supplied parameters from flash drive on Mac OSX

As the title suggests, I want to write a shell command to open google chrome portable (with supplied parameter) which is installed on the flash drive also. I am trying to do it as follows:
open "/Volumes/NDCS/chrome/chromium.app/Contents/MacOS/Portable Chromium" --allow-file-access-from-files
Portable Chrome opens up, but the setting hasn't worked. I also wanted it to launch a default homepage (parameter is --homepage="[url or file path]", but for now have done that by setting the default homepage in the browser.
If anyone has any knowledge/advice on this, would be much appreciated. I have done the same scenario on PC recently, just struggling with the Mac deployment.
From man open you need to supply thr parameters to the executable using the --args argument, otherwise they are taken as arguments to open not the app.
Also Open works on the App bundle
So
open "/Volumes/NDCS/chrome/chromium.app" --args --allow-file-access-from-files
oruse the plain unix executable as you are doinf without the open
"/Volumes/NDCS/chrome/chromium.app/Contents/MacOS/Portable Chromium" --allow-file-access-from-files

Any way to start Google Chrome in headless mode?

I carefully revised the list of switches at http://peter.sh/experiments/chromium-command-line-switches/#chrome-frame and I couldn't find anything that would launch Chrome in a hidden background process.
The closest I was able to is --keep-alive-for-test + custom packaged app, but the app fails to execute any passed code because (the way it reports) "no window - ChromeHidden".
TL;DR
google-chrome --headless --remote-debugging-port=9222 http://example.com
You'd also need --disable-gpu temporarily.
Tutorial:
https://developers.google.com/web/updates/2017/04/headless-chrome
There's a work in progress: https://code.google.com/p/chromium/issues/detail?id=546953
The main deliverables are:
A library which headless applications can link to to.
A sample application which demonstrates the use of headless APIs.
So it would be possible to create a simple application that runs in console without connecting to display.
Update Apr 18 '16: The work is mainly done. There's a public forum now:
https://groups.google.com/a/chromium.org/forum/#!forum/headless-dev
Documentation is being in progress:
https://chromium.googlesource.com/chromium/src/+/master/headless/README.md
Update Sep 20 '16: It looks like chrome will eventually get the "--headless" parameter:
https://bugs.chromium.org/p/chromium/issues/detail?id=612904
There was a presentation on BlinkOn 6 (June 16/17, 2016)
Update Nov 29 '16: Design doc for --headless flag: https://docs.google.com/document/d/1aIJUzQr3eougZQp90bp4mqGr5gY6hdUice8UPa-Ys90/edit#heading=h.qxqfzv2lj12s
Update Dec 13 '16: --headless flag is expected to be available in Canary builds soon
Update Mar 12 '17: Chrome 57 has a --headless flag working. Waiting for Selenium and other tools to catch up. User guide: https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md
This guy managed to run Chrome headlessly by using Xvfb (X virtual frame buffer) to trick Chrome into thinking it was displaying a window:
http://e-method.blogspot.fr/2010/11/google-chrome-with-xvfb-headless-server.html
If you're on Linux you could try that.
So basically you need to install X virtual frame buffer and Google Chrome via:
root#localhost: ~# apt-get install xvfb imagemagick
root#localhost: ~# apt-get install google-chrome
Then run the browser on the display:
root#localhost: ~# xvfb-run --server-args='-screen 0, 1024x768x24' \
google-chrome -start-maximized http://www.example.com \
> & /dev/null &
root#localhost: ~# DISPLAY=:99 import -window root myimage.png
Or you can look at PhantomJS project which is a headless WebKit implementation.
You could set up a linux VM and use xvfb in it.
Installation on debian / ubuntu:
sudo aptitude install xvfb
Start Chrome headless and visit http://example.com :
xvfb-run --server-args='-screen 0, 1024x768x16' google-chrome
-start-maximized http://example.com > /dev/null &
Turns out it starts in headless mode if you start it as a child subprocess. Besides that:
nircmd.exe can do win hide on chrome based on its PID
Autohotkey_L can also start Chrome hidden without a taskbar button
The Chromium Embedded Framework project seems like it might fit your usecase. I don't have personal experience with the project, but I've heard good things, and it has a solid API that you should be able to exploit for your purposes.
I don't have enough reputation to comment yet, but want to let you guys know that the chrome headless mode which Vanuan mentions actually works with Selenium webdriver.
In Java you can pass the flag to chrome through chromeDriver with the following code:
ChromeOptions options = new ChromeOptions();
options.addArguments("--headless");
ChromeDriver chromeDriver = new ChromeDriver(options);
I've recently found this article which mentions several commandline options that seem to do it. Using these keywords I googled out this piece of code which seem to confirm that these options exist.
// Does not automatically open a browser window on startup (used when
// launching Chrome for the purpose of hosting background apps).
const char kNoStartupWindow[] = "no-startup-window";
// Causes Chrome to launch without opening any windows by default. Useful if
// one wishes to use Chrome as an ash server.
const char kSilentLaunch[] = "silent-launch";
I managed to successfuly run Chrome with --no-startup-window and indeed it launched without any windows. It looked like it launched properly, it spawned all typical children, but the website I tried to make it load inside didn't seem to be actually visited. It maybe possible that this headless mode is only for running apps and not for visiting sites headless*), but it looks very promising as the normal worker tree is set up, just no windows.
The second option --silent-launch made chrome process very silent. I didn't notice any children spawned and the process exited promptly. I doubt it'll be usable for this case.
After I failed my attempts with these options, I focused on less sophisticated ways. On the bottom of the list there are two options:
// Specify the initial window position: --window-position=x,y
const char kWindowPosition[] = "window-position";
// Specify the initial window size: --window-size=w,h
const char kWindowSize[] = "window-size";
I ran Chrome with options to move it completely out of the working area:
--window-size=800,600 --window-position=-800,0
and as dirty as it feels, sure it's no true headless, but still the window is out of my sight, and everything done just with chrome's startup options, without external tools sending low-level window-hide messages.
*) yes, I know try to do weird things. Essentially I tried to get rid of Chrome window that's kept by Karma during tests. I know I can switch to PhantomJS, but I specifically needed to run them in Chrome, and the window popping out was.. eh.. disturbing in the long run.
It is currently under development, you can read more information about it from here:
https://chromium.googlesource.com/chromium/src/+/master/headless/README.md
Headless Chromium is a library for running Chromium in a
headless/server environment. Expected use cases include loading web
pages, extracting metadata (e.g., the DOM) and generating bitmaps from
page contents -- using all the modern web platform features provided
by Chromium and Blink.
It currently works on Linux, there is a nice presentation.
I was also able to make chrome headless work with NightwatchJS. Here is the config that let me use it:
"chromeHeadless": {
"desiredCapabilities": {
"browserName": "chrome",
"chromeOptions": {
"args": ["--headless"],
"binary": "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
}
}
}
Chrome 59 has the ability to create instance as headless.
Find the below tutorial
https://www.automation99.com/2017/07/how-to-use-chrome-headless-using.html?m=1