Summary of tools:
watir-webdriver 1.8.17
Mac OS X 10.7.3
Chrome 18.0.1025.151
I'm currently using Watir WebDriver to automate Chrome sessions across a number of websites. I need to backup the state of the web browser (cookies, cache, etc.) at certain points throughout the session. Originally, I figured I could do this with Ruby's file IO library by copying ~/Library/Application Support/Google/Chrome/Default at the necessary points. However, it does not appear that Chrome sessions created with Watir WebDriver store the needed information in this default location. How can I locate this data to back it up? Is this information stored elsewhere? Is there something other than Watir that would make this easier?
I finally have a solution!
It appears that watir-webdriver stores the browser state/user data in random path. By default this can be found here (where XXXXXX is the random identifier):
/private/var/folders/2v/vkd2v3vs5njf69m59nqzc16m0000gn/T/.com.google.Chrome.XXXXXX/Default/
Instead of relying on this default and randomized path, you can specify a precise location for the user data using the following flag:
Watir::Browser.new :chrome, :switches => %w[--user-data-dir=/path/to/user/data]
Then the cache, cookies, etc. can be backed up, deleted, etc. using Ruby's standard library. Hopefully this helps someone else.
Edit: If you are unable to find where watir-webdriver is storing your user data by default, find Chrome's process id by running watir-webdriver and top. Once you have the pid, type lsof -p <pid> into terminal to find the path to the user data.
Another thing I like to do is serialize(save) the Watir::Browser object into a file using YAML, like so:
require "yaml"
File.open("browserObj.yaml", 'w').write YAML::dump(#browser)
This browserObj.yaml file will then contain all sorts of internal details in easily readable/parseable text, including PID of whichever browser, path to temp profile, etc. Eg.
profile_dir: /tmp/webdriver-rb-profilecopy20121201-1981-9o9t9a
Related
We are working on an automated system for memory performance monitoring.
We start Chrome with proper flags --trace-gc --trace-gc-verbose both on Windows7 and MacOSX High Sierra.
The output is printed and visible on the console. However capturing the output with standard stream redirection does not work:
What do we do:
MacOS X:
./Google\ Chrome --js-flags="--trace-gc --trace-gc-verbose" > log.txt 2>&1
Windows:
chrome --no-sandbox --js-flags="--trace_gc --trace_gc_verbose" > log.txt 2>&1
What gets written into the log file is only a part of what you can see on the screen: all gc-related data is missing in the file although abundant in the terminal window (you can see all the data when you run the app without stream capture at all).
We suppose a new process for V8 is run and it's output is not directed to stdout, though somewhat it gets printed to the console itself.
The V8 C++ code shows no mark of such a strange redirect, pretty clean standard code, in fact one can record gc-output of node --trace-gc script.js without any problem with standard stdout/stderr. It seems Chrome adds the undesired behavior.
Question how to record Chrome gc-related data to a file both on Windows and MacOS X.
Partial, unacceptable solution on POSIX-based systems we can grab all the terminal output with script command. The dump contains all the data we need, but the solution does not cover Windows scenario (we do need tests on Windows) and in fact it does not solve the problem neither explains the reason behind the lacking data, it simply hides the problem away.
I think there are two things worth being pointed out.
The only output that is actually being redirected is Chrome-related. I mean if we redirect stderr to a file we can see the errors in this file, but only Chrome's errors not V8's.
I wanted to use script command as I tried Powershell Start-Transcript, Tee and a few other options. It has turned out (no suprises here) that Windows has no command that can be used to achieve desired effect (AFAIK). So I used Babun (http://babun.github.io/) which is basically preconfigured Cygwin. No GC log has been displayed. Only Chrome's related stuff. I think that V8's logs are not being forwarded to Cygwin.
UPDATE:
This issue may be somehow related with these:
Chrome spawns many processes and from what I can see from logs - GC logs are gathered from child process (different PID than one related with opened terminal window). I do not know how this alone could affect STDOUT redirection. It's just a fact I noticed.
I found out that an application may know if its STOUD is being redirected and handle this situation differently - e. i. using buffers without flushing when STDOUT is being redirected to a file but not if it is printed on the console.
UPDATE:
I have created a following bug: https://bugs.chromium.org/p/chromium/issues/detail?id=865876#c3
It seems like it was noticed and someone is going to take care of it.
UPDATE:
It seems like ChromeDriver can do exactly what we need - http://chromedriver.chromium.org/logging/performance-log
Another workaround would be to run Chrome with --enable-precise-memory-info and the periodically invoke window.performance.memory, but that command does not provide us with nearly as much data as ChromeDriver
I'd like to use Selenium alongside my current Chrome profile, which may or may not be in use. I'd like to be able to launch some Selenium automation that is aware of (for example) any currently set cookies from my current Chrome session. I'd also like my Selenium automation to be able to change cookies that will still persist in my local profile.
Example:
I'd like to be able to manually log into a website (without
Selenium)
I'd like to then launch some Selenium automation that
assumes I'm already logged in (which I would be)
I'd like to then make some type of change through the Selenium automation
I'd like to close out the Selenium automation and see the changes that were made reflected in my original, manually-initiated, session
I know this can technically be achieved by setting user-data-dir in ChromeOptions, however that results in the following errors:
[20644:39092:1124/205239:ERROR:cache_util_win.cc(20)] Unable to move the cache: 0
[20644:39092:1124/205239:ERROR:cache_util.cc(134)] Unable to move cache folder C:\Users\****\AppData\Local\Google\Chrome\User Data\Default\ShaderCache\GPUCache to C:\Users\****\AppData\Local\Google\Chrome\User Data\Default\ShaderCache\old_GPUCache_000
[20644:39092:1124/205239:ERROR:cache_creator.cc(134)] Unable to create cache
[20644:39092:1124/205239:ERROR:shader_disk_cache.cc(585)] Shader Cache Creation failed: -2
Have you reviewed the permissions of the directories listed in the error message? They should share the same group and have write permissions.
This should solve the messages, but your implementation looks to be correct.
Let's say I set a cookie using the setcookie() function in PHP:
setcookie('name','foo',false,'/',false);
I can see it in:
chrome://settings/cookies
However, I can not find the actual file stored on my hard disk. Can anyone tell me where this specific cookie is stored on the hard disk?
The answer is due to the fact that Google Chrome uses an SQLite file to save cookies. It resides under:
C:\Users\<your_username>\AppData\Local\Google\Chrome\User Data\Default\
inside Cookies file. (which is an SQLite database file)
So it's not a file stored on hard drive but a row in an SQLite database file which can be read by a third party program such as: SQLite Database Browser
EDIT: Thanks to #Chexpir, it is also good to know that the values are stored encrypted.
For Google chrome Version 97.0.4692.71 (Latest Release) cookies are found inside the Network folder.
There is a file called "Cookies".
Path : C:\Users\user_name\AppData\Local\Google\Chrome\User Data\Default\Network
Remember to replace user_name.
You can find a solution on SuperUser :
Chrome cookies folder in Windows 7:-
C:\Users\your_username\AppData\Local\Google\Chrome\User Data\Default\
You'll need a program like SQLite Database Browser to read it.
For Mac OS X, the file is located at :-
~/Library/Application Support/Google/Chrome/Default/Cookies
Actually the current browsing path to the Chrome cookies in the address bar is:
chrome://settings/content/cookies
On Windows the path is:
C:\Users\<current_user>\AppData\Local\Google\Chrome\User Data\<Profile 1>\Cookies(Type:File)
Chrome doesn't store each cookies in separate text file. It stores all of the cookies together in a single file in the profile folder. That file is not readable.
For Google chrome Version 56.0.2924.87 (Latest Release) cookies are found inside profile1 folder.
If you browse that you can find variety of information.
There is a separate file called "Cookies". Also the Cache folder is inside this folder.
Path :
C:\Users\user_name\AppData\Local\Google\Chrome\User Data\Profile 1
Remember to replace user_name.
For Version 61.0.3163.100
Path :
C:\Users\user_name\AppData\Local\Google\Chrome\User Data\Default
Inside this folder there is Cookies file and Cache folder.
Since the expiration time is zero (the third argument, the first false) the cookie is a session cookie, which will expire when the current session ends. (See the setcookie reference).
Therefore it doesn't need to be saved.
Chromium on Linux: it's an SQLite3 database, located at:
~/.config/chromium/Default/Cookies
Google Chrome is going to be similar, try replace with
In case you came here to find out how to see info about the cookie of a particular website in Chrome, open Inspector (press F12) navigating the website, go to the tab Application/Aplicativo and look below in the left tree, there is Storage/cookies with all info:
cookie variables
content, length
expiration dates, etc
I've got a slightly unusual scenario. A web app running on a local network can perform various operations on any file on the network it can access. At present the user copies/pastes the UNC path to the file into a text input and clicks submit.
The server retrieves the file, performs some operations and returns the results to the user.
I'd like to allow the user to browse for the file using the webpage - but I don't want to upload the file, just get the full path to it. Is this possible?
I'm aware there will be a couple of scenarios which are doomed to failure - eg browsing to a local path not a UNC share but I can cover this with some validation. There will also be scenarios when the server can access a path the user can't (this is intentional) so browsing wouldn't work here.
All users will be techies who should get the point. Of course, if there were a way to limit the browse dialog to a UNC path, that would be even better but I suspect it's impossible.
Note, we already limit support to the latest versions of the main browsers and since this is just a utility feature, limited support is acceptable.
Sorry, that can't be done. It's a security feature.
When doing webkitRequestFileSystem in window.PERSISTENT option in Google Chrome, where on my filesystem do files get written? I'd like to drop files there and have Chrome interact with them while I'm building and debugging this app.
For me, at least on Mac OSX, they're stored under /Users/USERNAME/Library/Application Support/Google/Chrome/Default/File System for me. If you're using profiles, there will be profile directories instead of Default. However, each origin's saved files/folders are obfuscated under directories that won't be easy for you to interact with.
For debugging the Filesystem API, you have a few options:
Use this extension to view/remove files.
See the tips here: http://updates.html5rocks.com/2011/08/Debugging-the-Filesystem-API
That includes viewing stored files very easily using the filesystem: URLs.
Drop the Filesystem Playground demo (http://html5-demos.appspot.com/static/filesystem/filer.js/demos/index.html) into your origin. You can use that to view/rename/delete files/folders easily.
Chrome DevTools now has support for the Filesystem API for viewing the files stored under an origin. To use that, you will need to enable the Developer Tools experiments in about:flags, restart, hit the gear in the devtools (lower right corner), and enable the 'FileSystem inspection' under the experimental tab.
Just for completeness: on linux it goes into ~/.config/google-chrome/Default/File\ System/
On Windows XP, it is here: c:\Documents and Settings\USERNAME\Local Settings\Application Data\Google\Chrome\User Data\Default\File System\.
On Windows 7, the location is C:\Users\USERNAME\AppData\Local\Google\Chrome\User Data\Default\File System.
It's not very useful to browse it because file and dir names are obfuscated (but content in files is unchanged).
As ebidel wrote the best way is using browser of filesystem: urls that incorporated into Chrome. It's excellent! You can get the url using fs.root.toURL() where fs is a FileSystem object that you get, for example, from window.webkitRequestFileSystem().
Seems like the filesystem storage is encoded to prevent exactly what was trying to do. I ended up writing a very simple file manager available here. Start up any web server (I like mongoose for its 0 setup) and go to the /filemanager.html route
I saved a file called log.txt on MAC
It ended up at
~/Library/Application\Support/Google/Chrome/Default/Storage/ext/panbljeniblfmcakpphmjmmnpcaibipi/def/File\ System/iso/p/00/
with file name 00000 and no ext
If you are using MAC OX, and you have more than one profile on your chrome, or you cannot find default in the path, replace default with profile. But depending on number of profiles you have, it could be profile 1, profile 2, etc