Capybara, Chrome Headless: File Download is not working - google-chrome

I am trying to download a file with chrome headless.
My Chrome version is 67.0.3396.87 and my chromedriver has the 2.4.
The file does not appear on my file system. As far as i researched, its a safety function of chrome headless to prevent file downloads, but which can be turned on again.
Thats what i tried to do, regarding to this Thread:
https://bugs.chromium.org/p/chromium/issues/detail?id=696481
Still nothing works. I tried different approaches with
Page.setDownloadBehavior
eg. i copied the content of comment 78 but Chrome does not respond to it, or atleast it still does not work:
def enable_chrome_headless_downloads(driver, directory)
bridge = driver.send(:bridge)
path = '/session/:session_id/chromium/send_command'
path[':session_id'] = bridge.session_id
bridge.http.call(:post, path, {
"cmd" => "Page.setDownloadBehavior",
"params" => {
"behavior" => "allow",
"downloadPath" => directory,
}
})
end
I also checked if i could download manually a file with headless chrome with
'--remote-debugging-port=9222'
but it was not possible either.
Has anyone an idea, what I could do to make it work?
Thanks already!

Related

How to add bookmarks in Chromium/Chrome in automated fashion

I am trying to test a Chrome extension that searches bookmarks. Puppeteer loads Chromium with a clean profile each time, which is great, but my bookmarks are empty.
I was hoping to find a way to load a bookmarks file, so I don't have to use the Chrome API to manually create a bookmark tree under a testing flag in my code.
You can load an existing user data directory with all of its data, including bookmarks and browser settings:
puppeteer.launch({
userDataDir: '/path/to/user-data-directory',
})
The profile must be from a Chrome/Chromium version close to the puppeteer's Chromium.
Also — in my experience path shouldn't contain spaces on Windows.

shardTestFiles unexpected behaviour with chromeoptions setting of --user-dir-data

I have the following settings in my config file.
specs : ['../specs/sample1.js', '../specs/sample2.js']
capabilities : {
'browserName' : 'chrome',
'chromeOptions' : {
'args' : [ '--user-data-dir=C:/Chrome/User Data' ]
},
shardTestFiles: true,
maxInstances: 2
}
There is nothing special about the tests in the spec file, just prints out the title of the url in the console.
When I run the test two chrome windows are opened up. But only one
(random choice) of the spec file tests are run. Once the tests in that spec file are completed both the browsers close at the same time. The second spec file is orphaned out and errors out after some time. Both the browser icons are overlapping each other. Check image below.
When I comment out the args in chromoptions containing the user-data-dir option, everything works perfectly. Two chrome windows are opened and the spec files are divided among the two browser windows and run all tests to completion. In this case the browser icons are separate.
I want to use the existing chrome profile with parallel option as it speeds up page loading and has default cookies setup.
What is the solution to get this to work?
On latest versions of protractor, chrome, webdrivers running on windows 8

Source map is absolute path but Chrome DevTool assumes it's a URL

I am using LESS compiler via a gulp-less in my gulpjs build. I have enabled sourceMap generation, and it kind of works, the proper file names and line numbers are there.
The gulp line that generates LESS with source maps is very straight-forward:
var less = require('gulp-less');
// [...]
gulp.src('web/css/**/*.less')
.pipe(less({
sourceMap: true
}))
.pipe(gulp.dest('dist/css'));
The only problem is that the source map contains a URL to the original .less file and it is generated as an absolute path to my file system, e.g. /Users/myuser/projects/project/web/css/myfile.less.
During development, I serve the site via Apache. When I open the site with Chrome DevTools, I can inspect elements, and the source map is working because I can see the myfile.less including correct line numbers on the Styles panel. However, Chrome is trying to load the less file, and it is using the full path is the sourcemap output but assuming it's a relative url to my site, so it tries to load http://localhost/Users/myuser/projects/project/web/css/myfile.less which doesn't exists.
I tried using workspaces to fix it, but didn't really manage. Is there something wrong in this setup? Am I missing something?
I think you're missing these final steps:
Create a workspace for your site. Point Chrome to your site's root directory on your system.
Right-click on a source-mapped less file in the Sources panel, then select 'Map to File System Resource...'. Select the less source file on filesystem.
Reload the dev tools (you'll get a prompt) and you should be good. Now, when you click on the foo.less:45 in the Styles panel, you should be brought to that spot in foo.less. Now that you've setup the workspace, if you save your changes in the dev tools, it will persist to the source less file. Nice!
Note there's a pretty big bug with Chrome which you'll probably notice right away. When you go to add style rules via the dev tools, the source mapping gets busted:
https://github.com/less/less.js/issues/1050#issuecomment-26195595
Let me know if this helps. I can post screenshots later, too, if needed.
Try to use sourceMapRootpath instead of sourceMapBasepath
gulp.src('web/css/**/*.less')
.pipe(less({
sourceMap: true
sourceMapRootpath: './my_less_path'
}))
For me this solution works in part because the files I'm importing inside another less files (eg. bower_components / .. / grid.less) still come with the absolute path.
Still could not find a solution for this.
If anyone can help would be great ... :)
To answer my own question, the proper way to do this is to use gulp-sourcemaps in the gulp pipe instead of passing properties to the LESS plugin. So, the following syntax works for me:
gulp.src('web/css/**/*.less')
.pipe(sourcemaps.init())
.pipe(less())
.pipe(sourcemaps.write())
.pipe(gulp.dest('dist/css'));
i was having a similar issue with typescript source maps -- the path in the maps would be generated as '../src/file.ts' where src was on the same level as output dir.
dir
|- src/*.ts
|- output/*.js
my webserver runs straight on output, so chrome was trying to download an inexisting source file from http://localhost/src/file.ts
my solution was to create a symlink 'src' inside the output folder and it is now happy. not a real 'fix' but solves the problem.
Trying the following:
gulp.src('web/css/**/*.less')
.pipe(less({
sourceMap: true
sourceMapBasepath: './my_less_path'
}))

Chrome 33 does not allow video capture although allowed in Preferences

I am trying to allow a http website to use my camera without the prompt without luck.
I changed the Preference file (when Chrome is closed) and changed the Profile part as follows:
"profile": {
"avatar_index": 0,
"content_settings": {
"clear_on_exit_migrated": true,
"pattern_pairs": {
"http://mywebsite.com:80,*": {
"media-stream-camera": 1,
"media-stream-mic": 1
},
...
However, for some reason it keeps showing the prompt.
Any idea? I'm using Chrome version 33.
Thanks!
It seems like Chrome overwrites the preference file every time it starts. I couldn't get any of my changes to stick.
Solution 1. Packaged Chrome app.
One way to grant permissions is to run your site as a chrome app. Just include videoCapture in the manifest.json. This will definitely work.
"permissions": [
"videoCapture"
],
Solution 2. Get an SSL Certificate.
I know you specifically said 'http', but if you have serve your site over https, it will only require the user to grant permissions once.
Solution 3. Modify device policy.
If you can't package your site as a chrome app for some reason, there might be another way to get it to work. It involves editing Chrome's policy list to allow camera access. I can't get this to work, currently, but I'm looking into it.
http://www.chromium.org/administrators/policy-list-3#VideoCaptureAllowedUrls
VideoCaptureAllowedUrls are a list of urls to automatically grant webcam permission to. The problem is setting this property. I know where the file lives, but there are no examples on HOW to set properties.
/Applications/Google\ Chrome.app/Contents/Resources/com.google.Chrome.manifest/Contents/Resources/com.google.Chrome.manifest
There are 'instructions' on editing the policy list, but they're totally useless. All it does is point to that file, and gives templates for Windows and Linux (I'm on OSX).
Note that this solution will only work in kiosk mode. To run Chrome in kiosk mode, add the flag --kiosk http://yoursite.com. This forces the app into fullscreen mode.
If you're on Windows/Linux, this SO post might help you.
Solution 4. Use command-line flag
use --use-fake-ui-for-media-stream command-line flag
example (OS X) : /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome http://html5-demos.appspot.com/static/getusermedia/record-user-webm.html --use-fake-ui-for-media-stream
More info here http://creativcoders.wordpress.com/2014/08/18/chrome-always-allow-access-to-webcam-and-microphone-over-http-webrtc/

HTML5 offline "Application Cache Error event: Manifest fetch failed (-1)"

I'm trying to write an HTML5 offline application but can't seem to get Chrome to accept the cache manifest file.
Chrome logs the following output to its console while loading the application:
Creating Application Cache with manifest http://localhost/cache.manifest
Application Cache Checking event
Application Cache Error event: Manifest fetch failed (-1) http://localhost/cache.manifest
However, if I remove all lines from the manifest file except for the first line (i.e. "CACHE MANIFEST") Chrome accepts the manifest:
Creating Application Cache with manifest http://localhost/cache.manifest
Application Cache Checking event
Application Cache Downloading event
Application Cache Progress event (0 of 0)
Application Cache Cached event
But, as soon as I add a new line to the manifest (even if that next line is empty) Chrome reverts to complaining that the fetch failed.
All files are being served locally from a Windows 7 PC via Python using SimpleHTTPServer on port 80. I've updated the types_map in %PYTHON%/Lib/mimetypes.py with the following line:
'.manifest': 'text/cache-manifest',
The manifest should contain the following:
CACHE MANIFEST
scripts/africa.js
scripts/main.js
scripts/offline.js
scripts/libs/raphael-min.js
favicon.ico
apple-touch-icon.png
To cache a website offline (HTML5) you need to specify all the files needed for it to run. In short specify the site main components needed.
Easy way to create a manifest is in Note Pad.
Note: CACHE MANIFEST needs to be first line and your files will follow after a line space as follows:
CACHE MANIFEST
Scripts/script.js
Content/Site.css
Scripts/jquery-ui-1.8.20.min.js
Scripts/modernizr-2.5.3.js
SESOL.png
Scripts/jquery.formatCurrency-1.4.0.min.js
http://code.jquery.com/mobile/1.2.0/jquery.mobile-1.2.0.min.css
http://code.jquery.com/jquery-1.8.2.min.js
http://code.jquery.com/mobile/1.2.0/jquery.mobile-1.2.0.min.js
Content/themes/images/icons-18-white.png
Controllers/AccountController
Controllers/HomeController
Models/AccountModels
Account/Login
Home/CheckOut
Note2: remove all spaces after each line.
Note:3 you need to follow the exact format FOLDER/File or FOLDER/FOLDER/FILE ect....
Just because you have a manifest file doesnt mean it will load. you need to add the following to the Tag:
<html manifest="~/cache.manifest" type="text/cache-manifest">
Don't forget that after you add this it's cached the first time the page loads. So you need to register a cache event in the 'mobileinit' event.
$(document).on("mobileinit", function () {
//register event to cache site for offline use
cache = window.applicationCache;
cache.addEventListener('updateready', cacheUpdatereadyListener, false);
cache.addEventListener('error', cacheErrorListener, false);
function cacheUpdatereadyListener (){
window.applicationCache.update();
window.applicationCache.swapCache();
}
function cacheErrorListener() {
alert('site not availble offline')
}
}
Download Safari and use the web inspector to find errors.
http://developer.apple.com/library/safari/#documentation/appleapplications/Conceptual/Safari_Developer_Guide/1Introduction/Introduction.html#//apple_ref/doc/uid/TP40007874-CH1-SW1
Tip: Chrome's developer tools "F12" will show you the errors in the manifest load. ie the files you still need to add.
Hope this helps, covers the entire process. I assuming if you are at this stage in development you new to add these to the mobile init:
$.mobile.allowCrossDomainPages = true; // cross domain page loading
$.mobile.phonegapNavigationEnabled = true; //Android enabled mobile
$.mobile.page.prototype.options.domCache = true; //page caching prefech rendering
$.support.touchOverflow = true; //Android enhanced scrolling
$.mobile.touchOverflowEnabled = true; // enhanced scrolling transition availible in iOS 5
Safari Developer Guide:
https://developer.apple.com/library/safari/#documentation/AppleApplications/Reference/SafariWebContent/Client-SideStorage/Client-SideStorage.html#//apple_ref/doc/uid/TP40002051-CH4-SW4
Have you tried anything like https://manifest-validator.appspot.com/ to validate your manifest?
I've struggled with my manifest file for quite a while, it is really hard to pinpoint what is wrong. Could be something as simple as wrong encoding to an extra line break at the start.
Today I experienced exactly the same problem. After hours of working I came the the key point: the format of manifest file. In short, the file must begin a new line ONLY with ascii(0A), not ascii(0D), or ascii(0D + 0A). Only in this way can I live with Chrome, or I will get a blank page, and the error info in the console window.
According to w3c, (http://www.w3.org/TR/html5/offline.html), in “5.6.3.2 Writing cache manifests”,both 0A, 0D and 0D + 0A are all acceptable. So, my opinion is: Chrome is not compatible with w3c in the point.
Further more, say, if myapp.js is to be cached, it MUST follow the same rule: begins a new line only with ascii(0A), or Chrome will throw the same info in the console windows.
My Chrome is 13.0.782.107
I have now resolved this issue by switching to CherryPy for serving these files :)
If anyone else becomes similarly stuck but wants to keep the server part simple, the following Python may be sufficient for getting started:
import cherrypy
class SimpleStaticServer:
#cherrypy.expose
def index(self):
return '<html><body>Go to the static index page</body></html>'
cherrypy.config.update({
# global
'server.socket_host': '192.168.0.3',
'server.socket_port': 80,
# /static
'tools.staticdir.on': True,
'tools.staticdir.dir': "(directory where static files are stored)",
})
cherrypy.quickstart(SimpleStaticServer())
If you want to visit the "site" from another device, you'll need to use the external IP address (for me this was 192.168.0.3). Otherwise, you can just use '127.0.0.1' for the 'server.socket_host' value. I then point my browser to http://192.168.0.3/index.html to get my static index page.
I have resolved this issue in visual studio for MVC application.
follow below steps:
I have created .appcache file in notepad and copy manifest file content into it.
(you don't need to create .Manifest file OR not create Manifest.cshtml view. just create .appcache file in notepad.)
give reference as
<html manifest="~/example.appcache"> in view
and issue will be resolved
I think the line
CACHE:
is missing in the manifest file (should be the 2nd line, before the list of files.