What is the expected behaviour of the changes feed with drive.file scope? - google-drive-api

My expectation is that if I query the Changes Feed with a scope of drive.file, I will only receive changes to files owned by my application.
However, in testing that I have done, I am seeing files in the feed that have nothing to do with my app. At least some of them are files that have been shared with me.
Anybody know exactly how this is supposed to work?

Edit 0
Similar or duplicate StackOverflow questions
Listing files with search query returns out-of-scope results (drive.files.list call, using drive.files scope)
List ignores drive.file scope and shows shared files not created by the calling app
The files returned will not be specific to your app. Files that are "public on the web" are also reported back, regardless of whether or not your app created them or they were ever opened by the user in your app.
There is a parameter (includeSubscribed) that will filter out shared docs but this is also a bit limited (see below).
From Detect Changes:
For Google Drive apps that need to keep track of changes to files,
polling repeatedly can be both inefficient and resource-intensive. The
Changes feed provides a more efficient way to detect changes to all
files, including those that have been shared with a user. The feed
works by providing the current state of each file, if and only if the
file has changed since the given changestamp.
Here is a relevant parameter from Changes:list.
includeSubscribed boolean
Whether to include shared files and public
files the user has opened. When set to false, the list will include
owned files plus any shared or public files the user has explictly
added to a folder in Drive. (Default: true)

Scope(https://www.googleapis.com/auth/drive.file)
Meaning(Per-file access to files created or opened by the app)
The scope https://www.googleapis.com/auth/drive.file strikes this balance in a practical way. Presumably, users only open or create a file with an app that they trust, for reasons they understand.
But though, to your point, please refer to this q&a.

Related

How do I avoid Google Drive API audit? -- Only Read access is needed to list files from folder and to download them

The product I'm working on currently uses the scope "https://www.googleapis.com/auth/drive" (which is now "restricted" by Google), which gives full read and write access to a user's Drive account, including app metadata. But we only need read access to list all files and folders inside a specific folder, and we need to be able to download those files, that's all.
Google Drive API will soon apply the new "restricted" scope policy (https://support.google.com/cloud/answer/9110914#restricted-scopes), which will require us to go through a very expensive audit (tens of thousands of dollars...). Is there a possible workaround to get 'read-only' access on a specific folder, and avoid the audit (note that https://www.googleapis.com/auth/drive.readonly is also a restricted mode)?
I'm aware of the "https://www.googleapis.com/auth/drive.file" scope (which is "recommended" by Google, so no audit required), which almost solves this problem. But we have thousands of users bringing in data from multiple Drive Folders, and pushing new files daily. This scope would introduce a manual step for a client each morning to have to "approve" every new file, and this would be a big scalability/usability problem.
Ideally, I would like Google to add a new scope, like a read-only access to anything inside a folder, before they go forward with their audit... but i doubt that this will happen soon.
Does anyone know of a better option?
[EDIT] For reference, here is the list of scopes and we can see which ones are "restricted", "sensitive" and "recommended" : https://developers.google.com/drive/api/v2/about-auth
Solution
Hi! So after taking a better look at this it seems that restricted scopes do NOT require any paid audit. The main difference is that they will have a wider access to user's data and thus it requires you to go through a restricted scope verification process.
You can use these restrictive scopes (the one that best fits your application) without the need of paying any audit. See more information about how to implement restrictive scopes here.

How can we add a file to a user's files.list via the sdk?

We are having issues where sometimes a file that a user can access is not returned when the user issues a files.list. This can happen in many ways. For example, new members of a Google group will not see previously shared files, as described in this question. Moreover, acording to Google documentation there are other limits on sharing which can prevent shared files from appearing in the "Shared with me" view. Finally, a user can issue a files.delete on a file she doesn't own, and the file will disappear from files.list but will still exist.
What can a user do via the SDK alone to cause a file which she can access via files.get to appear in the list of files retrieved via files.list? We are using a service account which impersonates users; the user never authenticates to Google via a browser. A link in an email that the user needs to click won't work for us, unfortunately. Accessing the file via the Google Drive UI has the desired effect, but the analogous files.get call does not.
The Google Calendar API explicitly exposes a CalendarList interface where a user can issue an insert to add an existing calendar to her list. The Google Drive SDK seems like a hybrid Files/FilesList interface with some of the functionality missing (nothing like FilesList.insert) and some of the functionality mixed together (issuing a delete as a non-owner acts like FilesList.delete but issuing it as the owner acts like Files.delete).
If we can't manage the user's files list programmatically then it is not useful for our service. We could ignore the files.list call entirely and just start recursively performing children.list queries on all shared folders, but this is incredibly expensive (unless someone knows how to issue a single query which returns all the Files resources in a folder and not just the IDs of those resources).
Any help would be appreciated. We've been trying this many different ways and have been frustrated at every turn. Thanks!

files.list() reproducibly returns incomplete list in "drive.files" scope

Our application needs a full list of the user's files and folders. We use files.list() via the Javascript library (essentially the same code as shown in the official API reference as an example).
We use the "drive.files" scope.
Examining the response to the list, we find that some files are always missing. I did various tests to understand the problem:
The files clearly exist. They show up in the Google Drive Webapp and, if I explicitly request them via ID, I can get them via the API without problems.
It's reproducible, always the same files are missing.
It is not transient. I tried a day after and still the same files are missing. I know of a few strange effects in the API that go away after some time but not this one.
It is not a one time thing (e.g. some weird things went wrong during upload). If I repeat with a completely different Google Account again files are missing. Of a small set of 147 uploaded files in one test 4 are missed by the files.list call, in another test with the same 147 files on another account 23 files are missing.
It only occurs when I use the drive.files scope. If I relax the scope to drive all files are returned. If look at "Details" in the Google Drive Webapp also the missing files are shown as created by our Application. So it does not seem that they lost their origin somehow.
It also occurs when I specify a search query. If I call files.list with a search term "q: modifiedDate > '2012-06-04T12:00:00'" which also should return all files, the same files are missing.
I re-implemented the same thing as pure REST call to the API to rule out that it is an issue with the Javascript library. The error remains.
Update: I could track it down to an issue with the paging and the maxResults parameter. If I use different values the API returns different number of items:
With maxResults=100 I get 100+100+7=207.
With maxResults=99 I get 99+99+28=226.
With maxResults=101 I get 101+101+0=202.
The last result is interesting which gave me a nextLink indicating there are more results but the items array in the last response was actually empty. This alone might indicate a bug.
Still, this only occurs in drive.file scope, the counts are consistent in the full drive scope.
I'd be glad to hear ideas for a workaround. I'm aware of other ways to keep track of the users files, e.g. using the changes feed. I'm using that already but for a specific part in our application I simply need a reliable and complete list of all our application's items in a user's account.
One more note: We had other issues with the "drive.files" scope before (see Listing files with search query returns out-of-scope results (drive.files.list call, using drive.files scope)). This turned out to be an easy fix. Perhaps this issue is related.
Are there any difference in the files belonging to "shared to me" and own files/folders, was the issue for me ?
The way it is presented in Google Drive was not the same result I got when searching without the correct flags.
I found out when I did this file list with all the folders, that I did have to include from where the search scope of files should be.
- Include deleted files
- Include shared to me files

Is there any way to have private data?

I'm aware of shortcut links. Looking for behavior similar to that of a native Google doc. File exists, possibly takes up storage, can be renamed/moved/deleted, but the data inside shouldn't be modified except by the app. Possibly, defining export formats/links.
I believe the answer is a simple "no" - Google Drive is for storing user files, not protected application data or configuration data. So you could put a file to a users drive, but only the owner of the drive can control whether the file is shared or changed. So they can edit it, you can't stop them, and there's no reason to think that'll ever be a feature in the future.
To have such control you will need to store such data on your own server, or some other such storage medium.
The only other thing that you would do with only Google Drive is encrypt the configuration file you store, for instance, so it couldn't be easily edited - but that's probably just a bad idea. If you must save a configuration file to a persons drive, bury it inside an application folder and sanity check it to ensure it isn't corrupt - but don't count on a person or application never opening and editing it. If it's something a person shouldn't be able to read or change, don't save it to their drive.
As of April 2012, application data is supported: What is the Application Data folder?.
"Export format links" could be done with Custom file properties, though, I'm unsure of what kind datatypes are supported for the value beyond the example string.

How to make a copy of a file between users of different domains?

I would like to make a copy of a Google Spreadsheet file between users that are not on the same domain. The source file has a sharing setting of Anyone with the link can edit.
I created a Web App (with execution permissions set to User accessing the web app) that call an install function:
I was hoping the code would read the source file which is open to everybody and make a local copy to the Google Drive of the user running the Web App (of course assuming the user accept to run the script). The behavior works when it's me running it (because I am the owner of the source) but do not work if the user is from another domain.
Any suggestions on how to achieve this?
function doInstall() {
// The source file is readable by everybody with the link;
var file = DocsList.getFileById('0AlVPTKz1xoevdHc1ZTQ2OGFMXXXXXXXYYYYZZZZ');
file.makeCopy('Test Copy Spreadsheet');
}
I had a different but similar use case and problem. Like you, my source file was accessible to anyone with the link and my domain settings allowed users to share content outside my domain. It worked for me but not for them.
This might work for you, it has for me (so far):
Assuming you have your original script...
Create a separate script for 'Script as web app'.
Write a function (e.g. loadFileIds()) to store the necessary fileIds using ScriptDb. Run the loadFileIds() function.
Write another function (e.g. getFileId()) in the same project that returns your desired file Id from ScriptDb.getMyDb when called. Avoid using DocsList in this function. I originally thought that because getFileId() would 'run as me' that it would be OK but it was always the DocsList line that caused failures when other users triggered the script.
In File > Manage Versions: Save a version of the project.
Under Publish: Deploy that version of your project as a web app. Choose Execute the app as "me(me#mydomain.com)" and set Who has access to the web app to "Anyone".
In your original project go to Manage Resources and enter the Project Key of the web app, choose a version and copy the identifier.
Write a function that calls yourIdentifier.getFileId() and - hopefully - returns the fileId.
In my experience so far external users have been able to access files that they could not when DocsList methods returned null. But there is one more thing to add... You might find you have to share your web app script with your users. I haven't quite made certain that's necessary but I've encountered bugs when I don't. I untick the option to email them the link.
I learned how to do this from examples in the documentation, this video and of course StackOverflow.
As you can see, I'm still working this out myself and testing of the system has so far been just me simulating other users. So no promises. Good luck! I'll star your issue.