Upload CSV to Google Drive Spreadsheet using Drive v2 API - csv

How can I upload a local CSV file to Google Drive using the Drive API v2 so that the uploaded file is in the native Google Spreadsheet format. Preferably in Python, but a raw HTTP request will suffice.
What I tried:
request body content-type: 'application/vnd.google-apps.spreadsheet', media_body content-type: 'text/csv'. --> 401 Bad Request
request body content-type: 'application/vnd.google-apps.spreadsheet', media_body content-type: 'application/vnd.google-apps.spreadsheet'. --> 400 Bad Request
... (a couple of others such as leaving a property out and similar, usually got 400 or Drive didn't recognise it as a native spreadsheet)

Your insert request should specify text/csv as the content-type.
The trick to get the file converted is to add the ?convert=true query parameter to the request url:
https://developers.google.com/drive/v2/reference/files/insert

(Mar 2017) Note, while the question specifically asks about Drive API v2, developers should know that the Google Drive API team released v3 at the end of 2015, and in that release, insert() changed names to create() so as to better reflect the file operation. There's also no more convert flag -- you just specify MIMEtypes... imagine that!
The documentation has also been improved: there's now a special guide devoted to uploads (simple, multipart, and resumable) that comes with sample code in Java, Python, PHP, C#/.NET, Ruby, JavaScript/Node.js, and iOS/Obj-C to upload a file and another that imports a CSV file as a Google Sheet.
Just to show how straightforward it is, below is one alternate Python solution (to the sample in the docs) for short files ("simple upload") where you don't need the apiclient.http.MediaFileUpload class. This snippet assumes your auth code works where your service endpoint is DRIVE with a minimum auth scope of https://www.googleapis.com/auth/drive.file.
# filenames & MIMEtypes
DST_FILENAME = 'inventory'
SRC_FILENAME = DST_FILENAME + '.csv'
SHT_MIMETYPE = 'application/vnd.google-apps.spreadsheet'
CSV_MIMETYPE = 'text/csv'
# Import CSV file to Google Drive as a Google Sheets file
METADATA = {'name': DST_FILENAME, 'mimeType': SHT_MIMETYPE}
rsp = DRIVE.files().create(body=METADATA, media_body=SRC_FILENAME).execute()
if rsp:
print('Imported %r to %r (as %s)' % (SRC_FILENAME, DST_FILENAME, rsp['mimeType']))

Claudio Cherubino's answer is correct -- you have to add the parameter manually. Since you asked in Python though, here's a concrete example:
body = {
'mimeType':'text/csv',
'title': 'title'
}
# service: your authenticated service
# media: your apiclient.http.MediaFileUpload object, with 'text/csv' mimeType
req = service.files().insert(media_body=media, body=body)
# patch the uri to ensure conversion, as the documented kwarg seems to be borked.
# you may need to use '?convert=true' depending on the uri, not taking that into
# account here for sake of simplicity.
req.uri = req.uri + '&convert=true'
# now we can execute the response.
resp = req.execute()
# should be OK
assert resp['mimeType'] == u'application/vnd.google-apps.spreadsheet'

Java :
//Insert a file
File body = new File();
body.setTitle("CSV");
body.setDescription("A test document");
body.setMimeType("text/csv");
java.io.File fileContent = new java.io.File("document.csv");
FileContent mediaContent = new FileContent("text/csv", fileContent);
Insert insert = service.files().insert(body, mediaContent);
insert.setConvert(true);
File file = insert.execute();
System.out.println("File ID: " + file.getId());

The best way to get started is using the web form at
https://developers.google.com/drive/v2/reference/files/insert#try-it

Related

Getting error - "The remote server returned an error: (401) Unauthorized." while requesting file with Google Drive API

While downloading files from Google Drive, specifically with extensions .docx, .doc, or .pdf, we are now using the "WebContentLink" property through filePicker API as per the recent modification in the Google Drive API to get the file. (Previously, we were using downloadUrl file property to download the file through filePicker API).
However, we are getting this unwanted exception "401 - Unauthorized access" while authorizing download requests to the API using HttpWebRequest by passing header using bearer token.
Google Drive API version - v2
Used - File Picker API to get the file data, i.e. file ID, webContentLink, alternateLink
Here's the piece of code where we have implemented the logic
//fileurl - webcontentlink of the file (previously, we used downloadUrl)
//accessToken - client's accessToken
//filepath - location to store the downloaded file
public void downloadGdriveFileThroughWebRequest(string fileurl, string filepath, string accessToken)
{
HttpWebRequest rq = (HttpWebRequest)WebRequest.Create(fileurl);
rq.Method = "GET";
rq.PreAuthenticate = true;
rq.Headers.Add("Authorization", "Bearer " + accessToken);
try {
HttpWebResponse resp = (HttpWebResponse)rq.GetResponse();
using (Stream output = File.OpenWrite(filepath))
{
using (Stream s = resp.GetResponseStream())
{
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = s.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
}
}
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
reader.ReadLine();
Console.WriteLine(reader.ReadToEnd());
}
}
}
If we do not pass headers, we are able to download the file but not able to read and write its data so that's of no use!
Is there any alternate solution to download the requested Google Drive file through API requests with HTTP headers?
NOTE - Here's the recent modification to the Google Drive and Picker APIs - https://cloud.google.com/blog/products/application-development/upcoming-changes-to-the-google-drive-api-and-google-picker-api
and we have already made the changes.
Solution
This solution is brought to you by #RuchiYevle , the original poster who in the comments posted what worked for him. Therefore, this will be posted as a community wiki.
As Google Drive API now no longer supports the functionality to download files using downloadURL using access token, we will have to use the URL googleapis.com/drive/v2/files{fileId}?alt=media instead.
This worked for me and also, it is the intended behaviour as mentioned in their issue tracker.

Stackdriver export to .txt or PDF on drive/mail

I've set up an script which reads data from a spreadsheet and sends emails according this data.
Now, I've also set it up to do some simple logging via stackdriver.
What I'd like to do is to export these logs (after/at the end of every execution of the mail-script) to a .txt or .pdf file which then get saved to a specific Google Drive folder or been send by mail.
Unfortunately I can't seem to find out how to do this, or if its even posible?
There is no way to edit a Google docs file if this is what you where thinking of doing. Your going to have to create your .txt or .pdf file locally then upload the file to Google drive or send it as an email. Technically if you upload the file as a .txt i think that Google drive will allow you to export it as pdf but i haven't tried with the new version of Google drive.
var fileId = '1ZdR3L3qP4Bkq8noWLJHSr_iBau0DNT4Kli4SxNc2YEo';
var dest = fs.createWriteStream('/tmp/resume.pdf');
drive.files.export({
fileId: fileId,
mimeType: 'application/pdf'
})
.on('end', function () {
console.log('Done');
})
.on('error', function (err) {
console.log('Error during download', err);
})
.pipe(dest);
Downloading google Documents
I also dont think that you will be able to email a file directly from Google Drive you will have to download the file locally then add send your email.
Stackdriver has an error reporting API. Documentation for Stackdriver The API has REST capability, which means that you can call it from Apps Script using UrlFetchApp.fetch(url) where url is the url needed to get error reporting information. The base url for the Stackdriver API is: https://clouderrorreporting.googleapis.com The API must be enabled.
There are multiple methods that can be used with the API.
The method that you probably need is the list method, which requires the url:
https://clouderrorreporting.googleapis.com/v1beta1/{projectName=projects/*}/events
where the projectName parameter must be a Google Cloud Platform project ID.
See documentation on list at: projects.events.list
The return value for that HTTPS Request, if successful, is a "response body" with the following structure and data:
{
"errorEvents": [
{
object (ErrorEvent)
}
],
"nextPageToken": string,
"timeRangeBegin": string
}
The ErrorEvent is a JSON object with the following structure and data:
{
"eventTime": string,
"serviceContext": {
object (ServiceContext)
},
"message": string,
"context": {
object (ErrorContext)
}
}
So, if you want to send an email with error data from Stackdriver, it won't be sent directly from Stackdriver, you need to make a request to Stackdriver from Apps Script, get the error information, and then send an email from Apps Script.
Of course, you could have your own error handling system, that logged error information to some external target, (Eg. your spreadsheet, or a database) using UrlFetchApp.fetch(url);
To make the request to the Stackdriver API you would need code something like this:
var projectID = "Enter project ID";
var url = 'https://clouderrorreporting.googleapis.com/v1beta1/' + projectID
+ '/events';
var tkn = ScriptApp.getOAuthToken();
var options = {};
options.headers = {Authorization: 'Bearer ' + tkn}
options.muteHttpExceptions = true;
var rtrnObj = UrlFetchApp.fetch(url,options);
Logger.log(rtrnObj.getContentText())
I haven't use this API and I haven't tested this code. If anyone uses it, and has information or finds an error, please make a comment.

Unable to update Google Drive files using Drive API v3 -- The resource body includes fields which are not directly writable

I am trying to use Google Drive API (v3) to make updates to documents
in Google Drive.
I have read this migration guide:
Google Drive API v3 Migration
And coded it to make a new empty File() with the details I want to update
and then calling execute() with that and the file ID.
But i am still getting an error. Can anyone point out where I am doing wrong?
thanks alot!!
Error:
{
"code" : 403,
"errors" : [{
"domain" : "global",
"message" : "The resource body includes fields which are not directly writable.",
"reason" : "fieldNotWritable"
}],
"message" : "The resource body includes fields which are not directly writable."
}
Code snippet below:
File newFileDetails = new File();
FileList result = service2.files().list()
.setPageSize(10)
.setFields("nextPageToken, files(id, name)")
.execute();
List<File> files = result.getFiles();
if (files == null || files.size() == 0) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
if (file.getName().equals("first_sheet")) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
newFileDetails.setShared(true);
service2.files().update(file.getId(), newFileDetails).execute();
}
}
}
I had the same issue and found a solution. The key point is: you must create a new File object without Id and use it in update() method. Here is a piece of my code:
val oldMetadata = service!!.files().get(fileId.id).execute()
val newMetadata = File()
newMetadata.name = oldMetadata.name
newMetadata.parents = oldMetadata.parents
newMetadata.description = idHashPair.toDriveString()
val content = ByteArrayContent("application/octet-stream", fileContent)
val result = service!!.files().update(fileId.id, newMetadata, content).execute()
It works. I hope it'll help you.
Referring to https://developers.google.com/drive/v3/reference/files#resource-representations, you can see that shared isn't a writable field. If you think about it, this makes perfect sense. You can share a file by adding a new permission, and you can check if a file has been shared by reading the shared property. But saying a file is shared, other than by actually sharing it, makes no sense.
in the code it looks like this
Drive service... // your own declared implementation of service
File file = new File(); //using the com.google.api.services.drive.model package
// part where you set your data to file like:
file.setName("new name for file");
String fileID = "id of file, which you want to change";
service.files().update(fileID,file).execute();
trying to change the fields from remote files, and rewriting to this file can throw the security exception like exception below.
but it is not a solution for your question.
If you want to share file to another google account by email, you can do it with reimplementing authorization to authorization with using service account of your app, and the add the needed email, as owner of the file.
I was doing the same thing. My goal was to share my file programmatically with my Python code.
And yes, I was getting the same error:
"The resource body includes fields which are not directly writable"
I solved this problem by adding the service's email address of my Virtual Machine (I created it on my Compute Engine dashboard) to Editors of the file.
Then I ran this Python code in my VM:
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
# Took the json file from my Google Cloud Platform (GCP) → IAM & Admin → Service Accounts:
service_key_file = 'service_key.json'
scope = 'https://www.googleapis.com/auth/drive'
credentials = ServiceAccountCredentials.from_json_keyfile_name(service_key_file, scopes=scope)
driveV3 = build('drive', 'v3', credentials=credentials)
fileId = '1ZP1xZ0WaH8w2yaQTSx99gafNZWawQabcdVW5DSngavQ' # A spreadsheet file on my GDrive.
newGmailUser = 'testtest#gmail.com'
permNewBody = {
'role': 'reader',
'type': 'user',
'emailAddress': newGmailUser,
}
driveV3.permissions().create(fileId=fileId, body=permNewBody).execute()
print(f"""The file is now shared with this user:
{newGmailUser}\n
See the file here:
https://docs.google.com/spreadsheets/d/1ZP1xZ0WaH8w2yaQTSx99gafNZWawQabcdVW5DSngavQ""")

Import Google app script project from JSON file

In Google Drive, it's possible to download an app script project as a .json file.
When such file is imported back to a Google Drive it's not properly associated with Google Script editor app.
Is there any way to do it properly?
Importing and exporting of Apps Script files requires the use of the import/export API.
To modify an existing script you will need to have a Oauth2 token with the scope of: https://www.googleapis.com/auth/drive.scripts
For updating a file you will "PUT" the updated JSON to:
https://www.googleapis.com/upload/drive/v2/files/{FileId}
The Apps Script file looks like
{
files:
[
{
name:{fileName},
type:{/* server_js or html */},
source:{/* source code for this file */},
id:{ /* Autogenerated. Omit this key for a new file, or leave value unmodified for an updated file */},
},
{...}
]
}
To add a file:
Add an object to the files array with the keys name, type, source
To modify a file:
Modify the values of name, type, or source of the file object but do not modify the id.
When you PUT the file back make sure you put the entire files array with your modifications, not just the new file object.
To make the modification in GAS itself would look like:
var scriptFiles = JSON.parse(downloadedJSONFile);
scriptFiles.files.push({"name":fileName,"type":fileType,"source":source});
var url = "https://www.googleapis.com/upload/drive/v2/files/"+scriptId;
var parameters = { method : 'PUT',
headers : {'Authorization': 'Bearer '+ tokenWithProperScope,
payload : JSON.stringify(scriptFiles),
contentType:'application/vnd.google-apps.script+json',
muteHttpExceptions:true};
var response = UrlFetchApp.fetch(url,parameters);
You will get a response code of 200 for a successful change. The response text will include the entire new JSON files with the assigned id to the file you added.
Fine more at:
https://developers.google.com/apps-script/import-export
Set the mimetype as application/vnd.google-apps.script

Get shared link through Google Drive API

I am working on an app using Google Drive. I want the user to be able to share files by link, setting the permissions to anyone and withLink as described in the Google Developers documentation.
However, I cannot figure out what link to share. When I share a file in the Google Drive browser interface, I see the Link to share in the format:
https://docs.google.com/presentation/d/[...]/edit?usp=sharing
A link in this format is not part of the file resource object, nor it is returned from the http call setting the permissions. I hope someone can explain to me how to get this link through the REST api?
You can use the alternateLink property in the File resource to get a link that can be shared for opening the file in the relevant Google editor or viewer:
https://developers.google.com/drive/v2/reference/files
Update
[With API V3](https://developers.google.com/drive/api/v3/reference/files it is suggested to use the webViewLink property.
To actually enable link sharing using Google Drive API:
drive.permissions.create({
fileId: '......',
requestBody: {
role: 'reader',
type: 'anyone',
}
});
Get the webLinkView value using:
const webViewLink = await drive.files.get({
fileId: file.id,
fields: 'webViewLink'
}).then(response =>
response.data.webViewLink
);
In my case using the PHP Api v3, for the link to be non-empty you must define that you request this field... and if you have the right permissions:
so something like this:
$file =self::$service->files->get("1ogXyGxcJdMXt7nJddTpVqwd6_G8Hd5sUfq4b4cxvotest",array("fields"=>"webViewLink"));
Here's a practical example on how to get the WebViewLink file property (A.K.A. File edit link):
$file = $service->files->get($file_id, array('fields' => 'webViewLink'));
$web_link_view = $file->getWebViewLink();
OR
$sheetsList = $drive_service->files->listFiles([
'fields' => 'files(id, name, webViewLink, webContentLink)',
]);
$web_link_view = $sheetsList->current()->getWebViewLink();
Pay attention that you should load the file specifying which fields you wanna bring with it (In this case, webViewLink). If you don't do that, only id and name will be available.
In case you need to adjust the file sharing settings, this is how you do it:
$permissions = new \Google_Service_Drive_Permission();
$permissions->setRole('writer');
$permissions->setType('anyone');
$drive_service->permissions->create($file_id, $permissions);
Possible values for setRole() and setType() can be found here: https://developers.google.com/drive/api/v3/reference/permissions/create
For python, I only needed to get the file "id".
Then "created" the link like this:
def create_folder(folder_name, folder_id):
"""Create a folder and prints the folder ID and Folder link
Returns : Folder Id
"""
try:
# create drive api client
service = build("drive", "v3", credentials=creds)
file_metadata = {
"name": folder_name,
"mimeType": "application/vnd.google-apps.folder",
"parents": [folder_id],
}
file = (
service.files().create(body=file_metadata, fields="id").execute()
)
id = file.get("id")
print(
f'Folder ID: "{id}".',
f'https://drive.google.com/drive/u/0/folders/{id}',
)
return id
except HttpError as error:
print(f"An error occurred: {error}")
return None