Google Apps Script: UrlFetchApp Post File - google-apps-script

I'm trying to POST a file to a REST API via Google Apps Script. The idea is that I have a process that is creating copies of a Google Doc, and I want to be able to post those newly created Docs to a third party system.
I found in UrlFetchApp that I can send files. However, I'm having issues sending the correct header values.
My request looks like so:
var file = DriveApp.getFileById(fileId);
var body = {
"file": file.getAs(MimeType.PDF)
};
var headers = {
'Content-Disposition': 'attachment; filename="'+ file.getName() +'"',
'Content-Length': file.getSize()
};
My options when I call UrlFetchApp.fetch(url, options) looks like :
({
method:"POST",
headers:{
'Content-Disposition':"attachment; filename=\"My Merge Development_row_1.pdf\"",
'Content-Length':90665,
Authorization:"Bearer TOKEN"
},
contentType:"application/x-www-form-urlencoded",
muteHttpExceptions:true,
payload:{file:Blob}
})
The API that I'm sending the files to requires the 'Content-Length' header. But, when I try to set a value for 'Content-Length' header I get an Apps Script error, "Attribute provided with invalid value: Header:Content-Length". If I don't set the Content-Length header then the API responds that the Content-Length and file size don't match.
Any ideas on how I set the Content-Length header so I can POST the file?

There is an existing ticket highlighting that the documentation is not clear on this very issue
The solution is:
Move content length value from "content-Length" name/value pair in
headers to the advanced argument "contentLength"
So in your example your options should look like
({
method:"POST",
headers:{
'Content-Disposition':"attachment; filename=\"My Merge Development_row_1.pdf\"",
Authorization:"Bearer TOKEN"
},
contentLength: 90665,
contentType:"application/x-www-form-urlencoded",
muteHttpExceptions:true,
payload:{file:Blob}
})
EDIT: Added a full example function to get contentLength and blob shown below:
function testFilePost() {
var file = DriveApp.getFileById(doc_id).getAs(MimeType.PDF);
var headers = {
'Content-Disposition': 'attachment; filename="'+ file.getName() +'"',
};
var options =
{
"method" : "post",
"payload": file.getBytes(),
"headers": headers,
"contentLength": file.getBytes().length,
};
var result = JSON.parse(UrlFetchApp.fetch('http://httpbin.org/post', options).getContentText());
Logger.log(result);
}

Related

Convert API requests to Google Apps Script

This post is following my previous one ->
How to turn Postman API's request into Apps Script code?
I'm trying to convert 3 API requests into a single piece of code via Google Apps Script.
The purpose is to export automatically a set of data from my CRM platform into Google Sheet.
Based on my CRM platform documentation, I need to do 3 API requests in order to get the final set of data I want:
Creating an export job (using a POST request)
Retrieve the job status from previous request (using GET request)
Retrieve the exported data (using GET request)
Below is the code for the 1st API request (kindly provided by Tanaike):
function exportjob() {
var url = 'https://api.intercom.io/export/content/data';
var options = {
"method": "post",
"headers": {
"Authorization": "Bearer 123456789",
"Accept": "application/json",
},
"contentType": "application/json",
"payload": JSON.stringify({
"created_at_after": 1654041600,
"created_at_before": 1656547200
})
}
var response = UrlFetchApp.fetch(url, options);
console.log(response.getContentText())
}
The result of this request is the following
Info {"job_identifier":"5gf58ty4y5y45229r", "status":"pending", "download_url":"", "download_expires_at":""}
Next step is to add the 2 other API requests in my script (Retrieve the job status and retrieve the data).
However I have a couple of issues:
For the 2nd API request
How do I include the job_identifier ID in my code? (given in the 1st request)
For the 3rd API request
How do I retrieve the data with the URL provided in the 2nd API request?
The data comes as ZIP file, how do I convert it to CSV so Google Sheet can open it?
How do get the final set of data to be exported in a google sheet?
Apologies for the long post, I'm trying to summarize the documentation from my CRM platform as much as I can.
Any help on how to retrieve the job POST and retrieve the set of data to a google sheet via CSV would be highly appreciated.
Thank you
1. About For the 2nd API request
If this is for Retrieve a job status, it seems that the sample curl command is as follows.
curl https://api.intercom.io/export/content/data/v5xsp8c19ji0s82 \
-X GET \
-H 'Authorization:Bearer <Your access token>'
-H 'Accept: application/json'
It seems that the value of job_identifier can be retrieved from your 1st request shown in your question. Ref
When this request is done for the first time, it seems that status returns pending. By this, it seems that until status is changed to complete, it is required to wait. Ref
2. About For the 3rd API request
If this is for Retrieve the exported data, it seems that the sample curl command is as follows.
curl https://api.intercom.io/download/content/data/xyz1234 \
-X GET \
-H 'Authorization:Bearer <Your access token>' \
-H 'Accept: application/octet-stream’
In this case, the document says Your exported message data will be streamed continuously back down to you in a gzipped CSV format.. I thought that in this case, the returned data might be able to be ungzipped with Utilities.ungzip, and the ungzipped data might be able to be parsed with Utilities.parseCsv.
3. Using your 3 requests, how about the following sample script?
function exportjob2() {
var accessToken = "###your access token###"; // Please set your access token.
// 1st request: This is from https://stackoverflow.com/a/73032528
var url1 = 'https://api.intercom.io/export/content/data';
var options1 = {
"method": "post",
"headers": {
"Authorization": "Bearer " + accessToken,
"Accept": "application/json",
},
"contentType": "application/json",
"payload": JSON.stringify({
"created_at_after": 1654041600,
"created_at_before": 1656547200
})
}
var response1 = UrlFetchApp.fetch(url1, options1);
var { job_identifier } = JSON.parse(response1.getContentText());
// 2nd request <--- Modified
var url2 = 'https://api.intercom.io/export/content/data/' + job_identifier;
var options2 = {
"headers": {
"Authorization": "Bearer " + accessToken,
"Accept": "application/json",
},
}
var response2 = UrlFetchApp.fetch(url2, options2);
// console.log(response2.getContentText()); // for debug.
var { download_url, status } = JSON.parse(response2.getContentText());
while (status == "pending") {
Utilities.sleep(5000); // Please adjust this value. The current wait time is 5 seconds.
response2 = UrlFetchApp.fetch(url2, options2);
// console.log(response2.getContentText()); // for debug.
var obj = JSON.parse(response2.getContentText());
status = obj.status;
download_url = obj.download_url;
}
if (!download_url) {
throw new Error("download_url has no value.");
}
// 3rd request
var options3 = {
"headers": {
"Authorization": "Bearer " + accessToken,
"Accept": "application/octet-stream",
},
}
var response3 = UrlFetchApp.fetch(download_url, options3);
var blob = response3.getBlob().setContentType("application/zip");
var csvs = Utilities.unzip(blob);
// Create a new Spreadsheet and put the CSV data to the 1st tab.
var ss = SpreadsheetApp.create("sample Spreadsheet");
csvs.forEach((b, i) => {
var ar = Utilities.parseCsv(b.getDataAsString());
var sheet = i == 0 ? ss.getSheets()[i] : ss.insertSheet("sample" + (i + 1));
sheet.getRange(1, 1, ar.length, ar[0].length).setValues(ar);
});
}
I'm not sure whether the downloaded the gzip file has the correct mimeType. So I added the mimeType like var blob = response3.getBlob().setContentType("application/x-gzip").
Note:
When this script is run, the flow of your showing question is done. But, I cannot test this API because I have no account. So, when an error occurs, please check each value and your access token again. And, please provide the error message. By this, I would like to confirm it.
I thought that the value of download_url returned from the 1st request might be the same with the value of download_url returned from 2nd request. But, I cannot test this API because I have no account. So, please check it, and when my understanding is correct, you can modify the above script.
This sample script creates a new Spreadsheet. But, if you want to put the CSV data to the existing Spreadsheet, please modify the above script.
Reference:
fetch(url, params)

using youtube API with Google Apps Scripts

I am not able to test my scripts.
If you can help with the code here - it is not working
As per documentation - https://developers.google.com/youtube/v3/docs/videos/insert
if (serviceYT.hasAccess()) {
url = 'https://youtube.googleapis.com/youtube/v3/videos?part=snippet%2CcontentDetails%2Cstatistics&mine=true&key='+ API_KEY;
var data;
data = '{"snippet":{"title":"testtitle","description":"testdes","categoryId":"19","tags":["asdf","sadfds"]},"status":{"privacyStatus":"public"}}';
var options = {
'headers': {
'Authorization': 'Bearer ' + serviceYT.getAccessToken()
,'Accept': 'application/json'
},
'contentType': 'application/json',
'method' : 'POST',
'payload' : data,
'muteHttpExceptions' : true
};
//execute and handle the response
var response = UrlFetchApp.fetch(url, options);
var responseCode = response.getResponseCode();
var result = JSON.parse(response.getContentText());
Logger.log(result);
}
My questions -
where to put the video?
Resolution of the error I am getting:
{error={message='status', code=400.0, errors=[{reason=unexpectedPart, domain=youtube.part, message='status', location=part, locationType=parameter}]}}
A youtube object (a empty video basically) is successfully added to the youtube which is visible via studio. its a empty video but the title, description, status etc... other things are correcty setup.
Any ideas on how to add the media or video??
Where in the payload the video blob has to be added?
I am not able to get it in the documentation.
https://developers.google.com/youtube/v3/docs/videos/insert
I believe your goal is as follows.
You want to upload a movie file to your YouTube channel with the title, description, and so on.
The movie file is put in your Google Drive.
You want to achieve this by directly requesting to the endpoint using UrlFetchApp.
In this case, how about the following modified script?
Modified script:
Before you use this script, please enable YouTube Data API v3 and add the scope of https://www.googleapis.com/auth/youtube. And please set the file ID of the movie file in your Google Drive to const fileId = "###";.
serviceYT.getAccessToken() is from your script.
function myFunction() {
const fileId = "###"; // Please set the file ID of movie file on the Google Drive.
const metadata = {
"snippet": { "title": "testtitle", "description": "testdes", "categoryId": "19", "tags": ["asdf", "sadfds"] },
"status": { "privacyStatus": "public" }
};
const url = 'https://www.googleapis.com/upload/youtube/v3/videos?part=snippet%2Cstatus';
const file = DriveApp.getFileById(fileId);
const boundary = "xxxxxxxxxx";
let data = "--" + boundary + "\r\n";
data += "Content-Type: application/json; charset=UTF-8\r\n\r\n";
data += JSON.stringify(metadata) + "\r\n";
data += "--" + boundary + "\r\n";
data += "Content-Type: " + file.getMimeType() + "\r\n\r\n";
const payload = Utilities.newBlob(data).getBytes().concat(file.getBlob().getBytes()).concat(Utilities.newBlob("\r\n--" + boundary + "--").getBytes());
const options = {
method: "post",
contentType: "multipart/form-data; boundary=" + boundary,
payload: payload,
headers: { 'Authorization': 'Bearer ' + serviceYT.getAccessToken() },
muteHttpExceptions: true,
};
const res = UrlFetchApp.fetch(url, options).getContentText();
console.log(res);
}
In your script, you are using the endpoint of url = 'https://youtube.googleapis.com/youtube/v3/videos?part=snippet%2CcontentDetails%2Cstatistics&mine=true&key='+ API_KEY;. Your data has the properties of snippet and status. In this case, it is required to use the endpoint of https://www.googleapis.com/upload/youtube/v3/videos?part=snippet%2Cstatus.
In order to upload the movie file, it is required to request with multipart/form-data.
When this script is run, the movie file on Google Drive is uploaded to YouTube.
Note:
In the current stage, even when "privacyStatus": "public" is used, the uploaded video is not public. About this, you can see it at the official document as follows. Ref
All videos uploaded via the videos.insert endpoint from unverified API projects created after 28 July 2020 will be restricted to private viewing mode. To lift this restriction, each API project must undergo an audit to verify compliance with the Terms of Service. Please see the API Revision History for more details.
When you use the YouTube API at Advanced Google services, you can use the following simple script. Ref
function myFunction2() {
const fileId = "###"; // Please set the file ID of movie file on the Google Drive.
const res = YouTube.Videos.insert({
"snippet": { "title": "testtitle", "description": "testdes", "categoryId": "19", "tags": ["asdf", "sadfds"] },
"status": { "privacyStatus": "public" }
}, ["snippet", "status"], DriveApp.getFileById(fileId).getBlob());
console.log(res);
}
References:
Videos: insert
Multipart-POST Request Using Google Apps Script
fetch(url, params)

Sheets & Appscripts Hubspot POST Request with Authentication Token Problem

I've created a GAS app to provide better pipeline reporting from our Hubspot instance. So far the app works and I have successfully created a Sales Pipeline that shows up in Google sheets. I am trying to add a capability that requires a POST method to query Hubspot's CRM V3. I got it to work in Postman but cannot replicate it in GAS.
The error I get is "Authentication credentials not found." The headers print to the log so I assume they are being generated properly. My guess is that my access Token and payload are not being passed properly to the API during the request. Any help on the matter would be much appreciated.
function getConversions() {
// Prepare authentication to Hubspot
var service = getService();
var headers = {headers: {'Authorization': 'Bearer ' + service.getAccessToken()}};
Logger.log(headers);
var raw = JSON.stringify({"filterGroups":[{"filters":[{"propertyName":"hs_analytics_last_visit_timestamp","operator":"GT","value":"1561514165666"}]}],"limit":100,"after":0});
var options = {
'method' : 'post',
headers: headers,
'contentType': 'application/json',
// Convert the JavaScript object to a JSON string.
body : raw,
redirect: 'follow',
"muteHttpExceptions": true
};
var response = UrlFetchApp.fetch('https://api.hubapi.com/crm/v3/objects/contacts/search?', options);
var result = JSON.parse(response.getContentText());
Logger.log(result);
};
How about this modification?
Modification points:
When I checked the official document of Search of HubSpot API, I found the curl sample. When this sample is converted to Google Apps Script, I noticed several modification points in your script.
UrlFetchApp.fetch has no properties of body and redirect.
About followRedirects, the official document says as follows.
If false the fetch doesn't automatically follow HTTP redirects; it returns the original HTTP response. The default is true.
In your URL, https://api.hubapi.com/crm/v3/objects/contacts/search? is used. If you don't use the API key, how about modifying to https://api.hubapi.com/crm/v3/objects/contacts/search?
When above modification is reflected to your script, it becomes as follows.
Modified script:
Please modify as follows.
From:
var options = {
'method' : 'post',
headers: headers,
'contentType': 'application/json',
// Convert the JavaScript object to a JSON string.
body : raw,
redirect: 'follow',
"muteHttpExceptions": true
};
To:
var options = {
method : 'post',
headers: headers,
contentType: 'application/json',
payload : raw,
muteHttpExceptions: true
};
Note:
Above modification is required for your script. But I'm worry about the error of Authentication credentials not found.. In this modification, it supposes that your access token of service.getAccessToken() can be used for this request. When I saw the official document, the API key can be also used. If the access token cannot be used, how about using the API key? It's like below.
https://api.hubapi.com/crm/v3/objects/contacts/search?hapikey=YOUR_HUBSPOT_API_KEY
References:
Search of HubSpot API
fetch(url, params)
function getConversions() {
// Prepare authentication to Hubspot
var service = getService();
var headers = {headers: {'Authorization': 'Bearer ' + service.getAccessToken()}};
var url = 'https://api.hubapi.com/crm/v3/objects/contacts/search'
//Logger.log(headers);
var raw = {"filterGroups":[{"filters":[{"propertyName":"hs_analytics_last_visit_timestamp","operator":"GT","value":"1561514165666"}]}],"limit":100,"after":0};
var options = {
method : 'post',
contentType: "application/json",
// Convert the JavaScript object to a JSON string.
payload : JSON.stringify(raw),
muteHttpExceptions: true
};
var response = UrlFetchApp.fetch('https://api.hubapi.com/crm/v3/objects/contacts/search?hapikey=myapikey',options);
var result = JSON.parse(response.getContentText());
Logger.log(response)
Logger.log(result);
};

Google photos api adding photos not working, upload seems to work

Trying to use Google Apps Script and Google Photos API to add photos to Google Photos. Upload seems to work / returns a token, but then adding the photo to the library fails. The process consists of two steps: 1. Upload the photo data as described here, then 2. Add the photo to photo library as described here.
Step 1. works for me, as I get an upload token, but step 2 with source code below, throws an error, but my call has the one media item it needs.
{
"error": {
"code": 400,
"message": "Request must have at least one newMediaItem.",
"status": "INVALID_ARGUMENT"
}
}
My code after the upload step below. I have tried to stringify request body and have passed it to payload instead of body, but nothing worked. As the error seems specific enough, I've the feeling I'm just overlooking a tiny thing, but what??? Who has a working piece of code, preferably in apps script that I can have a look at?
requestHeader = {
"authorization": "Bearer " + photos.getAccessToken(),
"Content-Type": "application/json"
}
var requestBody = {
"newMediaItems": [
{
"description": "Photo description",
"simpleMediaItem": {
"fileName": fileName,
"uploadToken": uploadToken
}
}
]
}
var options = {
"muteHttpExceptions": true,
"method" : "post",
"headers": requestHeader,
"body" : requestBody
};
var response = UrlFetchApp.fetch("https://photoslibrary.googleapis.com/v1/mediaItems:batchCreate", options);
Logger.log("raw: " + response);
You want to add an image file to the album using Photo API with Google Apps Script.
You have already enabled Google Photo API at API console. And yout access token can be used for using the method of mediaItems.batchCreate.
If my understanding is correct, how about this answer? Please think of this as just one of several possible answers.
Usage:
1. Linking Cloud Platform Project to Google Apps Script Project:
About this, you can see the detail flow at here.
2. Add scope:
In this case, please addt the scope of https://www.googleapis.com/auth/photoslibrary to the manifest file (appsscript.json).
Although I think that from your question, above step 1 and 2 have already been done, I added them because I thought that this might be useful for other users.
3. Sample script:
In your script, I cannot see the detail of uploadToken. But in your question, I could confirm that you have alredy retrieved the value of uploadToken. So when you want to use your script for retrieving uploadToken, please replace uploadToken to yours. As the modification point of your script, 1. Include the album ID. 2. There is no body property of UrlFetchApp. 3. Please use JSON.stringify() to the payload.
function getUplaodToken_(imagefileId) {
var headers = {
"Authorization": "Bearer " + ScriptApp.getOAuthToken(),
"X-Goog-Upload-File-Name": "sampleFilename",
"X-Goog-Upload-Protocol": "raw",
};
var options = {
method: "post",
headers: headers,
contentType: "application/octet-stream",
payload: DriveApp.getFileById(imagefileId).getBlob()
};
var res = UrlFetchApp.fetch("https://photoslibrary.googleapis.com/v1/uploads", options);
return res.getContentText()
}
// Please run this.
function myFunction() {
var imagefileId = "###"; // Please set the file ID of the image file.
var albumId = "###"; // Please set the album ID.
var uploadToken = getUplaodToken_(imagefileId);
var requestHeader = {Authorization: "Bearer " + ScriptApp.getOAuthToken()};
var requestBody = {
"albumId": albumId,
"newMediaItems": [{
"description": "Photo description",
"simpleMediaItem": {
"fileName": "sampleName",
"uploadToken": uploadToken
}}]
};
var options = {
"muteHttpExceptions": true,
"method" : "post",
"headers": requestHeader,
"contentType": "application/json",
"payload" : JSON.stringify(requestBody)
};
var response = UrlFetchApp.fetch("https://photoslibrary.googleapis.com/v1/mediaItems:batchCreate", options);
Logger.log(response);
}
In this script, it supposes that the image file is put in Google Drive.
Note:
If the error of No permission to add media items to this album. occurs, please create the album by the script. The official document says as follows.
Media items can be created only within the albums created by your app.
In this case, please create new album by the following script, and please retrieve the album ID.
function createNewAlbum() {
var options = {
headers: {Authorization: "Bearer " + ScriptApp.getOAuthToken()},
payload: JSON.stringify({album: {title: "sample title"}}),
contentType: "application/json",
method: "post"
};
var res = UrlFetchApp.fetch("https://photoslibrary.googleapis.com/v1/albums", options);
Logger.log(res);
}
References:
Class UrlFetchApp
Upload media
Creating a media item
Method: mediaItems.batchCreate
If I misunderstood your question and this was not the direction you want, I apologize.
Found it! Not shown in the code I submitted, but still adding the fix, as it might help others making the same mistake I did. I directly assigned the response from UrlFetchApp to be the upload token, like so:
uploadToken = UrlFetchApp.fetch("https://photoslibrary.googleapis.com/v1/uploads", options);
but needed to call .getContentText() on it to get it as string, like so:
uploadToken = UrlFetchApp.fetch("https://photoslibrary.googleapis.com/v1/uploads", options).getContentText();

Stackdriver Logging API returns response code 200, but response is empty

I'm trying to fetch stackdriver logs via Stackdriver Logging API v2. I do this by making a POST request from google apps script project, in particular using UrlFetchApp. The thing is, it runs successfully, but the response shown in log is empty. However, when I made the same request using apirequest.io, curl and Google API explorer, I got the necessary response.
I searched extensively, but to no avail. Tried experimenting with header, url, but nothing.
function exportLogs () {
var options = {
"method" : "post",
"headers": {Authorization: 'Bearer ' + ScriptApp.getOAuthToken()},
"resourceNames": [
"projects/MyProject"
],
"pageSize": 1,
}
var response = UrlFetchApp.fetch('https://logging.googleapis.com/v2/entries:list?key=MyApiKey', options)
Logger.log(response)
}
What I want to get is some logs, but I'm only getting {}
Issue:
Unacceptable keys are used in options object.
Solution:
payload is the only acceptable parameter for including request body.
Code:
function exportLogs() {
var options = {
method: "post",
headers: { Authorization: 'Bearer ' + ScriptApp.getOAuthToken() }, //Include https://www.googleapis.com/auth/cloud-platform in scopes
payload: JSON.stringify({
resourceNames: ['projects/[PROJECT_ID]'],
pageSize: 1,
}),
};
var response = UrlFetchApp.fetch(
'https://logging.googleapis.com/v2/entries:list?key=MyApiKey',
options
);
Logger.log(response);
}
To Read:
Urlfetch#params
Logging api#entriesList
Setting scopes
For some strange reason, adding an orderBy sort property in the request body object is the only way I could get results to be retrieved.
Also, a filter should be added to get only Apps Script logs.
load.filter = 'resource.type="app_script_function"';//Only from Apps Script
See example code at GitHub:
Apps Script Download Stackdriver Logs
Code:
function exportStackdriverLogs() {
var id,load,options,param,response;
id = 'Enter your Cloud Project ID here';//See your console at:
//https://console.cloud.google.com/iam-admin/settings
param = "projects/" + id;
load = {};
load.resourceNames = [param];
load.orderBy = "timestamp desc";
load.filter = 'resource.type="app_script_function"';//Only get logs that
//came from Apps Script
load.pageSize = 1;//You will probably want more than 1 but this is for an example
options = {};
options.method = "post";
options.headers = { Authorization: 'Bearer ' + ScriptApp.getOAuthToken() };
options.payload = JSON.stringify(load);
options.muteHttpExceptions = true;
options.contentType = "application/json";
response = UrlFetchApp.fetch('https://logging.googleapis.com/v2/entries:list',options);
//Logger.log('response: ' + response.getResponseCode())
//Logger.log('content: ' + response.getContentText())
}