UrlFetchApp upload file multipart/form-data in Google Apps Script - google-apps-script

I need to upload files to a 3rd party service. The files are created on Google Drive, and I get the blob-data for it.
I want to create a multipart request which now looks like this when I do the UrlFetchApp Post.
This is the payload as string. I have some code that generates the payload. Getting it properly formatted is not the problem, it's the format it is supposed to be.
-----------------0.13accb4c42d338
Content-Disposition: form-data; name="source"; filename="Zzapps.jpg"
Content-Type: application/octet-stream
[[[IMAGE DATA HERE -- OMITTED FOR BREVITY]]]
-----------------0.13accb4c42d338
Content-Disposition: form-data; name="filename"
Zzapps.jpg
-----------------0.13accb4c42d338--
This is the piece of code that does the UrlFetchApp command.
var authHeaders = {
Authorization: 'OAuth2 '+access_token
}
var params = {
accept: "application/json",
method: 'POST',
payload: payload,
contentType: 'multipart/form-data; boundary='+boundaryKey,
headers: authHeaders,
ContentLength: payload.length,
muteHttpExceptions: true
}
var resx = UrlFetchApp.fetch(url, params);
The recieving party gives an error (missing source). I am not sure if my multipart-post is okay in the first place, I do not find any testing URL's to check if I do a proper upload.
How could I send blob-data as a multipart upload the proper way?
And now I use blob.getDataAsString() <-- is this correct ?

If you're building the payload string yourself, you want to do blob.getBytes(), not blob.getDataAsString().
However, there's an easier way. Rather than building the payload string yourself, you can just set a javascript object as the payload, and UrlFetchApp will automatically generate the appropriate payload string, select the correct content-type, boundary, and most of the other options. Just use the "name" attribute of the HTML form's input fields as your object keys. For files, use blobs as the key's values.
function sendReportToSteve() {
var url = "https://example.com/stevedore.html";
var form = {
date : new Date(),
subject : "Happy birthday!",
comment : "quakehashprismkeepkick",
attachment1 : DriveApp.getFileById("sH1proy0lradArgravha9ikE").getBlob(),
attachment2 : DriveApp.getFileById("traCetRacErI3hplVnkFax").getBlob()
};
uploadFile(url,form);
}
function uploadFile(url,form) {
var options = {
method : "POST",
payload : form
};
var request = UrlFetchApp.getRequest(url,options); // (OPTIONAL) generate the request so you
console.info("Request payload: " + request.payload); // can examine it (useful for debugging)
var response = UrlFetchApp.fetch(url,options);
console.info("Response body: " + response.getContentText());
}

Related

Apps Script - UrlFetchApp.fetch {url, method: "GET"} to a gzip gets failed with code 406

I'm on a multi months quest here. Any help would be much appreciated!
I'm trying to connect via API on App Store Connect using UrlFetchApp.fetch(). After having struggled in the desert to generate the correct JWT for authentification on GAS, I now face harsh reality: the server answer (content-type and content-encoding) is not JSON but GZIP.
Code 406 message:
"Truncated server response: The provided Accept header is not supported for this request. Requested: application/json Allowed: application/a-gzip"
Is there a way to still get access to the file?
Here is the part of the code doing the call only (JWT signature code for authentification is above -> "sJWT")
var url = "https://api.appstoreconnect.apple.com/v1/financeReports?filter[regionCode]=ZZ&filter[reportDate]=2019-11&filter[reportType]=FINANCIAL&filter[vendorNumber]=xxx"
var response = UrlFetchApp.fetch(url, { method : "GET", headers : { "Authorization" : "Bearer "+sJWT }});
Many thanks!
How about this answer? Please think of this as just one of several possible answers.
Modification points:
When I saw the official document, it is required to use application/a-gzip for Accept in the request headers.
And also, in this case, the response returns the content of gzip. So it is required to decompress the content.
When above points are reflected to your script, how about the following modification?
Modified script:
var url = "https://api.appstoreconnect.apple.com/v1/financeReports?filter[regionCode]=ZZ&filter[reportDate]=2019-11&filter[reportType]=FINANCIAL&filter[vendorNumber]=xxx"
var response = UrlFetchApp.fetch(url, {
method: "GET",
headers: {
"Authorization": "Bearer " + sJWT,
"Accept": "application/a-gzip" // Added
}
});
var res = Utilities.ungzip(response.getBlob()); // Added
Note:
Above modified script supposes that the values of your URL and sJWT are correct for using the API.
References:
Download Finance Reports
fetch(url, params)
ungzip(blob)
406 Not Acceptable
Unfortunately, I cannot test above script. I apologize for this. So if above modified script didn't resolve your issue, I apologize.
Try changing the blob content type to "application/x-gzip":
var url = "https://api.appstoreconnect.apple.com/v1/financeReports?filter[regionCode]=ZZ&filter[reportDate]=2019-11&filter[reportType]=FINANCIAL&filter[vendorNumber]=xxx"
var response = UrlFetchApp.fetch(url, {
method: "GET",
headers: {
"Authorization": "Bearer " + sJWT,
"Accept": "application/a-gzip"
}
});
var res = Utilities.ungzip(response.getBlob().setContentType("application/x-gzip")); // Changed

Quandl API returns HTML response on Google appscript

API request to quandl for fetching stock data returning HTML response instead of JSON. Its correctly returning JSON result in postman.
var url ='https://www.quandl.com/api/v3/datasets/BSE/BOM'+532540+'?start_date='+startDate+'&end_date='+endDate+'&collapse=weekly&api_key=myapikey'
console.log(url)
var options =
{
'muteHttpExceptions': true,
"contentType" : "application/json",
};
var response = UrlFetchApp.fetch(url, options);
console.log(response)
did anyone have a workaround?
Issue:
Xml response instead of JSON response from quandl api
Solution:
Explicitly mention the format in the url as mentioned in the documentation
GET https://www.quandl.com/api/v3/datasets/{database_code}/{dataset_code}/data.{return_format}
var url ='https://www.quandl.com/api/v3/datasets/BSE/BOM'+532540+'.json?start_date='+startDate+'&end_date='+endDate+'&collapse=weekly&api_key=myapikey'
AND/OR
Try mentioning that you only accept json response in the request using Accept header.
var options =
{
'muteHttpExceptions': true,
"contentType" : "application/json",
"headers":{"Accept":"application/json"}
};

node request module: parsing XML as JSON

Until recently I've been fetching XML data using the node request module, and then running that XML through an XML to JSON converter. I discovered by accident that if I set json: true as an option (even knowing the endpoint returns XML, not JSON), I was actually getting back JSON:
var request = require('request');
var options = { gzip: true, json: true, headers: { 'User-Agent': 'stackoverflow question (https://stackoverflow.com/q/52609246/4070848)' } };
options.uri = 'https://api.met.no/weatherapi/locationforecast/1.9/?lat=40.597&lon=-74.26';
request(options, function (error, response, body) {
console.log(`body for ${options.uri}: ${JSON.stringify(body)}`);
});
The above call returns JSON, whereas the raw URL is actually sending XML. Sure enough, with json: false the returned data is XML:
var request = require('request');
var options = { gzip: true, json: true, headers: { 'User-Agent': 'stackoverflow question (https://stackoverflow.com/q/52609246/4070848)' } };
options.uri = 'https://api.met.no/weatherapi/locationforecast/1.9/?lat=40.597&lon=-74.26';
options.json = false; // <<--- the only difference in the request
request(options, function (error, response, body) {
console.log(`body for ${options.uri}: ${body}`);
});
So I thought "that's handy", until I tried the same trick with a different URL that also returns XML, and in this case the returned data is still XML despite using the same request options:
var request = require('request');
var options = { gzip: true, json: true, headers: { 'User-Agent': 'stackoverflow question (https://stackoverflow.com/q/52609246/4070848)' } };
options.uri = 'https://graphical.weather.gov/xml/SOAP_server/ndfdXMLclient.php?whichClient=NDFDgen&lat=40.597&lon=-74.26&product=time-series&temp=tempSubmit=Submit';
request(options, function (error, response, body) {
console.log(`body for ${options.uri}: ${body}`);
});
What is the difference here? How do I get the latter request to return the data in JSON format (so that I can avoid the step of converting XML to JSON myself)? Maybe the endpoint in the first example can detect that JSON is requested and it does in fact return JSON rather than XML?
EDIT weirdly, the first request is now returning XML rather than JSON even with json: true. So maybe this behaviour was down to what was being sent from the endpoint, and they've changed this even since I posted a few hours ago
So now that the behavior is unrepeatable, the answer is less useful for your particular problem, but I think it's worth pointing out that when you set json:true on the request module, it does a few things under the hood for you:
Sets the Accept header to 'application/json'
Parses the response body using JSON.parse()
Request types with a body also get the body automatically serialized as JSON
Request types with a body also get the Content-Type header added as 'application/json'
So perhaps they did change it, but there are plenty of web services I've seen that will detect the content-type to send based on the Accept header and respond appropriately for some set of types that make sense (usually XML or JSON, but sometimes CSV, TXT, HTML, etc).
To handle XML query, I usually do something like this using the request module:
import parser from "xml2json";
const resp = await rp({
method: "POST",
url: 'some url',
form: {xml_query}, // set XML query to xml_query field
});
const parsedData = parser.toJson(resp, {
object: true, // returns a Javascript object instead of a JSON string
coerce: true, // makes type coercion.
});

Correct syntax for Google API PATCH request using UrlFetchApp for HTTPS Request

I'm trying to use UrlFetchApp.fetch(url) method in Apps Script to PATCH a groups resource using the Google Groups Settings API.
The code below allows me to GET the groups properties, but I'm unable to figure out the syntax for a PATCH request.
function doSomething (accessToken) {
var options = {
method: "GET",
headers: {
authorization: "Bearer " + accessToken
},
};
var result = UrlFetchApp.fetch("https://www.googleapis.com/groups/v1/groups/test_group_5#student.vis.ac.at", options);
return HtmlService.createHtmlOutput (result.getContentText());
}
A PATCH request needs a Header Override. You actually need to use a PUT request, and then override it to a PATCH request.
var payload = "{\"" + PropertyOne + "\":\"" + "Proptery Value" + "\"}";
Logger.log('payload: ' + payload);
var options = {"method" : "put", "headers": {"X-HTTP-Method-Override": "PATCH"}, "payload" : payload};
if (payload.length > 2) {
UrlFetchApp.fetch("https://www.googleapis.com/groups/v1/groups/test_group_5#student.vis.ac.at", options );
};
The code above won't be exactly what you want, and might not be error free, but the structure of it should be what you need. I'm sure the payload isn't configured correctly, because I don't know what the format is. It looks like the documentation calls it Patch body with an object.
Google Documentation - Group Settings API Patch
Key words: "Apps Script", patch

How to send a JSON payload with UrlFetchApp service?

I'm trying to POST to a web service that is expecting to get JSON as payload using Google Apps Script. I'm using the following code:
var options =
{
"method" : "post",
"contentType" : "application/json",
"headers" : {
"Authorization" : "Basic <Base64 of user:password>"
},
"payload" : { "endDate": "2012-06-03" }
};
var response = UrlFetchApp.fetch("http://www.example.com/service/expecting/json", options);
On the server side I'm getting the following error:
WARN [facade.SettingsServlet] 04 Jun 2012 15:30:26 - Unable to parse request body: endDate=2012-06-03
net.liftweb.json.JsonParser$ParseException: unknown token e
I'm assuming that the server is expecting to get
{ "endDate": "2012-06-03" }
instead of
endDate=2012-06-03
but I don't know how to make the UrlFetchApp do it.
I do not understand the server side error but the 'payload' parameter must be a string as specified here: https://developers.google.com/apps-script/class_urlfetchapp?hl=fr-FR#fetch.
try:
var options =
{
"method" : "post",
"contentType" : "application/json",
"headers" : {
"Authorization" : "Basic <Base64 of user:password>"
},
"payload" : '{ "endDate": "2012-06-03" }'
};
If you set payload as a String, it will be passed directly (as a
UTF-8 string).
If you set payload as an Object, it will be sent like
an HTML form (which means either 'application/x-www-form-urlencoded' if the
fields are simple, or 'multipart/form-data' if the Object includes a
blob/file).
For your use case (the server is expecting to receive JSON), it sounds like Utilities.jsonStringify() is the way to go.
Something like this worked for me in a similar situation:
Instead of creating payload and adding to options, I built the parameters into a string to append to the URL:
var params = "id=2179853&price=62755";
then, I appended params to the url string:
var result = UrlFetchApp.getRequest(url + '?' + params, options);
So, my options was passed containing only the header.
For my project, this worked like a charm.
Here goes the code that should work with some important comments:
function testMe() {
var products_authkey = "------------";
try {
var url = "https://app.ecwid.com/api/v1/---------/product?id=----------&secure_auth_key=" + products_authkey;
//url= "http://requestb.in/----------"; // you can actually debug what you send out with PUTs or POSTs using Requestb.in service
var payload = {
id: "21798583", // id is necessary and should be a string, or it might be sent in scientific representation (with E)
price: 62755
};
payload = JSON.stringify(payload); // the payload needs to be sent as a string, so we need this
var options = {
method: "put",
contentType: "application/json", // contentType property was mistyped as ContentType - case matters
payload: payload
};
var result = UrlFetchApp.getRequest(url, options);
Logger.log(result) // a better way to debug
var result = UrlFetchApp.fetch(url, options); // works perfectly in my case
Logger.log(result)
} catch (e) {
Logger.log(e)
}
}