Exception: Limit Exceeded: URLFetch URL Length - google-apps-script

Question

I am pulling data from an API which unfortunately has very big urls/end points.
Is there any way to shorten urls in order to bypass the Exception: Limit Exceeded: URLFetch URL Length error?
I found this: https://github.com/amitwilson/GoogleAppsScript/blob/master/LinkShortner but I don't know how to use it properly since most of the api calls i make are generated dynamically.
This is an example link (letters after "cursor=" is a cursor key which is different every time):
https://api.x.immutable.com/v1/orders?include_fees=false&status=filled&buy_token_address=0xacb3c6a43d15b907e8433077b6d38ae40936fe2c&order_by=updated_timestamp&direction=desc&user=0x72a2ffa8be361d4a5d575c55b7382d8c6317f77d&page_size=200&cursor=eyJvcmRlcl9pZCI6MTk0NTk2NzUyLCJzZXFfbnVtIjowLCJzdGF0dXMiOiJmaWxsZWQiLCJzdGFya19rZXkiOiIweDA3MWU2NjU0NTMzMGJiOGY5NDQ4ZWU1YTM3YTI5ODViZGYzYjQ1NzYwNDA5ODE4YmEwYTY0NmUxNjU4NTdhMWIiLCJ2YXVsdF9pZCI6MCwiZXRoZXJfa2V5IjoiMHg3MmEyZmZhOGJlMzYxZDRhNWQ1NzVjNTViNzM4MmQ4YzYzMTdmNzdkIiwic2VsbF90b2tlbl90eXBlIjoiRVRIIiwic2VsbF90b2tlbl9pZCI6IiIsInNlbGxfY2xpZW50X3Rva2VuX2lkIjoiIiwic2VsbF90b2tlbl9hZGRyZXNzIjoiIiwic2VsbF90b2tlbl9kZWNpbWFscyI6MTgsInNlbGxfcXVhbnRpdHkiOiIyODQzMDAwMDAwMDAwMDAwIiwic2VsbF9hc3NldF9uYW1lIjpudWxsLCJzZWxsX2Fzc2V0X2ltYWdlX3VybCI6bnVsbCwic2VsbF9jb2xsZWN0aW9uX25hbWUiOm51bGwsInNlbGxfY29sbGVjdGlvbl9pY29uX3VybCI6bnVsbCwiYnV5X3Rva2VuX3R5cGUiOiJFUkM3MjEiLCJidXlfY2xpZW50X3Rva2VuX2lkIjoiMjI0ODUzMTMiLCJidXlfdG9rZW5faWQiOiIweDU0ODRkZjA1NzM0OTkxNTg0NWNkNjM0YmVmNDAxNjY1YmE1YjAzZDEwNzE4YmU2ZTQxNmEyMTFhNmVhMGJkNzEiLCJidXlfdG9rZW5fYWRkcmVzcyI6IjB4YWNiM2M2YTQzZDE1YjkwN2U4NDMzMDc3YjZkMzhhZTQwOTM2ZmUyYyIsImJ1eV90b2tlbl9kZWNpbWFscyI6MCwiYnV5X3F1YW50aXR5IjoiMSIsImJ1eV9xdWFudGl0eV9pbmNsdXNpdmVfZmVlcyI6IjI4NDMwMDAwMDAwMDAxIiwiYnV5X2Fzc2V0X25hbWUiOiJFbGl4aXIgb2YgdGhlIFNuYWtlIiwiYnV5X2Fzc2V0X2ltYWdlX3VybCI6Imh0dHBzOi8vY2FyZC5nb2RzdW5jaGFpbmVkLmNvbS8_aWQ9ODU3XHUwMDI2cT00IiwiYnV5X2NvbGxlY3Rpb25fbmFtZSI6IkdvZHMgVW5jaGFpbmVkIiwiYnV5X2NvbGxlY3Rpb25faWNvbl91cmwiOiJodHRwczovL2ltYWdlcy5nb2RzdW5jaGFpbmVkLmNvbS9taXNjL2d1LXNpZ2VsLnBuZyIsImFtb3VudF9zb2xkIjoiMjg0MzAwMDAwMDAwMDAwMCIsImV4cGlyZWRfYXQiOiIyMTIxLTA1LTAzVDEwOjAwOjAwWiIsImNyZWF0ZWRfYXQiOiIyMDIyLTA1LTAzVDEwOjIwOjM0Ljc4ODkxN1oiLCJ1cGRhdGVkX2F0IjoiMjAyMi0wNS0wM1QxMDoyMDozNC43ODg5MTdaIiwiRmVlcyI6Ilczc2lZVzF2ZFc1MElqb2dJakk0TkRNd01EQXdNREF3TURBd0lpd2dJbVpsWlY5MGVYQmxJam9nSW0xaGEyVnlJaXdnSW05eVpHVnlYMmxrSWpvZ01UazBOVGsyTnpVeUxDQWljbVZqYVhCcFpXNTBYMlYwYUdWeVgydGxlU0k2SUNJd2VEVmpNell6WVRKaFlUZzJPV1F3WXpVeU1XTm1ZVE5sWWpCbE16YzBPVEpqTUdWaU5UbGxOVGdpTENBaWNtVmphWEJwWlc1MFgzTjBZWEpyWDJ0bGVTSTZJQ0l3ZURBMk1UY3haVEpoWlRRMU5qTXlNVE15TWpaak5XWm1aV1JpT1RSaE9UVXlOV0ppWm1Zell6bGtZMkpoWm1KaFpURTVabUk0TjJOa1ltSXpOMk5sTVdVaWZWMD0ifQ

Answer 1

In your situation, for example, how about the following sample script?
Sample script:
Before you use this script, please set apiKey and yourDynamicLinkDomain. Your yourDynamicLinkDomain might be like https://###.page.link.
const apiKey = "###"; // Please set your API key.
const yourDynamicLinkDomain = "###"; // Please set your dynamic link domain.
function getShortUrl_(longUrl) {
const url = "https://firebasedynamiclinks.googleapis.com/v1/shortLinks?key=" + apiKey;
const options = {
payload: JSON.stringify({
dynamicLinkInfo: {
dynamicLinkDomain: yourDynamicLinkDomain,
link: longUrl,
},
}),
contentType: "application/json",
};
const res = UrlFetchApp.fetch(url, options);
const { shortLink } = JSON.parse(res.getContentText());
return shortLink;
}
// Please run this function.
function main() {
// do something for retrieving the long URL in your actual script.
const longUrl = "###"; // It supposes that your long URL is put to longUrl.
const shortUrl = getShortUrl_(longUrl); // Here, the long URL is shortned. You can use this.
// If "options" is required to be used, please use this.
const options = {
};
const res = UrlFetchApp.fetch(shortUrl, options);
}
In this sample script, it supposes that by your current script, longUrl is obtained. And, this sample script converts from the long URL to the short URL with the function getShortUrl_. And, you can use this short URL using UrlFetchApp.
Note:
This sample script supposes that you have already been able to use Firebase Dynamic Links API. Please be careful about this.
If you have never been able to be used Firebase Dynamic Links API, please check the official document. And, the following flow might be useful.
please check the following flow.
Please create a new Firebase project and link it to your Google Cloud Platform Project. Ref
At this time, please set dynamicLinkDomain at "Dynamic Links".
Please enable Firebase Dynamic Links API at the API console.
Please create your API key from your Google Cloud Platform Project.
Please use this API in the above script.
When I tested this script, I could confirm that the long URL is converted to the short URL and the short URL can be used with UrlFetchApp.
If I misunderstood about but I don't know how to use it properly since most of the api calls i make are generated dynamically., please tell me.
References:
Create Dynamic Links with the REST API
Class UrlFetchApp
Added:
From your following reply,
Hey I made a sample sheet so you can see how I get the longurl it is highlighted with yellow color in 202 row. docs.google.com/spreadsheets/d/… The API works like this: it can pull 200 rows, then on the last row of each call it shows this cursor ID which must be called again in next row in order to show the next 200 rows.
Check the sample sheet on my previous reply. Longurl is highlighted with YELLOW color.
When I saw your sample Spreadsheet, it seems that you are using the long URL as =importjson(CONCATENATE("https://api.x.immutable.com/v1/orders?include_fees=false&status=filled&buy_token_address=0xacb3c6a43d15b907e8433077b6d38ae40936fe2c&order_by=updated_timestamp&direction=desc&user=0x8604808B824C4444Fe4dF3f94850ACd584C5aD7d&page_size=200&cursor=",W202),"","noTruncate,noHeaders"). But unfortunately, from your question and your reply, I couldn't notice that you are using it like that.
In this case, it is required to use my sample script by modifying as follows.
Sample script:
Please copy and paste the following script to the script editor of your Spreadsheet and save the script.
function getShortUrl(longUrl) {
const apiKey = "###"; // Please set your API key.
const yourDynamicLinkDomain = "###"; // Please set your dynamic link domain.
const url = "https://firebasedynamiclinks.googleapis.com/v1/shortLinks?key=" + apiKey;
const options = {
payload: JSON.stringify({
dynamicLinkInfo: {
dynamicLinkDomain: yourDynamicLinkDomain,
link: longUrl,
},
}),
contentType: "application/json",
};
const res = UrlFetchApp.fetch(url, options);
const { shortLink } = JSON.parse(res.getContentText());
return shortLink;
}
When this script is used for your sample Spreadsheet, please put the following formula to the cell "A203".
=importjson(getShortUrl(CONCATENATE("https://api.x.immutable.com/v1/orders?include_fees=false&status=filled&buy_token_address=0xacb3c6a43d15b907e8433077b6d38ae40936fe2c&order_by=updated_timestamp&direction=desc&user=0x8604808B824C4444Fe4dF3f94850ACd584C5aD7d&page_size=200&cursor=",W202)),"","noTruncate,noHeaders")
By this formula, the long URL retrieved with CONCATENATE("https://api.x.immutable.com/v1/orders?include_fees=false&status=filled&buy_token_address=0xacb3c6a43d15b907e8433077b6d38ae40936fe2c&order_by=updated_timestamp&direction=desc&user=0x8604808B824C4444Fe4dF3f94850ACd584C5aD7d&page_size=200&cursor=",W202) is shorten by shortenUrl(). And, use the shorten URL with importjson.
Note:
This sample script supposes that you have already been able to use Firebase Dynamic Links API. Please be careful about this.
When I tested the above script and formula to your sample Spreadsheet, I confirmed that the values are retrieved.