How can I access the contents of a (new-style) Google sheet a JSON? My aim is to access the values from JavaScript, so I need to be able to download the JSON via HTTP.
Example: how can I download the data from this sheet as JSON?
I tried to find the answer via a web search, but ultimately failed:
Many tutorials on the web start with the instruction to find the key=... value in the url. The URL I got when I exported the sheet is https://docs.google.com/spreadsheets/d/1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI/pubhtml?gid=1822753188&single=true and has no key=... in it.
The answer to "Accessing A Public Google Sheet" seems to indicate that I should try https://docs.google.com/spreadsheets/d/1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI/export?format=csv&id=1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI&gid=1822753188 to get a CSV version, but this does not work for me: I get a sign-in page instead of the data.
I found approaches using Google Apps Scripts, but these seem to require some user action in the browser instead of giving a download link.
If you want to use the latest API (v4), you'll need to do the following:
Generate a spreadsheets API key (see instructions below).
Make your sheet publicly accessible.
Use a request of the form:
https://sheets.googleapis.com/v4/spreadsheets/SPREADSHEET_ID/values/RANGE?key=API_KEY
You'll then get a clean JSON response back:
{
"range": "Sheet1!A1:D5",
"majorDimension": "ROWS",
"values": [
["Item", "Cost", "Stocked", "Ship Date"],
["Wheel", "$20.50", "4", "3/1/2016"],
["Door", "$15", "2", "3/15/2016"],
["Engine", "$100", "1", "30/20/2016"],
["Totals", "$135.5", "7", "3/20/2016"]
],
}
Note that if you want to specify the entire contents of a page, an identifier such as Sheet1 is perfectly valid.
See Basic Reading for more information.
As of v4 API, all requests must be accompanied by an identifier (e.g. API key):
Requests to the Google Sheets API for public data must be accompanied by an identifier, which can be an API key or an access token.
Follow the steps in the linked document to create an API key on the credentials page.
Make sure to:
Create a new app on Google Cloud Platform.
Create a new API key.
Add the Google Sheets API. (API Manager > Dashboard > Enable API)
Note that you can still access public data without forcing the user to log in:
In the new Sheets API v4, there is no explicit declaration of visibility. API calls are made using spreadsheet IDs. If the application does not have permission to access specified spreadsheet, an error is returned. Otherwise the call proceeds.
Note that you do not need to publish the sheet to the web. All you need to do is make sure anyone with the link can access the sheet.
(I.e. when you click Create credentials on the Google Sheets API, choose Other non-UI, User data, and it says "User data cannot be accessed from a platform without a UI because it requires user interaction for sign-in." you can safely ignore that message. The API Key is all you really need, since this is public data.)
Common error messages:
The request is missing a valid API key.
You didn't include the key= param in your call.
API key not valid. Please pass a valid API key.
Google developers console
You supplied an incorrect API key. Make sure that you typed in your key correctly. If you don't have a key yet, go to the Google developers console and create one.
API Key not found. Please pass a valid API key.
Google developer console API key
Your API Key is probably correct, but you most likely didn't add the Google Sheets permission. Go to the Google developer console API key page and add the sheets permission.
The caller does not have permission
Your sheet isn't set to be publicly accessible.
I have finally (kind of) solved my problem. Just for future reference, and in case somebody else runs into the same troubles, here the solution I came up with:
To make the worksheet publicly accessible, one needs to make the worksheet publicly accessible. This is done in the Google Sheets web interface, using the menu entries File > Publish to the web ... > link > publish. It is possible to either publish the whole spreadsheet or individual worksheets.
An API to access data from Google Sheets programmatically is described on the Google Sheets API web pages. This API uses URLS of the form https://spreadsheets.google.com/feeds/.../key/worksheetId/.... Slightly oddly, the meaning of key and worksheetId seems not to be explained in the API documentation.
My experiments show that the key value can be found by taking part of the URLs used to access the sheet via the web interface (see also here). The key is everything after the /d/, until the next slash. For the spreadsheet in the question, the key is thus 1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI. The worksheetId seems to be an integer, giving the position of the worksheet in the spreadsheet. For the example in the question one has to know that the sheet shown is the second worksheet, the worksheetId in this case is 2.
The API defined public and private requests. To access an exported resource without authentication, public requests must be used.
The API calls to get data from the spreadsheet are explained in the section "Retrieving a list-based feed" (click on the "Protocol" tab in the examples). The URL required extract the data from the spreadsheet in the question is
https://spreadsheets.google.com/feeds/list/1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI/2/public/full
A HTTP GET request to this URL returns that data as XML. (I have not found a way to get the data as JSON.)
The usual protections agains cross-site requests make it difficult to access the data via JavaScript XML RPC calls in a web app. One way around this problem is to proxy the API calls through the web server (e.g. using nginx's proxy_pass directive).
The above steps are at least a partial solution to the problem in the question. The only difficulty is that the data is returned as XML rather than as JSON. Since the API documentation does not mention JSON, maybe it is not possible any more to extract the data in this format?
Edit: (Aug 17, 2021) With the rollout of Sheets v4, the endpoint in the original answer has been deprecated. The updated endpoint and sample script included below:
Updated solution
Credits to the original answer here.
"https://docs.google.com/spreadsheets/d/" + spreadsheetId + "/gviz/tq?tqx=out:json&gid=0";
You don't technically have to include the gid if you just want the first sheet, but you can specify another sheet if you'd like using that parameter.
Here's a sample script to retrieve values of Spreadsheet as JSON, and then parsed as header row and values.
var sf = "https://docs.google.com/spreadsheets/d/1l7VfPOI3TYtPuBZlZ-JMMiZW1OK6rzIBt8RFd6KmwbA/gviz/tq?tqx=out:json";
$.ajax({url: sf, type: 'GET', dataType: 'text'})
.done(function(data) {
const r = data.match(/google\.visualization\.Query\.setResponse\(([\s\S\w]+)\)/);
if (r && r.length == 2) {
const obj = JSON.parse(r[1]);
const table = obj.table;
const header = table.cols.map(({label}) => label);
const rows = table.rows.map(({c}) => c.map(({v}) => v));
console.log(header);
console.log(rows);
}
})
.fail((e) => console.log(e.status));
Original solution
Note: This no longer works as Sheets v3 was deprecated in August 2021.
Here's how to get the JSON using those same URL parameters:
"https://spreadsheets.google.com/feeds/list/" + spreadsheetID + "/od6/public/values?alt=json";
Creds to #jochen on the answer with the path all the way up to XML "https://spreadsheets.google.com/feeds/list/" + spreadsheetID + "/od6/public/" + sheetID;
As #jochen's answer explains, this sheetID is based on the order of the sheets in the spreadsheet.
A faster solution here is to use this https://gist.github.com/ronaldsmartin/47f5239ab1834c47088e to wrap around your existing spreadsheet.
You first need to change your sheet access to Anyone with link can View
Add the id and sheet html param to the URL below.
https://script.google.com/macros/s/AKfycbzGvKKUIaqsMuCj7-A2YRhR-f7GZjl4kSxSN1YyLkS01_CfiyE/exec
Eg: your id is your sheet id which is
1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI
and your sheet which is
Sheet2
In your case you can actually see your data here as json at
https://script.google.com/macros/s/AKfycbzGvKKUIaqsMuCj7-A2YRhR-f7GZjl4kSxSN1YyLkS01_CfiyE/exec?id=1mhv1lpzufTruZhzrY-ms0ciDVKYiFJLWCMpi4OOuqvI&sheet=Sheet2
To be safe, you should deploy the code sheetAsJson.gs in the github gist above as your own in your Google Drive.
Here is the solution
Note your sheet id in the document url (don't use the published url to find the id!)
Publish your sheet, just as html page
Use the id from step 1,
and put it in this url https://spreadsheets.google.com/feeds/cells/{id}/1/public/full?alt=json
The /1 indicates the first sheet in your document
Related
I would like to use Apps Script to save anything sent to the Web App URL as a text file on Google Drive. I believe that this should be done with a basic doGet and doPost setup.
If I could get the file saved with a unique filename, that'd be great, but it's not important. I thought there would be a tutorial somewhere about this, but I can't seem to find it at all.
A basic workflow of what you need to do:
Create a WebApp with a doGet() function
Retrieve the data your application has sent to your WebApp url with e.parameter or e.parameters - depending on either it's a single string or an array, your retrieved data should be assigned to certain key
If the key is called "test" and your data is passed as https://script.google.com/a/XXX/macros/s/XXX/exec?test=Hello, you should query for myString = e.parameter.name
Create a text file on your drive with DriveApp.createFile(name, content, mimeType) - specifying a name of your choice, passing the retrieved string as content and chosing mimeType PLAIN_TEXT
I am writing a custom google scripts library for use within multiple google sheets. It uses the Google Script OAuth2 library to manage the OAuth flow with the Smartsheet API. Unfortunately it is returning 'Redirect URI missing or invalid'.
I had the OAuth2 flow working well in a single spreadsheet script, but it stopped working when I moved the code into a library.
I DID update the redirect URI within Smartsheet's developer menu and I verified that the URI returned from .getRedirectUri() was the one associated with my app within Smartsheet's app registration.
The other aspects of the code within the library are working well, and give me high confidence that it is still working as it did when it was tied to a sheet.
My goal with the library is that it can be used from multiple sheets to access the Smartsheet API, and thereby prevent me from creating a unique app (with unique redirect URI) within the Smartsheet developer menu for each sheet that might ever use this code.
Is this possible? What might I be doing wrong that it doesn't like the redirect URI?
When developing a library it is best to be agnostic to the source of token. A common design pattern is to have the user pass in a function that your library calls to get a token. This allows the user to choose how and where the OAuth workflow is implemented.
myLibrary.gs
var tokenService_ = function(){return null};
function setTokenService(tokenService){
tokenService_ = tokenService
}
//some hypothetical function in your library
function getData(options){
return fetchFromAPI(options,tokenService_());
}
userCode.gs
function requestData(){
myLibrary.setTokenService(function(){ return ScriptApp.getOAuthToken()});
var data = myLibrary.getData("foo");
}
If you create a non-container bound g-apps script (i.e. not as part of a gDoc or a gSheet), you can download it (however not view as a .json directly in the browser from the link) from gDrive as a .json. If you download a gDoc or gSheet, it converts to xlsx or docx and opening these with a zip viewer shows a number of files (many of type xml) however none contain the Google version's attached scripts.
Is there a way to read script files as a .json from within another Google Apps
Script? perhaps using the Drive-API or with g-a-s. DriveApp class?
Is there a way to download or read through DriveApp, the .jsons of
container bound scripts (which are usually invisible from all but within the original gFile)?
Update
Based on Kriggs, added a Logger.log(link) and this works great for stand-alone scripts.
How about for container-bound?
for stand alone script files:
exportLinks={
application/vnd.google-apps.script+json=
script.google.com/feeds/download/export?id=[scriptId]&format=json
}
for container-bound script files, there are links to csv, sheet and pdf, but no script json.
exportLinks= {
text/csv=docs.google.com/spreadsheets/export?id=[sheetId]&exportFormat=csv,
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet=
docs.google.com/spreadsheets/export?id=[sheetId]exportFormat=xlsx,
application/pdf=
docs.google.com/spreadsheets/export?id=[sheetId]&exportFormat=pdf
}
Update
In Google sheet, go to Tools->script Editor->
URL in address bar looks like:
https://script.google.com/macros/d/
[ProjectKey]/edit?uiv=2&mid=[aVeryLongAlphaNum]
this is the download json:
https://script.google.com/feeds/download/export?id=[ProjectKey]
Question is, can we use the Drive API to find [ProjectKey]
Have there been any feature requests for DriveApp/Drive-API methods to seek Project Keys in your account?
Would there be a way to test if a file has a container bound script? Then the question is, is the script included in the file size (this can be easily tested, however it is unknown to the asker at this point).
Something like this may work although it looks computationally costly:
var SizeOfFile = yourFile.getSize();//
var charsInFile = yourFile.getAsString();
var unicodeSizeReference = [];//get bytes per character array
charsInFile.sort()
//find frequency of characters then multiply from unicoseSizeReference.
//there could be other gotchas as well, however this is just testing for feasibility
var SizeOfTextInFile = [/*#of chars in file name and sheetname*/]+[/*#of chars in all sheets*/];
SizeOfTextInFile *= unicodeBytesPerCharacter;//ranges from 1 to 4
var someThreshold = 10;//bytes
var hasScript=0;
if ([SizeOfFile - SizeOfTextInFile] > someThreshold) hasScript=1
Yes you have to get it trough the Drive API with OAuth2, I used the DriveApp to get the fileId, but you can modify to use Drive api aswell. To enable the Drive API go to Resources -> Advanced Google Services, find the Drive API and turn on.
When you send a get with Drive you get back an object of the file which contains the property exportLinks, using it you fetch the URL with OAuth2 authentication (the ScriptApp.getOAuthToken()), the fetched string will be a JSON, which has the Array fileswith the colection of scripts.
function getAppsScriptAsJson( fileName ) {
var fileDrive = Drive.Files.get( DriveApp.getFilesByName( fileName ).next().getId() );
var link = JSON.parse(fileDrive)[ 'exportLinks' ][ 'application/vnd.google-apps.script+json' ];
var fetched = UrlFetchApp.fetch(link, {headers:{'Accept':'application/vnd.google-apps.script+json', "Authorization":'Bearer '+ScriptApp.getOAuthToken()}, method:'get'});
return JSON.parse(fetched.getContentText());
}
As for container bound:
DriveApp can't get it by name
It doesn't display an ID anywhere, just the project key
Drive API can't lookup by the project id, nor DriveApp
Drive API can't find by the name
There's no reference of the script from the returned object from Drive API nor the DriveApp
I guess it is pretty much incognito, doubt there's any way ATM.
You can always make a Standalone app and set it as a library for the Spreadsheet...
My app accesses private Google spreadsheet documents on behalf of an authorized user. It seems that Google's API expects developers to first request a list of all the spreadsheet documents available to an authorized user before they can get at a particular spreadsheet's keys. I wanted to find a workaround to this, and eventually did by extracting the key parameter value from URLs spreadsheet URLs that look like this:
https://docs.google.com/spreadsheet/ccc?key={some long key here}&usp=drive_web#gid=0
It was simple enough to just break down the string to point where I could retrieve key's value fairly easy without the need of a regex.
Recently, though I don't know how recent, I notice URLs to newly created Google Drive spreadsheets come in this form:
https://docs.google.com/spreadsheets/d/{some long key here}/edit#gid=0
I was also able to extract the key from this URL string, but am just curious about the difference between the two URLs:
What is the significance between the two URLs.
Why does Google's API force devs to first get a list of all available docs, when a dev might just want to extract a key from a direct URL to a Google Drive spreadsheet doc.
Thanks!
Old style sheets
They work online only and limited to about 400,000 cells per spreadsheet.
Old style URL
https://docs.google.com/spreadsheet/ccc?key={some long key here}&usp=drive_web#gid=0
New style sheets
Released about mid Dec 2013
Works offline and (if I remember) up to 2,000,000 cells per spreadsheet.
https://docs.google.com/spreadsheets/d/{some long key here}/edit#gid=0
Spreadsheet KEY
I get the key using Google-apps-script, as described here:
Get the spreadsheet key that is in the URL. Not ss.getId()
Where are you getting the URL from? You shouldn't rely on specific URL formats, these are subject to change and not intended to be reliable. You should be able get just the id by specifying the "fields" parameter in your request. See https://developers.google.com/drive/v2/reference/files/list
Cloudward has solved this through Cloud Snippets. Here's two that may be of help, there are lots of others to explore as well.
Publish Simple List from Google Sheet:
https://snippets.cloudward.com/app_listing.espx?template_id=0d367025e8b5f402cd510905cade1d29&account_id=&cat_id=c478885bb325028151eaa9060422c67f
Publish Google Doc by ID:
https://snippets.cloudward.com/app_listing.espx?template_id=51925e7ed2166d7d83a8c32fa1ee88dd&account_id=
Hope this helps.
Bob
I recently started using the "New Google Sheets" (spreadsheets) and they changed the URL to a shared public spreadsheet and I am unsure how to obtain a JSON feed of the spreadsheet data.
Based on data from this URL: https://developers.google.com/gdata/samples/spreadsheet_sample
I can obtain the JSON data from an older version spreadsheet using the key parameter found in a URL of this format: http://spreadsheets.google.com/feeds/feed/key/worksheet/public/basic?alt=json-in-script&callback=myFunc
However, the new sheets have a URL like this:
https://docs.google.com/spreadsheets/d/SOME-IDENTIFIER/pubhtml
Using "SOME-IDENTIFIER" in place of the key does not work, I'm not sure how I can pull the JSON feed from a new spreadsheet... Anyone have any experience with this?
The format you should use is this :
https://spreadsheets.google.com/feeds/cells/
SHEET-IDENTIFIER/
SHEET_INDEX/
public/basic?alt=json-in-script&callback=JSON_CALLBACK
You can find SHEET_INDEX by looking at the end of the URL while editing the sheet.
...SHEET-IDENTIFIER/edit#gid=SHEET_INDEX
Make sure the spreadsheet is published (not only shared).
File/Pulish to the web...
I tried Vlad's answer and it didn't work; the editing url had gid=0, but in order to get the feed I wanted I had to put in 1 as the sheet index.