Google Sheets: How to import the following data? - google-apps-script

Data Source
https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yieldAll
I am trying to get the following data onto a Google Sheet, but it is looking to be tricky to do so using IMPORTXML. Any idea how to do it?

You want to retrieve a table from the HTML data of the URL.
From I am trying to get the following data onto a Google Sheet, I thought like this.
If my understanding is correct, how about this answer?
Issue and workaround:
Unfortunately, it seems that the file size of HTML is large. So when =IMPORTXML("https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yieldAll","//title") is used, an error of Resource at url contents exceeded maximum size. occurs. When I retrieve HTML data from the URL, the size of HTML data was about 9 MB. It is considered that the reason of error is due to this. So as one of workaround, how about using Google Apps Script? In this workaround, the following flow is used.
Retrieve HTML data using UrlFetchApp
Parse the HTML data using Parser which is a GAS library.
Put the parsed data to the active sheet on the Spreadsheet using PasteDataRequest of Sheets API.
Usage:
Preparation:
Please install Parser. About the install of library, you can see it at here.
The project key of the library is M1lugvAXKKtUxn_vdAG9JZleS6DrsjUUV.
Please enable Sheets API at Advanced Google services.
Sample script:
Please copy and paste the following script to the script editor of the container-bound script of the Spreadsheet. After above settings were done, please run the function of myFunction(). When the script is run, the table of HTML is put to the active sheet on the Spreadsheet.
function myFunction() {
// Retrieve HTML data from URL.
var url = "https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yieldAll";
var html = UrlFetchApp.fetch(url).getContentText();
// Parse HTML data.
var table = "<table" + Parser.data(html).from("<table class=\"t-chart\"").to("</table>").build() + "</table>";
// Put the values to the Spreadsheet.
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getActiveSheet();
var resource = {requests: [{pasteData: {html: true, data: table, coordinate: {sheetId: sheet.getSheetId()}}}]};
Sheets.Spreadsheets.batchUpdate(resource, ss.getId());
}
References:
Parser
PasteDataRequest
Advanced Google services
If I misunderstood your question and this was not the direction you want, I apologize.
Updated at April, 23, 2021:
New IDE for Google Apps Script has finally been released at December 7, 2020. Ref By this, in the current stage, in order to install Google Apps Script library, it is required to use the script ID of Google Apps Script project.
In this case, when the Google Apps Script library of Parser is installed, unfortunately, this ID M1lugvAXKKtUxn_vdAG9JZleS6DrsjUUV cannot be used.
So when you use new IDE, please use the following script ID.
1Mc8BthYthXx6CoIz90-JiSzSafVnT6U3t0z_W3hLTAX5ek4w0G_EIrNw
This script ID is the ID of Google Apps Script project of M1lugvAXKKtUxn_vdAG9JZleS6DrsjUUV. By this, the library of Parser can be installed to the new IDE.
About the method for installing the library, you can see the official document.
Reference:
Libraries

Related

Bulk copy files on google drive from a list of hyperlinks

I have a list of 200 hyperlinks saved on a spreadsheet. Those links are for files (particularly Google Slides files) all saved in Google Drive. They are scattered in sub folders under the same root folder that has ~1500 files
Link 1
Link 2
Link 3
...
Link 200
I want to make a copy of those 200 files only. There is no common search term or filter to pull them up on Google Drive search. So I need to work off that list
Thoughts on doing this? Thanks in advance!
I believe your current situation and your goal as follows.
You have the Spreadsheet including 200 hyperlinks of Google Slides like https://docs.google.com/presentation/d/FILE_ID.
You want to copy the Google Slides to the specific folder in your Google Drive.
You want to achieve this using Google Apps Script.
From the number of hyperlinks, I thought that in this case, the batch request might be useful for your situation. When the batch request is used, the process cost will become low because the batch request is run with the asynchronous process. So, in this answer, I would like to propose to copy the files of 200 hyperlinks using the batch request. The sample script is as follows.
Usage:
1. Install a Google Apps Script library.
In this script, in order to achieve the batch request, a Google Apps Script library is used. Ref I thought that the request of the batch request might be a bit complecate. Ref So I created this library for using the batch request with Google Apps Script. The library's project key is as follows.
1HLv6tWz0oXFOJHerBTP8HsNmhpRqssijJatC92bv9Ym6HSN69_UuzcDk
The method for installing the library can be seen at the official document. Ref
2. Sample script.
Please copy and paste the following script to the script editor of your Google Spreadsheet including 200 hyperlinks like https://docs.google.com/presentation/d/FILE_ID. This script used Drive API. So, before you use this script, please enable Drive API at Advanced Google services. And, run the function "myFunction".
function myFunction() {
const sheetName = "Sheet1"; // Please set the sheet name.
const destinationFolderId = "###"; // Please set the destination folder ID.
// 1. Retrieve file IDs from Spreadsheet.
const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName);
const fileIds = sheet.getRange("A1:A" + sheet.getLastRow()).getValues().reduce((ar, [a]) => {
if ((/^https:\/\/docs.google.com\/presentation\/d/).test(a)) ar.push(a.split("/")[5]);
return ar;
}, []);
console.log(fileIds) // You can check the retrieved file ID.
// 2. Retrieve the filenames form the file IDs using the "Files: get" method with the batch request.
const requests1 = fileIds.map(id => ({
method: "GET",
endpoint: `https://www.googleapis.com/drive/v3/files/${id}?supportsAllDrives=true`,
}));
const res1 = BatchRequest.EDo({batchPath: "batch/drive/v3", requests: requests1});
console.log(res1) // You can check the retrieved file metadata.
// 3. Copy the files using the file IDs and filenames using the "Files: copy" method with the batch request.
const requests2 = res1 .map(({id, name}) => ({
method: "POST",
endpoint: `https://www.googleapis.com/drive/v3/files/${id}/copy?supportsAllDrives=true`,
requestBody: {parents: [destinationFolderId || "root"], name},
}));
const res2 = BatchRequest.EDo({batchPath: "batch/drive/v3", requests: requests2});
console.log(res2);
}
Note:
In this sample script, it supposes that the 200 hyperlinks are put in the column "A" of "Sheet1". So, please modify this for your actual situation. Please be careful this.
References:
Batch request of official document
Files: get
Files: copy
BatchRequest of Google Apps Script library
Assuming the links look something similar to this and they're stored in the A column:
https://docs.google.com/presentation/d/SLIDE_ID/edit
You can easily extract the slideId from the hyperlink which corresponds to the fileId by making use of this formula (by dragging it down the whole A column):
=REGEXEXTRACT(A1,"[-\w]{25,}")
Finally, in order to copy each file, you can make use of Apps Script’s DriveApp, something similar to this:
DriveApp.getFileById(“fileId”).makeCopy(“destination”);
However, since the fileId corresponds to a range in the sheet, you can pass directly the range - so instead of using “fileId”, you could use this:
let sheet = SpreadsheetApp.openById(“spreadsheetId”).getSheetByName(“sheetName”);
let fileId = sheet.getRange(1,2).getValue();
The snippet above is retrieving the sheet where the links are stored and then by making use of the getRange and getValue methods it retrieves the value from the B1 cell (assuming that the ids of the files will be in the B column after REGEXEXTRACT).
Note
Please bear in mind that you can extract the fileId as well directly in your script, depending on the workaround and programming language .
Reference
Files Class Apps Script;
Spreadsheet Class Apps Script;
Range Class Apps Script;
REGEXEXTRACT function.

Resource URl Not Found with Yahoo Finance

I wish to grab the historical stock price from Yahoo Finance into Google Sheet and received this error. Please assist. If using import xml, how will it be?
https://au.finance.yahoo.com/quote/ASX.AX/history?p=ASX.AX
=IMPORTHTML(D7,"table",1)
I believe your goal as follows.
You want to retrieve the table from the URL of https://au.finance.yahoo.com/quote/ASX.AX/history?p=ASX.AX and put it to the Spreadsheet.
Issue and workaround:
Unfortunately, it seems that the table cannot be retrieved using IMPORTHTML and IMPORTXML from the URL. This has already been mentioned in Jason E.'s answer.
But, fortunately, when I tested to retrieve the table using UrlFetchApp of Google Apps Script, I confirmed that the table can be retrieved. So, in this answer, as a workaround, I would like to propose to achieve your goal using Google Apps Script. The sample script is as follows.
Sample script:
Please copy and paste the following sample script to the script editor of Spreadsheet. And, before you use this script, please enable Sheets API at Advanced Google services. And, run the function of myFunction and please authorize the scopes. By this flow, the table is retrieved from the URL and put it to the active sheet.
function myFunction() {
const url = "https://au.finance.yahoo.com/quote/ASX.AX/history?p=ASX.AX";
const res = UrlFetchApp.fetch(url, {muteHttpExceptions: true});
const tables = res.getContentText().match(/(<table[\w\s\S]+?<\/table>)/g);
if (!tables || tables.length == 0) throw new Error("No tables. Please confirm URL again.");
const spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
const sheet = spreadsheet.getActiveSheet();
const resource = {requests: [{pasteData: {html: true, data: tables[0], coordinate: {sheetId: sheet.getSheetId()}}}]};
Sheets.Spreadsheets.batchUpdate(resource, spreadsheet.getId());
}
Result:
When above script is run, the following result is obtained.
Note:
This sample script is for the URL of https://au.finance.yahoo.com/quote/ASX.AX/history?p=ASX.AX. So when you changed the URL, the script might not be able to be used. Please be careful this.
References:
Class UrlFetchApp
Method: spreadsheets.batchUpdate
Yahoo seems to have made some changes to their website resulting for the IMPORT functions of Google Sheet not to work. This affected some(not all) of their webpage as well as the tickers. Using IMPORTXML will still give you the same error.
I suggest using the built in GOOGLEFINANCE() function or find another website that is scrape-able by IMPORT functions and will give you the same data as you wanted.

Getting data from a public readable Google Sheet in a script

I'm trying to create a Google Script to access a range of cells from a Google Docs spreadsheet that I don't own but is shared. The spreadsheet is shared in a way that means it is publicly readable, but can't be written to. (Normal users can comment, but not edit)
When I try to run this script, I get an error: "You do not have permissions to access the requested document." I've authorized Google Scripts to my account, but that doesn't seem to be enough to do a getValues() call.
Here's my script so far:
function myFunction() {
var spreadsheet = SpreadsheetApp.openByUrl("https://docs.google.com/spreadsheets/d/1g24uo61ITTveZrUGx4ZY5nMoJp-n1NCxq5r4WMhAJU0/");
// I can access the name, this runs fine:
Logger.log(spreadsheet.getName());
SpreadsheetApp.setActiveSpreadsheet(spreadsheet);
SpreadsheetApp.setActiveSheet(spreadsheet.getSheets()[0]);
// fetch this sheet
sheet = spreadsheet.getActiveSheet();
var range = sheet.getRange(3,3,22,1);
// This next line fails:
var slicesAndCounts = range.getValues();
Logger.log(slicesAndCounts);
}
(The URL is the actual public spreadsheet, so this code should run as is)
Is there some way I can access the values in the spreadsheet from Google Scripts? I've considered trying to make a private copy of the spreadsheet, but I only need it for the duration of the script and I'm not sure if I can create a temporary copy easily. That solution also seems a bit heavy, considering I feel like I should be able to read the spreadsheet given I can access it via my browser or the Google Sheets app.
How about using Sheets API? At SpreadsheetApp, it seems that users cannot retrieve values from the shared spreadsheet. I think that the reason may be due to that the access token cannot use at SpreadsheetApp. In my environment, the same error also occurs. So I would like to propose to use Sheets API. When Sheets API is used, the access token is also used. By this, users can retrieve values from the shared spreadsheet.
In order to use this sample script, please enable Sheets API at Advanced Google Services and API console.
Enable Sheets API v4 at Advanced Google Services
On script editor
Resources -> Advanced Google Services
Turn on Google Sheets API v4
Enable Sheets API v4 at API console
On script editor
Resources -> Cloud Platform project
View API console
At Getting started, click Enable APIs and get credentials like keys.
At left side, click Library.
At Search for APIs & services, input "sheets". And click Google Sheets API.
Click Enable button.
If API has already been enabled, please don't turn off.
If now you are opening the script editor with the script for using Sheets API, you can enable Sheets API for the project by accessing this URL https://console.cloud.google.com/apis/library/sheets.googleapis.com/
About sample script :
The result of this sample is the same to that of your script. The values are retrieved from the range of 3,3,22,1 of 1st sheet in the shared spreadsheet.
In this sample script, it uses file ID of Spreadsheet. In your case, that is 1g24uo61ITTveZrUGx4ZY5nMoJp-n1NCxq5r4WMhAJU0 for https://docs.google.com/spreadsheets/d/1g24uo61ITTveZrUGx4ZY5nMoJp-n1NCxq5r4WMhAJU0/.
In this sample script, a1Notation is used for the range. In your case, getRange(3,3,22,1) is C3:C24.
For this script, when {ranges: "C3:C24"} is used, it means the 1st sheet of the spreadsheet. If you want to retrieve values from more than 2nd sheet, please use the sheet name like sheet2!C3:C24 and sheet3!C3:C24.
Script :
function myFunction() {
var spreadsheetId = "1g24uo61ITTveZrUGx4ZY5nMoJp-n1NCxq5r4WMhAJU0";
var slicesAndCounts = Sheets.Spreadsheets.Values.batchGet(spreadsheetId, {ranges: "C3:C24"}).valueRanges[0].values;
Logger.log(slicesAndCounts);
}
References :
Advanced Google Services : https://developers.google.com/apps-script/guides/services/advanced
Sheets API v4: https://developers.google.com/sheets/api/
If this was not useful for you, I'm sorry.

Google Script for formula value Return #N/A

Column F have a Formula value when I read in script is show #N/A.
Below is a very simple script code, but I'm unable to understand why it is showing #N/A, I didn't implement any trigger.
var spreadsheet =
SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Sheet1');
var values = spreadsheet.getDataRange().getValues();
Logger.log(values[1][0]);
Logger.log(values[1][5]);
https://docs.google.com/spreadsheets/d/18EsMMrC1IbzkZaIb5aIx3OfwvN1VzdmHm5B-YVuCMs4/edit?usp=sharing
How about a following modification?
From :
var values = spreadsheet.getDataRange().getValues();
To :
var values = spreadsheet.getDataRange().getDisplayValues();
The detail information of getDisplayValues() is here.
If I misunderstand your question, I'm sorry.
Edit
When I tried to retrieve the data using Sheet API v4, following error message occurred.
"#N/A (Historical GOOGLEFINANCE data is not available outside of the Google Sheets UI.)"
So I investigated about this error. And I found a following document.
We want to make you aware of a small change to the GOOGLEFINANCE function, which makes it easy to pull current or historical securities information directly into your spreadsheets in Google Sheets. Starting today, it will not be possible to download historical data or access it outside of Sheets using Apps Script or an API. If you have a spreadsheet with historical data generated from the GOOGLEFINANCE function and you try to download it or access it via Apps Script or an API, the corresponding cells will show “#N/A.” You will still be able to view that historical data from within the Sheets application, and you will still be able to download current data and access current data via Apps Script or an API. Please keep this in mind when using the GOOGLEFINANCE function going forward.
Document : https://gsuiteupdates.googleblog.com/2016/09/historical-googlefinance-data-no-longer.html
Updated at July 30, 2022:
At May 18, 2022, I noticed that the values from GOOGLEFINANCE on Spreadsheet can be retrieved using a Google Apps Script.
Testing:
Put the following sample formula into the Spreadsheet.
=GOOGLEFINANCE("NASDAQ:GOOG","price",DATE(2022,5,1),DATE(2022,5,10),"DAILY")
The image of this sample situation is as follows.
When the following sample scripts with Spreadsheet service (SpreadsheetApp) are used,
const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1");
const values = sheet.getDataRange().getDisplayValues(); // or getValues()
console.log(values);
And, when the following sample scripts with Sheets API are used,
const spreadsheetId = SpreadsheetApp.getActiveSpreadsheet().getId();
const values = Sheets.Spreadsheets.Values.get(spreadsheetId, "Sheet1").values;
console.log(values);
the following result is obtained.
[
["Date", "Close"],
["2022/05/02 16:00:00", "2343.14"],
["2022/05/03 16:00:00", "2362.59"],
["2022/05/04 16:00:00", "2451.5"],
["2022/05/05 16:00:00", "2334.93"],
["2022/05/06 16:00:00", "2313.2"],
["2022/05/09 16:00:00", "2261.68"]
]
Note:
Unfortunately, I cannot find the change in this specification in the official document. And, I'm not sure whether this is the current specification. So, this situation might be changed in the future update. Please be careful about this.
References:
Report: Obtaining Values from GOOGLEFINANCE using Google Apps Script
Checking Exchange Rate using GOOGLEFINANCE with Google Apps Script

programmatically edit script in Google Drive document

I have a Google Drive document which writes values to a Google Drive spreadsheet using Google Apps Scripts.
The script associated with the document looks a lot like this:
// must change value to actual spreadsheet ID
RobProject.spreadsheetID = "spreadsheetID";
function onOpen()
{
// do stuff;
}
Each time I create a spreadsheet and its related documents, I manually change the value spreadsheetID to the spreadsheet's ID so the documents know to which spreadsheet they should write their values.
I would like a programmatic way to fill in the correct value for spreadsheetID into the Documents' scripts.
When I search for "edit scripts programmatically," I get tutorials for creating Google Apps Scripts, not editing scripts with scripts. Is there any way to edit Google Apps Scripts with a Google Apps Script?
If I understand correctly, you are working with a document and a spreadsheet. The document needs to know the Id of the spreadsheet.
There are some ways to access the Google Apps Script code through the API, but that is only for standalone projects, not for container-bound scripts (unfortunately).
You could consider using a naming convention for the document and spreadsheet so that you could use the Drive service to get from the document to the spreadsheet (DriveApp.getFilesByName()). Or possibly organize them by folder (DriveApp.getFoldersByName(), folder.getFiles()).
If you wanted to store the spreadsheet Id in a project property, you could build a UI in the document that let the user open up the list of files in Drive and pick the associated spreadsheet and then store the Id (ScriptProperties.setProperty('SpreadsheetId')).
Don't forget that onOpen has a parameter. You could use following code:
// Define global variable somewhere
RobProject = {};
function onOpen(e) {
RobProject.sSheet = e.source; // maybe the spreadsheet object is as useful as the ID
RobProject.spreadsheetID = e.source.getId();
// do stuff;
}
Please, for your own sake, don't try to write selfmodifying code.