Pulling current price change % (Google Finance) with Google Apps Script - google-apps-script

Question

I'm trying to pull current price change % from Google Finance with Google Apps Script. With the following coding, I couldn't figure out why it doesn't pull current price change % (0.72%), though it retrieves "After Hours" price change % (0.081%). Can anyone help me out? Thank you!!
function test() {
var url = 'https://www.google.com/finance/quote/AAPL:NASDAQ';
var res = UrlFetchApp.fetch(url, { muteHttpExceptions: true }).getContentText();
var location = '', sub = '', index = [], price = '', change = [];
// From google finance, scrape whole plain string from first <div class="YMlKec fxKbKc"> tag where current price is.
location = '<div class="YMlKec fxKbKc">';
index = res.indexOf(location);
sub = res.substring(index + location.length, index + location.length + 1000);
// Logger.log(sub);
// Pull current price from the sub variable. This is succesful.
price = sub.substring(0, sub.indexOf('<'));
Logger.log(price);
// Pull current price change, which should be 0.72%. But it didn't work.
// This retrieved only "After Hours" price change %, 0.081%, but not 0.72%, current price change % that I'm looking for.
location = '%';
index = [sub.indexOf(location)];
for (var i = 0; i < index.length; ++i) {
change = sub.substring(index[i] - 5, index[i] + 1);
Logger.log(change)
}
}

Answer 1

Is it possible to use Google Apps Script to get an info from the code I see at DevTools?
First thing that you should be aware is that the code shown in the Elements tab of Chrome Developers Tools is the current DOM but UrlFetchApp.fetch is only able to get the code from the source file referred directly by the URL.
To review the source code you might right click on the webpage and select View page source or go open the file from the Chrome Developers Tool's Sources tab. If the value that you want to read is not there, the best is to use a stock market API, otherwise you have to implement a headless browser.
There have being a several questions about getting data from Google Finance. Most of the question that involves Google Apps Script are related to using the GOOGLEFINANCE built-in function in Google Sheets, where most of the OP didn't reviewed the official help article about GOOGLEFINANCE. From this doc:
Historical data cannot be downloaded or accessed via the Sheets API or Apps Script. If you attempt to do so, you'll see a #N/A error in place of the values in the corresponding cells of your spreadsheet.
The relevance of the above is that Google is taking some measures to prevent that certain data be retrieved automatically.
In order to make Google Apps Script able to "see" what is shown in the web browser using only server-side code you should have to implement a headless browser:
A headless browser is a web browser without a graphical user interface.
Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but they are executed via a command-line interface or using network communication. They are particularly useful for testing web pages as they are able to render and understand HTML the same way a browser would, including styling elements such as page layout, colour, font selection and execution of JavaScript and Ajax which are usually not available when using other testing methods.
Another alternative might be to u use client-side code as the HTML Service could be used to create a dialog/sidebar/web-application to serve it including google.script API support. The later could be used to send to the server-side code some data from the client-side with certain limitations, i.e. Date, Function and DOM objects can't be sent to server side but you might pass them as JSON.
Related
How can I get stock quotes using Google Finance API?
Using Apps Script to scrape javascript rendered web page
Can we use Chrome V8 functionalities from GAS?

Answer 2

You can get current price change using axios and cheerio, the code below shows you how to do this. And check the code in the online IDE
const cheerio = require("cheerio");
const axios = require("axios");
const currencyName = "AAPL:NASDAQ"; // currency name from browser URL
const BASE_URL = "https://www.google.com/finance/quote/";
const AXIOS_OPTIONS = {
headers: {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36",
}, // adding the User-Agent header as one way to prevent the request from being blocked
};
function getCurrentPrice() {
return axios(BASE_URL + currencyName, AXIOS_OPTIONS).then(({ data }) => {
const $ = cheerio.load(data);
const pattern = /data:\[{3}(?<data>\[.{100,500}\])\]{3}, side/gm //https://regex101.com/r/VLKbBt/1
const currencyData = JSON.parse([...data.matchAll(pattern)].map(({groups}) => ({data: groups.data})).find(el=> el.data.includes(currencyName)).data);
const priceChange = currencyData[5][2]
return priceChange;
});
}
getCurrentPrice().then(console.log);
Output:
-0.53461844
And as you can see in the screenshot below, the data that we received is exactly as on the page.
If for some reason you want to make a Google Finance page parser in Python, have a look at the dedicated Google Finance blog post at SerpApi.
Disclaimer, I work for SerpApi