Fetch Telegram Post's views with Google Apps Script - google-apps-script

I'm trying to fetch this number:
Tested this Telegram post:
https://t.me/s/google_sheets/393
I want to do it with a script. Interesting, it was possible for me to achieve this with a formula:
=INDEX( REGEXEXTRACT(SUBSTITUTE(QUERY(FLATTEN(IMPORTDATA("https://t.me/s/google_sheets/393")),,2^99),CHAR(10),), "(.*tgme_widget_message_views"">)(.+)(<\/span><span class=""copyonly"".*)"),2)
This is the script I've tried:
function GET() {
var reg1 = /tgme_widget_message_views">(.+)<\/span><span class="copyonly"/i;
var response = UrlFetchApp.fetch('https://t.me/s/google_sheets/393').getContentText();
var match = response.match(new RegExp(reg1))[1];
Logger.log(match); // 13.4K, expected: 11.2K
}
It gives me the wrong result. I'm willing to do the work with a script because I want an afficient way of fetching multiple URLs at once. The formula will crash in this situation.
My script gives me the wrong result, it may be the number for other Telegram post (https://t.me/google_sheets/374 ), I'm not sure.
Please help me to resolve this issue.

Since the one you are looking for is that specific post and not the others, you may want to limit what you are searching for by appending the id of the post at the end of your regex. Thus we are sure we are looking just until post id 393. As you can see, I added .+google_sheets\/393 in your regex.
function GET() {
var reg1 = /tgme_widget_message_views">(.+)<\/span><span class="copyonly".+google_sheets\/393/i;
var response = UrlFetchApp.fetch('https://t.me/s/google_sheets/393').getContentText();
var match = response.match(new RegExp(reg1))[1];
Logger.log(match); //11.2K
}

Related

How to get the correct XPath for Google Sheets formula ImportXML

I tried to set up importXML in my Google Sheets, I used the method to copy full Xpath.
It seems not working at all.
After reading Xpath still not sure how to get the right path just for the token price.
Hope can get some idea or document to read to get the value I need.
Thanks a lot for reading this.
Wish you to have a nice day.
=IMPORTXML("https://info.osmosis.zone/token/DSM","/html/body/div[1]/div/div[3]/div/div/p")
Create a custom function by:
Opening Script Editor (Tools > Script Editor or Extensions > Apps Scripts)
And then enter the following within the script:
/**
* #return Specific value out of the value of different fields
* #customfunction
*/
function PARSEVALUE(Url,itemKey) {
var res = UrlFetchApp.fetch(Url);
var content = res.getContentText();
var jsonObject = JSON.parse(content);
return jsonObject[0][itemKey];
}
In your spreadsheet, use the function like:
=PARSEVALUE("https://api-osmosis.imperator.co/tokens/v1/DSM","price")
There are different values for different keys, as in price,symbol,name,liquidity,volume_24h e.t.c. you can grab using this function.
The page contents
You need to enable JavaScript to run this app.
JavaScript content is not supported by any IMPORT formulae. the best course of action would be to find a different source for your scrapping.
You need to use a specific url (api) which contains the json.
edit :
According to the url provided by #QHarr and if you want to retrieve all informations from url, try
function parseValues(url) {
const jsn = JSON.parse(UrlFetchApp.fetch(url).getContentText())
const headers = Object.keys(jsn[0]);
return ([headers, ...jsn.map(obj => headers.map(header => Array.isArray(obj[header]) ? obj[header].join(",") : obj[header]))]);
}
and in your sheet
=parseValues("https://api-osmosis.imperator.co/tokens/v1/DSM")

appendRow() adds blank row in google sheets (app script)

I've setup a google app script that would be triggered from an external system. This script would fetch the details from the third party system and add them to google sheet row.
function doPost(request) {
try{
var jsonString = request.postData.getDataAsString(); //get the request from KF as JSON String
setLog("\n postData*********************"+jsonString+"************************************* \n");
setLog("before the row append");
ss.appendRow([jsonString["Name"], jsonString["Age"], jsonString["Contact"]]);
setLog("After the row append");
var returnJson = '{"status": "success"}';
//used to send the return value to the calling function
setLog("/n returnJson****************************"+returnJson+"************************************* /n")
return ContentService.createTextOutput(returnJson).setMimeType(ContentService.MimeType.JSON);
}
There's absolutely no errors or warnings, but somehow it keeps adding the blank rows into the sheet.
Note: setLog() is a function where I print the values into google doc for debugging.
Maybe the reason your script is not working has to do with the value of jsonString.
I could not find any reference to request.postData.getDataAsString() inside GAS Documentation, so maybe you are trying to call a method on an object which does not support it, which would not raise an Error, but would return undefined.
One quick way to debug this would be to LOG the value (using your custom function or Logger.log(jsonString)) BEFORE you call .appendRow(). Then, you can verify if your variable has the value you expect it to have.
On the other hand, my suggestion is to use this method:
var jsonString = JSON.parse(request.postData.contents) //Gets the content of your request, then parses it
This method is present in the Documentation, and has been consistently working on all of my projects.
I think you should sort the coulmns with google app script. Write this code after ss.appendRow. The column will be sorted and all blank rows gets down.
// Sorts the sheet by the first column, ascending
ss.sort(1)
or if errors try this one also
var fl = SpreadsheetApp.getActiveSpreadsheet();
var sheet = fl.getSheets()[0];
fl.sort(1)

Parse JSON data from ASX into Google Sheets for Exchange Traded Products - not companies

I am trying to develop a Google Sheets-based portfolio tracking sheet that is able to retrieve daily prices for the securities in the Australian (ASX) and US markets.
For US market securities the GoogleFinance function works well enough. However for the ASX the ability for GoogleFinance to retrieve information is a bit hit and miss.
Ruben had asked a similar question to which Ian Finlay provided a solution that works in most instances, i.e. listed companies, but not for Exchange Traded Products that such as PMGOLD.
Ian Finlay's solution using apps script to parse json data was:
<code>
function AsxPrice(asx_stock) {
var url = "https://www.asx.com.au/asx/1/share/" + asx_stock +"/";
var response = UrlFetchApp.fetch(url);
var content = response.getContentText();
Logger.log(content);
var json = JSON.parse(content);
var last_price = json["last_price"];
return last_price;
}
For a 'normal' company such as NAB = asx_stock, the script works well, however for a exchange traded product such as PMGOLD, it does not.
With some basic searching an experimentation, the reason seems to be that the url that is in the script does not point to the information required.
For NAB = asx_stock, the url reponse is
{"code":"NAB","isin_code":"AU000000NAB4","desc_full":"Ordinary Fully Paid","last_price":23.77,"open_price":24.11,"day_high_price":24.21,"day_low_price":23.74,"change_price":-0.15,"change_in_percent":"-0.627%","volume":1469971,"bid_price":23.75,"offer_price":23.77,"previous_close_price":23.92,"previous_day_percentage_change":"-1.239%","year_high_price":27.49,"last_trade_date":"2021-01-29T00:00:00+1100","year_high_date":"2020-02-20T00:00:00+1100","year_low_price":13.195,"year_low_date":"2020-03-23T00:00:00+1100","year_open_price":34.51,"year_open_date":"2014-02-25T11:00:00+1100","year_change_price":-10.74,"year_change_in_percentage":"-31.121%","pe":29.12,"eps":0.8214,"average_daily_volume":6578117,"annual_dividend_yield":2.51,"market_cap":-1,"number_of_shares":3297132657,"deprecated_market_cap":78636614000,"deprecated_number_of_shares":3297132657,"suspended":false}
However, for PMGOLD = asx_stock, the url reponse is:
{"code":"PMGOLD","isin_code":"AU000PMGOLD8","desc_full":"Perth Mint Gold","suspended":false}
Conducting some relatively 'non-code qualified person' type research, looks like the actual url for an Exchange Listed Product should be:
https://www.asx.com.au/asx/1/share/PMGOLD/prices?interval=daily&count=1
The url reponse for this is:
{"data":[{"code":"PMGOLD","close_date":"2021-01-28T00:00:00+1100","close_price":24.12,"change_price":0.19,"volume":98132,"day_high_price":24.2,"day_low_price":23.9,"change_in_percent":"0.794%"}]}
When I substitute this url into Ian Finlay's code and rename the var as 'close_price' instead of 'last_price' there is nothing retrieved. The code used is:
function AskPrice(asx) {
var url = "https://www.asx.com.au/asx/1/share/"+ asx +"/prices?interval=daily&count=1";
var response = UrlFetchApp.fetch(url);
var content = response.getContentText();
Logger.log(content);
var json = JSON.parse(content);
var data = json["data"];
return data;
}
I suspect this is due to the structure of the url response being formatted differently for the two different url types. Maybe nested? - I am not sure.
Can someone please help point out what mistake(s) I am making?
Thank you
Yes, the structure is different. I've done this in Python so I know exactly your problem.
The 1/share API (first example) returns a simple dictionary of name:value pairs so you can easily reference the value.
The "prices" version gives you a list of daily values under the data element. Even though your example only returns one day, it is a list with one value. (Notice the [square] brackets around it?
So you need to go to the "data" element to get the list, then reference the first (only) item of the list and then reference close_price.
I don't know this language but it's probably something like:
var data = json["data"][0]["close_price"];
Let me know if this helps.

List all Gmail History details

function history(){
var newResponse = Gmail.Users.History.list('email ID / 'me'');
console.log(newResponse);
for(var i = 0; i< newResponse.history.length; i++){
var label = newResponse.history[i];
console.log(JSON.stringify(label));
}
}
GOAL I have been trying to list all my history using the above given function but it is not working. I have been trying to search for a way to make this work but I keep receiving the same error over and over: "GoogleJsonResponseException: API call to gmail.users.history.list failed with error: Missing/invalid parameter: startHistoryId (line 13, file "GMailAPI")"
I hope my question is understandable.
I believe your goal as follows.
You want to retrieve the value from the method "Users.history: list" in Gmail API using Google Apps Script.
How about this answer?
Modification points:
In your case, it is required to include startHistoryId in the request of Gmail.Users.History.list. At the official document, startHistoryId is "Required".
I think that the reason of your issue is due to this.
In order to retrieve startHistoryId, please use the method "Users: getProfile" in Gmail API.
When above points are reflected to your script, it becomes as follows.
Usage:
In order to retrieve the values from Gmail.Users.History.list, please run the following flow.
Please run the function getStartHistoryId().
By this, startHistoryId is saved to PropertiesService.
For example, as a sample situation, please send a sample email on Gmail using browser.
Please run the function history().
By this, you can retrieve the values from Gmail.Users.History.list.
When history() is run just after getStartHistoryId() was run, no values are returned from Gmail.Users.History.list. Please be careful this. The official document of startHistoryId is as follows.
startHistoryId : Required. Returns history records after the specified startHistoryId. The supplied startHistoryId should be obtained from the historyId of a message, thread, or previous list response. History IDs increase chronologically but are not contiguous with random gaps in between valid IDs. Supplying an invalid or out of date startHistoryId typically returns an HTTP 404 error code. A historyId is typically valid for at least a week, but in some rare circumstances may be valid for only a few hours. If you receive an HTTP 404 error response, your application should perform a full sync. If you receive no nextPageToken in the response, there are no updates to retrieve and you can store the returned historyId for a future request.
Modified script:
// Added: At first, please run this function and save the startHistoryId to the PropertiesService.
function getStartHistoryId() {
var startHistoryId = Gmail.Users.getProfile('me').historyId;
PropertiesService.getScriptProperties().setProperty("startHistoryId", startHistoryId);
}
// Modified: As the next step, please run this function.
function history() {
var startHistoryId = PropertiesService.getScriptProperties().getProperty("startHistoryId");
var newResponse = Gmail.Users.History.list('me', {startHistoryId: Number(startHistoryId)});
if ("history" in newResponse) {
for(var i = 0; i< newResponse.history.length; i++){
var label = newResponse.history[i];
console.log(JSON.stringify(label));
}
}
}
In my environment, I could confirm that {startHistoryId: startHistoryId} can be also used.
Note:
This modified script supposes that Gmail API has already been enabled at Advanced Google services. Ref
References:
Users.history: list
Users: getProfile

Cannot find method open()

Cannot find method open().
If it can't find it, why does it let me choose it when I type the dot?
So how am I supposed to open a file? openByID()? Really? Where do I get this magical ID? From getURL? Seriously? This is the only thing that comes up:
SpreadsheetApp.openById(id).getUrl()
but I fail to understand how I'm supposed to get the URL if I don't have an ID to hang it on... leading to a catch 22. Further, I presume in the event I do get the URL I still need to parse it to get the ID... and what magical method have they supplied for that particular incantation?
Clearly, my understanding is lacking. Any help?
function copyTemplate() {
var targetSpreadsheetName="The file";
var targetSpreadsheetID=SpreadsheetApp.open(targetSpreadsheetName).getId();
targetSpreadsheetID.insertSheet("CellData", 1, {template:temp});
// The code below will duplicate the sheet named "CellData" and insert it after
// the 1st sheet and call it "CellData"
var ss = SpreadsheetApp.getActiveSpreadsheet();
var temp = ss.getSheetByName("CellData");
ss.insertSheet("CellData", 1, {template:temp});
}
The question is multi-part.
The error message returned was "Cannot find method open()."
The first question is "Why can't it find method open()?".
The second question is "If in fact, it can't find it, why does it let me choose it when I type the dot?"
The third question is "Considering the fact that the open() method fails to work, how am I supposed to open a file?" The presumed answer I expect to receive from the community is "with openByID()".
That leads to the fourth question which is "Where do I get the ID?". The apparent answer seems to be "getURL()".
And that leads to the fifth question which is "How do I use getURL()?" getURL seems to require an ID. If getURL requires an ID to get the URL, and openByID requires a URL to get the ID, you have an infinite loop. Surely I'm misunderstanding something.
The sixth question is "In the event getURL() ends up being part of the solution, how does one distinguish the ID from the rest of the string returned?"
I hope that clarifies my question.
Well I'll finally try to answer your question, hoping you'll find it clear enough... ;-)
First thing first :
var ss = SpreadsheetApp.open()
does indeed appear in the autocomplete after the dot, what you didn't pay attention to is that the argument is a file, ie an object returned by an appropriate statement.
Now let's see how to get it and , with this example how the other parameters of this file can be obtained :
function myFunction() {
var files = DocsList.find('thisisthesheetIwant');// this is an array of file objects that include the term 'thisisthesheetIwant'
var file = files[0];// I take the first one
var filename = file.getName();//and get its name
var fileId = file.getId();// its ID
var fileurl = file.getUrl();//and its url
// then I show the results in the logger
Logger.log('number of file found = '+files.length+'\n\n'+filename+'\n\n'+fileId+'\n\n'+fileurl+'\n\n')
var ss = SpreadsheetApp.open(file);// using that file object I can open a spreadsheet
var content = ss.getSheets()[0].getDataRange().getValues().toString();// and get the whole content of the first sheet
Logger.log('content = '+content);// there it is
}
The spreadsheet with its code is available here so you can test it by yourself, I named it thisisthesheetIwant hoping you don't have any file with a similar name or content since it wouldn't work as I expected if more than 1 file was returned.
Look at the logger and I hope it will answer your question(s).
it should appear like this below :
And the sheet itself is like this :
EDIT : note that the ID and the URL have a common part, the url is what you can see in your Browser's adress bar, the ID is just a part of it.
Now you can open the same spreadsheet with
SpreadsheetApp.openById(fileId)