I'm exporting data from Google Analytics to put it in Google Sheets. I have created a sheet called "raw" where I will dump all the data. In order to avoid exporting and dumping the data, I need to check if the data is already there. I don't need to know where it is, I just need to know where it is.
I have written a function to get the data out of Google Analytics and it works as when I display the output in the logs, I get what I need.
result = getData(tableId,dimensions,filters,'2021-01-01','2021-01-01',metrics);
Logger.log(result['rows']);
Now I want to loop through those results and check if the data is already available in the sheet. If it's not yet available, I would add it to the bottom of the sheet
var raw = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("raw");
var raw_content = raw.getDataRange();
From here, I used the script available here: How to check if the value exist in google spreadsheet or not using apps script
for (var b=0;b<result['rows'].length;b++)
{
temp_date = result['rows'][b][0];
temp_value = result['rows'][b][1];
unique_id = label+'_'+temp_date;
Logger.log(unique_id);
var textFinder = raw_content.createTextFinder(unique_id);
var occurrences = textFinder.findAll().map(x => x.getA1Notation());
if (occurrences == [])
{
Logger.log('not found');
}
else
{
Logger.log('found');
}
}
Even if the sheet is empty, it always return "found". If I add manually one of the unique ids, it still returns "found" all the time.
I have also tried the second alternative in this post but it doesn't work.
I'd like to avoid using a loop to check cells one by one because at some point, there will be a lot of data in there.
Any idea?
thanks
From your code and what you are trying to accomplish, you would need to define textFinder for each iteration of the loop, with different values of label+'_'+temp_date.
Update: Two arrays in JavaScript cannot be compared with == as it would always return false. The best way in this case is to check its length if zero or not.
for (var b=0;b<result['rows'].length;b++)
{
temp_date = result['rows'][b][0];
temp_value = result['rows'][b][1];
unique_id = label+'_'+temp_date;
var textFinder = raw_content.createTextFinder(unique_id);
var occurrences = textFinder.findAll().map(x => x.getA1Notation());
if (occurrences.length)
{
Logger.log('found');
}
else
{
Logger.log('not found');
}
}
Sample Result:
Reference:
Class TextFinder
Related
I am using Google Apps Script and a custom function to call an external API to verify phone numbers.
Below is the code for my function.
/**
* This CUSTOM FUNCTION uses the numVerify API to validate
* a phone number based on the input from JotForm and a
* country code which is derived from the JotForm country
*
* Numverify website: https://numverify.com/dashboard (account via LastPass)
* Numverify docs: https://numverify.com/documentation
*/
function PHONE_CHECK(number, country){
if(country == "")
return [["", "country_not_set"]]
// check the API result has already been retrieved
var range = SpreadsheetApp.getActiveSheet().getActiveRange()
var apires = range.offset(0, 1).getValue()
if(apires.length > 0)
return range.offset(0, 0, 1, 2).getValues()
var url = 'http://apilayer.net/api/validate'
+ '?access_key=' + NUMVERIFY_KEY
+ '&number=' + encodeURIComponent(number)
+ '&country_code=' + encodeURIComponent(country)
+ '&format=1';
var response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});
var json = response.getContentText();
var data = JSON.parse(json);
if(data.valid !== undefined){
if(data.valid){
return [[data.international_format, "OK"]]
}else{
return [["", "invalid_number"]] // overflows data to the next column (API Error) while keeping the phone field clear for import into TL
}
}else if(data.success !== undefined){
if(data.error.type.length > 0){
return [[number, data.error.type]]
}else{
return [[number, "no_error_type"]]
}
}else{
return [[number, "unexpected_error"]] // this generally shouldn't happen...
}
}
Given this formula, which takes a phone number and country code, it will then check the phone number against the numverify API and return the result in the cell and overflow to the cell to the right of it. The overflow is used to indicate whether the API was called successfully and to check if the result was already retrieved.
Example:
=PHONE_CHECK("+32123456789", "BE")
Note that the first cell is empty because the API returns an 'invalid phone number' code. Because of privacy, I won't put any real phone numbers here. In case I would've used a real phone number, the first cell would contain the phone number formatted in the international number format.
Since I'm using the free plan, I don't want to rerun the function every time if I already know what the result is, as I don't want to run up against the rate limit. Unfortunately, this doesn't seem to work and periodically (it looks like once every day), it will refresh the results for each row in the sheet.
So two questions:
Is something wrong with my logic in checking the API result and then just exiting the function? (see below for the code)
If the logic is right, why does Google Sheets seem to periodically ignore (or refresh?) the values in that second column and call the external API anyhow?
var range = SpreadsheetApp.getActiveSheet().getActiveRange() // get the cell from which the function is called
var apires = range.offset(0, 1).getValue() // get the values directly to the right of the cell
if(apires.length > 0) // check if there's anything there...
return range.offset(0, 0, 1, 2).getValues() // return an array that basically just resets the same values, effectively stopping the script from running
Your Aim:
You want a custom function, AKA a formula to only run once, or as many times as is necessary to produce a certain result.
You want the same formula to write a value to the another cell, for example the adjacent cell, that will tell the formula in future, if it should be run again or not.
Short Answer:
I'm afraid that values that are evaluated from custom functions AKA formulas are transient, and what you want to accomplish is not possible with them.
Explanation:
You can run a quick test with this custom function:
function arrayTest() {
return [[1, 2, 3, 4 ,5]]
}
If you put this in a cell as below:
You will see that if you delete the formula in the original cell, the overflow values also dissapear.
Therefore something like the following code will almost always produce the same value:
function checkTest() {
var cell = SpreadsheetApp.getActiveRange()
var status = cell.offset(0, 1).getValue();
if (status != "") {
return "already executed" // in your case without calling API
} else {
return [["OK","executed"]] // in your case making API call - will happen ~90% of the time.
}
}
// OUTPUT [["OK","executed"]]
Here I am inserting a row and deleting it to force re-calculation of the formulas.
The first thing that Sheets does before re-calculating a formula is that it clears the previous values populated by formula. Since the conditional statment depends on the value of its previous execution, it will always evaluate to the same result. In your case, it will almost always make the API call.
Confusingly, this is not 100% reliable! You will find that sometimes, it will work as you intend. Though in my tests, this only happened around 1 times out of 10, and most often when the formulas updated when saving changes to the script editor.
Ideally, though not possible, you would want to be able to write something like this:
function checkTest() {
var cell = SpreadsheetApp.getActiveRange();
var cellValue = cell.getValue();
var adjacentCell = cell.offset(0, 1);
var status = adjacentCell.getValue();
if (status == "") {
cell.setValue(cellValue)
adjacentCell.setValue("executed")
}
}
Which would clear the formula once it has run, alas, setValue() is disabled for formulas! If you wanted to use setValue() you would need to run your script from a menu, trigger or the script editor. In which case it would no longer make sense as a formula.z
References
https://developers.google.com/apps-script/guides/sheets/functions
I am very new to Google Apps Script (as well as JavaScript, for that matter), but I have been trying to tinker with it for fun.
I have tried writing a script to fetch API price data in Google Sheets, but am finding that the returned value is not updating when re-evaluating the script in the same cell.
Below is a script to fetch bitcoin price data from Coinbase's API. The script parses the JSON response of the request, as is described here.
function getBTCPrice() {
var url = "https://api.coinbase.com/v2/prices/BTC-USD/spot";
var response = UrlFetchApp.fetch(url);
var jsonSpotPrice = response.getContentText();
var parseSpotPrice = JSON.parse(jsonSpotPrice);
var price = "$" + parseSpotPrice.data.amount;
return price
}
Now, if I type =getBTCPrice() in some cell, and then re-evaluate a few moments later, I get the same price; however, if I evaluate the script in a different cell, I get a different result.
I've read some stuff about Google caching values in cells, so that perhaps the script isn't evaluated because the value of the cell has not changed. Is this the case here? If so, is there a workaround?
Any help is greatly appreciated!
I finally figured it out! Instead of trying to call the custom function from an actual sheet cell (which apparently stores cached values), the trick is to call the function within a script.
Using my above script:
function getBTCPrice(url) {
var response = UrlFetchApp.fetch(url);
var jsonSpotPrice = response.getContentText();
var parseSpotPrice = JSON.parse(jsonSpotPrice);
var price = "$" + parseSpotPrice.data.amount;
return price;
}
You can then call this function from another script. Specifically, I was looking to assign the updated price to a cell. Below is an example, which assigns the price to the active spreadsheet, in cell A1:
function updatePrice(){
var a = getBTCPrice("https://api.coinbase.com/v2/prices/BTC-USD/spot");
SpreadsheetApp.getActiveSpreadsheet().getRange('A1').setValue(a);
}
You can then proceed to set an appropriate time trigger. And that's all there is to it!
Have a look at this answer on Refresh data retrieved by a custom function in google spreadsheet.
As the answerer says, the trick is to
My solution was to add another parameter to my script, which I don't even use. Now, when you call the function with a parameter that is different than previous calls, it will have to rerun the script because the result for these parameters will not be in the cache.
Vik
In addition of Vikramaditya Gaonkar answer, you can use a installable trigger to get a refresh result each minute.
function getBTCPrice(input) {
url = "https://api.coinbase.com/v2/prices/BTC-USD/spot";
response = UrlFetchApp.fetch(url);
var jsonSpotPrice = response.getContentText();
var parseSpotPrice = JSON.parse(jsonSpotPrice);
var price = "$" + parseSpotPrice.data.amount;
return price
}
function up(){
SpreadsheetApp.getActiveSheet().getRange('A1').setValue(Math.random());
}
The parameter of getBTCPrice function is, in my case, cell A1 which is randomize each minute. For this, I create a installable trigger on up function
function up, time-driven, minute timer, every minute
I was also trying to make my custom function update, after searching I came up with the following function:
function updateFormulas() {
range = SpreadsheetApp.getActiveSpreadsheet().getDataRange();
formulas = range.getFormulas();
range.clear();
SpreadsheetApp.flush();
range.setValues(formulas);
}
The function above update all formulas of the spreadsheet. In my experience to make a custom function update I had to change its value, so I get all the data of the sheet, then I get the formulas and store them into a variable, then I clear their values and apply this change with "flush", finally I update the values I have just cleared with the formulas I have stored.
I created this function and in my case I have set the trigger for 1 minute to execute it, every minute all functions of the table are updated.
I hope this helps you.
So, I'm trying to write a script using the onEdit() event, which will basically remove links that are duplicates (technically, it removes everything, and only puts back things which aren't duplicates).
My code works fine all the way until it's time to write back non-duplicates. Namely, the line in which I use range.setValues(). I understand that it needs an array of arrays of cells which to edit, and that said array needs to fit in the range.
So far, I have :
if (unique)
{
newData.push(editedRow[0]);
Browser.msgBox(newData);
}
Unique is a variable I use that is false if an exact entry was found. With the msgBox command, I can verify that newData contains what it needs to contain. Further down, I have :
newDataFinal = [newData];
Browser.msgBox('Put values '+newDataFinal+' in range ' +range.getA1Notation());
range.setValues(newDataFinal);
To my knowledge, this should make NewDataFinal an array of arrays, which I can verify if I change setValues() to setValue(), which writes [[22.0, 13.0, 23.0]] (for my example) in the spreadsheet, which looks like an array of arrays to me.
The range should also match, since for this example, I get a prompt along the lines of "Put values 22,13,23 in range B2:B4" from the msgBox, which seems as a fitting range.
So, what am I doing wrong?
Here's the rest of the code (please excuse the abundancy of comments/msgboxes and lack of elegancy, the priority is to get it to work, I can probably optimize it and clean it up a bunch afterwards) :
function onEdit(e)
{
var range = e.range;
var values = range.getValues();
var sheet = SpreadsheetApp.getActiveSheet();
if (sheet.getName() != 'testiranje') return;
newData = new Array();
// Browser.msgBox(range.getA1Notation());
range.clear();
var data = sheet.getDataRange().getValues();
var counter = 0;
for (editedRowIndex in values)
{
unique = true;
editedRow = values[editedRowIndex];
// Browser.msgBox('Edited Row ' +editedRow);
for(i in data)
{
var row = data[i];
// Browser.msgBox('Old Row '+row);
for (j in row)
{
// Browser.msgBox(row[j] + ' vs ' + editedRow[0])
if (editedRow[0] == row[j])
{
Browser.msgBox('Hit! '+editedRow[0]);
unique = false;
}
}
}
if (unique)
{
// Browser.msgBox('Pushing '+editedRow[0]+' in newdata');
newData.push(editedRow[0]);
Browser.msgBox(newData);
}
}
newDataFinal = [newData];
Browser.msgBox('Put values '+newDataFinal+' in range ' +range.getA1Notation());
range.setValues(newDataFinal);
// range.setNote('SCIENCE');
}
I didn't test your code because I didn't feel like creating a sheet for it but what I can suggest (that should solve this issue in any case) is to replace your range.setValues(newDataFinal); with this :
sheet.getRange(range.getRowIndex(),range.getColumnIndex(),newDataFinal.length,newDataFinal[0].length).setValues(newDataFinal);
And if you want to know why the range and array didn't fit you can use this code :
(I used Browser because you seem to like it... I prefer Logger.log)
Browser.msgBox('data height = '+newDataFinal.length+', data width = '+newDataFinal[0].length+' and range height is '+range.getHeight()+', range width is '+range.getWidth()+' ... does it fit ?');
Note : I'm almost sure that your initial range is bigger than the newData array since you remove elements from the initial data... My best guess would be that heights don't fit. (but that's only a guess ;-) since you didn't mention the error message you get...)
the problem is that you cant change cells from an onEdit handler. see the docs. instead install your own onEditCustom handler.
I'm writing some scripts for a client with the end goal of complete autonomy -- when complete, the spreadsheet will always work forever. Ideally, anyways.
Because I need information from other sheets, I have to access them in a way other than .getActiveSheet(). Because the client might re-name or re-order the sheets, I have to access the sheet in a way that works even after those changes. This rules out getSheetByName() and getSheets()[SHEET_NUMBER] (again, the client might re-name or re-order the sheets). However, it should be possible because of the "gid." Each sheet has a different gid and they do not change when you re-order or re-name the sheets (scroll to the end of the URL for each sheet to see what I mean).
All of the URL accesses only open the FIRST sheet. For instance,
SpreadsheetApp.openById(SHEET_ID).getDataRange().getValues()
returns the values of the first sheet, even if I include the "gid" part at the end. Same with openById and openFile.
So my question is, how do I access a sheet in a way that will work even after renaming the sheet or reordering the sheets within the spreadsheet?
There's no getSheetById method, but you can build your own using getSheetId(). Here:
function sheetsIdMap() {
var sheetsById = {};
SpreadsheetApp.getActive().getSheets().forEach(function(s){ sheetsById[s.getSheetId()] = s; });
//just checking that it worked
for( var id in sheetsById )
Logger.log(id+' - '+sheetsById[id].getName());
//usage example
var sId2 = sheetsById[2];
Logger.log('\n'+sId2.getName());
}
-- edit
Let's try a more straightforward function (although I don't like to do such a loop and don't store the data on a map for subsequent use o(1)).
function getSheetById(ssID, sheetID) {
var sheets = SpreadsheetApp.openById(ssID).getSheets();
for( var i in sheets )
if( sheets[i].getSheetId() == sheetID )
return sheets[i];
return null; //sheet id not found in spreadsheet, probably deleted?
}
Yes there is a sheet id. Its sheet.getSheetId. this id can be used from apps script and can also be transformed into a "real" gid for making a sheet url. Do (sheetId ^ 31578).toString(36) to get the gid.
I had to reverse-eng it to get it and I cant guarantee it will work forever.
I am running a project in Google Docs and I need help with a script. I need one that will perform the following tasks in a two column spreadsheet.
Insert a new blank row between every existing row of data.
Shift the data in the second column down one place so that they occupy the newly inserted blank row
Add curly brackets {} to the text in the data of the second column ie anytext --> {anytext}
Here is an example of how it is supposed to work:Sample GDocs spreadsheet
Your question was not far from the previous one that I just tried to answer so I though I could continue and suggest a working solution to this (easy) problem...
so there it is, not the most elegant but easy and working.
function insertBlank() {
var sh = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet()
var last = lastRowinCol(sh,'A');
var colA = sh.getRange('A1:A'+last).getValues();
var colB = sh.getRange('B1:B'+last).getValues();
var nA = [];
var nB = [];
for(n=0;n<last;++n){
nA.push([colA[n]])
nA.push(['']);
nB.push([''])
nB.push(['{'+colB[n][0]+'}']);
}
Logger.log(nA.length+' = '+nA)
Logger.log(nB.length+' = '+nB)
sh.getRange(1,1,nA.length,1).setValues(nA);
sh.getRange(1,2,nB.length,1).setValues(nB);
}
function lastRowinCol(sh,col){
var coldata = sh.getRange(col+'1:'+col).getValues();
for(var c in coldata){if(coldata[c][0]==''){break}}
return c
}
That said, you should preferably ask more appropriate questions and show that you have tried before coming on sto make your shopping for free ;-)
EDIT : following a very pertinent comment from Mogsdad on this other post I suggest you replace the for-next loop in function lastRowinCol with his code that iterates backwards so that it handles empty cells in the column. Moreover his code has an interesting construction since the loop limits and the condition are in the same statement.
function lastRowinCol(sh,col){
var coldata = sh.getRange(col+'1:'+col).getValues();
for (var c=coldata.length; (c) && coldata[c-1][0]==''; c--) {}
return c
}