refresh cell instead of pressing ctrl-Shift-E manually - google-apps-script

I have a few queries on the sheet where I extract info to a database.
The problem I have is that the code implemented was fine till some sheets won't display the query showing the: "warning: one or more of these results' entries may not be displayed. Select ctrl+Shift+E to show them.
I tried clearing the values and pasting them back to no avail.
what else can I do?

IMHO I don't think that there is a canonical answer to this question - there has always been a loud clamour from users around certain topics which merely dissipate into an eerie silence of the Google docs engineering team (this issue seems to be shrouded in the same eternal mystery as IMPORTRANGE frequently failing or SPLIT not creating a perfect matrix).
I have been bitten by this myself several times so have some experience. I have used a heath-robinson workaround, which has worked for me. But YMMV.
The OP's question discusses the situation where there is a formula which was working well, but after some time of operation, a Ctrl+Shift+E situation has started to occur.
(There are other situations where Ctrl+Shift+E might occur due to results of one formula attempting to overwrite onto cells written by another formula. In this scenario, sometimes adding EXPAND at the beginning of the desired formula does the trick. Also using Filter functions, rather than 'IF' comparisons, does mop-up previous =CONTINUE(1,2,3) cells ... but perhaps neither of these are likely to be of help to the OPs question).
In my experience the spreadsheet might develop such a 'confused' behaviour over time, so requiring Ctrl+Shift+E, meaning a period of time where there are several uses-of-the-spreadsheet resulting in changes to the data. I might further speculate that there is more likelihood of this confused behaviour when the referred-to data is more dynamic e.g. rows added, rows removed OR that there are other formulas which also calculate from the same data set (let alone that these two formulas relate to each other).
I think that the idiom 'confused-behaviour' is appropriate since the actual engineering of the spreadsheet is an open box only to google engineers; we cannot rationalise its workings. So it seems mysterious and magical how, in the OP's case and my experience, the spreadsheet was working well, but then altered behaviour to require Ctrl+Shift+E without there being any change in the formulas, only in the sheets' accumulated data changes through usage.
I am giving an example of my workaround using an example case of mine. Note that in my case the data and also the problematic formulas are in one single sheet which I am calling the production sheet
The workaround involves using a script to duplicate a 'template-sheet', in the same spreadsheet as the production sheet. The template-sheet is structurally identical to the production sheet with the same column headings, but with only a few lines of sample data. It has the same formulas as the production sheet, which refer to the sample data in the self-same template sheet (not to the production sheet). Importantly, the template sheet is not showing 'confused-behaviour' - no Ctrl+Shift+E is required in the formula cells should there be any changes to the template's sample data. It also contains the formating of the production sheet.
So, when the script runs, it creates a duplicate of the template-sheet; it subsequently copies over the production sheet's data to this template duplicate and reapplies the formats. This duplicate becomes the new production sheet. The old production sheet is hidden (in my case, but could be deleted).
I have this system operational on a Tasks List spreadsheet being used by 12 people daily, where Tasks come in by google form throughout the day (and are themselves copied over to the production sheet by another script, not shown as not relevant to this OP). Once a task is completed, the task's row is removed from the production sheet. So the data grows and shrinks constantly throughout the day. The duplicate sheet is created each night, by timed trigger upon the insertSheet() function. Here is the script I use:
function insertSheet(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet =ss.getSheets()[0]; //first sheet is the production sheet
var sheetRange = sheet.getRange('A3:P');
var sheetValues= sheetRange.getValues();
var d = Utilities.formatDate(new Date(), Session.getTimeZone(), 'ddMMMyyyy-hh:mm:ss');
var ex = sheet.setName('CCEs' + d);
var templateSheet = ss.getSheetByName('templateSheet');
var s2 = ss.insertSheet(0,{template: templateSheet});
var height = sheetRange.getHeight();
s2.insertRowsAfter(2, height)
s2.setName('CCEs');
templateSheet.hideSheet();
ex.hideSheet();
s2.getRange('A3:P' + (sheetValues.length+2)).setValues(sheetValues);
format(); //
templateSheet.hideSheet();
ex.hideSheet();
}
function format(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var formatRange = ss.getSheetByName('templateSheet').getRange('B2:N2').copyTo(ss.getSheetByName('CCEs').getRange(2,2, ss.getSheetByName('CCEs').getMaxRows() -1,14), {formatOnly:true});
}
Prior to deploying this strategy, the spreadsheet would experience 'confused-behaviour' after a couple of days or so.
The sheet contained three formulas which would become confused. They are not important to the OP, but I duplicate here just to give the reader a feeling for the success of this strategy:
=arrayformula(IFERROR(FILTER(if(row(O:O) =1,"Auto Time Stamp ",iferror(1/0)) &O:O&if(row(O:O) =1,"copy",),len(A:A)),"Error"))
=Arrayformula(iferror(if(filter(L:L, len(A:A)) - filter(A:A, len(A:A))>0, if( int(filter(L:L, len(A:A)) - filter(A:A, len(A:A))) = 0 , text( text(filter(L:L, len(A:A)), "HH:mm") -text(filter(A:A, len(A:A)), "HH:mm") , "H:mm") , int(filter(L:L, len(A:A)) - filter(A:A, len(A:A))) & "Day(s), " & text( text(filter(L:L, len(A:A)), "HH:mm") -text(filter(A:A, len(A:A)), "HH:mm") , "H:mm") ) ,iferror(1/0)),"Time Taken"))
=arrayformula( IFERROR(if((ISBLANK(FILTER(J1:J,LEN(A1:A))) * (FILTER(G1:G,LEN(A1:A)) = "Normal")* (now()-FILTER(A1:A,LEN(A1:A))> OverdueTimings!A1 )),"OVERDUE Normal", if((ISBLANK(FILTER(J1:J,LEN(A1:A))) * (FILTER(G1:G,LEN(A1:A)) = "Urgent")* (now()-FILTER(A1:A,LEN(A1:A))> OverdueTimings!A2 )),"OVERDUE Urgent", if((ISBLANK(FILTER(J1:J,LEN(A1:A))) * (FILTER(G1:G,LEN(A1:A)) = "Very Urgent")* (now()-FILTER(A1:A,LEN(A1:A))> OverdueTimings!A3 )),"OVERDUE V. Urgent", IFERROR(1/0)))),countif(if((ISBLANK(FILTER(J1:J,LEN(A1:A))) * (FILTER(G1:G,LEN(A1:A)) = "Normal")* (now()-FILTER(A1:A,LEN(A1:A))> OverdueTimings!A1 )),"OVERDUE Normal", if((ISBLANK(FILTER(J1:J,LEN(A1:A))) * (FILTER(G1:G,LEN(A1:A)) = "Urgent")* (now()-FILTER(A1:A,LEN(A1:A))> OverdueTimings!A2 )),"OVERDUE Urgent", if((ISBLANK(FILTER(J1:J,LEN(A1:A))) * (FILTER(G1:G,LEN(A1:A)) = "Very Urgent")* (now()-FILTER(A1:A,LEN(A1:A))> OverdueTimings!A3 )),"OVERDUE V. Urgent", iferror(1/0)))),"OVERDUE *")& " OVERDUE"))
After several days, we delete all the accumulated old production sheets.

Related

Apps Script - How could this code be streamlined?

I've recently started working with Apps Script to improve the scope of what my google sheets can do, and I wanted to ask more experienced people how I might make my script more efficient. I used a mixture of tutorials, documentation, and trial & error to make it. I find that although it usually completes the task it's meant for, sometimes it takes an unreasonably long time or exceeds its runtime and simply stops.
I would like to know which best practices I could implement to make it run more quickly overall, and which things I might be able to include in future scripts to avoid any pitfalls I'd landed in here.
Scope:
The script is meant to take each day's new data and apply it to a new sheet called 'TODAY.' It works as follows.
Rename the tab labeled 'TODAY' to the previous workday's date (if today is 2.3, it renames the sheet to 2.2.)
Hide this renamed tab.
Duplicate the 'TEMPLATE' tab, and rename it to 'TODAY.'
Pull data from the 'RAW DATA' tab, and paste it into the new 'TODAY' tab.
Paste a formula into the new 'TODAY' tab and drag it down to the bottom of the table so that the correct values populate and the conditional formatting occurs.
Any help would be greatly appreciated, I really just need some direction for how to improve my work.
Here is a link to an example sheet with editing permissions enabled: https://docs.google.com/spreadsheets/d/1F7bAd2DjKgk53e-haPgjWfFphMfu5YBn8iRQ3qwC3n0/edit?usp=sharing
In my humble opinion, a good Google Sheet App Script doesn't need to use activate to control the source or destination of data. The sheet and script developer should know what and where they want the data to come from and go. Activate is like using the mouse to click on something.
I've taken your script and rewritten to minimize the use of variables. I have only one sheet variable and reuse it throughout. In fact for the majority of the time it is the copy of the TEMPLATE called TODAY.
Also unless I have to use a sheet last row many times, I avoid using a variable and instead just use sheet.getLastRow(). Same for columns.
I always wrap my code in a try catch block as a matter of habit.
As a last note, unless you change the notation in column C and N you could have used your script to fill in column B.
function myDailyUpdate() {
try {
let spread = SpreadsheetApp.getActiveSpreadsheet();
// Step 1
let sheet = spread.getSheetByName("TODAY");
let oldDate = sheet.getRange("Q4").getValue();
let prevDate = Utilities.formatDate(oldDate,"GMT-5","M.d");
// Renames old 'TODAY' sheet to previous workday's date.
sheet.setName(prevDate);
// Sets the color to red.
sheet.setTabColor("990000");
// Hides the old 'TODAY' sheet
sheet.hideSheet();
// Step 2
sheet = spread.getSheetByName("TEMPLATE");
// Copies the contents of the 'TEMPLATE' sheet to a new sheet called 'TODAY.'
sheet = sheet.copyTo(spread);
sheet.setName("TODAY");
sheet.activate(); // required to move to 1st position
// Move TEMPLATE to first position
spread.moveActiveSheet(1);
// Step 3
// Colors the 'TODAY' tab green to signify it being active.
sheet.setTabColor("6aa85f")
// Identifies the 'RAWDATA' sheet for later use.
let source = spread.getSheetByName("RAWDATA");
// Identifies ranges and values for later use.
let values = source.getDataRange().getValues();
// sheet is still the "TODAY" sheet
// Identifies 'TODAY' sheet as the recipient of 'RAWDATA' data, and identifies the range.
// Sets the values from 'RAWDATA' into 'TODAY.'
sheet.getRange(12,2,values.length,values[0].length).setValues(values);
// Step 4
// sheet is still the "TODAY" sheet
let range = sheet.getRange("C12");
range.setFormula(
'=IFERROR(IFERROR(IFS(VLOOKUP($B12,INDIRECT'
+'('
+'"'
+'\''
+'"'
+'&$Q$4&'
+'"'
+'\''
+'!"&"!"&"A1:O2000")'
+',15,false)="D","D",$N12="Quote","Q",$N12="Important","I",$N12="On Hold","H",$N12="IN TRANSIT","T",$N12="REQUEST","R",$N12="INCOMPLETE","N",$N12="COMMENT","C"),VLOOKUP($N12,$B$3:$C$9,2,FALSE)),"")');
// Pastes the above formula into cell C12.
let fillRange = sheet.getRange(12,3,values.length,1);
range.copyTo(fillRange);
sheet.activate();
}
catch(err) {
console.log(err);
}
}

Google Script pulling cached data

I have a sheet that is pulling data from an API every minute. Each row contains information on an individual NFT that shows what has been done to the NFT for a given day. The data resets once per day at a different time for each row/NFT. The script I have archives each row just before it resets to log what happened with it during that 24-hour period.
The issue is, the script does not always pull the live time updates in my helper cells. I have been logging debugs for the last day and I can see that sometimes it pulls live data, sometimes it pulls the same data for up to 4 minutes. In other words, the countdown clock for a row may show 10 minutes on the first script run, then in the sheet it changes to 9 minutes, but the script will pull 10 minutes again. This may continue again at 8 and 7 minutes remaining, where the script still says 10 minutes. Finally, at 6 minutes, the script will get live data again and show 6 minutes. My script begins with a Spreadsheet flush as well.
I have only found this similar issue reported a year ago, but I did not see it resolved anywhere:
Script pulling old data
I agree with that poster's assumption, it seems Google Script is pulling from a cached version of the sheet. The randomness of this bug is causing major problems with my archive, and so I'm trying to figure out if it is possible to force a fresh read of the sheet, or else if there is known information about this delay so that I can account for this with proper error handling (for example, currently the helper cells continuously flag a row to be archived for 4 minutes, and remove the flag once it's been archived, but I'm not sure if that needs to be even longer)
For reference, this is the code that pulls cached data sometimes, in case there is something I can add to it to force a fresh read (perhaps adding and removing data from a cell just to make it access the sheet?):
function saveHabs() {
var dateStamp = Utilities.formatDate(new Date(), "GMT", "yyyy-MM-dd");
var ss = SpreadsheetApp.getActiveSpreadsheet();
var habSheet = ss.getSheetByName("Harvests");
SpreadsheetApp.flush;
var beVals = habSheet.getRange("be2:be").getDisplayValues();
var habs = beVals.filter(String).length;
var dataSheet = ss.getSheetByName("New Historical");
var aVals = dataSheet.getRange("a:a").getDisplayValues();
var firstRow = aVals.filter(String).length+1;
Logger.log(habs)
var archiveDetails;
if (habs>0){
archiveDetails = habSheet.getRange("be2:bi"+(habs+1)).getDisplayValues();
dataSheet.getRange(firstRow,1,habs,5).setValues(archiveDetails);
dataSheet.getRange(firstRow,6,habs,1).setValue(dateStamp);
dataSheet.getRange(firstRow,6,habs,1).setNumberFormat("#");
}
Logger.log(archiveDetails);
//debugging logs
var totalCount = habSheet.getRange("BM:BM").getDisplayValues().filter(String).length;
Logger.log(totalCount);
var logFlags = habSheet.getRange("BM2:BP"+totalCount).getDisplayValues();
console.log(habs+" habitats to archive, below are the flags and time remaining (ignore second column, that's only for unharvested habs)");
console.log(logFlags);
}
This is a simplified version of the sheet for a MRE: https://docs.google.com/spreadsheets/d/1od2G4i2YOJleQ6Ph0ICH-XC7L-SsINF8uBEivOsicTY/edit?usp=sharing
It is currently running every minute with logs that should show the same error - that is, the countdown in column BM will not always go down on each new script run, but rather the cached time remaining will be pulled by the script.
I'm not positive if this will fix the issue in all cases, but I came across this problem and solved it as you mentioned. I added a simple value (in my case dummyCell.setValue("Running")) to a blank cell on the source sheet. At the end of the script, the script deletes it. That has, so far, caused the script to always read the live sheet, nothing cached.
This seems to only happen when the source tab is only read from, never written to, so it would appear forcing the write action updates the sheet to the live version.
Hopefully this will solve this for you as well!

Google sheets script - copy varying data from one sheet to another in same spreadsheet

I wonder if you could please help me solve this. My knowledge of scripts is quite poor, despite attempts to learn.
This is an example of my spreadsheet.
Each month, in the Monthly sheet, I copy data from all the rows below the frozen ones (columns B, C and D only) and make a note in my head of the number of rows copied (the figure shown in cell E2, which can vary). I then go to the Daily sheet, insert the required number of rows at the top of the unfrozen section, and paste in the data values (CTRL Shift V).
I've been trying unsuccessfully to automate this with a script. I've tried piecing bits of code together from other solutions, but the different methods used and my lack of knowledge mean I can't get them to work.
Hoping you can help.
Many thanks.
You can do the following:
Retrieve the desired range from Monthly, using Sheet.getRange.
Taking into account the source range number of rows via Range.getNumRows(), insert the corresponding number of blank rows to Daily (starting at row 4, first unfrozen one) using Sheet.insertRowsBefore.
Copy the retrieved range to Daily!B4 (first unfrozen row) using Range.copyTo.
Code snippet:
function copyRange() {
const ss = SpreadsheetApp.getActive();
const monthlySheet = ss.getSheetByName("Monthly");
const dailySheet = ss.getSheetByName("Daily");
const firstRow = 5;
const sourceRange = monthlySheet.getRange(firstRow,2,monthlySheet.getLastRow()-firstRow+1,3);
dailySheet.insertRowsBefore(4, sourceRange.getNumRows());
const destRange = dailySheet.getRange("B4");
sourceRange.copyTo(destRange,SpreadsheetApp.CopyPasteType.PASTE_VALUES,false);
}
Having tried and failed to do this quite a few times, I've now managed to produce something that seems to work by recording a macro, working out what each line or section in the resulting script does, and editing it, with some additional online research required. It might be a bit clunky, but here's the result of my efforts:
function CommitMonthly() {
// Script is only ever run from the Monthly sheet via the Commit button. Get the number of rows.
var spreadsheet = SpreadsheetApp.getActive();
var NumberOfRows = spreadsheet.getRange('E2').getValue();
// Select the Daily sheet, select top row, insert required number of rows above it.
spreadsheet.setActiveSheet(spreadsheet.getSheetByName('Daily'), true);
spreadsheet.getRange('B4').activate();
spreadsheet.getActiveSheet().insertRowsBefore(spreadsheet.getActiveRange().getRow(), NumberOfRows);
// Copy and paste the data from Monthly to new rows in Daily.
spreadsheet.getRange('Monthly!B5:D').copyTo(spreadsheet.getActiveRange(), SpreadsheetApp.CopyPasteType.PASTE_VALUES, false);
};
Many thanks for your help.

How to link various Google spreadsheets using IMPORTRANGEs that contain VLOOKUP formulas without getting #N/A returned?

The problem:
When using IMPORTRANGE to pull data from a different spreadsheet, cells in the origin spreadsheet that contain a formule containing VLOOKUP often (but not always) return #N/A (ERROR: Did not find value 'xxx' in VLOOKUP evaluation). In the origin sheet the formulas are calculated correctly and showing a value though. This does not always happen, sometimes it does pull in the proper value.
The intent:
To take data from different spreadsheets, combine them in a different spreadsheet and do some calculations on them. And then pull info from this calculation spreadsheet (or from multiple calculation spreadsheets) into a reporting spreadsheet for some further calculations and formatting.
The set-up:
There are several source data spreadsheets,say dataspreadsheet1, dataspreadsheet2 and dataspreadsheet3. A calculation spreadsheet (calcspreadsheet) is created that creates a copy of sheet1 in each of the data spreadsheets and names these sheets datasheet1, datasheet2 and datasheet3 respectively. The IMPORTRANGE statement used for this is created as follows: importrange(+VLOOKUP("dataspreadsheet1",filelist!A1:C1000,3,FALSE),"sheet1!a1:Z1000") where
filelist!A1:C1000 is a sheet in calcspreadsheet that contains Name, Type and Id in respective columns.
The values in each of these sheets datasheet1-3 are then used for calculations in another sheet, calcsheet1 in the calcspreadsheet. As the main goal of this is to add up daily values from the 3 dataspreadsheets, but those sourcesheets do not all have the same data on the same line a VLOOKUP is used again to make sure additions for a date use the line for the date in datasheet1-3 regardless of its row number. E.g. VLOOKUP($A11,'datasheet1'!$A:$P,4) + VLOOKUP($A11,'datasheet2'!$A:$P,4) + VLOOKUP($A11,'datasheet3'!$A:$P,4) where column A is the date column in all sheets.
This appears to work fine, although upon opening calcspreadsheet it can take a long time for it to go through an update, during which time lots of #N/A's are displayed. Eventually it comes right though.
The problem arise when a reportspreadsheet is created that in turn used an IMPORTRANGE call to pull the info from calcsheet1 in order to be able to work with it. This often, but not always, results in the problem states at the start. The IMPORTRANGE call in this reportspreadsheet is generated in a similar way as that in the calcspreadsheet: =importrange(+VLOOKUP(calc!B1,sheetcodes!A1:C3000,3,FALSE),"sheet1!a1:Z1000") where calc!B1 contains the name of the source spreadsheet (in this calc that would be 'calcspreadsheet' and sheetcodes!A1:C3000 again contains a list of sheets with Name, Type and Id in respective columns
A work-around I tried:
What I did notice that IMPORTRANGE works better on cells that do not contain VLOOKUP So I tried to copy the content of calcsheet to another sheet in calcspreadsheet, called exportsheet but having only the values in there, not formulas and then use IMPORTRANGE on this exportsheet. The copy script used is as follows:
function exportPrep() {
// Get the active spreadsheet and the active sheet
//var ss = SpreadsheetApp.getActiveSpreadsheet();
//var sheet = ss.getSheetByName("stream");
//sheet.activate();
var source = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("calcsheet");
var sourceDataRange = source.getDataRange();
var sourceSheetValues = sourceDataRange.getValues();
var sourceRows = sourceDataRange.getNumRows();
var sourceColumns = sourceDataRange.getNumColumns();
var destination = SpreadsheetApp.getActiveSpreadsheet();
SpreadsheetApp.setActiveSheet(destination.setActiveSheet("exportsheet"));
destination.getDataRange().offset(0, 0, sourceRows, sourceColumns).setValues(sourceSheetValues);
}
This seemed to work, but unfortunately the copy script used to copy the value of calcsheet into exportsheet now showed the same behaviour, it would sometimes work and sometimes give the #N/A and so leaves me with the same problem.
My questions:
I've read various posts with similar issues and responses that mentioned this function was temperamental or had a bug. Other stated it is not possible to use dynamic references in IMPORTRANGE. Given that it sometimes works and sometimes not I suspect the function itself might be correct but that there are sync or time-out issues in the set-up.
How can I set up the above functionality. Either with the use of IMPORTRANGE and VLOOKUP at all with some changes/additions, or built in a different way from the get-go.
So I've not done this with importrange but I when I have this issue with Vlookup I wrap the vlookup in an IF and test for the #N/A.
Try something like: =IF(ISNA(VLOOKUP(...)),"",VLOOKUP(...))
=IFERROR(VLOOKUP(B81,'KAM_Q3_OPPS STAGE_(H/I/L/M/N)'!$F$3:$G$111,2,False),)
This is the best way I found to do it and the last two values [ , ) ] make the new output, in this case it is nothing.
IF you wanted 0 put in then you would do [ ,0) ]

Google spreadsheet - #N/A error when imported range has a custom function

In my Google spreadsheet (the newest version, as of August 2014) I'm using =IMPORTRANGE(P5,"Tasks!b6:o7") where P5 cell contains another spreadhsheet's key. One of the cells in the other spreadsheet contains my custom function, that sometimes takes time to calculate (there are dozens of cells using that function). The problem is that in the first spr. that cell's value is not displayed properly, the #N/A error message is displayed instead.
The function is defined as follows:
// global variable
var ONEDAY = 24*60*60*1000; // hours*minutes*seconds*milliseconds
function _mySimpleCalculateDaysDifference(DateToBeChecked)
{
var todaysDate = new Date();
var todaysDayModifier = todaysDate.getHours()<6?-1:0;
var daysDiff = (todaysDate.getTime() - DateToBeChecked.getTime())/ONEDAY;
return Math.floor(daysDiff) + todaysDayModifier;
}
It's being called the following way:
=if((H18>0),(H18-F18),if((F18>0),_mySimpleCalculateDaysDifference(F18),""))
where F and H columns contain date values.
Is there any way to solve that problem?
Edit:
I mentioned that the main spreadhsheet is the newest Google spreadsheet, but those being pulled in were 'old'.
This answer relates to a situation when the importing spreadsheet is the 'new' one, and imported - the 'old' (as of September 2014):
Make sure that both spreadsheets are of the same version, i.e. old<-old or new<-new (although as #eddieparkinson commented, in his case similar issue existed for an old<-old situation).
In my case converting the imported spreadsheet to the new helped and I cannot observe the issue now.
PS
Probably the easiest way to convert is to save it as an Excel file, then import it back into Google Drive and convert into Google spreadsheet. I found it the easiest and most reliable way, although I lost my charts and scripts. There were also problems with validations, so I had to recreate them. If you're not using any of these - it should be OK.