=INDEX(IF(E2:E = "", "", IF(COUNTIFS(C2:C, C2:C, ROW(C2:C), "<=" & ROW(C2:C)) = 1, "Unique", "Duplicate"))) takes a minute for each new entry to calculate when there are 50,000 rows. Is there a way to optimize this without reducing the range?
Related
I am learning right now scripts functionally in Google Sheet, however, can't twist my head around constructing a very simple App script.
I have the following table (Snoopi Tab)
https://docs.google.com/spreadsheets/d/1l6nYBAqB1GWoMkIOwlykhiuMpaXdWHTo7UhgZdq6hT8/edit?usp=sharing
I want it to do this simple action:
EXAMPLE: If today is not Sunday or Saturday and the date is 14.2.14 and cell BF5 is
---> go down 3 rows and paste current time "Clocking in" working-shift
When button "IN" is clicked:
If (TODAYDATE = Value in cell in row 5) & (row 3 ==!"S") both true
Set current time in (same column just row 8)
Same with "OUT" button, but this I'll try to figure by myself.
The other answer is acceptable, but is very resource intensive and have a lot of loops to do resulting to very slow execution time especially when it gets later on the year since it will loop all those dates.
Also, the run you did on the other answer did finish successfully but didn't write anything due to it missing the actual date value. This might have been caused by a timezone issue, or by only modifying the actual date while getting the raw time of the cell value.
A better alternative would be to make use of the 4th row where it contains x value when the date is equal to the current date. By using that, you wouldn't need to loop thus resulting in faster execution time and wouldn't need to convert time thus making it safer. As long as row 4 is populated on all columns (which your problem is), there should be no issue of using this script.
Script:
function WorkClock() {
var currentDate = new Date();
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = SpreadsheetApp.getActiveSheet();
// you only need 3rd and 4th row of data
var data = sheet.getRange("E3:NE4").getValues();
// 4th row contains 'x' when today matches the column, find that index
var indexToday = data[1].indexOf('x');
// if that column's row 3 is not 'S'
if(data[0][indexToday] != 'S')
// write the time on row 8
sheet.getRange(8, indexToday + 5).setValue(Utilities.formatDate(currentDate, ss.getSpreadsheetTimeZone(), 'HH:mm'));
}
Output:
Note:
Timezone used is based on the spreadsheet's timezone which is GMT-8. Wherever the user is, it will use GMT-8, not its local time which should be helpful in some cases.
Performance difference between this and looping all dates would be vast if we are now dealing with the later months of the year (e.g. November, December)
For the OUT button, create another function by duplicating the current function. Then replace where you write the time. Instead of row 8 (Start), write it in row 10 (Finish).
function myFunction() {
var actualDate = new Date(new Date().setHours(0, 0, 0, 0)).getTime();
var sheet = SpreadsheetApp.getActiveSheet();
var data = sheet.getRange("E3:NE6").getValues();
for(var i = 0; i < data[0].length; i++) {
if (data[2][i].getTime() === actualDate) {
if (data[0][i] !== "S") {
sheet.getRange(8, (5+i)).setValue(new Date().getHours() + ":" + new Date().getMinutes());
}
break;
}
}
}
I am trying to create an attendance system that calculates tardies based on certain criteria. First off, each student member has a row dedicated to them in the spreadsheet. (Ie, student1's attendance is found on row 3, student2's attendance is found on row 4, etc.). Each student has personal information in the first few columns. Then each student has an attendance record for the meeting. All attendance information comes in pairs with a sign in time and a sign out time.
A student receives a tardy if they show up after 6:30 PM. They also receive a tardy if they are present for less than 2 hours. So I wrote a code to calculate the number of tardies.
/**
each student has one row dedicated to their attendance. Sign in and sign out are always in pairs. I will ensure that the range
begins with range[0][0] being the first sign in time.
#param range the row for the student's attendance
#return the total tardies for each student
**/
function calculateTardies(range){
var search = range[0].length;
var tardies = 0;//initially set to 0
for(k = 0; k<search; k+2){ //go through the entire row of the student's sign in and sign out times (k+2 because they always come in pairs)
var timeIn = new Date(range[0][k]); //sign in time will always be at a K value
var date = Utilities.formatDate(timeIn, "EST", "EEE"); //get the day of week that time stamp is
var timeOut = new Date(range[0][k+1]); //sign out time is to the right of the sign in time
if(date == "Tue" || date == "Wed" || date == "Thu"){ //they only need to be there for 2 hours Tues-Thurs
var checkHour = Number(Utilities.formatDate(timeIn, "EST", "HH")); //The hour student arrived
var checkMin = Number(Utilities.formatDate(timeIn, "EST", "mm")); //The minute student arrived
var outHour = Number(Utilities.formatDate(timeOut, "EST", "HH")); //hour student left
var outHour = Number(Utilities.formatDate(timeOut, "EST", "HH")); // min student left
//If the student wasn't present for at least 2 hours, add a tardy
if((checkHour+2)*60+checkMin < (outHour*60)+outMin){
tardies = tardies+1;
}
//if the student left before 6:30 PM, add a tardy
if((checkHour*60)+checkMin > 1830){ //11100 is 6:30 PM in minutes 6PM=> 18 hours * 60 min = 1080 minutes + 30 additional minutes
tardies = tardies+1;
}
}
}
return tardies;
}
To check if they are there for two hours I had to change the time stamps to minutes. I found that I couldn't simply compare the times as is. The same applies for why I changed the arrival time to minutes before comparing it to 6:30 PM.
I am running into an error that states exceeded maximum execution time (line 0). Any advice on how I can change the code or fix this error would be greatly appreciated.
I am running into an error that states exceeded maximum execution time
You have an endless loop that makes your code run until it exceeds maximum execution time
This endless loop is created by the definition
for(k = 0; k<search; k+2)
depending on what you want to do, the correct syntax is either
for(k = 0; k<search; k=k+2)
(if you want to increase k by 2 after each iteration) or
for(k = 0; k<search; k=k++)
if you want to increase k by 1 after each iteration.
I found that I couldn't simply compare the times as is.
Dates are proceeded in Apps Script the same way like in Javascript.
The documentation provides many useful methods and examples of how to wokr with dates.
In a nutshell:
If you want to calculate the difference between two timestamps - the easiest way might be to convert both dates to ms with getTime(), substract them and convert the difference back to hours (1 h = 1000*60*60 ms)
If you want to know the weekday of a date, you need to use getDay()
For working with dates it is recommendable that they are formatted as such in your spreadsheet, rather than manually converting them from numbers inside your script.
Thanks in advance for your help. I'm new to apps script.
I have a gsheet with 98 columns and 25000 rows. All I want to do is copy 24 of the 98 columns to a new sheet.
Currently I am using filter & map, and it works:
var data = sourceSheet.getDataRange().getValues(); //read all columns & rows into array
var filterData = data.filter(function(e, j){return j > 0 && e}).map(function(e){return [e[88], e[14], e[13], e[4], e[17], e[87], e[91], e[48], e[57], e[31], e[89], e[82], e[70], e[97], e[47], e[30], e[72], e[71], e[67], e[34], e[33], e[00], e[38], e[39]]}); //extract just the columns I want into a new array
but that 2nd line takes almost an hour to execute! I presume because it is processing every element 1-by-1, even though it is just to return each one.
I haven't tried getValue'ing and setValue'ing columns one at a time because everything I read says limit external calls and do everything in memory. And I can't imagine pushing elements 1-by-1 would be faster than filtering.
Suggestions for faster execution?
function doCopy(SpreadID, OrgSheet, DestSheet, OrgRange, Sql, DestCell) {
var mySpread = SpreadsheetApp.openById(SpreadID);
var myQry = '=QUERY(' + OrgSheet + "!" + OrgRange + ',\"'+ Sql + '\")';
var myDestSheet = mySpread.getSheetByName(DestSheet);
myDestSheet.getRange(DestCell).setFormula(myQry);
}
Sample to call, but the destination sheet must be blank:
doCopy(spreadsheetId,"Sheet1", "Sheet2", "A:G", "Select A, C, F", "A1");
Another way to look at the problem which might be more time-effective and easy to implement is the following :
Duplicate the original sheet into a new one with copyTo(spreadsheet) (https://developers.google.com/apps-script/reference/spreadsheet/sheet#copytospreadsheet). You might also want to check this post for more information about creating a sheet into the same spreadsheet (Google Script to Duplicate Sheet that is Not Active)
Delete the columns you don't want to keep with deleteColumn(columnPosition) or deleteColumns(columnPosition, howMany)(see https://developers.google.com/apps-script/reference/spreadsheet/sheet#deletecolumncolumnposition).
You might also want to start deleting columns from the right side of the sheet to make the implementation easier.
Let me know if it helps.
I am working on a kinda large sheet which I would like to update on a regular basis (weekly) via Google Apps Scripts.
Every week I need to add a new column at the "end" (lastDataColumn) of my sheet and then move the last two colums (with Fomulas to calculate weekly relative changes) to the "end" (ie move them one column to the right). This leaves me with a blank column addressed with lastDataColumn - 2. This is where the new report data will go.
I have two functions.
copyCols and getReports.
They both work fine on their own, so copyCols creates a new empty column at position lastDataColumn - 2 using the method explained above - and getReports fetches report Data from Analytics, third party APIs and other sheets and then writes these values in the column at position lastDataColumn - 2.
However, if I group these two functions in let's say my main function which I then want to trigger on a 7-day basis, copyCols seems to only execute to the point of creating a new empty column. Then getReports executes fully and writes all data in lastDataColumn - 2. But no columns were moved, so getReports overwrites last weeks data. After executing everything from getReports copyCols starts moving the rows (ie copying).
This leaves me with a duplicate column of lastDataColumn - 3 (which should have last weeks data, but was overwritten with this weeks data because it was still in lastDataColumn - 2 before the execution of getReports) in lastDataColumn - 2.
To clarify: Executing copyCols and getReports afterwards (each on it's own) works perfectly fine.
Is Google Apps Script threadless? If so why does the problem described happen? Do "bulk" operations (like copying ranges) execute at the end of the script?
Code:
var today = new Date();
var start = new Date(today.getTime() - 7 * 24 * 60 * 60 * 1000);
var end = new Date(today.getTime() - 1 * 24 * 60 * 60 * 1000);
var dateString=Utilities.formatDate(start, Session.getScriptTimeZone(),'dd.MM.')+'-'+Utilities.formatDate(end, Session.getScriptTimeZone(),'dd.MM.')
var nc=2 //num of cols with growth rates to move
function main() {
copyCols();
getReports();
}
function copyCols(){
var ss=SpreadsheetApp.openById('1dSpy7teczLwViKbfr-VjfQRuOq3iaRfSm3LghFldjZk')
var sh_DB=ss.getSheetByName('data')
var w=sh_DB.getLastColumn(); //width
var h = sh_DB.getLastRow(); //heigth
// insert new column
sh_DB.insertColumnAfter(w);
// copy last n cols to next col
for (i=0;i<=nc;i++){
sh_DB.getRange(1,w-i,h,1).copyTo(sh_DB.getRange(1,w-i+1,h,1));
}
sh_DB.getRange(1,w-nc+1).setValue(dateString);
}
function getReports(){
var sh_DB=SpreadsheetApp.getActiveSpreadsheet().getSheetByName('data')
var w=sh_DB.getLastColumn(); //width
var h = sh_DB.getLastRow(); //heigth
dc=sh_DB.getRange(1,w-nc); //lastDataColumn
data = [50, 60, 870, 2];
report = {'rows':[2,3,4,5]};
for (i in data){
sh_DB.getRange(report['rows'][i],w-nc).setValue(data[i]);
}
}
Thank you for any help provided.
Add SpreadsheetApp.flush() between copyCols() and getReports() in order to tell to the Google Apps Script engine to apply the changes made by the first before running the second.
I am working on a kinda large sheet which I would like to update on a regular basis (weekly) via Google Apps Scripts.
Every week I need to add a new column at the "end" (lastDataColumn) of my sheet and then move the last two colums (with Fomulas to calculate weekly relative changes) to the "end" (ie move them one column to the right). This leaves me with a blank column addressed with lastDataColumn - 2. This is where the new report data will go.
I have two functions.
copyCols and getReports.
They both work fine on their own, so copyCols creates a new empty column at position lastDataColumn - 2 using the method explained above - and getReports fetches report Data from Analytics, third party APIs and other sheets and then writes these values in the column at position lastDataColumn - 2.
However, if I group these two functions in let's say my main function which I then want to trigger on a 7-day basis, copyCols seems to only execute to the point of creating a new empty column. Then getReports executes fully and writes all data in lastDataColumn - 2. But no columns were moved, so getReports overwrites last weeks data. After executing everything from getReports copyCols starts moving the rows (ie copying).
This leaves me with a duplicate column of lastDataColumn - 3 (which should have last weeks data, but was overwritten with this weeks data because it was still in lastDataColumn - 2 before the execution of getReports) in lastDataColumn - 2.
To clarify: Executing copyCols and getReports afterwards (each on it's own) works perfectly fine.
Is Google Apps Script threadless? If so why does the problem described happen? Do "bulk" operations (like copying ranges) execute at the end of the script?
Code:
var today = new Date();
var start = new Date(today.getTime() - 7 * 24 * 60 * 60 * 1000);
var end = new Date(today.getTime() - 1 * 24 * 60 * 60 * 1000);
var dateString=Utilities.formatDate(start, Session.getScriptTimeZone(),'dd.MM.')+'-'+Utilities.formatDate(end, Session.getScriptTimeZone(),'dd.MM.')
var nc=2 //num of cols with growth rates to move
function main() {
copyCols();
getReports();
}
function copyCols(){
var ss=SpreadsheetApp.openById('1dSpy7teczLwViKbfr-VjfQRuOq3iaRfSm3LghFldjZk')
var sh_DB=ss.getSheetByName('data')
var w=sh_DB.getLastColumn(); //width
var h = sh_DB.getLastRow(); //heigth
// insert new column
sh_DB.insertColumnAfter(w);
// copy last n cols to next col
for (i=0;i<=nc;i++){
sh_DB.getRange(1,w-i,h,1).copyTo(sh_DB.getRange(1,w-i+1,h,1));
}
sh_DB.getRange(1,w-nc+1).setValue(dateString);
}
function getReports(){
var sh_DB=SpreadsheetApp.getActiveSpreadsheet().getSheetByName('data')
var w=sh_DB.getLastColumn(); //width
var h = sh_DB.getLastRow(); //heigth
dc=sh_DB.getRange(1,w-nc); //lastDataColumn
data = [50, 60, 870, 2];
report = {'rows':[2,3,4,5]};
for (i in data){
sh_DB.getRange(report['rows'][i],w-nc).setValue(data[i]);
}
}
Thank you for any help provided.
Add SpreadsheetApp.flush() between copyCols() and getReports() in order to tell to the Google Apps Script engine to apply the changes made by the first before running the second.