I'd like to know whether it is possible to write a Google Apps Script that will trigger daily importhtml in Google spreadsheets. For example let's assume I want to import updates of following table everyday at 1 pm.
=IMPORTHTML("https://en.wikipedia.org/wiki/Lubbock,_Texas","table",5)
I need this automatic checking for table updates even when my computer is offline.
Thank you
Yes, it is possible to write a script to make a copy of the table every day.
In short, you will have to make a copy of the imported table and paste it into a new Sheet. ImportHTML a gets updated every time you open/access the spreadsheet. So only way to store table is to make a copy and paste it into a new sheet.
Here is code that does just that:
function getImportData()
{
var ss = SpreadsheetApp.getActive()
var sheet =ss.getSheetByName("ImportSheet")
if (sheet == null)
{
sheet = ss.insertSheet("ImportSheet")
sheet.getRange(1,1).setValue( "=IMPORTHTML(\"https://en.wikipedia.org/wiki/Lubbock,_Texas\",\"table\",5)")
}
var dataRange = sheet.getDataRange()
//waitforLoading waits for the function importHTML to finish loading the table, only a problem if table takes a while to load
//for more discussion on this visit : http://stackoverflow.com/questions/12711072/how-to-pause-app-scripts-until-spreadsheet-finishes-calculation
var wait = waitForLoading(dataRange,sheet, 10)
var logSheet = ss.getSheetByName("ImportLogSheet")
if (logSheet == null)
{
logSheet = ss.insertSheet("ImportLogSheet")
}
var timeStampRow = []
timeStampRow[0] = new Date()
if(wait)
{
logSheet.appendRow(timeStampRow)
var lastRow = logSheet.getLastRow() + 1
var destinationRange = logSheet.getRange(lastRow,1)
dataRange.copyTo(destinationRange, {contentsOnly:true})
}
else {
timeStampRow[1] = "Scripted timeout waiting for the table to Load "
logSheet.appendRow(timeStampRow)
}
}
function waitForLoading(dataRange, sheet, maxWaitTimeInSec)
{
// This function is only required if it takes a while for the importHTML to load your data!
// if not you can skip this function
// Function looks to see if the value of the
for(i = 0; i< maxWaitTimeInSec ; i++)
{
var value = dataRange.getCell(1,1).getValue()
if(value.search("Loading") !== -1) {
Utilities.sleep(1000);
dataRange = sheet.getDataRange()
} else {
return true
}
}
return false
}
Edit: Forgot to mention about setting up triggers. You can setup any function to be triggered using time-driven triggers. Details on how to set it up is given here: https://developers.google.com/apps-script/guides/triggers/installable#managing_triggers_manually
Related
I have a large sheet with around 30 importxml functions that obtain data from a website that updates usually twice a day.
I would like to run the importxml function on a timely basis (every 8 hours) for my Google Spreadsheet to save the data in another sheet. The saving already works, however the updating does not!
I read in Google Spreadsheet row update that it might run every 2 hours, however I do not believe that this is true, because since I added it to my sheet nothing has changed or updated, when the spreadsheet is NOT opened.
How can I "trigger" the importxml function in my Google Spreadsheet in an easy way, as I have a lot of importxml functions in it?
I made a couple of adjustments to Mogsdad's answer:
Fixed the releaseLock() call placement
Updates (or adds) a querystring parameter to the url in the import function (as opposed to storing, removing, waiting 5 seconds, and then restoring all relevant formulas)
Works on a specific sheet in your spreadsheet
Shows time of last update
...
function RefreshImports() {
var lock = LockService.getScriptLock();
if (!lock.tryLock(5000)) return; // Wait up to 5s for previous refresh to end.
var id = "[YOUR SPREADSHEET ID]";
var ss = SpreadsheetApp.openById(id);
var sheet = ss.getSheetByName("[SHEET NAME]");
var dataRange = sheet.getDataRange();
var formulas = dataRange.getFormulas();
var content = "";
var now = new Date();
var time = now.getTime();
var re = /.*[^a-z0-9]import(?:xml|data|feed|html|range)\(.*/gi;
var re2 = /((\?|&)(update=[0-9]*))/gi;
var re3 = /(",)/gi;
for (var row=0; row<formulas.length; row++) {
for (var col=0; col<formulas[0].length; col++) {
content = formulas[row][col];
if (content != "") {
var match = content.search(re);
if (match !== -1 ) {
// import function is used in this cell
var updatedContent = content.toString().replace(re2,"$2update=" + time);
if (updatedContent == content) {
// No querystring exists yet in url
updatedContent = content.toString().replace(re3,"?update=" + time + "$1");
}
// Update url in formula with querystring param
sheet.getRange(row+1, col+1).setFormula(updatedContent);
}
}
}
}
// Done refresh; release the lock.
lock.releaseLock();
// Show last updated time on sheet somewhere
sheet.getRange(7,2).setValue("Rates were last updated at " + now.toLocaleTimeString())
}
The Google Spreadsheet row update question and its answers refer to the "Old Sheets", which had different behaviour than the 2015 version of Google Sheets does. There is no automatic refresh of content with "New Sheets"; changes are only evaluated now in response to edits.
While Sheets no longer provides this capability natively, we can use a script to refresh the "import" formulas (IMPORTXML, IMPORTDATA, IMPORTHTML and IMPORTANGE).
Utility script
For periodic refresh of IMPORT formulas, set this function up as a time-driven trigger.
Caveats:
Import function Formula changes made to the spreadsheet by other scripts or users during the refresh period COULD BE OVERWRITTEN.
Overlapping refreshes might make your spreadsheet unstable. To mitigate that, the utility script uses a ScriptLock. This may conflict with other uses of that lock in your script.
/**
* Go through all sheets in a spreadsheet, identify and remove all spreadsheet
* import functions, then replace them a while later. This causes a "refresh"
* of the "import" functions. For periodic refresh of these formulas, set this
* function up as a time-based trigger.
*
* Caution: Formula changes made to the spreadsheet by other scripts or users
* during the refresh period COULD BE OVERWRITTEN.
*
* From: https://stackoverflow.com/a/33875957/1677912
*/
function RefreshImports() {
var lock = LockService.getScriptLock();
if (!lock.tryLock(5000)) return; // Wait up to 5s for previous refresh to end.
// At this point, we are holding the lock.
var id = "YOUR-SHEET-ID";
var ss = SpreadsheetApp.openById(id);
var sheets = ss.getSheets();
for (var sheetNum=0; sheetNum<sheets.length; sheetNum++) {
var sheet = sheets[sheetNum];
var dataRange = sheet.getDataRange();
var formulas = dataRange.getFormulas();
var tempFormulas = [];
for (var row=0; row<formulas.length; row++) {
for (col=0; col<formulas[0].length; col++) {
// Blank all formulas containing any "import" function
// See https://regex101.com/r/bE7fJ6/2
var re = /.*[^a-z0-9]import(?:xml|data|feed|html|range)\(.*/gi;
if (formulas[row][col].search(re) !== -1 ) {
tempFormulas.push({row:row+1,
col:col+1,
formula:formulas[row][col]});
sheet.getRange(row+1, col+1).setFormula("");
}
}
}
// After a pause, replace the import functions
Utilities.sleep(5000);
for (var i=0; i<tempFormulas.length; i++) {
var cell = tempFormulas[i];
sheet.getRange( cell.row, cell.col ).setFormula(cell.formula)
}
// Done refresh; release the lock.
lock.releaseLock();
}
}
To answer your question for an easy "trigger" to force the function to reload:
add an additional not used parameter to the url you are loading, while referencing a cell for the value of that parameter.
Once you alter the content of that cell, the function reloads.
example:
importxml("http://www.example.com/?noop=" & $A$1,"...")
unfortunately you cannot put a date calculating function into the referenced cell, that throws an error that this is not allowed.
You can also put each XML formula as a comment in the respective cells and record a macro to copy and paste it in the same cell. Later use the Scripts and then the Trigger functionality to schedule this macro.
I have an html form which I'm passing data to and taking the values here. I want to create a system to catch whenever the same data is repeated. I'm doing the if statement below which is supposed to catch whenever the same data is entered, but it is not working properly. The issue is writes the same data multiple times.
function processFormClients(formObject) {
var url = "LINK";
var ss = SpreadsheetApp.openByUrl(url);
var Clients = ss.getSheetByName("Clients");
var data = Clients.getDataRange().getValues();
for (var i = 0; i < data.length; i++) {
if(data[i][1] !== formObject.client_name) {
Clients.appendRow([
Math.floor(Math.random() * Date.now()),
formObject.client_name,
formObject.client_company,
formObject.client_budget,
]);
} else if (data[i][1] === formObject.client_name) {
console.log('failed')
}
}
Modification points:
In your script, when the column "B" of var data = Clients.getDataRange().getValues() is not same with the value of formObject.client_name, Clients.appendRow is run every time. Only when the value of formObject.client_name is the same with data[i][1], console.log('failed') is run. I thought that this might be the reason of your issue.
In order to append the data of [Math.floor(Math.random() * Date.now()), formObject.client_name, formObject.client_company, formObject.client_budget] when formObject.client_name is not existing in the column "B", how about the following modified script?
Modified script:
function processFormClients(formObject) {
var url = "LINK";
var ss = SpreadsheetApp.openByUrl(url);
var Clients = ss.getSheetByName("Clients");
var search = Clients.getRange("B1:B" + Clients.getLastRow()).createTextFinder(formObject.client_name).findNext();
if (search) {
console.log('failed');
return;
}
Clients.appendRow([
Math.floor(Math.random() * Date.now()),
formObject.client_name,
formObject.client_company,
formObject.client_budget,
]);
}
In this modification, the duplicated value of column "B" is checked using TextFinder. When TextFinder is used, the process cost can be reduced a little. Ref
When this script is run, only when formObject.client_name is not existing in the column "B" of the sheet "Clients", Clients.appendRow is run.
References:
createTextFinder(findText)
Class TextFinder
I've gotten to the point where I have a pretty solid script that:
1) Takes data from a new data tab and posts it to an existing data tab
2) Clears data from the new data tab
3) Deletes duplicates from the existing data tab
When I originally put this script in, it worked great. But after running it a few times, it seems to stall at the de-duping portion of the script. So when I run it, the first two scripts run, but not the third. If I select the de-dupe script to run on it's own, it works just fine.
Has anyone else seen this issue? Is there any way to tweak the script to have a more reliable run so that it will always process all three scripts?
Not sure how to optimize from there.
function Run(){
insert();
clear1();
removeDuplicates();
}
function insert() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var source = ss.getSheetByName('Candidate Refresh'); // change here
var des = ss.getSheetByName('Candidate Listing'); // change here
var sv = source
.getDataRange()
.getValues();
sv.shift();
des.insertRowsAfter(1, sv.length);
des.getRange(2, 1, sv.length, source.getLastColumn()).setValues(sv);
}
function clear1() {
var sheet = SpreadsheetApp.getActive().getSheetByName('Candidate Refresh');
sheet.getRange('A2:K100').clearContent()
}
function removeDuplicates() {
var sheet=SpreadsheetApp.getActiveSheet();
var rows=sheet.getLastRow();
var firstColumn=sheet.getRange(1, 2, rows, 1).getValues();
firstColumn = firstColumn.map(function(e){return e[0]})
var uA=[];
for (var i=rows;i>0;i--) {
if (uA.indexOf(firstColumn[i-1])!=-1) {
sheet.deleteRow(i);
}else{
uA.push(firstColumn[i-1]);
}
}
}
All three scripts should fire when the Run script is played.
I'm trying to programmatically create a trigger for a Google Spreadsheet when a form is submitted with code from another Spreadsheet, but it won't work. Here is my code:
function createOnFormSubmitTrigger()
{
//Id of "Spreadsheet 1"
var ssId = "18bq-67nP4y7F9Hp4jzpbKPCyAsR6hglgcfmxCi_zj14";
ScriptApp.newTrigger("formInput").forSpreadsheet(ssId).onFormSubmit().create();
}
If I put that method into "Spreadsheet 1" and run it, it works fine and creates the script in Spreadsheet 1 as intended. However, if I put that method into say "Spreadsheet 2" and run it, it creates the trigger in Spreadsheet 2 instead of Spreadsheet 1, which is not as intended. What am I doing wrong?
Here is the code for formInput for the original script and the dummy script I made for testing:
Original:
function formInput()
{
var spreadSheet = SpreadsheetApp.getActiveSpreadsheet();
var inSheet = spreadSheet.getSheetByName("Form Responses");
var data = inSheet.getSheetValues(2,2,1,7);
var minutes = data[0][0];
var seconds = data[0][1];
var numMissed = data[0][2];
var sgIncorrect = data[0][3];
var sfMissed = data[0][4];
var year = data[0][5];
var testLevel = data[0][6];
var date = new Date();
var outSheet = spreadSheet.getSheetByName(testLevel);
var testLevelExists = (outSheet != null);
if(testLevelExists)
{
outSheet.insertRowBefore(2);
outSheet.getRange(2,1).setValue(date.getMonth()+1+ "/" + date.getDate());
outSheet.getRange(2,2).setValue(year);
var secStr = ("0" + seconds);
outSheet.getRange(2,3).setValue(minutes + ":" + secStr.substring(secStr.length-2));
outSheet.getRange(2,4).setValue(numMissed);
outSheet.getRange(2,5).setValue(sgIncorrect);
outSheet.getRange(2,6).setValue(sfMissed);
outSheet.getRange(2,7).setValue(350-7*(sgIncorrect+numMissed)-2*(sfMissed));
shiftBox(outSheet);
setFormulas(outSheet);
updateAverageTime(outSheet);
inSheet.deleteRow(2);
copyToMaster(spreadSheet,outSheet);
}
}
Dummy:
function formInput()
{
var hi = "hello";
}
I'd like to thank #JSmith for all his help first of all. I seemed to have misunderstood the creation of new triggers using the ScriptApp API. When the script is created, it links the trigger to actions performed on the target sheet, but the code must be in the original sheet which created the trigger. The trigger is also only displayed on the sheet which created the trigger.
As seen together,
try all your functions from google's IDE once.
Also it seems a trigger created from a certain spreadsheet is viewable from the trigger menu of the original spreadsheet script even if this trigger is linked to another spreadsheet.
I have a simple function to get some cell value
function getValue() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheets()[4];
var range = sheet.getRange("C2:C4");
var cell = range.getCell(1, 1); // "C2"
if (cell.isBlank()) {
return 'error'
} else {
return cell.getValue()
}
}
But when I change data in C2, cell, which contains =getValue() function does not refresh itself instantly. Only if I run script again and get back to sheet. Is it possible to speed this process up? Any code for this? Thanks.
If you have to use the custom functions for this situation, how about this workaround? I don't know whether this is the best way for you. Please think of this as one of several answers.
The flow of script is as follows.
Flow :
Retrieve all values and formulas on the sheet.
Remove values of cells which have formulas.
Reflect values to the sheet using SpreadsheetApp.flush().
Import formulas to the removed cells.
By onEdit(), when you edit the cell, this sample script is launched.
Sample script :
function onEdit(e) {
var range = e.source.getDataRange();
var data = range.getValues();
var formulas = range.getFormulas();
var values = data.map(function(e){return e.slice()});
for (var i in formulas) {
for (var j in formulas[i]) {
if (formulas[i][j]) {
data[i][j] = formulas[i][j];
values[i][j] = "";
}
}
}
range.setValues(values);
SpreadsheetApp.flush();
range.setValues(data);
}
Note :
In this situation which imports a value at "C2" to the cell at =getValue(), the refresh speed is slower than that of #random-parts's method.
To use onEdit() is also proposed from #Cooper.
If this was not useful for you, I'm sorry.