Related
There is a collection of data which needs to be shown in the spreadsheet. I mean data will have values for each row(in this case Housing, CapitalRaised, SizePerSquare etc). How to plot below data in the spreadsheet in following format?
the data comes in this format
[
{
"Housing": "Before housing price",
"Price": 5,
"Rate": 0.75
},
{
"CapitalRaised": 5000,
"SizePerSquare": 12,
"Price": null,
"RatePerSquare": 1.25
},
{
"CapitalRaised": 6000,
"SizePerSquare": 24,
"Price": null,
"RatePerSquare": 1
},
{
"CapitalRaised": 7000,
"SizePerSquare": 24,
"Price": null,
"RatePerSquare": 0.75,
}
]
Here is the code
function plotData() {
var data =[
{
"Housing": "Before Capital Raised",
"Price": 5,
"Rate": 0.75
},
{
"CapitalRaised": 5000,
"SizePerSquare": 12,
"Price": null,
"RatePerSquare": 1.25
},
{
"CapitalRaised": 6000,
"SizePerSquare": 24,
"Price": null,
"RatePerSquare": 1
},
{
"AmountRaised": 7000,
"SizePerSquare": 24,
"Price": null,
"RatePerSquare": 0.75,
}
]
var activeSheet = SpreadsheetApp.getActiveSpreadsheet();
var sheetname = "PlotData2";
var sheet = activeSheet.getSheetByName(sheetname);
var startRow = 4;
var range = "'" + sheetname + "'!B" + startRow;
}
Here is the link to spreadsheet
https://docs.google.com/spreadsheets/d/1tLNZv4F4lpBAnmHN5H0pBiirW4MVIfTexll9jPA03hI/edit#gid=1286090443
You want to achieve the following situation using Google Apps Script.
From:
var data = [
{"Housing":"Before housing price","Price":5,"Rate":0.75},
{"CapitalRaised":5000,"SizePerSquare":12,"Price":null,"RatePerSquare":1.25},
{"CapitalRaised":6000,"SizePerSquare":24,"Price":null,"RatePerSquare":1},
{"CapitalRaised":7000,"SizePerSquare":24,"Price":null,"RatePerSquare":0.75}
];
To:
The header titles of rows are constant.
If my understanding is correct, how about this answer? In this answer, from your question, I thought that the header titles of rows are constant. I used this situation.
Sample script:
Before you use the following scripts, please enable Sheets API at Advanced Google services. When the all column length of the array is not the same, the method of Spreadsheets.Values.update is easy to put the array to Spreadsheet. So I used Sheets API.
function plotData() {
var data = [
{"Housing":"Before housing price","Price":5,"Rate":0.75},
{"CapitalRaised":5000,"SizePerSquare":12,"Price":null,"RatePerSquare":1.25},
{"CapitalRaised":6000,"SizePerSquare":24,"Price":null,"RatePerSquare":1},
{"CapitalRaised":7000,"SizePerSquare":24,"Price":null,"RatePerSquare":0.75}
];
// Convert "data" to an array for putting to Spreadsheet.
var rowHeaders = ["Housing", "Price", "CapitalRaised", "RatePerSquare", "SizePerSquare"];
var values = data.reduce(function(ar, e, i) {
rowHeaders.forEach(function(g, n) {
if (!Array.isArray(ar[n])) ar[n] = [g];
ar[n][i + 1] = e[g];
});
return ar;
}, []);
// Put the converted array to Spreadsheet.
var activeSheet = SpreadsheetApp.getActiveSpreadsheet();
var sheetname = "PlotData2";
var sheet = activeSheet.getSheetByName(sheetname);
var startRow = 4;
var range = "'" + sheetname + "'!B" + startRow;
Sheets.Spreadsheets.Values.update({values: values}, activeSheet.getId(), range, {valueInputOption: "USER_ENTERED"});
}
In above script, if there are the keys, which are not included in rowHeaders, in the keys of object of data, the values of the keys are not used. Please be careful this.
For example, Rate of the first element of data is not used.
Note:
In your data, it seems that the data of the data comes in this format is different from the data of Here is the code. "CapitalRaised": 7000, and "AmountRaised": 7000 are different. From your image of the expected result, I supposed that you want to use the data of the data comes in this format. If I misunderstood your goal, I apologize.
Reference:
Method: spreadsheets.values.update
If I misunderstood your question and this was not the direction you want, I apologize.
I am currently using ImportJSON to import Sendgrid Email with data Keenio Extraction Query API URL by calling the ImportJSON function in a Google Spreadsheet cell of Sheet DATA.
=ImportJSON("https://api.keen.io/3.0/projects/"& PROJECT_KEY & "/queries/extraction?api_key=" & API_KEY & "&event_collection=" & EVT_COL & "&timezone=" & TIMEZONE & "&latest=" & LATEST & "&property_names..........", PTDATA!$AB$1)
In Sheet PTDATA, in the last column cell i am setting a random number for ImportJSON to recalculate. The function runs on Spreadsheet open event. I have also added a custom menu to call the ReCalcCell custom function.
function onOpen() {
var ui = SpreadsheetApp.getUi();
// Or DocumentApp or FormApp.
ui.createMenu('IMPORT DATA')
.addItem('KEENIO DATA', 'ReCalcCell')
.addToUi();
}
function ReCalcCell(){
var min = Math.ceil(0);
var max = Math.floor(9999);
var randomNum = Math.floor(Math.random() * (max - min + 1)) + min
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sh = ss.getSheetByName("PTDATA");
sh.getRange("$AB$1").setValue(randomNum);
}
PTDATA sheet has specific column header names for which i want to pull the data from DATA sheet. Towards the right of these columns, i have other calculation columns which work on these specific columns.
Since the columns in DATA sheet always appear in a random / shuffled order, i had to write a small custom function GCL which takes in a header name and returns its datarange address from DATA sheet as a string.
function GCL(header,dummy) {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("DATA");
var headings = sheet.getRange(1, 1, 1, sheet.getLastColumn()); // get the range representing the whole sheet
var width = headings.getWidth();
var lrow = sheet.getLastRow();
// search every cell in row 1 from A1 till the last column
for (var i = 1; i <= width; i++) {
var data = headings.getCell(1,i).getValue();
if (data == header) {
return ((sheet.getSheetName() + "!" + columnToLetter(i)+"2:" + columnToLetter(i) + lrow).toString()); // return the column range if we find it
break; // exit when found
}
}
return(-1); // return -1 if it doesn't exist
}
function columnToLetter(column)
{
var temp, letter = '';
while (column > 0)
{
temp = (column - 1) % 26;
letter = String.fromCharCode(temp + 65) + letter;
column = (column - temp - 1) / 26;
}
return letter;
}
Then i use the custom function GCL in each specific column to get it's datarange. Once data is populated, the PDATA sheet is used to create different Pivots for reporting purposes.
=ARRAYFORMULA(INDIRECT(GCL(A1,$AB$1)))
The problems i am facing is that though the ImportJSON data populates the DATA sheet:
DATA Sheet:
The columns appear shuffled everytime, so my calculation columns cannot calculate as the references go away. This renders the pivots useless! To counter this issue, i had to create the PDATA sheet to pull in specific columns using the custom function GCL.
The custom function GCL does not always refresh and most of the time shows #Ref error.
PDATA Sheet:
BTW, my JSON output from Keenio looks like this:
{
"result":
[
{
"sg_event_id": "92-OndRfTs6fZjNdHWzLBw",
"timestamp": 1529618395,
"url": "https://noname.com?utm_campaign=website&utm_source=sendgrid.com&utm_medium=email",
"ip": "192.168.1.1",
"event": "click",
"keen": {
"timestamp": "2018-06-21T21:59:55.000Z",
"created_at": "2018-06-21T22:00:28.532Z",
"id": "555c1f7c5asdf7000167d87b"
},
"url_offset": {
"index": 38,
"type": "text"
},
"sg_message_id": "F5mwV1rESdyKFA_2bn1IEQ.filter0042p3las1-15933-5B2A68E8-36.0",
"useragent": "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)",
"email": "no.name#noname.com"
}, {
"sg_event_id": "bjMlfsSfRyuXEVy8LndsYA",
"timestamp": 1529618349,
"url": "https://noname.com?utm_campaign=website&utm_source=sendgrid.com&utm_medium=email",
"ip": "192.168.1.1",
"event": "click",
"keen": {
"timestamp": "2018-06-21T21:59:09.000Z",
"created_at": "2018-06-21T21:59:39.491Z",
"id": "555c1f7c5asdf7000167d87b"
},
"url_offset": {
"index": 36,
"type": "text"
},
"sg_message_id": "F5mwV1rESdyKFA_2bn1IEQ.filter0042p3las1-15933-5B2A68E8-36.0",
"useragent": "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)",
"email": "no.name#noname.com"
}, {
"sg_event_id": "fru_s2s1RtueuqBMNoIoTg",
"timestamp": 1529618255,
"url": "https://noname.com?utm_campaign=website&utm_source=sendgrid.com&utm_medium=email",
"ip": "192.168.1.1",
"event": "click",
"keen": {
"timestamp": "2018-06-21T21:57:35.000Z",
"created_at": "2018-06-21T21:58:20.374Z",
"id": "555c1f7c5asdf7000167d87b"
},
"url_offset": {
"index": 29,
"type": "text"
},
"sg_message_id": "F5mwV1rESdyKFA_2bn1IEQ.filter0042p3las1-15933-5B2A68E8-36.0",
"useragent": "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)",
"email": "no.name#noname.com"
}
]
}
My questions are:
Is there a way to parse the JSON result without use of ImportJSON, which has to be entered as a custom function in a cell that also depends on recalculation? ImportJSON sometimes doesn't work properly.
How can this code be refactored or optimized so that it can always return data to PDATA sheet columns?
Is there a better way of accomplishing what i want without resorting to custom functions like GCL in the PDATA Sheet or ImportJSON in DATA sheet?
How about this sample script? This script parses the values retrieved from API using UrlFetchApp and put them to the sheet "DATA". You can run this at the menu of spreadsheet. Before you run this, please put the endpoint.
Sample script :
function onOpen() {
var ui = SpreadsheetApp.getUi();
// Or DocumentApp or FormApp.
ui.createMenu('IMPORT DATA')
.addItem('KEENIO DATA', 'ReCalcCell')
.addItem('main', 'main')
.addToUi();
}
function main() {
var url = "###"; // Please put the endpoint with your token.
var res = UrlFetchApp.fetch(url).getContentText(); // Modified
var values = JSON.parse(res);
var putData = values.result.map(function(e) {return [e.useragent, e.sg_event_id, e.timestamp, e.ip, e.url, e.event, e.keen.timestamp, e.keen.created_at, e.keen.id, e.url_offset.index, e.url_offset.type, e.sg_message_id, e.email]});
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("DATA");
sheet.getRange(sheet.getLastRow() + 1, 1, putData.length, putData[0].length).setValues(putData);
}
Note :
When you use this, please put the endpoint including your token to url.
I confirmed this script using the JSON object in your question. So if the structure of the object is changed, it is required to also modify the script. Please be careful this.
Reference :
UrlFetchApp.fetch()
If I misunderstand about your issue, please tell me. I would like to modify it.
Edit 1 :
Pattern 1 :
var putData = values.result.map(function(e) {return [e.useragent, e.sg_event_id, e.timestamp, e.ip, e.url, e.event, e.keen.timestamp, e.keen.created_at, e.keen.id, JSON.parse(e["url_offset"]).index, JSON.parse(e["url_offset"]).type, e.sg_message_id, e.email]});
Pattern 2 :
var putData = values.result.map(function(e) {return [e.useragent, e.sg_event_id, e.timestamp, e.ip, e.url, e.event, e.keen.timestamp, e.keen.created_at, e.keen.id, e["url_offset"].index, e["url_offset"].type, e.sg_message_id, e.email]});
Edit 2 :
Could you please run this script and provide the values of the created file? Of course, please remove the personal information. But please don't modify the structure of the object. If you cannot do it, I would like to think of other ways.
var url = "###"; // Please put the endpoint with your token.
var res = UrlFetchApp.fetch(url).getContentText();
DriveApp.createFile("sample.txt", res, MimeType.PLAIN_TEXT)
Edit 3 :
Please copy and paste this script in your script editor, run myFunction(). Then, please show the values of file. When you run this function, please confirm whether there are NOT the same function name in your project.
function myFunction() {
var url = "###"; // Please put the endpoint with your token.
var res = UrlFetchApp.fetch(url).getContentText();
DriveApp.createFile("sample.txt", res, MimeType.PLAIN_TEXT)
}
Edit 4 :
Please copy and paste this script in your script editor, run myFunction2(). Then, please show the results. When you run this function, please confirm whether there are NOT the same function name in your project.
Please confirm whether the keys and values of keen and url_offset are retrieved.
function myFunction2() {
var url = "###";
var res = UrlFetchApp.fetch(url).getContentText();
var values = JSON.parse(res);
for (var key in values.result[0]) {
Logger.log("key: %s, value: %s", key, values.result[0][key])
if (typeof values.result[0][key] == "object") {
for (var dkey in values.result[0][key]) {
Logger.log("key: %s, dkey: %s, value: %s", key, dkey, values.result[0][key][dkey])
}
}
}
}
Edit 5 :
Please copy and paste this script in your script editor, run myFunction3(). Then, please show the results. When you run this function, please confirm whether there are NOT the same function name in your project.
function myFunction3() {
var url = "###"; // Please set this.
var res = UrlFetchApp.fetch(url).getContentText();
var values = JSON.parse(res);
var obj = [];
for (var i = 0; i < values.result.length; i++) {
var temp = {};
var v = values.result[i];
for (var key in v) {
temp[key.replace(/_/g, "")] = v[key];
if (typeof v[key] == "object") {
for (var dkey in v[key]) {
temp[key.replace(/_/g, "") + dkey.replace(/_/g, "")] = v[key][dkey];
}
}
}
obj.push(temp);
}
var putData = obj.map(function(e) {return [e.useragent, e.sgeventid, e.timestamp, e.ip, e.url, e.event, e.keentimestamp, e.keencreatedat, e.keenid, e.urloffsetindex, e.urloffsettype, e.sgmessageid, e.email]});
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("DATA");
sheet.getRange(sheet.getLastRow() + 1, 1, putData.length, putData[0].length).setValues(putData);
}
Looking at what you are doing here, it might be much easier to design your spreadsheet in an "append-only" format with a Zapier integration.
Zapier is able to handle SendGrid events directly, and append those events to your spreadsheet, if that is what you want.
And then you can have your "calculation columns" on a separate Sheet in the spreadsheet.
Just an idea.
I have the following script to create simple pivot table. It is a working script. However, I am struggle to find sample codes to create a filter. Would appreciate if somebody can show me sample script to create a pivot table filter.
var sheetName = "Sheet1";
var pivotTableParams = {};
pivotTableParams.source = {
sheetId: ss.getSheetByName(sheetName).getSheetId()
};
pivotTableParams.rows = [{
sourceColumnOffset: 8,
"showTotals": true,
sortOrder: "ASCENDING"
}];
pivotTableParams.values = [{
summarizeFunction: "SUM",
sourceColumnOffset: 10
}];
var sheet = ss.getSheetByName('Sheet2');
ss.deleteSheet(sheet);
var pivotTableSheet = ss.insertSheet('Sheet2');
var pivotTableSheetId = pivotTableSheet.getSheetId();
var request = {
"updateCells": {
"rows": {
"values": [{
"pivotTable": pivotTableParams
}]
},
"start": {
"sheetId": pivotTableSheetId
},
"fields": "pivotTable"
}
};
The documentation could use an example.
In the context of your code:
pivotTableParams.criteria = {
8: {visibleValues: ["asdf", "fdsa"]},
9: {visibleValues: ["sdaf", "fads"]},
//and so on
};
Where the keys 8 and 9 above reference the columns in the pivot's source. Similar to how sourceColumnOffset references them.
The criteria field is documented here: Pivot Tables | Sheets API | Google Developers
The PivotFilterCriteria type is documented here: PivotFilterCriteria | Sheets API | Google Developers
From my investigation, I believe that visibleValues is the only key that you can pass and so can't filter by exclusion as far as I know.
In case anyone comes across this thread trying to find an example of a working script with more detailed params being passed to the filter, here's a working example of a filter that removes all blank values and defaults all other values to being shown:
// Filter blank cells
var condition = new Boolean();
condition.type = "NOT_BLANK";
pivotTableParams.criteria = { 0: {"condition": condition, "visibleByDefault": true}};
In this example, the "0" is the column index on the pivot table.
Here's the docs from the Google Sheets API, which are unfortunately short on examples so it can take a lot of trial and error to get things working:
https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets/pivot-tables#PivotFilterCriteria
If somebody it´s trying to call the Sheets API in Apps Script, yo can add a filter to your pivot table objet with the following code
var filter1 = Sheets.newPivotFilterCriteria()
filter1.visibleValues = ['Sí']
var spec1 = Sheets.newPivotFilterSpec()
spec1.filterCriteria = filter1
spec1.columnOffsetIndex = 0
var filter2 = Sheets.newPivotFilterCriteria()
filter2.visibleValues = dates
var spec2 = Sheets.newPivotFilterSpec()
spec2.filterCriteria = filter2
spec2.columnOffsetIndex = 10
var filter3 = Sheets.newPivotFilterCriteria()
filter3.visibleValues = ['Negocio']
var spec3 = Sheets.newPivotFilterSpec()
spec3.filterCriteria = filter3
spec3.columnOffsetIndex = 23
pivotTable.filterSpecs = [spec1,spec2,spec3]
This is my first post here and I am new to coding. I have been tasked with creating an automated report which will send a google form submitter a graph to help them monitor their production versus their daily goal. To do this I am using the new developer Google sheets script to refresh a pivot table. I found this code online, and it works great, however, I want to add a line which will filter based to the unique submitter's data. Here is the code I have so far:
function updatePivotTable() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var pivotTableSheetName = "Lunch Chart";
var pivotTableSheetId = ss.getSheetByName(pivotTableSheetName).getSheetId();
var fields = "sheets(properties.sheetId,data.rowData.values.pivotTable)";
var sheets = Sheets.Spreadsheets.get(ss.getId(), {fields: fields}).sheets;
for (var i in sheets) {
if (sheets[i].properties.sheetId == pivotTableSheetId) {
var pivotTableParams = sheets[i].data[0].rowData[0].values[0].pivotTable;
break;
}
}
// Update source range:
pivotTableParams.source.endRowIndex = 40;
// Send back the updated params
var request = {
"updateCells": {
"rows": {
"values": [{
"pivotTable": pivotTableParams
}]
},
"start": {
"sheetId": pivotTableSheetId
},
"fields": "pivotTable"
}
};
Sheets.Spreadsheets.batchUpdate({'requests': [request]}, ss.getId());
}
Is this possible? Where would I add in the filter piece? I found this on the google developer site, but I am very new to coding so I don't really know where to put it or how to make it conditional. https://developers.google.com/sheets/api/reference/rest/v4/FilterCriteria
Thank you!
I am not sure if this is still relevant to you but you can add a filter using following piece of code-
"criteria": {
<col_index>: {"visibleValues": <filter criteria>},
<col_index>: {"visibleValues": <filter criteria>},
I've had the same problem and didn't find an easy explanation or code. Here is what I've done and it works:
function updatePivotTable() {
var ss = SpreadsheetApp.openById(SHEET_ID);
var pivotTableSheetName = "Pivot";
var pivotTableSheetId = ss.getSheetByName(pivotTableSheetName).getSheetId();
var fields = "sheets(properties.sheetId,data.rowData.values.pivotTable)";
var sheets = Sheets.Spreadsheets.get(ss.getId(), {fields: fields}).sheets;
for (var i in sheets) {
if (sheets[i].properties.sheetId == pivotTableSheetId) {
var pivotTableParams = sheets[i].data[0].rowData[0].values[0].pivotTable;
break;
}
}
// Update source range:
pivotTableParams.source.endRowIndex = 111;
pivotTableParams.criteria = { 3: {"visibleValues": ["foo"]}};
// Send back the updated params
var request = {
"updateCells": {
"rows": {
"values": [{
"pivotTable": pivotTableParams
}]
},
"start": {
"sheetId": pivotTableSheetId
},
"fields": "pivotTable",
}
};
Sheets.Spreadsheets.batchUpdate({'requests': [request]}, ss.getId());
}
So basically you need to pass the criteria as a pivotTableParams where the key of the object is the index of the column that you want to query and the value should be passed as another object with the format {"visibleValues": ["foo"]}.
Trying to write all users.organizations.name (title, primary, type, customType, etc) via GAS Directory API to a Google Sheet. Script below separates fullName, primaryEmail, & id into separate columns but writes organizations into a single column. I'd like to spilt organizations values up into separate columns.
function getAllUsers2(){
var values = [],
users = [],
userListQuery = {},
nextPageToken = '',
listObject = {
domain:'hhs1.com',
maxResults: 500,
projection: 'full',
customerId: 'my_customer'
},
i = 0;
do {
if (nextPageToken && nextPageToken !== '') {
listObject.pageToken = nextPageToken;
}
userListQuery = AdminDirectory.Users.list(listObject);
// if there are more users than fit in the query a nextPageToken is returned
nextPageToken = userListQuery.nextPageToken;
// Add the query results to the users array
users = users.concat(userListQuery.users);
} while (nextPageToken);
for (i = 0; i < 5; i += 1) {
values.push([users[i].name.fullName,
users[i].primaryEmail,
users[i].id,
users[i].organizations]);
}
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheetName = 'USERS2';
var header = ["Name", "Email","ID","Organizations"]
sheetExists(sheetName);
var sheet = ss.getSheetByName(sheetName);
sheet.getRange(1, 1, 1, header.length).setValues([header]); //Note: I had to add brackets around header for this to work. Was getting "Cannot convert Array to Object[][]" before I did so
sheet.getRange(2, 1, values.length, values[0].length).setValues(values);
Logger.log(values);
}
Here's what is added in column D after running:
partial product from script
Here is part of the USERS JSON Template for reference:
"organizations": [
{
"name": string,
"title": string,
"primary": boolean,
"type": string,
"customType": string,
"department": string,
"symbol": string,
"location": string,
"description": string,
"domain": string,
"costCenter": string
}
],
Thank you!