I have this function for extracting the timestamp from two JSON objects:
lineReader.on('line', function (line) {
var obj = JSON.parse(line);
if(obj.Event == "SparkListenerApplicationStart" || obj.Event == "SparkListenerApplicationEnd") {
console.log('Line from file:', obj.Timestamp);
}
});
The JSON comes from a log file(not JSON) where each line represents an entry in the log and each line also happens to be in JSON format on its own.
The two objects represent the start and finish of a job. These can be identified by the event key(SparkListenerApplicationStart and SparkListenerApplicationEnd). They also both contain a timestamp key. I want to subtract the end time from the start time to get the duration.
My thinking is to assign the timestamp from the JSON where Event key = SparkListenerApplicationStart to one variable and assign the timestamp from the JSON where Event key = SparkListenerApplicationEnd to another variable and subtract one from the other. How can I do this? I know I can't simply do anything like:
var startTime = if(obj.Event == "SparkListenerApplicationStart"){
return obj.Timestamp;
}
I'm not sure if I understood, but if are reading rows and want get the Timestamp of each row I would re-write a new object;
const collection = []
lineReader.on('line', function (line) {
var obj = JSON.parse(line);
if(obj.Event == "SparkListenerApplicationStart" || obj.Event == "SparkListenerApplicationEnd") {
// console.log('Line from file:', obj.Timestamp);
collection.push(obj.Timestamp)
}
});
console.log(collection);
Where collection could be a LocalStorage, Global Variable, or something alike.
Additional info
With regard to my comment where I queried how to identify the start and end times, I ended up setting start as the smallest value and end as the largest. Here is my final code:
const collection = []
lineReader.on('line', function (line) {
var obj = JSON.parse(line);
if((obj.Event == "SparkListenerApplicationStart" || obj.Event == "SparkListenerApplicationEnd")) {
collection.push(obj.Timestamp);
if(collection.length == 2){
startTime = Math.min.apply(null, collection);
finishTime = Math.max.apply(null, collection);
duration = finishTime - startTime;
console.log(duration);
}
}
});
Related
How do you write a filter function in GAS that, rather than returning values of a dataset that pass the condition, returns the instance numbers of the data that passes the condition?
For example, let's say our condition is divisibility by 10.
(value % 10 = 0)
And our dataset is
[1,5,10,20,7,40]
Items # 0, 1, and 4 fail the condition; items # 2, 3, and 5 pass.
Desired result:
[2,3,5]
EDIT: Cooper's answer below solves the problem as a single instance console command. In my Comments I have parsed his answer as a function that can take the array and condition as variables. For sake of better readability, I'll post that here:
function indicesOfEntriesPassingCondition (array,condition) {
return array.map((value, i) => (condition(value))? i:'').filter(value => value !== '')
}
function testIndicesOfEntriesPassingCondition () {
var condition = value => (value % 10 == 0);
var array = [1,5,10,20,7,40]
var results = indicesOfEntriesPassingCondition (array,condition);
console.log(results)
}
function lfunko() {
Logger.log([1,5,10,20,7,40].map((e,i) => (e % 10 == 0)? i: '').filter(e => e !== '').join(','))
}
Execution log
4:42:39 PM Notice Execution started
4:42:40 PM Info 2,3,5
4:42:40 PM Notice Execution completed
Array.map
Alternate Solution
Try this code using simple JavaScript loop and indexOf:
function indexOfDivisibleByTen(){
var arr = [1,5,10,20,7,40];
var numberIndex = [];
//loop through array to find those divisible by 10
for (i = 0; i < arr.length; i++){
if (arr[i] % 10 ==0){
//enter the index of the value in array instead of the actual value
numberIndex.push(arr.indexOf(arr[i]));
}
}
console.log(numberIndex);
};
Works the same as finding the value in the array but instead of entering the actual value into a new array, you enter the index or the position of the value from the array.
Result:
References:
JavaScript Looping
JavaScript Array indexOf
On selecting any date and hitting enter an API call should be made. And there's a x icon in the input on clicking it, it should call the API with date 01/01/12 Also this has feature like if you type 2/3 and hit enter it will automatically make it 02/03/20. The problem is if the input is empty and if I hit Enter same API calls are made thrice.
But the feature should be like if you select a date then without hitting Enter an API call should be made. I can't just use change function because if 2/3 is typed and Tab is pressed then it will not adjust the date automatically and also multiple API calls on hitting Enter. Is there a way to stop multiple API calls?
(change)="startDate($event)" (keydown.enter)="CallAPI($event)"
startDate(event) {
if (event.target.value == '' || event.target.value == null)
this.cutoverFilterApi(event)
}
CallAPI(event) {
let data = event.target.value;
if (data != '' && data != null && data != "NaN/NaN/NaN") {
data = data;
} else {
data = "01/01/12";
}
this.httpService.getData('PATH' + data).subscribe((response: any) => {
this.dateChangeData = response.results;
this.rowData = response.results;
this.gridApi.setRowData(this.rowData);
});
}
You could keep the last valid value and avoid request if it is the same.
Something like this,
lastDate = null; // <- variable to keep last value
CallAPI(event) {
let data = event.target.value;
if (data != '' && data != null && data != "NaN/NaN/NaN") {
data = data;
} else {
data = "01/01/12";
}
// check if data is not the same as last request
if (this.lastDate === data) {
return;
}
this.lastDate = data; // <- update new request date
this.httpService.getData('PATH' + data).subscribe((response: any) => {
this.dateChangeData = response.results;
this.rowData = response.results;
this.gridApi.setRowData(this.rowData);
});
}
You can use this
(dateInput)="addEvent('input', $event)" (dateChange)="addEvent('change', $event)"
instead of
(change)="startDate($event)" (keydown.enter)="CallAPI($event)"
I have an example of angular material datepicker, which will make your code easier.
Reference link
I hope it is helpful for you. :)
So, I aim to pass 2 states into the following function and have it determine whether the states are identical, or are neighbors. If they are, I want the function to return true, and if not, false.
function shouldWeRun(origin, destination) {
var shouldWeRun = false;
switch (origin) {
case destination:
shouldWeRun = true;
case "Massachusetts":
if ( destination == "Connecticut" ||
destination == "New Hampshire" ||
destination == "New York" ||
destination == "Rhode Island" ||
destination == "Vermont" ) { shouldWeRun = true; };
break;
default:
shouldWeRun = false;
};
return shouldWeRun;
};
When I declare states as an array of strings, like this...
var states = ["Massachusetts","Massachusetts","Connecticut","Virginia"];
...and run this:
Logger.log("same state should return true: " + shouldWeRun(states[0],states[1]));
Logger.log("state neighbors should return true: " + shouldWeRun(states[0],states[2]));
Logger.log("non-neighbor states should return false: " + shouldWeRun(states[0],states[3]));
...the function works as advertised.
However, here's the problem: if I instead get the states values from the spreadsheet (how I need to) like this...
var states = sheet.getRange("H2:H31").getValues();
...the function always returns the default case and therefore, false.
Assume that every cell in the range H2:H31 is "Massachusetts".
The object returned by sheet.getRange("H2:H31").getValues() is always a 2D array, ie an array of arrays, even when it is a single column.
You are comparing strings with arrays, that's why the result is always false.
You should "flatten" that 2D array to get a simple one.
You can do this simply with a small code like this :
var states = sheet.getRange("H2:H31").getValues().join().split();
I am using the following functions on a library and then calling them like this. The issue with the code is that I am not able return the values from the code below:
Would be great if some one suggests a way to return the value back to my test. (I will post the full working code once this is solved). I have not worked with promises so if some one can suggest a solution that be great!
Resolved this!!! check my answer:
My Testcase
iit("Should Find the OrderID and update task and submit", function () {
var job_id_data= lib.getTestData('MYPROJ_TESTCASE_001'); //Problem area
console.log(job_id_data);
element(by.xpath('//input[#type=\'search\']')).sendKeys(job_id_data);
//Do other stuff
}
The below code in my function (lib) needs to return a promise, and I don't know how to do that :(
csvConverter.on("end_parsed",function(jsonObj){
//console.log(jsonObj); //here is your result json object
var foundTestData = getObjects(jsonObj, 'TC', jobreference);
console.log(returnKeyValue ); //I can see this value
returnKeyValue = getValues(foundTestData, 'JOBID'); // I cannot return this??
});
Full Not working code ...Code
var lib = require('./lib/library.js');
iit("should go to logout page", function () {
var id_data= lib.getTestData('Test.3');
//plan to use this value in my tests
});
//Library
function getTestData(jobreference) {
//Converter Class
var Converter=require("csvtojson").core.Converter;
var fs=require("fs");
var csvFileName="C:\\TestData.csv";
var fileStream=fs.createReadStream(csvFileName);
//new converter instance
var param={};
var csvConverter=new Converter(param);
var returnKeyValue="";
var result = {};
//This requires a code change:
csvConverter.on("end_parsed",function(jsonObj){
//console.log(jsonObj); //here is your result json object
var foundTestData = getObjects(jsonObj, 'TC', jobreference);
console.log(returnKeyValue ); //I can see this value
returnKeyValue = getValues(foundTestData, 'JOBID'); // I cannot return this??
});
//read from file
fileStream.pipe(csvConverter);
return returnKeyValue;
}
function getValues(obj, key) {
var objects = [];
for (var i in obj) {
if (!obj.hasOwnProperty(i)) continue;
if (typeof obj[i] == 'object') {
objects = objects.concat(getValues(obj[i], key));
} else if (i == key) {
objects.push(obj[i]);
}
}
return objects;
}
function getObjects(obj, key, val) {
var objects = [];
for (var i in obj) {
if (!obj.hasOwnProperty(i)) continue;
if (typeof obj[i] == 'object') {
objects = objects.concat(getObjects(obj[i], key, val));
} else
//if key matches and value matches or if key matches and value is not passed (eliminating the case where key matches but passed value does not)
if (i == key && obj[i] == val || i == key && val == '') { //
objects.push(obj);
} else if (obj[i] == val && key == ''){
//only add if the object is not already in the array
if (objects.lastIndexOf(obj) == -1){
objects.push(obj);
}
}
}
return objects;
}
Managed to resolve this :) with some help from my colleague (thanks :))
This post here helped me get quickly to the point
http://know.cujojs.com/tutorials/promises/creating-promises
Solution is I updated the function to the following, which basically works with Protractor Promises. Which is great.
function getTestData(jobreference) {
var Converter=require("csvtojson").core.Converter;
var fs=require("fs");
var csvFileName="TESTJOB.csv";
var fileStream=fs.createReadStream(csvFileName);
var csvConverter=new Converter(param);
//new converter instance
var param={};
var csvConverter=new Converter(param);
var d = protractor.promise.defer();
csvConverter.on("end_parsed",function(jsonObj){
var foundTestData = getObjects(jsonObj, 'TCaseID', jobreference);
returnKeyValue = getValues(foundTestData, 'ID');
console.log(returnKeyValue.toString());
d.fulfill(returnKeyValue.toString());
});
//d.reject("fail!!!!");
fileStream.pipe(csvConverter);
return d.promise;
}
I'm trying to estimate the limits of my current GAS project. I use ScriptDB to chunk out processing to get around the 6 min execution limit. If I have an object like
var userObj{
id: //user email address
count: //integer 1-1000
trigger: //trigger ID
label: //string ~30 char or less
folder: //Google Drive folder ID
sendto: //'true' or 'false'
shareto: //'true' or 'false'
}
How would I calculate the size that this object takes up in the DB? I would like to project how many of these objects can exist concurrently before I reach the 200MB limit for our domain.
Whenever you've got a question about google-apps-script that isn't about the API, try searching for javascript questions first. In this case, I found JavaScript object size, and tried out the accepted answer in apps-script. (Actually, the "improved" accepted answer.) I've made no changes at all, but have reproduced it here with a test function so you can just cut & paste to try it out.
Here's what I got with the test stud object, in the debugger.
Now, it's not perfect - for instance, it doesn't factor in the size of the keys you'll use in ScriptDB. Another answer took a stab at that. But since your object contains some potentially huge values, such as an email address which can be 256 characters long, the key lengths may be of little concern.
// https://stackoverflow.com/questions/1248302/javascript-object-size/11900218#11900218
function roughSizeOfObject( object ) {
var objectList = [];
var stack = [ object ];
var bytes = 0;
while ( stack.length ) {
var value = stack.pop();
if ( typeof value === 'boolean' ) {
bytes += 4;
}
else if ( typeof value === 'string' ) {
bytes += value.length * 2;
}
else if ( typeof value === 'number' ) {
bytes += 8;
}
else if
(
typeof value === 'object'
&& objectList.indexOf( value ) === -1
)
{
objectList.push( value );
for( i in value ) {
stack.push( value[ i ] );
}
}
}
return bytes;
}
function Marks()
{
this.maxMarks = 100;
}
function Student()
{
this.firstName = "firstName";
this.lastName = "lastName";
this.marks = new Marks();
}
function test () {
var stud = new Student();
var studSize = roughSizeOfObject(stud);
debugger;
}