I have nested JSON like -
"disks" : [ {
"name" : "v2.16",
"diskAggregate" : "aggr0",
"diskRPM" : 15000,
"totalSizeBytes" : 1077477376,
"vendorId" : "NETAPP ",
"usedBytes" : 1070071808,
"diskType" : "FCAL",
"uuid" : "4E455441:50502020:56442D31:3030304D:422D465A:2D353230:32353836:30303030:00000000:00000000",
"portName" : "FC:A ",
"raidGroup" : "rg0"
},
{
"name" : "v4.16",
"diskAggregate" : "aggr0",
"diskRPM" : 15000,
"totalSizeBytes" : 1077477376,
"vendorId" : "NETAPP ",
"usedBytes" : 1070071808,
"diskType" : "FCAL",
"uuid" : "4E455441:50502020:56442D31:3030304D:422D465A:2D353230:32353633:34333030:00000000:00000000",
"portName" : "FC:B ",
"raidGroup" : "rg0"
}]
I want to get addition 'totalSizeBytes' from above list of objects.
I used following code to get it -
val storageDevices = "above given json".toList
val totalCapacity = storageDevices.foldLeft(0) {
case (sumOfAllDevices, storageDevice) =>
val sumOfTotalBytesOnStorageDevice = storageDevice.disks.foldLeft(0) {
case (totalBytesOnDevice, disk) =>
totalBytesOnDevice + disk.usedBytes.getOrElse(0).toString.toInt
}
sumOfAllDevices + sumOfTotalBytesOnStorageDevice
// Logger.info("dss"+sumOfTotalBytesOnStorageDevice.toString.toInt)
}
This code gives me total capacity in Integer format. But as there are too many objects in disks array, the totalCapacity will get exceed int. So I wanted to convert it to Long while doing addition.
I want following output-
"totalCapacity": [
{
"name": "192.168.20.22",
"y": 123456789
}
]
How do I convert it to Long to get exact sum of all 'totalBytesAvailable' from array/list???
Cast zero values as 0L (by default assumed Int), both in foldLeft(0L) and in getOrElse(0L), so the compiler will enforce arithmetic additions on Long.
Related
How do I access the JSON array to display the output of "AdjustedScheduleTime" from the Trip section?
I got it working for StopLabel as shown below, but I'm struggling to access AdjustedScheduleTime.
I tried the following:
["GetNextTripsForStopResponse"]["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"]["AdjustedScheduleTime"]
but doesn't work.
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
let parameters = [
"appID": "5rt5rydg", //incorrect appID
"apiKey": "3b5fb15rdgy5454hdrfhr", //incorrect apiKey
"routeNo": "14",
"stopNo": "8600",
"format": "JSON"
]
AF.request("https://api.octranspo1.com/v1.2/GetNextTripsForStop?", method: .post, parameters: parameters,encoding:
URLEncoding.httpBody, headers: nil).responseJSON{ response in
let swiftyJsonVar = JSON(response.result.value!)
print(swiftyJsonVar)
if let busInfo = swiftyJsonVar["GetNextTripsForStopResult"]["StopLabel"].string {
print(": ",busInfo)
print("Label1: ", self.label1.text = busInfo)
}
}
}
This is the results:
{
"GetNextTripsForStopResult" : {
"Error" : "",
"Route" : {
"RouteDirection" : {
"RouteLabel" : "St-Laurent",
"Error" : "",
"RequestProcessingTime" : "20190112151425",
"Trips" : {
"Trip" : [
{
"AdjustmentAge" : "0.38",
"GPSSpeed" : "0.5",
"Latitude" : "45.429457",
"Longitude" : "-75.684117",
"TripDestination" : "St-Laurent",
"LastTripOfSchedule" : false,
"TripStartTime" : "14:31",
"BusType" : "4LB - IN",
"AdjustedScheduleTime" : "11"
},
{
"AdjustmentAge" : "4.32",
"GPSSpeed" : "0.5",
"Latitude" : "45.413749",
"Longitude" : "-75.689748",
"TripDestination" : "St-Laurent",
"LastTripOfSchedule" : false,
"TripStartTime" : "14:46",
"BusType" : "4LB - IN",
"AdjustedScheduleTime" : "22"
},
{
"AdjustmentAge" : "0.55",
"GPSSpeed" : "31.3",
"Latitude" : "45.399587",
"Longitude" : "-75.727631",
"TripDestination" : "St-Laurent",
"LastTripOfSchedule" : false,
"TripStartTime" : "15:01",
"BusType" : "4L - IN",
"AdjustedScheduleTime" : "37"
}
]
},
"RouteNo" : 14,
"Direction" : "Eastbound"
}
},
"StopLabel" : "MCARTHUR \/ IRWIN MILLER",
"StopNo" : "8600"
}
}
: MCARTHUR / IRWIN MILLER //This is the desired output for StopLabel
Ok, so do you explain JSON. Here's a shot.
First some rules:
When you see opening { it means dictionary, you have to pick a key next
When you see opening [ it means array. you have to pick an index
When you see "SomeString": its a key in an array.
Dictionaries have keys, arrays have index. Pick accordingly..
So when we walk through this response:
We see that we start with {. We have a dictionary! We're expecting to see some keys next.
So lets pick a key: We only have one and it's "GetNextTripsForStopResult". so far we have: swiftyJsonVar["GetNextTripsForStopResult"]
We now look at the content of "GetNextTripsForStopResult". We see it's also a dictionary. Again we should have some keys. We do. We have Error, Route, StopLabel and more. Let's pick a key. Since we're trying to get to a "AdjustedScheduleTime", lets pick Route. so far we have ["GetNextTripsForStopResult"]["Route"]
Now lets look at the contents of Route. Its a dictionary again.
Again we pick a key and keep repeating till we hit Trip. You should have ["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"]
Lets look at what we have in Trip Whats this?..its an array!
We have to pick an index now. We need to chose somehow. Thats the tricky part. In order to do that we need some more information. So lets just ARBITRARILY chose one. Lets take the last one. so we have: ["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"][2]
Now we can get our final key AdjustedScheduleTime. So let's pick it!
["GetNextTripsForStopResult"]["Route"]["RouteDirection"]["Trips"]["Trip"][2]["AdjustedScheduleTime"]
Keep in mind:
These hard coded indexes are almost NEVER what you want. Maybe you need to show all the AdjustedScheduleTime to the user or let the user chose one, or add all of them up. That really depends on your application and what you're trying to accomplish. I chose the last index (2) arbitrarily without having any knowledge of your application, the api you're calling and what you're trying to achieve. Its VERY possible that you don't want the last index.
Below is the JSON response I receive when I am hitting a particular web service:
[
{
"sId" : "0001",
"sName" : "abc1",
"sPlace" : "abc11"
}, {
"sId" : "0002",
"sName" : "abc2",
"sPlace" : "abc12"
}, {
"sId" : "0003",
"sName" : "abc3",
"sPlace" : "abc13"
}, {
"sId" : "0004",
"sName" : "abc4",
"sPlace" : "abc14"
}
]
I don't know which index has my expected values (I need to validate multiple values after identifying which has sId == '0003'), this is dynamic. Don't want to user hard coded value.
And match response.[3].sId == '0003'
because this will be changed next time.
I have two questions regarding this:
How can I pass response to java code and get the array index which having sId == '0003' so that I can use this index to validate?
How can I pass a variable value as an array index in response?
The code below is not working.
def ind = Java.type('karate.Utility.FindIndex')
response.['#ind'].sId == '0003'
karate uses json-path which allows writing conditions to read data from JSON.
example:
* def sId = "0003"
* def sValue = karate.jsonPath(response, "$[?(#.sId == '" + sId + "')]")
* match sValue[0] == {"sId" : "0003","sName" : "abc3","sPlace" : "abc13"}
now if there is a match in sId on the response JSON array, all such matches will be returned.
No need to do * match sValue[0].sId == "0003" as this is your filter
criteria
More about JSON path
online JSON path evaluator
karate doc refernce
i 'm actually working on a stream, receiving a bunch of strings and need to make a count of all the strings. the sums is aggragated, that mean for the second record the sum was added to the day before
the output must be some json file looking like
{
"aggregationType" : "day",
"days before" : 2,
"aggregates" : [
{"date" : "2018-03-03",
"sum" : 120},
{"date" :"2018-03-04",
"sum" : 203}
]
}
i created a stream looking like :
val eventStream : DataStream [String] =
eventStream
.addSource(source)
.keyBy("")
.TimeWindow(Time.days(1), Time.days(1))
.trigger(new MyTriggerFunc)
.aggregation(new MyAggregationFunc)
.addSink(sink)
thank you in advance for the help :)
Note on working with JSON in Flink:
Use JSONDeserializationSchema to deserialize the events, which will produce ObjectNodes. You can map the ObjectNode to YourObject for convenience or continue working with the ObjectNode.
Tutorial on working with ObjectNode: http://www.baeldung.com/jackson-json-node-tree-model
Back to your case, you can do it like the following:
val eventStream : DataStream [ObjectNode] =
oneMinuteAgg
.addSource(source)
.windowAll()
.TimeWindow(Time.minutes(1))
.trigger(new MyTriggerFunc)
.aggregation(new MyAggregationFunc)
will output a stream of 1min aggregates
[
{
"date" :2018-03-03
"sum" : 120
},
{
"date" :2018-03-03
"sum" : 120
}
]
then chain another operator to the "oneMinuteAgg" that will add the 1min aggregates into 1day aggregates:
[...]
oneMinuteAgg
.windowAll()
.TimeWindow(Time.days(1))
.trigger(new Whatever)
.aggregation(new YourDayAggF)
that will output what you need
{
"aggregationType" : "day"
"days before" : 4
"aggregates : [{
"date" :2018-03-03
"sum" : 120
},
{
"date" :2018-03-03
"sum" : 120
}]
}
I used windowAll() assuming you don't need to key the stream.
This is the structure of a document I have in one monggodb collection.
I wanted to understand how one can do a mongo aggregate of grouped count over key "code" and the index position in the nested json (not the priority as it can be any number but within schedules nested there can be just 5 values):
{
"_id" : ObjectId("5749e9fde4b0064e7362b560"),
"_class" : "com.weirdcompanyname.core.collectionname",
"rfId" : 1,
"scheduleds" : [
{
"code" : "556e4835f1eae40bdfa2f2001f2afc76",
"type" : "HT",
"priority" : 0
},
{
"code" : "8b2ab67af4f60e42f7ea64813b5795cf",
"type" : "HT",
"priority" : 1
},
{
"code" : "ed17101eb918b4d8c7c598e4884523ea",
"type" : "HT",
"priority" : 2
},
{
"code" : "7e0ffb4db",
"type" : "QZ",
"priority" : 3
},
{
"code" : "1453dfa1794f39b05f0259ad04699073",
"type" : "HT",
"priority" : 4
}
],
"created" : ISODate("2016-05-28T18:57:00.878Z")
}
The result I'm trying to find is:
code index_position count
556e4835f1eae40bdfa2f2001f2afc76 0 100
8b2ab67af4f60e42f7ea64813b5795cf 1 100
ed17101eb918b4d8c7c598e4884523ea 2 100
7e0ffb4db 3 100
1453dfa1794f39b05f0259ad04699073 4 100
I could get my head around unwinding the nested json in single arrays and then grouping the code over code and maybe other column, let's say priority and have the count but the problem is to get the index position.
Is this even doable on mongo, I've read around a lot of stuff about it and I figured if I have value for which I need a position then it can be doable but I don't really have a value to look for, what I'm looking for is each code and its index position in the "scheduleds" and count.
This is what I could do with my limited mongo querying skills:
db.collectionname.aggregate([{'$match':{'date_key':{'$gte': yesterday_beginning, '$lte': yesterday_end}}}, {'$unwind':'$scheduleds'}, {'$group':{'_id':{'code':'$scheduleds.code','priority':'$scheduleds.priority'}, 'rfid':{'$addToSet':'$rfId'}}}, {'$project':{'_id':0, 'code':'$_id.code', 'priority':'$_id.priority', 'totalRfid':{'$size':'$rfid'}}}, { $limit : 1000 }],{ allowDiskUse:true})
Alain1405 says here that MongoDB 3.2 supports unwinding of the array index.
Instead of passing a path the $unwind operator, you can pass an
object with the field path and the field includeArrayIndex which
will hold the array index.
From MongoDB official documentation:
{
$unwind:
{
path: <field path>,
includeArrayIndex: <string>,
preserveNullAndEmptyArrays: <boolean>
}
}
I need to retrieve information from a JsValue that may be structured in a few different ways buy the specific values I'm looking for will always be under similar names.
So for example I could have something like:
{
"name" : "Watership Down",
"location" : {
"lat" : 51.235685,
"long" : -1.309197
}
}
OR
{
"title" : "Watership Down",
"size" : "M",
"location" : {
"latitude" : 51.235685,
"longitude" : -1.309197
}
}
and I'd want to be able to do: val text = json \\ "name"|"title"
I know what I want is either under name or title but not sure which in a given scenario. Is there any way to do something similar to what I did above with an "or" similar to Scala's .getOrElse() method?
You can do:
val text = (json \\ "name").asOpt[String]
.getOrElse((json \\ "title").as[String])