beginner with js here. I'm trying to parse a json string with node6. The interesting bit of json goes like this:
{
"Metadata" : {
"AWS::CloudFormation::Interface" : {
"ParameterGroups" : [
{
"Label" : {
"default": "Group1"
},
"Parameters" : [
"One",
"Two"
]
},
{
"Label" : {
"default": "Group2"
},
"Parameters" : [
"Three"
]
}
]
}
}
}
I'm trying to list all Parameters (One, Two, Three), but I cannot get through "AWS::CloudFormation::Interface". Accessing AWS::CloudFormation::Interface.ParameterGroups fails, and trying to walk AWS::CloudFormation::Interface subtree
for ( a in Metadata ) {
for ( b in a ) {}
}
get's me an array of single characters.
thanks.
Related
I'm having some problem to write a query to return a triple nested value from a document. The documents I'm using are structured like this
{
"areaname": "name1",
"places": [
{
"placename": "place1",
"objects": [
{
"objname": "obj1",
"tags": [
"tag1",
"tag2"
]
},
{
"objname": "obj2",
"tags": [
"tag6",
"tag7"
]
}
]
},
{
"placename": "place2",
"objects": [
{
"objname": "obj45",
"tags": [
"tag46",
"tag34"
]
},
{
"objname": "obj77",
"tags": [
"tag56",
"tag11"
]
}
]
}
]
}
It is quite simple actually but I can't find a solution to a simple query like:
"return the objname of the object that contains tag1 inside their tag"
So for the give document if I use "tag1" as a parameter it is expected for the query to return "obj1"
It should give me the same result if I use "tag2" as a parameter
Other example: using "tag56" it should return only "obj77"
Right now i have no problem returning the whole document using the dot-notation or top level field such as areaname or others
db.users.find( {"places.objects.tags":"tag1"}, { areaname: 1, _id:0 } )
Is this even possible?
Keeping it simple:
[
{
"$match" : {
"places.objects.tags" : "tag1"
}
},
{
"$unwind" : "$places"
},
{
"$unwind" : "$places.objects"
},
{
"$match" : {
"places.objects.tags" : "tag1"
}
},
{
"$group" : {
"_id" : "$_id",
"obj_names" : {
"$push" : "$places.objects.objname"
}
}
}
],
You should add any other fields you want to keep to the group stage,
this can also be done without the double $unwind stage but i choose this for read-ability.
I have a JSON file, with a structure like this:
{
"106" : {
"id54011" : [
{
"partno1" : "16690617"
},
{
"partno2" : "5899180"
}
],
"parts" : [
"0899180",
"16920617"
],
"id5632" : [
{
"partno1" : "090699180"
}
]
},
"560" : {
"id9452" : [
{
"partno2" : "1569855"
}
],
"parts" : [
"03653624",
"15899855"
],
"id578" : [
{
"partno3" : "0366393624"
},
{
"partno4" : "0363213624"
}
]
}
}
I need to split this JSON into multiple files, using this method:
Each JSON file will consist of one object. Using the example file above, I should end up with 000106.json, and 000560.json. (All names, must have 6 digits, so zeros must be added.)
I have tried to use an iteration grouper, in python, and jq, for this, but no luck up to now.
Expected output:
JSON file 1, named 000106.json:
{
"106" : {
"id54011" : [
{
"partno1" : "16690617"
},
{
"partno2" : "5899180"
}
],
"parts" : [
"0899180",
"16920617"
],
"id5632" : [
{
"partno1" : "090699180"
}
]
}
}
JSON file 2, named 000560.json:
{
"560" : {
"id9452" : [
{
"partno2" : "1569855"
}
],
"parts" : [
"03653624",
"15899855"
],
"id578" : [
{
"partno3" : "0366393624"
},
{
"partno4" : "0363213624"
}
]
}
}
Since this question has both jq and awk tags, I'd recoomend using jq and awk as explained here: Split a JSON file into separate files
You can easily pad the key names in jq or awk.
Using MongoDb how do you get back the date, and 3rd "obs" back from below?
{ "data" : [
{ "val" : [
{ "obs" : "2/3/2016"
},
{ "obs" : 41.8599992990494
},
{ "obs" : 41.3111999630928
},
{ "obs" : 5.048
}
]
},
{ "val" : [
{ "obs" : "2/4/2016"
},
{ "obs" : 39.394998550415
},
{ "obs" : 41.8486998975277
},
{ "obs" : NumberInt(0)
}
]
},
{ "val" : [
{ "obs" : "2/5/2016"
},
{ "obs" : NumberInt(0)
},
{ "obs" : 40.2090013027191
},
{ "obs" : 24.2410004138947
},
{ "obs" : 3.629
}
]
}
]
}
Started with this:
db.myColl.find({},{"_id":0, "data.val.obs": 1, })
would like:
["2/3/2016", 41.3111], ["2/4/2016", 41.8486]
Here is how you could do this in MongoDB starting from v 3.4
db.getCollection('test').aggregate([
{
$addFields: {
data: {
$map: {
input: "$data",
as: "item",
in: {$concatArrays: [{$slice: ['$$item.val', 1]}, {$slice: ['$$item.val', 2, 1]}]}
}
}
}
}
]);
So basically I'm using $addFields not to lose other properties of a root document (as you might need them). If you don't need them you can switch to $project.
Example: collection records look like this: {_id: ..., data: [...], data_2: [...]}.
If you run the query as is you'll have 'data' array filtered. But you'll still have data_2 unchanged. If you replace $addFields with $project you'll lose data_2. (or you need to explicitly tell mongo to keep it by passing data_2: true)
Then I'm mapping each element of 'data' array and assign the result back to 'data' array so in fact data property is overridden by filtered array.
To get 1st and 3rd elements I use $slice (each $slice returns an array of one document). And then I join them into a single array by $concatArrays.
This question based on MongoDB,How to retrieve selected items retrieve by selecting multiple condition.It is like IN condition in Mysql
SELECT * FROM venuelist WHERE venueid IN (venueid1, venueid2)
I have attached json data structure that I have used.[Ref: JSON STRUCTUE OF MONGODB ].
As an example, it has a venueList then inside the venue list, It has several attribute venue id and sum of user agents name and total count as value.user agents mean user Os,browser and device information. In this case I used os distribution.In that case i was count linux,ubuntu count on particular venueid.
it is like that,
"sum" : [
{
"name" : "linux",
"value" : 12
},
{
"name" : "ubuntu",
"value" : 4
}
],
Finally I want to get count of all linux user count by selecting venueid list in one find query in MongoDB.
As example, I want to select all count of linux users by conditioning if venue id VID1212 or VID4343
Ref: JSON STRUCTUE OF MONGODB
{
"_id" : ObjectId("57f940c4932a00aba387b0b0"),
"tenantID" : 1,
"date" : "2016-10-09 00:23:56",
"venueList" : [
{
"id" : “VID1212”,
"sum" : [
{
"name" : "linux",
"value" : 12
},
{
"name" : "ubuntu",
"value" : 4
}
],
“ssidList” : [ // this is list of ssid’s in venue
{
"id" : “SSID1212”,
"sum" : [
{
"name" : "linux",
"value" : 8
},
{
"name" : "ubuntu",
"value" : 6
}
],
“macList” : [ // this is mac list inside particular ssid ex: this is mac list inside the SSID1212
{
"id" : “12:12:12:12:12:12”,
"sum" : [
{
"name" : "linux",
"value" : 12
},
{
"name" : "ubuntu",
"value" : 1
}
]
}
]
}
]
},
{
"id" : “VID4343”,
"sum" : [
{
"name" : "linux",
"value" : 2
}
],
"ssidList" : [
{
"id" : “SSID4343”,
"sum" : [
{
"name" : "linux",
"value" : 2
}
],
"macList" : [
{
"id" : “43:43:43:43:43:34”,
"sum" : [
{
"name" : "linux",
"value" : 2
}
]
}
]
}
]
}
]
}
I am using golang as language to manipulation data with mongoldb using mgo.v2 package
expected out put is :
output
linux : 12+2 = 14
ubuntu : 4+0 = 4
Don't consider inner list in venuelist.
You'd need to use the aggregation framework where you would run an aggregation pipeline that first filters the documents in the collection based on
the venueList ids using the $match operator.
The second pipeline would entail flattening the venueList and sum subdocument arrays in order for the data in the documents to be processed further down the pipeline as denormalised entries. The $unwind operator is useful here.
A further filter using $match is necessary after unwinding so that only the documents you want to aggregate are allowed into the next pipeline.
The main pipeline would be the $group operator stage which aggregates the filtered documents to create the desired sums using the accumulator operator $sum. For the desired result, you would need to use a tenary operator like $cond to create the independent count fields since that will feed the number of documents to the $sum expression depending on the name value.
Putting this altogether, consider running the following pipeline:
db.collection.aggregate([
{ "$match": { "venueList.id": { "$in": ["VID1212", "VID4343"] } } },
{ "$unwind": "$venueList" },
{ "$match": { "venueList.id": { "$in": ["VID1212", "VID4343"] } } },
{ "$unwind": "$venueList.sum" },
{
"$group": {
"_id": null,
"linux": {
"$sum": {
"$cond": [
{ "$eq": [ "$venueList.sum.name", "linux" ] },
"$venueList.sum.value", 0
]
}
},
"ubuntu": {
"$sum": {
"$cond": [
{ "$eq": [ "$venueList.sum.name", "ubuntu" ] },
"$venueList.sum.value", 0
]
}
}
}
}
])
For usage with mGo, you can convert the above pipeline using the guidance in http://godoc.org/labix.org/v2/mgo#Collection.Pipe
For a more flexible and better performant alternative which executes much faster than the above, and also takes into consideration unknown values for the sum list, run the alternative pipeline as follows
db.collection.aggregate([
{ "$match": { "venueList.id": { "$in": ["VID1212", "VID4343"] } } },
{ "$unwind": "$venueList" },
{ "$match": { "venueList.id": { "$in": ["VID1212", "VID4343"] } } },
{ "$unwind": "$venueList.sum" },
{
"$group": {
"_id": "$venueList.sum.name",
"count": { "$sum": "$venueList.sum.value" }
}
},
{
"$group": {
"_id": null,
"counts": {
"$push": {
"name": "$_id",
"count": "$count"
}
}
}
}
])
I have the following JSON structure but am wondering if there would be any way to simplify it further. Can 'ingredient' and 'quantity' be removed from all the entries somehow to help reduce it?
var cooking = {
"recipes" : [
{
"name":"pizza",
"ingredients" : [
{
"ingredient" : "cheese",
"quantity" : "100g"
},
{
"ingredient" : "tomato",
"quantity" : "200g"
}
]
},
{
"name":"pizza 2",
"ingredients" : [
{
"ingredient" : "ham",
"quantity" : "300g"
},
{
"ingredient" : "pineapple",
"quantity" : "300g"
}
]
}
]
};
Yes, you can simplify that quite a bit:
var recipes = {
"pizza": {
"cheese": "100g",
"tomato": "200g"
},
"pizza 2": {
"ham": "300g",
"pineapple": "300g"
}
}
An explanation:
The top level of your example is a single-item object: {"recipes": <...>}. Unless this is a simplified version of an object that will actually have other items in it, that's redundant. Your code knows what it's sending/recieving, so there's no extra information there.
The value of your {"recipes": <...>} object is an array of two-item objects, with the keys "name" and "ingredients". Whenever you have an array like this, it makes more sense (and is more compact) to replace it with an object. As a rule of thumb:
If the keys in an array of objects can be replaced by "key" and "value" and still make sense, replace the array with a single {"key_name": <value>, ...} object.
The same rule applies to your [{"ingredient": <...>, "quantity": <...>}, ...] array: each object can be replaced by a key-value pair and continue to make sense.
The end result is that this representation of the information is 87 characters in length (with extraneous whitespace removed), compared to your original's 249 characters - a 65% reduction.
Definitely. One way would be :
var cooking = {
"recipes" : [
{
"name":"pizza",
"ingredients" : [
"cheese",
"tomato"
],
"quantities" : [ // Have to be in order of ingredients
"100g",
"200g"
]
}
]
}
Or
var cooking = {
"recipes" : [
{
"name":"pizza",
"ingredients" : [ // Putting ingredient and quantity together
"cheese:100g",
"tomato:200g"
]
}
]
}
Since they are all pizzas you can remove the name.
var cooking = {
"recipes" : [
{
"ingredients" : [
"cheese:100g",
"tomato:200g"
]
},
{
"ingredients" : [
"ham:100g",
"pineapple:200g"
]
}
]
}
Hope this simplifies it for you ! Json must be written in a way so that its minimal and comprehensible for both computers and humans.
var cooking = {
"recipes" :
[
{
"name":"pizza",
"cheese": "100g"
"tomato": "200g"
}
,
{
"name":"pizza 2",
"ham": "300g"
"pineapple": "300g"
}
]
}
};