I'm relatively new to postgres and I have a problem manipulating nested JSON arrays.
My JSON structure:
{
"rows":[
{
"columns":[
[
{
"text":"Some text",
"type":"report"
},
{
"type":"feedback"
}
],
[
{
"key":"p0",
"type":"publications",
"title":""
}
]
]
},
{
"columns":[
[
{
"key":"p1",
"type":"publication",
"title":""
}
]
]
}
]
}
As you can see, the json structure is a bit redundant, which makes it even more difficult. My goal is just to add a new title element to each nested column of type="report" (as is already present for the type="publication").
I am currently looping over the arrays and trying to reconstruct the modified json structure again via concatentation, but this is a tedious and annoying process for this simple task.
FOR currentColumn IN SELECT * FROM json_array_elements(extractedRows)
-- more for loops to get to the type="publication" elements
-- set the new title element
SELECT jsonb_set(...) into result;
-- finally reconstruct the json and persist it
END LOOP;
Is there a clever way to do this more efficiently? I've read through the documentation but I have a hard time coming up with a better way.
Related
I've been wondering for some days what kind of scheme would be more appropriate to use a data list in json in a web application.
I'm developing a REST Web Application, and im using Angular for front end, i should order, filter and print these data list also in xml ...
For you what scheme is better and why?
1) {
"datas": [
{ "first":"","second":""},
{ "first":"","second":""},
{ "first":"","second":""}
]
}
2) {
"datas": [{
"data": { "first":"","second":""},
"data": { "first":"","second":""},
"data": { "first":"","second":""}
}]
}
3) [
{ "first":"","second":""},
{ "first":"","second":""},
{ "first":"","second":""}
]
Thanks so much.
The first and third notations are quite similar because the third notation is included in your first.
So the question is "Should I return my datas as an array or should I return an object with a property that contain the array ?
It will depend on either you want to have more information alongside your datas or not.
For exemple, if your API might return an error, you will want to manage it from the front end.
In case of error, the JSON will looks like this :
{
"datas": null,
"error": "An error occured because of some reasons..."
}
At the opposite, if everything goes well and your API actually return the results, it will looks like this :
{
"datas": [
{ "first":"","second":""},
{ "first":"","second":""},
{ "first":"","second":""}
],
"error": null
}
Then your front end can use the error property to manage errors sent from the API.
var result = getDatas(); // Load datas from the API
if(result.error){
// Handle the error, display a message to the user, ...
} else {
doSomething(result.datas); // Use your datas
}
If you don't need to have extra properties like error then you can stick with the third schema.
The second notation is invalid. The datas array will contain only one object which will have one property named data. In this case data is a property that is defined multiple times so the object in the array will contain only the last occurence:
var result = {
"datas": [{
"data": { "first":"a","second":"b"},
"data": { "first":"c","second":"d"},
"data": { "first":"e","second":"f"}
}]
}
console.log("Content of result.datas[0].data : ")
console.log(result.datas[0].data)
Obviously the first option would be easy to use. Once you will access datas it'll give you an array. Any operation (filter, sort, print) on that array will be easy in comparison to anything else. Everywhere you just need to pass datas not datas.data.
I have a REST call that is working properly. I can pass parameters, it returns data.
The app uses TRESTClient, TRESTResponse, TRESTRequest, TRESTAdapter, feeding into TClientDataSet and TDataSource.
The end result is that when the JSON data comes in, I can iterate through it as if it was a table. With simple JSON, I can get this working.
I am now querying a REST service which is providing data that is one level deeper than normal. See JSON below.
Everything I need to access is UNDER the mycursor element, which is under the items element.
I can't change the REST service, so how can I tell one of these components to ignore the items level and look at the mycursor level?
The data I am looking to parse has a first element of id.
{
"next":
{
"$ref":"https://<internal URL>/?page=1"
},
"items":
[
{
"mycursor":
[
{
"id":13372,
…
},
{
"id":13373,
…
},
{
"id":13374,
…
},
{
"id":13375,
…
},
{
"id":13376,
…
}
]
}
]
}
Consider a subset of a sample output from http://demo.nginx.com/status:
{
"timestamp": 1516053885198,
"server_zones": {
"hg.nginx.org": {
... // Data for "hg.nginx.org"
},
"trac.nginx.org": {
... // Data for "trac.nginx.org"
}
}
}
The keys "hg.nginx.org" and "track.nginx.org" are quite arbitrary, and I would like to parse them into something meaningful for Elasticsearch. In other words, each key under "server_zones" should be transformed into a separate event. Logstash should thus emit the following events:
[
{
"timestamp": 1516053885198,
"server_zone": "hg.nginx.org",
... // Data for "hg.nginx.org"
},
{
"timestamp": 1516053885198,
"server_zone": "trac.nginx.org",
... // Data for "trac.nginx.org"
}
]
What is the best way to go about doing this?
You can try using the ruby filter. Get the server zones and create a new object using the key value pairs you want to include. From the top of my head, something like below should work. Obviously you then need to map the object to your field in the index. Change the snipped based on your custom format i.e. build the array or object as you want.
filter {
ruby {
code => " time = event.get('timestamp')
myArr = []
event.to_hash.select {|k,v| ['server_zones'].include?(k)}.each do |key,value|
myCustomObject = {}
#map the key value pairs into myCustomObject
myCustomObject[timestamp] = time
myCustomObject[key] = value
myArr.push(myCustomObject) #you'd probably move this out based on nesting level
end
map['my_indexed_field'] = myArr
"
}
}
In the output section use rubydebug for error debugging
output {
stdout { codec => rubydebug }
}
Let's say this is the table inside my collection:
{
"_id" : ObjectId("557cf6bbd8efe38c627bffdf"),
"name" : "John Doe",
"rating" : 9,
"newF" : [
"milk",
"Eggs",
"Beans",
"Cream"
]
}
Once a user types in some input, it is sent to my node server, and my node server then adds that item to the list "newF", which is then sent back to my MongoDB and saved.
I'm trying to use update, which can successfully change the values inside of this table, but I'm not sure how to add new items onto that list. I did it with $push inside the MongoDB shell, but not sure how to do it on node.
Here's a snippet of my code:
db.collection('connlist').update({ _id: new ObjectId("e57cf6bb28efe38c6a7bf6df")}, { name: "JohnDoe", rating: 9, newF: ["Milk, Eggs", "Beans"] }, function(err,doc){
console.log(doc);
});
Well the syntax for adding new items is just the same as in the shell:
// make sure you actually imported "ObjectID"
var ObjectId = require('mongodb').ObjectID;
db.collection('conlist').update(
{ "_id": new ObjectId("e57cf6bb28efe38c6a7bf6df") },
{ "$push": { "newF": { "$each": [ "cream", "butter" ] } } },
function(err,numAffected) {
// do something in the callback
}
)
Or perhaps use .findOneAndUpdate() if you want to return the modified document instead of just making the alteration.
Of course use $push and possibly with $each which allows multiple array elements to be added when adding to an array. If you want "unique" items then use $addToSet where your operation allows.
And generally speaking for other items you should use $set or other operators in the update portion of your document. Without these operators you are just "replacing" the document content with whatever structure you place in the "update" portion of your statement.
I'm new to MongoDB, and I'm trying to get results in a different way.
if I execute the query db.collection.find().toArray() I get something like:
[
{
"_id":"34234...",
"first":"Mark",
"last":"Marker"
},
{
"_id": "34235...",
"first":"Adam",
"last":"Smith"
}
]
is there an api that lets you to receive the results as the following?:
{
"results" : {
"34234..." :{
"_id":"34234...",
"first":"Mark",
"last":"Marker"
},
"4235..." :{
"_id": "34235...",
"first":"Adam",
"last":"Smith"
}
}
Or I need to get the results array and iterate every single object and build my response? (I would like to avoid the single cursor iteration)
I don't believe there's a native API function for that. cursor.toArray() goes through each item in the cursor begin with, so I wouldn't worry too much about that. We can just skip the toArray() and do our own iteration:
var obj = {}
db.collection.find().each(function(item){
obj[item._id] = item;
});
I don't think that would really be any slower.