With jq, how can I transform the following:
{
"root": {
"branch1": {
"leaf": 1
},
"branch2": {
"leaf": 2
},
"branch3": {
"leaf": 3
}
},
"another-root": {
"branch": 123
},
"foo": "bar"
}
to this:
{
"root": {
"branch1": {
"leaf": "updated"
},
"branch2": {
"leaf": "updated"
},
"branch3": {
"leaf": "updated"
}
},
"another-root": {
"branch": 123
},
"foo": "bar"
}
🤦 Apparently [] can be used on object too. I had though it was only for lists.
The following was all I needed.
.root[].leaf="updated"
First you need to parse the json and then modify the resulting object as required using for ... in statement (example below)
const flatJSON = '{"root":{"branch1":{"leaf":1},"branch2":{"leaf":2},"branch3":{"leaf":3}},"another-root":{"branch":123},"foo":"bar"}';
const parsedJSON = JSON.parse(flatJSON);
const root = parsedJSON.root;
for (let property in root) {
root[property].leaf = "updated"; (or root[property]["leaf"] = "updated";)
}
If you want to use jquery you have to replace for ... in statement with jQuery.each() method that iterates over both objects and arrays.
Don't forget to convert it back to json with JSON.stringify() method (if required).
Hope that this helps.
All the best.
Related
I am writing a code in python3 where i am struggling with usage of variables with "pyjq", the code works without variables but variables are not getting parsed inside pyjq.
The documentation referred is https://github.com/doloopwhile/pyjq/blob/master/README.md#api
Please check the code given below and suggest -
My code
import json, os
import pyjq
from flask import Flask, request, jsonify
def query_records():
args = {"meta.antivirus.enabled": "true"}
for key, value in args.items():
with open('/tmp/data.txt', 'r') as f:
print (key)
print (value)
data = f.read()
records = json.loads(data)
query = ("." + key)
print (query)
#jq '.[]|select(.meta.antivirus.enabled=="true")' filename.json works,issue with variable substitution in python
match = pyjq.all('.[]|select(["$query"]==$value)', records, vars={"value": value,"query": query})
print (match)
query_records()
Content of file "/tmp/data.txt"
[
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "false"
},
"calories": 230,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "1g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "sandwich-nutrition"
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "true"
},
"calories": 440,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "2g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "random-nutrition"
}
]
Expected output(which works without variables)
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
}
Current output []
seems like some issue with variables not being passed in case of "query" , help would be appreciated.
Edit 1
It works if I hardcode "query" -
match = pyjq.all('.[]|select(.meta.antivirus.enabled==$value)', records, vars={"value": value,"query": query})
but not vice-versa
which probably narrows it down to issue with the variable "query"
JQ is not a necessity and I can use other libraries too,given that json is returned
Variables are intended to be used for values, not for jq expressions (at least not directly).
I think the easiest option here is to go for an fstring:
match = pyjq.all(f'.[]|select({query}==$value)', records, vars={"value": value})
and it probably makes sense to prepend the period inside the fstring:
match = pyjq.all(f'.[]|select(.{key}==$value)', records, vars={"value": value})
I have this JSON
{
"srv_config": [{
"name": "db1",
"servers": ["srv1", "srv2"],
"prop": [{"source":"aa"},"destination":"bb"},{"source":"cc"},"destination":"cc"},]
}, {
"name": "db2",
"servers": ["srv2", "srv2"],
"prop": [{"source":"dd"},"destination":"dd"},{"source":"ee"},"destination":"ee"},]
}
]
}
I try to build a JMESPath expression to select the prop application in each object in the main array, but based on the existence of a string in the servers element.
To select all props, I can do:
*.props [*]
But how do I add condition that says "select only if srv1 is in servers list"?
You can use the contains function in order to filter based on a array containing something.
Given the query:
*[?contains(servers, `srv1`)].prop | [][]
This gives us:
[
{
"source": "aa",
"destination": "bb"
},
{
"source": "cc",
"destination": "cc"
}
]
Please mind that I am also using a bit of flattening here.
All this run towards a corrected version of you JSON:
{
"srv_config":[
{
"name":"db1",
"servers":[
"srv1",
"srv2"
],
"prop":[
{
"source":"aa",
"destination":"bb"
},
{
"source":"cc",
"destination":"cc"
}
]
},
{
"name":"db2",
"servers":[
"srv2",
"srv2"
],
"prop":[
{
"source":"dd",
"destination":"dd"
},
{
"source":"ee",
"destination":"ee"
}
]
}
]
}
Here is the sample JSON
Sample JSON:
[
{
"_id": "123456789",
"YEAR": "2019",
"VERSION": "2019.Version",
"QUESTION_GROUPS": [
{
"QUESTIONS": [
{
"QUESTION_NAME": "STATE_CODE",
"QUESTION_VALUE": "MH"
},
{
"QUESTION_NAME": "COUNTY_NAME",
"QUESTION_VALUE": "IN"
}
]
},
{
"QUESTIONS": [
{
"QUESTION_NAME": "STATE_CODE",
"QUESTION_VALUE": "UP"
},
{
"QUESTION_NAME": "COUNTY_NAME",
"QUESTION_VALUE": "IN"
}
]
}
]
}
]
Query that am using :
db.collection.find({},
{
"QUESTION_GROUPS.QUESTIONS.QUESTION_NAME": "STATE_CODE"
})
My requirement is retrive all QUESTION_VALUE whose QUESTION_NAME is equals to STATE_CODE.
Thanks in Advance.
If I get you well, What you are trying to do is something like:
db.collection.find(
{
"QUESTION_GROUPS.QUESTIONS.QUESTION_NAME": "STATE_CODE"
},
{
"QUESTION_GROUPS.QUESTIONS.QUESTION_VALUE": 1
})
Attention: you will get ALL the "QUESTION_VALUE" for ANY document which has a QUESTION_GROUPS.QUESTIONS.QUESTION_NAME with that value.
Attention 2: You will get also the _Id. It is by default.
In case you would like to skip those issues, you may need to use Aggregations, and unwind the "QUESTION_GROUPS"-> "QUESTIONS". This way you can skip both the irrelevant results, and the _id field.
It sounds like you want to unwind the arrays and grab only the question values back
Try this
db.collection.aggregate([
{
$unwind: "$QUESTION_GROUPS"
},
{
$unwind: "$QUESTION_GROUPS.QUESTIONS"
},
{
$match: {
"QUESTION_GROUPS.QUESTIONS.QUESTION_NAME": "STATE_CODE"
}
},
{
$project: {
"QUESTION_GROUPS.QUESTIONS.QUESTION_VALUE": 1
}
}
])
I have a rather large json schema. The problematic part is a smaller schema within the schema called "translations", and which looks like this:
"translations": {
"bsonType": "object",
"patternProperties": {
"id": {
"bsonType": "string"
},
"^[a-z]{2}$": {
"anyOf": [
{
"bsonType": "object"
},
{
"bsonType": "array"
}
]
}
}
}
Where the object defined by the regex contains many more properties (a field called "text", for example) and the array is an array of these objects, but I only left the parts that are important for understanding the structure.
My issue is that when I validate my files against this schema, it fails every single one of them, but when I remove the "bsonType": "object" from the first object in the anyOf array, it works properly.
All of my files are such that at least one of the objects in the translation objects, which have the regular expression as key, are of type "object". so I don't understand why it fails them.
I use mongoDB 3.6.0.
Here is an example for a file that would fail:
"translations":{
"id":"12345",
"br":{
"text":"string1"
},
"en":{
"text":"string2"
},
"ja":[
{
"text":"string3"
},
{
"text":"string4"
}
],
"no":[
{
"text":"string6"
},
{
"text":"string7"
}
]
}
In case it wasn't clear- the problem is that files like this one fail when the schema is defined with "bsonType": "object" in the first object of the anyOf array, and works when i take that off. The "bsonType": "array" in the second object of the anyOf array works fine.
I think your problem that id collide with the regex try this:
let MongoClient = require('mongodb').MongoClient;
let collectionName = 'translations';
let scheme = {
$jsonSchema:{
"bsonType": "object",
"patternProperties": {
"^id$":{
"bsonType":"string"
},
"^(?!id)([a-z]{2})$": {
"anyOf": [
{
"bsonType": "object"
},
{
"bsonType": "array"
}
]
}
},
}
};
let goodJson ={
"id": "12345",
"br":{
"text":"string1"
},
"en":{
"text":"string2"
},
"ja":[
{
"text":"string3"
},
{
"text":"string4"
}
],
"no":[
{
"text":"string6"
},
{
"text":"string7"
}
]
};
let badJson ={
"id": "12345",
"br":{
"text":"string1"
},
"en":{
"text":"string2"
},
"ja":[
{
"text":"string3"
},
{
"text":"string4"
}
],
"no":[
{
"text":"string6"
},
{
"text":"string7"
}
],
"nt": "not_object_or_array"
};
async function run() {
let db = await MongoClient.connect('mongodb://localhost:27017/exampleDb');
let dbo = db.db('mydb');
let collections = await dbo.collections();
let collectionsNames = collections.map(c => c.s.name);
if (collectionsNames.includes(collectionName)) {
console.log('dropping collection');
await dbo.collection(collectionName).drop();
}
console.log('creating collection');
await dbo.createCollection(collectionName, {validator: scheme});
let translationCollection = dbo.collection(collectionName);
console.log('this will validate successfully');
await translationCollection.insertOne(goodJson);
console.log('this will raise validation error because: "nt": "not_object_or_array"');
try {
await translationCollection.insertOne(badJson);
} catch(error) {
console.log(error);
}
await db.close();
}
run();
I am having a JSON data like below.
{
"divisions": [{
"name": "division1",
"id": "div1",
"subdivisions": [{
"name": "Sub1Div1",
"id": "div1sub1",
"schemes": [{
"name": "Scheme1",
"id": "scheme1"
}, {
"name": "Scheme2",
"id": "scheme2"
}]
}, {
"name": "Sub2Div1",
"id": "div1sub2",
"schemes": [{
"name": "Scheme3",
"id": "scheme3"
}]
}
]
}]
}
I want to read this into a TreeStore, but cannot change the subfields ( divisions, subdivisions, schemes ) to be the same (eg, children).
How can achieve I this?
When nested JSON is loaded into a TreeStore, essentially the children nodes are loaded through a recursive calls between TreeStore.fillNode() method and NodeInterface.appendChild().
The actual retrieval of each node's children field is done within TreeStore.onNodeAdded() on this line:
dataRoot = reader.getRoot(data);
The getRoot() of the reader is dynamically created in the reader's buildExtractors() method, which is what you'll need to override in order to deal with varying children fields within nested JSON. Here is how it's done:
Ext.define('MyVariJsonReader', {
extend: 'Ext.data.reader.Json',
alias : 'reader.varijson',
buildExtractors : function()
{
var me = this;
me.callParent(arguments);
me.getRoot = function ( aObj ) {
// Special cases
switch( aObj.name )
{
case 'Bill': return aObj[ 'children' ];
case 'Norman': return aObj[ 'sons' ];
}
// Default root is `people`
return aObj[ 'people' ];
};
}
});
This will be able to interpret such JSON:
{
"people":[
{
"name":"Bill",
"expanded":true,
"children":[
{
"name":"Kate",
"leaf":true
},
{
"name":"John",
"leaf":true
}
]
},
{
"name":"Norman",
"expanded":true,
"sons":[
{
"name":"Mike",
"leaf":true
},
{
"name":"Harry",
"leaf":true
}
]
}
]
}
See this JsFiddle for fully working code.