How to extract all keys / values from JSON, using Python - json

I need to extract the "string-value" and "value" from the "contains" element in the following JSON snippet:
{
"pricing-model": {
"qualifier": {
"and": {
"all": {
"equals": {
"constant": {
"string-value": "AFN",
"data-type": "java.lang.String"
},
"value": "item.priceInfo.currencyCode"
},
"collection-name": "items",
"element-name": "item"
},
"or": {
"contains": [
{
"constant": {
"string-value": "AF",
"data-type": "java.lang.String"
},
"value": "locale.country"
},
{
"constant": {
"string-value": "IR",
"data-type": "java.lang.String"
},
"value": "locale.country"
}
]
}
}
}
}
My current approach is too simple example below only extracts one string-value, when I need to extract both:
with open('my.json') as f:
data = json.load(f)
my_dict={}
for item in data:
value = item.get('pricing-model').get('qualifier').get('and').get('or')\
.get('constant')[0].get('string-value')

If you're unsure the keys exist, you can chain dict.get() with an empty dict as a default. That way .get() will always be a valid method of the result.
items = data.get('pricing-model', {}).get('qualifier', {}).get('and', {}).get('or', {}).get('contains', [])
values = [item.get('constant', {}).get('string-value') for item in items if item]
print(values) # ['AF', 'IR']
If you're certain that the keys will exist you can just access them directly as you would a standard dict.
items = data['pricing-model']['qualifier']['and']['or']['contains']
values = [item['constant']['string-value'] for item in items]
print(values)

Related

pyjq - how to use "select" with both query and value as variables

I am writing a code in python3 where i am struggling with usage of variables with "pyjq", the code works without variables but variables are not getting parsed inside pyjq.
The documentation referred is https://github.com/doloopwhile/pyjq/blob/master/README.md#api
Please check the code given below and suggest -
My code
import json, os
import pyjq
from flask import Flask, request, jsonify
def query_records():
args = {"meta.antivirus.enabled": "true"}
for key, value in args.items():
with open('/tmp/data.txt', 'r') as f:
print (key)
print (value)
data = f.read()
records = json.loads(data)
query = ("." + key)
print (query)
#jq '.[]|select(.meta.antivirus.enabled=="true")' filename.json works,issue with variable substitution in python
match = pyjq.all('.[]|select(["$query"]==$value)', records, vars={"value": value,"query": query})
print (match)
query_records()
Content of file "/tmp/data.txt"
[
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "false"
},
"calories": 230,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "1g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "sandwich-nutrition"
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "true"
},
"calories": 440,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "2g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "random-nutrition"
}
]
Expected output(which works without variables)
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
}
Current output []
seems like some issue with variables not being passed in case of "query" , help would be appreciated.
Edit 1
It works if I hardcode "query" -
match = pyjq.all('.[]|select(.meta.antivirus.enabled==$value)', records, vars={"value": value,"query": query})
but not vice-versa
which probably narrows it down to issue with the variable "query"
JQ is not a necessity and I can use other libraries too,given that json is returned
Variables are intended to be used for values, not for jq expressions (at least not directly).
I think the easiest option here is to go for an fstring:
match = pyjq.all(f'.[]|select({query}==$value)', records, vars={"value": value})
and it probably makes sense to prepend the period inside the fstring:
match = pyjq.all(f'.[]|select(.{key}==$value)', records, vars={"value": value})

update value of a map of objects

With jq, how can I transform the following:
{
"root": {
"branch1": {
"leaf": 1
},
"branch2": {
"leaf": 2
},
"branch3": {
"leaf": 3
}
},
"another-root": {
"branch": 123
},
"foo": "bar"
}
to this:
{
"root": {
"branch1": {
"leaf": "updated"
},
"branch2": {
"leaf": "updated"
},
"branch3": {
"leaf": "updated"
}
},
"another-root": {
"branch": 123
},
"foo": "bar"
}
🤦 Apparently [] can be used on object too. I had though it was only for lists.
The following was all I needed.
.root[].leaf="updated"
First you need to parse the json and then modify the resulting object as required using for ... in statement (example below)
const flatJSON = '{"root":{"branch1":{"leaf":1},"branch2":{"leaf":2},"branch3":{"leaf":3}},"another-root":{"branch":123},"foo":"bar"}';
const parsedJSON = JSON.parse(flatJSON);
const root = parsedJSON.root;
for (let property in root) {
root[property].leaf = "updated"; (or root[property]["leaf"] = "updated";)
}
If you want to use jquery you have to replace for ... in statement with jQuery.each() method that iterates over both objects and arrays.
Don't forget to convert it back to json with JSON.stringify() method (if required).
Hope that this helps.
All the best.

jmespath :select json object element based on other (array) element in the object

I have this JSON
{
"srv_config": [{
"name": "db1",
"servers": ["srv1", "srv2"],
"prop": [{"source":"aa"},"destination":"bb"},{"source":"cc"},"destination":"cc"},]
}, {
"name": "db2",
"servers": ["srv2", "srv2"],
"prop": [{"source":"dd"},"destination":"dd"},{"source":"ee"},"destination":"ee"},]
}
]
}
I try to build a JMESPath expression to select the prop application in each object in the main array, but based on the existence of a string in the servers element.
To select all props, I can do:
*.props [*]
But how do I add condition that says "select only if srv1 is in servers list"?
You can use the contains function in order to filter based on a array containing something.
Given the query:
*[?contains(servers, `srv1`)].prop | [][]
This gives us:
[
{
"source": "aa",
"destination": "bb"
},
{
"source": "cc",
"destination": "cc"
}
]
Please mind that I am also using a bit of flattening here.
All this run towards a corrected version of you JSON:
{
"srv_config":[
{
"name":"db1",
"servers":[
"srv1",
"srv2"
],
"prop":[
{
"source":"aa",
"destination":"bb"
},
{
"source":"cc",
"destination":"cc"
}
]
},
{
"name":"db2",
"servers":[
"srv2",
"srv2"
],
"prop":[
{
"source":"dd",
"destination":"dd"
},
{
"source":"ee",
"destination":"ee"
}
]
}
]
}

Convert Nested Dictionary to a graphable format

So I'm trying to convert a nested dictionary like:
A = {
"root":
{
"child1":
{
"child11":"hmm",
"child12":"not_hmm"
},
"child2":"hello"
}
}
To this:
{
"name":"root",
"children": [
{"name":"child1",
"children" :
[{"name":"child11",
"children":[{"name":"hmm"}]}
{"name":"child12",
"children":[{"name":"not_hmm"}]}
]
},
{"name":"child2",
"children":[{"name":"hello"}]
}
]
}
I need this, since I'm trying to visualize it with this graph drawing template: Collapsible Tree
I'm having some trouble creating a recursive method that is capable of this transformation.
Preferably in python3. So far I have:
def visit(node, parent=None):
B = {}
for k,v in node.items():
B["name"]=k
B["children"] = []
if isinstance(v,dict):
print("Key value pair is",k,v)
B["children"].append(visit(v,k))
new_dict = {}
new_dict["name"]=v
return [new_dict]
C = visit(A) # This should have the final result
But its wrong. Any help is appreciated.
We'll have a function that takes a root (assuming it has only one entry), and returns a dict, as well as a helper function that returns lists of dicts.
def convert(d):
for k, v in d.items():
return {"name": k, "children": convert_helper(v)}
def convert_helper(d):
if isinstance(d, dict):
return [{"name": k, "children": convert_helper(v)} for k, v in d.items()]
else:
return [{"name": d}]
which gives us
json.dumps(convert(A), indent=2)
{
"name": "root",
"children": [
{
"name": "child1",
"children": [
{
"name": "child11",
"children": [
{
"name": "hmm"
}
]
},
{
"name": "child12",
"children": [
{
"name": "not_hmm"
}
]
}
]
},
{
"name": "child2",
"children": [
{
"name": "hello"
}
]
}
]
}

Groovy JSONBuilder issues

I'm trying to use JsonBuilder with Groovy to dynamically generate JSON. I want to create a JSON block like:
{
"type": {
"__type": "urn",
"value": "myCustomValue1"
},
"urn": {
"__type": "urn",
"value": "myCustomValue2"
},
"date": {
"epoch": 1265662800000,
"str": "2010-02-08T21:00:00Z"
},
"metadata": [{
"ratings": [{
"rating": "NR",
"scheme": "eirin",
"_type": {
"__type": "urn",
"value": "myCustomValue3"
}
}],
"creators": [Jim, Bob, Joe]
}]
}
I've written:
def addUrn(parent, type, urnVal) {
parent."$type" {
__type "urn"
"value" urnVal
}
}
String getEpisode(String myCustomVal1, String myCustomVal2, String myCustomVal3) {
def builder = new groovy.json.JsonBuilder()
def root = builder {
addUrn(builder, "type", myCustomVal1)
addUrn(builder, "urn", "some:urn:$myCustomVal2")
"date" {
epoch 1265662800000
str "2010-02-08T21:00:00Z"
}
"metadata" ({
ratings ({
rating "G"
scheme "eirin"
addUrn(builder, "_type", "$myCustomVal3")
})
creators "Jim", "Bob", "Joe"
})
}
return root.toString();
}
But I've run into the following issues:
Whenever I call addUrn, nothing is returned in the string. Am I misunderstanding how to use methods in Groovy?
None of the values are encapsulated in double (or single) quotes in the returned string.
Anytime I use a {, I get a '_getEpisode_closure2_closure2#(insert hex)' in the returned value.
Is there something wrong with my syntax? Or can someone point me to some example/tutorial that uses methods and/or examples beyond simple values (e.g. nested values within arrays).
NOTE: This is a watered down example, but I tried to maintain the complexity around the areas that were giving me issues.
You have to use delegate in addUrn method instead of
passing the builder on which you are working.
It is because you are doing a toSting() or toPrettyString() on root instead of builder.
Solved if #2 is followed.
Sample:
def builder = new groovy.json.JsonBuilder()
def root = builder {
name "Devin"
data {
type "Test"
note "Dummy"
}
addUrn(delegate, "gender", "male")
addUrn(delegate, "zip", "43230")
}
def addUrn(parent, type, urnVal) {
parent."$type" {
__type "urn"
"value" urnVal
}
}
println builder.toPrettyString()
Output:-
{
"name": "Devin",
"data": {
"type": "Test",
"note": "Dummy"
},
"gender": {
"__type": "urn",
"value": "male"
},
"zip": {
"__type": "urn",
"value": "43230"
}
}