I have a JSON file that looks like the below:
{"Africa":
{
"invitee_event_type_page":
{
"Total Events": "194"
},
"invitee_meeting_cancelled":
{
"Total Events": "9"
}
},
"Asia":
{
"invitee_event_type_page":
{
"Total Events": "127"
},
"invitee_meeting_scheduled":
{
"Total Events": "16"
}
}
}
I am trying to format this so that I something along the lines of the below:
output = [{"Asia - invitee_event_type_page": 127, "Asia - invitee_meeting_scheduled":16, "Africa - invitee_event_type_page": 194,"Africa - invitee_meeting_cancelled":9}]
I was trying the following but I've not had much luck. I'm quite new to all of this and I'm struggling to piece the different pieces of other answers together into something that is coherent. Any help would be greatly appreciated.
import re
import json
with open("ga2.json", "r") as f:
data =json.load(f)
africa = (data)["Africa"]
output=[]
for key,value in africa.items():
print(key,value["Total Events"])
output= output+key,value["Total Events"]
print(output)
I get TypeError: can only concatenate list (not "str") to list.
You can use dict-comprehension to get your output:
dct = {
"Africa": {
"invitee_event_type_page": {"Total Events": "194"},
"invitee_meeting_cancelled": {"Total Events": "9"},
},
"Asia": {
"invitee_event_type_page": {"Total Events": "127"},
"invitee_meeting_scheduled": {"Total Events": "16"},
},
}
out = [
{
f"{k} - {kk}": int(vv["Total Events"])
for k, v in dct.items()
for kk, vv in v.items()
}
]
print(out)
Prints:
[
{
"Africa - invitee_event_type_page": 194,
"Africa - invitee_meeting_cancelled": 9,
"Asia - invitee_event_type_page": 127,
"Asia - invitee_meeting_scheduled": 16,
}
]
Related
I am writing a code in python3 where i am struggling with usage of variables with "pyjq", the code works without variables but variables are not getting parsed inside pyjq.
The documentation referred is https://github.com/doloopwhile/pyjq/blob/master/README.md#api
Please check the code given below and suggest -
My code
import json, os
import pyjq
from flask import Flask, request, jsonify
def query_records():
args = {"meta.antivirus.enabled": "true"}
for key, value in args.items():
with open('/tmp/data.txt', 'r') as f:
print (key)
print (value)
data = f.read()
records = json.loads(data)
query = ("." + key)
print (query)
#jq '.[]|select(.meta.antivirus.enabled=="true")' filename.json works,issue with variable substitution in python
match = pyjq.all('.[]|select(["$query"]==$value)', records, vars={"value": value,"query": query})
print (match)
query_records()
Content of file "/tmp/data.txt"
[
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "false"
},
"calories": 230,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "1g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "sandwich-nutrition"
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "true"
},
"calories": 440,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "2g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "random-nutrition"
}
]
Expected output(which works without variables)
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
}
Current output []
seems like some issue with variables not being passed in case of "query" , help would be appreciated.
Edit 1
It works if I hardcode "query" -
match = pyjq.all('.[]|select(.meta.antivirus.enabled==$value)', records, vars={"value": value,"query": query})
but not vice-versa
which probably narrows it down to issue with the variable "query"
JQ is not a necessity and I can use other libraries too,given that json is returned
Variables are intended to be used for values, not for jq expressions (at least not directly).
I think the easiest option here is to go for an fstring:
match = pyjq.all(f'.[]|select({query}==$value)', records, vars={"value": value})
and it probably makes sense to prepend the period inside the fstring:
match = pyjq.all(f'.[]|select(.{key}==$value)', records, vars={"value": value})
I have a json that I need to import and then return a certain value. The json has two keys, like
{
"NUM_High_Objects": {
"abseta_pt": {
"field1:[0.0,0.9]": {
"field2:[15,20]": {
"tagIso": 0.00012,
"value": 0.99
},
"field2:[20,25]": {
"tagIso": 0.00035,
"value": 0.98
}
},
"field1:[0.91,1.2]": {
"field2:[15,20]": {
"tagIso": 0.00013,
"value": 0.991
},
"field2:[20,25]": {
"tagIso": 0.00036,
"value": 0.975
}
},
"binning": [
{
"binning": [
0.0,
0.9,
1.2,
2.1,
2.4
],
"variable": "abseta"
},
{
"binning": [
15,
20,
25,
30,
40,
50,
60,
120
],
"variable": "pt"
}
]
}
},
What I need is to search if a pair of values is within the range of "field1" and "field2" and return the corresponding "value"
I tried following this Search nested json / dict for multiple key values matching specified keys but could not make it to work...
I ve tried something like
class checkJSON() :
def __init__(self,filein) :
self.good, self.bad = 0, 0
print 'inside json function : will use the JSON', filein
input_file = open (filein)
self.json_array = json.load(input_file)
def checkJSON(self,LS,run) :
try :
LSlist = self.json_array[str(run)]
for LSrange in LSlist :print LSrange, run
except KeyError :
pass
self.bad += 1
return False
CJ=''
CJ=checkJSON(filein='test.json')
isInJSON = CJ.checkJSON("0.5", "20")
print isInJSON
but this does not work as I am not sure how to loop inside the keys
If I am understanding your question correctly then the relevant portion of your JSON is:
{
"field1:[0.0,0.9]": {
"field2:[15,20]": {
"tagIso": 0.00012,
"value": 0.99
},
"field2:[20,25]": {
"tagIso": 0.00035,
"value": 0.98
}
},
"field1:[0.91,1.2]": {
"field2:[15,20]": {
"tagIso": 0.00013,
"value": 0.991
},
"field2:[20,25]": {
"tagIso": 0.00036,
"value": 0.975
}
},
"binning": [
{
"binning": [
0.0,
0.9,
1.2,
2.1,
2.4
],
"variable": "abseta"
},
{
"binning": [
15,
20,
25,
30,
40,
50,
60,
120
],
"variable": "pt"
}
]
}
Then the following code should do what you are trying to achieve. It doesn't look like you need to search for nested keys, you simply need to parse your field1[...] and field2[...]. The code below is a quick implementation of what I understand you are trying to achieve. It will return the value if the first parameter is in the range of a field1[...] and the second parameter is in the range of a field2[...]. Otherwise, it will return None.
import json
def check_json(jsondict, l1val, l2val):
def parse_key(keystr):
level, lrange = keystr.split(':')
return level, eval(lrange)
for l1key, l2dict in jsondict.items():
if 'field' in l1key:
l1, l1range = parse_key(l1key)
if l1val >= l1range[0] and l1val <= l1range[1]:
for l2key, vals in l2dict.items():
l2, l2range = parse_key(l2key)
if l2val >= l2range[0] and l2val <= l2range[1]:
return vals['value']
return None
Here is a driver code to test the implementation above.
if __name__ == '__main__':
with open('data.json', 'r') as f:
myjson = json.load(f)
print(check_json(myjson, 0.5, 20))
I'm trying to parse and mofidy JSON with Groovy. Source JSON from REST API looks like:
[
{
"id":27858,
"type":"ad",
"stats":[
{
"day":"2021-01-21",
"sex":[
{
"impressions_rate":0.349,
"value":"f"
},
{
"impressions_rate":0.621,
"value":"m",
"clicks_rate":0.22
}
],
"age":[
{
"impressions_rate":0.217,
"value":"18-21"
}
]
},
{
"day":"2021-02-25",
"sex":[
{
"impressions_rate":0.349,
"value":"f"
},
{
"impressions_rate":0.651,
"value":"m"
}
],
"age":[
{
"impressions_rate":0.217,
"value":"18-21"
}
]
}
]
},
{
"id":565291,
"type":"ad",
"stats":[
{
"day":"2021-03-21",
"sex":[
{
"impressions_rate":0.78,
"value":"f",
"clicks_rate":0.33
},
{
"impressions_rate":0.551,
"value":"m"
}
],
"age":[
{
"impressions_rate":0.17,
"value":"18-21"
}
]
}
]
}
]
It's an array with some ids and data for them. I want to grab id, day inside stats array and elements from sex array. After all manipulations my JSON should be like this:
[
{
"id": 27858,
"day": "2021-01-21",
"impression_rate": 0.349,
"value": "f"
},
{
"id": 27858,
"day": "2021-01-21",
"impression_rate": 0.621,
"value": "f",
"clicks_rate": 0.22
},
{
"id": 27858,
"day": "2021-02-25",
"impressions_rate":0.349,
"value":"f"
},
{
"id": 27858,
"day": "2021-02-25",
"impressions_rate":0.651,
"value":"m"
},
{
"id": 565291,
"day": "2021-03-21",
"impressions_rate":0.78,
"value":"f",
"clicks_rate":0.33
},
{
"id": 565291,
"day": "2021-03-21",
"impressions_rate":0.78,
"value":"f",
"clicks_rate":0.33
}
]
So, the main goal is - loop through all ids -> elements in sex array (for each id) and add to these elements day and id mapped fields. I tried to start with empty map with inject, but after 1 hour of debugging i still can't achieve desired output, so maybe better to loop through existed values in array? But I can't even reach sex array.
import groovy.json.*
def json = new JsonSlurper().parseText '''...'''
List expected = json.inject([]){ r, e ->
Map ids = e.findAll {
k, v -> k == "id"
}
e.each{ k, v ->
if( (v.any{ it.sex } )
v.each{ r << ids + it }
}
return r
}
If you have nested structures, that contain again nested structures,
a good option, to get a flat result, is to use collectMany; like
collect it transforms each item of the iterated container, but the
results gets concated.
E.g. you can collectMany on your outer data, then again on the
stats, and finally just collect over sex.
def data = new groovy.json.JsonSlurper().parse("data.json" as File)
println data.collectMany{ o ->
o.stats.collectMany{ i ->
i.sex.collect{ it + [id: o.id, day: i.day] }
}
}
// [[impressions_rate:0.349, value:f, id:27858, day:2021-01-21],
// [impressions_rate:0.621, value:m, clicks_rate:0.22, id:27858, day:2021-01-21],
// [impressions_rate:0.349, value:f, id:27858, day:2021-02-25],
// [impressions_rate:0.651, value:m, id:27858, day:2021-02-25],
// [impressions_rate:0.78, value:f, clicks_rate:0.33, id:565291, day:2021-03-21],
// [impressions_rate:0.551, value:m, id:565291, day:2021-03-21]]
This question already has an answer here:
Karate API json response - How to validate the presence of a key which sometimes come and sometimes not in the API response
(1 answer)
Closed 2 years ago.
I am trying to validate a JSON which has Optional keys depending on a condition.
Json response is as follows:
{ "hits": [
{
"_type": "sessions",
"_source": {
"movie_name": "The Greatest Showman (U/A) - English",
"session_data": {
"freeSeating": false,
"mandatoryItems": [],
"areas": [
{
"seatingType": "fixed"
},
{
"seatingType": "free"
},
{
"seatingType": "mixed"
}
]
}
}
},
{
"_type": "sessions",
"_source": {
"movie_name": "The Greatest Showman (U/A) - English",
"session_data": {
"freeSeating": false,
"mandatoryItems": [],
"areas": [
{
"seatingType": "fixed"
},
{
"seatingType": "free"
}
]
}
}
},
{
"_type": "sessions",
"_source": {
"movie_name": "The Greatest Showman 3D (U/A) - English",
"session_data": {
"freeSeating": false,
"mandatoryItems": [
{
"quantity": 1,
"level": "ticket",
"price": 30,
"type": "3dglasses"
}
],
"areas": [
{
"seatingType": "fixed"
}
]
}
}
}
]
}
If movie_name contains "3D" in it then mandatoryItems = [{"quantity": 1,"level": "ticket", "price": 30, "type": "3dglasses"}]
If movie_name does not contains "3D" in it then mandatoryItems = []
I want to achieve above assertions in my feature file.
Note: "movie_name" and "mandatoryItems" are present each element of an array. So I want to assert this condition on entire array.
Thanks in advance!!
Sorry #Peter for the inconvenience caused by me. I worked on this problem statement by referring all possible sources and wrote following code which is giving me desired output:
Given url api_url
When method Get
And def mandatoryItems_present =
"""
{
"quantity": 1,
"level": "ticket",
"price": '#number',
"type": "3dglasses",
}
"""
Then status 200
And print response
And def source_list = karate.jsonPath(response, '$.._source')
And print source_list
And match each source_list[*].session_data contains {'freeSeating': '#boolean','mandatoryItems':'##[] mandatoryItems_present'}
And def movie_names = get source_list[*].movie_name
And def mandatoryItems_list = get source_list[*].session_data.mandatoryItems
And def name_size = names.size();
And print name_size
And def threeD_movie_list = new ArrayList()
And eval for(var i = 0; i < name_size; i++) {if (names[i].match('3D')) threeD_movie_list.add(names[i])}
And print threeD_movie_list
And def threeD_movies_array_size = threeD_movie_list.size();
And print threeD_movies_array_size
And print expected
And def expected = (threeD_movies_array_size == 0 ? {"mandatoryItems" : []} : {'mandatoryItems' : [mandatoryItems_present]} )
And print expected
And match each response.hits[*]._source.session_data[*].mandatoryItems == expected
Please let me know whether this approach is correct.
I have a json File that is in the format. I would like to access each branch and sub branch of this file using d3.js
"x":
{
"a":
{
no.of employees:30
total salary:2500000
count_email:25
}
"b":
{
no.of employees:20
total salary:350000
count_email:25
}
}
"y":
{
"c":
{
no.of employees:30
total salary:4500000
count_email:30
}
}
"z":
{
"d":
{
no.of employees:40
total salary:550000
count_email:40
}
"e":
{
no.of employees:10
total salary:100000
count_email:15
}
"f":
{
no.of employees:15
total salary:1500000
count_email:15
}
}
How can i access each branch and sub branch for processing. How to know the keys of this json format
Your Json is in a wrong format :
Before anything, validate it with a tool like JsonLint.com.
It should look something like that :
data = {
"x": [{
"a": {
"ff": 30,
"fff": 2500000,
"fff": 25
},
"b": {
"ff": 30,
"fff": 2500000,
"fff": 25
}
}],
"y": [{
"a": {
"ff": 30,
"fff": 2500000,
"fff": 25
},
"b": {
"ff": 30,
"fff": 2500000,
"fff": 25
}
}]
};
You could iterate into your object like that :
Employee = [];
for (x in data) {
// Access to the first level
for (y in x) {
//Access to the second level
console.log(data[x][y]);
// Here you could store each of your object into a new tab that you could use in d3.js
Employee.push(data[x][y]);
}
}
Nothing special in D3 to do this.
You can use simple javascript to deal with this case.
Something like this:
d3.json("my.json", function(error, json) { //load json using d3
if (error) return console.warn(error);
var keys = Object.keys(json); //get all the keys
console.log(keys, "Branch") //print on console
keys.forEach(function(key){
var subBranch = Object.keys(json[key]); //get all subbranch for key
console.log(subBranch, "SubBranch of " + key); //print subbranch and key
})
});
will produce output like this:
["x", "y", "z"] "Branch"
["a", "b"] "SubBranch of x"
["c"] "SubBranch of y"
["d", "e", "f"] "SubBranch of z"
working code here