I want to create a map of data with the pid value as the key and the name as the array for a json array. Here is the json structure:
{
"measurements": [
{
"pid": 6691,
"name": "lung",
"measurement": "qualityFactor",
},
{
"pid": 1106,
"name": "kidney",
"measurement": "qualityFactor",
},
{
"id": 119,
"name": "pancreas",
"measurement": "qualityFactor",
},
]
}
Here is my attempt with Groovy but I'm stuck:
def jsonSlurper= new JsonSlurper()
Object objs=jsonSlurper.parseText(jsonData)
List pp =objs.data
Map<String,String> m=new HashMap()
pp.each{ it ->
it.collect{Map mm ->
println "Map m is ${mm}"
}
}
I want the map to look like
["6691" : "lung" , "1106" :"kidney" ....] etc.
Hiw can I accomplish this?
As a full answer with corrected JSON:
def json = '''{
"measurements": [
{
"pid": 6691,
"name": "lung",
"measurement": "qualityFactor",
},
{
"pid": 1106,
"name": "kidney",
"measurement": "qualityFactor",
},
{
"pid": 119,
"name": "pancreas",
"measurement": "qualityFactor",
}
]
}
'''
import groovy.json.*
def parsed = new JsonSlurper().parseText(json)
def map = map.measurements.collectEntries { [it.pid.toString(), it.name] }
Related
This is my JSON file
{
"Research": {
"#xmlns": "http://www.rixml.org/2002/6/RIXML",
"#xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"#createDateTime": "2022-03-25T12:18:03.647Z",
"#language": "eng",
"#researchID": "ABCDEFG_1194968",
"#xsi:schemaLocation": "http://www.rixml.org/2002/6/RIXML http://www.rixml.org/newsite/specifications/v20/RIXML2.xsd",
"Product": {
"#productID": "ABCDEFG_1194968",
"StatusInfo": {
"#currentStatusIndicator": "Yes",
"#statusDateTime": "2022-03-25T12:17:53.686Z",
"#statusType": "Published"
},
"Source": {
"Organization": {
"#primaryIndicator": "Yes",
"#type": "SellSideFirm",
"OrganizationID": [
{
"#idType": "Multex",
"#text": "767"
},
{
"#idType": "cr pep",
"#text": "Americas"
}
],
"OrganizationName": {
"#nameType": "Display",
"#text": "VFGH"
},
"PersonGroup": {
"PersonGroupMember": {
"#primaryIndicator": "Yes",
"Person": {
"#personID": "ABCDEFG12275",
"ContactInfo": {
"#nature": "Business",
"Email": "su.ppe#cr-pep.com"
}
}
}
}
}
},
"Content": {
"Title": "Weekly Insights: March 25, 2022",
"Synopsis": null,
"Resource": {
"#language": "eng",
"#primaryIndicator": "Yes",
"#resourceID": "ABCDEFG_1194968",
"MIMEType": "application/pdf",
"Name": "ABCDEFG_1194968.pdf"
}
}
}
}
}
I need to to parse JSON and get
researchID ,productID and resourceID
I am trying to read file from s3 and then want to parse and process the JSON .
This is the simple code to start with .
import json
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
s3_clientobj = s3.get_object(Bucket='avcdf-bulk-dev', Key='RIXML/josnfile.json')
s3_clientdata = s3_clientobj['Body'].read().decode('utf-8')
#print(s3_clientdata)
print (s3_clientdata[0]['Research'])
Apology in advance as this is very basic question to ask but i am new to python and started with AWS lambda .
Please suggest something which does not required any lambda layers in AWS if possible .
Tree view is
Convert into dict and then try
import json
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
s3_clientobj = s3.get_object(Bucket='avcdf-bulk-dev', Key='RIXML/josnfile.json')
s3_clientdata = s3_clientobj['Body'].read().decode('utf-8')
#print(s3_clientdata)
clientdata_dict = json.loads(s3_clientdata)
print(clientdata_dict)
print(clientdata_dict['Research']['#researchID'])
You can access the above mentioned properties like below,
s3_clientobj = s3.get_object(Bucket='avcdf-bulk-dev', Key='RIXML/josnfile.json')
s3_clientdata = json.loads(s3_clientobj['Body'].read().decode('utf-8'))
print("%s, %s, %s" % (s3_clientdata['Research']['#researchID'], s3_clientdata['Research']['Product']['#productID'], s3_clientdata['Research']['Product']['Content']['Resource']['#resourceID']))
My original file is in CSV format which I have converted to python JSON array to JSON Sring.
jsonfile
<class 'list'>
<class 'dict'>
[
{
"key": "timestamp",
"source": "eia007",
"turnover": "65million",
"url": "abc.com",
"record": "",
"loc.reg": "nord000",
"loc.count": "abs39i5",
"loc.town": "cold54",
"co.gdp": "nscrt77",
"co.pop.min": "min50",
"co.pop.max": "max75",
"co.rev": "",
"chain.system": "5t5t5",
"chain.type": "765ef",
"chain.strat": "",
}
]
I would like to get the output as below:
{
"timestamp001": {
"key": "timestamp001",
"phNo": "ner007",
"turnover": "65million",
"url": "abc.com",
"record": "",
"loc": {
"reg": "nord000",
"count": "abs39i5",
"town": "cold54"
},
"co": {
"form": "nscrt77",
"pop": {
"min": "min50",
"max": "max75"
},
"rev: ""
},
"chain":{
"system": "5t5t5",
"type": "765ef",
"strat": ""
}
...
}
...
}
]
I have tried different options; tried to enumerate, but cannot get the required output. Please help me with this. Thanks in advance.
You can use something like this to create the nested dict:
import json
def unflatten(somedict):
unflattened = {}
for key, value in somedict.items():
splitkey = key.split(".")
print(f"doing {key} {value} {splitkey}")
# subdict is the dict that goes deeper in the nested structure
subdict = unflattened
for subkey in splitkey[:-1]:
# if this is the first time we see this key, add it
if subkey not in subdict:
subdict[subkey] = {}
# shift the subdict a level deeper
subdict = subdict[subkey]
# add the value
subdict[splitkey[-1]] = value
return unflattened
data = {
"key": "timestamp",
"source": "eia007",
"turnover": "65million",
"url": "abc.com",
"record": "",
"loc.reg": "nord000",
"loc.count": "abs39i5",
"loc.town": "cold54",
"co.gdp": "nscrt77",
"co.pop.min": "min50",
"co.pop.max": "max75",
"co.rev": "",
"chain.system": "5t5t5",
"chain.type": "765ef",
"chain.strat": "",
}
unflattened = unflatten(data)
print(json.dumps(unflattened, indent=4))
Which produces:
{
"key": "timestamp",
"source": "eia007",
"turnover": "65million",
"url": "abc.com",
"record": "",
"loc": {
"reg": "nord000",
"count": "abs39i5",
"town": "cold54"
},
"co": {
"gdp": "nscrt77",
"pop": {
"min": "min50",
"max": "max75"
},
"rev": ""
},
"chain": {
"system": "5t5t5",
"type": "765ef",
"strat": ""
}
}
Cheers!
I'm trying to parse json file to csv. Could you help?
example json:
{
"expand": "schema,names",
"startAt": 0,
"maxResults": 50,
"total": 21,
"issues": [
{
"expand": "operations,versionedRepresentations",
"id": "217580",
"self": "issue/217580",
"key": "ART-4070",
"fields": {"summary": "#[ART] Pre.3 Verification \\"S\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217579",
"self": "issue/217579",
"key": "ART-4069",
"fields": {"summary": "Verification \\"C\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217577",
"self": "issue/217577",
"key": "ART-4068",
"fields": {"summary": "#[ART] Enum type"}
}
]
}
result csv should be like:
key;summary
ART-4070;#[ART] Pre.3 Verification \"S\"
ART-4069;Verification \"C\"
ART-4068;#[ART] Enum type
I've tried such a code:
import groovy.json.*
def jsonSlurper = new JsonSlurper()
def json = '''
{
"expand": "schema,names",
"startAt": 0,
"maxResults": 50,
"total": 21,
"issues": [
{
"expand": "operations,versionedRepresentations",
"id": "217580",
"self": "issue/217580",
"key": "ART-4070",
"fields": {"summary": "#[ART] Pre.3 Verification \\"S\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217579",
"self": "issue/217579",
"key": "ART-4069",
"fields": {"summary": "Verification \\"C\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217577",
"self": "issue/217577",
"key": "ART-4068",
"fields": {"summary": "#[ART] Enum type"}
}
]
}
'''
def obj = jsonSlurper.parse(json)
def columns = obj.issues*.keySet().flatten().unique()
// remove nulls
def encode = { e -> e == null ? '' : e }
// Print all the column names
println columns.collect { c -> encode( c ) }.join( ';' )
// create all the rows
println obj.issues.collect { row ->
// A row at a time
columns.collect { colName -> encode( row[ colName ] ) }.join( ';' )
}.join( '\n' )
but result is wrong:
expand;id;self;key;fields
operations,versionedRepresentations;217580;issue/217580;ART-4070;[summary:#[ART] Pre.3 Verification "S"]
operations,versionedRepresentations;217579;issue/217579;ART-4069;[summary:Verification "C"]
operations,versionedRepresentations;217577;issue/217577;ART-4068;[summary:#[ART] Enum type]
how can i extract only what i want from json file? I need only two columns:key,summary and values for them.
You want to extract only specific information from your list of issues
and you need different strategies to extract those. So I'd use
a "configuration" to describe the extraction (see the map config
below). Then the code is quite close to your original one (extracted
some common code etc)
import groovy.json.*
def config = [ // header -> extractor
"key": { it.key },
"summary": { it.fields.summary }
]
def encode(e) { // help with nulls; quote the separator
(e ?: "").replaceAll(";", "\\;")
}
def csvLine(items) { // write items as "CSV"
println(items.collect{ encode it }.join(";"))
}
// main
def obj = new JsonSlurper().parse("data.json" as File)
csvLine(config.keySet())
obj.issues.each{ issue ->
csvLine(config.values().collect{ f -> f issue })
}
I want to extract part of an existing JSON file based on a list of keys and save it into another JSON file
For eg-
{
"211": {
"year": "2020",
"field": "chemistry"
},
"51": {
"year": "2019",
"field":"physics"
},
"5": {
"year": "2014",
"field":"Literature"
}
Lets say the list =[5,51]
Output json file should contain
{
"5": {
"year": "2014",
"field":"Literature"
},
"51": {
"year": "2019",
"field":"physics"
}
}
}
It should not contain data for key 211
I believe this will work for you:
import json
# Open input file and deserialize JSON to a dict
with open("input_file.json", "r", encoding="utf8") as read_file:
input_file_dict = json.load(read_file)
# List of id's you want
inc_id_list = [5,51,2,101]
output_dict = dict()
# Iterate over the input file JSON and only add the items in the list
for id in input_file_dict.keys():
if int(id) in inc_id_list:
output_dict[id] = input_file_dict.get(id)
# Serialize output dict to JSON and write to output file
with open('output_file.json', 'w') as output_json_file:
json.dump(output_dict, output_json_file)
try this :
l= {
"211": {
"year": "2020",
"field": "chemistry"
},
"51": {
"year": "2019",
"field":"physics"
},
"5": {
"year": "2014",
"field":"Literature"
} }
for i,v in l.items():
if(i!="211"):
print(i,v)
It's a dictionary of JSON, with a missing bracket at the end, so that is easily done with a comprehension:
mydict = { "211": { "year": "2020", "field": "chemistry" }, "51": { "year": "2019", "field":"physics" }, "5": { "year": "2014", "field":"Literature" }}
incList = [5, 51]
myAnswer = {k:v for (k,v) in mydict.items() if int(k) in incList}
Hi i am trying to parse the below json file. I tried using jsonsluper and parsed the file. I executed below command. Nothing works.
def test =newjsonslurper().parsetext(organist)
test.resources.each{
println it.resources.metadata. "guid"
println it.resources.entity. "name"
}
This is the json file format
resources: [
{
"metadata" :{
"guid":"cya"
"url": "dummy.test"
},
"entity" :
{
"name": "system"
"status": "active"
}
}
{
"metadata" :
{
"guid":"cya"
"url": "dummy.test"
},
"entity" :
{
"name": "system"
"status": "active"
}
}
]
There were a couple of problems:
JsonSlurper().parseText() expects a String. If you're wanting to parse a file, use something like def response = new JsonSlurper().parse(new File('JsonFile.json'))
The JSON payload is not valid: it's missing a few brackets and commas.
The following code should work:
import groovy.json.JsonSlurper
def test = new JsonSlurper().parseText '''
{"resources": [
{
"metadata": {
"guid": "cya",
"url": "dummy.test"
},
"entity": {
"name": "system",
"status": "active"
}
},
{
"metadata": {
"guid": "cya",
"url": "dummy.test"
},
"entity": {
"name": "system",
"status": "active"
}
}
]}
'''
test.resources.each {
println it.metadata.guid
println it.entity.name
}