I trying to write to the jsong file. I want my json file should be written with indentation. But It has only one line. It is possible in python. How to do it in ballerina.
[
{
"employeeId": 2413,
"odometerReading": 4089,
"gallons": 21.682,
"gasPrice": 3.46
},
{
"employeeId": 3423,
"odometerReading": 6582,
"gallons": 15.248,
"gasPrice": 4.56
},
{
"employeeId": 2413,
"odometerReading": 4127,
"gallons": 4.221,
"gasPrice": 3.40
},
{
"employeeId": 2413,
"odometerReading": 4349,
"gallons": 11.192,
"gasPrice": 4.10
},
{
"employeeId": 3423,
"odometerReading": 6767,
"gallons": 8.696,
"gasPrice": 3.34
},
{
"employeeId": 2413,
"odometerReading": 4547,
"gallons": 9.197,
"gasPrice": 2.90
}
]
But My file has
[{"employeeId":2413, "gasFillUpCount":4, "totalFuelCost":161.92962, "totalGallons":46.292, "totalMilesAccured":458}, {"employeeId":3423, "gasFillUpCount":2, "totalFuelCost":98.57552, "totalGallons":23.944, "totalMilesAccured":185}]
Adding to the #ThisaruG answer, if you want the indentation(without the escape characters) in the JSON file, you can use the io:fileWriteString instead of io:fileWriteJson. Please refer to the following example,
import ballerina/io;
import thisarug/prettify;
public function main() returns error? {
// Initializes the JSON file path and content.
string jsonFilePath = "./files/jsonFile.json";
json jsonContent = [{"employeeId": 2413, "odometerReading": 4089, "gallons": 21.682, "gasPrice": 3.46}];
string prettified = prettify:prettify(jsonContent);
check io:fileWriteString(jsonFilePath, prettified);
}
You can use the prettify library in Ballerina for this. It is a third party library though.
import thisarug/prettify;
public function main() {
json value = { name: “Sam” };
string prettified = prettify:prettify(value);
}
You can check out the documentation here: https://central.ballerina.io/thisarug/prettify
Related
I am writing a code in python3 where i am struggling with usage of variables with "pyjq", the code works without variables but variables are not getting parsed inside pyjq.
The documentation referred is https://github.com/doloopwhile/pyjq/blob/master/README.md#api
Please check the code given below and suggest -
My code
import json, os
import pyjq
from flask import Flask, request, jsonify
def query_records():
args = {"meta.antivirus.enabled": "true"}
for key, value in args.items():
with open('/tmp/data.txt', 'r') as f:
print (key)
print (value)
data = f.read()
records = json.loads(data)
query = ("." + key)
print (query)
#jq '.[]|select(.meta.antivirus.enabled=="true")' filename.json works,issue with variable substitution in python
match = pyjq.all('.[]|select(["$query"]==$value)', records, vars={"value": value,"query": query})
print (match)
query_records()
Content of file "/tmp/data.txt"
[
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "false"
},
"calories": 230,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "1g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "sandwich-nutrition"
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "true"
},
"calories": 440,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "2g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "random-nutrition"
}
]
Expected output(which works without variables)
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
}
Current output []
seems like some issue with variables not being passed in case of "query" , help would be appreciated.
Edit 1
It works if I hardcode "query" -
match = pyjq.all('.[]|select(.meta.antivirus.enabled==$value)', records, vars={"value": value,"query": query})
but not vice-versa
which probably narrows it down to issue with the variable "query"
JQ is not a necessity and I can use other libraries too,given that json is returned
Variables are intended to be used for values, not for jq expressions (at least not directly).
I think the easiest option here is to go for an fstring:
match = pyjq.all(f'.[]|select({query}==$value)', records, vars={"value": value})
and it probably makes sense to prepend the period inside the fstring:
match = pyjq.all(f'.[]|select(.{key}==$value)', records, vars={"value": value})
This is my JSON file
{
"Research": {
"#xmlns": "http://www.rixml.org/2002/6/RIXML",
"#xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"#createDateTime": "2022-03-25T12:18:03.647Z",
"#language": "eng",
"#researchID": "ABCDEFG_1194968",
"#xsi:schemaLocation": "http://www.rixml.org/2002/6/RIXML http://www.rixml.org/newsite/specifications/v20/RIXML2.xsd",
"Product": {
"#productID": "ABCDEFG_1194968",
"StatusInfo": {
"#currentStatusIndicator": "Yes",
"#statusDateTime": "2022-03-25T12:17:53.686Z",
"#statusType": "Published"
},
"Source": {
"Organization": {
"#primaryIndicator": "Yes",
"#type": "SellSideFirm",
"OrganizationID": [
{
"#idType": "Multex",
"#text": "767"
},
{
"#idType": "cr pep",
"#text": "Americas"
}
],
"OrganizationName": {
"#nameType": "Display",
"#text": "VFGH"
},
"PersonGroup": {
"PersonGroupMember": {
"#primaryIndicator": "Yes",
"Person": {
"#personID": "ABCDEFG12275",
"ContactInfo": {
"#nature": "Business",
"Email": "su.ppe#cr-pep.com"
}
}
}
}
}
},
"Content": {
"Title": "Weekly Insights: March 25, 2022",
"Synopsis": null,
"Resource": {
"#language": "eng",
"#primaryIndicator": "Yes",
"#resourceID": "ABCDEFG_1194968",
"MIMEType": "application/pdf",
"Name": "ABCDEFG_1194968.pdf"
}
}
}
}
}
I need to to parse JSON and get
researchID ,productID and resourceID
I am trying to read file from s3 and then want to parse and process the JSON .
This is the simple code to start with .
import json
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
s3_clientobj = s3.get_object(Bucket='avcdf-bulk-dev', Key='RIXML/josnfile.json')
s3_clientdata = s3_clientobj['Body'].read().decode('utf-8')
#print(s3_clientdata)
print (s3_clientdata[0]['Research'])
Apology in advance as this is very basic question to ask but i am new to python and started with AWS lambda .
Please suggest something which does not required any lambda layers in AWS if possible .
Tree view is
Convert into dict and then try
import json
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
s3_clientobj = s3.get_object(Bucket='avcdf-bulk-dev', Key='RIXML/josnfile.json')
s3_clientdata = s3_clientobj['Body'].read().decode('utf-8')
#print(s3_clientdata)
clientdata_dict = json.loads(s3_clientdata)
print(clientdata_dict)
print(clientdata_dict['Research']['#researchID'])
You can access the above mentioned properties like below,
s3_clientobj = s3.get_object(Bucket='avcdf-bulk-dev', Key='RIXML/josnfile.json')
s3_clientdata = json.loads(s3_clientobj['Body'].read().decode('utf-8'))
print("%s, %s, %s" % (s3_clientdata['Research']['#researchID'], s3_clientdata['Research']['Product']['#productID'], s3_clientdata['Research']['Product']['Content']['Resource']['#resourceID']))
I would like to extract only a small fraction of my JSON response in a .csv file. However, I need to go to 4 levels deep and I am currently only able to go to 3 level deep. My goal is to have a .csv with 3 columns campaign_id, campaign_name, cost_per_click and 3 lines for each of my campaigns.
Original JSON
{
"318429215527453": {
"conversion_events": {
"data": [
{
"id": "djdfhdf",
"name": "Total",
"cost": 328.14,
"metrics_breakdown": {
"data": [
{
"campaign_id": 2364,
"campaign_name": "uk",
"cost_per_click": 1345
},
{
"campaign_id": 7483,
"campaign_name": "fr",
"cost_per_click": 756
},
{
"campaign_id": 8374,
"campaign_name": "spain",
"cost_per_click": 545
},
{
"campaign_id": 2431,
"campaign_name": "ge",
"cost_per_click": 321
}
],
"paging": {
"cursors": {
"after": "MjUZD"
},
"next": "https://graph.facebook.com/v9.0/xxxx"
}
}
}
],
"summary": {
"count": 1,
"metric_date_range": {
"date_range": {
"begin_date": "2021-01-09T00:00:00+0100",
"end_date": "2021-02-08T00:00:00+0100",
"time_zone": "Europe/Paris"
},
"prior_period_date_range": {
"begin_date": "2020-12-10T00:00:00+0100",
"end_date": "2021-01-09T00:00:00+0100"
}
}
}
},
"id": "xxx"
}
}
reformated.py
import json
with open('campaigns.json') as json_file:
data = json.load(json_file)
reformated_json = data['318429215527453']['conversion_events']['data']
with open('data.json', 'w') as outfile:
json.dump(reformated_json, outfile)
I tried to add ['metrics_breakdown'] or another ['data'] at the end of reformated_json but I am getting TypeError: list indices must be integers or slices, not str.
{
"id": "djdfhdf",
"name": "Total",
"cost": 328.14,
"metrics_breakdown": {
"data": [
{
"campaign_id": 2364,
"campaign_name": "uk",
"cost_per_click": 1345,
},
{
"campaign_id": 7483,
"campaign_name": "fr",
"cost_per_click": 756,
},
{
"campaign_id": 8374,
"campaign_name": "spain",
"cost_per_click": 545,
},
{
"campaign_id": 2431,
"campaign_name": "ge",
"cost_per_click": 321,
},
],
"paging": {
"cursors": {
"after": "MjUZD"
},
"next": "https://graph.facebook.com/v9.0/xxxx"
}
}
}
]
import csv
import json
from typing import Dict, List, Union # typing for easy development
# read json function
def read_json(json_path: str) -> Union[Dict, List]:
with open(json_path, 'r') as file_io:
return json.load(file_io)
# write csv function
def write_csv(data: List[Dict], csv_path: str) -> None:
with open(csv_path, 'w') as file:
fieldnames = set().union(*data)
writer = csv.DictWriter(file, fieldnames=fieldnames,
lineterminator='\n')
writer.writeheader()
writer.writerows(data)
# parse campaigns using a comprehension
def parse_campaigns(data: Dict) -> List[Dict]:
return [row
for value in data.values() # first level (conversion events)
for root_data in value['conversion_events']['data'] # conversion events/data
for row in root_data['metrics_breakdown']['data']] # data/metrics_breakdown/data
json_data = read_json('./campaigns.json')
campaign_data = parse_campaigns(json_data)
write_csv(campaign_data, 'campaigns.csv')
campaigns.csv (I copied the data to multiple root dictionary objects):
cost_per_click,campaign_id,campaign_name
1345,2364,uk
756,7483,fr
545,8374,spain
321,2431,ge
1345,2364,uk
756,7483,fr
545,8374,spain
321,2431,ge
The first data subkey contains a single-element list. Dereference with [0] to get the element, then fetch the next layers of keys. Then a DictWriter can be used to write the CSV lines:
import json
import csv
with open('campaigns.json') as json_file:
data = json.load(json_file)
items = data['318429215527453']['conversion_events']['data'][0]['metrics_breakdown']['data']
with open('data.csv', 'w', newline='') as outfile:
w = csv.DictWriter(outfile,fieldnames=items[0].keys())
w.writeheader()
w.writerows(items)
Output:
campaign_id,campaign_name,cost_per_click
2364,uk,1345
7483,fr,756
8374,spain,545
2431,ge,321
I want to extract part of an existing JSON file based on a list of keys and save it into another JSON file
For eg-
{
"211": {
"year": "2020",
"field": "chemistry"
},
"51": {
"year": "2019",
"field":"physics"
},
"5": {
"year": "2014",
"field":"Literature"
}
Lets say the list =[5,51]
Output json file should contain
{
"5": {
"year": "2014",
"field":"Literature"
},
"51": {
"year": "2019",
"field":"physics"
}
}
}
It should not contain data for key 211
I believe this will work for you:
import json
# Open input file and deserialize JSON to a dict
with open("input_file.json", "r", encoding="utf8") as read_file:
input_file_dict = json.load(read_file)
# List of id's you want
inc_id_list = [5,51,2,101]
output_dict = dict()
# Iterate over the input file JSON and only add the items in the list
for id in input_file_dict.keys():
if int(id) in inc_id_list:
output_dict[id] = input_file_dict.get(id)
# Serialize output dict to JSON and write to output file
with open('output_file.json', 'w') as output_json_file:
json.dump(output_dict, output_json_file)
try this :
l= {
"211": {
"year": "2020",
"field": "chemistry"
},
"51": {
"year": "2019",
"field":"physics"
},
"5": {
"year": "2014",
"field":"Literature"
} }
for i,v in l.items():
if(i!="211"):
print(i,v)
It's a dictionary of JSON, with a missing bracket at the end, so that is easily done with a comprehension:
mydict = { "211": { "year": "2020", "field": "chemistry" }, "51": { "year": "2019", "field":"physics" }, "5": { "year": "2014", "field":"Literature" }}
incList = [5, 51]
myAnswer = {k:v for (k,v) in mydict.items() if int(k) in incList}
I have a JSON file with the following format:
const data = [
{
"quantity": "200",
"prodType": "stock",
"symbol": "LOL",
"prodDesc": "Εθνική τράπεζα",
"market": "Greece",
"averageCost": "131,16",
"totalCost": "123,47",
"lastPrice": "121,123",
"value": "123,34",
"positionPercentage": "10",
"valueEUR": "113,23",
"pl": "1300",
"plPercentage": "12",
"plEuro": "1238",
"results": "12-01-2017",
"dividend": "12-03-2017",
"isin": "1234566"
}
]
I want to filter the data using their Product Type.
As a result, im creating an action
export function fetchSearchData(product_type) {
const filtered_data = data.filter(record=>
{
return record.prodType.match(product_type)
});
return {
type: FETCH_SEARCH_DATA,
payload: filtered_data
};
}
But it does not seem to work. Do you have any idea why?