I am new to Powershell.
I have a gz zip file which consists one json will have json data in the below format.
{
"results": [{
"_id": "1",
"AccountPartyRelationshipList": {
"AccountPartyRelationship": [{
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}, {
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}]
},
"AlternateIdList": {
"AlternateId": [{
"value": "A",
"name": "B"
}, {
"value": "C",
"name": "D"
}]
},
"SourceInfo": {
"AccountClosedDate": "A",
"SystemCSIID": "B",
"SourceName": "C",
"AccountLastUpdateDate": "D",
"SourceCSIID": "E",
"AccountOpenDate": "F",
"SystemName": "G"
},
"CustomerAccount": {
"Status": "CLOSED",
"Address": [{
"AddressId": "A",
"AddressType": ["B"],
"FormatCode": "C",
"FreeFormatAddress": "D"
}]
}
}, {
"_id": "2",
"AccountPartyRelationshipList": {
"AccountPartyRelationship": [{
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}, {
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}]
},
"AlternateIdList": {
"AlternateId": [{
"value": "A",
"name": "B"
}, {
"value": "C",
"name": "D"
}]
},
"SourceInfo": {
"AccountClosedDate": "A",
"SystemCSIID": "B",
"SourceName": "B",
"AccountLastUpdateDate": "C",
"SourceCSIID": "D",
"AccountOpenDate": "E",
"SystemName": "F"
},
"CustomerAccount": {
"Status": "CLOSED",
"Address": [{
"AddressId": "A",
"AddressType": ["B"],
"FormatCode": "C",
"FreeFormatAddress": "D"
}]
}
}]}{
"results": [{
"_id": "3",
"AccountPartyRelationshipList": {
"AccountPartyRelationship": [{
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}, {
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}]
},
"AlternateIdList": {
"AlternateId": [{
"value": "A",
"name": "B"
}, {
"value": "C",
"name": "D"
}]
},
"SourceInfo": {
"AccountClosedDate": "A",
"SystemCSIID": "B",
"SourceName": "C",
"AccountLastUpdateDate": "D",
"SourceCSIID": "E",
"AccountOpenDate": "F",
"SystemName": "G"
},
"CustomerAccount": {
"Status": "CLOSED",
"Address": [{
"AddressId": A,
"AddressType": ["B"],
"FormatCode": "C",
"FreeFormatAddress": "D"
}]
}
}, {
"_id": "4",
"AccountPartyRelationshipList": {
"AccountPartyRelationship": [{
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}, {
"PartyIdValue": "A",
"SourceSystem": "B",
"PartyIdType": "C",
"RelationshipType": "D",
"PartyId": "E"
}]
},
"AlternateIdList": {
"AlternateId": [{
"value": "A",
"name": "B"
}, {
"value": "A",
"name": "B"
}, {
"value": "C",
"name": "D"
}]
},
"SourceInfo": {
"AccountClosedDate": "A",
"SystemCSIID": "B",
"SourceName": "B",
"AccountLastUpdateDate": "C",
"SourceCSIID": "D",
"AccountOpenDate": "E",
"SystemName": "F"
},
"CustomerAccount": {
"Status": "CLOSED",
"Address": [{
"AddressId": "A",
"AddressType": ["B"],
"FormatCode": "C",
"FreeFormatAddress": "D"
}]
}
}]}
I know we can find many solutions here in stackoverflow, but as I am new to powershell and the json looks pretty complex for me.
I request to help me on parsing the above json format and make it to a csv data.
The output of CSV and the columns should be as below mentioned list.
Name
_id
AccountPartyRelationship.PartyIdValue
AccountPartyRelationship.SourceSystem
AccountPartyRelationship.PartyIdType
AccountPartyRelationship.RelationshipType
AccountPartyRelationship.PartyId
AlternateIdList.AlternateId.value
AlternateIdList.AlternateId.name
SourceInfo.AccountClosedDate
SourceInfo.SystemCSIID
SourceInfo.SourceName
SourceInfo.AccountLastUpdateDate
SourceInfo.SourceCSIID
SourceInfo.AccountOpenDate
SourceInfo.SystemName
CustomerAccount.Status
CustomerAccount.Address.AddressId
CustomerAccount.Address.AddressType
CustomerAccount.Address.FormatCode
CustomerAccount.Address.FreeFormatAddress
Thanks,
Mohan V.
Related
I'm want to make my json to csv so that i can upload it on google sheets and make it as json api. Whenever i have change data i will just change it on google sheets. But I'm having problems on converting my json file to csv because it changes the variables whenever i convert it. I'm using https://toolslick.com/csv-to-json-converter to convert my json file to csv.
What is the best way to convert json nested to csv ?
JSON
{
"options": [
{
"id": "1",
"value": "Jumbo",
"shortcut": "J",
"textColor": "#FFFFFF",
"backgroundColor": "#00000"
},
{
"id": "2",
"value": "Hot",
"shortcut": "D",
"textColor": "#FFFFFF",
"backgroundColor": "#FFFFFF"
}
],
"categories": [
{
"id": "1",
"order": 1,
"name": "First Category",
"active": true
},
{
"id": "2",
"order": 2,
"name": "Second Category",
"shortcut": "MT",
"active": true
}
],
"products": [
{
"id": "03c6787c-fc2a-4aa8-93a3-5e0f0f98cfb2",
"categoryId": "1",
"name": "First Product",
"shortcut": "First",
"options": [
{
"optionId": "1",
"price": 23
},
{
"optionId": "2",
"price": 45
}
],
"active": true
},
{
"id": "e8669cea-4c9c-431c-84ba-0b014f0f9bc2",
"categoryId": "2",
"name": "Second Product",
"shortcut": "Second",
"options": [
{
"optionId": "1",
"price": 11
},
{
"optionId": "2",
"price": 20
}
],
"active": true
}
],
"discounts": [
{
"id": "1",
"name": "S",
"type": 1,
"amount": 20,
"active": true
},
{
"id": "2",
"name": "P",
"type": 1,
"amount": 20,
"active": true
},
{
"id": "3",
"name": "G",
"type": 2,
"amount": 5,
"active": true
}
]
}
Using python, this can be easily done or almost done. Maybe this code will help you in some way to understand that.
import json,csv
data = []
with open('your_json_file_here.json') as file:
for line in file:
data.append(json.loads(line))
length = len(data)
with open('create_new_file.csv','w') as f:
writer = csv.writer(f)
writers = csv.DictWriter(f, fieldnames=['header1','header2'])
writers.writeheader()
for iter in range(length):
writer.writerow((data[iter]['specific_col_name1'],data[iter]['specific_col_name2']))
f.close()
my data is structured as below:
Basically it contains Way Point information from google distance matrix. i need to find the routes that start from point C and ending at point E. if i use $and operator and query for
db.Routes.find( {
$and : [
{routes.legs.start_address:"C"},
{routes.legs.end_address:"E"}
]
}
I still get both this results. what are the possible approaches do i have for to resolve this. I'm trying Mongo for the first time so any help or guidance is much appreciated.
[
{
"_id": {
"$oid": "596e004151cc92046c28dd39"
},
"routes": [
{
"legs": [
{
"end_address": "B",
"start_address": "A"
},
{
"end_address": "C",
"start_address": "B"
},
{
"end_address": "D",
"start_address": "C"
},
{
"end_address": "E",
"start_address": "D"
}
]
}
],
"status": "OK",
"Date": {
"$date": "2017-07-18T12:34:07.781Z"
}
},
{
"_id": {
"$oid": "596e007d51cc9231a8117607"
},
"routes": [
{
"legs": [
{
"end_address": "E",
"start_address": "F"
},
{
"end_address": "D",
"start_address": "E"
},
{
"end_address": "C",
"start_address": "D"
},
{
"end_address": "B",
"start_address": "C"
}
]
}
],
"status": "OK",
"Date": {
"$date": "2017-07-18T12:35:09.121Z"
}
}
]
I need to convert a json(got after multiple calls and then merged it) into a specific structure.
here is my input json payload
[{
"shops": [{
"shop": {
"code": "AU5",
"streetName": "a",
"city": "a",
"district": "a",
"state": "a",
"postalCode": "a",
"country": "a"
}
}, {
"shop": {
"code": "b",
"streetName": "b",
"city": "b",
"district": "b",
"state": "b",
"postalCode": "b",
"country": "b"
}
}]
},
[
[{
"salesOffice": {
"shop": {
"code": "AU5"
},
"office": "MEL",
"branch": "MEL",
"district": "SPR",
"subRegion": "SPR",
"region": "AP"
}
}],
[{
"salesOffice": {
"shop": {
"code": "b"
},
"office": "999",
"branch": "999",
"district": "999",
"subRegion": "999",
"region": "999"
}
}
]
]]
below is the expected output json
{
"shops": [
{
"shop": {
"code": "AU5",
"streetName": "a",
"city": "a",
"district": "a",
"state": "a",
"postalCode": "a",
"country": "a",
"salesOffice": {
"office": "MEL",
"branch": "MEL",
"district": "SPR",
"subRegion": "SPR",
"region": "AP"
}
}
},
{
"shop": {
"code": "b",
"streetName": "b",
"city": "b",
"district": "b",
"state": "b",
"postalCode": "b",
"country": "b",
"salesOffice": {
"office": "999",
"branch": "999",
"district": "999",
"subRegion": "999",
"region": "999"
}
}
}
]}
while transformation, 'code' inside shop should match with 'code' inside salesOffice>>shop>>'code'
below is Json Schema for the output payload(should validate against the output)
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"shops": {
"type": "array",
"items": {
"type": "object",
"properties": {
"shop": {
"type": "object",
"properties": {
"code": {
"type": "string"
},
"streetName": {
"type": "string"
},
"city": {
"type": "string"
},
"district": {
"type": "string"
},
"state": {
"type": "string"
},
"postalCode": {
"type": "string"
},
"country": {
"type": "string"
}
},
"salesOffice": {
"type": "object",
"properties": {
"office": {
"type": "string"
},
"branch": {
"type": "string"
},
"district": {
"type": "string"
},
"subRegion": {
"type": "string"
},
"region": {
"type": "string"
}
}
},
"required": [
"code",
"streetName",
"city",
"district",
"state",
"postalCode",
"country",
"salesOffice"
]
}
},
"required": [
"shop"
]
}
}
},
"required": [
"shops"
]}
Any solution or any pointer would be a great help
Refer below dataweave:
%dw 1.0
%output application/json
---
{
shops: using (salesOffice = payload[1]..salesOffice)( payload[0].shops map {
shop :
{
code: $.shop.code,
streetName:$.shop.streetName,
city:$.shop.city,
district:$.shop.district,
state:$.shop.state,
postalCode:$.shop.postalCode,
country:$.shop.country,
salesOffice: using (code= $.shop.code) ( salesOffice[?(code == $.shop.code)] map {
office:$.office,
branch:$.branch,
district:$.district,
subRegion:$.subRegion,
region:$.region
})[0]
}
})
}
You can use Validate JSON Schema mule components.
There are the same JSON data.
I have tried to show the value in Highchart to extract values from the JSON data.
scope.jsondata = [
{
"Name": "A",
"Categories": "03.01",
"timestamp":"133409520000",
"Locate": "1",
"Value": 90
},
{
"Name": "A",
"Categories": "03.02",
"timestamp":"133409530000",
"Locate": "1",
"Value": 110
},{
"Name": "A",
"Categories": "03.03",
"timestamp":"133409630000",
"Locate": "1",
"Value": 120
},{
"Name": "A",
"Categories": "03.04",
"timestamp":"133409730000",
"Locate": "1",
"Value": 140
},{
"Name": "A",
"Categories": "03.05",
"timestamp":"133409830000",
"Locate": "1",
"Value": 70
},{
"Name": "A",
"Categories": "03.06",
"timestamp":"133409930000",
"Locate": "1",
"Value": 70
},{
"Name": "A",
"Categories": "03.07",
"timestamp":"133410530000",
"Locate": "1",
"Value": 70
},{
"Name": "A",
"Categories": "03.08",
"timestamp":"133420530000",
"Locate": "1",
"Value": 70
},{
"Name": "A",
"Categories": "03.09",
"timestamp":"133430530000",
"Locate": "1",
"Value": 70
},{
"Name": "B",
"Categories": "03.05",
"Locate": "2",
"Value": 70
}
];
However, Y values in the chart, but to go to the Data, Timestamp does get into the x value.
What should I Can give a timestamp value to the value of output x axis?
I have this database http://ashleyw.co.uk/project/food-nutrient-database, and I would like to extract the group Dairy and Egg Products into a separate Json file.
Also, within the group some of the data is triplicated. For example:
{
"id": 1008,
"description": "Cheese, caraway",
"tags": [],
"manufacturer": "",
"group": "Dairy and Egg Products",
"portions": [
{
"amount": 1,
"unit": "oz",
"grams": 28.35
}
],
"nutrients": [
{
"value": 25.18,
"units": "g",
"description": "Protein",
"group": "Composition"
},
{
"value": 29.2,
"units": "g",
"description": "Total lipid (fat)",
"group": "Composition"
},
{
"value": 3.06,
"units": "g",
"description": "Carbohydrate, by difference",
"group": "Composition"
},
{
"value": 25.18,
"units": "g",
"description": "Protein",
"group": "Composition"
},
{
"value": 29.2,
"units": "g",
"description": "Total lipid (fat)",
"group": "Composition"
},
{
"value": 3.06,
"units": "g",
"description": "Carbohydrate, by difference",
"group": "Composition"
},
{
"value": 25.18,
"units": "g",
"description": "Protein",
"group": "Composition"
},
{
"value": 29.2,
"units": "g",
"description": "Total lipid (fat)",
"group": "Composition"
},
{
"value": 3.06,
"units": "g",
"description": "Carbohydrate, by difference",
"group": "Composition"
}
]
}
Every child of the nutrients node is triplicated. How can the extras be stripped away?
Using Python:
import json
with open("data.json") as f, open("nutrients.json", "w") as g:
d = json.loads(f.read())
json.dump(d["nutrients"], g)