Related
I just need to query all information of the doctor in the first index.
Here is my sample XML data
Here is my sample JSON data:
Both XML and JSON data are same. I just converted the XML to JSON format.
I have more documents like this.
I tried this query in this image
this query showing all information but I just need first doctors information.
I also tried in command prompt.Here is the query done in cmd
Here is my sample JSON code block: but I have more similar documents
{
"doctors": {
"doctor":[
{
"ssn": "257-79-xxxx",
"name": "Mavis Bxx",
"address": "xxxx Rusk Drive",
"country": "France",
"email": "",
"phone": "",
"patients": {
"patient": [
{
"gender": "Male",
"name": "Itch xxxx",
"address": "xx Cottonwood Avenue",
"revenue": "254",
"_country": "Spain",
"_id": "27"
},
{
"gender": "Male",
"name": "Damon xxxxx",
"address": "xx David Trail",
"revenue": "370",
"_country": "Germany",
"_id": "21"
}
]
},
"_id": "6"
},
{
"ssn": "179-45-xxxx",
"name": "Tobie Conxxxx",
"address": "x Comanche Center",
"country": "Spain",
"email": "tconringh#xxx.xxx",
"phone": "+86 998 262 xxxx",
"patients": {
"patient": {
"gender": "Male",
"name": "Vergil Tome",
"address": "x Melody Drive",
"revenue": "254",
"_country": "Germany",
"_id": "15"
}
},
"_id": "18"
},
{
"ssn": "777-59-xxxx",
"name": "Gertrud Macxxxx",
"address": "x Buell Drive",
"country": "USA",
"email": "gmaclaig2#xxx.com",
"phone": "+62 975 394 xxxx",
"patients": {
"patient": [
{
"gender": "Non-binary",
"name": "Dre Skxxxx",
"address": "x Becker Circle",
"revenue": "400",
"_country": "Germany",
"_id": "20"
},
{
"gender": "Female",
"name": "Arleyne Lestxxxx",
"address": "xx Farragut Court",
"revenue": "225",
"_country": "France",
"_id": "22"
}
]
},
"_id": "3"
}
You needs to change your collect data format. Your current format is single big size data. Not a quarriable format.
So
From current format
{
"doctors": {
"docker": [
{
...
"patients": {
"patient": [
{
...
},
{
...
}
]
},
},
{
...
}
]
}
}
To this format
[
{
...
"patients": [
{
...
},
{
...
}
]
},
{
...
}
]
Test JSON
[
{
"ssn": "2xx-7x-4xxx",
"name": "Mavis Bxxxed",
"address": "9xxx Rusk Drive",
"country": "France",
"email": "",
"phone": "",
"patients": [
{
"gender": "Male",
"name": "Itch Txxx",
"address": "5xx Cottonwood Avenue",
"revenue": "25x",
"_country": "Spain",
"_id": "27"
},
{
"gender": "Male",
"name": "Damon Wxxx",
"address": "09xx Dxxxx Trail",
"revenue": "370",
"_country": "Germany",
"_id": "21"
}
],
"_id": "6"
},
{
"ssn": "19-45-xxxx",
"name": "Tobie Coxxxx",
"address": "8x Comxxxx Center",
"country": "Spain",
"email": "tconxxxxx#usa.gov",
"phone": "",
"patients": [
{
"gender": "Male",
"name": "Some one",
"address": "1 Mexxxx Drive",
"revenue": "254",
"_country": "Germany",
"_id": "15"
}
],
"_id": "18"
}
]
In Compass View after add data upper data
Find doctor by name query
{ name: "Mavis Bxxxed" }
Find doctor by patient name
{ "patients.name": "Some one" }
If same search with this query
{ "patients.name": "Itch Txxx" }
OR
{ "patients.name": "Damon Wxxx" }
will same result to find the first docker (Mavis Bxxxed)
Due to it's patients start [ and end ]. it is array type a single size of data.
You can filter by project option
{
patients: {
$filter:
{
input: "$patients",
cond: { $eq: [ "$$patient.name", "Itch Txxx"] },
as: "patient"
}
}
}
OR shows only address
{ "patients.address" : 1 }
I would like to transform my nested json message and take only required filed using Jolt spec-
My Input JSON :
{
"results": [
{
"user": {
"gender": "male",
"name": {
"title": "mr",
"first": "moritz",
"last": "zimmer"
},
"location": {
"street": "3667 breslauer straße",
"city": "gera",
"state": "mecklenburg-vorpommern",
"zip": 35662
},
"email": "moritz.zimmer#example.com",
"username": "purplegorilla905",
"password": "kkkkk",
"salt": "O5KFiJMB",
"md5": "d5659f39f9102452dca5e9afbfe06f07",
"sha1": "bb8790f5e22a8a4c33f12d623cd7f5c45adba275",
"sha256": "5fc5dd30d75c323404474e2c5bec2372a145f363742162b36a6414aa10e2b5e6",
"registered": 968202500,
"dob": 68248460,
"phone": "0450-7475136",
"cell": "0179-3008357",
"picture": {
"large": "https://randomuser.me/api/portraits/men/15.jpg",
"medium": "https://randomuser.me/api/portraits/med/men/15.jpg",
"thumbnail": "https://randomuser.me/api/portraits/thumb/men/15.jpg"
}
}
},
{
"user": {
"gender": "female",
"name": {
"title": "ms",
"first": "julia",
"last": "bayer"
},
"location": {
"street": "5913 burgstraße",
"city": "mühldorf a. inn",
"state": "bremen",
"zip": 17247
},
"email": "julia.bayer#example.com",
"username": "bluedog987",
"password": "sheba1",
"salt": "qJ3KuIu7",
"md5": "c29e59c439ebc41b6772344b53863a17",
"sha1": "7fdb58f22eb85cb89ffb0c2cbb13bbf3bbb37238",
"sha256": "d6114912b8476e1c825a3487feb1cc7319c0c26f29f583ccd6864fa4f11214db",
"registered": 1224249359,
"dob": 1307176045,
"phone": "0846-0635929",
"cell": "0172-0881456",
"picture": {
"large": "https://randomuser.me/api/portraits/women/15.jpg",
"medium": "https://randomuser.me/api/portraits/med/women/15.jpg",
"thumbnail": "https://randomuser.me/api/portraits/thumb/women/15.jpg"
}
}
},
{
"user": {
"gender": "male",
"name": {
"title": "mr",
"first": "sascha",
"last": "vogel"
},
"location": {
"street": "8276 lerchenweg",
"city": "straubing",
"state": "niedersachsen",
"zip": 73954
},
"email": "sascha.vogel#example.com",
"username": "purplefrog135",
"password": "picard",
"salt": "kBv4gmHR",
"md5": "79a388de1832ab7f14f4e5ddf0d88012",
"sha1": "8fc12f95217bf04eaf7e950154485dd35b84c65e",
"sha256": "a5df2bfd3ee84a869c43f92fe28ce5f56f7700bc75ce2e12bbee7cad2524af82",
"registered": 950549849,
"dob": 310749405,
"phone": "0098-5667969",
"cell": "0179-8473990",
"picture": {
"large": "https://randomuser.me/api/portraits/men/83.jpg",
"medium": "https://randomuser.me/api/portraits/med/men/83.jpg",
"thumbnail": "https://randomuser.me/api/portraits/thumb/men/83.jpg"
}
}
},
{
"user": {
"gender": "female",
"name": {
"title": "mrs",
"first": "luisa",
"last": "döring"
},
"location": {
"street": "9027 am bahnhof",
"city": "salzgitter",
"state": "bayern",
"zip": 12419
},
"email": "luisa.döring#example.com",
"username": "purpleelephant855",
"password": "californ",
"salt": "gsnCNl8c",
"md5": "905d16801137f3368277889a783c02f3",
"sha1": "a5027752e9228bbc1e99fa56a6f2d09a1901e7e2",
"sha256": "51b7e09a415fef7e71eb3aec2721e91d0b51c2e264a7cdd41f3a692777771de3",
"registered": 1068886905,
"dob": 733574914,
"phone": "0648-0740938",
"cell": "0172-1199322",
"picture": {
"large": "https://randomuser.me/api/portraits/women/65.jpg",
"medium": "https://randomuser.me/api/portraits/med/women/65.jpg",
"thumbnail": "https://randomuser.me/api/portraits/thumb/women/65.jpg"
}
}
}
],
"nationality": "DE",
"seed": "23e269027e0ea51404",
"version": "0.8"
}
Expected output :
[
{
"Gender": "male",
"FirstName": "moritz",
"LastName": "zimmer",
"City": "gera",
"State": "mecklenburg-vorpommern",
"Email": "moritz.zimmer#example.com",
"DateOfBirth": 68248460,
"MobileNo": "0179-3008357",
"Nationality": "DE"
},
{
"Gender": "female",
"FirstName": "julia",
"LastName": "bayer",
"City": "mühldorf a. inn",
"State": "bremen",
"Email": "julia.bayer#example.com",
"DateOfBirth": 1307176045,
"MobileNo": "0172-0881456",
"Nationality": "DE"
},
{
"Gender": "male",
"FirstName": "sascha",
"LastName": "vogel",
"City": "straubing",
"State": "niedersachsen",
"Email": "sascha.vogel#example.com",
"DateOfBirth": 310749405,
"MobileNo": "0179-8473990",
"Nationality": "DE"
},
{
"Gender": "female",
"FirstName": "luisa",
"LastName": "döring",
"City": "salzgitter",
"State": "bayern",
"Email": "luisa.döring#example.com",
"DateOfBirth": 733574914,
"MobileNo": "0172-1199322",
"Nationality": "DE"
}
]
I am using below Spec and output not coming as per expectation -
My Spec :
[
{
"operation": "shift",
"spec": {
"results": {
"*": {
"user": {
"gender": "[&2].Gender",
"name": {
"first": "[&3].FirstName",
"last": "[&3].LastName"
},
"location": {
"city": "[&3].City",
"state": "[&3].State"
},
"email": "[&2].Email",
"dob": "[&2].DateOfBirth",
"cell": "[&2].MobileNo"
}
}
}
}
}
]
Below my Spec output which is not as expected -
[
{
"Gender": "male",
"FirstName": "moritz",
"LastName": "zimmer",
"City": "gera",
"State": "mecklenburg-vorpommern",
"Email": "moritz.zimmer#example.com",
"DateOfBirth": 68248460,
"MobileNo": "0179-3008357"
},
{
"Gender": "female",
"FirstName": "julia",
"LastName": "bayer",
"City": "mühldorf a. inn",
"State": "bremen",
"Email": "julia.bayer#example.com",
"DateOfBirth": 1307176045,
"MobileNo": "0172-0881456"
},
{
"Gender": "male",
"FirstName": "sascha",
"LastName": "vogel",
"City": "straubing",
"State": "niedersachsen",
"Email": "sascha.vogel#example.com",
"DateOfBirth": 310749405,
"MobileNo": "0179-8473990"
},
{
"Gender": "female",
"FirstName": "luisa",
"LastName": "döring",
"City": "salzgitter",
"State": "bayern",
"Email": "luisa.döring#example.com",
"DateOfBirth": 733574914,
"MobileNo": "0172-1199322"
}
]
I tried many option but nationality is not coming inside as my expected output.
Please help here in Jolt transformation
Tip : Looking at outer part of the results array by using "*": "&" while looking at the inner part of that, and combining them under name key which seems to be unique as a grouping by criteria for the desired result set such as
[
{
"operation": "shift",
"spec": {
"results": {
"*": {
"user": {
"*": "&"
}
}
},
"*": "&"
}
},
{
"operation": "shift",
"spec": {
"name": {
"*": {
"#(2,gender[&])": "[&].Gender",
"#(0,first)": "[&1].First&2",
"#(0,last)": "[&1].Last&2",
"#(2,location[&].city)": "[&].City",
"#(2,location[&].state)": "[&].State",
"#(2,email[&])": "[&].Email",
"#(2,dob[&])": "[&].DateOfBirth",
"#(2,phone[&])": "[&].MobileNo",
"#(2,nationality)": "[&].Nationality"
}
}
}
}
]
I have a JSON file with nested elements that I'm trying to manipulate into an unnested JSON file. How do I do this?
Using js, I have tried to isolate the percentages, which I was able to do. I'm not sure how to rename the percentage field. I've seen instances where it seems like value.gender or value.grade should work, but I'm also not sure how to combine it.
jq '.data[] | .id as $id | (.demographics[] | .percentage as $percentage | .gender as $gender | .grade as $grade | {"id":$id, "percentage":$percentage})' test2.json
From here, I want to be able to rename the percentage field to the gender and grades values. I then want to group by id.
Here is the original JSON file (test2.json):
{
"data": [{
"id": "abc",
"students": "elementary",
"demographics": [{
"grade": "K-2",
"percentage": "0.1",
"gender": "unspecified"
},
{
"grade": "K-2",
"gender": "male",
"percentage": "0.5"
},
{
"gender": "female",
"percentage": "0.4",
"grade": "K-2"
},
{
"grade": "3-6",
"percentage": "0.3",
"gender": "male"
},
{
"percentage": "0.2",
"gender": "unspecified",
"grade": "3-6"
},
{
"grade": "3-6",
"gender": "female",
"percentage": "0.5"
}
],
"neighborhood_name": [{
"percentage": "0.5",
"neighborhood": "atwood"
},
{
"region": "bluff",
"percentage": "0.5"
}
]
},
{
"id": "def",
"students": "midhigh",
"demographics": [{
"grade": "7-9",
"percentage": "0.2",
"gender": "unspecified"
},
{
"grade": "7-9",
"gender": "male",
"percentage": "0.2"
},
{
"gender": "female",
"percentage": "0.6",
"grade": "7-9"
},
{
"grade": "10-12",
"percentage": "0.1",
"gender": "male"
},
{
"percentage": "0.1",
"gender": "unspecified",
"grade": "10-12"
},
{
"grade": "10-12",
"gender": "female",
"percentage": "0.8"
}
],
"neighborhood_name": [{
"percentage": "0.2",
"neighborhood": "atwood"
},
{
"region": "bluff",
"percentage": "0.8"
}
]
}
]
}
Here is what I expect:
{
"id": "abc",
"students": "elementary",
"demo_K-2_unspecified": "0.1",
"demo_K-2_male": "0.5",
"demo_K-2_female": "0.4",
"demo_3-6_male": "0.3",
"demo_3-6_unspecified": "0.6",
"demo_3-6_female": "0.5",
},
{
"id": "def",
"students": "midhigh",
"demo_7-9_unspecified": "0.2",
"demo_7-9_male": "0.2",
"demo_7-9_female": "0.6",
"demo_10-12_male": "0.1",
"demo_10-12_unspecified": "0.1",
"demo_10-12_female": "0.8",
}
With the sample data, the following filter produces the desired output:
.data[]
| {id, students} as $ix
| .demographics
| map( {"demo_\(.grade)_\(.gender)": .percentage} )
| $ix + add
The main idea here is to use map to create a list of the key-value pairs, so that the composite object can then easily be created using add.
As a one-liner
jq '.data[] | {id,students} + (.demographics | map( {"demo_\(.grade)_\(.gender)": .percentage} ) | add)' test2.json
i wanted to simulate a http request by reading an Json File and its stored data. Here is how i tryed it
Data.json:
{ "jsonData": {
"data1": [{
"age": "18",
"name": "bernd",
"gender": "1"
}],
"data2": [{ "age": "18", "name": "bernd", "gender": "1"},
{ "age": "18", "name": "bernd", "gender": "1"},
{ "age": "15", "name": "bernd", "gender": "1"},
{ "age": "18", "name": "marting", "gender": "1"},
{ "age": "25", "name": "bernd", "gender": "1"},
{ "age": "29", "name": "bernd", "gender": "1"},
{ "age": "18", "name": "mike", "gender": "1"},
{ "age": "45", "name": "bernd", "gender": "1"},
{ "age": "18", "name": "bernd", "gender": "1"}
],
"data3": [{ "age": "18", "name": "mike", "gender": "1"},
{ "age": "18", "name": "monika", "gender": "2"},
{ "age": "18", "name": "martin", "gender": "1"},
{ "age": "18", "name": "monika", "gender": "2"},
{ "age": "18", "name": "monika", "gender": "2"},
{ "age": "18", "name": "monika", "gender": "2"},
{ "age": "18", "name": "bernd", "gender": "1"},
{ "age": "18", "name": "bernd", "gender": "1"},
{ "age": "18", "name": "lisa", "gender": "2"}
]}}
and a service which recieves the data
data.service:
getServers() {
return this.http.get('../assets/Data.json')
.subscribe((res: Response) => {
this.testData = res.json().data2;
} );}
The plan is to get for example the "data2" object Array into "this.testData". I tryed different versions of http.get, but just managed to get the Json object without successfully converting it into a object[].
I would really appreciate your help guys.
I have a mule flow which goes FILE -> CHOICE -> DATAMAPPER -> LOGGER
The FILE endpoint processes a csv file with 40 records
The DATAMAPPER converts the csv into JSON and shows all 40 records in JSON format when run in preview
The LOGGER shows that the payload as a byte and not JSON and when I convert JSON to object it shows the payload in JSON format but there is only 1 record.
My queries:
1. How do you setup the workflow to process all of the 40 records and convert all to JSON and output all from the flow?
Is it correct that the output is a byte because the datamapper output is JSON so I was expecting to see all records in JSON format in the payload when looking at it in debug mode?
You can do something like this.
<flow name="csv-to-jsonFlow">
<file:inbound-endpoint path="/src/main/resources/csv" connector-ref="File" responseTimeout="10000" doc:name="File">
<file:filename-regex-filter pattern=".*csv" caseSensitive="true"/>
</file:inbound-endpoint>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
payload]]></dw:set-payload>
</dw:transform-message>
<object-to-string-transformer doc:name="Object to String"/>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
Sample CSV file:
name,age,gender
janos,29,male
juan,40,male
pedro,22,male
carla,34,female
jose,30,male
anne,25,female
rey,33,male
joyce,27,female
pia,20,female
ard,27,male
janos,29,male
juan,40,male
pedro,22,male
carla,34,female
jose,30,male
anne,25,female
rey,33,male
joyce,27,female
pia,20,female
ard,27,male
janos,29,male
juan,40,male
pedro,22,male
carla,34,female
jose,30,male
anne,25,female
rey,33,male
joyce,27,female
pia,20,female
ard,27,male
janos,29,male
juan,40,male
pedro,22,male
carla,34,female
jose,30,male
anne,25,female
rey,33,male
joyce,27,female
pia,20,female
ard,27,male
Sample Output:
[
{
"name": "janos",
"age": "29",
"gender": "male"
},
{
"name": "juan",
"age": "40",
"gender": "male"
},
{
"name": "pedro",
"age": "22",
"gender": "male"
},
{
"name": "carla",
"age": "34",
"gender": "female"
},
{
"name": "jose",
"age": "30",
"gender": "male"
},
{
"name": "anne",
"age": "25",
"gender": "female"
},
{
"name": "rey",
"age": "33",
"gender": "male"
},
{
"name": "joyce",
"age": "27",
"gender": "female"
},
{
"name": "pia",
"age": "20",
"gender": "female"
},
{
"name": "ard",
"age": "27",
"gender": "male"
},
{
"name": "janos",
"age": "29",
"gender": "male"
},
{
"name": "juan",
"age": "40",
"gender": "male"
},
{
"name": "pedro",
"age": "22",
"gender": "male"
},
{
"name": "carla",
"age": "34",
"gender": "female"
},
{
"name": "jose",
"age": "30",
"gender": "male"
},
{
"name": "anne",
"age": "25",
"gender": "female"
},
{
"name": "rey",
"age": "33",
"gender": "male"
},
{
"name": "joyce",
"age": "27",
"gender": "female"
},
{
"name": "pia",
"age": "20",
"gender": "female"
},
{
"name": "ard",
"age": "27",
"gender": "male"
},
{
"name": "janos",
"age": "29",
"gender": "male"
},
{
"name": "juan",
"age": "40",
"gender": "male"
},
{
"name": "pedro",
"age": "22",
"gender": "male"
},
{
"name": "carla",
"age": "34",
"gender": "female"
},
{
"name": "jose",
"age": "30",
"gender": "male"
},
{
"name": "anne",
"age": "25",
"gender": "female"
},
{
"name": "rey",
"age": "33",
"gender": "male"
},
{
"name": "joyce",
"age": "27",
"gender": "female"
},
{
"name": "pia",
"age": "20",
"gender": "female"
},
{
"name": "ard",
"age": "27",
"gender": "male"
},
{
"name": "janos",
"age": "29",
"gender": "male"
},
{
"name": "juan",
"age": "40",
"gender": "male"
},
{
"name": "pedro",
"age": "22",
"gender": "male"
},
{
"name": "carla",
"age": "34",
"gender": "female"
},
{
"name": "jose",
"age": "30",
"gender": "male"
},
{
"name": "anne",
"age": "25",
"gender": "female"
},
{
"name": "rey",
"age": "33",
"gender": "male"
},
{
"name": "joyce",
"age": "27",
"gender": "female"
},
{
"name": "pia",
"age": "20",
"gender": "female"
},
{
"name": "ard",
"age": "27",
"gender": "male"
}
]
Hope this helps.