Related
I just need to query all information of the doctor in the first index.
Here is my sample XML data
Here is my sample JSON data:
Both XML and JSON data are same. I just converted the XML to JSON format.
I have more documents like this.
I tried this query in this image
this query showing all information but I just need first doctors information.
I also tried in command prompt.Here is the query done in cmd
Here is my sample JSON code block: but I have more similar documents
{
"doctors": {
"doctor":[
{
"ssn": "257-79-xxxx",
"name": "Mavis Bxx",
"address": "xxxx Rusk Drive",
"country": "France",
"email": "",
"phone": "",
"patients": {
"patient": [
{
"gender": "Male",
"name": "Itch xxxx",
"address": "xx Cottonwood Avenue",
"revenue": "254",
"_country": "Spain",
"_id": "27"
},
{
"gender": "Male",
"name": "Damon xxxxx",
"address": "xx David Trail",
"revenue": "370",
"_country": "Germany",
"_id": "21"
}
]
},
"_id": "6"
},
{
"ssn": "179-45-xxxx",
"name": "Tobie Conxxxx",
"address": "x Comanche Center",
"country": "Spain",
"email": "tconringh#xxx.xxx",
"phone": "+86 998 262 xxxx",
"patients": {
"patient": {
"gender": "Male",
"name": "Vergil Tome",
"address": "x Melody Drive",
"revenue": "254",
"_country": "Germany",
"_id": "15"
}
},
"_id": "18"
},
{
"ssn": "777-59-xxxx",
"name": "Gertrud Macxxxx",
"address": "x Buell Drive",
"country": "USA",
"email": "gmaclaig2#xxx.com",
"phone": "+62 975 394 xxxx",
"patients": {
"patient": [
{
"gender": "Non-binary",
"name": "Dre Skxxxx",
"address": "x Becker Circle",
"revenue": "400",
"_country": "Germany",
"_id": "20"
},
{
"gender": "Female",
"name": "Arleyne Lestxxxx",
"address": "xx Farragut Court",
"revenue": "225",
"_country": "France",
"_id": "22"
}
]
},
"_id": "3"
}
You needs to change your collect data format. Your current format is single big size data. Not a quarriable format.
So
From current format
{
"doctors": {
"docker": [
{
...
"patients": {
"patient": [
{
...
},
{
...
}
]
},
},
{
...
}
]
}
}
To this format
[
{
...
"patients": [
{
...
},
{
...
}
]
},
{
...
}
]
Test JSON
[
{
"ssn": "2xx-7x-4xxx",
"name": "Mavis Bxxxed",
"address": "9xxx Rusk Drive",
"country": "France",
"email": "",
"phone": "",
"patients": [
{
"gender": "Male",
"name": "Itch Txxx",
"address": "5xx Cottonwood Avenue",
"revenue": "25x",
"_country": "Spain",
"_id": "27"
},
{
"gender": "Male",
"name": "Damon Wxxx",
"address": "09xx Dxxxx Trail",
"revenue": "370",
"_country": "Germany",
"_id": "21"
}
],
"_id": "6"
},
{
"ssn": "19-45-xxxx",
"name": "Tobie Coxxxx",
"address": "8x Comxxxx Center",
"country": "Spain",
"email": "tconxxxxx#usa.gov",
"phone": "",
"patients": [
{
"gender": "Male",
"name": "Some one",
"address": "1 Mexxxx Drive",
"revenue": "254",
"_country": "Germany",
"_id": "15"
}
],
"_id": "18"
}
]
In Compass View after add data upper data
Find doctor by name query
{ name: "Mavis Bxxxed" }
Find doctor by patient name
{ "patients.name": "Some one" }
If same search with this query
{ "patients.name": "Itch Txxx" }
OR
{ "patients.name": "Damon Wxxx" }
will same result to find the first docker (Mavis Bxxxed)
Due to it's patients start [ and end ]. it is array type a single size of data.
You can filter by project option
{
patients: {
$filter:
{
input: "$patients",
cond: { $eq: [ "$$patient.name", "Itch Txxx"] },
as: "patient"
}
}
}
OR shows only address
{ "patients.address" : 1 }
I have the following data frame, df1:
A B C
123 B1 C1
456 B2 C2
And data frame df2:
A
[
{
"id": "123",
"details": {
"id": "123",
"color": null,
"param_1": {
"name": "mike"
},
"location": "US",
"items": [
{
"item_1": "#227858",
"offer_id": null,
"item_details": {
"detials_1": [{ "notes": "other:", "quantity": 1 }]
}
}
],
"version": 1,
}
}
]
[
{
"id": "456",
"details": {
"id": "456",
"color": null,
"param_1": {
"name": "james"
},
"location": "KR",
"items": [
{
"item_1": "#2221",
"offer_id": null,
"item_details": {
"detials_1": [{ "notes": "other", "quantity": 1 }]
}
}
],
"version": 2,
}
}
]
I want to find all values in df1[A] inside the JSON found inside df2[A] under the first instance of the id parameter. Once found, I want to replace the NULL values inside the color parameter with the df1[B] and offer_id with df1[C].
The output should create a new column with the appended values:
df2[B]:
[
{
"id": "123",
"details": {
"id": "123",
"color": B1,
"param_1": {
"name": "mike"
},
"location": "US",
"items": [
{
"item_1": "#227858",
"offer_id": C1,
"item_details": {
"detials_1": [{ "notes": "other:", "quantity": 1 }]
}
}
],
"version": 1,
}
}
]
[
{
"id": "456",
"details": {
"id": "456",
"color": B2,
"param_1": {
"name": "james"
},
"location": "KR",
"items": [
{
"item_1": "#2221",
"offer_id": C2,
"item_details": {
"detials_1": [{ "notes": "other", "quantity": 1 }]
}
}
],
"version": 2,
}
}
]
I just started researching how to approach this, but I need guidance on the most efficient way. Any insight would be greatly appreciated.
This question already has answers here:
How to use jq when the variable has reserved characters?
(3 answers)
Closed 6 months ago.
I have a JSON file that I am trying to query with jq. I am unable to retrieve the observations. I am trying to retieve each of the "observations using the following command and not able to get to the result:
cat sample3.json | jq .dataSets[0].series.0:0:0:0:0.observations.0[0]
I am able to retieve up to the series using:
cat sample3.json | jq .dataSets[0].series
But once I try to drill down further I am getting a compile error:
$ cat sample3.json | jq .dataSets[0].series.0:0:0:0:0
jq: error: syntax error, unexpected LITERAL, expecting end of file (Unix shell quoting issues?) at <top-level>, line 1:
.dataSets[0].series.0:0:0:0:0
jq: 1 compile error
I am not sure what I am doing wrong here....
The input file is:
{
"header": {
"id": "b8be2cd5-33bf-4687-9e81-eb032f6f8a71",
"test": false,
"prepared": "2022-09-01T13:30:57.013+02:00",
"sender": {
"id": "ECB"
}
},
"dataSets": [
{
"action": "Replace",
"validFrom": "2022-09-01T13:30:57.013+02:00",
"series": {
"0:0:0:0:0": {
"attributes": [
0,
null,
0,
null,
null,
null,
null,
null,
null,
null,
null,
null,
0,
null,
0,
null,
0,
0,
0,
0
],
"observations": {
"0": [
1.4529,
0,
0,
null,
null
],
"1": [
1.4472,
0,
0,
null,
null
],
"2": [
1.4591,
0,
0,
null,
null
]
}
}
}
}
],
"structure": {
"links": [
{
"title": "Exchange Rates",
"rel": "dataflow",
"href": "https://sdw-wsrest.ecb.europa.eu:443/service/dataflow/ECB/EXR/1.0"
}
],
"name": "Exchange Rates",
"dimensions": {
"series": [
{
"id": "FREQ",
"name": "Frequency",
"values": [
{
"id": "D",
"name": "Daily"
}
]
},
{
"id": "CURRENCY",
"name": "Currency",
"values": [
{
"id": "AUD",
"name": "Australian dollar"
}
]
},
{
"id": "CURRENCY_DENOM",
"name": "Currency denominator",
"values": [
{
"id": "EUR",
"name": "Euro"
}
]
},
{
"id": "EXR_TYPE",
"name": "Exchange rate type",
"values": [
{
"id": "SP00",
"name": "Spot"
}
]
},
{
"id": "EXR_SUFFIX",
"name": "Series variation - EXR context",
"values": [
{
"id": "A",
"name": "Average"
}
]
}
],
"observation": [
{
"id": "TIME_PERIOD",
"name": "Time period or range",
"role": "time",
"values": [
{
"id": "2022-08-29",
"name": "2022-08-29",
"start": "2022-08-29T00:00:00.000+02:00",
"end": "2022-08-29T23:59:59.999+02:00"
},
{
"id": "2022-08-30",
"name": "2022-08-30",
"start": "2022-08-30T00:00:00.000+02:00",
"end": "2022-08-30T23:59:59.999+02:00"
},
{
"id": "2022-08-31",
"name": "2022-08-31",
"start": "2022-08-31T00:00:00.000+02:00",
"end": "2022-08-31T23:59:59.999+02:00"
}
]
}
]
},
"attributes": {
"series": [
{
"id": "TIME_FORMAT",
"name": "Time format code",
"values": [
{
"name": "P1D"
}
]
},
{
"id": "BREAKS",
"name": "Breaks",
"values": []
},
{
"id": "COLLECTION",
"name": "Collection indicator",
"values": [
{
"id": "A",
"name": "Average of observations through period"
}
]
},
{
"id": "COMPILING_ORG",
"name": "Compiling organisation",
"values": []
},
{
"id": "DISS_ORG",
"name": "Data dissemination organisation",
"values": []
},
{
"id": "DOM_SER_IDS",
"name": "Domestic series ids",
"values": []
},
{
"id": "PUBL_ECB",
"name": "Source publication (ECB only)",
"values": []
},
{
"id": "PUBL_MU",
"name": "Source publication (Euro area only)",
"values": []
},
{
"id": "PUBL_PUBLIC",
"name": "Source publication (public)",
"values": []
},
{
"id": "UNIT_INDEX_BASE",
"name": "Unit index base",
"values": []
},
{
"id": "COMPILATION",
"name": "Compilation",
"values": []
},
{
"id": "COVERAGE",
"name": "Coverage",
"values": []
},
{
"id": "DECIMALS",
"name": "Decimals",
"values": [
{
"id": "4",
"name": "Four"
}
]
},
{
"id": "NAT_TITLE",
"name": "National language title",
"values": []
},
{
"id": "SOURCE_AGENCY",
"name": "Source agency",
"values": [
{
"id": "4F0",
"name": "European Central Bank (ECB)"
}
]
},
{
"id": "SOURCE_PUB",
"name": "Publication source",
"values": []
},
{
"id": "TITLE",
"name": "Title",
"values": [
{
"name": "Australian dollar/Euro"
}
]
},
{
"id": "TITLE_COMPL",
"name": "Title complement",
"values": [
{
"name": "ECB reference exchange rate, Australian dollar/Euro, 2:15 pm (C.E.T.)"
}
]
},
{
"id": "UNIT",
"name": "Unit",
"values": [
{
"id": "AUD",
"name": "Australian dollar"
}
]
},
{
"id": "UNIT_MULT",
"name": "Unit multiplier",
"values": [
{
"id": "0",
"name": "Units"
}
]
}
],
"observation": [
{
"id": "OBS_STATUS",
"name": "Observation status",
"values": [
{
"id": "A",
"name": "Normal value"
}
]
},
{
"id": "OBS_CONF",
"name": "Observation confidentiality",
"values": [
{
"id": "F",
"name": "Free"
}
]
},
{
"id": "OBS_PRE_BREAK",
"name": "Pre-break observation value",
"values": []
},
{
"id": "OBS_COM",
"name": "Observation comment",
"values": []
}
]
}
}
}
The .foo syntax cannot be used if the key name has anything but alphanumeric characters or the underscore, or if the first character of the key name is numeric.
Assuming you are using a recent version of jq,
you can always use the form: ."foo", which is actually an abbreviation of the basic form, .["foo"].
So assuming you're using a sufficiently recent version of jq, your query could begin with:
.dataSets[0].series."0:0:0:0:0"
If you are presenting the jq query on a command line, then you may have to escape the double-quotes appropriately, e.g. in a bash shell, by enclosing the jq query in single-quotes.
I have an elastic search index
like
{
"title": "A",
"comments": [
{
"id": "1"
},
{
"id": "2"
}
]
},
{
"title": "B",
"comments": [
{
"id": "1"
},
{
"id": "3"
}
]
},
{
"title": "C",
"comments": [
{
"id": "7"
},
{
"id": "3"
}
]
}
I want to collapse is the group by the nested object. In the above JSON, I want to group it by Id.
So the output will be like
hits:[{
"title": "A",
"comments": [
{
"id": "1"
},
{
"id": "2"
}
]
},
inner_hits {[
{
"title": "A",
"comments": [
{
"id": "1"
},
{
"id": "2"
}
]
},
{
"title": "B",
"comments": [
{
"id": "1"
},
{
"id": "3"
}
]
}
]}
}]
Baiscally I need collapse bases on the nested object property.
Tried this
/_search?track_total_hits=true
{
"collapse": {
"field": "comments.id",
"inner_hits": {
"name": "id",
"size": 10
},
"max_concurrent_group_searches": 3
}
}
But its always returing first object only in the inner hits
Within the mapping of the object comments , you should remove the nested type.
I have following json:
{
"id": "1",
"name": "profile1",
"userId": "0",
"groupId": "3",
"attributes": [
{
"id": "104",
"name": "Enable",
"value": "1"
},
{
"id": "105",
"name": "TargetNode",
"value": "system1"
},
{
"id": "106",
"name": "Timeout",
"value": "30"
}
],
"xconns": [
{
"id": "1",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "1",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "1",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
}
]
}
{
"id": "2",
"name": "profile2",
"userId": "7",
"groupId": "0",
"attributes": [
{
"id": "104",
"name": "Enable",
"value": "1"
},
{
"id": "105",
"name": "TargetNode",
"value": "system2"
},
{
"id": "106",
"name": "Timeout",
"value": "30"
}
],
"xconns": [
{
"id": "2",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "2",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "2",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
}
]
}
I can filter following:
$ jq -r 'select([.attributes[] | .name == "TargetNode" ] | any ) | [{userId, groupId, id, name}] | .[] | if (.userId == "0") then del(.userId) else . end | if (.groupId == "0") then del(.groupId) else . end | to_entries | map("\(.key | ascii_upcase):\(.value)") | #tsv' file.json
GROUPID:3 ID:1 NAME:profile1
USERID:7 ID:2 NAME:profile2
I need to add also value of TargetNode:
GROUPID:3 ID:1 NAME:profile1 TARGETNODE:system1
USERID:7 ID:2 NAME:profile2 TARGETNODE:system2
is there a way to include it in
[{userId, groupId, id, name, TargetNode}]
to get the value of TargetNode and not null?
GROUPID:3 ID:1 NAME:profile1 TARGETNODE:null
USERID:7 ID:2 NAME:profile2 TARGETNODE:null
Update:
the solution provided by RomanPerekhrest is nearly ok, but there is one issue because the json file in real is much bigger, there are more attrobutes in "main secttion", for example:
{
"id": "1",
"name": "profile1",
"userId": "0",
"groupId": "3",
"attrib101": "A",
"attrib102": "B",
"attributes": [
...
...
it is cousing that RomanPerekhrest's jq filter returns too much...
how to rid of them too?
ID:1 NAME:profile1 GROUPID:3 ATTRIB101:A ATTRIB102:B TARGETNODE:system1
ID:2 NAME:profile2 USERID:7 ATTRIB101:C ATTRIB102:D TARGETNODE:system2
jq solution:
jq -r '.attributes |= map(select(.name == "TargetNode"))
| if (.attributes | length != 0) then .targetNode = .attributes[0].value else . end
| if (.userId == "0") then del(.userId) else . end
| if (.groupId == "0") then del(.groupId) else . end
| del(.attributes, .xconns) | to_entries
| map("\(.key | ascii_upcase):\(.value)") | #tsv' file.json
If an object with "name": "TargetNode" pair not exists - TARGETNODE won't be added into resulting structure
The output:
ID:1 NAME:profile1 GROUPID:3 TARGETNODE:system1
ID:2 NAME:profile2 USERID:7 TARGETNODE:system2