I have been trying to extract a csv from the below json file using jq but not able to get so far. Does any experts out here can help?
{
"values": [
{
"resourceId": "xxxx-xxxx-xxx-8b16-xxxxxx",
"property-contents": {
"property-content": [
{
"statKey": "config|name",
"timestamps": [
1517591034069
],
"values": [
"somebname.UNIVERSE.test.com"
]
},
{
"statKey": "summary|guest|ipAddress",
"timestamps": [
1517591034069
],
"values": [
"100.xx.5.xx"
]
},
{
"statKey": "summary|parentCluster",
"timestamps": [
1551120506024
],
"values": [
"UFO-UFO"
]
},
{
"statKey": "summary|parentDatacenter",
"timestamps": [
1551120806021
],
"values": [
"GALAXY-D123"
]
},
{
"statKey": "summary|parentVcenter",
"timestamps": [
1517591334271
],
"values": [
"X-RAY123"
]
},
{
"statKey": "summary|runtime|powerState",
"timestamps": [
1517591034069
],
"values": [
"Powered On"
]
}
]
}
},
..
...
xxx-xxxx-xxx-8b16-xxxxxx,somebname.UNIVERSE.test.com,100.xx.5.xx,UFO-UFO,GALAXY-D123,X-RAY123,Powered On
Expected o/p is:
xxx-xxxx-xxx-8b16-xxxxxx,somebname.UNIVERSE.test.com,100.xx.5.xx,UFO-UFO,GALAXY-D123,X-RAY123,Powered On
Your expected output leaves some things unclear:
The second CSV column contains somebname.UNIVERSE.test.com, which was presumably derived from the section "property-content": [ { ..., "values": [ "somebname.UNIVERSE.test.com" ], ... }. How do you determine which element in the "property-content" list to pick for the second column? Is it because it's the first element? Is it because of its "statKey": "config|name"?
What if the "property-content" list is empty? What if it doesn't have the "statKey" entry you're looking for? What if the "values" list has zero or more than one element? The CSV row can only contain one scalar value. The same question applies for subsequent columns.
Making a wild guess here,
$ jq -r '.values[] | [ .resourceId, (."property-contents"."property-content"[] | .values[]) ] | join(",")' your.json
xxxx-xxxx-xxx-8b16-xxxxxx,somebname.UNIVERSE.test.com,100.xx.5.xx,UFO-UFO,GALAXY-D123,X-RAY123,Powered On
I cannot guarantee (and somewhat doubt) that this works in the general case, but I've been unable to extract a general case from your one example.
Related
I have file with 30, 000 JSON lines delimited by new line. I am using JQ to process it.
Below is each line schema (new.json).
{
"indexed": {
"date-parts": [
[
2020,
8,
13
]
],
"date-time": "2020-08-13T06:27:26Z",
"timestamp": 1597300046660
},
"reference-count": 42,
"publisher": "American Chemical Society (ACS)",
"issue": "3",
"content-domain": {
"domain": [],
"crossmark-restriction": false
},
"short-container-title": [
"Org. Lett."
],
"published-print": {
"date-parts": [
[
2005,
2
]
]
},
"DOI": "10.1021/ol047829t",
"type": "journal-article",
"created": {
"date-parts": [
[
2005,
1,
27
]
],
"date-time": "2005-01-27T05:53:29Z",
"timestamp": 1106805209000
},
"page": "383-386",
"source": "Crossref",
"is-referenced-by-count": 38,
"title": [
"Liquid-Crystalline [60]Fullerene-TTF Dyads"
],
"prefix": "10.1021",
"volume": "7",
"author": [
{
"given": "Emmanuel",
"family": "Allard",
"affiliation": []
},
{
"given": "Frédéric",
"family": "Oswald",
"affiliation": []
},
{
"given": "Bertrand",
"family": "Donnio",
"affiliation": []
},
{
"given": "Daniel",
"family": "Guillon",
"affiliation": []
}
],
"member": "316",
"container-title": [
"Organic Letters"
],
"original-title": [],
"link": [
{
"URL": "https://pubs.acs.org/doi/pdf/10.1021/ol047829t",
"content-type": "unspecified",
"content-version": "vor",
"intended-application": "similarity-checking"
}
],
"deposited": {
"date-parts": [
[
2020,
4,
7
]
],
"date-time": "2020-04-07T13:39:55Z",
"timestamp": 1586266795000
},
"score": null,
"subtitle": [],
"short-title": [],
"issued": {
"date-parts": [
[
2005,
2
]
]
},
"references-count": 42,
"alternative-id": [
"10.1021/ol047829t"
],
"URL": "http://dx.doi.org/10.1021/ol047829t",
"relation": {},
"ISSN": [
"1523-7060",
"1523-7052"
],
"issn-type": [
{
"value": "1523-7060",
"type": "print"
},
{
"value": "1523-7052",
"type": "electronic"
}
],
"subject": [
"Physical and Theoretical Chemistry",
"Organic Chemistry",
"Biochemistry"
]
}
For every DOI, I need to obtain the values of given and family key in the same cell of the same row of that DOI in the CSV/TSV format.
The expected output for the above json is (in CSV/TSV format):
|DOI| givenName|familyName|
|10.1021/ol047829t|Emmanuel; Frédéric; Bertrand; Daniel;|Allard; Oswald; Donnio; Guillon|
I am using the below command line but it is throwing error and when I try to alter I am unable to get CSV/TSV output at all.
cat new.json | jq -r "[.DOI, .publisher, .author[] | .given] | #tsv" > manage.tsv
The same logic applies for subject key also. I am using the below command line to output values of subject key to CSV but it is throwing only the first element (in this case only: "Physical and Theoretical Chemistry")
cat new.json | jq -c -r "[.DOI, .publisher, .subject[0]] | #csv" > manage.csv
Any pointers for right jq command line will be of great help.
Join given and family names by semicolons separately, then pass resulting strings as fields to the TSV filter.
["DOI", "givenName", "familyName"],
(inputs | [.DOI, (.author | map(.given), map(.family) | join("; "))])
| #tsv
Online demo
Note that you need to invoke JQ with -r and -n flags for this to work and produce a valid TSV output.
Can somebody help me to extract with | jq the following:
{
"status": "success",
"data": {
"resultType": "matrix",
"result": [
{
"metric": {
"pod": "dev-cds-5c97cf7f78-sw6b9"
},
"values": [
[
1588204800,
"0.3561394483796914"
],
[
1588215600,
"0.3607968456046861"
],
[
1588226400,
"0.3813882532417868"
],
[
1588237200,
"0.6264355815408573"
]
]
},
{
"metric": {
"pod": "uat-cds-66ccc9685-b5tvh"
},
"values": [
[
1588204800,
"0.9969746974696218"
],
[
1588215600,
"0.7400881057270005"
],
[
1588226400,
"1.2298959318837195"
],
[
1588237200,
"0.9482296838254507"
]
]
}
]
}
}
I need to obtain all-values individually by given word dev-cds and not all the name dev-cds-5c97cf7f78-sw6b9.
Result desired:
{
"metric": {
"pod": "dev-cds-5c97cf7f78-sw6b9"
},
"values": [
[
1588204800,
"0.3561394483796914"
],
[
1588215600,
"0.3607968456046861"
],
[
1588226400,
"0.3813882532417868"
],
[
1588237200,
"0.6264355815408573"
]
]
}
You should first iterate over the result array. Check if the pod inside, metric object has the value that contains "dev-cds".
.data.result[] | if .metric.pod | contains("dev-cds") then . else empty end
https://jqplay.org/s/54OH83qHKP
I want to convert a complex JSON file into a simple JSON file using JQ. However, the query I'm using generates an incorrect output.
My (cut down) JSON file:
[
{
"id": 100,
"foo": [
{
"bar": [
{"type": "read"},
{"type": "write"}
],
"users": ["admin_1"],
"groups": []
},
{
"bar": [
{"type": "execute"},
{ "type": "read"}
],
"users": [],
"groups": ["admin_2"]
}
]
},
{
"id": 101,
"foo": [
{
"bar": [
{"type": "read"}
],
"users": [
"admin_3"
],
"groups": []
}
]
}
]
I need to generate a flatter JSON file and combine the users and groups into one field, similar to this:
[
{
"id": 100,
"users_groups": [
"admin_1",
"admin_2"
],
"bar": ["read"]
},
{
"id": 100,
"users_groups": ["admin_1"],
"bar": ["write"]
},
{
"id": 100,
"users_groups": ["admin_2"],
"bar": ["execute"]
},
{
"id": 101,
"users_groups": ["admin_3"],
"bar": ["read"]
}
]
Everything I try in JQ results in me getting an incorrect output (where admin_1 incorrectly has bar=execute and admin_2 incorrectly has bar=write), similar to the following:
[
{
"id": 100,
"users_groups": [
"admin_1",
"admin_2"
],
"bar": ["read", "write", "execute"]
},
{
"id": 101,
"users_groups": ["admin_3"],
"bar": ["read"]
}
]
I have tried many vairiats of this query - any idea what I should be doing instead?
cat file.json | jq -r '[.[] | select(has("foo")) |{"id", "users":(.foo[] | .users), "groups":(.foo[] | .groups), "bar":([.foo[].bar[] | .type])} ] '
The following filter groups by "type" as the question seems to require:
map(.id as $id
| [.foo[]
| {id: $id, bar: .bar[].type} +
{"users_groups": (.users + .groups)[]} ]
| group_by(.bar)
| map(.[0] + {"users_groups": [.[].users_groups]}) )
Output
[
[
{
"id": 100,
"bar": "execute",
"users_groups": [
"admin_2"
]
},
{
"id": 100,
"bar": "read",
"users_groups": [
"admin_1",
"admin_2"
]
},
{
"id": 100,
"bar": "write",
"users_groups": [
"admin_1"
]
}
],
[
{
"id": 101,
"bar": "read",
"users_groups": [
"admin_3"
]
}
]
]
Variations
To achieve the array-of-objects output format, simply tack on | [.[][]];
it would similarly be trivially easy to ensure that .bar is array-valued, though that might be pointless given that the grouping is by .type.
I have a json file, example.json:
[
[
"126",
1522767000
],
[
"122",
1522859400
],
[
"126",
1523348520
]
]
...and would like to add multiple parent items with the desired output:
{
"target": "Systolic",
"datapoints": [
[
"126",
1522767000
],
[
"122",
1522859400
],
[
"126",
1523348520
]
]
}
I'm having trouble, attempting things like:
cat example.json | jq -s '{target:.[]}', which adds the one key but not understanding how to add a value to the target and another key datapoints.
With straightforward jq expression:
jq '{target: "Systolic", datapoints: .}' example.json
The output:
{
"target": "Systolic",
"datapoints": [
[
"126",
1522767000
],
[
"122",
1522859400
],
[
"126",
1523348520
]
]
}
When I try to save my changes from the Google Apps Script file I get the following error:
Missing ] after element list.
This is my code:
var request = {
"name": "Name",
"id": 3,
"rules":[
{
"name": "Nested",
"tags": [
{
"tagId": 1,
"variables":[
[
"variable": "Var1"
]
],
"condition": false,
},
{
"tagId": 1,
"condition": false,
}
],
"ruleSetId": 3,
}
]
}
The error indicates that the problem is on the line that contains "variable": allScopes[i].variable, but I can't find where the problem is...
This is an example with the JSON object that I need to build:
Note that you are trying to use the array literal to construct an object:
"variables":[
[
"variable": "Var1"
]
],
As it looks like you need an object, not an array here, replace the inner [] with {}:
"variables":[
{
"id": null,
...,
"value": ".*"
}
],