I'm trying to reshape a json document and I assumed it would be easy to do using jq but I haven't been trying for several hours now and no success ...
(Please note that I'm not a jq jedi and the doc did not help)
I want to go from this :
{
"results": [
{
"profile": {
"birthYear": 1900,
"locale": "en_EN",
"city": "Somewhere, Around",
"timezone": "2",
"age": 52,
"gender": "m"
},
"UID": "SQSQSQerl7XSQSqSsqSQ"
}
]
}
to this :
{
"birthYear": 1900,
"locale": "en_EN",
"city": "Somewhere, Around",
"timezone": "2",
"age": 52,
"gender": "m",
"UID": "SQSQSQerl7XSQSqSsqSQ"
}
I got what below using this filter : .results[].profile , .results[].UID
{
"birthYear": 1900,
"locale": "en_EN",
"city": "Somewhere, Around",
"timezone": "2",
"age": 52,
"gender": "m"
}
"UID": "SQSQSQerl7XSQSqSsqSQ"
Thanks in advance for your help..
You can combine two objects with the addition operator.
jq '.results[] | .profile + {UID}'
.profile is already an object.
The other object is created with {}. {UID} is shorthand for {"UID" : .UID}
there are probably better ways but here you go
jq '.results[0].profile * .results[0] | del(.profile)'
explanation:
merge recursivly container with nested-container by means of A * B, then pipe to del( to remove nested container
Related
I have two json's that are a list of objects that share the same key and I am trying to combine them into one json using jq. The expected output is a single json that contains a list of the combined objects in list form. For example:
Json 1:
[
{"Id":"1", "FirstName":"firstName1", "LastName":"lastName1"},
{"Id":"2", "FirstName":"firstName2", "LastName":"lastName2"},
{"Id":"3", "FirstName":"firstName2", "LastName":"lastName3"}
]
Json 2:
[
{"School":"School1", "Id":"1", "Degree":"Degree1"},
{"School":"School2", "Id":"2", "Degree":"Degree2"},
{"School":"School3", "Id":"3", "Degree":"Degree3"}
]
Combined Json Based on Id
[
{"Id":"1", "FirstName":"firstName1", "LastName":"lastName1",
"School":"School1", "Degree":"Degree1"},
{"Id":"2", "FirstName":"firstName2", "LastName":"lastName2",
"School":"School2", "Degree":"Degree2"},
{"Id":"3", "FirstName":"firstName2", "LastName":"lastName3",
"School":"School3", "Degree":"Degree3"}
]
I have already tried a few ways to merge these jsons I found in this thread such as:
jq -s '.[0] * .[1]' file1 file2
I am still a novice in jq, so any help would be appreciated!
Use the SQL-Style Operators JOIN and INDEX
jq 'JOIN(INDEX(inputs[];.Id);.[];.Id;add)' json1 json2
[
{
"Id": "1",
"FirstName": "firstName1",
"LastName": "lastName1",
"School": "School1",
"Degree": "Degree1"
},
{
"Id": "2",
"FirstName": "firstName2",
"LastName": "lastName2",
"School": "School2",
"Degree": "Degree2"
},
{
"Id": "3",
"FirstName": "firstName2",
"LastName": "lastName3",
"School": "School3",
"Degree": "Degree3"
}
]
Demo
I'm trying to set up an alert for when the following JSON object state says anything but started. I'm beginning to play around with conditional jq but I'm unsure how to implement regex into this.
{
"page": 0,
"page_size": 100,
"total_pages": 10,
"total_rows": 929,
"headers": [
"*"
],
"rows": [
{
"id": "168",
"state": "STARTED"
},
{
"id": "169",
"state": "FAILED"
},
{
"id": "170",
"state": "STARTED"
}
]
}
I only want to display the id and state of the failed object, this is what I tried
jq '.rows[] | .id, select(.state | contains("!STARTED"))' test.json
I'd like my output to be something like
{
"id": "169",
"state": "FAILED"
}
If you simply want to print out the objects for which .state is NOT "STARTED", just use negation:
.rows[] | select(.state != "STARTED")
If the "started" state is associated with multiple values, please give further details. There might not be any need to use regular expressions. If you really do need to use regular expressions, then you will probably want to use test.
I have the following json format that I need to convert to CSV
[{
"name": "joe",
"age": 21,
"skills": [{
"lang": "spanish",
"grade": "47",
"school": {
"name": "my school",
"url": "example.com/sp-school"
}
}, {
"lang": "english",
"grade": "87"
}]
},
{
"name": "sarah",
"age": 34,
"skills": [{
"lang": "french",
"grade": "47",
"school": {
"name": "my school",
"url": "example.com/sp-school"
}
}, {
"lang": "english",
"grade": "87"
}]
}, {
"name": "jim",
"age": 26,
"skills": [{
"lang": "spanish",
"grade": "60"
}, {
"lang": "english",
"grade": "66",
"school": {
"name": "eg school",
"url": "eg-school.com"
}
}]
}
]
to convert to csv
name,age,grade,school,url,file,line_number
joe,21,47,"my school","example.com/sp-school",sample.json,1
jim,26,60,"","",sample.json,3
So add the top level fields and the object from the skills array if lang=spanish and the school hash from the skills object for spanish if it exists
I'd also like to add the file and line number it came from.
I would like to use jq for the job, but can't figure out the syntax , anyone help me out ?
With your data in input.json, and the following jq program in tocsv.jq:
.[]
| [.name, .age] +
(.skills[]
| select(.lang == "spanish")
| [.grade, .school.name, .school.url, input_filename, input_line_number] )
| #csv
the invocation:
jq -r -f tocsv.jq input.json
yields:
"joe",21,"47","my school","example.com/sp-school","input.json",51
"jim",26,"60",,,"input.json",51
If you want the number-valued strings converted to numbers, you could use the "tonumber" filter. If you want the null-valued fields replaced by strings, use e.g. .school.name // ""
Of course this approach doesn't yield a very useful line number. One approach that would yield higher granularity would be to stream the individual objects into jq, but then you'd lose the filename. To recover the filename you could pass it in as an argument. So you would have a pipeline like so:
jq -c '.[]' input.json | jq -r --arg file input.json -f tocsv2.jq
where tocsv2.jq would be like tscsv.jq above but without the initial .[] |, and with $file instead of input_filename.
Finally, please also consider using the TSV format (#tsv) rather than the rather messy CSV format (#csv).
Given the following test.json that I received as a response from the Pocket API,
{
"complete": 1,
"error": null,
"list": {
"1000055792": {
"excerpt": "Some Text",
"favorite": "0",
"given_title": "Some Title",
"given_url": "Some URL",
"has_image": "0",
"has_video": "0",
"is_article": "1",
"is_index": "0",
"item_id": "1000055792",
"resolved_id": "1000055792",
"resolved_title": "Title",
"resolved_url": "Some URL",
"sort_id": 700,
"status": "1",
"time_added": "1438646514",
"time_favorited": "0",
"time_read": "1439025088",
"time_updated": "1439025090",
"word_count": "10549"
},
"1000102810": {
"excerpt": "Some Text",
"favorite": "0",
"given_title": "Title",
"given_url": "Some URL",
"has_image": "1",
"has_video": "0",
"is_article": "1",
"is_index": "0",
"item_id": "1000102810",
"resolved_id": "1000102810",
"resolved_title": "Title",
"resolved_url": "Resolved URL",
"sort_id": 650,
"status": "1",
"time_added": "1440303789",
"time_favorited": "0",
"time_read": "1440320729",
"time_updated": "1440320731",
"word_count": "3219"
}
How can I access the values of keys like resolved_title and word_count. They are nested inside an object which is a number, the same as the id, which in itself is nested inside list. I've searched and found a way to access nested objects using jq. But how can I access the values that are nested inside another object within the main list object?
Also, the IDs are different and not sequential, so I don't think recursion is possible, but I could be wrong. What I'm intending to do with this data is to only extract the resolved_title and word_count values for each item and save them to a two-column spreadsheet.
Thanks in advance!
The following can easily be extended and/or adapted:
> jq ".list[] | {resolved_title, word_count}" input.json
Output:
{
"resolved_title": "Title",
"word_count": "10549"
}
{
"resolved_title": "Title",
"word_count": "3219"
}
You can use the .[] operator to iterate over all elements in an array (or in this case all the keys). The following will give you output with each field on a separate line:
cat <file_with_json> | jq '.list | .[] | .resolved_title, .word_count'
The first filter operates on only the list element. The second filter says for every element and finally the output is just the resolved_title and .word_count fields. This produces the following:
"Title"
"3219"
"Title"
"10549"
Try map():
$ cat myfile.json | jq '.list | map({resolved_title: .resolved_title, word_count: .word_count})'
[
{
"resolved_title": "Title",
"word_count": "10549"
},
{
"resolved_title": "Title",
"word_count": "3219"
}
]
I using JQ 1.4 on Windows 64 bit machine.
Below are the contents input file (JSON.txt)
{
"name": "Google",
"location":
{
"street": "1600 Amphitheatre Parkway",
"city": "Mountain View",
"state": "California",
"country": "US"
},
"employees":
[
{
"name": "Michael",
"division": "Engineering"
},
{
"name": "Laura",
"division": "HR"
},
{
"name": "Elise",
"division": "Marketing"
}
]
}
At the output I exect to see two results "Google" and "Laura"
I am able to get them with idividual filters.
1) jq ."name" JSON.txt
"Google"
2) jq ."employees|map(select(.division==\"HR\"))"[0].name JSON.txt
"Laura"
When I combine these two filters using comma ,, I receive below error:
3) jq ."name",."employees|map(select(.division==\"HR\"))"[0].name JSON.txt
jq: error: Cannot iterate over string
null
"Laura"
Can someone please help to me to get below result:
"Google"
"Laura"
Use brackets around the second part of the expression, and shorthand key filter syntax as in:
jq ".name, (.employees|map(select(.division==\"HR\"))[0].name)" JSON.txt