I am straggling some time with jq to parse json file. I have to grab ID and Name in the same line so I can do some logic after.
{
"elements": [{
"id": "e20a9cd8-8683-4986-b6c0-e5fbf51cbf7f",
"name": "Mike",
"components": [{
"id": "15f959fc-6d2d-451a-a59e-430a05a1852c",
"pid": "ZZZ1"
}],
"tenantIds": null,
"productIds": null
},
{
"id": "d892f2eb-d7f3-49f8-9176-2113351cccf8",
"name": "Steve",
"components": [{
"id": "0c44c917-e0e5-4fa3-b87c-89f9ac0815b4",
"pid": "XXX3"
}],
"tenantIds": null,
"productIds": null
}
]
}
With jq '{elements}[] | .[].id I am getting ID's but I can't find solution how to add appropriate name beside ID. I tried something like
{elements}[] | .[].id + " " + .[].name
and
{elements}[] | .[].id + .[].name
but it's not what I expected.
I want to get:
15f959fc-6d2d-451a-a59e-430a05a1852c Mike
d892f2eb-d7f3-49f8-9176-2113351cccf8 Steve
Any suggestion?
Thanks!
Apply string concatenation:
jq '.elements[] | .id +" "+ .name' file
The output:
"e20a9cd8-8683-4986-b6c0-e5fbf51cbf7f Mike"
"d892f2eb-d7f3-49f8-9176-2113351cccf8 Steve"
To output without double quotes use -r (--raw-output) option:
jq '.elements[] | .id +" "+ .name' -r file
e20a9cd8-8683-4986-b6c0-e5fbf51cbf7f Mike
d892f2eb-d7f3-49f8-9176-2113351cccf8 Steve
Related
I have a json that looks like this:
{
"HostedZones": [
{
"ResourceRecordSetCount": 2,
"CallerReference": "test20150527-2",
"Config": {
"Comment": "test2",
"PrivateZone": true
},
"Id": "/hostedzone/Z119WBBTVP5WFX",
"Name": "dev.devx.company.services."
},
{
"ResourceRecordSetCount": 2,
"CallerReference": "test20150527-1",
"Config": {
"Comment": "test",
"PrivateZone": true
},
"Id": "/hostedzone/Z3P5QSUBK4POTI",
"Name": "test.devx.company.services."
}
],
"IsTruncated": false,
"MaxItems": "100"
}
And my goal is to fetch a specific Name (in my case it's the test.devx.company.services), however the Name field contains an extra "." at the end that I'd like to remove from the output.
This is what I have so far:
jq --raw-output '.HostedZones[] | select(.Name | test("test")?) | (.Name[:-1] | sub("."; ""))'
The problem with that it is removing the first character from the output also.
So the output currently is: est.devx.company.services (JQ play snippet)
Not sure what I'm doing wrong :/
To always remove the last character, if it contains "test":
jq '(.HostedZones[].Name | select(contains("test"))) |= .[:-1]'
To remove it only if it is a dot:
jq '(.HostedZones[].Name | select(contains("test"))) |= sub("[.]$"; "")'
I'm trying to convert a json object into csv via jq. This is the json structure:
{
"totalCount": 4440,
"data": [
{
"company": {
"name": "My_company_name",
"countryCode": "US",
"portfolioName": "My_portfolio"
},
"eventDate": "2021-11-22T00:00:00",
"newValue": null,
"oldValue": null,
"ruleCode": 704,
"ruleName": "New Accounts",
"summary": "Explanations..."
},
{
"company": {
"name": "My_company_name 2",
"countryCode": "UK",
"portfolioName": "My_portfolio"
},
"eventDate": "2021-10-22T00:00:00",
"newValue": null,
"oldValue": null,
"ruleCode": 701,
"ruleName": "Hello",
"summary": "otherExplanations..."
}
...
]
}
For data in "first level", I've no problems:
jq -r '.data | map({eventDate, ruleCode, ruleName, summary, oldValue, newValue}) | (first | keys_unsorted) as $keys | map([to_entries[] | .value]) as $rows | $keys,$rows[] | #csv' input.json > output.csv
But I'ld like to add the company name and country code for example, and I don't kown to do this, with this king of data in second "level".
I'ld to obtain something like that:
"eventDate","ruleCode","ruleName","summary","oldValue","newValue","companyName", "companyCountryCode"
"2021-11-22T00:00:00",704,"New Accounts","Explanations...",,,"My_company_name", "US"
"2021-11-22T00:00:00",701,"Hello","otherExplanations...",,,"My_company_name 2", "UK"
Could you help me ?
Thanks
If you don't mind, I added a . to divide top-level from sub-level headers to make things easier (namely company.name and company.countryCode):
jq --raw-output '[
"eventDate",
"ruleCode",
"ruleName",
"summary",
"oldValue",
"newValue",
"company.name",
"company.countryCode"
] as $h
| $h, (.data[] | [getpath($h[] / ".")])
| #csv'
"eventDate","ruleCode","ruleName","summary","oldValue","newValue","company.name","company.countryCode"
"2021-11-22T00:00:00",704,"New Accounts","Explanations...",,,"My_company_name","US"
"2021-10-22T00:00:00",701,"Hello","otherExplanations...",,,"My_company_name 2","UK"
Demo
I need to print some results with jq to take json.
This is an example:
{
"data": [
{
"time": 20201606,
"event": {
"ip": "127.0.1",
"hostname": "srv1",
"locations": [
"UK",
"site1"
],
"num": 1
}
},
{
"time": 202016034,
"event": {
"ip": "127.0.2",
"hostname": "srv2",
"locations": [
"UK",
"site2"
],
"num": 3
}
}
]
}
Like to generate this output "num, ip, hostname, locations":
1, srv1, 127.0.1, UK,site1
2, srv2, 127.0.2, HK,site2
3, srv3, 127.0.3, LO,site3
How can I print this via jq?
Join locations by a comma, and put the result into an array with other fields. Then join again by a comma followed by a space to get the desired output format. E.g.:
.data[].event | [
.num,
.hostname,
.ip,
(.locations | join(",")) ?
] | join(", ")
Use --raw-output/-r option in the command line invocation to get raw strings instead of JSON strings.
Online demo
At its core, you want to build an array consisting of the values you want:
$ jq '.data[].event | [.num, .hostame, .ip, .locations]' tmp.json
[
1,
null,
"127.0.1",
[
"UK",
"site1"
]
]
[
3,
null,
"127.0.2",
[
"UK",
"site2"
]
]
From there, it's a matter of formatting. First, let's turn the list of locations into a single string:
$ jq '.data[].event | [.num, .hostame, .ip, (.locations|join(","))]' tmp.json
[
1,
null,
"127.0.1",
"UK,site1"
]
[
3,
null,
"127.0.2",
"UK,site2"
]
Next, let's join those strings into a ", "-separated string.
$ jq '.data[].event | [.num, .hostame, .ip, (.locations|join(","))] | join(", ")' tmp.json
"1, , 127.0.1, UK,site1"
"3, , 127.0.2, UK,site2"
Finally, you can use the -r flag to output raw text rather than a JSON string value.
$ jq -r '.data[].event | [.num, .hostame, .ip, (.locations|join(","))] | join(", ")' tmp.json
1, , 127.0.1, UK,site1
3, , 127.0.2, UK,site2
I have some database in JSON file, I had already sort and remove some data from object by using ./jq
But I'm stuck at adding new variables in object.
Here is a part of my JSON file:
{
"Name": "Forrest.Gump.1994.MULTi.1080p.AMZN.WEB-DL.DDP5.1.H264-Ao",
"ID": "SMwIkBoC2blXeWnBa9Hjge9YPs90"
},
{
"Name": "Point.Blank.2019.MULTi.1080p.NF.WEB-DL.DDP5.1.x264-Ao",
"ID": "OZI4mOuBXuJ7b89FLgXJoozyhHe9"
},
{
"Name": "The.Incredible.Hulk.2008.MULTi.2160p.UHD.BluRay.REMUX.HDR.HEVC.DTS-HD.MA.7.1",
"ID": "jZzR4_B_vjm593cYKR7j97XAMv6d"
},
Is it possible by using jq and for example RegExp to extract some data and insert it as new variable in object, I wish to achive something like this:
{
"Name": "Forrest.Gump.1994.MULTi.1080p.AMZN.WEB-DL.DDP5.1.H264-Ao",
"ID": "SMwIkBoC2blXeWnBa9Hjge9YPs90",
"Year": "1994",
"Res": "1080p"
},
{
"Name": "Point.Blank.2019.MULTi.1080p.NF.WEB-DL.DDP5.1.x264-Ao",
"ID": "OZI4mOuBXuJ7b89FLgXJoozyhHe9",
"Year": "2019",
"Res": "1080p"
},
{
"Name": "The.Incredible.Hulk.2008.MULTi.2160p.UHD.BluRay.REMUX.HDR.HEVC.DTS-HD.MA.7.1",
"ID": "jZzR4_B_vjm593cYKR7j97XAMv6d",
"Year": "2008",
"Res": "2160p"
},
Thanks in advance
Here's one solution that assumes for simplicity that the fragment you've shown comes from an array:
map( . as $in
| .Name | capture(".*[.](?<year>[12][0-9]{3})[.](?<rest>.*)")
| .year as $year
| (.rest | split(".") | .[1]) as $res
| $in + {Year: $year, Res: $res} )
Hopefully, once you're familiar with some jq basics, such as map, capture, and the EXP as $var syntax, the above will be more-or-less self-explanatory.
As a one-liner
Here's the same thing but as a one-liner:
map(. + (.Name | capture(".*[.](?<Year>[12][0-9]{3})[.](?<Res>.*)") | {Year, Res: (.Res | split(".")[1])}))
I'm trying to convert 7z file content list to json and can't fix missing separator between output converted blocks.
I'm little bit newbie in json conversion, but found that jq could do the job.
I read the jq documentation and found examples inside here and there also elsewhere without solution.
Please find the use case:
The command line:
jq -f pf_7z.jq -R
The input file demo.lst:
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2018-06-23 14:02:16 D.... 0 0 Installer
2018-06-23 14:02:16 ..... 3381 1157 Installer\Readme
2018-06-23 14:02:16 ..... 4646 1157 Installer\License.txt
2018-06-23 14:02:16 ..... 138892 136152 Installer\Setup.exe
The filter file pf7z.jq:
def parse:
def parse_line:
. | map(match("(\\d+-\\d+-\\d+) (\\d+:\\d+:\\d+) (D|.).* +(\\d+) +(\\d+) +(.*\\\\)([^\\\\]*)\\.(.*)")) | .[] |
({
"date" :(.captures[0].string),
"time" :(.captures[1].string),
"attr" :(.captures[2].string),
"size" :(.captures[3].string),
"path" :(.captures[5].string),
"name" :(.captures[6].string),
"extn" :(.captures[7].string)
});
split("\n") | ( {} + (parse_line));
parse
The expected result should be:
{
"date": "2018-06-23",
"time": "14:02:16",
"attr": ".",
"size": "4646",
"path": "Installer\",
"name": "License",
"extn": "txt"
},
{
"date": "2018-06-23",
"time": "14:02:16",
"attr": ".",
"size": "138892",
"path": "Installer\",
"name": "Setup",
"extn": "exe"
}
And I only got :
{
"date": "2018-06-23",
"time": "14:02:16",
"attr": ".",
"size": "4646",
"path": "Installer\",
"name": "License",
"extn": "txt"
}
{
"date": "2018-06-23",
"time": "14:02:16",
"attr": ".",
"size": "138892",
"path": "Installer\",
"name": "Setup",
"extn": "exe"
}
without the comma separator between blocks.
Thanks ;-)
Your def for parse_line produces a stream of JSON entities, whereas you evidently want a JSON array. Using your regex, you could write:
def parse:
def parse_line:
match("(\\d+-\\d+-\\d+) (\\d+:\\d+:\\d+) (D|.).* +(\\d+) +(\\d+) +(.*\\\\)([^\\\\]*)\\.(.*)")
| .captures
| map(.string)
| { "date" :.[0],
"time" :.[1],
"attr" :.[2],
"size" :.[3],
"path" :.[5],
"name" :.[6],
"extn" :.[7] } ;
[inputs | parse_line];
parse
Invocation
jq -nR -f 7z.jq 7z.txt
Alternative regex
The regex fragment (D|.).* does not make much sense.
You should consider replacing it by (.)[^ ]* or some such.
A simpler solution
def parse_line:
capture("(?<date>\\d+-\\d+-\\d+) "
+ "(?<time>\\d+:\\d+:\\d+) "
+ "(?<attr>.)[^ ]* +"
+ "(?<size>\\d+) +\\d+ +"
+ "(?<path>.*\\\\)"
+ "(?<name>[^\\\\]*)\\."
+ "(?<extn>.*)");
[inputs | parse_line]
An alternative approach
From the comment about JSONEdit, it seems likely to me that your overall approach might be suboptimal. Have you considered using jq rather than jq with JSONEdit?