I have been working on a Bash script. I am having a curl response as follows.
{
"range": "'PR-DETAILS'!A1:Z1000",
"majorDimension": "ROWS",
"values": [
[
"PR ID",
"PR Owner"
],
[
"1929",
"Angel"
],
[
"73",
"Martin"
],
[
"142"
]
]
}
Here I just want to get the last available element for the second column.
Expected Answer:- Martin
Here's one way:
last(.values[] | select(has(1))) [1]
Online demo
If the second column does not contain false values (null, false) this will also work:
last(.values[][1] // empty)
Here's a different solution:
.values | map(.[1] | select(.)) | last
probably less efficient, but quite readable.
The difference to the other answer is that select(.) will filter any falsy value, not only non-existent values. So if your second columns were to contain false or null, these wouldn't show up in your result.
Related
I've got some JSON like the following (I've filtered the output here):
[
{
"Tags": [
{
"Key": "Name",
"Value": "example1"
},
{
"Key": "Irrelevant",
"Value": "irrelevant"
}
],
"c7n:MatchedFilters": [
"tag: example_tag_rule"
],
"another_key": "another_value_I_dont_want"
},
{
"Tags": [
{
"Key": "Name",
"Value": "example2"
}
],
"c7n:MatchedFilters": [
"tag:example_tag_rule",
"tag: example_tag_rule2"
]
}
]
I'd like to create a csv file with the value within the Name key and all of the "c7n:MatchedFilters" in the array. I've made a few attempts but still can't get quite the output I expect. There's some example code and the output below:
#Prints the key that I'm after.
cat new.jq | jq '.[] | [.Tags[], {"c7n:MatchedFilters"}] | .[] | select(.Key=="Name")|.Value'
"example1"
"example2"
#Prints all the filters in an array I'm after.
cat new.jq | jq -r '.[] | [.Tags[], {"c7n:MatchedFilters"}] | .[] | select(."c7n:MatchedFilters") | .[]'
[
"tag: example_tag_rule"
]
[
"tag:example_tag_rule",
"tag: example_tag_rule2"
]
#Prints *all* the tags (including ones I don't want) and all the filters in the array I'm after.
cat new.jq | jq '.[] | [.Tags[], {"c7n:MatchedFilters"}] | select((.[].Key=="Name") and (.[]."c7n:MatchedFilters"))'
[
{
"Key": "Name",
"Value": "example1"
},
{
"Key": "Irrelevant",
"Value": "irrelevant"
},
{
"c7n:MatchedFilters": [
"tag: example_tag_rule"
]
}
]
[
{
"Key": "Name",
"Value": "example2"
},
{
"c7n:MatchedFilters": [
"tag:example_tag_rule",
"tag: example_tag_rule2"
]
}
]
I hope this makes sense, let me know if I've missed anything.
Your attempts are not working because you start out with [.Tags[], {"c7n:MatchedFilters"}] to construct one array containing all the tags and an object containing the filters. You are then struggling to find a way to process this entire array at once because it jumbles together these unrelated things without any distinction. You will find it much easier if you don't combine them in the first place!
You want to find the single tag with a Key of "Name". Here's one way to find that:
first(
.Tags[]|
select(.Key=="Name")
).Value as $name
By using a variable binding we can save it for later and worry about constructing the array separately.
You say (in the comments) that you just want to concatenate the filters with spaces. You can do that easily enough:
(
."c7n:MatchedFilters"|
join(" ")
) as $filters
You can combine all this together like follows. Note that each variable binding leaves the input stream unchanged, so it's easy to compose everything.
jq --raw-output '
.[]|
first(
.Tags[]|
select(.Key=="Name")
).Value as $name|
(
."c7n:MatchedFilters"|
join(" ")
) as $filters|
[$name, $filters]|
#csv
Hopefully that's easy enough to read and separates out each concept. We break up the array into a stream of objects. For each object, we find the name and bind it to $name, we concatenate the filters and bind them to $filters, then we construct an array containing both, then we convert the array to a CSV string.
We don't need to use variables. We could just have a big array constructor wrapped around the expression to find the name and the expression to find the filters. But I hope you can see the variables make things a bit flatter and easier to understand.
I have this JSON document:
{
"1": {
"a": "G1"
},
"2": {
"a": "GM1"
}
}
My expected result should be:
1,G1
2,GM1
With *.a i get
[
"G1",
"GM1"
]
but I am absolutely stuck for the rest.
Sadly there is not much you can do that would be totally matching your use case and that would scale properly.
This is because JMESPath does not have a way to reference its parent, although this has been requested before, to allow you something like
*.[join(',', [keys($), a])]
You can definitely extract a list of keys and values, thanks to the function keys:
#.{keys: keys(#), values: *.a}
That gives
{
"keys": [
"1",
"2"
],
"values": [
"G1",
"GM1"
]
}
But then you just fall under the same case as this other question, because keys will give you a list of keys.
You can also end with a list of lists:
#.[keys(#), *.a]
Will give you:
[
[
"1",
"2"
],
[
"G1",
"GM1"
]
]
And you can even go further and flatten it if needed:
#.[keys(#), *.a] []
Gives:
[
"1",
"2",
"G1",
"GM1"
]
With all this if you do happen to have a list of exactly two items, then a solution would be to use a combination of join and slice:
#.[join(',',[keys(#),*.a][] | [::2]), join(',',[keys(#),*.a][] | [1::2])]
That would give the expected:
[
"1,G1",
"2,GM1"
]
But, sadly, as soon as you have more than two items to consider you would end up with a buggy:
[
"1,3,G1,GM3",
"2,4,GM1,GM4"
]
With a data set of
{
"1": {
"a": "G1"
},
"2": {
"a": "GM1"
},
"3": {
"a": "GM3"
},
"4": {
"a": "GM4"
}
}
And then, of course, the same can be achieved hardcoding indexes:
#.[join(',', [keys(#)[0], *.a | [0]]), join(',', [keys(#)[1], *.a | [1]])]
That also gives the expected:
[
"1,G1",
"2,GM1"
]
But, sadly, this only works if you know in advance the number of rows that are going to be returned to you.
And if you want a single string, given that were you want to feed the data accepts \n as a new line, you can join he whole array again:
#.[join(',', [keys(#)[0], *.a | [0]]), join(',', [keys(#)[1], *.a | [1]])].join(`\n`,#)
Will give:
"1,G1\n2,GM1"
Finally this expression worked 100% for me:
[{key1:keys(#)[0],a:*.a| [0]},{key1:keys(#)[1],a:*.a| [1]}]
I'm trying to change # to %23 in every context value, but I'm having problem with null values.
The shortened JSON is:
{
"stats": {
"suites": 1
},
"results": [
{
"uuid": "676-a46b-47a1-a49f-4da4e46c1120",
"title": "",
"suites": [
{
"uuid": "gghjh-56a9-4713-b139-0d5b36bc7fbc",
"title": "Login process",
"tests": [
{
"pass": false,
"fail": true,
"pending": false,
"context": "\"screenshots/login.spec.js/Login process -- should login #11 (failed).png\""
},
{
"pass": false,
"fail": false,
"pending": true,
"context": null
}
]
}
]
}
]
}
And the JQ command I think it's closest to correct is:
jq '.results[].suites[].tests[].context | strings | sub("#";"%23")'
But the problem is that I need to get in return full edited file. How could I achieve that?
You were close. To retain the original structure, you need to use the update operator (|=) instead of pipe. Enclosing the entire expression to the left of it in parentheses is also necessary, otherwise the original input will be invisible to |=.
(.results[].suites[].tests[].context | strings) |= sub("#"; "%23")
Online demo
change # to %23 in every context value
You might wish to consider:
walk( if type=="object" and (.context|type)=="string"
then .context |= gsub("#"; "%23")
else . end )
I have the following JSON
[
{
"name": "alpha"
},
{
"fields": [
{
"name": "beta_sub_1"
},
{
"name": "beta_sub_2"
}
],
"name": "beta"
},
{
"fields": [
{
"fields": [
{
"name": "gamma_sub_sub_1"
}
],
"name": "gamma_sub_1"
}
],
"name": "gamma"
}
]
and I would like to get the paths of "name" needed to get to each "name" values. Considering the above code, I would like the following result:
"alpha"
"beta.beta_sub_1"
"beta.beta_sub_2"
"beta"
"gamma.gamma_sub_1.gamma_sub_sub_1"
"gamma.gamma_sub_1"
"gamma"
I've been searching around but I couldn't get to this result. So far, I have this:
tostream as [$p,$v] | select($p[-1] == "name" and $v != null) | "\([$p[0,1]] | join(".")).\($v)"
but this gives me the path with the key name of the parents (and doesn't keep all the intermediary parents.
"0.name.alpha"
"1.fields.beta_sub_1"
"1.fields.beta_sub_2"
"1.name.beta"
"2.fields.gamma_sub_sub_1"
"2.fields.gamma_sub_1"
"2.name.gamma"
Any ideas?
P.S.: I've been searching for very detailed doc on jq but couldn't find anything good enough. If anyone has any recommendations, I'd appreciate.
The problem description does not seem to match the sample input and output, but the following jq program produces the required output:
def descend:
select( type == "object" and has("name") )
| if has("fields") then ([.name] + (.fields[] | descend)) else empty end,
[.name] ;
.[]
| descend
| join(".")
With your input, and using the -r command-line option, this produces:
alpha
beta.beta_sub_1
beta.beta_sub_2
beta
gamma.gamma_sub_1.gamma_sub_sub_1
gamma.gamma_sub_1
gamma
Resources
Apart from the jq manual, FAQ, and Cookbook, you might find the following helpful:
"jq Language Description"
"A Stream-Oriented Introduction to jq"
All the examples I've seen so far "reduce" the output (filter out) some part. I understand how to operate on the part of the input I want to, but I haven't figured out how to output the rest of the content "untouched".
The particular example would be an input file with several high level entries "array1", "field1", "array2", "array3" say. Each array contents is different. The specific processing I want to do is to sort "array1" entries by a "name" field which is doable by:
jq '.array1 | sort_by(.name)' test.json
but I also want this output as "array1" as well as all the other data to be preserved.
Example input:
{
"field1": "value1",
"array1":
[
{ "name": "B", "otherdata": "Bstuff" },
{ "name": "A", "otherdata": "Astuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
Expected output:
{
"field1": "value1",
"array1":
[
{ "name": "A", "otherdata": "Astuff" },
{ "name": "B", "otherdata": "Bstuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
I've tried using map but I can't seem to get the syntax correct to be able to handle any type of input other than the array I want to be sorted by name.
Whenever you use the assignment operators (=, |=, +=, etc.), the context of the expression is kept unchanged. So as long as your top-level filter(s) are assignments, in the end, you'll get the rest of the data (with your changes applied).
In this case, you're just sorting the array1 array so you could just update the array.
.array1 |= sort_by(.name)