I get the following JSON result from an external system:
{
"key1": "val1",
"key2": "val2",
"key3": "val3"
}
Now I want to display all keys and all values by using JSONPath. So I am looking for something to get key1, key2 and key3 as a result. Additionally I would like to use the index of a property, e. g. $....[2].key to get "key3" etc.
Is there a way to do something like this?
I found that the tilda ~ symbol is able to retrieve the keys of the values it's called upon. So for your example a query like this:
$.*~
Returns this:
[
"key1",
"key2",
"key3"
]
Another example, if we had a JSON document like this:
{
"key1": "val1",
"key2": "val2",
"key3": {
"key31":"val31",
"key32":"val32"
}
}
A query like this:
$.key3.*~
Would return this:
[
"key31",
"key32"
]
It's important to note that these examples work on JSONPath.com and some other simulators/online tools, but on some they don't. It might come from the fact that I found out about the tilda(~) operator in the JSONPath plus documentation and not the official one.
Related
I'm trying to mimic a post request, using python requests.
The post body is
nonce=b3272d453ca8734f8df1c78ce201f00c&from=01%2F12%2F22&to=31%2F12%2F22&columns%5B%5D=Transaction.DateTimeConverted&columns%5B%5D=Terminal.Id
I have managed to create
{"nonce": "b3272d453ca8734f8df1c78ce201f00c",
"from": "01/12/22",
"to": "31/12/22",
"columns"[]: "Transaction.DateTimeConverted"
"columns"[]: "Terminal.Id"
}
But something isn't working, I suspect it is around my misunderstanding of percentage enncoding representation in Json, aroud the columns? Can someone help?
"key"[] : "value1", "key"[]: "value2" is not valid JSON. you would need to convert it to "key": [ "value1", "value2" ].
I have an object that I can filter to several paths I wish to keep. The paths are of the form:
[
"key1",
"key2",
"mykey"
]
[
"key3",
"key4",
"mykey"
]
What I want is:
{ "key1":{
"key2": .key2
},
"key3":{
"key4": .key4
}
}
The closest I can get is:
{ "key1":{
"key2": .key2
}
}
{ "key3":{
"key4": .key4
}
using:
(paths(objects)|select(last(.[])=="mykey")) as $path|
getpath([$path[0],$path[1]]) as $getpath|
{($path[0]):{($path[1]):$getpath}}
Though I can pipe this output to a jq -s '.' command, I cannot find a way to sum the reconstructions of the paths together within the original set of filters. It appears that the filters reset at the end of each object. $path appears to hold only one path array at a time, rather than be an array of paths. This prevents me from iterating over $path in a reduce function.
I have created the following script that works but I am interested in finding how to use the paths() function, as well. I have not figured out how to make it very useful to me, as yet.
(to_entries |
map(select(.["value"][]?|has("mykey")?))|[.[].key]) as $rooms|
(to_entries |
map(select(.["value"][]?|has("mykey")?))|[.[].value]) as $roomvals| #allows room paths to be avail
##### creates object containing only those locations and sensors within the locations that include "mykey" objs
reduce
range(0;$rooms|length) as $i ({};
.+{($rooms[$i]): ($roomvals[$i] | to_entries |
map(select(.["value"]|has("mykey")))|{(.[]["key"]):.[]["value"]})})
Any assistance on these approaches or the suggestion of alternative approaches is appreciated.
With guidance from peak's responses I can now better pose and answer my question.
I have expanded the data object that peak used in his example so it better represents the problem I was trying to solve. Now the data better illustrates that "key2" and "key10" have "mykey" while "key6", "key8", and "key11" do not. What I was trying to express in my question is that I wanted to retain the full paths of "key2" and "key10" which would include "another" as well as "mykey". I realized that a small modification to peak's example selection criterion would do that. I only need to add a filter to the selection to retain the 1st two elements of each path that includes "mykey":
def data:
{ "key1": { "key2": {"mykey": {"a": 123},
"another":{"q": "six"} },
"key6": {"key7": {"d": 997} } },
"key3": { "key8": {"key9": {"b": 234} },
"key10": {"mykey": {"d": 997},
"another":{"q": "seven"} } },
"key4": { "key11": {"key5" : {"a": 123} } }
};
def selection(mykey):
. as $in
| reduce (paths(objects) | select(last(.[])==mykey)|.[0:2]) as $path
(null; setpath($path; $in|getpath($path)) );
data | selection("mykey")
Which returns what I desired:
{"key1":{"key2":{"mykey":{"a":123},
"another":{"q":"six"}}},
"key3":{"key6":{"mykey":{"d":997},
"another":{"q":"seven"}}}
}
You will note the selection now indcludes "|.[0:2]".
Thanks to peak for setting my on the right track. I have voted for his answer as it helped me better phrase my question and give me the insight to solve it.
The output that you have indicated you want is not JSON, and it's unclear to me what the desired output is, but the following seems to come close:
reduce inputs as $d (null; setpath( $d[0: ($d|length) - 1]; $d[-1] ) )
With your input, and using jq -n, this produces:
{
"key1": {
"key2": "mykey"
},
"key3": {
"key4": "mykey"
}
}
Hopefully, you'll be able to take it from here.
Example:
def data:
{ "key1": { "key2": {"mykey": {"a": 123} } },
"key3": { "key4": {"mykey": {"b": 234} } },
"key4": { "key1": {"key2" : {"a": 123} } }
};
def selection(mykey):
. as $in
| reduce (paths(objects) | select(.[-1] == mykey)) as $path
(null; setpath($path; $in|getpath($path)) );
data | selection("mykey")
Invocation: jq -n -c -f example.jq
Output:
{"key1":{"key2":{"mykey":{"a":123}}},
"key3":{"key4":{"mykey":{"b":234}}}}
I'm trying to flatten a JSON consisting of nested objects. The top layer contains several key/value pairs, where each value is itself an array of a number of objects (the bottom layer).
What I would like to get, using jq, is simply an array of objects containing all the objects of the bottom layer, each of which with an additional key/value pair identifying the top-layer key it originally belonged to.
In other words, I would like to turn a JSON
{
"key1": [obj1, obj2],
"key2": [obj3]
}
into a plain array
[OBJ1, OBJ2, OBJ3]
where each OBJi is simply the original object with an extra key/value pair
"parent-key-name": keyx
where keyx would be the top-layer key obji belonged to, i.e. "key1" for obj1 and obj2, and "key2" for obj3.
I'm struggling with the fact that when referencing the objects in the bottom layer, e.g. via .[], jq does not seem to have inbuilt functionality to access associated top-layer information. However, I'm new to jq, and hope there is an easy solution after all.
Given the following input :
{
"key1": [{"name":"Emma"},{"name":"Bob"}],
"key2": [{"name":"Jean"}]
}
You can divide your items to entries, store the key in a variable and add the value for each item in value object:
jq '[ to_entries[] | .key as $parent | .value[] |
.["parent-key-name"] |= (.+ $parent) ] ' test.json
which gives the following output :
[
{
"name": "Emma",
"parent-key-name": "key1"
},
{
"name": "Bob",
"parent-key-name": "key1"
},
{
"name": "Jean",
"parent-key-name": "key2"
}
]
The solution presented below consists of two steps, each of which might be helpful separately, e.g. if someone wants to "flatten" the JSON in a slightly different way.
First, let's make the changes to obj[i] "in-place":
with_entries( .key as $k | .value[] |= ( . + {"parent-key-name": $k} ) )
Example:
$ jq -n -c -f program.jq
Input:
{
"key1": [{a:1}, {a:2}],
"key2": [{b:3}]
}
Output:
{
"key1": [
{
"a": 1,
"parent-key-name": "key1"
},
{
"a": 2,
"parent-key-name": "key1"
}
],
"key2": [
{
"b": 3,
"parent-key-name": "key2"
}
]
}
To flatten, simply append | [.[]] to the above filter. This produces:
[[{"a":1,"parent-key-name":"key1"},{"a":2,"parent-key-name":"key1"}],[{"b":3,"parent-key-name":"key2"}]]
All the examples I've seen so far "reduce" the output (filter out) some part. I understand how to operate on the part of the input I want to, but I haven't figured out how to output the rest of the content "untouched".
The particular example would be an input file with several high level entries "array1", "field1", "array2", "array3" say. Each array contents is different. The specific processing I want to do is to sort "array1" entries by a "name" field which is doable by:
jq '.array1 | sort_by(.name)' test.json
but I also want this output as "array1" as well as all the other data to be preserved.
Example input:
{
"field1": "value1",
"array1":
[
{ "name": "B", "otherdata": "Bstuff" },
{ "name": "A", "otherdata": "Astuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
Expected output:
{
"field1": "value1",
"array1":
[
{ "name": "A", "otherdata": "Astuff" },
{ "name": "B", "otherdata": "Bstuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
I've tried using map but I can't seem to get the syntax correct to be able to handle any type of input other than the array I want to be sorted by name.
Whenever you use the assignment operators (=, |=, +=, etc.), the context of the expression is kept unchanged. So as long as your top-level filter(s) are assignments, in the end, you'll get the rest of the data (with your changes applied).
In this case, you're just sorting the array1 array so you could just update the array.
.array1 |= sort_by(.name)
I would like to know whether, if a JSON Lines file is structured like this:
{"structure1":"some kind of file header"}
{"structure2": [ {"key1":"aaa", "key2":"bbb"}, {"key1":"one", "key2":"two"}]
{"structure2": [ {"key1":"xxx", "key2":"yyy"}, {"key1":"first", "key2":"second"}]
{"structure3":"maybe some kind of file footer"}
Is it considered a non-valid JSONLines format? I looked at the standard at http://jsonlines.org/ and I couldn't see anything one way or the other.
Thank you.
Your lines are each one valid ( but missing a '}' and the end of the second and third).
but if you want to put them all in one file it will be not valid. They have to be in array and separated with , or an object with key/value ( where the value can be array or object )
example of array :
[
{"structure1":"some kind of file header"},
{"structure2": [ {"key1":"aaa", "key2":"bbb"}, {"key1":"one", "key2":"two"}]},
{"structure2": [ {"key1":"xxx", "key2":"yyy"}, {"key1":"first", "key2":"second"}]},
{"structure3":"maybe some kind of file footer"}
]
or an object :
{
"structure1": "some kind of file header",
"structure2": [{
"key1": "aaa",
"key2": "bbb"
}, {
"key1": "one",
"key2": "two"
}],
"structure2": [{
"key1": "xxx",
"key2": "yyy"
}, {
"key1": "first",
"key2": "second"
}],
"structure3": "maybe some kind of file footer"
}
You can test your json on this site to see if valid or not :
http://www.jslint.com/ or http://jsonlint.com/