Get value of JSON object using jq --stream - json

I'm trying to extract the value of an JSON object using jq --stream, because the real data can the size of multiple GigaBytes.
This is the JSON I'm using for my tests, where I want to extract the value of item:
{
"other": "content here",
"item": {
"A": {
"B": "C"
}
},
"test": "test"
}
The jq options I'm using:
jq --stream --null-input 'fromstream(inputs | select(.[0][0] == "item"))[]' example.json
However, I don't get any output with this command.
A strange thing I found is that when removing the object after the item the above command seems to work:
{
"other": "content here",
"item": {
"A": {
"B": "C"
}
}
}
The result looks as expected:
❯ jq --stream --null-input 'fromstream(inputs | select(.[0][0] == "item"))[]' example.json
{
"A": {
"B": "C"
}
}
But as I cannot control the input JSON this is not the solution.
I'm using jq version 1.6 on MacOS.

You didn't truncate the stream, therefore after filtering it to only include the parts below .item, fromstream is missing the final back-tracking item [["item"]]. Either add it manually at the end (not recommended, this would also include the top-level object in the result), or, much simpler, use 1 | truncate_stream to strip the first level altogether:
jq --stream --null-input '
fromstream(1 | truncate_stream(inputs | select(.[0][0] == "item")))
' example.json
{
"A": {
"B": "C"
}
}
Alternatively, you can use reduce and setpath to build up the result object yourself:
jq --stream --null-input '
reduce inputs as $in (null;
if $in | .[0][0] == "item" and has(1) then setpath($in[0];$in[1]) else . end
)
' example.json
{
"item": {
"A": {
"B": "C"
}
}
}
To remove the top level object, either filter for .item at the end, or, similarly to truncate_stream, remove the path's first item using [1:] to strip the first level:
jq --stream --null-input '
reduce inputs as $in (null;
if $in | .[0][0] == "item" and has(1) then setpath($in[0][1:];$in[1]) else . end
)
' example.json
{
"A": {
"B": "C"
}
}

Related

JQ - Deep child value replace through wildcard search and merge to the original JSON

My question is similar to Unix jq parsing wildcards but want to merge to the original JSON.
Say, input JSON:
{
"a": {
"1": {
"c": "text1"
},
"999": {
"c": "text99"
}
}
}
I want to manipulate the inner "c": "text1" and modify it to "c": "newtext". But, also need to merge with the original JSON. IOW, it is not about extracting, but about manipulating.
Expected output:
{
"a": {
"1": {
"c": "newtext"
},
"999": {
"c": "text99"
}
}
}
I tried:
.. | .c? |= (sub("^text1$";"newtext"))
But, it throws null (null) cannot be matched, as it is not a string
jqplay: https://jqplay.org/s/2nFAus6Umz
Just walk the path with the expression to select an object type when .c equals to your desired value
jq 'walk(if type == "object" and .c == "text1" then .c |= "newtext" else . end)'
jqplay demo
Along the lines of your attempt:
(.. | objects | select(has("c")) | .c) |= (sub("^text1$";"newtext"))
or in order of increasing brevity:
(.. | select(try has("c")) | .c) |= (sub("^text1$";"newtext"))
(.. | select(has("c")?) | .c) |= (sub("^text1$";"newtext"))

jq simple json object merge/combination

a.json
{"a": 1}
b.json
{"b": 1}
Desired outcome
{"a": 1, "b": 1}
jq -s "." a.json b.json
[
{
"a": 1
},
{
"b": 1
}
]
It's wrapped in an array
jq "." a.json b.json
{
"a": 1
}
{
"b": 1
}
That's not even valid json
Is jq the wrong tool here? What is more appropriate?
Try:
jq -s 'add' a.json b.json
Result:
{
"a": 1,
"b": 1
}
In some cases it may be desirable to avoid “slurping” the objects, as that requires more memory than necessary.
In any case, to accomplish the task economically, use -n in conjunction with inputs as follows:
reduce inputs as $i ({}; . + $i)
sigma/1
If you don't mind that sigma(empty) evaluates to null, you could define a polymorphic sigma as follows:
def sigma(s): reduce s as $x (null; . +$x);
This works on streams of numbers, streams of objects, streams of arrays, and streams of strings, and so would be suitable for your standard library.
In any case, with this def, for the task at hand, you could write: simga(inputs).

jq: How can a nested object be sliced to include the parent-key hierarchy?

Using Bash and jq, if I have a Bash variable filter F of the form .<key1>.<key2>...<keyN> and I want to slice a Bash variable JSON object O so that the result is just that slice of the object including all keys in F, how can this be done with jq?
For example, suppose:
O='
{
"a":
{
"b":
{
"c": { "p":1 },
"x": 1
},
"x": 2
},
"x": 3
}'
Then, doing:
F='.a.b.c'; jq -r "$F" <<<"$O"
results in:
{
"p": 1
}
But, I want the slice to include parent key hierarchy.
Inelegant Solution
I have come up with a solution, but it involves 2 calls to jq:
F='.a.b.c'; S="$(jq -r "$F" <<<"$O"); jq --null-input -r "$F |= $S"
that results in:
{
"a": {
"b": {
"c": {
"p": 1
}
}
}
}
The solution must work for any valid O and F Bash variable where O stores a JSON object and F is a simple filter of key names only as described above. For example:
F='.a.b'; S="$(jq -r "$F" <<<"$O")"; jq --null-input -r "$F |= $S"
results in:
{
"a": {
"b": {
"c": {
"p": 1
},
"x": 1
}
}
}
Can slicing an object with a key-hierarchy filter be done more simply in jq?
Provided $F is a valid jq path expression (i.e., so that jq -n "$F" works):
jq "$F as \$v | null | $F |= \$v" <<< "$O"
(I included the |= from your solution to show the similarity, but here you could drop the |.)

Using jq to update objects within a JSON document if the value corresponding to a given key starts with a specified string

I have the given JSON and want to change the id value of all elements, which starts with test in the name element:
{
"other-value": "some-id",
"values": [
{
"name": "test-2017-12-01",
"id": "1"
},
{
"name": "othert",
"id": "2"
}
]
}
The following jq commands works jqplay
jq (.values[] | select(.name == "test-afs").id) |= "NEWID"
But when I try it with startswith it stops working, what am I missing? jqplay
(.values[] | select(.name | startswith("test")).id) |= "NEWID"
jq: error (at :14): Invalid path expression near attempt to access element "id" of {"name":"test-afs","id":"id"}
exit status 5
You can also use map, like this:
jq '(.values)|=(map((if .name|startswith("test") then .id="NEWID" else . end)))' file
Output:
{
"other-value": "some-id",
"values": [
{
"name": "test-2017-12-01",
"id": "NEWID"
},
{
"name": "othert",
"id": "2"
}
]
}
Please note that since the release of jq 1.5, jq has been enhanced to support the query that previously failed. For example, using the current 'master' version:
jq -c '(.values[] | select(.name | startswith("test")).id) |= "NEWID"'
{"other-value":"some-id","values":[{"name":"test-2017-12-01","id":"NEWID"},{"name":"othert","id":"2"}]}
Using earlier versions of jq, if/then/else/end can be used in this type of situation as follows:
.values[] |= if .name | startswith("test") then .id = "NEWID" else . end
If using map, a minimalist expression would be:
.values |= map(if .name|startswith("test") then .id = "NEWID" else . end)

Json parsing on cli using jq

Let's say I have the below json object:
{
"d": {
"e": {
"bar": 2
}
},
"a": {
"b": {
"c": {
"foo": 1
}
}
}
}
I want to get the value foo without typing '.a.b.c.foo'
I realize I can do...
echo '{ "a":{"b":{"c":{ "foo":1}}},"d":{"e":{"bar":2}}}' | jq '.[][][].foo' but is there a recursive wild in jq? like **? I know for sure jq doesn't support *, is there a way to have jq support jsonpath?
Or maybe even just another cli tool that does support json path?
In jq 1.4 you could do this:
$ jq '..|.foo?' file.json
If you're stuck with 1.3 you could use
$ jq 'recurse(if type == "array" or type == "object" then .[] else empty end) | if type == "object" then . else empty end | .foo' file.json
which is a bit of a mouthful... That's why 1.4 has .., which recurses down through all iterables in ., and the ? operator, which doesn't bother indexing that which can't be.