I am very beginner into jq and I want to reshape my JSON file.
I have ve got JSON structured like this:
{
"a": [1, 2, 3, 4 ...],
"b": [
{
"x": 1000,
"value": 1
},
{
"x": 1000,
"value": 2
},
{
"x": 1000,
"value": 3
}
...
]
}
I am wondering how I can achieve result like this with jq:
[
{
"value": 1,
"from": "a",
},
{
"value": 2,
"from": "a"
},
...
{
"value": 1,
"from": "b"
},
{
"value": 2,
"from": "b"
}
...
]
Here is a very slightly generic, but hardly robust, solution:
map_values( if type == "array"
then map(if type == "object" then .value else . end)
else . end)
| [ keys_unsorted[] as $k
| .[$k][] as $v
| { value: $v, from: $k } ]
Create two lists, one from .a and one from .b, and merge them with +.
In the first list, create objects with value: set to the original content and add from: "a"; in the second list, remove .x from elements of .b and add the from again.
jq '[.a[] | {value:(.), from: "a"}] + [.b[] | del(.x) + {from: "b"}]
Related
This question already has answers here:
How do i add an index in jq
(3 answers)
Closed 2 years ago.
Considering input:
[
{
"a": 1
},
{
"a": 2
},
{
"a": 7
}
]
how do I add new field to each object, which value would be index in array? Producing:
[
{
"a": 1,
"index": 0
},
{
"a": 2,
"index": 1
},
{
"a": 7,
"index": 2
}
]
Using reduce, without disassembling/reassembling the input:
reduce range(length) as $index (.; .[$index] += {$index})
Online demo
Store the structure into a variable, then use keys to get the indices, retrieve the corresponding object from the variable using the index and add the index to it:
jq '[ . as $d | keys[] | $d[.] + {index:.} ]' file.json
to_entries takes an object and returns an array of key/value pairs.
It can be used effectively and intuitively.
jq/to_entries
input file
// file.json
[
{
"a": 1
},
{
"a": 2
},
{
"a": 7
}
]
commands
jq 'to_entries | map(.value+{index:.key})' file.json
results
[
{
"a": 1,
"index": 0
},
{
"a": 2,
"index": 1
},
{
"a": 7,
"index": 2
}
]
Here's also an alternative (non-jq) solution, using jtc:
bash $ <input.json jtc -w'[:]<I>k' -i'{"index":{I}}'
[
{
"a": 1,
"index": 0
},
{
"a": 2,
"index": 1
},
{
"a": 7,
"index": 2
}
]
bash $
PS. I'm a developer of jtc - unix JSON processor
PPS. The disclaimer is required by SO.
Try : Array[index_of_object].property = value
for example : array[0].a = 1
I want to sort this data structure by the object keys (easy with -S and sort the object values (the arrays) by the 'foo' property.
I can sort them with
jq -S '
. as $in
| keys[]
| . as $k
| $in[$k] | sort_by(.foo)
' < test.json
... but that loses the keys.
I've tried variations of adding | { "\($k)": . }, but then I end up with a list of objects instead of one object. I also tried variations of adding to $in (same problem) or using $in = $in * { ... }, but that gives me syntax errors.
The one solution I did find was to just have the separate objects and then pipe it into jq -s add, but ... I really wanted it to work the other way. :-)
Test data below:
{
"": [
{ "foo": "d" },
{ "foo": "g" },
{ "foo": "f" }
],
"c": [
{ "foo": "abc" },
{ "foo": "def" }
],
"e": [
{ "foo": "xyz" },
{ "foo": "def" }
],
"ab": [
{ "foo": "def" },
{ "foo": "abc" }
]
}
Maybe this?
jq -S '.[] |= sort_by(.foo)'
Output
{
"": [
{
"foo": "d"
},
{
"foo": "f"
},
{
"foo": "g"
}
],
"ab": [
{
"foo": "abc"
},
{
"foo": "def"
}
],
"c": [
{
"foo": "abc"
},
{
"foo": "def"
}
],
"e": [
{
"foo": "def"
},
{
"foo": "xyz"
}
]
}
#user197693 had a great answer. A suggestion I got in a private message elsewhere was to use
jq -S 'with_entries(.value |= sort_by(.foo))'
If for some reason using the -S command-line option is not a satisfactory option, you can also perform the by-key sort using the to_entries | sort_by(.key) | from_entries idiom. So a complete solution to the problem would be:
.[] |= sort_by(.foo)
| to_entries | sort_by(.key) | from_entries
I've got JSON in the format
{
"a": {
"size":3
},
"b": {
"size":2
},
"c": {
"size":1
}
}
I need to sort it by size, e.g.:
{
"c": {
"size": 1
},
"b": {
"size": 2
},
"a": {
"size": 3
}
}
I have found a way to do it, e.g.:
. as $in | keys_unsorted | map ({"key": ., "size" : $in[.].size}) | sort_by(.size) | map(.key | {(.) : $in[.]}) | add
but this seems quite complex so I'm hoping there's a simpler way that I've overlooked?
You can use to_entries / from_entries, like this:
jq 'to_entries|sort_by(.value.size)|from_entries' file.json
to_entries will transform your input object into a list of key/value pair objects:
[
{
"key": "a",
"value": {
"size": 3
}
},
...
{
"key": "c",
"value": {
"size": 1
}
}
]
That allows to apply sort_by(.value.size) to that list and then convert it back to an object using from_entries.
I have a JSON document that looks like the following. Note this is a simplified example of the real JSON, which is included at bottom of question:
{
"some_array": [
{
"k1": "A",
"k2": "XXX"
},
{
"k1": "B",
"k2": "YYY"
}
]
}
I would like to change the value of all the k2 keys in the some_array array where the value of the k1 key is "B".
Is this possible using jq ?
For reference this is the actual JSON document, which is an environment variable file for use in postman / newman tool. I am attempting this conversion using JQ because the tool does not yet support command line overrides of specific environment variables
Actual JSON
{
"name": "Local-Stack-Env-Config",
"values": [
{
"enabled": true,
"key": "KC_master_host",
"type": "text",
"value": "http://localhost:8087"
},
{
"enabled": true,
"key": "KC_user_guid",
"type": "text",
"value": "11111111-1111-1111-1111-11111111111"
}
],
"timestamp": 1502768145037,
"_postman_variable_scope": "environment",
"_postman_exported_at": "2017-08-15T03:36:41.474Z",
"_postman_exported_using": "Postman/5.1.3"
}
Here is a slightly simpler version of zayquan's filter:
.some_array |= map(if .k1=="B" then .k2="changed" else . end)
Here's another solution.
jq '(.some_array[] | select(.k1 == "B") | .k2) |= "new_value"'
Output
{
"some_array": [
{
"k1": "A",
"k2": "XXX"
},
{
"k1": "B",
"k2": "new_value"
}
]
}
Here is a viable solution:
cat some.json | jq '.some_array = (.some_array | map(if .k1 == "B" then . + {"k2":"changed"} else . end))'
produces the output:
"some_array": [
{
"k1": "A",
"k2": "XXX"
},
{
"k1": "B",
"k2": "changed"
}
]
}
jq strikes again. Trying to get the value of DATABASES_DEFAULT based on the name in a json file that has a whole lot of names and I'm completely lost.
My file looks like the following (output of an aws ecs describe-task-definition) only much more complex; I've stripped this to the most basic example I can where the structure is still intact.
{
"taskDefinition": {
"status": "bar",
"family": "bar2",
"volumes": [],
"taskDefinitionArn": "bar3",
"containerDefinitions": [
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo"
}
],
"name": "baz",
"links": []
},
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo2"
}
],
"name": "boo",
"links": []
}
],
"revision": 1
}
}
I need the value of DATABASES_DEFAULT where the name is baz. Note that there are a lot of keypairs with name, I'm specifically talking about the one outside of environment.
I've been tinkering with this but only got this far before realizing that I don't understand how to access nested values.
jq '.[] | select(.name==DATABASES_DEFAULT) | .value'
which is returning
jq: error: DATABASES_DEFAULT/0 is not defined at <top-level>, line 1:
.[] | select(.name==DATABASES_DEFAULT) | .value
jq: 1 compile error
Obviously this a) doesn't work, and b) even if it did, it's independant of the name value. My thought was to return all the db defaults and then identify the one with baz, but I don't know if that's the right approach.
I like to think of it as digging down into the structure, so first you open the outer layers:
.taskDefinition.containerDefinitions[]
Now select the one you want:
select(.name =="baz")
Open the inner structure:
.environment[]
Select the desired object:
select(.name == "DATABASES_DEFAULT")
Choose the key you want:
.value
Taken together:
parse.jq
.taskDefinition.containerDefinitions[] |
select(.name =="baz") |
.environment[] |
select(.name == "DATABASES_DEFAULT") |
.value
Run it like this:
<infile jq -f parse.jq
Output:
"foo"
The following seems to work:
.taskDefinition.containerDefinitions[] |
select(
select(
.environment[] | .name == "DATABASES_DEFAULT"
).name == "baz"
)
The output is the object with the name key mapped to "baz".
$ jq '.taskDefinition.containerDefinitions[] | select(select(.environment[]|.name == "DATABASES_DEFAULT").name=="baz")' tmp.json
{
"dnsSearchDomains": [],
"environment": [
{
"name": "bar4",
"value": "bar5"
},
{
"name": "bar6",
"value": "bar7"
},
{
"name": "DATABASES_DEFAULT",
"value": "foo"
}
],
"name": "baz",
"links": []
}