how to merge two JSON objects using jq? - json

Question

I have two file json. I want to append two array of SomeFile2.json to SomeFile1.json as below.
SomeFile1.json
[
{
"DNSName": "CLB-test-112a877451.ap-northeast-1.elb.amazonaws.com",
"Instances": [
{
"InstanceId": "i-0886ed703de64028a"
}
]
},
{
"DNSName": "CLB-test1-156925981.ap-northeast-1.elb.amazonaws.com",
"Instances": [
{
"InstanceId": "i-0561634c4g3b4fa25"
}
]
}
]
SomeFile2.json
[
{
"InstanceId": "i-0886ed703de64028a",
"State": "InService"
},
{
"InstanceId": "i-0561634c4g3b4fa25",
"State": "InService"
}
]
I want the result as below:
[
{
"DNSName": "CLB-test-112a877451.ap-northeast-1.elb.amazonaws.com",
"Instances": [
{
"InstanceId": "i-0886ed703de64028a"
"State": "InService"
}
]
},
{
"DNSName": "CLB-test1-156925981.ap-northeast-1.elb.amazonaws.com",
"Instances": [
{
"InstanceId": "i-0561634c4g3b4fa25"
"State": "InService"
}
]
}
]
I'm processing in bash shell via jq. But, unsuccessful.

Answer 1

Since the contents of the second file are evidently intended to define a mapping from InstanceId to State, let's start by hypothesizing the following invocation of jq:
jq --argfile dict SomeFile2.json -f program.jq SomeFile1.json
Next, let's create a suitable dictionary:
reduce $dict[] as $x ({}; . + ($x|{(.InstanceId): .State}))) as $d
Now the rest is easy:
map(.Instances |= map(. + {State: $d[.InstanceId]}))
Putting the pieces together in program.jq:
(reduce $dict[] as $x ({}; . + ($x|{(.InstanceId): .State}))) as $d
| map(.Instances |= map(. + {State: $d[.InstanceId]}))
Alternatives
The dictionary as above can be constructed without using reduce, as follows:
($dict | map( {(.InstanceId): .State}) | add) as $d
Another alternative is to use INDEX/2:
(INDEX($dict[]; .InstanceId) | map_values(.State))) as $d
If your jq does not have INDEX/2 you can snarf its def from
https://raw.githubusercontent.com/stedolan/jq/master/src/builtin.jq

Answer 2

Since I find jq pretty hard, I started in a procedural way: using ruby's json module:
ruby -rjson -e '
states = JSON.parse(File.read(ARGV.shift)).map {|o| [o["InstanceId"], o["State"]]}.to_h
data = JSON.parse(File.read(ARGV.shift))
data.each do |obj|
obj["Instances"].each do |instance|
instance["State"] = states[instance["InstanceId"]] || "unknown"
end
end
puts JSON.pretty_generate data
' SomeFile2.json SomeFile1.json
But we want jq, so after some trial and error, and finding this in the manual: https://stedolan.github.io/jq/manual/#Complexassignments -- (note, I changed the state for one of the instances so I could verify the output better)
$ cat SomeFile2.json
[
{
"InstanceId": "i-0886ed703de64028a",
"State": "InService"
},
{
"InstanceId": "i-0561634c4g3b4fa25",
"State": "NOTInService"
}
]
First, extract the states into an object mapping the id to the state:
$ state_map=$( jq -c 'map({"key":.InstanceId, "value":.State}) | from_entries' SomeFile2.json )
$ echo "$state_map"
{"i-0886ed703de64028a":"InService","i-0561634c4g3b4fa25":"NOTInService"}
Then, update the instances in the first file:
jq --argjson states "$state_map" '.[].Instances[] |= . + {"State": ($states[.InstanceId] // "unknown")}' SomeFile1.json
[
{
"DNSName": "CLB-test-112a877451.ap-northeast-1.elb.amazonaws.com",
"Instances": [
{
"InstanceId": "i-0886ed703de64028a",
"State": "InService"
}
]
},
{
"DNSName": "CLB-test1-156925981.ap-northeast-1.elb.amazonaws.com",
"Instances": [
{
"InstanceId": "i-0561634c4g3b4fa25",
"State": "NOTInService"
}
]
}
]