jq: Lifting fields from array of objects into the parent object - json

I have a JSON object an array at the top-level, and each array entry is an object with a nested array of objects as one of the fields.
I'd like to "lift" some fields from the sub-arrays into the objects within the first array. It's confusing to write, so, here's my input:
{ "emails": [
{ "email":"foo1#bar.com",
"events":[
{ "type":"open", "time":"t1", "ignore":"this" },
{ "type":"click", "time":"t2", "ignore":"this" } ] },
{ "email":"foo2#bar.com",
"events":[
{ "type":"open", "time":"t3", "ignore":"this" },
{ "type":"click", "time":"t4", "ignore":"this" },
{ "type":"open", "time":"t5", "ignore":"this" } ] }
] }
What I'd like to receive as output is:
[
{ "email":"foo1#bar.com", "type":"open", "time":"t1" },
{ "email":"foo1#bar.com", "type":"click", "time":"t2" },
{ "email":"foo2#bar.com", "type":"open", "time":"t3" },
{ "email":"foo2#bar.com", "type":"click", "time":"t4" },
{ "email":"foo2#bar.com", "type":"open", "time":"t5" }
]
I can jq this with multiple pipes already, like so:
[ .emails[] | { email:.email, event:(.events[] | { type:.type, time:.time }) } | { email:.email, type:.event.type, time:.event.time } ]
... but this seems way too verbose.
Is there an easier way to 'lift' those type and time fields from the deepest objects to the object one-level up?
When I try to use the .events[] iterator twice, I wind up with the Cartesian product of events, which is wrong :-/
I must be missing some simpler way (than my 'intermediate-object' approach above) to achieve this ... anyone know of a better way?

Let's proceed in two steps: first, the (heavy) lifting, and secondly, the (light) trimming.
Lifting
.emails | map( {email} + .events[])
or equivalently:
[.emails[] | {email} + .events[]]
Notice that {"email": .email} has been abbreviated to {email}.
Trimming
We can delete the "ignore" key using del(.ignore). With an eye to efficiency, we arrive at the following solution:
.emails | map( {email} + (.events[] | del(.ignore) ) )

Related

How to filter jq based on the key value?

I am quite new with JQ library, I want to filter the json file using their name(eg. release-1),Then I want to return key value of the all commitId in the same object as the name.
My json file
{
"releases":[
{
"name":[
"release-1"
],
"artifacts":[
{
"name":"pkg-1",
"commitId":"523asdc3"
},
{
"name":"pkg-2",
"commitId":"523asdc3"
},
{
"name":"pkg-3",
"commitId":"523asdc3"
}
]
},
{
"name":[
"release-2"
],
"artifacts":[
{
"name":"pkg-3",
"commitId":"523asdc3"
},
{
"name":"pkg-4",
"commitId":"523asdc3"
},
{
"name":"pkg-5",
"commitId":"523asdc3"
}
]
}
]
}
Expected Output
523asdc3
523asdc3
523asdc3
.releases[] | select(.name | index("release-1")) | .artifacts[].commitId
Will show each commitId where name contains (index()) release-1.
Result:
"523asdc3"
"523asdc3"
"523asdc3"
JqPlay demo
.releases[]|select(.name|contains(["release-1"]))|.artifacts[].commitId
is another way.
.releases[]|select(.name[0] =="release-1")|.artifacts[].commitId
is yet another way.

how to add a list of JSON values into keys with JQ

I have an input file:
{
"errands": [
{
"name": "broker-deregistrar",
"label": "Deregister and Purge Instances",
"impact_warning": null,
"pre_delete": true
},
{
"name": "delete-all-service-instances",
"label": "Delete All Service Instances",
"impact_warning": null,
"pre_delete": true
},
{
"name": "deregister-broker",
"label": "Deregister On-Demand Service Broker",
"impact_warning": null,
"pre_delete": true
}
]
}
I would like to reformat this to make the values of .name into a key with a fixed value like this:
{
"deploy_products": "all",
"errands": {
"product_1_guid": {
"run_pre_delete": {
"broker-deregistrar": true,
"delete-all-service-instances": true,
"deregister-broker": true
}
}
},
"ignore_warnings": true
}
I can subset the values I want with this filter:
.errands[].name
which gives me:
"broker-deregistrar"
"delete-all-service-instances"
"deregister-broker"
but I want to get the selected values into a new JSON as keys.
while this kind of works,
.errands=(.product_1_guid=(.run_pre_delete=(.xxx=true | .yyy=true | .zzz=true)))
the list of errand names is variable in that they have different names and counts. i.e. the list of errands may only be "delete-apps", or even nothing at all.
and in the above example I need .xxx, .yyy and .zzz to come from the original JSON.
Generate the name-true pairs within an array constructor so that you can easily merge them with add and place the result wherever it belongs.
{
deploy_products: "all",
errands: {
product_1_guid: {
run_pre_delete: [
{ (.errands[].name): true }
] | add
}
},
ignore_warnings: true
}
Online demo

convert a JSON to another w/ circe

I would like to convert this JSON
{
"l1k1": {
"l2k1": "l2v1",
"l2k2": 1
},
"l1k2": [
{
"e1l1": "e1v1",
"e1l2": "e1v2"
},
{
"e2l1": "e2v1",
"e2l2": "e2v2"
}
]
}
to this one
{
"papa": {
"l1k1c": {
"l2k1c": {
"string": "l2v1"
},
"l2k2c": {
"int": 1
}
},
"l1k2c": {
"array": [
{
"e1l1": "e1v1",
"e1l2": "e1v2"
},
{
"e2l1": "e2v1",
"e2l2": "e2v2"
}
]
}
}
}
where:
"l" stands for level
"k" for key, "v" for value
"e" for element
"c" for copy (where "*" maps to "*c")
I'm using circe's Json but having a hard time renaming the keys or creating parents or children with it. As I'm writing this, I'm thinking I may need to use its ACursor instead. As you may have guessed, I'm trying to generate an AVRO doc from an input JSON. I'm open to help w/ my approach or any suggestions about how to go about it in a cleaner way.

Obtaining the child values randomly in JSON

I have a JSON like below: I need to extract the Options -> Child as a Random and also Values within the options as randomly. How can we achieve in jmeter ?
{
"id":37,
"merchant_id":"39",
"title":"Parker Pens",
"subtitle":null,
"price":1000,
"description":null,
"images":[ ],
"image_thumbs":[ ],
"options":[
{
"code":"color",
"label":"Color",
"extra_info":"",
"values":[
{ },
{ },
{ }
]
},
{
"code":"size",
"label":"Size",
"extra_info":"",
"values":[
{ },
{ },
{ }
]
}
],"options_available":[
{ },
{ },
{ },
{ },
{ },
{ },
{ },
{ },
{ }
], "custom_options":[
]
}
I have to fetch the child of options randomly . In that i have to fetch the value of "Code" and its associated value within the "Value" .
Help is appreciated and useful
Your requirements are a little bit vague as you haven't indicated what is the desired output format. One of the solutions would be using JSR223 PostProcessor in order to obtain the random value from random options array like:
import com.jayway.jsonpath.JsonPath
import org.apache.commons.lang3.RandomUtils
import org.apache.jmeter.samplers.SampleResult
def options = JsonPath.read(prev.getResponseDataAsString(), '$.options')
def randomOption = options.get(RandomUtils.nextInt(0, options.size()))
def values = randomOption.get('values')
def randomValue = values.get(RandomUtils.nextInt(0, values.size())) as String
vars.put('randomValue', randomValue)
References:
Jayway JsonPath - A Java DSL for reading JSON documents
Apache Groovy - Why and How You Should Use It
Apache Groovy - Parsing and Producing JSON

jq: How can I combine data from duplicate keys

I have a fairly complex JSON data structure that I've managed to use jq to filter down to certain keys and their values. I need to combine the results though, so duplicate keys have only one array of values.
e.g.
{
"1.NBT.B": [
{
"id": 545
},
{
"id": 546
}
]
},
{
"1.NBT.B": [
{
"id": 1281
},
{
"id": 1077
}
]
}
would result in
{
"1.NBT.B": [
{
"id": 545
},
{
"id": 546
},
{
"id": 1281
},
{
"id": 1077
}
]
},
...
or even better:
[{"1.NBT.B": [545, 546, 1281, 1077]}, ...]
I need to do it without having to put in the key ("1.NBT.B") directly, since there are hundreds of these keys. I think what has me most stuck is that the objects here aren't named -- the keys are not the same between objects.
Something like this only gives me the 2nd set of ids, completing skipping the first:
reduce .[] as $item ({}; . + $item)
Part 1
The following jq function combines an array of objects in the manner envisioned by the first part of the question.
# Given an array of objects, produce a single object with an array at
# every key, the array at each key, k, being formed from all the values at k.
def merge:
reduce .[] as $o ({}; reduce ($o|keys)[] as $key (.; .[$key] += $o[$key] ));
With this definition together with the line:
merge
in a file, and with the example input modified to be a valid JSON array,
the result is:
{
"1.NBT.B": [
{
"id": 545
},
{
"id": 546
},
{
"id": 1281
},
{
"id": 1077
}
]
}
Part 2
With merge as defined above, the filter:
merge | with_entries( .value |= map(.id) )
produces:
{
"1.NBT.B": [
545,
546,
1281,
1077
]
}