Using JQ to inject element at specific position based on conditions - json

Pulling my hair here, trying to use jq to parse and extend a JSON file and adding an element at a specific position based on a certain condition.
Here is a sample file (also at https://jqplay.org/s/6cjmbnvrqu)
{
"TopA": { "stuff": "here"},
"TopB": {
"C591AB7E": {
"Type": "this",
"Properties": {
"lots": 1,
"of": 2,
"values" : 3
}
},
"7E16765A": {
"Type": "this",
"Properties": {
"lots": 4,
"of": 5,
"values" : 6
}
},
"AAD76465": {
"Type": "that",
"Properties": {
"lots": 7,
"of": 8,
"values" : 9
}
}
}
}
The goal is to add an element to the Properties node of any TopB child where .Type == "that". And the kicker is that I need to put the child node's key into the new element value with an added prefix.
So essentially I need the last element to look like this:
"AAD76465": {
"Type": "that",
"Properties": {
"lots": 7,
"of": 8,
"values": 9,
"newElement": "Prefix-AAD76465"
}
}
I also need to retain the whole rest of the file (or a new file for that matter). So I don't need a query but really a jq call to manipulate the existing file. Parallel to TopB there could be other elements that I'd still need in the file. And no, I don't know, neither do I have control over the naming of the children of TopB. All I have is that my targets are nested children of TopB with .Type == "that". There can be multiple of those.
Thanks for looking.

If you get the path to where you will be adding the new field first, you can simply extract the parent key from that.
reduce path(.TopB[] | select(.Type == "that") .Properties.newElement) as $p (.; setpath($p; "Prefix-\($p[1])"))
Online demo

Here'a solution that uses walk/1 and is therefore perhaps quite intuitive:
walk(if (type == "object") and .TopB
then .TopB |= with_entries(
.key as $key
| if .value.Type == "that"
then .value.Properties += { newElement: ("Prefix-" + $key) }
else . end)
else . end)
One advantage of this solution is that it meets the stated requirement:
The goal is to add an element to the Properties node of any TopB child ...
in the sense that no assumptions are made about the locations of objects with the "topB" key.

Related

Filtering JSON using jq with a condition

I have the following JSON:
{
"LaunchTemplates": [
{
"LaunchTemplateName": "bla-99",
"CreateTime": "2022-12-13T13:40:33+00:00"
},
{
"LaunchTemplateName": "abcabc",
"CreateTime": "2022-12-13T09:58:14+00:00"
},
{
"LaunchTemplateName": "bla-34",
"CreateTime": "2022-12-13T13:58:56+00:00"
},
{
"LaunchTemplateName": "bla-222",
"CreateTime": "2022-12-11T13:58:56+00:00"
},
{
"LaunchTemplateName": "bla-233",
"CreateTime": "2022-12-10T13:58:56+00:00"
}
]
}
I want to filter the JSON and print the oldest templates after filtering. I have the following jq query that prints the template names after filtering:
file.json | jq '.LaunchTemplates[].LaunchTemplateName|select(startswith("bla"))'
Output:
bla-99
bla-34
bla-222
bla-233
Now i want to add more logic to the query, and do something like that: If the number of bla lines is bigger than 3, then print the oldest bla lines (according to the date field). In my case, the output should be:
bla-233
Is that possible with jq or other shell commands? If so, how?
If you are only interested in the last two elements (and ignoring the fact that the input in the question is invalid JSON):
.LaunchTemplates
| sort_by(.CreateTime)
| map(.LaunchTemplateName|select(startswith("")))[:-3]
| reverse[]
sort_by(.CreateTime) sorts ascending by the CreateTime property.
map(.LaunchTemplateName|select(startswith("bla")) maps the input array to an array containing only the template names with a "bla" prefix.
[:-3] slices the input array to drop the last 3 elements (i.e. dropping the 3 newest elements.
reverse[] reverses the array and streams its elements.
Output:
"bla-222"
"bla-233"

JSON path filter for arrays

I'm learning JSON paths and I'm struggling with one issue. Unfortunatelly I cannot show the original file I'm working with but I recreated the issue with this simple JSON:
{
"store":{
"book":[
{
"category":"reference",
"author":"Nigel Rees",
"title":"Sayings of the Century",
"price":[
{
"originalEnglish": 8.95,
"currentEnglish":11
},
{
"originalSpanish":11,
"currentSpanish":13
}
]
}
]
}
}
In the original one there's more objects and arrays but basically what I'm trying to do is filter all books with category 'reference' and with originalEnglish price below 10.
I used:
$..book[?(#.price.originalEnglish < 10 && #.category == 'reference')]
but it doesn't work. If anyone could help me out with this, I'd really appreciate it.
This works for me on http://jsonpath.herokuapp.com/
$..book[?(#.price[?(#.originalEnglish < 10)] && #.category == 'reference')]

Expand references to sibling values in a JSON using JQ

I have a JSON document holding configurations with macros. I need to expand macros that reference other elements. For simplicity, consider this input document.
[
{
"first": "Tim",
"Full": "{first} {last}",
"last": "Smith"
},
{
"first": "Jane",
"Full": "{first} {last}",
"last": "Doe"
}
]
Performance is not paramount here so I don't mind blindly checking for every element occurring in every other element.
I worked out the following logic as a proof of concept. But can't figure out how to add a nested loop on $lookup to update the other with_entries value.
jq '
.[]
| . as $lookup
| with_entries(
.value=(
.value
| sub("{first}"; $lookup.first)
)
)
'
JQ processes streams of values/documents and it feels like I now need to work with two streams. Which I hoped to accomplish by using $lookup for the back reference. Now I am stuck.
SOLUTION
oguz ismail provided a great solution for the original question (Please give them a +1) that uses map_values(). It is very clear and I wanted to include it here for reference. You will note that it uses a named group (?<found_key>.*?) in the regular expression (see oniguruma's named group)
map(. as $lookup | map_values(gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])))
I asked how to process when there are non-string elements in the structure. Here is an example that includes an array of colors:
[
{
"first": "Tim",
"Full": "{first} {last}",
"last": "Smith",
"colors": ["red", "blue"]
}
]
oguz ismail provided a solution for this structure as well that only attempts to modify elements that are strings:
map(
. as $lookup
| (.[] | strings)
|= gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])
)
You can use gsub with a named capture group for expanding all the macros in a single run without hardcoding their names. And, with_entries doesn't help at all in this case; use map_values instead.
map(. as $lookup | map_values(gsub("\\{(?<found_key>.*?)\\}"; $lookup[.found_key])))
Online demo

Add a new entry to a Json after or before a specified existing entry with jq

I have the following input and I want to below output using jq.
I would like to add
an entry "zeiterfassungAktiviert" : truejust after the key gueltigBis (or alternatively before the key inhaltsverzeichnis)
add an entry to the end of the object inhaltsverzeichnis by adding the entry "zeiterfassung": "zeiterfassung"
Example input:
{
"fachbereich": "qp",
"produktTyp": "PRODUKT_ANFRAGE_V1",
"name": "Produkt Anfrage",
"kurzName": "anfrage",
"gueltigAb": "2019-01-01T00:00:00.000",
"gueltigBis": "2022-12-31T00:00:00.000",
"inhaltsverzeichnis": {
"versandumfang": "auftragsverwaltung/versandumfang",
"dokumentenerzeugung": "dokumentenerzeugung"
}
}
Example output:
{
"fachbereich": "qp",
"produktTyp": "PRODUKT_ANFRAGE_V1",
"name": "Produkt Anfrage",
"kurzName": "anfrage",
"gueltigAb": "2019-01-01T00:00:00.000",
"gueltigBis": "2022-12-31T00:00:00.000",
"zeiterfassungAktiviert": true,
"inhaltsverzeichnis": {
"versandumfang": "auftragsverwaltung/versandumfang",
"dokumentenerzeugung": "dokumentenerzeugung",
"zeiterfassung": "zeiterfassung"
},
}
I managed to do the second part but am not clear about how to go about the first part.
Command: jq '.zeiterfassungAktiviert += "zeiterfassung" | .inhaltsverzeichnis.zeiterfassung += "zeiterfassung"'
The result is as follows:
{
"fachbereich": "qp",
"produktTyp": "PRODUKT_ANFRAGE_V1",
"name": "Produkt Anfrage",
"kurzName": "anfrage",
"gueltigAb": "2019-01-01T00:00:00.000",
"gueltigBis": "2022-12-31T00:00:00.000",
"inhaltsverzeichnis": {
"versandumfang": "auftragsverwaltung/versandumfang",
"dokumentenerzeugung": "dokumentenerzeugung",
"zeiterfassung": "zeiterfassung"
},
"zeiterfassungAktiviert": true
}
As you can see it is added to the end of the root object. I would like to specify the position somehow, ideally without having to convert into array and convert back if possible but rather by saying please add entry after/before a specified key.
For the first part, it helps to have a helper function:
def insertkv($afterkey; $key; $value):
def insertafter($ix; $x): .[0:1+$ix] + [$x] + .[1+$ix:];
to_entries
| insertafter( map(.key) | index($afterkey); {$key, $value})
| from_entries;
Versions
The above is intended for use with jq 1.5 or later. Some minor fiddling is required for earlier versions.

jq - Selecting objects containing certain key

Let's say I have this JSON file below:
{
"team": {
"money": 100,
},
"group": {
"money": 200,
"snack": true,
}
}
I want to select the objects which has a "snack" key including its parent. The current command I'm using is:
jq '..|objects|select(has("snack"))' json
This however, does not include the parent, which in this case is "group". How do I select the parent of the selected object as well?
Instead of using .., you could use paths. That is, you'd select the paths that lead to the items of interest, and work from there. So you'd start with:
paths(objects) as $p
| select(getpath($p)|has("snack"))
| $p
For the given input (after having been corrected), this would yield:
["group"]
So you might want to replace the $p in the last line by $p[-1], but it's not altogether clear how useful that would be. More useful would be getpath( $p[:-1] )