jq update contents of one file to another as key value - json

I am trying to update branches.json.branch2 values from branch2.json.Employes values
Using jq, How can I merge content of one file to another file
Below are the files
I have tried this but it did work, it just prints the original data without updating the details
#!/bin/sh
#call file with branch name for example ./update.sh branch2
set -xe
branchName=$1
fullPath=`pwd`/$1".json"
list=$(cat ${fullPath})
branchDetails=$(echo ${list} | /usr/local/bin/jq -r '.Employes')
newJson=$(cat branches.json |
jq --arg updateKey "$1" --arg updateValue "$branchDetails" 'to_entries |
map(if .key == "$updateKey"
then . + {"value":"$updateValue"}
else .
end) |
from_entries')
echo $newJson &> results.json
branch1.json
{
"Employes": [
{
"Name": "Ikon",
"age": "30"
},
{
"Name": "Lenon",
"age": "35"
}
]
}
branch2.json
{
"Employes": [
{
"Name": "Ken",
"age": "40"
},
{
"Name": "Frank",
"age": "23"
}
]
}
brances.json / results.json fromat
{
"branch1": [
{
"Name": "Ikon",
"age": "30"
},
{
"Name": "Lenon",
"age": "35"
}
],
"branch2": [
{
"Name": "Ken",
"age": "40"
},
{
"Name": "Frank",
"age": "23"
}
]
}
Note: I dont have the list of all the branch files at any given point, so script is responsible only to update the that branch details.

If the file name is the name of the property you want to update, you could utilize input_filename to select the files. No testing needed, just pass in the files you want to update. Just be aware of the order you pass in the input files.
Merge the contents of the file as you see fit. To simply replace, just do a plain assignment.
$ jq 'reduce inputs as $i (.;
.[input_filename|rtrimstr(".json")] = $i.Employes
)' branches.json branch{1,2}.json
Your script would just need to be:
#!/bin/sh
#call file with branch name for example ./update.sh branch2
set -xe
branchName=$1
newJson=$(jq 'reduce inputs as $i (.; .[input_filename|rtrimstr(".json")] = $i.Employees)' branches.json "$branchName.json")

Related

Bash. How to access parameters while looping through json with jq?

i have a details.json file with a lot of entries and a shops.txt file like below. I like to have a little script which compares two values and just return the matching json entries.
[
{
"userName": "Anne",
"email": "anne#stack.com",
"company": {
"name": "Stack GmbH",
},
"details": {
"key": "EFHJKI-KJEFT-DHMNEB",
"prod": "Car",
},
"store": {
"id": "05611a7f-a679-12ad-a3u2-0745e3650a03",
"storeName": "shop-a57ca0a3-120c-1a73-153b-fa4231cab768",
}
},
{
"userName": "Tom",
"email": "tom#stack.com",
"company": {
"name": "Stack GmbH",
},
"details": {
"key": "DFSGSE-FGEAR-GWRTGW",
"prod": "Bike",
},
"store": null
},
]
This is the other file "shops.txt" (can be a lot more of shops inside)
shop-a57ca0a3-120c-1a73-153b-fa4231cab768
The script is looping through the shops, for every shop it loops through the json and should compare the currentShop with the store.shop from json and then echo the user and the shop.
But I can not access the specific parameters inside the json. How can I do this?
#!/bin/bash
shops="shops.txt"
while IFS= read -r line
do
currentShop="$line"
jq -c '.[.userName, .store.storeName]' details.json | while read i; do
if [[ $i.store.storeName == *$currentShop* ]]; then
echo $i.userName
echo $currentShop
fi
done
done < "$shops"
First of all, you might want to 'clean' your json, remove any trailing ,'s etc.
After looping through each line in the file we just need one select() to get the matching object.
The script could look something like:
#!/bin/bash
while read shop; do
echo "Check: $shop"
jq -r --arg storeName "$shop" '.[] | select(.store.storeName == "\($storeName)") | "\(.userName) - \(.store.storeName)"' details.json
done < "shops.txt"
Which will produce
Check: shop-a57ca0a3-120c-1a73-153b-fa4231cab768
Anne - shop-a57ca0a3-120c-1a73-153b-fa4231cab768
I guess this could be combined into a single jq call, but it seems like you want to loop over each entry found
You can test this jq selector on this online JqPlay Demo.
I was able to access the values with the following command:
echo $i | jq -r '.userName'

select specific values from json & convert into bash variables

I'm new to jq, I have the following JSON & I need to extract FOUR values i.e. SAUCE_KEY, sauceKey, SAUCE_VALUE, sauceValue etc. And I need to covert these bash variables as i.e.
SAUCE_KEY=sauceKey
SAUCE_VALUE=sauceValue
If I will echo it, it should print it's value ie. echo $SAUCE_KEY
I have used the code as:
jq -r '.env_vars[] | with_entries(select([.key] | inside([".env_vars[].name", ".env_vars[].value"])))' | jq -r "to_entries|map(\"\(.value)=\(.value|tostring)\")|.[]"
By doing so, i was able to get values as
name=SAUCE_KEY, value=sauceKey and so on.
{
"#type": "env_vars",
"env_vars": [
{
"#type": "env_var",
"#href": "/repo/xxxxx/env_var/xxxxx",
"#representation": "standard",
"#permissions": {
"read": true,
"write": true
},
"name": "SAUCE_KEY",
"value": "sauceKey"
},
{
"#type": "env_var",
"#href": "/repo/xxxxx/env_var/xxxxx",
"#representation": "standard",
"#permissions": {
"read": true,
"write": true
},
"name": "SAUCE_VALUE",
"value": "sauceValue"
}
]
}
If instead of trying to extract variable names from the JSON, you populate an associate array with the names as keys, it's pretty straightforward.
#!/usr/bin/env bash
declare -A sauces
while IFS=$'\001' read -r name value; do
sauces[$name]=$value
done < <(jq -r '.env_vars[] | "\(.name)\u0001\(.value)"' saucy.json)
for name in "${!sauces[#]}"; do
echo "$name is ${sauces[$name]}"
done
prints out
SAUCE_VALUE is sauceValue
SAUCE_KEY is sauceKey

increment a numerical value in a json with jq

I have a json which looks like this:-
{
"name": "abc",
"version": "20.02.01",
"tag": "24",
"abc_version": "20.02",
"registry": "registry.abc.com",
"vendor": "greenplanet",
"apps": [
{
"name": "abc-app",
"version": "20.02.01-16",
"volumes": [
"/dev/log:/dev/log"
],
"max_instances": "1"
},
{
"name": "xyz-app",
"version": "2.0.0-2",
"volumes": [
"/dev/log:/dev/log"
],
"max_instances": "1"
}
]
}
based on a condition I need to increment the abc-app's or xyz-app's version. At present its at "20.02.01-16" for abc-app and I need to change it to "20.02.01-17". Also I need to increment the tag of the parent app which is "abc" to "25"
I am able to increment the version with sed but that is not working for me:-
./jq -r ".apps" version.json | ./jq -r ".[] | .version" | grep -v '1.0.2' |sed -r 's/([0-9]+.[0-9]+.[0-9]+)(.*)([0-9]+)/echo "\1\2$((\3+1))"/e'
I need to increment all the above conditions in-place in json or maybe into a temporary file which I can move to original.
Thank you in advance.
First, let's define a helper function to perform the incrementation:
# return a string
def inc:
(capture("(?<pre>.*)-(?<post>[0-9]+$)") | "\(.pre)-\( (.post|tonumber) + 1 )")
// "\(tonumber+1)" ;
The remainder of the solution can now be written in two lines, one for each value to be incremented:
.tag |= inc
| .apps |= map(if .name == "abc-app" then .version |= inc else . end)
In-place editing
You could use sponge, or a temporary file, or ... (see e.g.
"How can "in-place" editing of a JSON file be accomplished?"
on jq's FAQ).

How to merge multiple Json files in folder in certain order

I have several JSON files in a folder and I want to combine them in a single one following certain order given in order.json
I´ve tested the code below that merges all the files but in alphabetical order based on file name.
jq -s . *.json > Merged.json
This is the file Shoes.json
{ "document": { "product": "Shoes", "info": [ { "day": "1", "month": "1", "entry": "Some text about Shoes for day 1 and month 1", "code": "AJKD" }, { "day": "2", "month": "1", "entry": "Some text about Shoes for day 2 and month 1", "code": "KKGIR" } ] } }
This is the file Watches.json
{ "document": { "product": "Watches", "info": [ { "day": "2", "month": "3", "entry": "Some text about Watches for day 2 and month 3", "code": "PEWQ" } ] } }
This is the file Accesories.json
{ "document": { "product": "Accesories", "info": [ { "day": "7", "month": "2", "entry": "Some text about Accesories for day 7 and month 2", "code": "UYAAC" } ] } }
This is the file that gives the order I want to get in output order.json
{
"order":{
"product 1":"Watches",
"product 2":"Accesories",
"product 3":"Shoes"
}
}
And the output file I´d like to get would be like this Merged.json:
{
"document":[
{
"product":"Watches",
"info":[
{
"day":"2",
"month":"3",
"entry":"Some text about Watches for day 2 and month 3",
"code":"PEWQ"
}
]
},
{
"product":"Accesories",
"info":[
{
"day":"7",
"month":"2",
"entry":"Some text about Accesories for day 7 and month 2",
"code":"UYAAC"
}
]
},
{
"product":"Shoes",
"info":[
{
"day":"1",
"month":"1",
"entry":"Some text about Shoes for day 1 and month 1",
"code":"AJKD"
},
{
"day":"2",
"month":"1",
"entry":"Some text about Shoes for day 2 and month 1",
"code":"KKGIR"
}
]
}
]
}
Maybe someone could help me with this case.
Any help would be very appreciated.
Here's a solution that uses order.json to determine which files are to be read by jq:
jq -n '{documents: [inputs.document]}' $(jq -r '.order[] + ".json"' order.json)
The approach exemplified immediately above has many advantages, but the above line makes various assumptions that might not be warranted. For example, it assumes there are no spaces in any of the file names.
Robust handling of file names
The following assumes bash-ful functionality:
mapfile -t args < <(jq -r '.order[] + ".json"' order.json)
jq -n '{documents: [inputs.document]}' "${args[#]}"
If your bash does not have mapfile, you could set the bash variable as follows:
args=()
while read -r f; do
args+=("$f")
done < <(jq -r '.order[] + ".json"' order.json)
The following solution has the advantage of not requiring any shell-specific functionality and only requiring a single invocation of jq. It assumes you want to be able to handle arbitrarily many files, as determined by order.json. Amongst other assumptions is that in the pwd, we can use the pattern [A-Z]*.json to select the relevant "documents".
jq -n --argfile order order.json '
INDEX(inputs.document; .product) as $dict
| reduce $order.order[] as $product ([]; . + [$dict[$product]])
| {document: .}
' [A-Z]*.json
def INDEX
If your jq does not have INDEX/2, then it might be a good time to upgrade; alternatively, you could simply add (prepend) its def:
def INDEX(stream; idx_expr):
reduce stream as $row ({}; .[$row|idx_expr|tostring] = $row);

json array into json stream with jq

This task is similar to this one but in my case I would like to go other way around.
So say we have input:
[
{
"name": "John",
"email": "john#company.com"
},
{
"name": "Brad",
"email": "brad#company.com"
}
]
and desired output is:
{
"name": "John",
"email": "john#company.com"
}
{
"name": "Brad",
"email": "brad#company.com"
}
I tried to write a bash function which will do it in loop:
#!/bin/bash
json=`cat $1`
length=`echo $json | jq '. | length'`
for (( i=0; i<$length ; i++ ))
do
echo $json | jq ".[$i]"
done
but it is obviously extremly slow...
Is there any way how to use jq better for this?
You can use this :
jq '.[]' file
If you use the .[index] syntax, but omit the index entirely, it will return all of the elements of an array.
Test:
$ jq '.[]' file
{
"email": "john#company.com",
"name": "John"
}
{
"email": "brad#company.com",
"name": "Brad"
}
you can apply ".[]" filter.
This tutorial is very informative
https://stedolan.github.io/jq/tutorial/