I want to get the IPs that has 'server.sh' value. My current script gets all the IPs
test.json
{
"nodes": {
"test1.local": {
":ip": "192.168.56.30",
":server": "server.sh",
":client": "client.sh"
},
"test2.local": {
":ip": "192.168.56.31",
":server": "server.sh",
":client": "client.sh"
},
"test3.local": {
":ip": "192.168.56.32",
":client": "client.sh"
}
}
}
test.sh
ips=`jq -c '.nodes | to_entries | map(.value.":ip")| map_values(.+":4648")' test.json`
echo $ips
["192.168.56.30:4648","192.168.56.31:4648","192.168.56.32:4648"]
Is it ok for your task?
jq '.nodes|.[]|select(.":server"=="server.sh")|.":ip"+":4648"' test.json
"192.168.56.30:4648"
"192.168.56.31:4648"
Related
I'm hoping someone could help, I'm trying to merge two json files. Here is my bash script:
script_directory="/home/joey/scripts/scripts/delete"
region_file="US.en.json"
cnmts_file="cnmts.json"
wget https://github.com/blawar/titledb/raw/master/$region_file -O $script_directory/$region_file
wget https://github.com/blawar/titledb/raw/master/$cnmts_file -O $script_directory/$cnmts_file
#This is here just to simplify the json files
cat $script_directory/$region_file | jq '.[] | {id: .id}' > $script_directory/region_file_id.txt
cat $script_directory/$cnmts_file | jq '.[] | .[] | {titleId: .titleId, otherApplicationId: .otherApplicationId}' > $script_directory/cnmts_titleId_otherApplicationId.txt
Essentially, I'm given two files:
region_file_id.txt:
{
"id": "01007EF00011E000"
}
{
"id": "0100225000FEE000"
}
{
"id": "0100BCE000598000"
}
{
"id": "0100B42001DB4000"
}
{
"id": "01008A9001DC2000"
}
and cnmts_titleId_otherApplicationId.txt:
{
"titleId": "0100000000010000",
"otherApplicationId": "0100000000010800"
}
{
"titleId": "0100000000010800",
"otherApplicationId": "0100000000010000"
}
{
"titleId": "010000000e5ee000",
"otherApplicationId": "010000000e5ee800"
}
{
"titleId": "010000000eef0000",
"otherApplicationId": "010000000eef0800"
}
{
"titleId": "010000000eef0800",
"otherApplicationId": "010000000eef0000"
}
{
"titleId": "0100000011d90000",
"otherApplicationId": "0100000011d90800"
}
{
"titleId": "0100000011d90800",
"otherApplicationId": "0100000011d90000"
}
{
"titleId": "0100000011d90800",
"otherApplicationId": "0100000011d90000"
}
Please note, this is only a snippet of the files, feel free to run the bash script to get a more accurate file.
All the "id" in the 'region_file_id' equal to a "titleId" somewhere in 'cnmts_titleId_otherApplicationId' (the reverse is not true though as it included id from different regions). I'm trying to grab the "otherApplicationId" values for each "id" in 'region_file_id' by cross referencing them and creating a json like: (repeated for every 'id' in region_file_id)
{
"id": "111000"
"titleId": "111000" (this one is optional as it is a duplicate from 'id')
"otherApplicationId": 111800"
}
I've tried searching and tried different snippets:
jq -s '.[0] * .[1]' $script_directory/region_file_id.txt cnmts_titleId_otherApplicationId.txt (only returned 1 object for some reason)
jq -s '{ .[0] as $u | .[1] | select(.id == $u.titleId) |= $u }' $script_directory/region_file_id.txt cnmts_titleId_otherApplicationId.txt
Update:
As peak pointed out:
jq -n --slurpfile ids region_file_id.txt '
INDEX(inputs; .titleId | ascii_upcase) as $dict
| $ids[].id as $id
| {$id} + $dict[$id]
' cnmts_titleId_otherApplicationId.txt > merged.txt
This seems to work until I hit "null" values where my file doesn't include the correct id, which is another problem all together!
All the "id" in the 'region_file_id' equal to a "titleId" somewhere in 'cnmts_titleId_otherApplicationId'
If that really is the case, then you could proceed as follows:
< cnmts_titleId_otherApplicationId.txt jq -n --slurpfile ids region_file_id.txt '
INDEX(inputs; .titleId) as $dict
| $ids[].id as $id
| {$id} + $dict[$id]
'
I have a following Json input in a text file json.txt:
{
"files":[
{
"id":49894894,
"list":[
{
"name":"one",
"animal_potato_carrot":{
"options":[
{
"id":4989,
"url":"https://example.com/text.txt"
},
{
"id":3994,
"url":"https://example.com/randomfile.json"
}
]
}
},
{
"name":"two",
"cat_dog_rabbit":[
{
"id":4989,
"url":"https://example.com/text2.txt"
},
{
"id":3994,
"url":"https://example.com/randomfile.json"
}
]
},
{
"name":"three",
"animal_potato_carrot":{
"options":[
{
"id":4989,
"url":"https://example.com/text3.txt"
},
{
"id":3994,
"url":"https://example.com/randomfile.json"
}
]
}
}
]
}
]
}
I want to get only the first url in the list of options for each animal_potato_carrot or cat_dog_rabbit nested tag only (note they have different structures)
So my output will be first three urls in those blocks:
["https://example.com/text.txt", "https://example.com/text2.txt, "https://example.com/text3.txt"]
I tried jq json.txt -c '.. |."animal_potato_carrot"? | select(. != null)' but that returns all the things inside the body, not just the FIRST url.
Edit:
these two commands return the urls for animal_potato_carrot and cat_dog_rabbit separately but is there a way to combine these commands?
jq -c '[..|.animal_potato_carrot?|select(. != null)|.options[0].url]' json.txt
jq -c '[..|.cat_dog_rabbit?|select(. != null)|.[0].url]' json.txt
If you want to concatenate two arrays you can use the + operator:
jq -c '[..|.animal_potato_carrot?|select(. != null)|.options[0].url] + [..|.cat_dog_rabbit?|select(. != null)|.[0].url]' json.txt
Please notice that the order of items in the result is not exactly as you requested, because first all animal_potato_carrot-urls are determined and then all cat_dog_rabbit-urls.
Combining two filters with , may come closest to your needs:
jq -c '[..|(.animal_potato_carrot?.options),(.cat_dog_rabbit?)|.[0].url|select(. != null)]' json.txt
Let's say that with a bash script I want to create a file, so to the command to create the file will be something like this:
myscript hostgroup_one 2
hostgroup_one and the number 2 are parameters.
How can I insert the parameters in the lines below and output all the lines as a file?
{
"bool": {
"should": [
{
"match_phrase": {
"hostgroup.keyword": "$1"
}
}
],
"minimum_should_match": $2
}
}
I'd use jq to build the JSON:
jq -n \
--arg hostgroup "$1" \
--argjson minimum "$2" \
'{bool: {should: [{match_phrase: {"hostgroup.keyword": $hostgroup}}], minimum_should_match: $minimum}}'
will produce your desired output.
While jq is a great tool for manipulating json, as glenn jackman recommends, if you don't have it and your sysadmin won't install it (?!?)...
You can use a "here document"
Your myscript could look something like this:
#!/bin/bash
echo "dollar-1 is '${1}' dollar-2 is '${2}' dollar-3 is '${3}'"
cat <<EOF >"${1}"
{
"bool": {
"should": [
{
"match_phrase": {
"hostgroup.keyword": "${2}"
}
}
],
"minimum_should_match": ${3}
}
}
EOF
echo "done"
I've made the output filename the first parameter, and then your two parameters, so it would be run like:
myscript output.json hostgroup_one 2
You don't need to do that, you could use 2 params and redirect output:
myscript hostgroup_one 2 > output.json
(note you don't have to use EOF as your here-document delimiter, I just like it)
Of course you don't need the echo statements, and you should have error checking (does ${#} equal 3 parameters?)
Here is my sample data, it's a list of objects in a storage bucket on Oracle cloud:
{
"objects": [
{
"name": "rhel/"
},
{
"name": "rhel/app-3.9.6.629089.txt"
},
{
"name": "rhel/app-3.11.4.629600.txt"
}
]
}
The part of the value before the '/' is a folder name, after is a filename. The last number in the filename is a build number. The desired output is the name of the object with the highest build number in the rhel folder:
$ jq -r 'some_program' file.json
rhel/app-3.11.4.629600.txt
I can somewhat process the data to exclude the bare "rhel/" folder as follows:
$ jq -r '.objects[] | select(.name|test("rhel/."))' file.json
{
"name": "rhel/app-3.9.6.629089.txt"
}
{
"name": "rhel/app-3.11.4.629600.txt"
}
When I try to split this on the period jq throws an error:
$ jq -r '.objects[] | select(.name|test("rhel/.")) | split(".")' file.json
jq: error (at file.json:1): split input and separator must be strings
I was expecting to use 'map(tonumber)[-2]' on the result of the split and wrap the entirety in 'max_by()'.
How can get closer to the desired output with jq?
[.objects[]
| select(.name|test("rhel/."))]
| max_by(.name|split(".")[-2]|tonumber)
produces:
{
"name": "rhel/app-3.11.4.629600.txt"
}
If you only want the names you could begin by extracting them:
[.objects[].name|select(test("rhel/."))]
| max_by(split(".")[-2]|tonumber)
I have below mentioned Json file. I wanted to do the below checks.
Get 1st 5 objects from the whole list and save them in a separate file (i.e FirstTopObject.json)
Get another set-off 5 objects and store them into another file (i.e SecondTopObject.json)
Get the last 5 objects and store them into another file (i.e ThirdTopObject.json)
Basically, wanted to split the Objects based on the Numbers and save them into a separate file.
Is there any solution is available to achieve through the “jq” function/method?
Input File:
{
"storeId": "0001"
}
{
"storeId": "0002"
}
{
"storeId": "0003"
}
{
"storeId": "0004"
}
{
"storeId": "0005"
}
{
"storeId": "0006"
}
{
"storeId": "0007"
}
{
"storeId": "0008"
}
{
"storeId": "0009"
}
{
"storeId": "00010"
}
{
"storeId": "00011"
}
{
"storeId": "00012"
}
{
"storeId": "00013"
}
{
"storeId": "00014"
}
{
"storeId": "00015"
}
enter code here
enter code here
Expecting output:
FirstTopObject.json should have the below set.
{
"storeId": "0001"
}
{
"storeId": "0002"
}
{
"storeId": "0003"
}
{
"storeId": "0004"
}
{
"storeId": "0005"
}
SecondTopObject.json - shold contain below setoff objects.
{
"storeId": "0006"
}
{
"storeId": "0007"
}
{
"storeId": "0008"
}
{
"storeId": "0009"
}
{
"storeId": "00010"
}
Like wise for other set.
It Would be more helpful if some help me.
Thanks in advance!
You could use JQ to process the input file - reformat it using -c (compact) and then use standard unix tools to split the files, i.e.
cat input | jq --slurp -c .[] | head -5 | jq . > FirstTopObject.json
cat input | jq --slurp -c .[] | sed '6,10!d' | jq . > SecondTopObject.json
cat input | jq --slurp -c .[] | tail -5 | jq . > ThirdTopObject.json
You can use a combination of the jq functions to_entries and group_by for this, along with a little bash.
This snippet will create 25 strings ("line 0", "line 1", etc.), group them by 5s, and write them into files 0.json, 1.json, etc. Everything before to_entries can be replaced with any list. In your case, you can use the slurp flag -s to get all your JSON objects in your input file into a list.
FILE_NUM=0
jq -nc '
# create input
["line " + (range(25) | tostring)] |
# process input
to_entries | group_by(.key / 5 | floor)[] | map(.value)
' | while read LINE; do echo "$LINE" > "/tmp/$((FILE_NUM++)).json"; done
There is no need to slurp the input file! Even if the output must be pretty-printed, there is no need for more than four invocations of jq altogether.
Handling a small input file
If the input is not so big, you can simply run
jq -c . input
directing the output to a temporary file, and then split that file into three using whichever standard command-line tools you find most convenient (a single invocation of awk might be worth considering ...).
Handling a very large input file
If the input file is very large, then it would make sense to use jq just to copy the (15) items of interest into a temporary file, and then process that file:
Step 1
Invoke the following program with jq -cn:
def echo($n1; $n2; $last):
foreach (inputs,null) as $in ({ix:-1, first:[], second:[], last:[]};
if $in then
.ix += 1
| if .ix < $n1
then .first += [$in]
elif .ix < $n1+n2 then .second += [$in]
else .last += [$in]
| .last = (.last[ - $last: ])
end
else . end;
if $in == null then del(.ix) else empty end
)
| .[];
echo(5;5;5)
(This program is somewhat complex because it makes no assumptions about the relative sizes of the three blocks.)
Step 2
Assuming the output from Step 1 is in input.tmp, then run:
sed -n 1p input.tmp | jq .[] > FirstTopObject.json
sed -n 2p input.tmp | jq .[] > SecondTopObject.json
sed -n 3p input.tmp | jq .[] > ThirdTopObject.json