shell parsing json and loop output the combine variable - json

Question

Jusk like my previous thread, I know how to parse simple json with spaces.
Now I have another question is that if I have multiple module structures, their keys are the same, but the values are different, I want the output value to be a combination of the values in each module, but actually the value in the last module will overwrite the previous one.
My test sample JSON like:
{
"WorkspaceName":"aaa bbb ccc ddd eee",
"ReportFileName":"xxx yyy zzz",
"StageName":"sit uat prod"
},
{
"WorkspaceName":"1111 2222 3333 4444 5555",
"ReportFileName":"6666 7777 8888",
"StageName":"sit1 uat1 prod1"
}
And my tried shell script(s) mian.sh is as follows:
InitialFile=$WORKSPACE/deployment/configuration/Initial.json
eval $(sed -n -e 's/^.*"\(.*\)":\(".*"\).*$/\1=\2/p' $InitialFile)
ConfigFile="$WorkspaceName"_"$ReportFileName"
echo The Config File is_$ConfigFile
The result is always The Config File is_1111 2222 3333 4444 5555_6666 7777 8888, I want get both values: aaa bbb ccc ddd eee_xxx yyy zzz and 1111 2222 3333 4444 5555_6666 7777 8888.
How do I achieve this?
A little background to understand why I'm doing this and some of my limitations：
I am executing my pipeline on jenkins and it will execute my mian.sh. So the entry is mian.sh. In addition, the jenkins server is maintained by a separate team, and we cannot directly access the server, so we cannot run shell code directly on the server.
Another, I need to combine variables in order to use this variable to match the name of the corresponding configuration file. Different results need to match different files for subsequent testing.

Answer 1

Important points for this answer:
Since OP can't install and use jq so going with an awk approach here.
I have provided 3 solutions here, 1st: is GNU awk approach and 2nd is NON-GNU awk approach and 3rd one is running NON-GNU awk code from a shell script.
1st 2 codes are stand along awk codes to run on terminal OR in an awk script
Then as per OP's request since their code is running in Jenkins I have posted a shell script which accepts an argument which is a Input_file name to be passed to it.
To save output into a shell variable could be done in 3rd code of this answer by changing 1st line to StageName=$(awk -v RS= ' and changing last line of 3rd code to ' "$1").
1st solution: With your shown samples please try following GNU awk code. Using match function of GNU awk where I am using regex [[:space:]]+"WorkspaceName":"([^"]*)",\n[[:space:]]+"ReportFileName":"([^"]*) to get the required values and creating 2 capturing groups out of it which further stores values into an array named arr to be get values later on as pre reuiqrement.
awk -v RS= '
{
while(match($0,/[[:space:]]+"WorkspaceName":"([^"]*)",\n[[:space:]]+"ReportFileName":"([^"]*)",/,arr)){
print arr[1]"_"arr[2]
$0=substr($0,RSTART+RLENGTH)
}
}
' Input_file
2nd solution: With your shown samples please try following code, should work in any POSIX awk. This solution also uses match function but it doesn't create array and doesn't have any capturing groups in it, since capturing group capability is part of GNU awk. So using split function here to split the matched values and get only required part out of it.
awk -v RS= '
{
while(match($0,/[[:space:]]+"WorkspaceName":"[^"]*",\n[[:space:]]+"ReportFileName":"[^"]*",/)){
val=substr($0,RSTART,RLENGTH)
split(val,arr,"\"WorkspaceName\":\"|\"ReportFileName\":\"|,\n")
sub(/"$/,"",arr[2])
sub(/",$/,"",arr[4])
print arr[2]"_"arr[4]
$0=substr($0,RSTART+RLENGTH)
}
}
' Input_file
To run code from shell script try like:
#!/bin/bash
awk -v RS= '
{
while(match($0,/[[:space:]]+"WorkspaceName":"[^"]*",\n[[:space:]]+"ReportFileName":"[^"]*",/)){
val=substr($0,RSTART,RLENGTH)
split(val,arr,"\"WorkspaceName\":\"|\"ReportFileName\":\"|,\n")
sub(/"$/,"",arr[2])
sub(/",$/,"",arr[4])
print arr[2]"_"arr[4]
$0=substr($0,RSTART+RLENGTH)
}
}
' "$1"