Actually my JSON looks like below:
"checksum": "sdkjjfj-shbdjfj23"
I wanted to replace the checksum value with another value. As per above I used :
sed -i 's/("checksum": ")[^"]*(")/\1$checksumVal\2/g' new.json
The new.json is updated as below, but i wanted the value of that variable.
"checksum": "$checksumVal",
Expected Result:
"checksum": "newval"
Any help would be appreciated.
First of all use one echo statement and check whether $checksumVal has some value or not and also try like below it work for me. How will it work
sed "/checksum will search for pattern "checksum once it found the pattern it will search for pattern : "*" (with wild card) in this example it is : "sdkjjfj-shbdjfj23" and this pattern will be replace with : "$checksumVal"
sed "/checksum/s/: ".*"/: \"$checksumVal\"/" new.json
What's wrong with jq?:
$ cat foo.json # added the required {}
{"checksum": "sdkjjfj-shbdjfj23"}
$ echo $checksum
foo
$ jq '.checksum = "'"$checksum"'"' foo.json
{
"checksum": "foo"
}
Related
I have a set of JSON files in a local folder. What I want to do is change a particular string value in it, permanently. That means, deleting or modifying the old entry, writing a new one, and saving it.
Below is the format of the file:
{
"name": "ABC #1",
"description": "This is the description",
"image": "ipfs://NewUriToReplace/1.png",
"dna": "a56c520f57ba2a861de8c78099b4691f9dad6e87",
"edition": 1,
"date": 1641634646966,
"creator": "Team Dreamlabs",
"attributes": [
{
I want to change ABA #1 to ABC #9501 in this file, ABC #2 to ABC #9502 in the text file, and so on. How do I do that on MAC in one go?
As I understand from the example, you are adding a value of 9500 to your integers after the symbol #.
Because this kind of a replacement is a kind of string operation, a cycle with command sed might be used:
for f in *.json; do sed -i.bak 's/\("name": "ABC #\)\([0-9]\)",/\1950\2",/' $f; done
it just replaces a single digit to the new composition... Despite it responses to the example, obviously, it would not work for more than number #9.
Then we need to use a bash function:
function add_number() { old_number=$(cat $1 | sed -n 's/[ ]*"name": "ABC #\([0-9]*\)",/\1/p'); new_number=$(($old_number+9500)); sed -i.bak "s/\(\"name\": \"ABC #\)\([0-9]*\)\",/\1${new_number}\",/" $1; }; for f in *.json; do add_number $f ; done
The function add_number extracts the integer value, then adds a desired number to it and then replaces content of the file.
For both extraction and replacing the sed is used again.
At extraction flag -n allows to limit the amount of lines at sed output and mode p prints the result of replacement. Also, we do not want spaces symbols to pass into this assignment.
At replacement double quotes used in order to enable the bash to use the variable value inside of sed. Also, the real quotes are masked.
Regarding addition from the comment below, in order to make replacement in another line with tag edition (and using the same number), just a new replacement sed operation should be added with amended regular expression to fit this line.
Finally, the overall code in a better look:
function add_number() {
old_number=$(cat $1 | sed -n 's/[ ]*"name": "ABC #\([0-9]*\)",/\1/p')
new_number=$(($old_number+9500))
sed -i.bak "s/\(\"name\": \"ABC #\)[0-9]*\",/\1${new_number}\",/" $1
sed -i.bak "s/\(\"edition\": \)[0-9]*,/\1${new_number},/" $1
}
for f in *.json
do add_number $f
done
Those previous answers helped me to write this code:
using variables inside of sed
assigning the variable
If you are going to manipulate your JSON files on more than just this one occasion, then you might want to consider using tools that are designed to accomplish such tasks with ease.
One popular choice could be jq which is a "lightweight and flexible command-line JSON processor" that "has zero runtime dependencies" and is also available for OS X. By using jq within your shell, the following would be one way to accomplish what you have asked for.
Adding the numeric value 9500 to the number sitting in the field called edition:
jq '.edition += 9500' file.json
Interpreting a part of a string as number, adding again 9500 to it, and recomposing the string:
jq '.name |= ((./"#" | .[1] |= "\(tonumber + 9500)") | join("#"))' file.json
On the whole, iterating over your files, making both changes at once, writing to a temporary file and replacing the original on success, while having the value to be added as external variable:
v=9500
for f in *.json; do jq --argjson v $v '
.edition += $v | .name |= ((./"#" | .[1] |= "\(tonumber + $v)") | join("#"))
' "$f" > "$f.new" && mv "$f.new" "$f"
done
Here is an online "playground for jq", set up to simulate the application of my code from above to three imaginary files of yours. Feel free to edit the jq filter and/or the input JSON in order to see what could be possible using jq.
I have one JSON file which contains three tags and I am trying to update the tag values using sed command.
demo.json
{
"useCaseName" : "demo",
"useCaseNumber" : 12,
"Case" : 221
}
I am trying to invoke the below function and pass the tag and value as an argument to update the JSON file
change(){
sed -i '/'${1}'/c\ \"'${1}'\" : "'${2}'",' demo.json
}
change "useCaseName" "hello"
change "useCaseNumber" 2
Now the problem is the above command is adding the double quotes to the useCaseNumber like below.
{
"useCaseName" : "hello",
"useCaseNumber" : "2",
"Case" : 221
}
Is there any way to update both string value and Integer value using the above function?
With GNU sed and bash if jq and Perl are not available:
change() { sed -Ei 's/(\"{0,1})'"$1"'(\"{0,1}) : (\"{0,1})[^,"]*(\"{0,1})/\1'"$1"'\2 : \3'"$2"'\4/' demo.json; }
You can just check if the second variable is number or not using regex. If it is number, then don't add the quotes and vice versa for text. The modified script is
change(){
re='^[0-9]+$'
if ! [[ $2 =~ $re ]] ; then
#if second argument is text
sed -i '/'${1}'/c\ \"'${1}'\" : "'${2}'",' demo.json
else
#if second argument is number
sed -i '/'${1}'/c\ \"'${1}'\" : '${2}',' demo.json
fi
}
change "useCaseName" "hello"
change "useCaseNumber" 2
Let's say 123.json with below content:
{
"LINE" : {
"A_serial" : "1234",
"B_serial" : "2345",
"C_serial" : "3456",
"X_serial" : "76"
}
}
If I want to use a shell script to change the parameter of X_serial by the original number +1 which is 77 in this example.
I have tried the below script to take out the parameter of X_serial:
grep "X_serial" 123.json | awk {print"$3"}
which outputs 76. But then I don't know how to make it into 77 and then put it back to the parameter of X_serial.
It's not a good idea to use line-oriented tools for parsing/manipulating JSON data. Use jq instead, for example:
$ jq '.LINE.X_serial |= "\(tonumber + 1)"' 123.json
{
"LINE": {
"A_serial": "1234",
"B_serial": "2345",
"C_serial": "3456",
"X_serial": "77"
}
}
This simply updates .LINE.X_serial by converting its value to a number, increasing the result by one, and converting it back to a string.
You need to install powerful JSON querying processor like jq processor. you can can easily install from here
once you install jq processor, try following command to extract the variable from JSON key value
value=($(jq -r '.X_serial' yourJsonFile.json))
you can modify the $value as you preferred operations
With pure Javascript: nodejs and bash :
node <<EOF
var o=$(</tmp/file);
o["LINE"]["X_serial"] = parseInt(o["LINE"]["X_serial"]) + 1;
console.log(o);
EOF
Output
{ LINE:
{ A_serial: '1234',
B_serial: '2345',
C_serial: '3456',
X_serial: 78 }
}
sed or perl, depending on whether you just need string substitution or something more sophisticated, like arithmetic.
Since you tried grep and awk, let's start with sed:
In all lines that contain TEXT, replace foo with bar
sed -n '/TEXT/ s/foo/bar/ p'
So in your case, something like:
sed -n '/X_serial/ s/\"76\"/\"77\"/ p'
or
$ cat 123.json | sed '/X_serial/ s/\"76\"/\"77\"/' > new.json
This performs a literal substiution: "76" -> "77"
If you would like to perform arithmetic, like "+1" or "+10" then use perl not sed:
$ cat 123.json | perl -pe 's/\d+/$&+10/e if /X_serial/'
{
"LINE" : {
"A_serial" : "1234",
"B_serial" : "2345",
"C_serial" : "3456",
"X_serial" : "86"
}
}
This operates on all lines containing X_serial (whether under "LINE" or under something else), as it is not a json parser.
I have set of json files where after last key value pair i have comma which needs to be replaced.
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime":"0.033",
}
It should look like:
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime": 0.033
}
How can i achive this using sed.
Edit: Changed output - There should not be any "" around 0.033.
sed -i \'7i'\\t'\"EchoTime\": \0.033\' sub-285345_task-WM_acq-RL_bold.json
is not helping me. I have tried few other options but no success..
I trioed using simplejson and json package in python too. But given that the files are incorrct json, json.loads(file) throws errors..
I would prefer sed over python for now..
sed -Ei.bak 's/^([[:blank:]]*"EchoTime[^"]*":)"([^"]*)",$/\1\2/' file.json
will do it
Sample Output
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime":0.033
}
Notes
E to enable extended regular expressions.
i to enable inplace editing, a backup file with .bak extension is created.
Please try the following command.
sed -i 's#\(.*\)EchoTime":"\(.*\)",$#\1EchoTime":\2#' sub-285345_task-WM_acq-RL_bold.json
In case you are not limited to sed and open for awk , then following can be used :
awk ' BEGIN{FS=OFS=":"}/EchoTime/ {gsub(/\"|\,/,"",$2)}1' file.json
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime":0.033
}
Explanation:
FS=OFS=":" : This will set input and o/p field separator as ":"
/EchoTime/ : Search for the line containing EchoTime.
/EchoTime/{gsub(/\"|\,/,"",$2)}: Once echo time is found use global sub to replace , double quotes and comma in second field of that line.
1 : awk's default action is to print.
For making changes in original file:
awk ' BEGIN{FS=OFS=":"}/EchoTime/ {gsub(/\"|\,/,"",$2)}1' file.json >json.tmp && mv json.tmp file.json
I have a json file names test.json with the below content.
{
"run_list": ["recipe[cookbook-ics-op::setup_server]"],
"props": {
"install_home": "/test/inst1",
"tmp_dir": "/test/inst1/tmp",
"user": "tuser
}
}
I want to read this file into a variable in shell script & then extract the values of install_home,user & tmp_dir using expr. Can someone help, please?
props=cat test.json
works to get the json file into a variable. Now how can I extract the values using expr. Any help would be greatly appreciated.
Install jq
yum -y install epel-release
yum -y install jq
Get the values in the following way
install_home=$(cat test.json | jq -r '.props.install_home')
tmp_dir=$(cat test.json | jq -r '.props.tmp_dir')
user=$(cat test.json | jq -r '.props.user')
For a pure bash solution I suggest this:
github.com/dominictarr/JSON.sh
It could be used like this:
./json.sh -l -p < example.json
print output like:
["name"] "JSON.sh"
["version"] "0.2.1"
["description"] "JSON parser written in bash"
["homepage"] "http://github.com/dominictarr/JSON.sh"
["repository","type"] "git"
["repository","url"] "https://github.com/dominictarr/JSON.sh.git"
["bin","JSON.sh"] "./JSON.sh"
["author"] "Dominic Tarr <dominic.tarr#gmail.com> (http://bit.ly/dominictarr)"
["scripts","test"] "./all-tests.sh"
From here is pretty trivial achive what you are looking for
jq is a dedicated parser for JSON files. Install jq.
values in the json can be retrieved as:
jq .<top-level-attr>.<next-level-attr> <json-file-path>
if JSON contains an array
jq .<top-level-attr>.<next-level-array>[].<elem-in-array> <json-file-path>
if you want a value in a shell variable
id = $(jq -r .<top-level-attr>.<next-level-array>[].<next-level-attr> <json-file-path>)
echo id
use -r if you need unquoted value
For simple JSON, it may be treated as a plain text file.
In that case, we can use simple text pattern matching to extract the information we need.
If you observe the following lines:
"install_home": "/test/inst1",
"tmp_dir": "/test/inst1/tmp",
"user": "user"
There exists a pattern on each line that can be described as key and value:
"key" : "value"
We can use perl with regular expressions to exact the value for any given key:
"key" hardcoded for each case "install_home", "tmp_dir" and "user"
"value" as (.*) regular expression
Then we use the $1 matching group to retrieve the value.
i=$(perl -ne 'if (/"install_home": "(.*)"/) { print $1 . "\n" }' test.json)
t=$(perl -ne 'if (/"tmp_dir": "(.*)"/) { print $1 . "\n" }' test.json)
u=$(perl -ne 'if (/"user": "(.*)"/) { print $1 . "\n" }' test.json)
cat <<EOF
install_home: $i
tmp_dir : $t
user : $u
EOF
Which outputs:
install_home: /test/inst1
tmp_dir : /test/inst1/tmp
user : tuser