I have set of json files where after last key value pair i have comma which needs to be replaced.
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime":"0.033",
}
It should look like:
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime": 0.033
}
How can i achive this using sed.
Edit: Changed output - There should not be any "" around 0.033.
sed -i \'7i'\\t'\"EchoTime\": \0.033\' sub-285345_task-WM_acq-RL_bold.json
is not helping me. I have tried few other options but no success..
I trioed using simplejson and json package in python too. But given that the files are incorrct json, json.loads(file) throws errors..
I would prefer sed over python for now..
sed -Ei.bak 's/^([[:blank:]]*"EchoTime[^"]*":)"([^"]*)",$/\1\2/' file.json
will do it
Sample Output
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime":0.033
}
Notes
E to enable extended regular expressions.
i to enable inplace editing, a backup file with .bak extension is created.
Please try the following command.
sed -i 's#\(.*\)EchoTime":"\(.*\)",$#\1EchoTime":\2#' sub-285345_task-WM_acq-RL_bold.json
In case you are not limited to sed and open for awk , then following can be used :
awk ' BEGIN{FS=OFS=":"}/EchoTime/ {gsub(/\"|\,/,"",$2)}1' file.json
{
"RepetitionTime": 0.72,
"TaskName":"WM",
"Manufacturer": "Siemens",
"ManufacturerModelName": "Skyra",
"MagneticFieldStrength": 3.0,
"EchoTime":0.033
}
Explanation:
FS=OFS=":" : This will set input and o/p field separator as ":"
/EchoTime/ : Search for the line containing EchoTime.
/EchoTime/{gsub(/\"|\,/,"",$2)}: Once echo time is found use global sub to replace , double quotes and comma in second field of that line.
1 : awk's default action is to print.
For making changes in original file:
awk ' BEGIN{FS=OFS=":"}/EchoTime/ {gsub(/\"|\,/,"",$2)}1' file.json >json.tmp && mv json.tmp file.json
Related
I have a set of JSON files in a local folder. What I want to do is change a particular string value in it, permanently. That means, deleting or modifying the old entry, writing a new one, and saving it.
Below is the format of the file:
{
"name": "ABC #1",
"description": "This is the description",
"image": "ipfs://NewUriToReplace/1.png",
"dna": "a56c520f57ba2a861de8c78099b4691f9dad6e87",
"edition": 1,
"date": 1641634646966,
"creator": "Team Dreamlabs",
"attributes": [
{
I want to change ABA #1 to ABC #9501 in this file, ABC #2 to ABC #9502 in the text file, and so on. How do I do that on MAC in one go?
As I understand from the example, you are adding a value of 9500 to your integers after the symbol #.
Because this kind of a replacement is a kind of string operation, a cycle with command sed might be used:
for f in *.json; do sed -i.bak 's/\("name": "ABC #\)\([0-9]\)",/\1950\2",/' $f; done
it just replaces a single digit to the new composition... Despite it responses to the example, obviously, it would not work for more than number #9.
Then we need to use a bash function:
function add_number() { old_number=$(cat $1 | sed -n 's/[ ]*"name": "ABC #\([0-9]*\)",/\1/p'); new_number=$(($old_number+9500)); sed -i.bak "s/\(\"name\": \"ABC #\)\([0-9]*\)\",/\1${new_number}\",/" $1; }; for f in *.json; do add_number $f ; done
The function add_number extracts the integer value, then adds a desired number to it and then replaces content of the file.
For both extraction and replacing the sed is used again.
At extraction flag -n allows to limit the amount of lines at sed output and mode p prints the result of replacement. Also, we do not want spaces symbols to pass into this assignment.
At replacement double quotes used in order to enable the bash to use the variable value inside of sed. Also, the real quotes are masked.
Regarding addition from the comment below, in order to make replacement in another line with tag edition (and using the same number), just a new replacement sed operation should be added with amended regular expression to fit this line.
Finally, the overall code in a better look:
function add_number() {
old_number=$(cat $1 | sed -n 's/[ ]*"name": "ABC #\([0-9]*\)",/\1/p')
new_number=$(($old_number+9500))
sed -i.bak "s/\(\"name\": \"ABC #\)[0-9]*\",/\1${new_number}\",/" $1
sed -i.bak "s/\(\"edition\": \)[0-9]*,/\1${new_number},/" $1
}
for f in *.json
do add_number $f
done
Those previous answers helped me to write this code:
using variables inside of sed
assigning the variable
If you are going to manipulate your JSON files on more than just this one occasion, then you might want to consider using tools that are designed to accomplish such tasks with ease.
One popular choice could be jq which is a "lightweight and flexible command-line JSON processor" that "has zero runtime dependencies" and is also available for OS X. By using jq within your shell, the following would be one way to accomplish what you have asked for.
Adding the numeric value 9500 to the number sitting in the field called edition:
jq '.edition += 9500' file.json
Interpreting a part of a string as number, adding again 9500 to it, and recomposing the string:
jq '.name |= ((./"#" | .[1] |= "\(tonumber + 9500)") | join("#"))' file.json
On the whole, iterating over your files, making both changes at once, writing to a temporary file and replacing the original on success, while having the value to be added as external variable:
v=9500
for f in *.json; do jq --argjson v $v '
.edition += $v | .name |= ((./"#" | .[1] |= "\(tonumber + $v)") | join("#"))
' "$f" > "$f.new" && mv "$f.new" "$f"
done
Here is an online "playground for jq", set up to simulate the application of my code from above to three imaginary files of yours. Feel free to edit the jq filter and/or the input JSON in order to see what could be possible using jq.
I have a JSON like below coming from curl command and it is present in a output.txt file. I want to retreive the JIRA status, here it is "In Progress"
{
"self": "https://jira.com/jira/rest/api/2/issue/1",
"fields": {
"status": {
"self": "https://jira.com/jira/rest/api/2/status/10170",
"description": "",
"name": "In Progress",
"id": "10170"
}
}
}
I have a restriction to use only sed . I tried like below it does not work . I am not sure how to navigate to the name value. can you please suggest to print JIRA status
sed -n 's|.*"fields":{"status":{"name":"\([^"]*\)".*|\1|p' output.txt
You can use
sed -n 's/^[[:space:]]*"name": "\(.*\)",/\1/p' output.txt
# With GNU sed:
sed -n 's/^\s*"name":\s*"\(.*\)",/\1/p' output.txt
See the online demo
Details:
n - suppresses default line output
^\s*"name":\s*"\(.*\)", - matches
^ - start of string
\s* - zero or more whitespaces
"name": - a literal string
\s* - zero or more whitespaces
" - a " char
\(.*\) - a POSIX BRE capturing group matching any text up to
", - the last occurrence of ", (they are at the end of the targeted line anyway).
\1 - replaces the whole match with Group 1 value
p - only prints the replacement result.
With a GNU sed, you can also use the -z option to read the file as a single string and then use a more specific pattern:
sed -z 's/.*"fields":\s*{\s*"status": {.*\s*name": "\([^"]*\)".*/\1/' output.txt
See this online demo.
It does something very close to this demo.
Let's say 123.json with below content:
{
"LINE" : {
"A_serial" : "1234",
"B_serial" : "2345",
"C_serial" : "3456",
"X_serial" : "76"
}
}
If I want to use a shell script to change the parameter of X_serial by the original number +1 which is 77 in this example.
I have tried the below script to take out the parameter of X_serial:
grep "X_serial" 123.json | awk {print"$3"}
which outputs 76. But then I don't know how to make it into 77 and then put it back to the parameter of X_serial.
It's not a good idea to use line-oriented tools for parsing/manipulating JSON data. Use jq instead, for example:
$ jq '.LINE.X_serial |= "\(tonumber + 1)"' 123.json
{
"LINE": {
"A_serial": "1234",
"B_serial": "2345",
"C_serial": "3456",
"X_serial": "77"
}
}
This simply updates .LINE.X_serial by converting its value to a number, increasing the result by one, and converting it back to a string.
You need to install powerful JSON querying processor like jq processor. you can can easily install from here
once you install jq processor, try following command to extract the variable from JSON key value
value=($(jq -r '.X_serial' yourJsonFile.json))
you can modify the $value as you preferred operations
With pure Javascript: nodejs and bash :
node <<EOF
var o=$(</tmp/file);
o["LINE"]["X_serial"] = parseInt(o["LINE"]["X_serial"]) + 1;
console.log(o);
EOF
Output
{ LINE:
{ A_serial: '1234',
B_serial: '2345',
C_serial: '3456',
X_serial: 78 }
}
sed or perl, depending on whether you just need string substitution or something more sophisticated, like arithmetic.
Since you tried grep and awk, let's start with sed:
In all lines that contain TEXT, replace foo with bar
sed -n '/TEXT/ s/foo/bar/ p'
So in your case, something like:
sed -n '/X_serial/ s/\"76\"/\"77\"/ p'
or
$ cat 123.json | sed '/X_serial/ s/\"76\"/\"77\"/' > new.json
This performs a literal substiution: "76" -> "77"
If you would like to perform arithmetic, like "+1" or "+10" then use perl not sed:
$ cat 123.json | perl -pe 's/\d+/$&+10/e if /X_serial/'
{
"LINE" : {
"A_serial" : "1234",
"B_serial" : "2345",
"C_serial" : "3456",
"X_serial" : "86"
}
}
This operates on all lines containing X_serial (whether under "LINE" or under something else), as it is not a json parser.
I have created one text file with all groupId which I have collected
from pom.xml and wanted to compare with predefined configuration it's
done using sed sh command. My text file is like
com.capgemini.psd2
org.springframework.boot
org.springframework.boot
com.capgemini.psd2
I wanted to convert as below format using sed/sh
groupId: {
"key1":"com.capgemini.psd2"
"key2":"org.springframework.boot"
"key3":"org.springframework.boot"
"key4": "com.capgemini.psd2"}
Although you would probably be better off using something like jq or python for processing JSON/XML data, sed is not ideal for pre-post processing. This solution proposes to use awk.
Considering you want NR to be keynum.
awk '
BEGIN {print "groupId: {"}
{print "\"key" NR "\":" "\""$0 "\""}
END {print "}"}' file
Output:
groupId: {
"key1":"com.capgemini.psd2"
"key2":"org.springframework.boot"
"key3":"org.springframework.boot"
"key4":"com.capgemini.psd2"
}
Actually my JSON looks like below:
"checksum": "sdkjjfj-shbdjfj23"
I wanted to replace the checksum value with another value. As per above I used :
sed -i 's/("checksum": ")[^"]*(")/\1$checksumVal\2/g' new.json
The new.json is updated as below, but i wanted the value of that variable.
"checksum": "$checksumVal",
Expected Result:
"checksum": "newval"
Any help would be appreciated.
First of all use one echo statement and check whether $checksumVal has some value or not and also try like below it work for me. How will it work
sed "/checksum will search for pattern "checksum once it found the pattern it will search for pattern : "*" (with wild card) in this example it is : "sdkjjfj-shbdjfj23" and this pattern will be replace with : "$checksumVal"
sed "/checksum/s/: ".*"/: \"$checksumVal\"/" new.json
What's wrong with jq?:
$ cat foo.json # added the required {}
{"checksum": "sdkjjfj-shbdjfj23"}
$ echo $checksum
foo
$ jq '.checksum = "'"$checksum"'"' foo.json
{
"checksum": "foo"
}