How to change a value in .spec file using sed? Want to change the $nmatch value of build to someother value say "build1.1".
{
"files": [
{
"aql": {
"items.find": {
"repo": {"$eq":"app-java-repo"},
"path": "archives/test/app",
"type": "folder",
"$or": [
{
"$and": [
{
"name": {"$nmatch" : "*build*"} //change value using sed
}
]
}
]
}
}
}
]
}
Tried below command but not working
sed -i -e '/name:/{s/\($nmatch\).*/\1: "'"build1.1"'" /}'
Your document format is json, so you should use jq for editing:
jq '.files[0].aql."items.find"."$or"[0]."$and"[0].name."$nmatch"="build1.1"' spec
With sed, you can't reliably access the correct name node. Your code doesn't work because you didn't include the quotes around name in your search pattern. Using sed would be ok if you have a unique identifier for build. Example:
sed -i 's/BUILD_NUMBER/build1.1/g' spec
A more "indirect" way is to simply set everything with $nmatch:
jq '(.. | select(has("$nmatch"))? ) = "build1.1"' input.json
Related
I try to preprocess a JSON file and want to remove all subobjects in it.
Note that my JSON string is inline (not have \r).
In input I have:
{
"total": 2,
"result": [
{
"id":1,
"createdOn" : 123456,
"obj1": {
...:...,
...:...
},
"obj2": {
...:...,
...:...
},
"otherattribute": "..."
},
{
"id":2,
"createdOn" : 123456,
"obj1": {
...:...,
...:...
},
"obj2": {
...:...,
...:...
},
"otherattribute": "..."
}
]
}
And want to have:
{
"total": 2,
"result": [
{
"id":1,
"createdOn" : 123456,
"otherattribute": "..."
},
{
"id":2,
"createdOn" : 123456,
"otherattribute": "..."
}
]
}
I've tried to use sed command with :
sed 's/"obj1":{[^}]*//g'
It will remove the "ojb1" subobject, but let remain the endind "}," of this subobject.
I didn't find the way to also remove "},".
How?
Second question: I know the list of suboject; but is there a way to remove all theses subjects directly without knowing there name? Something like:
sed 's/".*":{[^}]*//g'
So that I will have only one sed command and not having to chain commands like :
sed 's/obj1//g' | sed 's/obj2//g' | sed 's/obj3//g' ...
I'm giving a first answer:
sed 's/"obj1":{[^}},]*//g'
It will match "obj1": until character } and then match again },, so that these last 2 chars will be also deleted (replaced with nothing).
Now I try to do something more generic to match any object name, with something like this:
sed 's/"[[:alnum:]]":{[^}},]*//g'
But it's not working...
I've already tried to escape or escape escape or escape escape escape double quote without success...
I have some input, which you may recognize as a JSON stream, which I need to process from a simple shell script
I've opted to use sed, because half a dozen other manipulations I need to make are already working great with sed.
Below is the scenario.
input1:
{
"Expr": {
"Criteria": [
{
"Values": [
{
"displayValue": "MyObject",
"value": "UUID"
}
]
}
],
"Type": "SELECTED"
}
}
Intended result 1 (print the clause[s] containing displayValue + value pairs}:
{
"displayValue": "MyObject",
"value": "UUID"
}
Attempt:
$ sed -n -e '/"Expr":/,/"Type": "SELECTED"/ {/"Values":/,/^ }$/ {/^ {/,/^ }/p;};}' /tmp/input1.json
Actual result (good):
{
"displayValue": "MyObject",
"value": "UUID"
}
Problem is with this input2:
{
"Expr": {
"Criteria": [
{
"Values": [
{
"displayValue": "MyObject",
"value": "UUID"
}
]
}
],
"Type": "NOT_SELECTED"
}
}
Result is unexpected (since outer nest is NOT_SELECTED, do not want match unless SELECTED):
$ sed -n -e '/"Expr":/,/"Type": "SELECTED"/ {/"Values":/,/^ }$/ {/^ {/,/^ }/p;};}' /tmp/input2.json
Result 2:
{
"displayValue": "MyObject",
"value": "UUID"
}
Is it me, or is this a sed bug on my platform?
Otherwise, can you suggest a good approach to this problem?
The constraint is I do not want to print "Criteria" lines that are not within the "Type" is "SELECTED" range.
This might work for you (GNU sed):
sed -En '/"Expr"/{:a;n;H;/"Values"/h;/"Type"/!ba;/"SELECTED"/!d;g;s/.*\n((\s*)\{.*\2\}).*/\1/p}' file
Use a loop rather than a range.
Form a loop between "Expr" and "Type" and only proceed if the quoted word "SELECTED" is present.
While executing the loop save Values in the hold space and then manipulate the saved content to just show the parts needed.
N.B. This is a filtering operation so use the -n option.
Remember that sed does not look ahead in the file. So, multiline techniques must be used for this job. My approach is to "slurp" lines between toplevel { and }, which I assume they consist of a single character ({ or }, and no leading and trailing spaces), and examine the resultant pattern space:
sed '
/^{$/,/^}$/{
//!{H;d;}
g
s/.*"Values":.*\(\n[[:space:]]*{[^}]*"displayValue":[^}]*"value":[^}]*}\).*"Type":[[:space:]]"SELECTED".*/\1/p
s/.*//
h
d
}' file
Like this:
I have a json file with below format:
{
"Item": {
"summary": {
"B": "ABCDE"
},
"name": {
"S": "sider"
},
"age": {
"N": "1"
},
"data": {
"B": "abcde"
}
}
}
How can I get the object "Item" only using existed commands like sed/awk without installing any external tools in shell?
expected output:
{
"summary": {
"B": "ABCDE"
},
"name": {
"S": "sider"
},
"age": {
"N": "1"
},
"data": {
"B": "abcde"
}
}
As chepner suggests, you need to question your constraints. In some cases, you are dealt a bad hand and have to deal with it. So here is a sed approach:
Here's a revised solution with awk. Sed is thrown in also to un-indent:
awk '
/^ }/ { p = 0 ; print }
p == 1 { print }
/"Item": {/ { print " {" ; p = 1 }
' | sed 's/^ //'
Originally posted this (but noticed that the output didn't match your expectations):
sed -n '/^ "Item": {/,/^ }/p'
The above sed approach assumes that Item is indented as you have it in the sample input above.
Here's an approach using my favorite, jq:
jq '.Item'
You may do well to check if python with the json package is installed on your machine. Here's a python3 script that would suit your needs:
#!/usr/bin/env python3
import json
import sys
j = json.load( sys.stdin )
print(json.dumps(j["Item"]))
Idiomatically, it'd be incorrect to manipulate a nested data format like JSON with line-aware tools like sed/awk. However if you're limited in your choice, then the best approach is following:
convert multi-line file to a single line
using awk/sed extract your Item
Here's a sed based solution:
bash $ <file.json tr '\n' ' ' | sed -E 's/^ *{ *"Item": +//; s/ *}$//'
This is what you need:
sed -n '/^ "Item": {/,/^ }/{s/"Item": //;s/^ //;p}'
which essentially build upon Mark's solution by making two substitutions to remove "Item": and de-indent 4 spaces before printing.
I have a file:
{
"test_data": [
{
"id": "1",
"pm": "30",
"mp": "40"
}
],
"test": [
"id",
"pm",
"mp"
]
}
I want to extract test_data. Output:
"test_data": [
{
"id": "1",
"pm": "30",
"mp": "40"
}
],
I try this command: cat myFile | sed -n '/^"test_data": \[$/,/^\],$/p'
But it's don't work. An idea ?
Thanks you !
jq seems the right tool for the job :
$ jq '.|{test_data:.test_data}' filename
{
"test_data": [
{
"id": "1",
"pm": "30",
"mp": "40"
}
]
}
Solution 1st: With sed
sed -n '/"test_data"/,/],/p' Input_file
OR: as per OP, OP needs to append a string/data after a line matches:
sed -n '/"test_data"/,/],/p;/],/s/$/"test"/p' Input_file
OR2: If one wants to add an another file's content to a match then following may help in same:
sed -n '/"test_data"/,/],/p;/],/r another_file' Input_file
Solution 2nd: Following simple awk may help you in same.
awk '/test_data/, /],/' Input_file
Output will be as follows.
"test_data": [
{
"id": "1",
"pm": "30",
"mp": "40"
}
],
Logic for above solutions:
For sed: -n option in sed will turn OFF the printing of any line till it is explicitly mentioned to print it, then by doing /"test_data"/,/],/ I am letting sed know that I need to get the data from test_data to till ,/] and mentioning p after that will make sure those lines which are coming in this range are getting printed here/
For awk: Simply mentioning the range from /"test_data"/,/],/ and not mentioning any action so when any line comes into this range condition becomes true and since no action mentioned so by default print of that line happens then.
You can try that with gnu
csplit -s -z infile %test_data%-1 /],/1;rm xx01;echo "Add Text here" >> xx00;cat xx00
The right way is jq tool:
jq 'del(.test)' inputfile
The output:
{
"test_data": [
{
"id": "1",
"pm": "30",
"mp": "40"
}
]
}
I've been trying to use sed to remove agent4 as well as the comma after agent 3 (So it stays valid json). I originally tried sed 's/\"agent4\"/ which then morphed into sed 's/,\n\s+\"agent4\"/ but I found out that sed reads individual lines so I haven't been able to progress from here.
It has to use sed.
{
"environments": {
"default": {
"machines": {
"dev-machine": {
"agents": [
"agent1",
"agent2",
"agent3",
"agent4"
]
}
}
}
}
}
Using a proper JSON tool like jq:
% jq '.environments.default.machines["dev-machine"].agents |= .[:-1]' tmp.json
{
"environments": {
"default": {
"machines": {
"dev-machine": {
"agents": [
"agent1",
"agent2",
"agent3"
]
}
}
}
}
}
Try this command,
$ sed '/3\"/ s/,//g; N; s/\"agent4\"/,/' data.json
Reference: How can I use sed to replace a multi-line string?