I have a question about json and parsing with sed:
Here is what I get in json:
response='{"found":"true","downloadLink":"http:\/\/www.addic7ed.com\/updated\/1\/86593\/2"}'
If I use this:
downloadLink=`echo $response | sed -e 's/^.*"downloadLink"[ ]*:[ ]*"//' -e 's/".*//'`
then downloadLink will contain http:\/\/www.addic7ed.com\/updated\/1\/86593\/2.
I tried to put a variable instead of downloadLink:
downloadLink=`echo $response | sed -e 's/^.*"$value"[ ]*:[ ]*"//' -e 's/".*//'`
But it doesn't seem to work properly. Do you know how to do it?
The single quotes are not expanded in bash. Use double quotes and escape those already used: - like this:
echo $response | sed -e "s/^.*\"$value\"[ ]*:[ ]*\"//" -e 's/".*//'
Rather than using two sed commands, you can capture the value you are interested like this:
echo "$response" | sed -e "s/^.*\"$value\"\s*:\s*\"\([^\"]*\)\".*$/\1/"
The contents of the \( \) are captured into the variable \1. I have chosen to capture [^\"]* (any number of characters that are not a double quote), which works for your example.
I am also using the \s "whitespace" character class rather than [ ], as I believe it is clearer.
Testing it out:
$ echo "$response"
{"found":"true","downloadLink":"http:\/\/www.addic7ed.com\/updated\/1\/86593\/2"}
$ value=downloadLink
$ echo "$response" | sed -e "s/^.*\"$value\"\s*:\s*\"\([^\"]*\)\".*$/\1/"
http:\/\/www.addic7ed.com\/updated\/1\/86593\/2
$ value=found
$ echo "$response" | sed -e "s/^.*\"$value\"\s*:\s*\"\([^\"]*\)\".*$/\1/"
true
By the way, if you're using bash, you can avoid echo $var | sed by using <<<:
sed -e "s/^.*\"$value\"\s*:\s*\"\([^\"]*\)\".*$/\1/" <<<"$response"
Variables are not expanded inside single quotes. You could use double quotes insted like
sed "s/$variable/newlaue/g" ...
but then you should be extra careful with the contents of $variable, since sed will interpret any special characters in the variable (like the slash /) in this specific example.
Related
I have a plain text file with hex data information (32-bit word per line). Example :
cafef00d
deadbeef
That I need to convert to this :
11001010111111101111000000001101
11011110101011011011111011101111
BUT with 1 bit per line only. Starting from the LSB of the first 32-bit hex and so on. Final output file will be :
1
0
1
1
... and so on
Is there a unix command/s or can I do this in a Tcl proc ?
A tcl solution...
Assuming you've read the file into a string, the first thing is to convert the hex strings into numbers expressed in binary, LSB first. There's a few ways to do it, here's one (I like scan and format):
set binaryData [lmap hexValue $inputData {
scan $hexValue "%x" value
string reverse [format "%b" $value]
}]
For your input, that produces:
10110000000011110111111101010011 11110111011111011011010101111011
We can then convert that to be one digit per line with this:
set oneDigitPerLine [join [split [join $binaryData ""] ""] "\n"]
The innermost join gets rid of the whitespace, the split breaks it up into characters, and the outer join inserts the newline separators. (I'll not produce the result here.)
If you want to do it with linux commands, try the following:
tac: reverse text lines in a file
fold -w 1: fold a text file, column width 1
sed: replace strings
tac input_file | \
fold -w 1 | \
sed -e 's/0/0000/' | \
sed -e 's/1/0001/' | \
sed -e 's/2/0010/' | \
sed -e 's/3/0011/' | \
sed -e 's/4/0100/' | \
sed -e 's/5/0101/' | \
sed -e 's/6/0110/' | \
sed -e 's/7/0111/' | \
sed -e 's/8/1000/' | \
sed -e 's/9/1001/' | \
sed -e 's/a/1010/' | \
sed -e 's/b/1011/' | \
sed -e 's/c/1100/' | \
sed -e 's/d/1101/' | \
sed -e 's/e/1110/' | \
sed -e 's/f/1111/' | \
told -w 1 | \
tac
Another way, using a perl one-liner:
$ perl -nE 'say for split "", reverse sprintf("%032b", hex)' < input.txt
1
0
1
1
...
For each line, converts from a base-16 string to a number and turns that into a binary string, and then prints each individual character on its own line.
I've been trying to get the data from this command ioreg -r -c "AppleSmartBattery" and save each one of its inputs to a sql db
$ ioreg -r -c "AppleSmartBattery"
+-o AppleSmartBattery <class AppleSmartBattery, id 0x1000222c9, registered, ma$
{
"TimeRemaining" = 179
"AvgTimeToEmpty" = 179
"AdapterDetails" = {"FamilyCode"=0}
"ChargingOverride" = 0
"AppleRawCurrentCapacity" = 2373
"InstantTimeToEmpty" = 154
"AppleRawMaxCapacity" = 3811
"ExternalChargeCapable" = No
I would need to save it to a sql table, where one column is "*" and the next one is the value after the equal
I was trying to build a "for loop", I got this far I cant figure out how to continue
batstat=$(ioreg -r -c "AppleSmartBattery")
for i in ${batstat[#]}; do
sed 's/^[^{]*{\([^{}]*\)}.*/\1/' $i
echo $i
done
I would need to accomplish the following
get one single value in quotes "" out each time the for goes by the line
assign the correct value after the equals sign to the respective quoted value
thanks :)
Not that it's impossible, but I think doing this entirely in a shell script is a bit much when there are easier solutions available.
What I'd do here is convert the output to JSON and then use a Node module like JSON-to-SQL to generate the table from the JSON schema, and JSON-SQL to convert the output to an INSERT statement, which you can then use with any Node SQL client, like sql-client.
You can also probably parse the output more cleanly and easily in Node using a module like sh to capture the ioreg command output, but here's what I came up with for converting the command output into valid JSON.
#!/bin/bash
function parseData() {
tail -n +2 $1 | \
sed -re 's/\=/\:/g' | \
sed -re 's/</\"/g' | \
sed -re 's/>/\"/g' | \
sed -re 's/No/false/g' | \
sed -re 's/Yes/true/g' | \
sed -re 's/\(/\[/g' | \
sed -re 's/\)/\]/g' | \
sed '$d' | \
sed '$d' | \
sed 's/$/,/' | \
sed '1 s/\,//' | \
sed '$ s/\,//' | \
sed '52 s/,//'
}
ioreg -r -c "AppleSmartBattery" | parseData
The only issue is if the number of lines in the output ever changes, the 52 in the last line of the parseData function would need to be updated.
I need to prepare JSON which contains apostrophe to be sent via CURL.
Example of JSON:
{"myField":"Apos'test"}
Example of JSON I need as an output:
{"myField":"Apos'\''test"}
What I have tried:
sed -e "s/'/'\\\''/g" <<< {"myField":"Apos'test"}
which outputs:
{myField:Apos'\''test}
And I do not understand why it removes double quotes.
P.S. it is not obligatory to use sed, any other standard linux tool would work.
try this:
#/bin/bash
replacement=$((cat << EOT
{"myField":"Apos'test"}
EOT
) | sed "s|'|'\\\''|")
echo $replacement
output:
{"myField":"Apos'\''test"}
It doesn't
If it was because you used <<< , here, of which "" pair was parsed, expanded and dropped by the shell you're in
$ cat d
{"myField":"Apos'test"}
$ sed -E "s/'/'\\\''/g" d
{"myField":"Apos'\''test"}
I wanted to add a new line between </a> and <a><a>
</a><a><a>
</a>
<a><a>
I did this
sed 's#</a><a><a>#</a>\n<a><a>#g' filename but it didn't work.
Powered by mac in two Interpretation:
echo foo | sed 's/f/f\'$'\n/'
echo foo | gsed 's/f/f\n/g'
Some seds, notably Mac / BSD, don't interpret \n as a newline, you need to use an actual newline, preceded by a backslash:
$ echo foo | sed 's/f/f\n/'
fnoo
$ echo foo | sed 's/f/f\
> /'
f
oo
$
Or you can use:
echo foo | sed $'s/f/f\\\n/'
...or you just pound on it! worked for me on insert on mac / osx:
sed "2 i \\\n${TEXT}\n\n" -i ${FILE_PATH_NAME}
sed "2 i \\\nSomeText\n\n" -i textfile.txt
I have the json as below, i need to get only the mail from the above json in bash script
value={"count":5,"users":[{"username":"asa","name":"asa
Tran","mail":"asa#xyz.com"},{"username":"qq","name":"qq
Morris","mail":"qq#xyz.com"},{"username":"qwe","name":"qwe
Org","mail":"qwe#xyz.com"}]}
Output can be as
mail=asa#xyz.com,qq#xyz.com,qwe#xyz.com
All the above need to be done in the bash script (.sh)
I have already tried with the array iteration as but of no use
for key in "${!value[#]}"
do
#echo "key = $key"
echo "value = ${value[$key]}"
done
Even i have tried with the array conversion as
alias json-decode="php -r
'print_r(json_decode(file_get_contents(\"php://stdin\"),1));'"
value=$(curl --user $credentials -k $endPoint | json-decode)
Still i was not able to get the specific output.
jq is the tool to iterate through a json. In your case:
while read user; do
jq -r '.mail' <<< $user
done <<< $(jq -c '.users[]' users.json)
would give:
asa#xyz.com
qq#xyz.com
qwe#xyz.com
NOTE: I removed "value=" because that is not valid json. Users.json contains:
{"count":5,"users":[{"username":"asa","name":"asa Tran","mail":"asa#xyz.com"},{"username":"qq","name":"qq Morris","mail":"qq#xyz.com"},{"username":"qwe","name":"qwe Org","mail":"qwe#xyz.com"}]}
If this is valid json and the email field is the only one containing a # character, you can do something like this:
echo $value | tr '"' '\n' | grep #
It replaces double-quotes by new line character and only keeps lines containing #. It is really not json parsing, but it works.
You can store the result in a bash array
emails=($(echo $value | tr '"' '\n' | grep #))
and iterate on them
for email in ${emails[#]}
do
echo $email
done
You should use json_pp tool (in debian, it is part of the libjson-pp-perl package)
One would use it like this :
cat file.json | json_pp
And get a pretty print for your json.
So in your case, you could do :
#!/bin/bash
MAILS=""
LINES=`cat test.json | json_pp | grep '"mail"' | sed 's/.* : "\(.*\)".*/\1/'`
for LINE in $LINES ; do
MAILS="$LINE,$MAILS"
done
echo $MAILS | sed 's/.$//'
Output :
qwe#xyz.com,qq#xyz.com,asa#xyz.com
Using standard unix toolbox : sed command
cat so.json | sed "s/},/\n/g" | sed 's/.*"mail":"\([^"]*\)".*/\1/'
With R you could do this as follows:
$ value={"count":5,"users":[{"username":"asa","name":"asa Tran","mail":"asa#xyz.com"},{"username":"qq","name":"qq Morris","mail":"qq#xyz.com"},{"username":"qwe","name":"qwe Org","mail":"qwe#xyz.com"}]}
$ echo $value | R path users | R map path mail
["asa#xyz.com", "qq#xyz.com", "qwe#gyz.com"]