I've been trying to get the data from this command ioreg -r -c "AppleSmartBattery" and save each one of its inputs to a sql db
$ ioreg -r -c "AppleSmartBattery"
+-o AppleSmartBattery <class AppleSmartBattery, id 0x1000222c9, registered, ma$
{
"TimeRemaining" = 179
"AvgTimeToEmpty" = 179
"AdapterDetails" = {"FamilyCode"=0}
"ChargingOverride" = 0
"AppleRawCurrentCapacity" = 2373
"InstantTimeToEmpty" = 154
"AppleRawMaxCapacity" = 3811
"ExternalChargeCapable" = No
I would need to save it to a sql table, where one column is "*" and the next one is the value after the equal
I was trying to build a "for loop", I got this far I cant figure out how to continue
batstat=$(ioreg -r -c "AppleSmartBattery")
for i in ${batstat[#]}; do
sed 's/^[^{]*{\([^{}]*\)}.*/\1/' $i
echo $i
done
I would need to accomplish the following
get one single value in quotes "" out each time the for goes by the line
assign the correct value after the equals sign to the respective quoted value
thanks :)
Not that it's impossible, but I think doing this entirely in a shell script is a bit much when there are easier solutions available.
What I'd do here is convert the output to JSON and then use a Node module like JSON-to-SQL to generate the table from the JSON schema, and JSON-SQL to convert the output to an INSERT statement, which you can then use with any Node SQL client, like sql-client.
You can also probably parse the output more cleanly and easily in Node using a module like sh to capture the ioreg command output, but here's what I came up with for converting the command output into valid JSON.
#!/bin/bash
function parseData() {
tail -n +2 $1 | \
sed -re 's/\=/\:/g' | \
sed -re 's/</\"/g' | \
sed -re 's/>/\"/g' | \
sed -re 's/No/false/g' | \
sed -re 's/Yes/true/g' | \
sed -re 's/\(/\[/g' | \
sed -re 's/\)/\]/g' | \
sed '$d' | \
sed '$d' | \
sed 's/$/,/' | \
sed '1 s/\,//' | \
sed '$ s/\,//' | \
sed '52 s/,//'
}
ioreg -r -c "AppleSmartBattery" | parseData
The only issue is if the number of lines in the output ever changes, the 52 in the last line of the parseData function would need to be updated.
Related
I have a plain text file with hex data information (32-bit word per line). Example :
cafef00d
deadbeef
That I need to convert to this :
11001010111111101111000000001101
11011110101011011011111011101111
BUT with 1 bit per line only. Starting from the LSB of the first 32-bit hex and so on. Final output file will be :
1
0
1
1
... and so on
Is there a unix command/s or can I do this in a Tcl proc ?
A tcl solution...
Assuming you've read the file into a string, the first thing is to convert the hex strings into numbers expressed in binary, LSB first. There's a few ways to do it, here's one (I like scan and format):
set binaryData [lmap hexValue $inputData {
scan $hexValue "%x" value
string reverse [format "%b" $value]
}]
For your input, that produces:
10110000000011110111111101010011 11110111011111011011010101111011
We can then convert that to be one digit per line with this:
set oneDigitPerLine [join [split [join $binaryData ""] ""] "\n"]
The innermost join gets rid of the whitespace, the split breaks it up into characters, and the outer join inserts the newline separators. (I'll not produce the result here.)
If you want to do it with linux commands, try the following:
tac: reverse text lines in a file
fold -w 1: fold a text file, column width 1
sed: replace strings
tac input_file | \
fold -w 1 | \
sed -e 's/0/0000/' | \
sed -e 's/1/0001/' | \
sed -e 's/2/0010/' | \
sed -e 's/3/0011/' | \
sed -e 's/4/0100/' | \
sed -e 's/5/0101/' | \
sed -e 's/6/0110/' | \
sed -e 's/7/0111/' | \
sed -e 's/8/1000/' | \
sed -e 's/9/1001/' | \
sed -e 's/a/1010/' | \
sed -e 's/b/1011/' | \
sed -e 's/c/1100/' | \
sed -e 's/d/1101/' | \
sed -e 's/e/1110/' | \
sed -e 's/f/1111/' | \
told -w 1 | \
tac
Another way, using a perl one-liner:
$ perl -nE 'say for split "", reverse sprintf("%032b", hex)' < input.txt
1
0
1
1
...
For each line, converts from a base-16 string to a number and turns that into a binary string, and then prints each individual character on its own line.
so, I have the example code here:
#!/bin/bash
clear
curl -s https://www.cnbcindonesia.com/market-data/currencies/IDR=/USD-IDR |
html2text |
sed -n '/USD\/IDR/,$p' |
sed -n '/Last updated/q;p' |
tail -n-1 |
head -c+6 && printf "\n"
exit 0
this should print out some number range 14000~15000
lets start from the very basic one, what I have to do in order to print result + 1 ? so if the printout is 14000 and increment it to 1 become 14001. I suppose the result of the html2text is not calculatable since it should be something like string output not integer.
the more advance thing i want to know is how to calculate the result of 2 curl results?
What I would do, bash + xidel:
$ num=$(xidel -se '//div[#class="mark_val"]/span[1]/text()' 'https://url')
$ num=$((${num//,/}+1)) # num was 14050
$ echo $num
Output
14051
Explanations
$((...))
is an arithmetic substitution. After doing the arithmetic, the whole thing is replaced by the value of the expression. See http://mywiki.wooledge.org/ArithmeticExpression
Command Substitution: "$(cmd "foo bar")" causes the command 'cmd' to be executed with the argument 'foo bar' and "$(..)" will be replaced by the output. See http://mywiki.wooledge.org/BashFAQ/002 and http://mywiki.wooledge.org/CommandSubstitution
Bonus
You can compute directly in xidel, thanks Reino using xquery syntax :
$ xidel -s <url> e 'replace(//div[#class="mark_val"]/span[1],",","") + 1'
And to do addition arithmetic of 2 values :
$ xidel -s <url> -e '
let $num:=replace(//div[#class="mark_val"]/span[1],",","")
return $num + $num
'
I have a response trace file containing below response:
#RESPONSE BODY
#--------------------
{"totalItems":1,"member":[{"name":"name","title":"PatchedT","description":"My des_","id":"70EA96FB313349279EB089BA9DE2EC3B","type":"Product","modified":"2019 Jul 23 10:22:15","created":"2019 Jul 23 10:21:54",}]}
I need to fetch the value of the "id" key in a variable which I can put in my further code.
Expected result is
echo $id - should give me 70EA96FB313349279EB089BA9DE2EC3B value
With valid JSON (remove first to second row with sed and parse with jq):
id=$(sed '1,2d' file | jq -r '.member[]|.id')
Output to variable id:
70EA96FB313349279EB089BA9DE2EC3B
I would strongly suggest using jq to parse json.
But given that json is mostly compatible with python dictionaries and arrays, this HACK would work too:
$ cat resp
#RESPONSE BODY
#--------------------
{"totalItems":1,"member":[{"name":"name","title":"PatchedT","description":"My des_","id":"70EA96FB313349279EB089BA9DE2EC3B","type":"Product","modified":"2019 Jul 23 10:22:15","created":"2019 Jul 23 10:21:54",}]}
$ awk 'NR==3{print "a="$0;print "print a[\"member\"][0][\"id\"]"}' resp | python
70EA96FB313349279EB089BA9DE2EC3B
$ sed -n '3s|.*|a=\0\nprint a["member"][0]["id"]|p' resp | python
70EA96FB313349279EB089BA9DE2EC3B
Note that this code is
1. dirty hack, because your system does not have the right tool - jq
2. susceptible to shell injection attacks. Hence use it ONLY IF you trust the response received from your service.
Quick and dirty (don't use eval):
eval $(cat response_file | tail -1 | awk -F , '{ print $5 }' | sed -e 's/"//g' -e 's/:/=/')
It is based on the exact structure you gave, and hoping there is no , in any value before "id".
Or assign it yourself:
id=$(cat response_file | tail -1 | awk -F , '{ print $5 }' | cut -d: -f2 | sed -e 's/"//g')
Note that you can't access the name field with that trick, as it is the first item of the member array and will be "swallowed" by the { print $2 }. You can use an even-uglier hack to retrieve it though:
id=$(cat response_file | tail -1 | sed -e 's/:\[/,/g' -e 's/}\]//g' | awk -F , '{ print $5 }' | cut -d: -f2 | sed -e 's/"//g')
But, if you can, jq is the right tool for that work instead of ugly hacks like that (but if it works...).
When you can't use jq, you can consider
id=$(grep -Eo "[0-9A-F]{32}" file)
This is only working when the file looks like what I expect, so you might need to add extra checks like
id=$(grep "My des_" file | grep -Eo "[0-9A-F]{32}" | head -1)
I was wondering how to parse the CURL JSON output from the server into variables.
Currently, I have -
curl -X POST -H "Content: agent-type: application/x-www-form-urlencoded" https://www.toontownrewritten.com/api/login?format=json -d username="$USERNAME" -d password="$PASSWORD" | python -m json.tool
But it only outputs the JSON from the server and then have it parsed, like so:
{
"eta": "0",
"position": "0",
"queueToken": "6bee9e85-343f-41c7-a4d3-156f901da615",
"success": "delayed"
}
But how do I put - for example the success value above returned from the server into a variable $SUCCESS and have the value as delayed & have queueToken as a variable $queueToken and 6bee9e85-343f-41c7-a4d3-156f901da615 as a value?
Then when I use-
echo "$SUCCESS"
it shows this as the output -
delayed
And when I use
echo "$queueToken"
and the output as
6bee9e85-343f-41c7-a4d3-156f901da615
Thanks!
Find and install jq (https://stedolan.github.io/jq/). jq is a JSON parser. JSON is not reliably parsed by line-oriented tools like sed because, like XML, JSON is not a line-oriented data format.
In terms of your question:
source <(
curl -X POST -H "$content_type" "$url" -d username="$USERNAME" -d password="$PASSWORD" |
jq -r '. as $h | keys | map(. + "=\"" + $h[.] + "\"") | .[]'
)
The jq syntax is a bit weird, I'm still working on it. It's basically a series of filters, each pipe taking the previous input and transforming it. In this case, the end result is some lines that look like variable="value"
This answer uses bash's "process substitution" to take the results of the jq command, treat it like a file, and source it into the current shell. The variables will then be available to use.
Here's an example of Extract a JSON value from a BASH script
#!/bin/bash
function jsonval {
temp=`echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop`
echo ${temp##*|}
}
json=`curl -s -X GET http://twitter.com/users/show/$1.json`
prop='profile_image_url'
picurl=`jsonval`
`curl -s -X GET $picurl -o $1.png`
A bash script which demonstrates parsing a JSON string to extract a
property value. The script contains a jsonval function which operates
on two variables, json and prop. When the script is passed the name of
a twitter user it attempts to download the user's profile picture.
You could use perl module on command line:
1st, ensure they is installed, under debian based, you could
sudo apt-get install libjson-xs-perl
But for other OS, you could install perl modules via CPAN (the Comprehensive Perl Archive Network):
cpan App::cpanminus
cpan JSON::XS
Note: You may have to run this with superuser privileges.
then:
curlopts=(-X POST -H
"Content: apent-type: application/x-www-form-urlencoded"
-d username="$USERNAME" -d password="$PASSWORD")
curlurl=https://www.toontownrewritten.com/api/login?format=json
. <(
perl -MJSON::XS -e '
$/=undef;my $a=JSON::XS::decode_json <> ;
printf "declare -A Json=\047(%s)\047\n", join " ",map {
"[".$_."]=\"".$a->{$_}."\""
} qw|queueToken success eta position|;
' < <(
curl "${curlopts[#]}" $curlurl
)
)
The line qw|...| let you precise which variables you want to be driven... This could be replaced by keys $a, but could have to be debugged as some characters is forbiden is associative arrays values names.
echo ${Json[queueToken]}
6bee9e85-343f-41c7-a4d3-156f901da615
echo ${Json[eta]}
0
I have the json as below, i need to get only the mail from the above json in bash script
value={"count":5,"users":[{"username":"asa","name":"asa
Tran","mail":"asa#xyz.com"},{"username":"qq","name":"qq
Morris","mail":"qq#xyz.com"},{"username":"qwe","name":"qwe
Org","mail":"qwe#xyz.com"}]}
Output can be as
mail=asa#xyz.com,qq#xyz.com,qwe#xyz.com
All the above need to be done in the bash script (.sh)
I have already tried with the array iteration as but of no use
for key in "${!value[#]}"
do
#echo "key = $key"
echo "value = ${value[$key]}"
done
Even i have tried with the array conversion as
alias json-decode="php -r
'print_r(json_decode(file_get_contents(\"php://stdin\"),1));'"
value=$(curl --user $credentials -k $endPoint | json-decode)
Still i was not able to get the specific output.
jq is the tool to iterate through a json. In your case:
while read user; do
jq -r '.mail' <<< $user
done <<< $(jq -c '.users[]' users.json)
would give:
asa#xyz.com
qq#xyz.com
qwe#xyz.com
NOTE: I removed "value=" because that is not valid json. Users.json contains:
{"count":5,"users":[{"username":"asa","name":"asa Tran","mail":"asa#xyz.com"},{"username":"qq","name":"qq Morris","mail":"qq#xyz.com"},{"username":"qwe","name":"qwe Org","mail":"qwe#xyz.com"}]}
If this is valid json and the email field is the only one containing a # character, you can do something like this:
echo $value | tr '"' '\n' | grep #
It replaces double-quotes by new line character and only keeps lines containing #. It is really not json parsing, but it works.
You can store the result in a bash array
emails=($(echo $value | tr '"' '\n' | grep #))
and iterate on them
for email in ${emails[#]}
do
echo $email
done
You should use json_pp tool (in debian, it is part of the libjson-pp-perl package)
One would use it like this :
cat file.json | json_pp
And get a pretty print for your json.
So in your case, you could do :
#!/bin/bash
MAILS=""
LINES=`cat test.json | json_pp | grep '"mail"' | sed 's/.* : "\(.*\)".*/\1/'`
for LINE in $LINES ; do
MAILS="$LINE,$MAILS"
done
echo $MAILS | sed 's/.$//'
Output :
qwe#xyz.com,qq#xyz.com,asa#xyz.com
Using standard unix toolbox : sed command
cat so.json | sed "s/},/\n/g" | sed 's/.*"mail":"\([^"]*\)".*/\1/'
With R you could do this as follows:
$ value={"count":5,"users":[{"username":"asa","name":"asa Tran","mail":"asa#xyz.com"},{"username":"qq","name":"qq Morris","mail":"qq#xyz.com"},{"username":"qwe","name":"qwe Org","mail":"qwe#xyz.com"}]}
$ echo $value | R path users | R map path mail
["asa#xyz.com", "qq#xyz.com", "qwe#gyz.com"]