This question already has answers here:
Capturing Groups From a Grep RegEx
(10 answers)
Closed 3 years ago.
I need to capture a specific string using grep, on Bash Command Line. I cannot use sudo install/apt-install to install any addition software/packages like jq. I am not using any Python Interpreter; simply Bash Command Line.
I tested my regex string with the desired input string on an online regex tester, and it works. Yet when I try it with grep, it provides me the whole text, and not the captured text.
Regex: '":"(.+)","'
Input:
HTTP_RESPONSE='{"access_token":"eyJhbGciOiJSUzI1NiIsImtpZCI6IkE4M0UwQTFEQTY1MzE0NkZENUQxOTFDMzRDNTQ0RDJDODYyMzMzMzkiLCJ0eXAiOiJKV1QiLCJ4NXQiOiJxRDRLSGFaVEZHX1YwWkhEVEZSTkxJWWpNemsifQ.eyJuYmYiOjE1NTkzMjk5OTIsImV4cCI6MTU1OTMzMzU5MiwiaXNzIjoiaHR0cHM6Ly9jaW5jaHktbnByLmNsb3VkLnJlcy5ibmdmLmxvY2FsL2NpbmNoeXNzbyIsImF1ZCI6WyJodHRwczovL2NpbmNoeS1ucHIuY2xvdWQucmVzLmJuZ2YubG9jYWwvY2luY2h5c3NvL3Jlc291cmNlcyIsImpzX2FwaSJdLCJjbGllbnRfaWQiOiJhcGkiLCJzdWIiOiIxIiwiYXV0aF90aW1lIjoxNTU5MzI5OTkyLCJpZHAiOiJsb2NhbCIsInByb2ZpbGUiOiJBZG1pbmlzdHJhdG9yIiwiZW1haWwiOiJhZG1pbkBjaW5jaHkuY28iLCJyb2xlIjoiQ2luY2h5IFVzZXIgQWNjb3VudCIsImlkIjoiYWRtaW4iLCJzY29wZSI6WyJqc19hcGkiXSwiYW1yIjpbImN1c3RvbSJdfQ.gI_MYzkanGmDXPd8Wp1Q2XOOhAvnWxzOxSUngxYv8UFO1ozXm5ZxFDpUVXFodTQHKEk2B9h9VMXwyg80bxqqT7csHnjg43tylgEIKpupcubqXT1HzH_bMv8povGU6S75b8p8SWQKyYihm4nBdMDJFZMtKMg95ByBlaHpXV_6vuLUB0qFfcbWi5rHgHnJOT08ZJcHJUozS05FVZt_lU2zepNrwSqOvY3AnXpz9Z8KfrlARB46ukB3tBJzNvMDvxi44wfwpnky-4dxq0CcHoa766l6E66gjaLmR3ApHy4YKnP_DxDSR6mSHnVTEzPN7mt_1rklseK0TVd0kK3CpUg2aQ","expires_in":3600,"token_type":"Bearer"}'
echo $HTTP_RESPONSE | grep -E '":"(.+)","'
Obtained Result:
{"access_token":"eyJhbGciOiJSUzI1NiIsImtpZCI6IkE4M0UwQTFEQTY1MzE0NkZENUQxOTFDMzRDNTQ0RDJDODYyMzMzMzkiLCJ0eXAiOiJKV1QiLCJ4NXQiOiJxRDRLSGFaVEZHX1YwWkhEVEZSTkxJWWpNemsifQ.eyJuYmYiOjE1NTkzMjk5OTIsImV4cCI6MTU1OTMzMzU5MiwiaXNzIjoiaHR0cHM6Ly9jaW5jaHktbnByLmNsb3VkLnJlcy5ibmdmLmxvY2FsL2NpbmNoeXNzbyIsImF1ZCI6WyJodHRwczovL2NpbmNoeS1ucHIuY2xvdWQucmVzLmJuZ2YubG9jYWwvY2luY2h5c3NvL3Jlc291cmNlcyIsImpzX2FwaSJdLCJjbGllbnRfaWQiOiJhcGkiLCJzdWIiOiIxIiwiYXV0aF90aW1lIjoxNTU5MzI5OTkyLCJpZHAiOiJsb2NhbCIsInByb2ZpbGUiOiJBZG1pbmlzdHJhdG9yIiwiZW1haWwiOiJhZG1pbkBjaW5jaHkuY28iLCJyb2xlIjoiQ2luY2h5IFVzZXIgQWNjb3VudCIsImlkIjoiYWRtaW4iLCJzY29wZSI6WyJqc19hcGkiXSwiYW1yIjpbImN1c3RvbSJdfQ.gI_MYzkanGmDXPd8Wp1Q2XOOhAvnWxzOxSUngxYv8UFO1ozXm5ZxFDpUVXFodTQHKEk2B9h9VMXwyg80bxqqT7csHnjg43tylgEIKpupcubqXT1HzH_bMv8povGU6S75b8p8SWQKyYihm4nBdMDJFZMtKMg95ByBlaHpXV_6vuLUB0qFfcbWi5rHgHnJOT08ZJcHJUozS05FVZt_lU2zepNrwSqOvY3AnXpz9Z8KfrlARB46ukB3tBJzNvMDvxi44wfwpnky-4dxq0CcHoa766l6E66gjaLmR3ApHy4YKnP_DxDSR6mSHnVTEzPN7mt_1rklseK0TVd0kK3CpUg2aQ","expires_in":3600,"token_type":"Bearer"}
Expected:
eyJhbGciOiJSUzI1NiIsImtpZCI6IkE4M0UwQTFEQTY1MzE0NkZENUQxOTFDMzRDNTQ0RDJDODYyMzMzMzkiLCJ0eXAiOiJKV1QiLCJ4NXQiOiJxRDRLSGFaVEZHX1YwWkhEVEZSTkxJWWpNemsifQ.eyJuYmYiOjE1NTkzMjk5OTIsImV4cCI6MTU1OTMzMzU5MiwiaXNzIjoiaHR0cHM6Ly9jaW5jaHktbnByLmNsb3VkLnJlcy5ibmdmLmxvY2FsL2NpbmNoeXNzbyIsImF1ZCI6WyJodHRwczovL2NpbmNoeS1ucHIuY2xvdWQucmVzLmJuZ2YubG9jYWwvY2luY2h5c3NvL3Jlc291cmNlcyIsImpzX2FwaSJdLCJjbGllbnRfaWQiOiJhcGkiLCJzdWIiOiIxIiwiYXV0aF90aW1lIjoxNTU5MzI5OTkyLCJpZHAiOiJsb2NhbCIsInByb2ZpbGUiOiJBZG1pbmlzdHJhdG9yIiwiZW1haWwiOiJhZG1pbkBjaW5jaHkuY28iLCJyb2xlIjoiQ2luY2h5IFVzZXIgQWNjb3VudCIsImlkIjoiYWRtaW4iLCJzY29wZSI6WyJqc19hcGkiXSwiYW1yIjpbImN1c3RvbSJdfQ.gI_MYzkanGmDXPd8Wp1Q2XOOhAvnWxzOxSUngxYv8UFO1ozXm5ZxFDpUVXFodTQHKEk2B9h9VMXwyg80bxqqT7csHnjg43tylgEIKpupcubqXT1HzH_bMv8povGU6S75b8p8SWQKyYihm4nBdMDJFZMtKMg95ByBlaHpXV_6vuLUB0qFfcbWi5rHgHnJOT08ZJcHJUozS05FVZt_lU2zepNrwSqOvY3AnXpz9Z8KfrlARB46ukB3tBJzNvMDvxi44wfwpnky-4dxq0CcHoa766l6E66gjaLmR3ApHy4YKnP_DxDSR6mSHnVTEzPN7mt_1rklseK0TVd0kK3CpUg2aQ
With GNU grep and Perl-compatible regular expression (PCRE), if no more suitable tool (jq, e.g.) is available:
echo "$HTTP_RESPONSE" | grep -oP '(?<="access_token":").*?(?=")'
Related
This question already has an answer here:
"Argument list too long" while slurping JSON files [duplicate]
(1 answer)
Closed 1 year ago.
There are more than 6k JSON files, each containing exactly one JSON object. I want to prepare one list of objects from these JSONs.
When I am running below jq command I am getting an error.
Kedar.Javalkar#KD2806 MINGW64 /c/zz
$ jq -s '.' inventoryItem_*.json > inventory_items_result_$(date +"%Y%m%d_%H%M%S").json
bash: /usr/bin/jq: Argument list too long
I tried ulimit -s unlimited but the same error
I am using a windows 10 git bash
This is a job that xargs is created to fix -- splitting lists of items into individual command lines that are within the permitted limit.
Because running jq -s a single time is different from concatenating the results of multiple smaller runs, it's appropriate to use xargs to combine cat invocations using the manner described in the linked duplicate.
printf '%s\0' inventoryItem_*.json \
| xargs -0 cat \
| jq -s . \
>"inventory_items_result_$(date +"%Y%m%d_%H%M%S").json"
This question already has answers here:
Argument list too long error for rm, cp, mv commands
(31 answers)
Closed 1 year ago.
I have thousands of JSON files, and I want to merge them into a single one. I'm using the command below to do this.
jq -s . -- *.json > result.json
But I am getting argument list too long error, probably because of the number of files I'm trying to merge. Is there any workaround for this issue?
Built-in commands are immune to that limitation, and printf is one of them. In conjunction with xargs, it would help a lot to achieve this.
printf '%s\0' *.json | xargs -0 cat -- | jq -s .
This question already has answers here:
How to parse json response in the shell script?
(4 answers)
Extract json value with sed
(5 answers)
Closed 2 years ago.
This command
ubus -S call system board
gives me this output
{"kernel":"4.14.195","hostname":"OpenWrt","system":"ARMv7 Processor rev 1 (v7l)","model":"Linksys WRT32X","board_name":"linksys,venom","release":{"distribution":"OpenWrt","version":"19.07.4","revision":"r11208-ce6496d796","target":"mvebu/cortexa9","description":"OpenWrt 19.07.4 r11208-ce6496d796"}}
I want to just extract the model and, if there's a space, replace it with an underscore so I end up with
Linksys_WRT32X
Your command output is json, you can extract the value of the "model" field in pure text and use sed to replace any spaces with underscore.
<command> | jq -r '.model' | sed 's/ /_/g'
or using one jq command to select and replace the text value (thanks #Cyrus)
<command> | jq -r '.model | sub(" "; "_")'
If you don't have jq here is an awk for this
awk -v RS=, -F: '$1 ~ /model/{gsub(/\"/,""); gsub(" ","_"); print $2}'
Have in mind, that one awk or sed for this specific output is fine, but if this was for any json, it could break, json is not text, it can be printed in various lines, it can have additional spaces in places etc.
There are many ways to do this. Here's another one:
ubus -S call system board | sed 's/.*"model":"\([^"]*\)".*$/\1/' | tr ' ' _
I would attempt to answer this using my own version. Since GNU grep has support for Perl regex, we can get the result using:
ubus -S call system board | grep -oP '"model":"\K[^"]+' | tr ' ' _
We attempt to take advantage of the pattern that the JSON key-value pair is of the form "key-name":"<value>" and try to capture the <value> part using grep.
This question already has answers here:
Parsing JSON with Unix tools
(45 answers)
Closed 6 years ago.
Below is my JSON string available in file, out of which I need to extract the value for status in shell script.
Expected Output: status=success
response.json
{"eventDate":null,"dateProccessed":null,"fileName":null,"status":"success"}
Please let me know, if you need any information
Look into jq. It is a very handy json parser.
If you want to extract just the status you can do something like:
jq -r '.status' response.json
# output
success
You can format your output as well to follow the results you want.
jq -r '"status=\(.status)"' response.json
# output
status=success
You can try sed:
sed -n 's|.*"status":"\([^"]*\)".*|status=\1|p' response.json
This question already has answers here:
Read JSON data in a shell script [duplicate]
(4 answers)
Closed 7 years ago.
Let's say that we have this kind of JSON file:
{
...
"quotes":{
"SOMETHING":10,
...
"SOMETHING_ELSE":120.4,
...
} }
How can I obtain those values and use them in order to add them together?
Am I able to do even this?
#!/bin/bash
#code ...
echo "$SOMETHING + $SOMETHING_ELSE" | bc
#code ...
#exit
I will obtain the JSON file with wget command. All I want is the content from this file.
Can you help me, please? I am a beginner in shell programming.
I usually use jq, a really fast json parser, to do this kind of things (because parsing a json file with tools like awk or sed is really error-prone).
Given an input file like this:
# file: input.json
{
"quotes":{
"SOMETHING":10,
"SOMETHING_ELSE":120.4
}
}
You can obtain the sum of the 2 fields with a simple filter:
jq '.quotes.SOMETHING + .quotes.SOMETHING_ELSE' input.json
# output -> 130.4
NOTE: jq is available in every major linux distribution. In a debian-derivative system you can install with a sudo apt-get install jq.
This will print out the sum of the selected lines' floats.
#!/bin/bash
awk '{ if ($1 ~ /"SOMETHING":/) {print}; if ($1 ~ /"SOMETHING_ELSE":/) {print} }' $1 | cut -d: -f2 | cut -d, -f1 | awk '{s+=$1};END{print s}'
This finds the lines you want, the plucks out the numbers, and adds them.
You should look up and learn jq as shown in Read the json data in shell script.
The tools in a "normal" shell installation like awk and sed all predate JSON by decades, and are a very very bad fit. jq is worth the time to learn.
Or use Python instead.