I have a curl command which results in the following example json:
json={"id":"12345","key":"ABC-DEF","url":"https://google.com"}
Now, I want to parse this, and get the key out of it and store it in a variable. What I did was the following:
json={"id":"12345","key":"ABC-DEF","url":"https://google.com"}
ID=$(echo $json | jq '.key' )
But the above gives me a error as: parse error: Invalid numeric literal at line 1, column 4. Can someone help me with this? Thanks!
You will need to quote the input string so the shell doesn't do anything with your string
json='{"id":"12345","key":"ABC-DEF","url":"https://google.com"}'
ID=$(echo "$json" | jq '.key' )
Related
I am writing a bash script and I am looking to replace a character within a JSON field in a JSON array. In this case, I am trying to change the "." (period) character to a "-" (hyphen) in the name field. I am using jq to parse my JSON. Any tips on how I can achieve this will greatly help. Thank you!
Bash Script so far:
RAW=$(curl ${URL})
function manip() {
# Function for string manipulation.
}
echo "${RAW}" | jq '.data | .[].name = $manip' # Unable to make a function call in there.
Sample JSON:
[
{"id":"1","name":"a.a"},
{"id":"2","name":"b.b"},
{"id":"3","name":"c.c"}
]
Expected Output:
[
{"id":"1","name":"a-a"},
{"id":"2","name":"b-b"},
{"id":"3","name":"c-c"}
]
To replace a dot with a dash, use the sub function:
jq '.[].name |= sub("\\."; "-")' file.json
I'm trying to get jq to parse a JSON structure like:
{
"a" : 1,
"b" : 2,
"c" : "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}\n"
}
That is, an element in the JSON is a string with escaped json.
So, I have something along the lines of
$ jq [.c] myFile.json | jq [.id]
But that crashes with jq: error: Cannot index string with string
This is because the output of .c is a string, not more JSON.
How do I get jq to parse this string?
My initial solution is to use sed to replace all the escape chars (\":\", \",\" and \") but that's messy, I assume there's a way built into jq to do this?
Thanks!
edit:
Also, the jq version available here is:
$ jq --version
jq version 1.3
I guess I could update it if required.
jq has the fromjson builtin for this:
jq '.c | fromjson | .id' myFile.json
fromjson was added in version 1.4.
You can use the raw output (-r) that will unescape characters:
jq -r .c myfile.json | jq .id
ADDENDUM: This has the advantage that it works in jq 1.3 and up; indeed, it should work in every version of jq that has the -r option.
Motivation: you want to parse JSON string - you want to escape a JSON object that's wrapped with quotes and represented as a String buffer, and convert it to a valid JSON object. For example:
some JSON unescaped string :
"{\"name\":\"John Doe\",\"position\":\"developer\"}"
the expected result ( a JSON object ):
{"name":"John Doe","position":"developer"}
Solution: In order to escape a JSON string and convert it into a valid JSON object use the sed tool in command line and use regex expressions to remove/replace specific characters:
cat current_json.txt | sed -e 's/\\\"/\"/g' -e 's/^.//g' -e 's/.$//g'
s/\\\"/\"/g replacing all backslashes and quotes ( \" ) into quotes only (")
s/^.//g replacing the first character in the stream to none character
s/.$//g replacing the last character in the stream to none character
I have a JSON string without the separator ,. How do I parse it using jq?
$echo '{"access_token":"XXXX""expires_in":300"token_type":"Bearer"}' | jq -r .access_token
The above line gives me the below error:
parse error: Expected separator between values at line 1
I understand that the issue is because the JSON string provided is not comma-separated. But this is what I am getting as a response from the server. How do I parse such a string? I want to retrive the value for key "access_token".
You can use a regular expression with sed if you know the accesss token never contains quotes.
echo '{"access_token":"XXXX""expires_in":300"token_type":"Bearer"}' |
sed 's/"access_token":"\([^"]*\)/\1/'
The capture group between \( and \) captures the string between the quotes, and \1 in the replacement string extracts it.
Here are two just-jq solutions, each with its own degree of brittleness. The first one attempts to convert each entire input line into valid JSON:
Using fromjson
echo '{"access_token":"XXXX""expires_in":300"token_type":"Bearer"}' |
jq -rR 'gsub("(?<k>\"[^\"]*\")"; "," + .k )
| gsub("{,\"";"{\"") | gsub(":,\""; ":\"")
| fromjson | .access_token'
XXXX
Assume the value is a string on the same line
jq -rR 'sub(".*\"access_token\" *: *\"(?<v>[^\"]*)\".*"; .v )'
I understand that jq search needs to be blocked by {} and the key needs to be encased with ", for example:
{
"id": 36815684
}
But if I have something like this:
X-RateLimit-Reset: 1452786798
I get this error:
parse error: Invalid numeric literal at line 1, column 9
Do I need to fall back to sed/awk/perl .. or is there a more elegant way of using jq?
Apart from not using jq at all, you have two main options:
(1) pre-processing the non-JSON to make it JSON
(2) using the -R command-line option, e.g.
echo "X-RateLimit-Reset: 1452786798" | jq -R 'split(":")'
[
"X-RateLimit-Reset",
" 1452786798"
]
Thus, if you know the value is going to be numeric:
echo "X-RateLimit-Reset: 1452786798" |
jq -Rc 'split(":") | {(.[0]) : (.[1]|tonumber)}'
{"X-RateLimit-Reset":1452786798}
Note that although the "j" in jq is for JSON, jq (with the -R option) does just fine for text-processing.
I'm trying to get jq to parse a JSON structure like:
{
"a" : 1,
"b" : 2,
"c" : "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}\n"
}
That is, an element in the JSON is a string with escaped json.
So, I have something along the lines of
$ jq [.c] myFile.json | jq [.id]
But that crashes with jq: error: Cannot index string with string
This is because the output of .c is a string, not more JSON.
How do I get jq to parse this string?
My initial solution is to use sed to replace all the escape chars (\":\", \",\" and \") but that's messy, I assume there's a way built into jq to do this?
Thanks!
edit:
Also, the jq version available here is:
$ jq --version
jq version 1.3
I guess I could update it if required.
jq has the fromjson builtin for this:
jq '.c | fromjson | .id' myFile.json
fromjson was added in version 1.4.
You can use the raw output (-r) that will unescape characters:
jq -r .c myfile.json | jq .id
ADDENDUM: This has the advantage that it works in jq 1.3 and up; indeed, it should work in every version of jq that has the -r option.
Motivation: you want to parse JSON string - you want to escape a JSON object that's wrapped with quotes and represented as a String buffer, and convert it to a valid JSON object. For example:
some JSON unescaped string :
"{\"name\":\"John Doe\",\"position\":\"developer\"}"
the expected result ( a JSON object ):
{"name":"John Doe","position":"developer"}
Solution: In order to escape a JSON string and convert it into a valid JSON object use the sed tool in command line and use regex expressions to remove/replace specific characters:
cat current_json.txt | sed -e 's/\\\"/\"/g' -e 's/^.//g' -e 's/.$//g'
s/\\\"/\"/g replacing all backslashes and quotes ( \" ) into quotes only (")
s/^.//g replacing the first character in the stream to none character
s/.$//g replacing the last character in the stream to none character