‘.”line”’ not accepted as valid syntax by jq - "unexpected INVALID_CHARACTER" - json

I'm trying to make a GET request with curl in a bash file.
So I call curl, and get a JSON response from my server, and I try to get the object line which is part of the response, and I do so with jq, so I can print the value.
But its not working.
My response is something like this:
{"line":"Azul", "id":"j3453j45n35", "lat":"39.091937", "long":"-9.265441"}
My bash script:
#!/bin/bash
while :
do
cmd=$(curl -i -H "Content-Type: application/json" http://localhost:8080/cards/exists/1 | jq ‘.”line”’)
sleep 2
done
The result:
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at , line 1:
‘.”line”’
jq: 1 compile error
(23) Failed writing body
Whats going wrong?
I followed plenty of responses people advised online, but I keep getting the same response.
My json object seems to be correct.
I'm running this file on MacOS Sierra

There are two problems with your code:
You're using non-ASCII quotes (‘ ” ” ’), which jq doesn't understand:
Replace them with their ASCII equivalents: ' " " '
On macOS, avoid a text editor such as the standard TextEdit.app that has the "smart quotes" feature turned on, or turn that feature off via the menu system: Edit > Substitutions > Smart Quotes
It is this feature that replaces ASCII quoting chars. with their non-ASCII, typographical equivalents, as you type.
Generally, consider using a dedicated source-code editor for editing your code, such as Visual Studio Code (free) or Atom (free) or Sublime Text (paid).
Your curl command includes header output (-i), which jq chokes on - simply omit the -i:
#!/bin/bash
while :; do
cmd=$(curl -H 'Content-Type: application/json' 'http://localhost:8080/cards/exists/1' |
jq '.line')
sleep 2
done
While '."line"' would have worked too, there's no need for the extra layer of quoting for simple property names like that.
Also, it's better to enclose strings that you want to be interpreted literally in '...' in the shell.

Related

How can I fix JSON error on shell script?

I have a bash script which sends curl post requests. I want to pass data as bash script parameters. However one of the parameter has spaces in the string and it fails with the error below.
Error parsing JSON data.\n\tString not terminated on line
In shell script, I'm sending an argument like this format {"name":"'$2'"}
Could you please help me to solve that issue?
Thanks
jq is good not only for manipulating existing JSON data, but creating new data, as it does things like correctly handling characters that can't appear unescaped in JSON strings, and proper quoting. Something like
curl ... -d"$(jq -n --arg val "$2" '{name: $val}')"
It would be better if you add enough data while asking question.
I assume your json will be like
{
"name": "argument passed"
}
curl -XPOST "your/url/here" -H 'Content-Type:application/json' -d'{"name":"'$1'"}'
Save the above command as post_request.sh(Feel free to change the name).
Run using below comand.
sh post_request.sh "argument passed"
"argument passed" will be your name with space.

Tell Json this all one line

I am writing a JSON template for AWS SSM document.
Once of the commands I am trying to run looks like this:
"ssh-keygen -q -t rsa -N '' <<< ""$'\n'"y" 2>&1 >/dev/null",
However, due to the "" after the <<< JSON thinks that this is a new line and is expecting a comma.
Is there a way I can tell JSON that that is a single command and need to be treated as a single line?
You need to escape the double quotes. So " inside your json string becomes \".

How can I format a json file into a bash environment variable?

I'm trying to take the contents of a config file (JSON format), strip out extraneous new lines and spaces to be concise and then assign it to an environment variable before starting my application.
This is where I've got so far:
pwr_config=`echo "console.log(JSON.stringify(JSON.parse(require('fs').readFileSync(process.argv[2], 'utf-8'))));" | node - config.json | xargs -0 printf '%q\n'` npm run start
This pipes a short node.js app into the node runtime taking an argument of the file name and it parses and stringifies the JSON file to validate it and remove any unnecessary whitespace. So far so good.
The result of this is then piped to printf, or at least it would be but printf doesn't support input in this way, apparently, so I'm using xargs to pass it in in a way it supports.
I'm using the %q formatter to format the string escaping any characters that would be a problem as part of a command, but when calling printf through xargs, printf claims it doesn't support %q. I think this is perhaps because there is more than one version of printf but I'm not exactly sure how to resolve that.
Any help would be appreciated, even if the solution is completely different from what I've started :) Thanks!
Update
Here's the output I get on MacOS:
$ cat config.json | xargs -0 printf %q
printf: illegal format character q
My JSON file looks like this:
{
"hue_host": "192.168.1.2",
"hue_username": "myUsername",
"port": 12000,
"player_group_config": [
{
"name": "Family Room",
"player_uuid": "ATVUID",
"hue_group": "3",
"on_events": ["media.play", "media.resume"],
"off_events": ["media.stop", "media.pause"]
},
{
"name": "Lounge",
"player_uuid": "STVUID",
"hue_group": "1",
"on_events": ["media.play", "media.resume"],
"off_events": ["media.stop", "media.pause"]
}
]
}
Two ways:
Use xargs to pick up bash's printf builtin instead of the printf(1) executable, probably in /usr/bin/printf(thanks to #GordonDavisson):
pwr_config=`echo "console.log(JSON.stringify(JSON.parse(require('fs').readFileSync(process.argv[2], 'utf-8'))));" | node - config.json | xargs -0 bash -c 'printf "%q\n"'` npm run start
Simpler: you don't have to escape the output of a command if you quote it. In the same way that echo "<|>" is OK in bash, this should also work:
pwr_config="$(echo "console.log(JSON.stringify(JSON.parse(require('fs').readFileSync(process.argv[2], 'utf-8'))));" | node - config.json )" npm run start
This uses the newer $(...) form instead of `...`, and so the result of the command is a single word stored as-is into the pwr_config variable.*
Even simpler: if your npm run start script cares about the whitespace in your JSON, it's fundamentally broken :) . Just do:
pwr_config="$(< config.json)" npm run start
The $(<...) returns the contents of config.json. They are all stored as a single word ("") into pwr_config, newlines and all.* If something breaks, either config.json has an error and should be fixed, or the code you're running has an error and needs to be fixed.
* You actually don't need the "" around $(). E.g., foo=$(echo a b c) and foo="$(echo a b c)" have the same effect. However, I like to include the "" to remind myself that I am specifically asking for all the text to be kept together.

parsing json with shell script

Wanted to parse json:
{"FileStatus":"accessTime":1472892839430,"blockSize":134217728,"childrenNum":0,"fileId":17226,"group":"admin","length":115714,"modificationTime":1469649837471,"owner":"admin","pathSuffix":"","permission":"755","replication":2,"storagePolicy":0,"type":"FILE"}}
I tried something like this but not able to get it.
$ {"FileStatus":{"accessTime":1472892839430}} | jq '.FileStatus.accessTime'
Error:
-bash: {FileStatus:{accessTime:1472892839430}}: command not found`
Can someone help me to parse this whole json.
To make a command read a string on stdin in bash, use a "here string" like this:
$ jq '.FileStatus.accessTime' <<<'{"FileStatus":{"accessTime":1472892839430}}'
1472892839430
Also, you need to properly quote the text, so that bash doesn't try to interpret in some way you don't intend. When you want to preserve it literally, use single quotes (').

Shell: Replacing each New Line "\n" character with "\\n"

I'm inserting a git diff of changed files into a JSON object to send using a curl request.
The problem is it doesn't like the new-line characters being inserted into the JSON but I'm not sure how to get around that. Translate tool didn't work, this perl solution I'm using is close but just replaces with spaces:
changedfiles=$(git diff --name-only $3..$4 | perl -p -e 's/\n/ /')
and changing it to this didn't help:
changedfiles=$(git diff --name-only $3..$4 | perl -p -e 's/\n/\\n/')
Can anyone point me in the right direction? It doesn't need to use perl, it just needs to work
(...being simple would be nice too)
Instead of trying to do ad-hoc escaping for characters that your immediate testing finds problematic, how about using an actual JSON library that handles all of them in a solid way?
Here's an example in bash using inlined python:
python -c '
import json
import sys
print(json.dumps({"data": sys.argv[1]}))
' "$(git diff --name-only $3..$4)"
It prints the json object { "data": "your command output here" } with standards compliant escaping.
This is what I think you want to do to get a quoted list of files separated by commas (i.e. for inserting into a JSON string):
git diff --name-only $3..$4 | perl -p -e 's/(.*)/"$1",/;s/\n//;s/""/","/'
This works if your files don't contain double quotes or special characters that need to be JSON escaped.
First, we put the files in quotes followed by a comma, then remove newlines, then change the "" between files to ",". Although, this is kind of a hack. Somewhat better might be:
git diff --name-only $3..$4 | perl -p -e '$/="";s/(.*)\n/"$1",/g;s/,$//'
Here we read in the whole input, newlines and all, do our substitution and remove the final comma.