I'm currently developing a bash script that makes a query to a server with curl, and returns a JSON object. I'm trying to parse that object with jsawk. For example, here is some data I'm trying to parse:
{
"account":{
"quota":20,
"email":"test#example.com",
"uuid":"12ae7a0cbd",
"email_verified":true
}
}
In terminal I'm running cat test.json | jsawk -q 'account.quota'.
Assume test.json is the above object. Every time I run that command, I always get the following error: jsawk: js error: ReferenceError: account is not defined, even though account is very clearly defined.
Try
<test.json jsawk 'return this.account.quota'
I removed your useless use of cat.
Related
I have a bash script which sends curl post requests. I want to pass data as bash script parameters. However one of the parameter has spaces in the string and it fails with the error below.
Error parsing JSON data.\n\tString not terminated on line
In shell script, I'm sending an argument like this format {"name":"'$2'"}
Could you please help me to solve that issue?
Thanks
jq is good not only for manipulating existing JSON data, but creating new data, as it does things like correctly handling characters that can't appear unescaped in JSON strings, and proper quoting. Something like
curl ... -d"$(jq -n --arg val "$2" '{name: $val}')"
It would be better if you add enough data while asking question.
I assume your json will be like
{
"name": "argument passed"
}
curl -XPOST "your/url/here" -H 'Content-Type:application/json' -d'{"name":"'$1'"}'
Save the above command as post_request.sh(Feel free to change the name).
Run using below comand.
sh post_request.sh "argument passed"
"argument passed" will be your name with space.
Im checking inside a openwrt with very few shell commands to see If is possible to filter json string to have the values.
For example
{"address":"192.168.2.2","user":"user1","groups":"permissions"}
I receive from curl the string and I need to separate values to pass vars to other commands.
For now Im checking some examples but not works
#!/bin/sh
. /usr/share/libubox/jshn.sh
json_init
json_load '$(cat $STRING)'
json_get_keys keys
for k in $keys; do
json_get_var v "$k"
echo "$k : $v"
done
But produce error "Failed to parse message data"
My problem is justly that I cna't use jq, or python to choose data, so only solution is to separate first.
Suggestions?
I found other form more clean to do the same
eval $(jsonfilter -s $STRING -e 'ADDRESS=#.address' -e 'USER=#.user')
echo "address=$ADDRESS user=$USER"
With this form I can filter every value how parameter, without jq or python function.
I have JSON files on my server that needs to be passed into several different Raspberry Pis running Debian. Each of the Pis has their own JSON feed that they will pull from, but essentially, I need to automatically take the value of one key-value pair and use it as an argument for a command that is run in the terminal.
For instance: Fetching https://www.example.com/api/THDCRUG2899CGF8&/manifest.json
{
"version": "1.5.6",
"update_at": "201609010000",
"body": "172.16.1.1"
}
Then that value would be dynamically output into a command that uses the body's value as an argument. EG: ping [body value]
Edit:
The point of this is to have a task that executes every minute to receive weather updates to it.
You are looking for command substition, specifically wrapped around a command that can extract values from a JSON value. First, you can use jq as the JSON-processing command.
$ jq -r '.body' tmp.json
172.16.1.1
Command substitution allows you to capture the output of jq to use as an argument:
$ ping "$(jq -r '.body' tmp.json)"
PING 172.16.1.1 (172.16.1.1): 56 data bytes
...
I want to consume the raw output of some MongoDB commands in other programs that speak JSON. When I run commands in the mongo shell, they represent Extended JSON, fields in "shell mode", with special fields like NumberLong , Date, and Timestamp. I see references in the documentation to "strict mode", but I see no way to turn it on for the shell, or a way to run commands like db.serverStatus() in things that do output strict JSON, like mongodump. How can I force Mongo to output standards-compliant JSON?
There are several other questions on this topic, but I don't find any of their answers particularly satisfactory.
The MongoDB shell speaks Javascript, so the answer is simple: use JSON.stringify(). If your command is db.serverStatus(), then you can simply do this:
JSON.stringify(db.serverStatus())
This won't output the proper "strict mode" representation of each of the fields ({ "floatApprox": <number> } instead of { "$numberLong": "<number>" }), but if what you care about is getting standards-compliant JSON out, this'll do the trick.
I have not found a way to do this in the mongo shell, but as a workaround, mongoexport can run queries and its output uses strict mode and can be piped into other commands that expect JSON input (such as json_pp or jq). For example, suppose you have the following mongo shell command to run a query, and you want to create a pipeline using that data:
db.myItemsCollection.find({creationDate: {$gte: ISODate("2016-09-29")}}).pretty()
Convert that mongo shell command into this shell command, piping for the sake of example to `json_pp:
mongoexport --jsonArray -d myDbName -c myItemsCollection -q '{"creationDate": {"$gte": {"$date": "2016-09-29T00:00Z"}}}' | json_pp
You will need to convert the query into strict mode format, and pass the database name and collection name as arguments, as well as quote properly for your shell, as shown here.
In case of findOne
JSON.stringify(db.Bill.findOne({'a': '123'}))
In case of a cursor
db.Bill.find({'a': '123'}).forEach(r=>print(JSON.stringify(r)))
or
print('[') + db.Bill.find().limit(2).forEach(r=>print(JSON.stringify(r) + ',')) + print(']')
will output
[{a:123},{a:234},]
the last one will have a ',' after the last item...remove it
To build on the answer from #jbyler, you can strip out the numberLongs using sed after you get your data - that is if you're using linux.
mongoexport --jsonArray -d dbName -c collection -q '{fieldName: {$regex: ".*turkey.*"}}' | sed -r 's/\{ "[$]numberLong" : "([0-9]+)" }/"\1"/g' | json_pp
EDIT: This will transform a given document, but will not work on a list of documents. Changed find to findOne.
Adding
.forEach(function(results){results._id=results._id.toString();printjson(results)})`
to a findOne() will output valid JSON.
Example:
db
.users
.findOne()
.forEach(function (results) {
results._id = results._id.toString();
printjson(results)
})
Source: https://www.mydbaworld.com/mongodb-shell-output-valid-json/
Trying to use a query with mongoexport results in an error. But the same query is evaluated by the mongo-client without an error.
In mongo-client:
db.listing.find({"created_at":new Date(1221029382*1000)})
with mongoexport:
mongoexport -d event -c listing -q '{"created_at":new Date(1221029382*1000)}'
The generated error:
Fri Nov 11 17:44:08 Assertion: 10340:Failure parsing JSON string near:
$and: [ {
0x584102 0x528454 0x5287ce 0xa94ad1 0xa8e2ed 0xa92282 0x7fbd056a61c4
0x4fca29
mongoexport(_ZN5mongo11msgassertedEiPKc+0x112) [0x584102]
mongoexport(_ZN5mongo8fromjsonEPKcPi+0x444) [0x528454]
mongoexport(_ZN5mongo8fromjsonERKSs+0xe) [0x5287ce]
mongoexport(_ZN6Export3runEv+0x7b1) [0xa94ad1]
mongoexport(_ZN5mongo4Tool4mainEiPPc+0x169d) [0xa8e2ed]
mongoexport(main+0x32) [0xa92282]
/lib/libc.so.6(__libc_start_main+0xf4) [0x7fbd056a61c4]
mongoexport(__gxx_personality_v0+0x3d9) [0x4fca29]
assertion: 10340 Failure parsing JSON string near: $and: [ {
But doing the multiplication in Date beforehand in mongoexport:
mongoexport -d event -c listing -q '{"created_at":new Date(1221029382000)}'
works!
Why is mongo evaluating the queries differently in these two contexts?
The mongoexport command-line utility supports passing a query in JSON format, but you are trying to evaluate JavaScript in your query.
The JSON format was originally derived from JavaScript's object notation, but the contents of a JSON document can be parsed without eval()ing it in a JavaScript interpreter.
You should consider JSON as representing "structured data" and JavaScript as "executable code". So there are, in fact, two different contexts for the queries you are running.
The mongo command-line utility is an interactive JavaScript shell which includes a JavaScript interpreter as well as some helper functions for working with MongoDB. While the JavaScript object format looks similar to JSON, you can also use JavaScript objects, function calls, and operators.
Your example of 1221029382*1000 is the result of a math operation that would be executed by the JavaScript interpreter if you ran that in the mongo shell; in JSON it's an invalid value for a new Date so mongoexport is exiting with a "Failure parsing JSON string" error.
I also got this error doing a mongoexport, but for a different reason. I'll share my solution here though since I ended up on this SO page while trying to solve my issue.
I know it has little to do with this question, but the title of this post brought it up in Google, so since I was getting the exact same error I'll add an answer. Hopefully it helps someone.
I was trying to do a MongoId _id query in the Windows console. The problem was that I needed to wrap the JSON query in double quotes, and the ObjectId also had to be in double quotes (not single!). So I had to escape the ObjectId quotes.
mongoexport -u USERNAME -pPASSWORD -d DATABASE -c COLLECTION
--query "{_id : ObjectId(\"5148894d98981be01e000011\")}"
If I wrap the JSON query in single quote on Windows, I get this error:
ERROR: too many positional options
And if I use single quotes around the ObjectId, I get this error:
Assertion: 10340:Failure parsing JSON string near: _id
So, yeah. Good luck.