I want to export collection in mongodb using shell command:
I try the following command but fields with ":" (number:phone, number:fax) are not exported.
mongoexport --csv -d schemaName -c collectionName -q "{typeName:'user'}" -f "name, surname, e-mail, number:phone, number:fax" -o export.csv
I think that you have found a legitimate bug. The mongoexport tool is rarely used and the colon means something very specific when parsing JSON, so the tool is probably confused.
You can file the bug here: http://jira.mongodb.org/
Related
I am new to shell scripting and MQTT.
I need to publish a JSON file using MQTT. We can do it by storing the JSON contents in a shell variable. But it is not working for me.
my shell script:
#!/bin/sh
var1='{"apiVersion":"2.1","data":{"id":"4TSJhIZmL0A","uploaded":"2008-07-15T18:11:59.000Z","updated":"2013-05-01T21:01:49.000Z","uploader":"burloandbardsey","category":"News","title":"bbc news start up theme","description":"bbc","thumbnail":{"sqDefault":"http://i.ytimg.com/vi/4TSJhIZmL0A/default.jpg","hqDefault":"http://i.ytimg.com/vi/4TSJhIZmL0A/hqdefault.jpg"},"player":{"default":"http://www.youtube.com/watch?v=4TSJhIZmL0A&feature=youtube_gdata_player","mobile":"http://m.youtube.com/details?v=4TSJhIZmL0A"},"content":{"5":"http://www.youtube.com/v/4TSJhIZmL0A?version=3&f=videos&app=youtube_gdata","1":"rtsp://v5.cache7.c.youtube.com/CiILENy73wIaGQlAL2aGhIk04RMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp","6":"rtsp://v5.cache7.c.youtube.com/CiILENy73wIaGQlAL2aGhIk04RMYESARFEgGUgZ2aWRlb3MM/0/0/0/video.3gp"},"duration":15,"aspectRatio":"widescreen","rating":4.6683936,"likeCount":"354","ratingCount":386,"viewCount":341066,"favoriteCount":0,"commentCount":155,"accessControl":{"comment":"allowed","commentVote":"allowed","videoRespond":"allowed","rate":"allowed","embed":"allowed","list":"allowed","autoPlay":"allowed","syndicate":"allowed"}}}'
mosquitto_pub -h localhost -t test -m "$var1"
echo "$var1"
my Mosquitto commands:
Publisher: `mosquitto_pub -h localhost -t "test" -m "{"Contents":$var1}"
Subscriber: mosquitto_sub -h localhost -t "test"
Output I got:
{"Contents":}
Expected Output:
{"Contents":{"name":"Harini", "age":24, "city":"NewYork", "message":"Hello world"}}
I can get the output only at the terminal because of echo. But I want to publish and subscribe to the contents of the shell variable(var1)
Please help me out to get the output. Whether I need to add some more code in the shell script. I don't know how to proceed. Or can you suggest any other method.
The following works just fine, it's all about which quotes you use where:
#!/bin/sh
var1='{"name":"Harini", "age":24, "city":"NewYork","message":"Hello world"}'
echo $var1
mosquitto_pub -t test -m "{\"Content\": $var1}"
You need to wrap the -m argument in quotes because it contains spaces, which in turn means you need to escape the " round Content.
Wrapping the content of var1 in single quotes means you don't need to escape the double quotes in it.
i want to count how many times tag1 occurs
givin this 123.xml file ( streaming from the internet)
<startend>
<tag1 name=myname>
<date>10-10-10</date>
</tag1 >
<tag1 name=yourname>
<date>11-10-10</date>
</tag1 >
</startend>
using : xmlstarlet sel -t -v "count(//tag1)" 123.xml
output :
AttValue: " or ' expected
attributes construct error
how to ignore that the attribute has no " " ?
You input XML/HTML structure has invalid tags/attributes and should be recovered beforehand:
xmlstarlet solution:
xmlstarlet fo -o -R -H -D 123.xml 2>/dev/null | xmlstarlet sel -t -v "count(//tag1)" -n
The output:
2
Details:
fo (or format) - Format XML document(s)
-o or --omit-decl - omit xml declaration
-R or --recover - try to recover what is parsable
-D or --dropdtd - remove the DOCTYPE of the input docs
-H or --html - input is HTML
2>/dev/null - suppress errors/warnings
XML always requires quotes around attribute values. If you want to keep using XML, you first must produce valid XML from your input. You could use an SGML processor such as OpenSP (in particular, the osx program) to format your input into wellformed XML. It's as simple as invoking osx <your Input file> on it.
If you're on Ubuntu/Debian Linux, you can install osx by invoking sudo apt-get install opensp on the command line (and similarly on other Unix systems).
I'm trying to export just one object with mongoexport, filtering by its ID.
I tried:
mongoexport -d "kb_development" -c "articles" -q "{'_id': '4e3ca3bc38c4f10adf000002'}"
and many variations, but it keeps saying
connected to: 127.0.0.1
exported 0 records
(and I'm sure there is such an object in the collection)
In mongo shell I would use ObjectId('4e3ca3bc38c4f10adf000002'), but it does not seem to work in the mongoexport query.
I think you should be able to use ObjectId(...) in the query argument to mongoexport:
mongoexport -d kb_development -c articles -q '{_id: ObjectId("4e3ca3bc38c4f10adf000002")}'
If that does not work, you can use the "strict mode" javascript notation of ObjectIds, as documented here:
mongoexport -d kb_development -c articles -q '{_id: {"$oid": "4e3ca3bc38c4f10adf000002"}}'
(Also note that strict mode JSON is the format produced by mongoexport)
You have to specify the _id field by using the ObjectId type. In your question it was specified as a string.
CODE ::
mongoexport -h localhost -d my_database -c sample_collection -q '{key:ObjectId("50584580ff0f089602000155")}' -o my_output_file.json
NOTE :: dont forgot quotes in query
My MongoDB verion: 3.2.4. when I use mongoexport tool in mongo shell：
NOT WORK:
-q '{"_id":ObjectId("5719cd12b1168b9d45136295")}'
-q '{_id: {"$oid": "5719cd12b1168b9d45136295"}}'
WORKs:
-q "{_id:ObjectId('5719cd12b1168b9d45136295')}"
- Though in mongo doc , it says that
You must enclose the query in single quotes (e.g. ') to ensure that it
does not interact with your shell environment.
- But, single quote(') does not work! please use double quote(")!
for mongoexport version: r4.2.3
mongoexport -q '{"_id": {"$oid": "4e3ca3bc38c4f10adf000002"}}'
and for a nested field
mongoexport -q '{"_id": {"$oid": "4e3ca3bc38c4f10adf000002"}}' --fields parentField.childField
You do not have to add ObjectId or $oid as suggested by answers above. As has been mentioned by #Blacksad, just get your single and double quotes right.
mongoexport -d kb_development -c articles -q '{_id:"4e3ca3bc38c4f10adf000002"}'
many of the answers provided here didn't work for me, the error was with my double quotes. Here is what worked for me:
mongoexport -h localhost -d database_name -c collection_name -q {_id:ObjectId('50584580ff0f089602066633')} -o output_file.json
remember to use single quote only for the ObjectId string.
I have just dumped a riak db (back-up). But the backup file is a binary file.
Is there a lib that it deserialize it into a human readable file (JSON w/e) ?
I haven't found anything on google, neither on Stack Overflow.
Found a solution for my current problem:
Connect to the env and then run following command:
wget https://s3-us-west-2.amazonaws.com/ps-tools/riak-data-migrator-0.2.9-bin.tar.gz
tar -xvzf riak-data-migrator-0.2.9-bin.tar.gz
cd riak-data-migrator-0.2.9
java -jar riak-data-migrator-0.2.9.jar -d -r /var/riak_export -a -h 127.0.0.1 -p 8087 -H 8098
(source: https://github.com/basho-labs/riak-data-migrator)
EDIT
Another way to export riak db https://www.npmjs.com/package/riak-bucket-exporter
#!/bin/bash
for bucket in $(curl http://localhost:8098/riak?buckets=true | sed -e 's/[{}:"]//gi' -e 's/buckets\[//' -e 's/\]//' -e 's/,/ /g')
do
echo "Exporting bucket $bucket"
rm -f $bucket.json
riak-bucket-exporter -H localhost -p 8098 $bucket
done
echo "Export done"
As all the suggestions listed so far appear to be broken in one way or another (at least for me and riak-kv#2.x), I ultimately resorted to homegrow a bash shell script that leverages riak-kv's HTTP API with no other prerequisites than curl and jq to accomplish an export of sorts.
It can be found in this gist here: https://gist.github.com/cueedee/0b26ec746c4ef578cd98e93c93d2b6e8 hoping that someone will find it useful.
I'm trying to import a json file into mongo. When I import the file with this line, it works:
mongoimport -d reps_development -c users --jsonArray --file ~/reps/scripts/mockUserData.json
The script uses an environment variables $REPS_ROOT, which is set in my .bash_profile. This line fails:
mongoimport -d reps_development -c users --jsonArray --file $REPS_ROOT/scripts/mockUserData.json
I set $REPS_ROOT with the following command:
export REPS_ROOT="~/reps"
Any thoughts on why this isn't working? The error I get is:
file doesn't exist: ~/reps/scripts/mockUserData.json
Bash expands $REPS_ROOT to ~/reps, which becomes the value. Bash won't expand that value again. If the value of that variable contains a relative path, you need to make sure it is expanded. For example:
mongoimport -d reps_development -c users --jsonArray --file $(cd $REPS_ROOT; pwd)/scripts/mockUserData.json
If you are on Linux, you can use $(readlink -f $REPS_ROOT) instead. Alternatively you can use $HOME instead of ~:
export REPS_ROOT="$HOME/reps"