Ahoy! I'm having a very funny issue with MongoDB and, possibly more in general, with JSON. Basically, I accidentally created some MongoDB documents whose subdocuments contain an empty key, e.g. (I stripped ObjectIDs to make the code look nicer):
{
"_id" : ObjectId("..."),
"stats" :
{
"violations" : 0,
"cost" : 170,
},
"parameters" :
{
"" : "../instances/comp/comp20.ectt",
"repetition" : 29,
"time" : 600000
},
"batch" : ObjectId("..."),
"system" : "Linux 3.5.0-27-generic",
"host" : "host3",
"date_started" : ISODate("2013-05-14T16:46:46.788Z"),
"date_stopped" : ISODate("2013-05-14T16:56:48.483Z"),
"copy" : false
}
Of course the problem is line:
"" : "../instances/comp/comp20.ectt"
since I cannot get back the value of the field. If I query using:
db.experiments.find({"batch": ObjectId("...")}, { "parameters.": 1 })
what I get is the full content of the parameters subdocument. My guess is that . is probably ignored if followed by an empty selector. From the JSON specification (15.12.*) it looks like empty keys are allowed. Do you have any ideas about how to solve that?
Is that a known behavior? Is there a use for that?
Update I tried to $rename the field, but that won't work, for the same reasons. Keys that end with . are not allowed.
Update filed issue on MongoDB issue tracker.
Thanks,
Tommaso
I have this same problem. You can select your sub-documents with something like this:
db.foo.find({"parameters.":{$exists:true}})
The dot at the end of "parameters" tells Mongo to look for an empty key in that sub-document. This works for me with Mongo 2.4.x.
Empty keys are not well supported by Mongo, I don't think they are officially supported, but you can insert data with them. So you shouldn't be using them and should find the place in your system where these keys are inserted and eliminate it.
I just checked the code and this does not currently seem possible for the reasons you mention. Since it is allowed to create documents with zero length field names I would consider this a bug. You can report it here : https://jira.mongodb.org
By the way, ironically you can query on it :
> db.c.save({a:{"":1}})
> db.c.save({a:{"":2}})
> db.c.find({"a.":1})
{ "_id" : ObjectId("519349da6bd8a34a4985520a"), "a" : { "" : 1 } }
Related
I'm having a problem with querying a MongoDB dataset ("On Street Crime in Camden" from data.gov.uk)
The database name is Crime_Data_in_Camden and the collection name is Street_Crime_Camden. The query to find all records, db.Street_Crime_Camden.find(), works fine but anything else returns nothing at
all. Here is a portion of the metadata:
{
"id" : 509935,
"name" : "Ward Name",
"dataTypeName" : "text",
"fieldName" : "ward_name",
"position" : 13,
"renderTypeName" : "text",
"tableColumnId" : 258836,
"width" : 100,
"cachedContents" : {
"largest" : "West Hampstead",
"non_null" : 79813,
"null" : 0,
"top" : [ {
"item" : "Regent's Park",
"count" : 20
}, {
"item" : "Swiss Cottage",
"count" : 19
}, {
"item" : "Holborn and Covent Garden",
"count" : 18
}
}
}
I've tried 3 attempts at a basic query:
db.Street_Crime_Camden.find({"ward_name":"West Hampstead"});
db.Street_Crime_Camden.find({'meta.ward_name':'West Hampstead'});
db.Street_Crime_Camden.find({meta:{ward_name:"West Hampstead"} });
According to any documentation or tutorial that I've seen any of these approaches should be valid. And I know that there are hundreds of rows (or documents) that match those terms, so why are these queries returning nothing? Advice would be appreciated.
The common theme in the three aproaches you tried is some form of ward_name = West Hampstead but there is no attribute named ward_name in the document you shared with us.
Based on the document you show in your question the only way of addressing an attribute with the value West Hampstead is:
db.Street_Crime_Camden.find({"cachedContents.largest": "West Hampstead"});
For background; you address attributes in your documents by using dot notation so the document you included in your question could be found by any of the following find commands:
db.Street_Crime_Camden.find({"name": "Ward Name"});
db.Street_Crime_Camden.find({"position": 13});
db.Street_Crime_Camden.find({"cachedContents.top.item": "Swiss Cottage"});
db.Street_Crime_Camden.find({"cachedContents.top.1.count": 20});
... etc
These examples might help you to understand how to form find criteria. The MongoDB docs are also useful.
I have a data field that looks like this :
{ "field1" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
],
"field2" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
]
}
Is it possible to update a specific field with an update ?
create table t_json (
t_data json
);
insert into t_json values('{"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}]}');
select t_data->'field1'
from t_json;
I tried this :
update t_json
set t_data->'a' = '[{"value1" : true, "value2" : false}]';
But I get an error : "syntax error at or near ->
What is missing ?
I wanted to post this here in case it helps anybody else. By all means use JSON over JSONB unless you actually need features that JSONB affords you. In general, if you need to perform queries on the JSON data itself, use JSONB. If you are just needing to store data, use JSON.
Anyhow, here is how I am updating a JSON[] field:
UPDATE foo SET bar = ARRAY[$${"hello": "world"}$$, $${"baz": "bing"}$$]::JSON[]
The important things to notice are this:
The array is wrapped like this: ARRAY[ ... ]::JSON[]
Each item in the array is wrapped like this: $${ "foo": "bar" }$$
It is worth noting that this same technique can be used for other array types. For example, if you have a text[] column, the query would look like this:
UPDATE foo SET bar = ARRAY[$$hello world$$, $$baz bing$$]::TEXT[]`
Fixing your typos
Doubt it. This is not valid json. name1 and name2 must be double quoted. To ease working with json, ALWAYS use double quotes. ALWAYS query-quote with double-dollar.
{ "field1" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
],
"field2" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
]
}
And, what you INSERTED is also funky.. ALWAYS paste beautified valid JSON in your question.
{
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}]
}
Let's change that and fix it.
{
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],
"field2":[{"name":"name1","value" : true},{"name":"name2","value" : false}]
}
Now let's put it in a query..
TRUNCATE t_json;
INSERT INTO t_json (t_data) VALUES ($$
{
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],
"field2":[{"name":"name1","value" : true},{"name":"name2","value" : false}]
}
$$);
Making the update of the JSON
Now it works.. Now you can update it as you want..
UPDATE t_json
SET t_data = jsonb_set(
t_data::jsonb,
'{field1}',
$${"whatever":1}$$
);
Change from JSON to JSONB
Notice we're having to cast to jsonb. As a general rule, NEVER use JSON (not everyone agrees, see comments). There is no point. Instead use the newer JSONB.
ALTER TABLE t_json ALTER COLUMN t_data TYPE jsonb ;
Now you can do
UPDATE t_json
SET t_data = jsonb_set(
t_data,
'{field1}',
$${"whatever":1}$$
);
db.blog.save({ title : "My First Post", author: {name : "Jane", id : 1}})
what should below return as the key order does not match?
db.blog.find({"author" : {"id" : 1, "name" : "Jane"}})
EDIT:
based on official mongodb documentation , the keyorder must match (at least for findOne()). It wont return the match-only object using db.blog.findOne({"author" : {"id" : 1, "name" : "Jane"}})
The order of the keys in your query selector is irrelevant. It doesn't need to match the order of the keys you used when adding the document you're searching for.
UPDATE
If you're just looking for an order-independent way to query based on an embedded document, you need to use dot notation:
db.blog.find({"author.id" : 1, "author.name" : "Jane"})
Normally, as #JohnnyHK states the order of the query keys does not matter except for the example you have shown:
db.blog.find({"author" : {"id" : 1, "name" : "Jane"}})
This query will not return results that do not match exactly. Using the query he shows of:
db.blog.find({"author.id" : 1, "author.name" : "Jane"})
Will be key order independent. The reasons for this difference is because in the first query you are searching by an object as such the querier actually searches for exactly that object (in the simplest terms). The same applies for indexes created on the field which contains a set of sub documents, the order does matter.
According to the JSON definition, the key order doesn't matter.
An object is an unordered collection of zero or more name/value pairs
I don't know anything about MongoDB, but I assume it follows the normal rules of JSON, at which point it should return the "My First Post" entry.
On this post MongoDB finding nested elements, author claims that the mongodb document structure is
car : { "$ref" : "cars" , "$id" : { "$oid" : "4e8478ace4b0dea06288ad63"}}
When i tried to reproduce the problem, i encountered some weird behaviors with mongodb insertion
When run the insertion on above sample data, i got following error
> db.sample.insert({car:{ "$ref" : "cars" , "$id" : { "$oid" : "4e8478ace4b0dea06288ad63"}}})
Tue Jan 24 14:09:07 uncaught exception: field names cannot start with $ [$oid]
it says that field names cannot start with $.
If thats the case, it should not work if i remove $ from oid and left the remaining $ref & $id untouched
> db.sample.insert({car:{ "$ref" : "cars" , "$id" : { "oid" : "4e8478ace4b0dea06288ad63"}}})
> db.sample.find()
{ "_id" : ObjectId("4f1e6fbc403aae757ec6dea5"), "car" : { "$ref" : "cars", "$id" : { "oid" : "4e8478ace4b0dea06288ad63" } } }
surprisingly it worked. Now it accepts the fields start with $
Also when i tried this query
> db.sample.insert({ "$ref" : "cars" })
document to insert can't have $ fields
i got the error back.
I don't understand what causes this? anybody have a clear idea?
$id and $ref are special identifiers used for dbrefs. Otherwise, field names starting with a $ aren't allowed.
However, your first-level document must not be a dbref itself, hence the error "document to insert can't have $ fields".
However, dbrefs are allowed as subdocuments, e.g. (from the official docs)
"name" : "Joe",
"classes" : [
{
"$ref" : "courses",
"$id" : ObjectId("4b0552b0f0da7d1eb6f126a1")
}
]
Now $oid is not a special identifier, and is not allowed because the $ has special semantics: Think of $inc. That is an operator, but if $ field names were allowed, it could also be the name of a field.
You have to be careful when using the positional operator in updates:
The positional operator cannot be combined with an upsert since it requires a matching array element. If your update results in an insert then the "$" will literally be used as the field name.
$ref $id and $db are the only valid keys which begin with a dollar sign - they are related to the DBRef convention.
The reason you can't use your own keys starting with a dollar sign is becuase it has Special meaning and would otherwise create ambiguity.
In MongoDB, the update modifier unset works as follows:
Consider a Mongo DB Database db with a collection users. Users contain a Document, of the following format:
//Document for a user with username: joe
{
"_id" : ObjectId("4df5b9cf9f9a92b1584fff16"),
"relationships" : {
"enemies" : 2,
"friends" : 33,
"terminated" : "many"
},
"username" : "joe"
}
If I want to remove the terminated key, I have to specify the $unset update modifier as follows:
>db.users.update({"username":"joe"},{"$unset":{"relationships.terminated": "many"}});
My Question is, why do I have to specify the ENTIRE KEY VALUE PAIR for the $unset to work, instead of simply specifying:
>db.users.update({"username":"joe"},{"$unset":{"relationships.terminated"}});
Mon Jun 13 13:25:57 SyntaxError: missing : after property id (shell):1
Why not?
EDIT:
If the way to $unset is to specify the entire key value pair, in accordance with JSON specifications, or to add "1" as the value to the statement, why can't the Shell do the "1" substitution itself? Why isn't such a feature provided? Are there any pitfalls of providing such support?
The short answer is because {"relationships.terminated"} is not a valid json/bson object. A JSON object is composed of a key and a value, and {"relationships.terminated"} only has a key (or value, depends on how you look it).
Affortunately to unset a field in Mongo you do not need to set the actual value of the field you want to remove. You can use any value (1 is commonly used in Mongo docs) no matter the actual value of relationships.terminated:
db.users.update({"username":"joe"},{"$unset":{"relationships.terminated" : 1}});