On this post MongoDB finding nested elements, author claims that the mongodb document structure is
car : { "$ref" : "cars" , "$id" : { "$oid" : "4e8478ace4b0dea06288ad63"}}
When i tried to reproduce the problem, i encountered some weird behaviors with mongodb insertion
When run the insertion on above sample data, i got following error
> db.sample.insert({car:{ "$ref" : "cars" , "$id" : { "$oid" : "4e8478ace4b0dea06288ad63"}}})
Tue Jan 24 14:09:07 uncaught exception: field names cannot start with $ [$oid]
it says that field names cannot start with $.
If thats the case, it should not work if i remove $ from oid and left the remaining $ref & $id untouched
> db.sample.insert({car:{ "$ref" : "cars" , "$id" : { "oid" : "4e8478ace4b0dea06288ad63"}}})
> db.sample.find()
{ "_id" : ObjectId("4f1e6fbc403aae757ec6dea5"), "car" : { "$ref" : "cars", "$id" : { "oid" : "4e8478ace4b0dea06288ad63" } } }
surprisingly it worked. Now it accepts the fields start with $
Also when i tried this query
> db.sample.insert({ "$ref" : "cars" })
document to insert can't have $ fields
i got the error back.
I don't understand what causes this? anybody have a clear idea?
$id and $ref are special identifiers used for dbrefs. Otherwise, field names starting with a $ aren't allowed.
However, your first-level document must not be a dbref itself, hence the error "document to insert can't have $ fields".
However, dbrefs are allowed as subdocuments, e.g. (from the official docs)
"name" : "Joe",
"classes" : [
{
"$ref" : "courses",
"$id" : ObjectId("4b0552b0f0da7d1eb6f126a1")
}
]
Now $oid is not a special identifier, and is not allowed because the $ has special semantics: Think of $inc. That is an operator, but if $ field names were allowed, it could also be the name of a field.
You have to be careful when using the positional operator in updates:
The positional operator cannot be combined with an upsert since it requires a matching array element. If your update results in an insert then the "$" will literally be used as the field name.
$ref $id and $db are the only valid keys which begin with a dollar sign - they are related to the DBRef convention.
The reason you can't use your own keys starting with a dollar sign is becuase it has Special meaning and would otherwise create ambiguity.
Related
My JSON (simplified) looks like this:
[
{"name" : "foobar",
"id" : 123
},
{"name" : "bar",
"id" : 123
},
{"name" : "foobar",
"id" : 456
}, ...
]
I'm using https://jsonpath.herokuapp.com/ to try and find the right JSONPATH syntax to filter out anything not starting with foo, and having id == 123.
Getting it to filter the ones that do start with foo is easy:
$..[?(#.name =~ /foo.*/i)]
This yields the following results:
[
{
"name" : "foobar",
"id" : 123
},
{
"name" : "foobar",
"id" : 456
}
]
I can get rid of the id 456 by adding an additional filter like so:
$..[?(#.name =~ /foo.*/i && #.id==123)]
But how do I do the opposite of getting the name starting with foo? I want all entities that do not start with foo.
I tried something like this:
$..[?(!#.name =~ /foo.*/i && #.id==123)]
Which at least parses as valid JSONPATH, and should negate the filter, but for some reason it still happily only reports the foobar entry:
[
{
"name" : "foobar",
"id" : 123
}
]
How can I achieve a NOT LIKE in JSONPATH?
Thanks!
Regex to identify data not starting with a given string foo:
^([^f]|f[^o]|fo[^o])
If your regex engine supports negative lookahead, that reduces to
^(?!foo)
Note the starting anchor (^) that limits the permissible matching location to the start of the test string.
Your attempt $..[?(!#.name =~ /foo.*/i && #.id==123)] is almost correct. Surround the regex condition with parenthesis before negating with ! like so $..[?(!(#.name =~ /foo.*/i) && #.id==123)]. Tested at https://jsonpath.herokuapp.com/
Edit: This was assuming that you were using Jayway's jsonpath (Java, https://github.com/json-path/JsonPath), but from the documentation link you provided for SmartBear, it looks like it uses the Goessner jsonpath (Javascript, https://goessner.net/articles/JsonPath/). Both, for whatever reason use slightly differing syntaxes.
Thanks to #collapsar for nudging me in the correct direction, in that the key to solving it was in the regular expression (but specifically using the JavaScript Regular Expression syntax, and merging that with the JSONPath syntax).
What actually ended up doing the trick was reading the documentation for JASONPath a bit more careful. It states:
=~
Match a JavaScript regular expression. For example, [?(#.description =~ /cat.*/i)] matches items whose description starts with cat (case-insensitive).
Note: Not supported at locations that use Ready! API 1.1.
The link to Javascript Regular Expression in turn contains the following:
[^xyz]
A negated or complemented character set. That is, it matches anything that is not enclosed in the brackets. You can specify a range of characters by using a hyphen. Everything that works in the normal character set also works here.
For example, [^abc] is the same as [^a-c]. They initially match 'r' in "brisket" and 'h' in "chop."
The resulting expression that works is:
$..[?(#.name =~ /[^foo].*/ && #.id == 123)]
Result:
[
{
"name" : "bar",
"id" : 123
}
]
(I added an additional bar with id 456 to the JSON payload I had, to double-check the filter also worked).
I have a data field that looks like this :
{ "field1" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
],
"field2" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
]
}
Is it possible to update a specific field with an update ?
create table t_json (
t_data json
);
insert into t_json values('{"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}]}');
select t_data->'field1'
from t_json;
I tried this :
update t_json
set t_data->'a' = '[{"value1" : true, "value2" : false}]';
But I get an error : "syntax error at or near ->
What is missing ?
I wanted to post this here in case it helps anybody else. By all means use JSON over JSONB unless you actually need features that JSONB affords you. In general, if you need to perform queries on the JSON data itself, use JSONB. If you are just needing to store data, use JSON.
Anyhow, here is how I am updating a JSON[] field:
UPDATE foo SET bar = ARRAY[$${"hello": "world"}$$, $${"baz": "bing"}$$]::JSON[]
The important things to notice are this:
The array is wrapped like this: ARRAY[ ... ]::JSON[]
Each item in the array is wrapped like this: $${ "foo": "bar" }$$
It is worth noting that this same technique can be used for other array types. For example, if you have a text[] column, the query would look like this:
UPDATE foo SET bar = ARRAY[$$hello world$$, $$baz bing$$]::TEXT[]`
Fixing your typos
Doubt it. This is not valid json. name1 and name2 must be double quoted. To ease working with json, ALWAYS use double quotes. ALWAYS query-quote with double-dollar.
{ "field1" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
],
"field2" : [{"name":'name1',"value1":true},
{"name":'name2',"value2":false}
]
}
And, what you INSERTED is also funky.. ALWAYS paste beautified valid JSON in your question.
{
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}]
}
Let's change that and fix it.
{
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],
"field2":[{"name":"name1","value" : true},{"name":"name2","value" : false}]
}
Now let's put it in a query..
TRUNCATE t_json;
INSERT INTO t_json (t_data) VALUES ($$
{
"field1":[{"name":"name1","value" : true},{"name":"name2","value" : false}],
"field2":[{"name":"name1","value" : true},{"name":"name2","value" : false}]
}
$$);
Making the update of the JSON
Now it works.. Now you can update it as you want..
UPDATE t_json
SET t_data = jsonb_set(
t_data::jsonb,
'{field1}',
$${"whatever":1}$$
);
Change from JSON to JSONB
Notice we're having to cast to jsonb. As a general rule, NEVER use JSON (not everyone agrees, see comments). There is no point. Instead use the newer JSONB.
ALTER TABLE t_json ALTER COLUMN t_data TYPE jsonb ;
Now you can do
UPDATE t_json
SET t_data = jsonb_set(
t_data,
'{field1}',
$${"whatever":1}$$
);
I have a rather complex structure on my json and I cannot find how to query it to get the rows I am interested in. Here is a sample of my data:
{
"_id" : ObjectId("5282bf9ce4b05216ca1b68f8"),
"authorID" : ObjectId("5282a8c3e4b0d7f4f4d07b9a"),
"blogID" : "7180831558698033600",
"blogs" : {
"$" : {
"posts" : [
[
{
"author" : {
"displayName" : "mms",
...
...
...
}}}
So, I am interested in finding all json entries that have the author displayName equal to "mms".
My collection name is bz so, a find all query would be: db.dz.find()
What criteria do I have to put inside the find() to only get json document with author displayName equal to mms?
Any ideas?
Thank you in advance!
Suppose you have replaced field name "$" with "dollarSign".
Then db.dz.find({"blogs.dollarSign.posts.author.displayName": "mms"}) will fetch whole documents according to your requirements.
Ahoy! I'm having a very funny issue with MongoDB and, possibly more in general, with JSON. Basically, I accidentally created some MongoDB documents whose subdocuments contain an empty key, e.g. (I stripped ObjectIDs to make the code look nicer):
{
"_id" : ObjectId("..."),
"stats" :
{
"violations" : 0,
"cost" : 170,
},
"parameters" :
{
"" : "../instances/comp/comp20.ectt",
"repetition" : 29,
"time" : 600000
},
"batch" : ObjectId("..."),
"system" : "Linux 3.5.0-27-generic",
"host" : "host3",
"date_started" : ISODate("2013-05-14T16:46:46.788Z"),
"date_stopped" : ISODate("2013-05-14T16:56:48.483Z"),
"copy" : false
}
Of course the problem is line:
"" : "../instances/comp/comp20.ectt"
since I cannot get back the value of the field. If I query using:
db.experiments.find({"batch": ObjectId("...")}, { "parameters.": 1 })
what I get is the full content of the parameters subdocument. My guess is that . is probably ignored if followed by an empty selector. From the JSON specification (15.12.*) it looks like empty keys are allowed. Do you have any ideas about how to solve that?
Is that a known behavior? Is there a use for that?
Update I tried to $rename the field, but that won't work, for the same reasons. Keys that end with . are not allowed.
Update filed issue on MongoDB issue tracker.
Thanks,
Tommaso
I have this same problem. You can select your sub-documents with something like this:
db.foo.find({"parameters.":{$exists:true}})
The dot at the end of "parameters" tells Mongo to look for an empty key in that sub-document. This works for me with Mongo 2.4.x.
Empty keys are not well supported by Mongo, I don't think they are officially supported, but you can insert data with them. So you shouldn't be using them and should find the place in your system where these keys are inserted and eliminate it.
I just checked the code and this does not currently seem possible for the reasons you mention. Since it is allowed to create documents with zero length field names I would consider this a bug. You can report it here : https://jira.mongodb.org
By the way, ironically you can query on it :
> db.c.save({a:{"":1}})
> db.c.save({a:{"":2}})
> db.c.find({"a.":1})
{ "_id" : ObjectId("519349da6bd8a34a4985520a"), "a" : { "" : 1 } }
In MongoDB, the update modifier unset works as follows:
Consider a Mongo DB Database db with a collection users. Users contain a Document, of the following format:
//Document for a user with username: joe
{
"_id" : ObjectId("4df5b9cf9f9a92b1584fff16"),
"relationships" : {
"enemies" : 2,
"friends" : 33,
"terminated" : "many"
},
"username" : "joe"
}
If I want to remove the terminated key, I have to specify the $unset update modifier as follows:
>db.users.update({"username":"joe"},{"$unset":{"relationships.terminated": "many"}});
My Question is, why do I have to specify the ENTIRE KEY VALUE PAIR for the $unset to work, instead of simply specifying:
>db.users.update({"username":"joe"},{"$unset":{"relationships.terminated"}});
Mon Jun 13 13:25:57 SyntaxError: missing : after property id (shell):1
Why not?
EDIT:
If the way to $unset is to specify the entire key value pair, in accordance with JSON specifications, or to add "1" as the value to the statement, why can't the Shell do the "1" substitution itself? Why isn't such a feature provided? Are there any pitfalls of providing such support?
The short answer is because {"relationships.terminated"} is not a valid json/bson object. A JSON object is composed of a key and a value, and {"relationships.terminated"} only has a key (or value, depends on how you look it).
Affortunately to unset a field in Mongo you do not need to set the actual value of the field you want to remove. You can use any value (1 is commonly used in Mongo docs) no matter the actual value of relationships.terminated:
db.users.update({"username":"joe"},{"$unset":{"relationships.terminated" : 1}});