JSONPath regular expression - NOT starting with - json

My JSON (simplified) looks like this:
[
{"name" : "foobar",
"id" : 123
},
{"name" : "bar",
"id" : 123
},
{"name" : "foobar",
"id" : 456
}, ...
]
I'm using https://jsonpath.herokuapp.com/ to try and find the right JSONPATH syntax to filter out anything not starting with foo, and having id == 123.
Getting it to filter the ones that do start with foo is easy:
$..[?(#.name =~ /foo.*/i)]
This yields the following results:
[
{
"name" : "foobar",
"id" : 123
},
{
"name" : "foobar",
"id" : 456
}
]
I can get rid of the id 456 by adding an additional filter like so:
$..[?(#.name =~ /foo.*/i && #.id==123)]
But how do I do the opposite of getting the name starting with foo? I want all entities that do not start with foo.
I tried something like this:
$..[?(!#.name =~ /foo.*/i && #.id==123)]
Which at least parses as valid JSONPATH, and should negate the filter, but for some reason it still happily only reports the foobar entry:
[
{
"name" : "foobar",
"id" : 123
}
]
How can I achieve a NOT LIKE in JSONPATH?
Thanks!

Regex to identify data not starting with a given string foo:
^([^f]|f[^o]|fo[^o])
If your regex engine supports negative lookahead, that reduces to
^(?!foo)
Note the starting anchor (^) that limits the permissible matching location to the start of the test string.

Your attempt $..[?(!#.name =~ /foo.*/i && #.id==123)] is almost correct. Surround the regex condition with parenthesis before negating with ! like so $..[?(!(#.name =~ /foo.*/i) && #.id==123)]. Tested at https://jsonpath.herokuapp.com/
Edit: This was assuming that you were using Jayway's jsonpath (Java, https://github.com/json-path/JsonPath), but from the documentation link you provided for SmartBear, it looks like it uses the Goessner jsonpath (Javascript, https://goessner.net/articles/JsonPath/). Both, for whatever reason use slightly differing syntaxes.

Thanks to #collapsar for nudging me in the correct direction, in that the key to solving it was in the regular expression (but specifically using the JavaScript Regular Expression syntax, and merging that with the JSONPath syntax).
What actually ended up doing the trick was reading the documentation for JASONPath a bit more careful. It states:
=~
Match a JavaScript regular expression. For example, [?(#.description =~ /cat.*/i)] matches items whose description starts with cat (case-insensitive).
Note: Not supported at locations that use Ready! API 1.1.
The link to Javascript Regular Expression in turn contains the following:
[^xyz]
A negated or complemented character set. That is, it matches anything that is not enclosed in the brackets. You can specify a range of characters by using a hyphen. Everything that works in the normal character set also works here.
For example, [^abc] is the same as [^a-c]. They initially match 'r' in "brisket" and 'h' in "chop."
The resulting expression that works is:
$..[?(#.name =~ /[^foo].*/ && #.id == 123)]
Result:
[
{
"name" : "bar",
"id" : 123
}
]
(I added an additional bar with id 456 to the JSON payload I had, to double-check the filter also worked).

Related

How can I count objects in jq (not in an array)

I have some json input that comes as a list of objects (not as a json array) like, for which I want to count the number of active, and the number of inactive:
{
"key" : "state",
"value" : "active"
}
{
"key" : "state",
"value" : "active"
}
{
"key" : "state",
"value" : "active"
}
{
"key" : "state",
"value" : "inactive"
}
I want to transform using only JQ (in fact it will be handled by jackson-jq in java code, so I can not use some shell tricks).
I tried many things, like select(.value == "active") | length to get the number of active, but it is always handle object per object.
Even when using reduce, it is always handled object per object.
Example of output is :
2
2
2
(I have 3 objects with 2 fields that match value="active)
The output that I expect is :
3
Here is a playground with my example.
To avoid "slurping" the input, you would use either reduce or foreach. Here's a suitable abstraction built from reduce:
def count(s; f; g):
reduce s as $s ([0,0];
if $s|f then .[0]+=1 else . end
| if $s|g then .[1]+=1 else . end);
With this, you would invoke jq with the -n command-line option and run a query such as:
count(inputs|.value; .=="active"; .=="inactive")
Notice that one cannot use count/1 (defined analogously) twice since inputs consumes the input stream.

MongoDB regex string startswith and endswith [duplicate]

This question already has answers here:
Regex matching beginning AND end strings
(6 answers)
Regular Expression to find string starts with letter and ends with slash /
(3 answers)
Closed 4 years ago.
So i have something that looks like this
db.usuarios.insert
(
[
{
"nome" : "neymala",
"idade" : 40,
"status" : "solteira"
},
{
"nome" : "gabriel",
"idade" : 31,
"status" : "casado"
},
{
"nome" : "jose",
"idade" : 25,
"status" : "solteiro"
},
{
"nome" : "manoel",
"idade" : 25,
"status" : "solteiro",
"interesses" : [
"esporte",
"musica"
]
}
]
)
I would like to find names that starts with ma and ends with l, for example "manoel" or "manuel"
I have figured out how to do one or the other with the fallowing querys:
db.usuarios.find({nome:{$regex: /^ma/ }})
db.usuarios.find({nome:{$regex: /l$/ }})
Now i would like to combine them into a single query.
You can combine the two requirements into a single regex:
db.usuarios.find({nome: /^ma.*l$/})
In a regex, .* means to match 0 or more of any character. So this regex matches names that start with ma and end with l, ignoring whatever is between.
combine both querys with a AND opetator
db.usuarios.find({
$and:[
{nome:{$regex: /^ma/ }},
{nome:{$regex: /l$/ }}
]
})
In javascript /.../ is a regex, so no $regex needed.
db.usuarios.find({
$and:[
{nome: /^ma/ },
{nome: /l$/ }
]
})
Also note starts with can hit an index. ends with can't. This way you'd better have a selective starts with, otherwise there will be lots of object scans which may occupy extra CPU and possibly slow down your query.
May be this code help you.
you can search small letter string and capital letter string.......

How to Convert a list of tuples into a Json string

I have a Erlang list of tuples as follows:
[ {{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]} ,
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]} ]
I wanted this list of tuples in this form:
<<" [ {{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]} ,
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]}] ">>
So I tried using JSON parsing libraries in erlang (both jiffy and jsx )
Here is what I did:
A=[ {{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]} ,
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]} ],
B=erlang:iolist_to_binary(io_lib:write(A)),
jsx:encode(B).
and I get the following output(here I have changed the list to binary since jsx accepts binary):
<<"[{{[97]},[2],[{3,[98]},{4,[99]}],[5,[100]],[1,1],{e},[[102]]},{{[103]},
[3],[{6,[104]},{7,[105]}],[{8,[106]}],[1,1,1],{k},[[76]]}]">>
jiffy:encode(B) also gives the same output.
Can anyone help me to get the output as :
<<" [ {{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]} ,
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]}] ">>
instead of
<<"[{{[97]},[2],[{3,[98]},{4,[99]}],[5,[100]],[1,1],{e},[[102]]},{{[103]},
[3],[{6,[104]},{7,[105]}],[{8,[106]}],[1,1,1],{k},[[76]]}]">>
Thank you in advance
Instead of io_lib:write(A), use io_lib:format("~p", [A]). It tries to guess which lists are actually meant to be strings. (In Erlang, strings are actually lists of integers. Try it: "A" == [65])
> A=[ {{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]} ,
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]} ].
[{{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]},
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]}]
> B = erlang:iolist_to_binary(io_lib:format("~p", [A])).
<<"[{{\"a\"},[2],[{3,\"b\"},{4,\"c\"}],[5,\"d\"],[1,1],{e},[\"f\"]},\n {{\"g\"},[3],[{6,\"h\"},{7,\"i\"}],[{8,\"j\"}],[1,1,1],{k},[\"L\"]}]">>
If you don't want to see the backslashes before the double quotes, you can print the string to standard output:
> io:format("~s\n", [B]).
[{{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]},
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]}]
<<" [ {{"a"},[2],[{3,"b"},{4,"c"}],[5,"d"],[1,1],{e},["f"]} ,
{{"g"},[3],[{6,"h"},{7,"i"}],[{8,"j"}],[1,1,1],{k},["L"]}] ">>
This ^^ isn't a valid erlang term, but I think what you're getting at is that you want the "listy" strings, like "a" to be printed out like "a" instead of [97]. Unfortunately, I've found this to be a serious shortcoming of Erlang. The problem is that the string literal "a" is only syntactic sugar and is identical to the term [97], so any time you output it, you're subject to the vagaries of "is this thing a string or a list of integers?" The best way I know to get out of that is to use binaries as your strings wherever possible, like <<"a">> instead of "a".

MongoDB: how to select an empty-key subdocument?

Ahoy! I'm having a very funny issue with MongoDB and, possibly more in general, with JSON. Basically, I accidentally created some MongoDB documents whose subdocuments contain an empty key, e.g. (I stripped ObjectIDs to make the code look nicer):
{
"_id" : ObjectId("..."),
"stats" :
{
"violations" : 0,
"cost" : 170,
},
"parameters" :
{
"" : "../instances/comp/comp20.ectt",
"repetition" : 29,
"time" : 600000
},
"batch" : ObjectId("..."),
"system" : "Linux 3.5.0-27-generic",
"host" : "host3",
"date_started" : ISODate("2013-05-14T16:46:46.788Z"),
"date_stopped" : ISODate("2013-05-14T16:56:48.483Z"),
"copy" : false
}
Of course the problem is line:
"" : "../instances/comp/comp20.ectt"
since I cannot get back the value of the field. If I query using:
db.experiments.find({"batch": ObjectId("...")}, { "parameters.": 1 })
what I get is the full content of the parameters subdocument. My guess is that . is probably ignored if followed by an empty selector. From the JSON specification (15.12.*) it looks like empty keys are allowed. Do you have any ideas about how to solve that?
Is that a known behavior? Is there a use for that?
Update I tried to $rename the field, but that won't work, for the same reasons. Keys that end with . are not allowed.
Update filed issue on MongoDB issue tracker.
Thanks,
Tommaso
I have this same problem. You can select your sub-documents with something like this:
db.foo.find({"parameters.":{$exists:true}})
The dot at the end of "parameters" tells Mongo to look for an empty key in that sub-document. This works for me with Mongo 2.4.x.
Empty keys are not well supported by Mongo, I don't think they are officially supported, but you can insert data with them. So you shouldn't be using them and should find the place in your system where these keys are inserted and eliminate it.
I just checked the code and this does not currently seem possible for the reasons you mention. Since it is allowed to create documents with zero length field names I would consider this a bug. You can report it here : https://jira.mongodb.org
By the way, ironically you can query on it :
> db.c.save({a:{"":1}})
> db.c.save({a:{"":2}})
> db.c.find({"a.":1})
{ "_id" : ObjectId("519349da6bd8a34a4985520a"), "a" : { "" : 1 } }

Weird behavior with mongodb fields start with $

On this post MongoDB finding nested elements, author claims that the mongodb document structure is
car : { "$ref" : "cars" , "$id" : { "$oid" : "4e8478ace4b0dea06288ad63"}}
When i tried to reproduce the problem, i encountered some weird behaviors with mongodb insertion
When run the insertion on above sample data, i got following error
> db.sample.insert({car:{ "$ref" : "cars" , "$id" : { "$oid" : "4e8478ace4b0dea06288ad63"}}})
Tue Jan 24 14:09:07 uncaught exception: field names cannot start with $ [$oid]
it says that field names cannot start with $.
If thats the case, it should not work if i remove $ from oid and left the remaining $ref & $id untouched
> db.sample.insert({car:{ "$ref" : "cars" , "$id" : { "oid" : "4e8478ace4b0dea06288ad63"}}})
> db.sample.find()
{ "_id" : ObjectId("4f1e6fbc403aae757ec6dea5"), "car" : { "$ref" : "cars", "$id" : { "oid" : "4e8478ace4b0dea06288ad63" } } }
surprisingly it worked. Now it accepts the fields start with $
Also when i tried this query
> db.sample.insert({ "$ref" : "cars" })
document to insert can't have $ fields
i got the error back.
I don't understand what causes this? anybody have a clear idea?
$id and $ref are special identifiers used for dbrefs. Otherwise, field names starting with a $ aren't allowed.
However, your first-level document must not be a dbref itself, hence the error "document to insert can't have $ fields".
However, dbrefs are allowed as subdocuments, e.g. (from the official docs)
"name" : "Joe",
"classes" : [
{
"$ref" : "courses",
"$id" : ObjectId("4b0552b0f0da7d1eb6f126a1")
}
]
Now $oid is not a special identifier, and is not allowed because the $ has special semantics: Think of $inc. That is an operator, but if $ field names were allowed, it could also be the name of a field.
You have to be careful when using the positional operator in updates:
The positional operator cannot be combined with an upsert since it requires a matching array element. If your update results in an insert then the "$" will literally be used as the field name.
$ref $id and $db are the only valid keys which begin with a dollar sign - they are related to the DBRef convention.
The reason you can't use your own keys starting with a dollar sign is becuase it has Special meaning and would otherwise create ambiguity.