MongoDB find does not follow collation rule - json

I wanted to create an index on a membershipAccount collection, that had a collation rule set that would allow the find method to find a username while being case-insensitive.
[
2,
{
"username" : 1.0
},
"username_1",
"kollecto_prod.membershipAccounts",
{
"locale" : "da",
"caseLevel" : false,
"caseFirst" : "off",
"strength" : 1,
"numericOrdering" : false,
"alternate" : "non-ignorable",
"maxVariable" : "punct",
"normalization" : false,
"backwards" : false,
"version" : "57.1"
}
]
I have a user named - user#test.com in my membershipAccountscollection.
But this is my result of my find queries:
db.getCollection('membershipAccounts').find({username: "user#test.com"}) // 1 result - ok
db.getCollection('membershipAccounts').find({username: "USER#test.com"}) // 0 results - nope
db.getCollection('membershipAccounts').find({username: "USER#TEST.com"}) // 0 results - nope
db.getCollection('membershipAccounts').find({username: "USER#TEST.COM"}) // 0 results - nope
Is my collation incorect or have I misunterstood the concept?

You need to either set the desired collation in the query, or set the default collation on the collection. Creating an index with a collation does not make that collation the default one used in queries.
Per query:
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.foo.insert({v:'test'})
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.foo.find({v:'TEST'}).collation({locale:'en',caseLevel:false,strength:1})
{ "_id" : ObjectId("5ed8302ba6de3bc31c771682"), "v" : "test" }
Default collation:
MongoDB Enterprise ruby-driver-rs:PRIMARY> c={locale:'en',caseLevel:false,strength:1}
{ "locale" : "en", "caseLevel" : false, "strength" : 1 }
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.createCollection('bar',{collation:c})
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1591226649, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1591226649, 1)
}
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.bar.insert({v:'test'})
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise ruby-driver-rs:PRIMARY> db.bar.find({v:'TEST'})
{ "_id" : ObjectId("5ed8311fa6de3bc31c771683"), "v" : "test" }

So what I ended up doing instead was in my code as follows:
var account = DBHelper.membershipAccountCollection.Find(
x => x.Username.ToLower().Contains(username)
).FirstOrDefault();
var cu = DBHelper.customerUserCollection.AsQueryable().First(
x => x.Email.ToLower().Contains(model.UserName)
);
This works since I already had the collection object in my C# code, so I could that way utilize the power of C# instead. I realize this is not a general answer to the overall question, but it works in my case. If anyone has a more generalized answer to this question I am more than happy to accept this as an answer at a later date.

Related

If then else in json

Set values based on the ip address in JSON with if the else
Using perl JSON module.
I'm trying to setup my etc/config.json so that I never have to edit it when I move code from development->staging->production.
sql ip addresses will be different for different VM's
{
"html_title" : "Bogus",
"local_ip" :
"default" : "[production]",
"hosts" :
{
"production" :
{
"10.11.12.13" : "b102-vm",
"10.11.12.14" : "b103-vm"
},
"stage" :
{
"10.11.12.15" : "b102-vm-s",
"10.11.12.16" : "b103-vm-s"
},
},
"production" :
{
"some_schema" :
{
"host" : "10.11.12.13",
"database" : "some_table",
"dsn" : "DBI:mysql:[production.some_schema.database]:[production.some_schema.host]",
"user" : "user",
"pass" : "pass"
}
},
"stage" :
{
"some_schema" :
{
"host" : "10.11.12.16",
"database" : "some_table",
"dsn" : "DBI:mysql:[stage.some_schema.database]:[stage.some_schema.host]",
"user" : "user",
"pass" : "pass"
}
},
"if" : HERE IS WHERE I GET LOST.
{
"local_ip" : {
"anyOf" : ["[hosts.production]"]
}
},
"then" : { "default" : "[production]" }
"else" :
{
"if" :
{
"local_ip" : {
"anyOf" : ["[hosts.stage]"]
}
},
"then" : { "default" : "[production]" }
}
}
Would like to see "default" point to the production or stage values
As others have pointed out in the comments, JSON is not a programming language; it is a data storage format.
I can think of two obvious ways to do this. Yours is, I'm sorry to say, not one of them.
Both of my solutions involve your various systems having an environment variable set which identifies the kind of environment it is. I'm also going to assume some kind of get_config() function that reads a config file and returns a Perl data structure of the config.
Solution 1: Separate files
Have two files called "staging.json" and "production.json". The get_config() function reads the required one.
sub get_config {
my %valid_env = (
staging => 1,
production => 1,
);
die "$ENV{ENVIRONMENT} is not a valid environment type\n"
unless $valid_env{$ENV{ENVIRONMENT}};
my $json = read_file("$ENV{ENVIRONMENT}.json");
return JSON->new->decode($json);
}
As an extension to this, you could have three files - common.json, staging.json and production.json. You'd always read common.json along with the correct environment file and merge the two data structures into one before returning the data.
Solution 2: Munge the config
Stick with your existing file but munge the data structure after you've read it and before you return it.
sub get_config {
my %valid_env = (
staging => 1,
production => 1,
);
die "$ENV{ENVIRONMENT} is not a valid environment type\n"
unless $valid_env{$ENV{ENVIRONMENT}};
my $json = read_file('config.json');
my $config = JSON->new->decode($json);
# Use $ENV{ENVIRONMENT} to munge the contents of $config
# so it only contains data about the current environment
return $config;
}

Query date range on mongodb by passing parameters from datepicker [duplicate]

I've been playing around storing tweets inside mongodb, each object looks like this:
{
"_id" : ObjectId("4c02c58de500fe1be1000005"),
"contributors" : null,
"text" : "Hello world",
"user" : {
"following" : null,
"followers_count" : 5,
"utc_offset" : null,
"location" : "",
"profile_text_color" : "000000",
"friends_count" : 11,
"profile_link_color" : "0000ff",
"verified" : false,
"protected" : false,
"url" : null,
"contributors_enabled" : false,
"created_at" : "Sun May 30 18:47:06 +0000 2010",
"geo_enabled" : false,
"profile_sidebar_border_color" : "87bc44",
"statuses_count" : 13,
"favourites_count" : 0,
"description" : "",
"notifications" : null,
"profile_background_tile" : false,
"lang" : "en",
"id" : 149978111,
"time_zone" : null,
"profile_sidebar_fill_color" : "e0ff92"
},
"geo" : null,
"coordinates" : null,
"in_reply_to_user_id" : 149183152,
"place" : null,
"created_at" : "Sun May 30 20:07:35 +0000 2010",
"source" : "web",
"in_reply_to_status_id" : {
"floatApprox" : 15061797850
},
"truncated" : false,
"favorited" : false,
"id" : {
"floatApprox" : 15061838001
}
How would I write a query which checks the created_at and finds all objects between 18:47 and 19:00? Do I need to update my documents so the dates are stored in a specific format?
Querying for a Date Range (Specific Month or Day) in the MongoDB Cookbook has a very good explanation on the matter, but below is something I tried out myself and it seems to work.
items.save({
name: "example",
created_at: ISODate("2010-04-30T00:00:00.000Z")
})
items.find({
created_at: {
$gte: ISODate("2010-04-29T00:00:00.000Z"),
$lt: ISODate("2010-05-01T00:00:00.000Z")
}
})
=> { "_id" : ObjectId("4c0791e2b9ec877893f3363b"), "name" : "example", "created_at" : "Sun May 30 2010 00:00:00 GMT+0300 (EEST)" }
Based on my experiments you will need to serialize your dates into a format that MongoDB supports, because the following gave undesired search results.
items.save({
name: "example",
created_at: "Sun May 30 18.49:00 +0000 2010"
})
items.find({
created_at: {
$gte:"Mon May 30 18:47:00 +0000 2015",
$lt: "Sun May 30 20:40:36 +0000 2010"
}
})
=> { "_id" : ObjectId("4c079123b9ec877893f33638"), "name" : "example", "created_at" : "Sun May 30 18.49:00 +0000 2010" }
In the second example no results were expected, but there was still one gotten. This is because a basic string comparison is done.
To clarify. What is important to know is that:
Yes, you have to pass a Javascript Date object.
Yes, it has to be ISODate friendly
Yes, from my experience getting this to work, you need to manipulate the date to ISO
Yes, working with dates is generally always a tedious process, and mongo is no exception
Here is a working snippet of code, where we do a little bit of date manipulation to ensure Mongo (here i am using mongoose module and want results for rows whose date attribute is less than (before) the date given as myDate param) can handle it correctly:
var inputDate = new Date(myDate.toISOString());
MyModel.find({
'date': { $lte: inputDate }
})
Python and pymongo
Finding objects between two dates in Python with pymongo in collection posts (based on the tutorial):
from_date = datetime.datetime(2010, 12, 31, 12, 30, 30, 125000)
to_date = datetime.datetime(2011, 12, 31, 12, 30, 30, 125000)
for post in posts.find({"date": {"$gte": from_date, "$lt": to_date}}):
print(post)
Where {"$gte": from_date, "$lt": to_date} specifies the range in terms of datetime.datetime types.
db.collection.find({"createdDate":{$gte:new ISODate("2017-04-14T23:59:59Z"),$lte:new ISODate("2017-04-15T23:59:59Z")}}).count();
Replace collection with name of collection you want to execute query
MongoDB actually stores the millis of a date as an int(64), as prescribed by http://bsonspec.org/#/specification
However, it can get pretty confusing when you retrieve dates as the client driver will instantiate a date object with its own local timezone. The JavaScript driver in the mongo console will certainly do this.
So, if you care about your timezones, then make sure you know what it's supposed to be when you get it back. This shouldn't matter so much for the queries, as it will still equate to the same int(64), regardless of what timezone your date object is in (I hope). But I'd definitely make queries with actual date objects (not strings) and let the driver do its thing.
Use this code to find the record between two dates using $gte and $lt:
db.CollectionName.find({"whenCreated": {
'$gte': ISODate("2018-03-06T13:10:40.294Z"),
'$lt': ISODate("2018-05-06T13:10:40.294Z")
}});
Using with Moment.js and Comparison Query Operators
var today = moment().startOf('day');
// "2018-12-05T00:00:00.00
var tomorrow = moment(today).endOf('day');
// ("2018-12-05T23:59:59.999
Example.find(
{
// find in today
created: { '$gte': today, '$lte': tomorrow }
// Or greater than 5 days
// created: { $lt: moment().add(-5, 'days') },
}), function (err, docs) { ... });
db.collection.find({$and:
[
{date_time:{$gt:ISODate("2020-06-01T00:00:00.000Z")}},
{date_time:{$lt:ISODate("2020-06-30T00:00:00.000Z")}}
]
})
##In case you are making the query directly from your application ##
db.collection.find({$and:
[
{date_time:{$gt:"2020-06-01T00:00:00.000Z"}},
{date_time:{$lt:"2020-06-30T00:00:00.000Z"}}
]
})
You can also check this out. If you are using this method, then use the parse function to get values from Mongo Database:
db.getCollection('user').find({
createdOn: {
$gt: ISODate("2020-01-01T00:00:00.000Z"),
$lt: ISODate("2020-03-01T00:00:00.000Z")
}
})
Save created_at date in ISO Date Format then use $gte and $lte.
db.connection.find({
created_at: {
$gte: ISODate("2010-05-30T18:47:00.000Z"),
$lte: ISODate("2010-05-30T19:00:00.000Z")
}
})
use $gte and $lte to find between date data's in mongodb
var tomorrowDate = moment(new Date()).add(1, 'days').format("YYYY-MM-DD");
db.collection.find({"plannedDeliveryDate":{ $gte: new Date(tomorrowDate +"T00:00:00.000Z"),$lte: new Date(tomorrowDate + "T23:59:59.999Z")}})
mongoose.model('ModelName').aggregate([
{
$match: {
userId: mongoose.Types.ObjectId(userId)
}
},
{
$project: {
dataList: {
$filter: {
input: "$dataList",
as: "item",
cond: {
$and: [
{
$gte: [ "$$item.dateTime", new Date(`2017-01-01T00:00:00.000Z`) ]
},
{
$lte: [ "$$item.dateTime", new Date(`2019-12-01T00:00:00.000Z`) ]
},
]
}
}
}
}
}
])
For those using Make (formerly Integromat) and MongoDB:
I was struggling to find the right way to query all records between two dates. In the end, all I had to do was to remove ISODate as suggested in some of the solutions here.
So the full code would be:
"created": {
"$gte": "2016-01-01T00:00:00.000Z",
"$lt": "2017-01-01T00:00:00.000Z"
}
This article helped me achieve my goal.
UPDATE
Another way to achieve the above code in Make (formerly Integromat) would be to use the parseDate function. So the code below will return the same result as the one above :
"created": {
"$gte": "{{parseDate("2016-01-01"; "YYYY-MM-DD")}}",
"$lt": "{{parseDate("2017-01-01"; "YYYY-MM-DD")}}"
}
⚠️ Be sure to wrap {{parseDate("2017-01-01"; "YYYY-MM-DD")}} between quotation marks.
Convert your dates to GMT timezone as you're stuffing them into Mongo. That way there's never a timezone issue. Then just do the math on the twitter/timezone field when you pull the data back out for presentation.
Why not convert the string to an integer of the form YYYYMMDDHHMMSS? Each increment of time would then create a larger integer, and you can filter on the integers instead of worrying about converting to ISO time.
Scala:
With joda DateTime and BSON syntax (reactivemongo):
val queryDateRangeForOneField = (start: DateTime, end: DateTime) =>
BSONDocument(
"created_at" -> BSONDocument(
"$gte" -> BSONDateTime(start.millisOfDay().withMinimumValue().getMillis),
"$lte" -> BSONDateTime(end.millisOfDay().withMaximumValue().getMillis)),
)
where millisOfDay().withMinimumValue() for "2021-09-08T06:42:51.697Z" will be "2021-09-08T00:00:00.000Z"
and
where millisOfDay(). withMaximumValue() for "2021-09-08T06:42:51.697Z" will be "2021-09-08T23:59:99.999Z"
i tried in this model as per my requirements i need to store a date when ever a object is created later i want to retrieve all the records (documents ) between two dates
in my html file
i was using the following format mm/dd/yyyy
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<script>
//jquery
$(document).ready(function(){
$("#select_date").click(function() {
$.ajax({
type: "post",
url: "xxx",
datatype: "html",
data: $("#period").serialize(),
success: function(data){
alert(data);
} ,//success
}); //event triggered
});//ajax
});//jquery
</script>
<title></title>
</head>
<body>
<form id="period" name='period'>
from <input id="selecteddate" name="selecteddate1" type="text"> to
<input id="select_date" type="button" value="selected">
</form>
</body>
</html>
in my py (python) file i converted it into "iso fomate"
in following way
date_str1 = request.POST["SelectedDate1"]
SelectedDate1 = datetime.datetime.strptime(date_str1, '%m/%d/%Y').isoformat()
and saved in my dbmongo collection with "SelectedDate" as field in my collection
to retrieve data or documents between to 2 dates i used following query
db.collection.find( "SelectedDate": {'$gte': SelectedDate1,'$lt': SelectedDate2}})

How to use equalTo() query function on a child which is an array

I have a Json Tree for Teachers database
{ "Teachers":
{
"Teacher_id" : {
"email" : "teacher1#something.com",
"name" : "teacher1",
"Subjects" : {0 : "Maths",
1 : "science"
},
"Subject_Exp-in-years" :{ 0:"Maths_2",
1:"Science_4"
}
},
"Teacher_id" : {
"email" : "teacher2#something.com",
"name" : "teacher2",
"Subjects" : {0 : "Geography",
1 : "science"
},
"Subject_Exp-in-years" :{ "Geography_5",
"Science_1"
}
}
}
How can I get Set of Teachers who teach Science
I have used the query
var db_ref=firebase.database().ref();
db_ref.child("Teachers").orderByChild("Subjects").equalTo("Science")
How can I check if a value is present in an array in the database by querying functions?
When using .orderByChild("Subjects").equalTo("Science") doesn't work because none of the Subjects node children is equal to Science. So in order to solve this, I recomand you to change a little bit your database structure like this:
{ "Teachers":
{
"Teacher_id" : {
"email" : "teacher1#something.com",
"name" : "teacher1",
"Subjects" : {Maths: true,
science: true
},
As you can see i have changed the childrens of Subjects with: Maths: true and science: true. To check if a value is present under a node, just use exists() function on that node.
Hope it helps.
You can use for loop to read each of the element in an array.
When you read them one by one, you can include if else statement to check whether the value that you want existed or not. Let,s say the value is JOHN.
var index;
var arraylist = ["a", "b", "c"];
for (index = 0; index < arraylist.length; ++index) {
if(arraylist[index] = "JOHN") {
console.log(arraylist[index] + "is in the array");
};
}

Selected row not highlighted in jqgrid

I have written the function for row selection.It is not highlighting the selected row(sometimes highlighting sometimes other row highlighted) and not displaying the icons the way I have written it. following is the code
multiselect : true,
iconSet: "fontAwesome",
datatype : "json",
loadonce : true,
rowNum : 10,
rowList : [ 10, 20, 30 ],
toppager:true,
pager : '#prowed1',
sortname : 'id',
viewrecords : true,
sortorder : "asc",
editurl : "editGrid.html",
onSelectRow : function(rowId) {
var rowId = jQuery("#list1").jqGrid('getGridParam',
'selarrrow');
if (rowId.length > 1) {
$("#list1_iledit").addClass('ui-state-disabled');
}
},
$("#list1").jqGrid(
"navGrid",
"#prowed1",
{
cloneToTop:true,
formatter : "checkboxFontAwesome4",
addicon:"fa fa-plus ",
add : true,
delicon:"fa fa-trash",
del : true,
searchicon:"fa fa-search",
search : true,
refreshicon:"fa fa-refresh",
refresh : true,
editicon:"fa fa-edit ",
edit : true,
saveicon : 'fa fa-floppy-o',
save : true,
},`{ // edit options
afterSubmit : function() {
location.reload(true);
},
beforeShowForm : function(form) {
$("td .navButton navButton-ltr").hide();
},
closeAfterEdit : true
},
{ // add options
beforeShowForm : function(form) {
$("#buName").removeAttr("readonly");
},
closeAfterAdd : true,
clearAfterAdd : true
},
{ // del options
serializeDelData : function(postdata) {
return {
'buName' : $('#list1').jqGrid('getCell',
postdata.id, 'buName'),
'oper' : 'del'
}
}
}
);` $("#list1").jqGrid('inlineNav', "#prowed1", {
//cloneToTop:true,
//iconSet: "fontAwesome",
add : false,
edit : true,
editicon : 'fa fa-pencil-square-o',
save : true,
saveicon : 'fa fa-floppy-o',
editParams : {
aftersavefunc : function(id) {
jQuery('#list1').jqGrid('setSelection', id, false);
},
},
});`
You should provide the demo, which reproduce the problem. The reason of the most problems with highlighted: wrong input data or wrong colModel. Every row of jqGrid have always id attribute (rowid), which should be part of input data: see here. The id value must be unique. If you have id duplicates then you could problems with selection/highlighting of rows.
If you fill the data from the database than you can use native ids from the tables of the database to build unique rowids. Database tables don't allow id duplicates too.
To be able to edit the data in the database you will need to identify the edited data. Such unique value could be used as the rowid. If you fill the grid by JOIN from multiple tables, than composed id could be used. For example, if you fill the the grid with the data from two tables User and Location then the paar (user_id, location_id) are unique. If both user_id and location_id are numbers, then you can use user_id + "_" + location_id as rowid. The value will be sent to the server during editing and you will have full information to find the data in the tables, which need be modified.
I think that you have problems with unique id in the data. Check that the ids used when you insert data in the grid are unique and you do not have duplicate one.
Kind Regards
In your ColModel make sure the property key:true is only specified once and represents a unique row ID value. See http://www.trirand.com/jqgridwiki/doku.php?id=wiki:colmodel_options
Dumb on my part, but I had the property beforeSelectRow set in the grid code that I had copied from another project. This of course was blocking any selection of a grid row.
beforeSelectRow: function (rowid, e) {
return false;
},

Writing mongoDb queries using mongoose to create node api

I have some mongoDb collections 'classrooms' and 'students' of the form:
Classrooms:
{
"_id" : ObjectId("56c7219dbd5f92cd78ae4b7e"),
"name" : "My Students",
"user" : ObjectId("56c7218cbd5f92cd78ae4b7c"),
"updatedAt" : ISODate("2016-02-19T14:07:25.965+0000"),
"createdAt" : ISODate("2016-02-19T14:07:25.965+0000"),
"students" : [
ObjectId("56dbb26cff34aa686c0d9d25"),
ObjectId("56f7c2bf1982aa9219ae8843")
],
"role" : "user",
"allowDelete" : false,
"__v" : NumberInt(0)
}
Students:
{
"_id" : ObjectId("56dbb26cff34aa686c0d9d25"),
"email" : "1989manjari#gmail.com",
"createdBy" : ObjectId("56c7218cbd5f92cd78ae4b7c"),
"classRoom" : ObjectId("56c7219dbd5f92cd78ae4b7e"),
"mentorEmail" : "gauravatbits#gmail.com",
"studentId" : ObjectId("56ced54303b7cb7b0eda9862"),
"status" : true,
"updatedAt" : ISODate("2016-03-11T15:32:36.806+0000"),
"autoAdd" : true,
"createdAt" : ISODate("2016-03-06T04:30:36.073+0000"),
"__v" : NumberInt(0)
}
My query is:
id_list = db.classrooms.distinct("students");
db.students.find({_id: {$in: id_list}, studentId:{$exists:false}},{email:1, mentorEmail: 1}).pretty()
Now I want to create an endpoint in node api for this response. So i wanted to know how can I write these queries in Mongoose and create endpoint like: app.get('/api/myquery') to get the json result.
P.S. : Is it possible to do this, without creating schema in Mongoose because i also have some collections which have large no. of fields(38 fields). I just want to get some json data by applying queries in already existing collections.
something like this with mongoose
just pseudo code
var model = mongoose.model("collection", mongooseSchema)
app.get('/api/myquery', function(){
model.find({_id : id_list}, "email mentorEmail", function(err, data){
if(err)throw err;
res.json(data) // maybe use JSON.stringify() if didnt get json but that sets application/json in header of HTTP
})
})
You can directly insert jsons in mongoose but you should keep in mind that only that data will be read which are mentioned in the models in mongoose. Do keeping 38 fields is not a problem just insert in directly.