I'm storing a location of type POINT in my mysql database:
module.exports = function (sequelize, DataTypes) {
var Promotion = sequelize.define('promotion', {
userId: {
type: DataTypes.STRING,
allowNull: false
},
location: {
type: DataTypes.GEOMETRY('POINT'),
allowNull: false
}
It looks like the point is being stored correctly:
I'm trying find all rows within a set of bounds using the mysql native ST_MakeEnvelope function. I was using this with postgresql and it worked perfectly, but when I switch to mysql it start throwing errors:
if (northEastLng &&
northEastLat &&
southWestLat &&
southWestLng)
{
where.location = {
$overlap: db.sequelize.fn('ST_MakeEnvelope', southWestLng, southWestLat, northEastLng, northEastLat)
}
}
promotion.findAll({
where,
})
.then(promoters => {
res.json(promoters)
})
.catch(err => {
console.log('ERROR: ', err)
})
The error:
Error: ER_WRONG_PARAMCOUNT_TO_NATIVE_FCT:
Incorrect parameter count in the call to native function 'ST_MakeEnvelope'
So I tried passing in two points instead:
const POINT_ONE = `POINT(${southWestLng} ${southWestLat})`
const POINT_TWO = `POINT(${northEastLng} ${northEastLat})`
where.location = {
$overlap: db.sequelize.fn('ST_MakeEnvelope', POINT_ONE, POINT_TWO)
}
Now I'm getting this error:
SequelizeDatabaseError: UNKNOWN_CODE_PLEASE_REPORT:
Geometry byte string must be little endian.
Been searching around, and not too sure where to go from here. How do I use the ST_MakeEnvelope function with sequelize to query?
Edit
Adding the generated sql from piotrbienias response:
SELECT `promotion`.`id`, `promotion`.`userId`, `promotion`.`title`, `promotion`.`description`, `promotion`.`startDate`, `promotion`.`endDate`, `promotion`.`isIndefinite`, `promotion`.`isApproved`, `promotion`.`status`, `promotion`.`reach`, `promotion`.`trustRanking`, `promotion`.`isLocationBased`, `promotion`.`address`, `promotion`.`city`, `promotion`.`state`, `promotion`.`zip`, `promotion`.`location`, `promotion`.`createdAt`, `promotion`.`updatedAt`, `promotion`.`categoryId
`, `promotionImages`.`id` AS `promotionImages.id`, `promotionImages`.`url` AS `promotionImages.url`, `promotionImages`.`publicId` AS `promotionImages.publicId`, `promotionImages`.`secureUrl` AS `promotionImages.secureUrl`, `promotionImages`.`isApproved` AS `promotionImages.isApproved`, `promotionImages`.`createdAt` AS `promotionImages.createdAt`, `promotionImages`.`updatedAt` AS `promotionImages.updatedAt`, `promotionImages`.`promotionId` AS `promotionImages.promotionId`, `category`.`id` AS `cat
egory.id`, `category`.`title` AS `category.title` FROM `promotions` AS `promotion` LEFT OUTER JOIN `promotionImages` AS `promotionImages` ON `promotion`.`id` = `promotionImages`.`promotionId` LEFT OUTER JOIN `categories` AS `category` ON `promotion`.`categoryId` = `category`.`id` WHERE `promotion`.`location` && ST_MakeEnvelope(ST_GeomFromText('POINT(-80.30252222253421 25.802030960352745)'), ST_GeomFromText('POINT(-80.30252222253421 25.802030960352745)'));
I am not familiar with the Geometry part of MySQL, but shouldn't you use the ST_GeomFromText function on those points before using them?
where.location = {
$overlap: db.sequelize.fn(
'ST_MakeEnvelope',
db.sequelize.fn('ST_GeomFromText', POINT_ONE),
db.sequelize.fn('ST_GeomFromText', POINT_TWO),
)
}
};
Just as it is presented in the example of ST_MakeEnvelope function (and it takes two parameters, so that may be the reason of your first error Incorrect parameter count in the call to native function 'ST_MakeEnvelope'). And, as I said on the beginning, I am definitely not an expert in these, just my suggestion after taking a look at the MySQL documentation.
EDIT
Below is description of $overlap from Sequelize documentation
$overlap: [1, 2] // && [1, 2] (PG array overlap operator)
I think that you should construct your Sequelize query differently, without the $overlap. I suppose that you should use ST_Contains function and your where object should be as follows
{
where: db.sequelize.where(
db.sequelize.fn(
'ST_Contains',
db.sequelize.fn(
'ST_MakeEnvelope',
db.sequelize.fn('ST_GeomFromText', POINT_ONE),
db.sequelize.fn('ST_GeomFromText', POINT_TWO)
),
db.sequelize.col('promotion.location')
),
'=',
1
)
}
In my opinion the SQL you want should be something like:
WHERE ST_Contains(POLYGON(...), location)
Above Sequelize query would generate
WHERE ST_Contains(ST_MakeEnvelope(ST_GeomFromText(POINT_ONE), ST_GeomFromText(POINT_TWO)), location) = 1;
Which should check if polygon created from two points contains value of location column. ST_Contains returns 1 or 0, so I think that = 1 condition should be ok in this case.
Related
Given this query here,
let output = [];
const sql = `select * from coredb.account LIMIT ${offset},${limit}`;
let data = await sequelize.query(sql, null, {raw: true, type: sequelize.QueryTypes.SELECT});
data.forEach((item) => {
console.log(item['id'], item.id); // <-- output says "undefined, undefined"
});
the data variable is indeed hydrated with the right row data when using console.log to inspect it.
But, when I try to access the individual properties, they only ever come back as undefined. This TextRow object that Sequelize seems to return the result in doesn't seem to want to let me access then explicit rows.
Just curious what i'm missing here, am I missing an option?
I agree, Sequalize raw queries are not intuitive. You don't need the null or raw: true flag. Something like this should work:
let data = await sequelize.query(sql, {type: sequelize.QueryTypes.SELECT});
When I tried this, "data" was an array of two objects, each being the query result. So, the properties can be accessed by using index [0].... e.g.
data[0].forEach((item) => {
console.log(item['id'], item.id); // <-- output says "undefined, undefined"
});
Not yet sure WHY this occurs!
EDIT - it's because .query() should have only two arguments. Changing the call to: sequelize.query(sql, {raw: true, type: sequelize.QueryTypes.SELECT}) resulted in data being a single array (as expected).
Finally I was able to find the solution for it.
You just need to make a new array and push data into it by finding bases on key name like this:
suppose we have data in students object:
let finalArray = new Array();
for (var k in students ) {
finalArray.push(students[k])
}
console.log(finalArray) // Normal JSON array object :)
m.sequelize.query(sql, {
model,
mapToModel: true
})
.then(model => res.status(200).send(model))
.catch(error => res.status(400).send(error.toString())
})
I'm using nodeJs Express Framework.
I'm using mysql database with sequelizejs library and using querying for retrieve data.
I am getting timeout error when I fired select query for almost 50,00,000 records.
I have done the server timeout but not worked.
I have done the pooling method in sequlizeJs But not worked.
function fetchNamesData(req, name) {
return new Promise((resolve, reject) => {
const names = req.app.locals.models.names_data;
names.findAll({
where: {
name: name
},
order: [['date', 'DESC']],
limit: 50
})
.then(function (dbRes) {
console.log(dbRes.length);
resolve(dbRes);
})
.catch(function (dbErr) {
console.log(dbErr);
return reject(dbErr);
});
});
}
allNames.forEach(element => {
//console.log(element.dataValues.name);
fetchNamesData(req, element.dataValues.name).then((dbRes) => {
//here I will have all the records
}).catch((dbErr) => { console.log(dbErr) });
var allNames = {having almost 7000 names}
now I iterate this obj and each names having 50 record in database
I want to get that all record like 50*7000 = 3,50,000.
What happens in your case is :
Looping through 7000 names and at same time hitting 7000 queries in mySql , and mysql will create queue for executing 7000 queries at same time cause load on machine. Either you can update your configuration to handle such load OR
Solution to this : Try to put some timeout b/w each queries , this way you will be able to fetch more records ,
allNames.forEach(element => {
setTimeout(() => { // <----------- HERE -------------
fetchNamesData(req, element.dataValues.name).then((dbRes) => {
//here I will have all the records
}).catch((dbErr) => {
console.log(dbErr)
});
},500); // <----------- HERE -------------
});
I have found the solution like
- Remove the unwanted console.log() and
- Also your hardware configuration depend upon it for timeout error.
- When query is firing do not start or run any other work it will lead to time out error[when there are multiple crud operation is going on].
- Also give index to table field when particular field is going to use in where clause.
I've been playing with KnexJS lately and I've got most of what I need from KnexJS documentation, however I have a bit more complex MySQL query that I can't 'port' on my own to Knex. I know there's an option to use .raw(), however I'd like to avoid that if possible.
My working MySQL query looks like this:
SELECT A.profile_id, C.model_name, D.brand_name, A.car_plate
FROM carsdb.profiles_has_cars A,
carsdb.profiles B,
carsdb.brands_cars C,
carsdb.brands D
WHERE A.profile_id = B.user_id AND
A.car_id = C.id AND
A.brand_id = D.id;
What I got so far is:
return new Promise((resolve, reject) => {
db({
A: "profiles_has_cars",
B: "profiles",
C: "brands_cars",
D: "brands"
})
.select("A.profile_id", "C.model_name", "D.brand_name", "A.car_plate")
.where({
"A.profile_id": userId,
"A.car_id": "C.id",
"A.brand_id": "D.id"
})
.then(results => {
if (results > 0) {
resolve(results);
} else {
reject("There is a problem with a query");
}
});
});
I've also tried using an object as an argument in .where(), but that didn't do anything either.
Any help or suggestion?
Oh, I've got it. knex doesn’t understand that .where values are actually references to another fields and interpret them as strings.
Try to replace such with one more for each .whereRaw("A.car_id = C.id"). (For references only, not for actual values).
I am trying to select all records from table called fairCustomer where business_type_id equal array of value.
I am using sailsjs for server and mysql for DB, in frontend I am using nativescript with typescript.
and this is my code:
filterShopType(){
this.service.serviceName = 'fairCustomers';
this.service.Get({ populate:'country_id,business_type_id',where:{ business_type_id:{ in:this.business_ids_filter } }, limit:20 }).then((data: any) => {
this.fairCustomers.splice(0, this.fairCustomers.length);
this.fairCustomers.push(data);
this.Refresh('fairCustomers');
}).catch((err) => {
console.log('failed get fairCustomers from server ', err);
});
}
service refer to http.request(), where i am using it in another place in my code.
business_ids_filter is an array of ids.
When I run this code the I am getting this error:
"message": "Could not parse the provided where clause. Refer to the Sails documentation for up-to-date info on supported query language syntax:\n(http://sailsjs.com/documentation/concepts/models-and-orm/query-language)\nDetails: Unrecognized sub-attribute modifier (in) for business_type_id. Make sure to use a recognized sub-attribute modifier such as startsWith, <=, !, etc. )",
and if I removed where I got an error result.
please anyone have any idea or solution?
you may try with a native query as described here, https://sailsjs.com/documentation/reference/waterline-orm/datastores/send-native-query
As far as I can tell, you could simply provide an array of items. Matches against ANY of the items in the array means that the record should be returned as part of the result. Not sure if it works in a where-clause though.
Docs:Query Language (scroll to: In Modifier)
I do share your confusion as to why the given query does not work though, as the docs state that in should be valid, but perhaps it's not valid inside where:{...}? And I am assuming you are using .find() inside .Get(...)? Simply proxying the query unchanged through to .find()?
filterShopType(){
this.service.serviceName = 'fairCustomers';
this.service.Get({
populate:'country_id, business_type_id',
business_type_id: this.business_ids_filter,
limit:20
})
.then((data: any) => {
this.fairCustomers.splice(0, this.fairCustomers.length);
this.fairCustomers.push(data);
this.Refresh('fairCustomers');
}).catch((err) => {
console.log('failed get fairCustomers from server ', err);
});
}
Now, you of course need to make sure that the array actually is an array if you use in:.
It is working with me by using [or] instead of [in], this is the code
filterShopType(){
if(this.business_ids_filter){
this.service.serviceName = 'fairCustomers';
var arr = [];
this.business_ids_filter.forEach( (id) => {
arr.push({ business_type_id: id })
});
let where = {};
if(arr.length > 0){
where = {or: arr};
}
this.service.Get({ populate: 'country_id,business_type_id',where:where , limit:20 }).then((data: any) => {
this.filterItems.splice(0, this.filterItems.length);
this.filterItems.push(data);
this.Refresh('filterItems');
}).catch((err) => {
console.log('failed get fair Customers from server ', err);
});
}else{
this.set('isFilter', false);
}
}
I`m using sequelize with mysql and I want to know if there is a way to use sequelize to create SQLs like these:
SELECT * FROM table WHERE DAY(dateAttribute) = 'chosen_day'
SELECT * FROM table WHERE MONTH(dateAttribute) = 'chosen_month'
I`ve tried to do something like this:
this.test= function () {
let searchingDate = "2017-05%";
return model.table.findAll({
where: {
dateAttribute: searchingDate
}
})
}
This works but it gives a deprecation warning saying that it will be remove in future versions.
Thanks for any help!
Found the solution!
So I used the sequelize.fn() with the Op.and function from sequelize. So the solution was:
this.test = function () {
return model.tableName.findAll({
attributes: ['myAttribute'],
where: {
dateAttribute: {
[sequelize.Op.and]: [
sequelize.where(sequelize.fn('MONTH', sequelize.col('dateAttribute')), 5),
sequelize.where(sequelize.fn('DAY', sequelize.col('dateAttribute')), 5),
]
},
otherAttribute: 'someValue'
}
})
}
This code generated this SQL query:
SELECT myAttribute FROM tableName WHERE (MONTH('dateAttribute') = 5 AND DAY('dateAttribute') = 5) AND otherAttribute = 'someValue'
I hope to save you guys from doing this long researching that I made!
Remember be HAPPY!