Waterline - Where with sum of fields - mysql

I've got the following model Test
module.exports = {
attributes: {
a: {
type: 'number'
},
b: {
type: 'number'
}
}
}
I would like to build a query that allows me to put sum of fields a and b in where statement.
SQL equavilent:
SELECT * FROM Test WHERE a + b = myValue
I read in sails doc's about Criteria modifiers but there is no word about that.
Is there any clever way to do that? Of course I can use native query but I would like to avoid that because I must use the sum along with other modifiers. The reason is I'm generating dynamic queries from separate files and with native queries I will have to also handle already defined functionality like or, and, etc.

I found a workaround. Maybe it will be useful to someone.
It is not stricte sails/node solution, but database one, however, it fits my case perfectly.
From MySQL 5.7 there is something like generated columns.
Columns are generated because the data in these columns are computed based on predefined expressions.
All I had to do was add an extra, auto generated column to my Test model:
module.exports = {
attributes: {
a: {
type: 'number',
columnType: 'int'
},
b: {
type: 'number',
columnType: 'int'
},
c: {
type: 'number',
columnType: 'int GENERATED ALWAYS AS(a + b) VIRTUAL'
}
}
}
Now I'm able to do such query:
const result = await Test.find({ c: 2 })
...and I get the correct result. Waterline treats my column like any other, database does everything instead of me.
Of course I can mix it with other modifiers with no problems.
I haven't seen any complications so far.

Related

Sequelize converted float to int NodeJS for MySQL DB while read

I have the following data in MySQL database:
Column: Volume
Value:
3495303.0
3495123.8
3484616.8
3482624.8
3474865.5
3434217.5
3407878.0
I use the following code to read from the database:
let query = {
where: {
id: id
},
order: [
['time', 'DESC']
],
}
let result = await markets.findAll(query);
At the following is markets.js
let markets = dbConnection.connection.define(
'markets', {
...,
volume: Sequelize.FLOAT,
...,
}, {
timestamps: false
}
);
module.exports = markets;
However, when column "volume" is read from the database, all the number is converted into int rather than float.
I manually executed SQL script generated by Sequelize, and I can see the value of volume has decimals.
However, when the data is loaded into my JSON object "result", the value for volume becomes integer.
I have tried with changing the definition in markets.js
from
volume: Sequelize.FLOAT,
to
volume: Sequelize.DOUBLE,
But it does not change a thing.
The definition of volume in MySQL is:
volume float
Alright I figured it out.
The reason being it is doing the wrong thing is the number for volume is overwhelming. There are volume value like "1.0775351E7" which is beyond the capability of FLOAT calculation.
I changed the data type in both database and node code to DOUBLE and it is working fine now.

How to query for Null or Missing Fields in Mysql X DevAPI?

How to query for null or missing field in Mysql X DevAPI?
I tried .find("age IS NULL") and .find("age = null") but both not work.
> db.createCollection('users')
> db.getCollection('users').add({ name: "foo", age: 30 })
> db.getCollection('users').add({ name: "bar", age: null })
> db.getCollection('users').find("age IS NULL")
Empty set (0.0003 sec)
> db.getCollection('users').find("age = null")
Empty set (0.0004 sec)
I'm able to reproduce that with the MySQL Shell (which I guess is what you are using) and with Connector/Node.js. I know for a fact that Connector/Node.js sends the correct datatype to the server (with the Shell you can check that with the --trace-proto option).
Mysqlx.Crud.Find {
collection {
name: "<some_schema>"
schema: "users"
}
data_model: DOCUMENT
criteria {
type: OPERATOR
operator {
name: "is"
param {
type: IDENT
identifier {
document_path {
type: MEMBER
value: "age"
}
}
}
param {
type: LITERAL
literal {
type: V_NULL
}
}
}
}
}
Which means it's some issue in the server.
In this case, looks like that X DevAPI expression is not resulting in the proper SQL query. If you look in the general log, you should see something like
SELECT doc FROM `<some_schema>`.`users` WHERE (JSON_EXTRACT(doc,'$.age') IS NULL)
The problem is that JSON_EXTRACT is returning null (JSON type) and not NULL (SQL "type"), and it is a limitation of the X Plugin.
One way for this to be fixed by the plugin is to replace JSON_EXTRACT() with JSON_VALUE(), which will return the proper NULL value in that case, but I don't know the implications of that.
As a workaround, you can always use
session.sql("select doc from `<some_schema>`.`users` where json_value(doc, '$.age') is null").execute()
In the meantime, I encourage you to report a bug at https://bugs.mysql.com/ using either the MySQL Server: Document Store: X Plugin or the MySQL Server: Document Store: MySQL Shell categories.
Disclaimer: I'm the lead developer of the MySQL X DevAPI Connector for Node.js

How can I pass parameters to getter in Sequelize model?

What I'm doing:
I'm reading csv which contains data like townships, religions and so on. In the csv, they are text values. But of course in my table in the database, they are foreign_key. So to insert those, I need to map those text values to id.
It is possible to get the id like below.
const { townshipId } = await models.Township.findOne({
where: { name: township },
attributes: ["townshipId"],
raw: true,
});
The problem:
But the problem what I think is that those findOne(s) will be populated in my entire method because I have like 50 other properties besides township.
What I'm trying
I'm trying to pass township-name to a getter method, expecting its equivalent townshipId.
I found out that it is possible to use getter and setter in sequelize. But I don't find how to pass parameters to those.
My question
Am I trying the correct way to import the csv data? How can I use getters (and setters) with parameters in sequelize?
Maybe if you want to exclude columns from the result, you pass the "attributes" field like:
attributes: { exclude: ['field1','field2'] }
This way you wont show the columns "field1" and "field2" in the result.

How to model endpoint with different data types?

I have a model in the DB with the following structure
id
key
value_type (can be 'String', 'Integer' and 'Boolean')
string_value
int_value
bool_value
Where only one of string_value, int_value or bool_value have value while the other 2 are nulls depends on the value of value_type. for example:
id:1, key:'key1', value_type:'String', string_value:'random string', int_value:null, bool_value: null
id:2, key:'key2', value_type:'Integer', string_value:NULL, int_value:777, bool_value:NULL
id:3, key:'key3', value_type:'Boolean', string_value:NULL, int_value:NULL, bool_value:false
I want to open an endpoint to allow GET for entities of this type and consider the two following JSON structure for the return object:
Option 1:
{
id:<id>,
key:<key>,
value_type:<value_type>,
value:<a string representation of the value>
}
for instance:
{
id:2,
key:"key2",
value_type:"Integer",
value:"777"
}
Option 2:
{
id:<id>,
key:<key>,
value_type:<value_type>,
string_value:<string_value>,
integer_value:<integer_value>,
boolean_value:<boolean_value>
}
for instance:
{
id:2,
key:"key2",
value_type:"Integer",
string_value:null,
integer_value:777
boolean_value:null
}
The first option is cleaner while the second one is more typesafe. Is there any standard or consideration to take before making a choice?

Sequelize datatype TEXT not working with mySQL

I am using sequelize ORM with mySQL database.
I have a model with attribute type TEXT as :
description: {
type: Sequelize.TEXT,
unique: true
},
When I am trying to create table for the corresponding model, its giving an error message as :
Unhandled rejection SequelizeDatabaseError:
ER_BLOB_KEY_WITHOUT_LENGTH: BLOB/TEXT column 'description' used in key
specification without a key length
This worked fine when used with postgreSQL.
Possible reason for this error which i could think of can be that mySQL doesn't support TEXT datatype and therefore, i have to specify it as LONGTEXT.
If I am thinking correct or is there some other reason for the same, if someone can help.
You are using unique: true with data-type TEXT. You can not use it. Do you delete unique: true and try again?
esmrkbr is correct, mySQL does not accept UNIQUE KEY on a TEXT field, you need to use VARCHAR instead (see: make text column as unique key). That being said, depending on the Database being used, you may need to explicitly specify a size for a TEXT (or BLOB) type. The documentation (http://docs.sequelizejs.com/en/latest/api/datatypes/) is pretty terse on this point and other than a link to the code, currently only has the following information:
An (un)limited length text column. Available lengths: tiny, medium,
long
You can pass the size as a string argument to the type. For example, to have it defined as LONGTEXT you will need:
description: {
type: Sequelize.TEXT('long')
},
The code in lib/data-types.js has the following mapping to SQL for TEXT (there is a similar one for BLOB also):
TEXT.prototype.toSql = function() {
switch (this._length.toLowerCase()) {
case 'tiny':
return 'TINYTEXT';
case 'medium':
return 'MEDIUMTEXT';
case 'long':
return 'LONGTEXT';
default:
return this.key;
}
};
Define as string like this:
var filters = sequelize.define('filters', {
description: {
type: DataTypes.STRING,
validate: { notEmpty: true }
}
}