MySQL FullText Search with Sequelize - mysql

I want to implement MySQL full text search with sequelize. The version "sequelize": "^3.23.6". I tried to research about this but could not find =the documentation that guides how to implement this.
Here is the link that says FullText is supported by sequelize:
https://github.com/sequelize/sequelize/issues/2979
But there is not exact documentation on how to do it and how to do a full text search query with sequelize.
Any links advice would be helpful
Thanks !

Since we now have the error message in recent Sequelize that looks like this:
Unhandled rejection Error: Support for literal replacements in the where object has been removed.
The solution would be to provide replacements manually
Payments.findAll({
where: Sequelize.literal('MATCH (SomeField) AGAINST (:name)'),
replacements: {
name: 'Alex'
}
});
Use arrays for more complex conditions:
Payments.findAll({
where: [
{ State: 'Paid' },
Sequelize.literal('MATCH (SomeField) AGAINST (:name)')
],
replacements: {
name: 'Alex'
}
});

Sequelize doesn’t fully support the full-text search feature. We can add a FULLTEXT index as easy as any other index. But operators supporting the MATCH (column) AGAINST (value) syntax haven’t been implemented.
My current solution to the problem consists of creating a regular model:
module.exports = (sequelize, DataTypes) => {
const Book = sequelize.define('Book', {
title: DataTypes.STRING,
description: DataTypes.TEXT,
isActive: DataTypes.BOOLEAN
}, {
indexes: [
// add a FULLTEXT index
{ type: 'FULLTEXT', name: 'text_idx', fields: ['description'] }
]
});
return Book;
};
And using a raw query for querying:
const against = 'more or less';
models.Book.find({
where: ['isActive = 1 AND MATCH (description) AGAINST(?)', [against]]
}).then((result) => {
console.log(result.title);
});
Using only MySQL it's not possible to get correct results if you trying to search for inflectional words, synonyms etc. MySQL developers consider adding dictionaries for full-text search (https://dev.mysql.com/worklog/task/?id=2428), but who knows when we will see it.
If you have to stick with MySQL, I suggest to take a look at Sphinx. It works properly with synonyms and inflectional words.

Since Sequlize does not support for fullText search.
Here is another approach for searching a string from a table
models.sequelize.query(
"SELECT * FROM tableName WHERE CONCAT(field1, '', field2, '', field3, '', field4 ) LIKE \"%" + keyword + "%\"",
{type: models.sequelize.QueryTypes.SELECT}).then(res => {
})

another approach, using single migration file
module.exports = {
up: (queryInterface, Sequelize) => queryInterface.createTable(
'Products',
{
id: {
type: Sequelize.UUID,
defaultValue: Sequelize.UUIDV4,
allowNull: false,
primaryKey: true,
},
name: {
type: Sequelize.STRING,
},
},
).then(() => queryInterface.addIndex('Products', ['name'], { type: 'FULLTEXT' })),
down: (queryInterface) => queryInterface.dropTable('Products'),
};

New updated Answer, Here I am searching for Locations that match my searchString.
Location.findAll({
where: {
name: sequelize.where(
sequelize.fn("LOWER", sequelize.col("name")),
"LIKE",
"%" + "You search text here"+ "%". <==== add your searchString Here
),
},
})

Related

How to speed up a change column migration using Sequelize and MySQL

I have several models with the fields: createdAt, updatedAt and deletedAt. These are set to the type DATE which results in a timestamp that is accurate to one second. I want these fields to be precise to a millisecond and the way to do that is to set their type to DATE(6) using Sequelize. This is the migration that I am using:
module.exports = {
up: (queryInterface, Sequelize) => {
return Promise.all([
queryInterface.changeColumn('transactions', 'createdAt', {
type: Sequelize.DATE(6),
allowNull: true,
}),
queryInterface.changeColumn('transactions', 'updatedAt', {
type: Sequelize.DATE(6),
allowNull: true,
}),
queryInterface.changeColumn('transactions', 'deletedAt', {
type: Sequelize.DATE(6),
allowNull: true,
}),
])
},
down: (queryInterface, Sequelize) => {
return Promise.all([
queryInterface.changeColumn('transactions', 'createdAt', {
type: Sequelize.DATE,
allowNull: true,
}),
queryInterface.changeColumn('transactions', 'updatedAt', {
type: Sequelize.DATE,
allowNull: true,
}),
queryInterface.changeColumn('transactions', 'deletedAt', {
type: Sequelize.DATE,
allowNull: true,
}),
])
}
};
The transactions table has 93 million rows and this migration ran for 10 hours before an internet problem caused it to time out. As such, is there any way to speed up it up? This is a MySQL database, the Sequelize version is 5.8.6 and the sequelize-cli version is 4.0.
This migration runs the ALTER TABLE command on the given database table. Under the hood, ALTER TABLE creates a copy of the original table, makes the given changes and copies over the original data. For a large database, this will take time. The problem is that the table gets locked whilst this operation is running so any updates on it will not go through. For this situation, there are two tools available (that I am aware of):
pt-online-schema-change
gh-ost
These tools also create a copy of the table but they don't lock up the table so we can continue updating it whilst the operation runs. They also record any updates to the original table during the operation and update the new table accordingly.

Unable to create BLOB/Binary types with LoopBack 4

I'm trying to use Loopback for my new projects, but I've been facing some problems...
I have the habit of storing my UUIDs ID in a binary format at my databases, here's an example:
#model({
settings: { mysql: { table: 'application' } },
})
export class Application extends Entity {
#property({
type: 'buffer',
required: true,
generated: false,
id: true,
dataLength: 16,
})
id: BinaryType;
[...]
}
But when I try to do the migration, I've been receiving that error from mysql:
"BLOB/TEXT column 'id' used in key specification without a key length"
I really tried everything and nothing works. Hope that you'll be able to help me!
Thanks a lot!
I'll show the answer for this question that I made.
Just define your model with the following info:
#property({
required: true,
mysql: {
columnName: 'application_id',
dataType: 'VARBINARY',
dataLength: 16,
nullable: 'NO'
}
})
application_id: string;
It worked like a charm for me :)
Thank you all!

Sequelize create with associations

I have been trying to define a relationship between 3 tables and then create them all in one create function. For some reason, while creating the 3 models, the linking IDs (foreign keys) are undefined and are not passing on. Here are the associations:
Person.js:
models.person.Lead = models.person.hasMany(models.lead, {
onDelete: "CASCADE",
foreignKey: "person_id"
});
Lead.js:
models.lead.Person = models.lead.belongsTo(models.person, {foreignKey: 'person_id'});
models.lead.Sealant_customer = models.lead.hasOne(models.sealant_customer, {
onDelete: "CASCADE",
foreignKey: 'lead_id'
})
sealantCustomer.js:
models.sealant_customer.Lead = models.sealant_customer.belongsTo(models.lead);
The build function:
let sealantCustomer = models.sealant_customer.build({
address: body.address,
city: body.city,
roof_size: body.roofSize,
last_sealed: body.lastSealed,
existingSealant: body.existingSealant,
leaks_freq: body.leaksFrequency,
floor: body.floor,
elevator: body.elevator,
panels: body.panels,
home_type: body.homeType,
urgency: body.urgency,
next_step: body.nextStep,
more_info: body.moreInfo,
lead: {
site,
url: body.url,
date,
ip: body.ip,
person: {
name: body.name,
email: body.email,
phone: body.phone,
date,
city: body.city ? body.city : undefined,
address: body.address ? body.address : undefined,
}
}
}, {
include: [{
model: models.lead,
association: models.sealant_customer.Lead,
include: [{
model: models.person,
association: models.lead.Person
}]
}]
})
The outputted object is good except for the fact that lead_id and person_id are nulls (Each model has its own ID, but not the associated model's id). I also should note there are no validation errors and the data is good.
The library has a bug in the build function as far as I can tell. Same syntax with create worked perfectly.
In Sequelize v6, the association identifier in the include section is not valid. Otherwise, this build function should properly work.

Add MariaDB Constraint CHECK for JSON column using sequelize

I'm using MariaDB v10.2.11, and since v10.2.7 you can have JSON columns with a constraint that validates the JSON value of a row.
I want to add a that constraint through a Sequelize migration, but I don't know how to.
MariaDB JSON Documentation:
https://mariadb.com/resources/blog/json-mariadb-102
https://mariadb.com/kb/en/library/json-data-type/
I'm sharing the solution I came in this issue (https://github.com/sequelize/sequelize/issues/8767) (it's mine).
module.exports = {
up: (queryInterface, Sequelize) =>
queryInterface
.createTable('tableName', {
// id, other columns
column1: Sequelize.JSON,
// more columns
})
.then(() =>
queryInterface.addConstraint('tableName', ['column1'], {
type: 'check',
where: {
column1: Sequelize.literal('JSON_VALID(column1)'),
},
name: 'check_column1_has_valid_json',
}),
),
down: (queryInterface) => queryInterface.dropTable('tableName'),
};
Since createTable and addConstraint returns a promise, it's possible to chain multiple operations in one migration :)

Unique doesn't work on Node.js Sails.js "sails-mysql"

I've just started to get into the framework of Sails for Node. But it seems like I can't get the unique- requirements to work when adding for example users to the sails-mysql database. I can atm add unlimited number of new users with the same username and email.
From what I have read it should work, I did also try with sails-memory and there this exact code did work. Is it something I have missed out?
module.exports = {
attributes: {
username: {
type: 'string',
required: true,
unique: true
},
firstname: {
type: 'string',
required: true
},
lastname: {
type: 'string',
required: true
},
password: {
type: 'string',
required: true
},
birthdate: {
type: 'date',
required: true
},
email: {
type: 'email',
required: true,
unique: true
},
phonenumber: 'string',
// Create users full name automaticly
fullname: function(){
return this.firstname + ' ' + this.lastname;
}
}
};
As I mentioned above, this does work with the memory-storage. And now I have also tried with mongodb where it does work fins as well.
Got support from Sails.js on twitter: "it uses the db layer- suspect it's an issue with automigrations. Would you try in a new MySQL db?"
This answer did work, a new db and everything was just working :)
Just to add to this, since sails uses auto-migrations, if you initially start the server and your model does not have an attribute as unique, the table is built without the unique (index) switch. If you then change an existing attribute in the model to unique, the table will not be rebuilt the subsequent times you start the server.
One remedy during development is to set migrations in your model to drop like this:
module.exports = {
migrate: 'drop' // drops all your tables and then re-create them Note: You loose underlying.
attributes: {
...
}
};
That way, the db would be rebuilt each time you start the server. This would of course drop any existing data as well.