Sequelize - Association Many to Many - 3 Foreign Keys - mysql

I'm looking for some help on properly defining an association table with 3 foreign keys in a Sequelize model:
Situation: I'm building an email client, the relative models to this question are:
User Model (user records)
Thread Model (thread records for each new email-thread)
Folder Model (folders for default folders e.g. inbox, sent, etc and custom folders)
ThreadFolder Model (association linking a specific a) User Model, to a specific b) Thread Model, to a specific c) Folder Model)
Problem: My problem is the association model/table (ThreadFolder), I'm not able to create an association for all 3 tables in the ThreadFolder association Model.
First Attempt I'm able to create an association with Sequelize that allows the ThreadFolder model to create foreign keys for 2 of the three models above, but not all 3. Here is the association for that:
Thread.belongsToMany(Folder, { through: ThreadFolder, foreignKey: 'thread_id', otherKey: 'folder_id' })
Folder.belongsToMany(Thread, { through: ThreadFolder, foreignKey: 'folder_id', otherKey: 'thread_id' })
SQL Input Attempts:
user_id: 1 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 1 | thread_id: 1 | folder_id: 2 | Successful Insert
user_id: 1 | thread_id: 2 | folder_id: 1 | Failed Insert -- Error below...
user_id: 2 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 2 | thread_id: 1 | folder_id: 2 | Successful Insert
First Attempt's Error:
Executing:
INSERT INTO `iwantmail-core`.`thread_folders` (`user_id`, `deleted`, `archived`, `created_at`, `updated_at`, `thread_id`, `folder_id`) VALUES ('1', '0', '0', '2020-03-05 23:34:16', '2020-03-05 23:34:16', '30', '1');
Operation failed: There was an error while applying the SQL script to the database.
ERROR 1062: 1062: Duplicate entry '30-1' for key 'PRIMARY'
SQL Statement:
INSERT INTO `iwantmail-core`.`thread_folders` (`user_id`, `deleted`, `archived`, `created_at`, `updated_at`, `thread_id`, `folder_id`) VALUES ('1', '0', '0', '2020-03-05 23:34:16', '2020-03-05 23:34:16', '30', '1')
Second Attempt I can specify the association as shown below, to allow me to add records with different user_id and folder_id, however, if I use a different thread_id, I get an error shown below.
Folder.hasMany(ThreadFolder, { foreignKey: 'folder_id' })
ThreadFolder.belongsTo(Folder, { foreignKey: 'folder_id' })
Thread.belongsToMany(Folder, { through: ThreadFolder })
Thread.hasMany(ThreadFolder, { foreignKey: 'thread_id' })
ThreadFolder.belongsTo(Thread, { foreignKey: 'thread_id' })
User.hasMany(ThreadFolder, { foreignKey: 'user_id' })
ThreadFolder.belongsTo(User, { foreignKey: 'user_id' })
Folder.belongsToMany(User, { through: ThreadFolder })
SQL Input Attempts:
user_id: 1 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 1 | thread_id: 1 | folder_id: 2 | Successful Insert
user_id: 1 | thread_id: 2 | folder_id: 1 | Failed Insert -- Error below...
user_id: 2 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 2 | thread_id: 1 | folder_id: 2 | Successful Insert
Second Attempt's Error
Operation failed: There was an error while applying the SQL script to the database.
ERROR 1062: 1062: Duplicate entry '1-1' for key 'PRIMARY'
SQL Statement:
INSERT INTO `mail-core`.`thread_folders` (`user_id`, `thread_id`, `folder_id`) VALUES ('1', '2', '1')
Note that I'm basically trying to indicate that User#1 of Thread#1 are in Folder#1 and as soon as I try to indicate that User#1 of Thread#2 are in Folder#1, the above error occurs.
Help:
Could someone please point me towards the right direction / show how the association should be written to take into account the 3rd association?
Is there a different way to write this association all together so all 3 foreign keys are taken into account in the association table?
Thanks for any help/assistance in advance!
(Relative technologies used: MySQL, MySQL Workbench, Node 12.x, TypeScript, Serverless Framework)
EDIT: Made edits to the post, 2nd attempt was presented as a partial solution, after further testing, both 1st and second attempts fail when a 2nd user is added to the same thread and folder in the association table ThreadFolder.

After looking at your create statements I think you have defined associations thread_folders properly. In my opinion, your second attempt at defining association is correct.
You are getting an error while inserting records because your primary key is the combined key of two attributes namely thread_id and folder_id. Let us say that in your thread_folders table there is already record for thread_id 1 and folder_id 1 then you can not insert another record with thread_id 1 and folder_id 1.
If you remove the combined primary key of thread_id and folder_id then you will be able to insert the records that you want to insert in the thread_folders table.
I hope it helps!

Working solution ended up being a variance of the second attempt:
User.hasMany(ThreadFolder, { foreignKey: 'user_id' })
ThreadFolder.belongsTo(User, { foreignKey: 'user_id' })
Folder.hasMany(ThreadFolder, { foreignKey: 'folder_id' })
ThreadFolder.belongsTo(Folder, { foreignKey: 'folder_id' })
Thread.hasMany(ThreadFolder, { foreignKey: 'thread_id' })
ThreadFolder.belongsTo(Thread, { foreignKey: 'thread_id' })
Most of my model calls (based on how they were written before) will end up changing to start with the association table first e.g.
ThreadFolder.findAll({
where: {
user_id: 1,
folder_id: 1,
},
include: [
{
model: Thread,
include: [
'recipient',
'sender'
]
}
]
})
Hope this helps others that have attempted to do a multi-foreign key association table beyond 2 foreign keys, presumable this approach should work with any amount of foreign keys in the association table.

Related

updateOnDuplicate not effect

hello i want insert data with bulkCreate ex:
[
{
"typeId": 5,
"devEui": "0094E796CBFCFEF9",
"application_name": "Pressure No.10",
"createdAt": "2020-02-05T08:07:17.000Z",
"updatedAt": "2020-02-05T08:07:17.000Z"
}
]
and my sequelize code :
return models.sequelize.transaction(t=>{
return models.iot_nodes.bulkCreate(data,{
updateOnDuplicate: ["devEui",]
})
})
when i hit this code in first data that will be insert to db
my problem is when i hit again whit same data that not update, just insert in new row
iam using mysql db, laragon
log:
Executing (f202b84c-c5d8-4c67-954c-e22f96fb93d8): START TRANSACTION;
Executing (default): INSERT INTO `iot_nodes` (`id`,`typeId`,`devEui`,`application_name`,`createdAt`,`updatedAt`) VALUES (NULL,5,'0094E796CBFCFEF9','Pressure No.10','2020-02-05 08:07:17','2020-02-05 08:07:17') ON DUPLICATE KEY UPDATE `id`=VALUES(`id`),`devEui`=VALUES(`devEui`);
Executing (f202b84c-c5d8-4c67-954c-e22f96fb93d8): COMMIT;
It seems to fit this scenario based on the information. You want to update devEui field. updateOnDuplicate option:
Fields to update if row key already exists (on duplicate key update)?
So, the row key already exists means the table must have a unique key or the primary key is duplicated when you insert the data.
E.g.
import { sequelize } from '../../db';
import { Model, DataTypes } from 'sequelize';
class IotNode extends Model {}
IotNode.init(
{
typeId: {
type: DataTypes.INTEGER,
unique: true,
},
devEui: DataTypes.STRING,
application_name: DataTypes.STRING,
},
{ sequelize, modelName: 'iot_nodes' },
);
(async function test() {
try {
await sequelize.sync({ force: true });
const datas = [
{
typeId: 5,
devEui: '0094E796CBFCFEF9',
application_name: 'Pressure No.10',
createdAt: '2020-02-05T08:07:17.000Z',
updatedAt: '2020-02-05T08:07:17.000Z',
},
];
await IotNode.bulkCreate(datas, { updateOnDuplicate: ['devEui'] });
await IotNode.bulkCreate(datas, { updateOnDuplicate: ['devEui'] });
} catch (error) {
console.log(error);
} finally {
await sequelize.close();
}
})();
As you can see, I make the typeId unique and execute IotNode.bulkCreate twice. The generated SQL logs:
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("typeId") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("typeId") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
sequelize use the unique typeId field as the duplicate key. Check the rows in the database:
=# select * from iot_nodes;
id | typeId | devEui | application_name
----+--------+------------------+------------------
1 | 5 | 0094E796CBFCFEF9 | Pressure No.10
(1 row)
The data row is upserted as expected.
If we remove the unique: true from typeId field. sequelize will use primary key as the duplicate key. Take a look below generated SQL and data rows in the database:
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("id") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("id") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
=# select * from iot_nodes;
id | typeId | devEui | application_name
----+--------+------------------+------------------
1 | 5 | 0094E796CBFCFEF9 | Pressure No.10
2 | 5 | 0094E796CBFCFEF9 | Pressure No.10
(2 rows)

Sequelize Upsert is Creating instead of Updating

According to the documentation found here it states as follows
upsert(values, [options]) -> Promise.<created>
Insert or update a single row. An update will be executed if a row which matches the supplied values on either the primary key or a unique key is found. Note that the unique index must be defined in your sequelize model and not just in the table. Otherwise you may experience a unique constraint violation, because sequelize fails to identify the row that should be updated.
So my expectation is that upserting using a unique key should replace the existing value. However when my code runs instead of updating the existing database record, it adds a new one. What am I doing wrong?
here is a sample of my model
'use strict'
module.exports = (db, dataTypes) => {
const titanJob = db.define('titanJob', {
titanId: {
type: dataTypes.STRING,
allowNull: false,
unique: true
},
name: {
type: dataTypes.STRING,
allowNull: false
}
}, {
timestamps: true
})
return titanJob
}
and here is an example of my upsert
await asyncForEach(res.data.hits.hits, async es => {
const src = es._source
try {
await titanJob.upsert({
name: src.name,
titanId: src.id,
}, { titanId: src.id })
logger.debug(`[${file}] upsert successful`)
} catch (err) {
logger.warn(`[${file}] failed to save to database`)
logger.warn(`[${file}] ${err}`)
}
})
First you should add a unique index (constraint) to your table. The data you upserting should contain the field set of the unique index (constraint).
It should work. Here is an example using "sequelize": "^5.21.3":
index.ts:
import { Model, DataTypes } from 'sequelize';
import { sequelize } from '../../db';
import assert from 'assert';
class TitanJob extends Model {}
TitanJob.init(
{
titanId: {
type: DataTypes.STRING,
allowNull: false,
unique: true,
},
name: {
type: DataTypes.STRING,
allowNull: false,
},
},
{ sequelize, modelName: 'titanJob', timestamps: true },
);
(async function test() {
try {
await sequelize.sync({ force: true });
const datas = [
{ titanId: '1', name: 'programmer' },
{ titanId: '2', name: 'teacher' },
];
const jobs = await TitanJob.bulkCreate(datas);
assert.deepEqual(
jobs.map((job) => ({ titanId: job.id, name: job.name })),
datas,
'Should bulk create programmer and teacher datas',
);
const rval = await TitanJob.upsert({ titanId: '1', name: 'driver' }, { returning: true });
assert.equal(rval[0].titanId, '1', 'Should update the row which titanId is "1"');
} catch (error) {
console.log(error);
} finally {
await sequelize.close();
}
})();
Execution results:
{ POSTGRES_HOST: '127.0.0.1',
POSTGRES_PORT: '5430',
POSTGRES_PASSWORD: 'testpass',
POSTGRES_USER: 'testuser',
POSTGRES_DB: 'node-sequelize-examples' }
Executing (default): DROP TABLE IF EXISTS "titanJob" CASCADE;
Executing (default): DROP TABLE IF EXISTS "titanJob" CASCADE;
Executing (default): CREATE TABLE IF NOT EXISTS "titanJob" ("id" SERIAL , "titanId" VARCHAR(255) NOT NULL UNIQUE, "name" VARCHAR(255) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "updatedAt" TIMESTAMP WITH TIME ZONE NOT NULL, PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'titanJob' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): INSERT INTO "titanJob" ("id","titanId","name","createdAt","updatedAt") VALUES (DEFAULT,'1','programmer','2020-02-14 08:09:45.506 +00:00','2020-02-14 08:09:45.506 +00:00'),(DEFAULT,'2','teacher','2020-02-14 08:09:45.506 +00:00','2020-02-14 08:09:45.506 +00:00') RETURNING *;
Executing (default): CREATE OR REPLACE FUNCTION pg_temp.sequelize_upsert(OUT created boolean, OUT primary_key text) AS $func$ BEGIN INSERT INTO "titanJob" ("titanId","name","createdAt","updatedAt") VALUES ('1','driver','2020-02-14 08:09:45.524 +00:00','2020-02-14 08:09:45.524 +00:00') RETURNING "id" INTO primary_key; created := true; EXCEPTION WHEN unique_violation THEN UPDATE "titanJob" SET "titanId"='1',"name"='driver',"updatedAt"='2020-02-14 08:09:45.524 +00:00' WHERE ("id" IS NULL OR "titanId" = '1') RETURNING "id" INTO primary_key; created := false; END; $func$ LANGUAGE plpgsql; SELECT * FROM pg_temp.sequelize_upsert();
Executing (default): SELECT "id", "titanId", "name", "createdAt", "updatedAt" FROM "titanJob" AS "titanJob" WHERE "titanJob"."id" = '1';
No assertion fails. It works as expected. Check the data rows in the database:
node-sequelize-examples=# select * from "titanJob";
id | titanId | name | createdAt | updatedAt
----+---------+---------+----------------------------+----------------------------
2 | 2 | teacher | 2020-02-14 08:09:45.506+00 | 2020-02-14 08:09:45.506+00
1 | 1 | driver | 2020-02-14 08:09:45.506+00 | 2020-02-14 08:09:45.524+00
(2 rows)
source code: https://github.com/mrdulin/node-sequelize-examples/tree/master/src/examples/stackoverflow/59686743

jumping ID aunto increment after delete

When I was delete record Users, the ID primary key auto increment always jumping, I want ID auto increment is sequence again, this sample
| ID | UserName |
| 1 | Budi |
| 2 | Joko |
| 3 | Amir |
when I delete users Joko, then I add new other user, the ID number is jumping
| ID | UserName |
| 1 | Budi |
| 3 | Amir |
| 4 | Faris |
while I've browsing solution, I get some solution, but doesn't work.
here I've add modified file
config/app.php
'SQLkonek' => [
'className' => 'Cake\Database\Connection',
'driver' => 'Cake\Database\Driver\Mysql',
'persistent' => false,
'host' => 'localhost',
'username' => 'root',
'paassword' => ' ',
'database' => 'klinikucing',
'encoding' => 'utf8mb4',
'timezone' => ' ',
'cacheMetadata' => true
]
then I call modified above through
controller/UsersController.php
public function delete ($id = null)
{
$this->request->allowMethod(['post', 'delete']);
$kon = ConnectionManager::get('SQLkonek');
$user = $this->Users->get($id);
$stm = $kon->execute(array(
['SET #count = 0'],
['DELETE FROM users WHERE ID = :ID'],
['UPDATE users SET users.ID = #count:= #count + 1'],
['ALTER TABLE users AUTO_INCREMENT =1']
))
->fetchAll('assoc');
If($this->Users->$stm) {
$this->Flash->success(__('Users success delete.'));
} else {
$this->Flash->error(__('User delete failed, try again.'));
}
return $this->redirect(['action' => 'index']);
}
The error message was shown
Warning (2): PDO::prepare() expects parameter 1 to be string, array given [CORE\src\Database\Driver\Mysql.php, line 138]
Warning (512): Unable to emit headers. Headers sent in file=C:\xampp\htdocs\klinikucing\vendor\cakephp\cakephp\src\Error\Debugger.php line=853 [CORE\src\Http\ResponseEmitter.php, line 48]
Warning (2): Cannot modify header information - headers already sent by (output started at C:\xampp\htdocs\klinikucing\vendor\cakephp\cakephp\src\Error\Debugger.php:853) [CORE\src\Http\ResponseEmitter.php, line 148
Warning (2): Cannot modify header information - headers already sent by (output started at C:\xampp\htdocs\klinikucing\vendor\cakephp\cakephp\src\Error\Debugger.php:853) [CORE\src\Http\ResponseEmitter.php, line 181]
Argument 1 passed to Cake\Database\Statement\PDOStatement::__construct() must be an instance of PDOStatement or null, boolean given, called in C:\xampp\htdocs\klinikucing\vendor\cakephp\cakephp\src\Database\Driver\Mysql.php on line 139
Error in: ROOT\vendor\cakephp\cakephp\src\Database\Statement\PDOStatement.php, line 33
My CakePHP version is 3.7.2
I hopefull that someone can help me, thanx
Don't renumber. Accept there will be gaps in AI due to deleting an other cause (like aborted transaction, INSERT IGNORE etc.).
As you change PK values you change the FK relationships they have.
With ID of a type of INT UNSIGNED or BIGINT UNSIGNED you're not going to run out of ID ever.

Associating a reference table in sequelize

I have a couple of tables I'm trying to associate in Sequelize -- a jobaids_sections_messages table containing messages a user enters, and a jobaids_sections_messages_levels table which is a static reference used by the messages table. They're set up like so
this.jobaidMessage = sequelize.define('jobaids_sections_messages', {
message: Sequelize.STRING,
attuid: Sequelize.STRING,
level: Sequelize.INTEGER
}, {
paranoid: true
});
this.jobaidMessageLevel = sequelize.define('jobaids_sections_messages_levels', {
name: Sequelize.STRING
}, {
timestamps: false
});
The jobaids_sections_messages_levels table is set up like so:
| id | name |
| --- | -------- |
| 1 | Critical |
| 2 | Major |
| 3 | Warning |
| 4 | Info |
I want to make is so that when I create a new message, I can pass the level in as a key to the jobaids_sections_messages_levels table, and upon retrieving a message, I get the level back as
{
...
level: {
id: 2,
name: 'Major'
}
}
How should I set my associations up here? So far, I have
this.jobaidMessageLevel.belongsTo(this.jobaidMessage, {
foreignKey: 'level'
});
Though I'm not sure about the reversal of this association. Would it be a "many-to-one" relationship of some sorts?
Thank you!
Your message has a single level and technically your levels can have many messages. So simply stating that your message hasOne level will do the association needed. Then when you pull down a message and include the level, it will come back.
this.jobaidMessage.hasOne(this.jobaidMessageLevel, {
foreignKey: 'levelId'
});

Storing nested JSON with Cassandra

I am trying to store nested JSON object using composite tables in Cassandra and the nodejs bindings.
Let's say my data looks like this (friends and foes actually have more complex data structures than simple map):
{
id: 001,
name: 'luke',
lastname: 'skywalker',
friends: [
{ id: 002,
name: 'han',
lastname: 'solo' },
{ id: 003,
name: 'obiwan',
lastname: 'kenobi' },
{ id: 004,
name: 'leila',
lastname: 'skywalker' }
],
foes: [
{ id: 005,
name: 'dark',
lastname: 'vador' },
{ id: 006,
name: 'boba',
lastname: 'feet' }
]
}
From what I understood from composite keys (here: https://pkghosh.wordpress.com/2013/07/14/storing-nested-objects-in-cassandra-composite_columns/), I expected to store my data like this:
001 | luke | skywalker | friend:002:han:solo | friend:003:obiwan:kenobi | ... | foe:006:boba:feet
I created my table like this:
CREATE TABLE heroes (
id int,
name text,
lastname text,
friend_id int,
friend_name text,
friend_lastname text,
foe_id int,
foe_name text,
foe_lastname text,
PRIMARY KEY ((id, name, lastname), friend_id, foe_id)
);
And then run for each friends or foes:
client.execute(
'INSERT INTO heros (id, name, lastname, friend_id, friend_name, friend_lastname) VALUES (?, ?, ?, ?, ?)',
[001, 'luke', skywalker', 002, 'han', 'solo'],
function(err) {
//some code
}
)
Now, when runing the query 'SELECT * FROM heroes WHERE id=001' I expected to get only one row with all the friends or foes added as column.
Instead I get as many row as there as friends and foes. On top of this, a row for a friend looks like this:
{ rows:
[ { __columns: [Object],
id: 001,
name: 'luke',
lastname: 'skywalker',
friend_id: 002,
friend_name: 'han',
friend_lastname: 'solo',
foe_id: null,
foe_name: null,
foe_lastname: null } ],
meta:
// some data
}
I would have expected it not to have foe_* field at all.
Am I doing something wrong or is it the way cassandra handles composite items?
The result you are getting is be expected, because you have included friend and foes as part of the primary key. Hero has one to many association with friend and foe. In the blog post of mine that you are referring, I have used only the primary entity attributes as primary key. All the children entity attributes are mapped as dynamic columns.