updateOnDuplicate not effect - mysql

hello i want insert data with bulkCreate ex:
[
{
"typeId": 5,
"devEui": "0094E796CBFCFEF9",
"application_name": "Pressure No.10",
"createdAt": "2020-02-05T08:07:17.000Z",
"updatedAt": "2020-02-05T08:07:17.000Z"
}
]
and my sequelize code :
return models.sequelize.transaction(t=>{
return models.iot_nodes.bulkCreate(data,{
updateOnDuplicate: ["devEui",]
})
})
when i hit this code in first data that will be insert to db
my problem is when i hit again whit same data that not update, just insert in new row
iam using mysql db, laragon
log:
Executing (f202b84c-c5d8-4c67-954c-e22f96fb93d8): START TRANSACTION;
Executing (default): INSERT INTO `iot_nodes` (`id`,`typeId`,`devEui`,`application_name`,`createdAt`,`updatedAt`) VALUES (NULL,5,'0094E796CBFCFEF9','Pressure No.10','2020-02-05 08:07:17','2020-02-05 08:07:17') ON DUPLICATE KEY UPDATE `id`=VALUES(`id`),`devEui`=VALUES(`devEui`);
Executing (f202b84c-c5d8-4c67-954c-e22f96fb93d8): COMMIT;

It seems to fit this scenario based on the information. You want to update devEui field. updateOnDuplicate option:
Fields to update if row key already exists (on duplicate key update)?
So, the row key already exists means the table must have a unique key or the primary key is duplicated when you insert the data.
E.g.
import { sequelize } from '../../db';
import { Model, DataTypes } from 'sequelize';
class IotNode extends Model {}
IotNode.init(
{
typeId: {
type: DataTypes.INTEGER,
unique: true,
},
devEui: DataTypes.STRING,
application_name: DataTypes.STRING,
},
{ sequelize, modelName: 'iot_nodes' },
);
(async function test() {
try {
await sequelize.sync({ force: true });
const datas = [
{
typeId: 5,
devEui: '0094E796CBFCFEF9',
application_name: 'Pressure No.10',
createdAt: '2020-02-05T08:07:17.000Z',
updatedAt: '2020-02-05T08:07:17.000Z',
},
];
await IotNode.bulkCreate(datas, { updateOnDuplicate: ['devEui'] });
await IotNode.bulkCreate(datas, { updateOnDuplicate: ['devEui'] });
} catch (error) {
console.log(error);
} finally {
await sequelize.close();
}
})();
As you can see, I make the typeId unique and execute IotNode.bulkCreate twice. The generated SQL logs:
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("typeId") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("typeId") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
sequelize use the unique typeId field as the duplicate key. Check the rows in the database:
=# select * from iot_nodes;
id | typeId | devEui | application_name
----+--------+------------------+------------------
1 | 5 | 0094E796CBFCFEF9 | Pressure No.10
(1 row)
The data row is upserted as expected.
If we remove the unique: true from typeId field. sequelize will use primary key as the duplicate key. Take a look below generated SQL and data rows in the database:
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("id") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
Executing (default): INSERT INTO "iot_nodes" ("id","typeId","devEui","application_name") VALUES (DEFAULT,5,'0094E796CBFCFEF9','Pressure No.10') ON CONFLICT ("id") DO UPDATE SET "devEui"=EXCLUDED."devEui" RETURNING *;
=# select * from iot_nodes;
id | typeId | devEui | application_name
----+--------+------------------+------------------
1 | 5 | 0094E796CBFCFEF9 | Pressure No.10
2 | 5 | 0094E796CBFCFEF9 | Pressure No.10
(2 rows)

Related

How to update values for all rows having null value | Prisma | Type script | MySQL

I have prisma schema like below
model t1 {
id Int #id #default(autoincrement())
title String
previousNodeId Int? #unique
}
and I wanted to update title of rows who have null values in previousNodeId
I have written query
await prisma.t1.update({
where: { previousNodeId: null },
data: {
title:"update title"
}
});
but I am not able to update it.
Getting error Type 'null' is not assignable to type 'number | undefined'.
The expected type comes from property 'previousNodeId' which is declared here on type 'T1WhereUniqueInput'
I am able to get all rows using findMany.
await prisma.t1.findMany({
where: { previousNodeId: null },
});
Result of findMany
[
{
id: 3,
title: 'demo t1 1',
previousNodeId: null
}
]

Selecting data from table with UUID as binary()

I have an AWS RDS Aurora MySQL connection. Following are the steps that I'm following to operate create, insert & fetch the data:
create table:
CREATE TABLE user (
user_id BINARY(16) PRIMARY KEY NOT NULL,
first_name VARCHAR(25) NOT NULL,
last_name VARCHAR(25) NOT NULL,
);
insert into table:
INSERT INTO user VALUES (
UNHEX(REPLACE("9a9cbfga-7426-471a-af27-31be7tb53ii2", "-","")),
"Jon",
"Smith"
);
fetch from table:
SELECT * FROM user WHERE user_id = UNHEX(REPLACE("9a9cbfga-7426-471a-af27-31be7tb53ii2", "-",""));
The handler function that triggers my function to fetch the data goes like:
module.exports.fetchUser = async (event, context, callback) => {
const params = {
resourceArn: 'arn:aws:rds:*********:cluster:************',
secretArn: 'arn:aws:secretsmanager:*************',
sql: 'SELECT * FROM user WHERE user_id = UNHEX(REPLACE("9a9cbfga-7426-471a-af27-31be7tb53ii2", "-",""))',
database: 'dev_db1',
continueAfterTimeout: true,
includeResultMetadata: true
}
try {
const db_res = await rdsDataService.executeStatement(params).promise();
const response = {
body: {
message: 'Data fetched',
data: JSON.stringify(db_res, null, 2)
}
};
callback(null, response);
} catch (error) {
callback(null, error)
}
};
Executing the http data API endpoint for the same results me in this:
{
"body": {
"message": "Data fetched",
"data": [
[
{
"blobValue": {
"type": "Buffer",
"data": [
202,
136,
22,
206,
45,
214,
75,
233,
156,
28,
163,
223,
186,
115,
89,
107
]
}
},
{
"stringValue": "Jon"
},
{
"stringValue": "Smith"
},
]
]
}
}
All I need is to fetch the complete details of all the users (for now I've inserted just one record for ease) in the table user and show the exact user_id instead of the blobValue. Not sure how to convert this UUID binary into redable form like 9a9cbfga7426471aaf2731be7tb53ii2 or 9a9cbfga-7426-471a-af27-31be7tb53ii2 in the response.
I'am bit new to handling queries in MYSQL & Aurora, any help to get me through this will be really appretiated. Thanks in advance.
Mysql 8 has a function for that as https://dev.mysql.com/doc/refman/8.0/en/miscellaneous-functions.html#function_bin-to-uuid
and https://dev.mysql.com/doc/refman/8.0/en/miscellaneous-functions.html#function_uuid-to-bin
But you have to do it manually.
As i can't insert your data anywhere the select will do
CREATE TABLE user (
user_id BINARY(16) PRIMARY KEY NOT NULL,
first_name VARCHAR(25) NOT NULL,
last_name VARCHAR(25) NOT NULL
)
INSERT INTO user VALUES (UNHEX(REPLACE("3f06af63-a93c-11e4-9797-00505690773f", "-","")),
"Jon",
"Smith");
SELECT * from user where user_id = UNHEX(REPLACE("3f06af63-a93c-11e4-9797-00505690773f","-",""))
user_id | first_name | last_name
:------ | :--------- | :--------
null | Jon | Smith
SELECT CONCAT(SUBSTRING(HEX(user_id),1,8),'-',
SUBSTRING(HEX(user_id),9,4),'-',
SUBSTRING(HEX(user_id),13,4),'-',
SUBSTRING(HEX(user_id),17,12)) user_id,first_name,last_name FROM user
user_id | first_name | last_name
:------------------------------ | :--------- | :--------
3F06AF63-A93C-11E4-979700505690 | Jon | Smith
db<>fiddle here
But as you need the text anyway, keep the original so you don't need to unhex hex the data again
CREATE TABLE user (
user_id BINARY(16) PRIMARY KEY NOT NULL,
first_name VARCHAR(25) NOT NULL,
last_name VARCHAR(25) NOT NULL,
userid_txt Varchar(3&)
);

How to fetch a serial number from a mysql table using Sequelize in nodeJS?

I have a mysql Query:-
SELECT #a:=#a+1 serial_number,rule FROM pledges,(SELECT #a:= 0) AS a;
This gives me a serial number along with the rule from the table.
How can I do that in Sequelize?
This is the query I wrote in the model which gives me id and rule:-
Pledge.getPledgeList = function(lang_code='EN') {
return this.findAll({
attributes:['id','rule'],
where: {
status:'a',
deleted_at:null
},
include:[
{ association: 'local', where: {lang_code:lang_code} ,required: false},
]
})
}

Sequelize - Association Many to Many - 3 Foreign Keys

I'm looking for some help on properly defining an association table with 3 foreign keys in a Sequelize model:
Situation: I'm building an email client, the relative models to this question are:
User Model (user records)
Thread Model (thread records for each new email-thread)
Folder Model (folders for default folders e.g. inbox, sent, etc and custom folders)
ThreadFolder Model (association linking a specific a) User Model, to a specific b) Thread Model, to a specific c) Folder Model)
Problem: My problem is the association model/table (ThreadFolder), I'm not able to create an association for all 3 tables in the ThreadFolder association Model.
First Attempt I'm able to create an association with Sequelize that allows the ThreadFolder model to create foreign keys for 2 of the three models above, but not all 3. Here is the association for that:
Thread.belongsToMany(Folder, { through: ThreadFolder, foreignKey: 'thread_id', otherKey: 'folder_id' })
Folder.belongsToMany(Thread, { through: ThreadFolder, foreignKey: 'folder_id', otherKey: 'thread_id' })
SQL Input Attempts:
user_id: 1 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 1 | thread_id: 1 | folder_id: 2 | Successful Insert
user_id: 1 | thread_id: 2 | folder_id: 1 | Failed Insert -- Error below...
user_id: 2 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 2 | thread_id: 1 | folder_id: 2 | Successful Insert
First Attempt's Error:
Executing:
INSERT INTO `iwantmail-core`.`thread_folders` (`user_id`, `deleted`, `archived`, `created_at`, `updated_at`, `thread_id`, `folder_id`) VALUES ('1', '0', '0', '2020-03-05 23:34:16', '2020-03-05 23:34:16', '30', '1');
Operation failed: There was an error while applying the SQL script to the database.
ERROR 1062: 1062: Duplicate entry '30-1' for key 'PRIMARY'
SQL Statement:
INSERT INTO `iwantmail-core`.`thread_folders` (`user_id`, `deleted`, `archived`, `created_at`, `updated_at`, `thread_id`, `folder_id`) VALUES ('1', '0', '0', '2020-03-05 23:34:16', '2020-03-05 23:34:16', '30', '1')
Second Attempt I can specify the association as shown below, to allow me to add records with different user_id and folder_id, however, if I use a different thread_id, I get an error shown below.
Folder.hasMany(ThreadFolder, { foreignKey: 'folder_id' })
ThreadFolder.belongsTo(Folder, { foreignKey: 'folder_id' })
Thread.belongsToMany(Folder, { through: ThreadFolder })
Thread.hasMany(ThreadFolder, { foreignKey: 'thread_id' })
ThreadFolder.belongsTo(Thread, { foreignKey: 'thread_id' })
User.hasMany(ThreadFolder, { foreignKey: 'user_id' })
ThreadFolder.belongsTo(User, { foreignKey: 'user_id' })
Folder.belongsToMany(User, { through: ThreadFolder })
SQL Input Attempts:
user_id: 1 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 1 | thread_id: 1 | folder_id: 2 | Successful Insert
user_id: 1 | thread_id: 2 | folder_id: 1 | Failed Insert -- Error below...
user_id: 2 | thread_id: 1 | folder_id: 1 | Successful Insert
user_id: 2 | thread_id: 1 | folder_id: 2 | Successful Insert
Second Attempt's Error
Operation failed: There was an error while applying the SQL script to the database.
ERROR 1062: 1062: Duplicate entry '1-1' for key 'PRIMARY'
SQL Statement:
INSERT INTO `mail-core`.`thread_folders` (`user_id`, `thread_id`, `folder_id`) VALUES ('1', '2', '1')
Note that I'm basically trying to indicate that User#1 of Thread#1 are in Folder#1 and as soon as I try to indicate that User#1 of Thread#2 are in Folder#1, the above error occurs.
Help:
Could someone please point me towards the right direction / show how the association should be written to take into account the 3rd association?
Is there a different way to write this association all together so all 3 foreign keys are taken into account in the association table?
Thanks for any help/assistance in advance!
(Relative technologies used: MySQL, MySQL Workbench, Node 12.x, TypeScript, Serverless Framework)
EDIT: Made edits to the post, 2nd attempt was presented as a partial solution, after further testing, both 1st and second attempts fail when a 2nd user is added to the same thread and folder in the association table ThreadFolder.
After looking at your create statements I think you have defined associations thread_folders properly. In my opinion, your second attempt at defining association is correct.
You are getting an error while inserting records because your primary key is the combined key of two attributes namely thread_id and folder_id. Let us say that in your thread_folders table there is already record for thread_id 1 and folder_id 1 then you can not insert another record with thread_id 1 and folder_id 1.
If you remove the combined primary key of thread_id and folder_id then you will be able to insert the records that you want to insert in the thread_folders table.
I hope it helps!
Working solution ended up being a variance of the second attempt:
User.hasMany(ThreadFolder, { foreignKey: 'user_id' })
ThreadFolder.belongsTo(User, { foreignKey: 'user_id' })
Folder.hasMany(ThreadFolder, { foreignKey: 'folder_id' })
ThreadFolder.belongsTo(Folder, { foreignKey: 'folder_id' })
Thread.hasMany(ThreadFolder, { foreignKey: 'thread_id' })
ThreadFolder.belongsTo(Thread, { foreignKey: 'thread_id' })
Most of my model calls (based on how they were written before) will end up changing to start with the association table first e.g.
ThreadFolder.findAll({
where: {
user_id: 1,
folder_id: 1,
},
include: [
{
model: Thread,
include: [
'recipient',
'sender'
]
}
]
})
Hope this helps others that have attempted to do a multi-foreign key association table beyond 2 foreign keys, presumable this approach should work with any amount of foreign keys in the association table.

Sequelize Upsert is Creating instead of Updating

According to the documentation found here it states as follows
upsert(values, [options]) -> Promise.<created>
Insert or update a single row. An update will be executed if a row which matches the supplied values on either the primary key or a unique key is found. Note that the unique index must be defined in your sequelize model and not just in the table. Otherwise you may experience a unique constraint violation, because sequelize fails to identify the row that should be updated.
So my expectation is that upserting using a unique key should replace the existing value. However when my code runs instead of updating the existing database record, it adds a new one. What am I doing wrong?
here is a sample of my model
'use strict'
module.exports = (db, dataTypes) => {
const titanJob = db.define('titanJob', {
titanId: {
type: dataTypes.STRING,
allowNull: false,
unique: true
},
name: {
type: dataTypes.STRING,
allowNull: false
}
}, {
timestamps: true
})
return titanJob
}
and here is an example of my upsert
await asyncForEach(res.data.hits.hits, async es => {
const src = es._source
try {
await titanJob.upsert({
name: src.name,
titanId: src.id,
}, { titanId: src.id })
logger.debug(`[${file}] upsert successful`)
} catch (err) {
logger.warn(`[${file}] failed to save to database`)
logger.warn(`[${file}] ${err}`)
}
})
First you should add a unique index (constraint) to your table. The data you upserting should contain the field set of the unique index (constraint).
It should work. Here is an example using "sequelize": "^5.21.3":
index.ts:
import { Model, DataTypes } from 'sequelize';
import { sequelize } from '../../db';
import assert from 'assert';
class TitanJob extends Model {}
TitanJob.init(
{
titanId: {
type: DataTypes.STRING,
allowNull: false,
unique: true,
},
name: {
type: DataTypes.STRING,
allowNull: false,
},
},
{ sequelize, modelName: 'titanJob', timestamps: true },
);
(async function test() {
try {
await sequelize.sync({ force: true });
const datas = [
{ titanId: '1', name: 'programmer' },
{ titanId: '2', name: 'teacher' },
];
const jobs = await TitanJob.bulkCreate(datas);
assert.deepEqual(
jobs.map((job) => ({ titanId: job.id, name: job.name })),
datas,
'Should bulk create programmer and teacher datas',
);
const rval = await TitanJob.upsert({ titanId: '1', name: 'driver' }, { returning: true });
assert.equal(rval[0].titanId, '1', 'Should update the row which titanId is "1"');
} catch (error) {
console.log(error);
} finally {
await sequelize.close();
}
})();
Execution results:
{ POSTGRES_HOST: '127.0.0.1',
POSTGRES_PORT: '5430',
POSTGRES_PASSWORD: 'testpass',
POSTGRES_USER: 'testuser',
POSTGRES_DB: 'node-sequelize-examples' }
Executing (default): DROP TABLE IF EXISTS "titanJob" CASCADE;
Executing (default): DROP TABLE IF EXISTS "titanJob" CASCADE;
Executing (default): CREATE TABLE IF NOT EXISTS "titanJob" ("id" SERIAL , "titanId" VARCHAR(255) NOT NULL UNIQUE, "name" VARCHAR(255) NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "updatedAt" TIMESTAMP WITH TIME ZONE NOT NULL, PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'titanJob' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): INSERT INTO "titanJob" ("id","titanId","name","createdAt","updatedAt") VALUES (DEFAULT,'1','programmer','2020-02-14 08:09:45.506 +00:00','2020-02-14 08:09:45.506 +00:00'),(DEFAULT,'2','teacher','2020-02-14 08:09:45.506 +00:00','2020-02-14 08:09:45.506 +00:00') RETURNING *;
Executing (default): CREATE OR REPLACE FUNCTION pg_temp.sequelize_upsert(OUT created boolean, OUT primary_key text) AS $func$ BEGIN INSERT INTO "titanJob" ("titanId","name","createdAt","updatedAt") VALUES ('1','driver','2020-02-14 08:09:45.524 +00:00','2020-02-14 08:09:45.524 +00:00') RETURNING "id" INTO primary_key; created := true; EXCEPTION WHEN unique_violation THEN UPDATE "titanJob" SET "titanId"='1',"name"='driver',"updatedAt"='2020-02-14 08:09:45.524 +00:00' WHERE ("id" IS NULL OR "titanId" = '1') RETURNING "id" INTO primary_key; created := false; END; $func$ LANGUAGE plpgsql; SELECT * FROM pg_temp.sequelize_upsert();
Executing (default): SELECT "id", "titanId", "name", "createdAt", "updatedAt" FROM "titanJob" AS "titanJob" WHERE "titanJob"."id" = '1';
No assertion fails. It works as expected. Check the data rows in the database:
node-sequelize-examples=# select * from "titanJob";
id | titanId | name | createdAt | updatedAt
----+---------+---------+----------------------------+----------------------------
2 | 2 | teacher | 2020-02-14 08:09:45.506+00 | 2020-02-14 08:09:45.506+00
1 | 1 | driver | 2020-02-14 08:09:45.506+00 | 2020-02-14 08:09:45.524+00
(2 rows)
source code: https://github.com/mrdulin/node-sequelize-examples/tree/master/src/examples/stackoverflow/59686743