Why postgresql's single insert statement is faster than mysql? - mysql

I wonder why postgresql's single "insert" statement is completely faster than MySQL's when autocommit is turned on? The following is the same code that I did on them.
Version:
MySQL: 5.6.10
PostgreSQL: PostgreSQL 9.3.2 on x86_64
Table definition:
MySQL:
CREATE TABLE `user` (
`username` char(36) NOT NULL,
`password` char(32) NOT NULL,
`register_time` datetime NOT NULL,
`mobile_phone` char(11) NOT NULL,
`is_admin` enum('yes','no') NOT NULL,
PRIMARY KEY (`username`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
PostgreSQL:
CREATE TYPE ytt_enum AS ENUM ('yes','no');
CREATE TABLE ytt."user" (
"username" char(36) NOT NULL,
"password" char(32) NOT NULL,
"register_time" timestamp NOT NULL,
"mobile_phone" char(11) NOT NULL,
"is_admin" ytt_enum NOT NULL,
PRIMARY KEY ("username")
) ;
Store functions:
MySQL:
DELIMITER $$
USE `t_girl`$$
DROP PROCEDURE IF EXISTS `sp_insert_user_simple`$$
CREATE DEFINER=`root`#`localhost` PROCEDURE `sp_insert_user_simple`(
IN f_input INT
)
BEGIN
DECLARE i INT DEFAULT 0;
WHILE i <= f_input
DO
INSERT INTO t_girl.user (`username`, `password`, register_time,mobile_phone,is_admin)
VALUES (UUID(),MD5(REPLACE(UUID(),'-','')),DATE_SUB(NOW(),INTERVAL CEIL(RAND()*40) DAY),CEIL(RAND()*10000)+13800000000,IF(TRUNCATE(RAND()*2,0)=1,'yes','no'));
SET i = i + 1;
END WHILE;
END$$
DELIMITER ;
PostgreSQL:
CREATE or replace function sp_insert_user_simple(
IN f_input INT
) returns void as
$ytt$
declare i int := 0;
v_username char(36);
v_password char(32);
v_register_time timestamp;
v_mobile_phone char(11);
v_is_admin ytt_enum;
BEGIN
WHILE i < f_input
loop
v_username := uuid_generate_v1();
v_password :=MD5(REPLACE(uuid_generate_v1()::text,'-',''));
v_register_time := to_timestamp((now() - '1 day'::interval*ceil(random()*40))::text,'yyyy-mm-dd HH24:MI:SS');
v_mobile_phone :=CEIL(RANDOM()*10000)+13800000000;
v_is_admin := (case TRUNC(RANDOM()*2) when 1 then 'yes' else'no' end)::ytt_enum;
INSERT INTO ytt.user (username, password, register_time,mobile_phone,is_admin)
VALUES (v_username,v_password,v_register_time,v_mobile_phone,v_is_admin);
i := i + 1;
END loop;
END;
$ytt$language plpgsql;
Parameters:
MySQL:
innodb_buffer_pool_size=32M
bulk_insert_buffer_size=20M
autocommit=on
PostgreSQL:
shared_memory=32M
effective_cache_size=20M
autocommit=on
Test result:
MySQL:
mysql> call sp_insert_user_simple(10000);
Query OK, 1 row affected (1 min 9.93 sec)
PostgreSQL:
ytt=# select sp_insert_user_simple(10000);
sp_insert_user_simple
-----------------------
(1 row)
Time: 1177.043 ms
The above test shows that MySQL's running time is 69.93 second but PostgreSQL's is only 1.17 second.
Any answer is appreciated. Thanks.

I think what's happening here is that MySQL's procedures may be doing a commit for each individual INSERT. In PostgreSQL the whole procedure commits at the end; procedures cannot run individual transactions. (I'm not totally sure if that's how MySQL's procedures behave with autocommit=off, but it seems to be from a quick look at the docs).
You should really be doing this INSERT as a single statement anyway, using INSERT ... SELECT:
CREATE or replace function sp_insert_user_simple(
IN f_input integer
) returns void AS $$
INSERT INTO ytt.user (username, password, register_time,mobile_phone,is_admin)
SELECT
uuid_generate_v1(),
MD5(REPLACE(uuid_generate_v1()::text,'-','')),
to_timestamp((now() - '1 day'::interval*ceil(random()*40))::text,'yyyy-mm-dd HH24:MI:SS'),
CEIL(RANDOM()*10000)+13800000000,
case TRUNC(RANDOM()*2) when 1 then 'yes' else'no' end
FROM generate_series(1,$1);
$$ LANGUAGE sql;
(I assume this is dummy user-data generation?).
Also, use char, not varchar. char is an awful data type and should be avoided. Also, consider using boolean for the is_admin column.

Try testing simple insert queries:
INSERT INTO ytt.user (username, password) VALUES ('a', 'b');
and loop it in a procedure, thus making time measurement more accurate. Avoid using other built-in functions (like rng and timestamp), since their perfomance can differ singnificantly on a large sample, unless, of course, you tested those first.

Related

How to build a conditional Procedure to operate (if-then-insert-else-update) on multiple tables using MySQL in phpMyAdmin

I use MySQL to manage a database.
I have 2 tables named IDENTITY and OPTIONS in a database named WIFI.
IDENTITY contains 2 fields: USERNAME and PASSWORD.
OPTIONS contains 3 fields: USERNAME, WIFI_SSID and WIFI_PASSWORD.
And there is a procedure accepting these variables as argument:
arg01_USERNAME
arg02_PASSWORD
arg03_WIFI_SSID
arg04_WIFI_PASSWORD
I want to build a procedure in phpMyAdmin to do this operation:
This procedure should check if the arg01_USERNAME and arg02_PASSWORD match the data in IDENTITY. If data not found, then do nothing. Else search for arg01_USERNAME in table OPTIONS and update WIFI_SSID and WIFI_PASSWORD with arg03_WIFI_SSID and arg04_WIFI_PASSWORD. If arg01_USERNAME not found in OPTIONS, then insert a new record into OPTIONS.
Here is the SQL query to define procedure:
CREATE DEFINER=`WIFI`#`localhost` PROCEDURE `INSERT`(IN `arg01_USERNAME` INT(20) UNSIGNED, IN `arg02_PASSWORD` VARCHAR(32), IN `arg03_WIFI_SSID` VARCHAR(32) CHARSET utf8, IN `arg04_WIFI_PASSWORD` VARCHAR(32) CHARSET utf8)
NO SQL
IF EXISTS (
SELECT COUNT(*)
FROM `WIFI`.`IDENTITY`
WHERE `USERNAME` = arg01_USERNAME AND `PASSWORD` = arg02_PASSWORD)
THEN
INSERT INTO `WIFI`.`OPTIONS` (
`USERNAME`,
`WIFI_SSID`,
`WIFI_PASSWORD`
) VALUES (
arg01_USERNAME,
arg03_WIFI_SSID,
arg04_WIFI_PASSWORD
)
ON DUPLICATE KEY UPDATE
`WIFI_SSID` = arg03_WIFI_SSID,
`WIFI_PASSWORD` = arg04_WIFI_PASSWORD
END IF;
ENGINE=InnoDB DEFAULT CHARSET=utf8
Could you please point me out what is wrong with this code?
When I try to define the procedure using this code, phpMyAdmin tells me there is a syntax error near "END IF" part of the code(error 1064).
Sorry for my poor English.
You need to redefined Delimiter to something else, for eg: $$, so that parser does not trigger query execution when it sees ;
At the end, redefine the Delimiter back to ;.
Try:
DELIMITER $$
CREATE DEFINER=`WIFI`#`localhost` PROCEDURE `INSERT`(IN `arg01_USERNAME` INT(20) UNSIGNED, IN `arg02_PASSWORD` VARCHAR(32), IN `arg03_WIFI_SSID` VARCHAR(32) CHARSET utf8, IN `arg04_WIFI_PASSWORD` VARCHAR(32) CHARSET utf8)
NO SQL
IF EXISTS (
SELECT COUNT(*)
FROM `WIFI`.`IDENTITY`
WHERE `USERNAME` = arg01_USERNAME AND `PASSWORD` = arg02_PASSWORD)
THEN
INSERT INTO `WIFI`.`OPTIONS` (
`USERNAME`,
`WIFI_SSID`,
`WIFI_PASSWORD`
) VALUES (
arg01_USERNAME,
arg03_WIFI_SSID,
arg04_WIFI_PASSWORD
)
ON DUPLICATE KEY UPDATE
`WIFI_SSID` = arg03_WIFI_SSID,
`WIFI_PASSWORD` = arg04_WIFI_PASSWORD;
END IF $$
DELIMITER ;

MySQL Stored procedure for updating two tables (2nd with auto increment value from 1st)

I want to update two tables at the same time in my database. One table is for groups, and the other table is for members of groups:
CREATE TABLE IF NOT EXISTS groups (
group_id INTEGER UNSIGNED AUTO_INCREMENT,
group_name VARCHAR(150) NOT NULL DEFAULT '',
group_created TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (group_id)
) ENGINE=InnoDB DEFAULT CHARACTER SET utf8;
CREATE TABLE IF NOT EXISTS group_members (
group_mem_user_id INTEGER UNSIGNED NOT NULL,
group_mem_group_id INTEGER UNSIGNED NOT NULL,
group_mem_role TINYINT DEFAULT 1,
group_mem_created TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT group_mem_pk PRIMARY KEY (group_mem_user_id, group_mem_group_id),
FOREIGN KEY (group_mem_user_id) REFERENCES user (user_id),
FOREIGN KEY (group_mem_group_id) REFERENCES groups (group_id)
) ENGINE=InnoDB DEFAULT CHARACTER SET utf8;
I want to use a stored procedure to create an entry in group and create an entry in group_members with the id that was just created for the group.
I know how to do this on the server (I have a java server and I'm using Spring's JdbcTemplate to make calls to the database) but I thought it would be better and more efficient to do this in a stored procedure.
The two individual queries are (im using prepared statements):
INSERT INTO groups (group_name) VALUES (?)
and
INSERT INTO group_members (group_mem_user_id, group_mem_group_id, group_mem_role) VALUES (?,?,?)
But I'm not sure how to merge these into one stored procedure.
DELIMITER //
DROP PROCEDURE IF EXISTS create_group //
CREATE PROCEDURE create_group(
#in/out here
)
BEGIN
#no idea
END //
DELIMITER ;
Ideally I would like it to return some value describing whether the operation was sucessful or not.
I use the following procedure:
DELIMITER //
DROP PROCEDURE IF EXISTS create_group //
CREATE PROCEDURE create_group(
IN create_group_group_name VARCHAR(150),
IN create_group_user_id INT
)
BEGIN
DECLARE EXIT HANDLER FOR SQLEXCEPTION, SQLWARNING
BEGIN
ROLLBACK;
END;
START TRANSACTION;
INSERT INTO groups (group_name) VALUES (create_group_group_name);
INSERT INTO group_members (group_mem_user_id, group_mem_group_id, group_mem_role) VALUES (create_group_user_id, LAST_INSERT_ID(), 2);
COMMIT;
END //
DELIMITER ;
In my server I use it like:
JdbcTemplate jt = new JdbcTemplate(DB.getDataSource(DB_USER));
int i = jt.update("CALL create_group (?,?)", new Object[] {groupName, userId});
if (i != 1)
throw new SQLException("Error creating group with name=" + groupName + " for userid=" + userId);
i == 1 if everything went well. Groups will never be created if the user is not added as a member (fixing the problem with my first iteration below).
OLD
(non-transactional, causes a problem if the second insert fails then the group is created with no members, it might work in some cases which is why I leave it here but it doesn't work for me)
The following procedure works. It does not return anything and I am just using the fact that the procedure completes without error to assume that it was all ok.
DELIMITER //
DROP PROCEDURE IF EXISTS create_group //
CREATE PROCEDURE create_group(
IN create_group_group_name VARCHAR(150),
IN create_group_user_id INT
)
BEGIN
INSERT INTO groups (group_name) VALUES (create_group_group_name);
INSERT INTO group_members (group_mem_user_id, group_mem_group_id, group_mem_role) VALUES (create_group_user_id, LAST_INSERT_ID(), 2);
END //
DELIMITER ;

Using stored procedure in MySQL

I have a weather database in MySQL. My database get data from arduino, but sometimes the arduino have some error and send error value in my database. I want to make a stored procedure to reject this error. I want using if then in stored procedure. Example if temperature < 20 then MySQL reject this data. Is it possible? Help me please with the coding
this is my table
CREATE TABLE `cuaca_maritim`.`weather_data` (
`idweather` INT(10) NOT NULL,
`temperature` DECIMAL(4,2) NOT NULL,
`HUMID` DECIMAL(4,2) NOT NULL,
`AIRPRESSURE` DECIMAL(6,2) NOT NULL,
`WIND` DECIMAL(4,2) NOT NULL,
PRIMARY KEY (`idweather`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_bin;
Perhaps you just want a view:
create view good_weather_data as
select wd.*
from weather_data
where temperature >= 20;
I don't really see why a stored procedure would be necessary. You might want a trigger that rejects invalid data values when they are loaded.
I think you can use the following stored procedure to do your work.
DELIMITER #
CREATE OR REPLACE PROCEDURE add_weather_data (IN temp INT)
proc_main: BEGIN
IF (temp > 20) THEN
INSERT INTO weather_data(temperature) VALUES (temp);
END IF;
END proc_main #
DELIMITER;
Here I have only consider the temp you can do the same for other parameters also

Operand should contain 1 column(s)

Getting a Operand should contain 1 column(s) mysql error whenever I try to insert into the table sets.
I googled and found a hoard of similar questions but they are always pin point specific to solving their immediate problem. I have mysql 5.6 by the way. I am allowed multiple TIMESTAMPS.
Here is my code:
INSERT INTO `sets` (`tabler_name`) VALUES ("leads_auto");
Here is my table:
CREATE TABLE IF NOT EXISTS `lms`.`sets` (
`set_id` BIGINT NOT NULL AUTO_INCREMENT,
`on_off` SMALLINT NOT NULL DEFAULT 0,
`tabler_name` VARCHAR(45) NULL,
`origin_date` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
`last_modified_date` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
`original_count` INT NULL,
`current_count` INT NULL,
`source_type` VARCHAR(45) NULL,
`source` VARCHAR(45) NULL,
`method` VARCHAR(45) NULL,
`agent` VARCHAR(45) NULL,
`dupes` INT NULL,
`bads` INT NULL,
`aged` INT NULL COMMENT 'This table keeps track of the record sets that enter the system. Example: a set of leads imported into the database.',
PRIMARY KEY (`set_id`)
) ENGINE = InnoDB;
Stored Procedure:
DELIMITER //
CREATE PROCEDURE `lms`.`leads_to_bak` ()
BEGIN
SET #table1 = (SELECT `tabler_name` FROM sets WHERE on_off=0 LIMIT 1);
SET #table2 = CONCAT(#table1, '_bak');
SET #SQL1 = CONCAT('INSERT INTO ',#table2, '(', (SELECT
REPLACE(GROUP_CONCAT(COLUMN_NAME), 'lead_id,', '') FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #table2), ')', ' SELECT ', (SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME), 'lead_id,', '') FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #table1), ' FROM ', #table1);
PREPARE stmt FROM #sql1;
EXECUTE stmt;
END//
DELIMITER ;
USE `lms`;
Trigger
DELIMITER $$
USE `lms`$$
CREATE TRIGGER `lms`.`after_insert_into_leads`
AFTER INSERT ON `sets` FOR EACH ROW
BEGIN
IF (SELECT * FROM sets WHERE on_off=0 LIMIT 1) THEN
CALL lms.leads_to_bak();
END IF;
END$$
DELIMITER ;
USE `lms`;
I don't see anything wrong with my routines. Removing the routines and trigger seems to make the problem go away.
In your trigger, did you mean to put EXISTS after IF? Like this:
CREATE TRIGGER `lms`.`after_insert_into_leads`
AFTER INSERT ON `sets` FOR EACH ROW
BEGIN
IF EXISTS (SELECT * FROM sets WHERE on_off=0 LIMIT 1) THEN
CALL lms.leads_to_bak();
END IF;
END$$
Escape your table name, it seems to be a reserved function. I'm not sure if you've defined one locally.
INSERT INTO `sets` (tabler_name) VALUES ("leads_auto");
Also, you can't have two timestamp fields in a single database afaik. Change one of the two timestamps to a DATETIME field if it caused you issues as well
Besides escaping the field name in your INSERT-statement, it cannot be improved very much. But it doesn't generate any error in my test enviroment. Is this really the exact statement throwing you an error?
However, there's a slight problem in your table definition, it will throw you an
Incorrect table definition; there can be only one TIMESTAMP column
with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause.
As the error message indicates, you can only use one timestamp column with CURRENT_TIMESTAMP, if you need more than one, you can do this using a trigger.

Using a check contraint in MySQL for controlling string length

I'm tumbled with a problem!
I've set up my first check constraint using MySQL, but unfortunately I'm having a problem. When inserting a row that should fail the test, the row is inserted anyway.
The structure:
CREATE TABLE user (
id INT UNSIGNED NOT NULL AUTO_INCREMENT,
uname VARCHAR(10) NOT NULL,
fname VARCHAR(50) NOT NULL,
lname VARCHAR(50) NOT NULL,
mail VARCHAR(50) NOT NULL,
PRIMARY KEY (id),
CHECK (LENGTH(fname) > 30)
);
The insert statement:
INSERT INTO user VALUES (null, 'user', 'Fname', 'Lname', 'mail#me.now');
The length of the string in the fname column should be too short, but it's inserted anyway.
I'm pretty sure I'm missing something basic here.
MySQL doesn't enforce CHECK constraints, on any engine.
Which leads me to ask, why would you declare the fname column as VARCHAR(50), but want to enforce that it can only be 30 characters long?
That said, the only alternative is to use a trigger:
CREATE TRIGGER t1 BEFORE INSERT ON user
FOR EACH ROW BEGIN
DECLARE numLength INT;
SET numLength = (SELECT LENGTH(NEW.fname));
IF (numLength > 30) THEN
SET NEW.col = 1/0;
END IF;
END;
As mentioned above you have to use a trigger, MySQL doesn't support check, also when you have multiple statements inside your trigger block, like declaring variables or control flows, you need to start it with begin and end and enclose your trigger inside two delimiters:
Note: If you use MariaDB use // after the first delimiter and before the second delimiter, otherwise if you use MySQL use $$ instead.
delimiter //
create trigger `user_insert_trigger` before insert on `user` for each row
begin
declare maximumFnameLength int unsigned;
declare fNameLength int unsigned;
set maximumFnameLength = 30;
set fNameLength = (select length(new.fNameLength));
if (fNameLength > maximumFnameLength) then
signal sqlstate '45000'
set message_text = 'First name is more than 30 characters long.';
end if;
end
//
delimiter ;