I was wondering if it was possible to speed up this MySQL query. Currently, I am calling it 20,000+ times, and it takes a while to run (a while being about 10 to 20 minutes).
Here is the basic table layout:
db1:
INT(11) id
VARCHAR(45) col1
VARCHAR(100) col2
VARCHAR(100) col3
db2:
VARCHAR(45) id
db3:
VARCHAR(45) fk_db2
INT(11) fk_db1
Here is the stored routine:
DELIMITER $$
CREATE DEFINER=`root`#`localhost` PROCEDURE `proc`(IN fk VARCHAR(45),
IN a VARCHAR(45),
IN b VARCHAR(100),
IN c VARCHAR(65))
BEGIN
SET #id=0;
SELECT id INTO #id FROM db1 WHERE db1.col1=a
AND db1.col2=b
AND db1.col3=c;
IF #id=0 THEN
INSERT INTO db1 (col1, col2, col3)
VALUES (a, b, c);
SELECT LAST_INSERT_ID() INTO #id;
END IF;
-- Association table for db2 and db1.
INSERT IGNORE INTO db3 (fk_db1, fk_db2)
VALUES(#id, fk);
END
The main point of this routine, is I want to get the ID of a specific record, or create one if it doesn't exist. Then I want to associate the fk passed in and the ID that I just found out. I'm sure there is a MySQL one-liner for this, but I have been unable to figure it out.
Any help is greatly appreciated!!
Thank you!
By the way, the names of columns are much better in the actual database, but I can't share the names with you all.
Actually, unless I'm missing something, that doesn't look all that slow. Can you define "a while" in "takes a while"?
Do you have an index on col1/col2/col3?
How exactly are you calling the procedure in your application? You are reusing the database connections, right?
Can you bundle multiple calls into a single transaction if you're not already?
Related
MySQL has LAST_INSERT_ID(); function which gets the last insert.
But this is not safe: If i run some query then get LAST_INSERT_ID() and between the two another query is executed I can get the wrong id.
This can happen in multiple threads using the same connection, or using pconnect (persistend connection for multiple users)
Is there safe method for getting the ID that i want 100% ?
Thanks
Store procedure may help in the case:
Create table test (id int AUTO_INCREMENT primary key, name varchar(50) )
Store procedure as:
delimiter $$
DROP PROCEDURE IF EXISTS InserData$$
CREATE PROCEDURE InserData(IN _Name VARCHAR(50))
BEGIN
START TRANSACTION;
INSERT INTO test(name) VALUES (_Name);
SELECT LAST_INSERT_ID() AS InsertID;
COMMIT;
END
Call the stored procedure using
CALL InserData('TESTER')
Give it a try as we have transaction statement but it can't ensure the value in multi threaded environment.
The link Mysql thread safety of last_insert_id explain it will work based on per connection model.
Is using SELECT Max(ID) FROM table safer than using SELECT last_insert_id(), where they run as 2 separate queries?
According to your question the table must have a primary key.
So you can get last record from MAX(ID)
I have two columns in my DB table, where the first one is auto incremented ID. In the second one I would like to have that ID mirrored. (I know it sound like design error, but I really need it)
Is it possible to configure it that way?
If you're using MySql 5.7.6 or later you can define the second id as a virtual column like this:
CREATE TABLE my_table (
id INT UNSIGNED AUTO_INCREMENT,
id_mirrored INT UNSIGNED AS (id)
);
This way the id_mirrored column isn't really stored in your database but instead it's evaluated when the row is read.
If you're using an earlier version of MySql creating a view is probably your best option. Views are basically virtual tables.
CREATE VIEW my_view AS
SELECT
t.id AS id,
t.id AS id_mirrored
FROM my_table t
Third option is to define the id_mirrored as a real column and add a trigger to give it it's value. The way to do this has already been described in other answers.
use TRIGGER
DELIMITER |: # switch delimiter to prevent execution of ;
CREATE TRIGGER `copy_id2c_name` BEFORE INSERT ON tb_name
FOR EACH ROW BEGIN
SELECT AUTO_INCREMENT INTO #AI FROM information_schema.tables WHERE table_schema = 'db_name' and table_name = 'tb_name';
set NEW.c_name = #AI;
END;
|: # execute code
DELIMITER ; // switch back original delimiter
You could use smth like LAST_INSERT_ID() in case you are using stored procedure.
Good evening,
I have 1 table where i store a element with the type : datetime,
I am going to insert a date into this element by a script on php.
When the date is reached i want to increase "datepast" from table2,
we can do it by comparing "name" from table1 with "person_name" from table2.
Now the question is how to trigger a sql script to do this job for me, it would be great if it was real time.
Already thanks,
create table if not exists table1 (
name varchar(64) NOT NULL,
finishtime DATETIME,
id_table1 int NOT NULL AUTO_INCREMENT,
primary key ( id_table1 ));
create table if not exists table2 (
person_name varchar(64) NOT NULL,
datepast int,
id_table1 int NOT NULL AUTO_INCREMENT,
primary key ( id_table1 ));
Cronjobs and MySQL events can do a good job handling such things. But the queries you put in there must be set up to be idempotent -- that is, they must be set up so if you run them more than once they have the same effect as running them once. Otherwise you will have a brittle solution.
When you're handling data based on expiration times like your finishtime, it's usually a good idea to try to use a query or a view, rather than an update.
For example you could create this view
CREATE OR REPLACE VIEW table2 AS
SELECT name AS person_name
COUNT(*) AS datepast
FROM table1
WHERE datepast <= NOW()
GROUP BY name
Then, SELECT * FROM table2 will generate a result set just like SELECT person_name, datepast FROM table2 might generate. But the SELECT resultset will always be precisely accurate in time.
Wait! you say, isn't that inefficient? The answer is, probably not unless you have several hundred thousand rows or more in your table. SQL is built for this kind of declarative data stuff.
You can use MySQL events or a Cronjob. It is not real-time, but it can be close to.
Because your posting is not complete and table2 is missing, I can only give you an example how to setup an event. Inside DO BEGIN and END $$ you can add your MySQL query to update datepast.
DELIMITER $$
CREATE
EVENT `increase_date_past`
ON SCHEDULE EVERY 30 SECOND
DO BEGIN
{{READ DATE AND UPDATE TABLE2}}
END $$
DELIMITER ;
Events are not activated by default, you have to go to
In /etc/mysql/my.cnf
and activate events by placing a line inside [mysqld] tag.
[mysqld]
event_scheduler=1
and then sudo service mysql restart
Is it possible that each time a table has a record inserted, delete a view, then re-create a view using the data in the table. For example, my table structure is like such
Create Table NeedView (
Databasename varchar(100)
,DateAdded datetime
)
And the information in that table is just a fully qualified database name like so servername.database.dbo.table My thought is to have a trigger applied that each time a record is inserted into this table, 1st drop the view (it would be called dbo.ViewMaster then recreate it with a quick select of
Select fullname, phone
from each table in the database. Is this achievable through a trigger?
EDIT
I now just need to figure out how to cycle the table to create a view for each databasename in the table...I have this rough syntax ready, with the exception of actually iterating each database. Can someone help me with the missing piece?
CREATE TRIGGER GetViewReady ON [FullOn]
FOR INSERT
AS
IF OBJECT_ID('FullInformation', 'V') IS NOT NULL
DROP VIEW FullInformation
Declare #database varchar(100), #campaignID varchar(10)
Declare C1 Cursor FOR
Select database
FROM [FullOn]
Open C1
Fetch Next FROM C1 INTO #database
WHILE ##Fetch_Status = 0
Begin
'Here is where the actual iteration to create the view would go...
'How would that statement actually need to be syntactically written?
Fetch Next From C1 Into #database
End
GO
EDIT --
A few database names would be hellfire.master.dbo.hennigar, hellfire.mainframe.dbo.dekalb, hellfire.master.dbo.ftworth and the syntax I am after would be like so:
Select fullname, phone
FROM hellfire.master.dbo.hennigar
UNION ALL
select fullname, phone
from hellfire.mainframe.dbo.dekalb
UNION ALL
select fullname, phone
from hellfire.master.dbo.ftworth
I've recently moved from MSSQL to MySQL.
I would like to use a table variable (or equivalent) inside a MySQL 5.5 stored routine, to populate a dataset for an online report.
In MS SQL, I would do it this way
...
...
DECLARE #tblName TABLE
WHILE <condition>
BEGIN
Insert Row based on iteration value
END
...
...
From what I understand, I can't declare table variables in MySQL (correct me if I'm wrong) How do I implement the above logic in a MySQL stored procedure?
You could create a table or temporary table and populate it with data you need.
CREATE TABLE Syntax
You understand that limitation correctly. The MySQL user manual clearly states that user-defined variables cannot refer to a table:
http://dev.mysql.com/doc/refman/5.5/en/user-variables.html
User variables are intended to provide data values. They cannot be used directly in an SQL statement as an identifier or as part of an identifier, such as in contexts where a table or database name is expected, or as a reserved word such as SELECT.
create temporary table tmp
(
id int unsigned not null,
name varchar(32) not null
)
engine=memory; -- change engine type if required e.g myisam/innodb
insert into tmp (id, name) select id, name from foo... ;
-- do more work...
select * from tmp order by id;
drop temporary table if exists tmp;
I think this covers it. Also, this may be helpful.