This question already has an answer here:
MySql - using dynamic table names in one query
(1 answer)
Closed 1 year ago.
I have a table generated from SQL Query itself. Now I need to add an auto incremental id column into this table.
Usual syntax to add auto incremental id column is-
ALTER TABLE *Table_Name* ADD id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
But I don't have a specific table name, the table is generated from a query.
You can do it with a query like this:
## Your Query to generate the Tablename
SELECT "myTable" INTO #mytab;
EXECUTE IMMEDIATE CONCAT("ALTER TABLE ",#mytab," ADD id INT NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST");
sample
MariaDB [bernd]> desc myTable;
+----------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------+------+-----+---------+-------+
| insertat | date | YES | | NULL | |
| myval | int(11) | YES | | NULL | |
+----------+---------+------+-----+---------+-------+
2 rows in set (0.02 sec)
MariaDB [bernd]> select * from myTable;
+------------+-------+
| insertat | myval |
+------------+-------+
| 2021-01-01 | 44 |
| 2021-01-02 | 99 |
| 2021-01-02 | 134 |
| 2021-01-03 | 45 |
| 2021-01-04 | 2 |
| 2021-01-04 | 17 |
+------------+-------+
6 rows in set (0.06 sec)
MariaDB [bernd]> ## Your Query to generate the Tablename
MariaDB [bernd]> SELECT "myTable" INTO #mytab;
Query OK, 1 row affected (0.01 sec)
MariaDB [bernd]>
MariaDB [bernd]> EXECUTE IMMEDIATE CONCAT("ALTER TABLE ",#mytab," ADD id INT NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST");
Query OK, 0 rows affected (0.14 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB [bernd]> desc myTable;
+----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| insertat | date | YES | | NULL | |
| myval | int(11) | YES | | NULL | |
+----------+---------+------+-----+---------+----------------+
3 rows in set (0.01 sec)
MariaDB [bernd]> select * from myTable;
+----+------------+-------+
| id | insertat | myval |
+----+------------+-------+
| 1 | 2021-01-01 | 44 |
| 2 | 2021-01-02 | 99 |
| 3 | 2021-01-02 | 134 |
| 4 | 2021-01-03 | 45 |
| 5 | 2021-01-04 | 2 |
| 6 | 2021-01-04 | 17 |
+----+------------+-------+
6 rows in set (0.00 sec)
MariaDB [bernd]>
We just upgraded our database from mariadb 5.5.56 to MySQL 8. We are facing issue that some update queries to an InnoDB takes a long time randomly. Please note that 99% of the time, query is fast but then randomly suddenly it takes huge time and then again back to normal. We DO NOT face this issue in mariadb 5.5.56
CREATE TABLE `users` (
`uid` int(10) unsigned NOT NULL AUTO_INCREMENT,
`pid` int(10) unsigned NOT NULL DEFAULT '0',
`ipaddr` int(10) unsigned DEFAULT '0',
PRIMARY KEY (`uid`,`pid`),
KEY `pid` (`pid`)
) ENGINE=InnoDB AUTO_INCREMENT=161089 DEFAULT CHARSET=utf8
/*!50100 PARTITION BY HASH (`pid`)
PARTITIONS 101
mysql> update users set ipaddr=2148888835 where uid=1 limit 1;
Query OK, 1 row affected **(39.51 sec)**
Rows matched: 1 Changed: 1 Warnings: 0
mysql> update users set ipaddr=2148888835 where uid=1 limit 1;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
query is very simple and it uses primary key as shown below
mysql> explain update users set ipaddr=123 where uid=1;
+----+-------------+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+---------------+---------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+---------------+---------+---------+-------+------+----------+-------------+
| 1 | UPDATE | users | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20,p21,p22,p23,p24,p25,p26,p27,p28,p29,p30,p31,p32,p33,p34,p35,p36,p37,p38,p39,p40,p41,p42,p43,p44,p45,p46,p47,p48,p49,p50,p51,p52,p53,p54,p55,p56,p57,p58,p59,p60,p61,p62,p63,p64,p65,p66,p67,p68,p69,p70,p71,p72,p73,p74,p75,p76,p77,p78,p79,p80,p81,p82,p83,p84,p85,p86,p87,p88,p89,p90,p91,p92,p93,p94,p95,p96,p97,p98,p99,p100 | range | PRIMARY | PRIMARY | 4 | const | 1 | 100.00 | Using where |
+----+-------------+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+---------------+---------+---------+-------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
I enabled profiling and it shows updating takes long time.
mysql> SHOW PROFILE; +--------------------------------+-----------+
| Status | Duration |
+--------------------------------+-----------+
| starting | 0.000045 |
| Executing hook on transaction | 0.000005 |
| starting | 0.000005 |
| checking permissions | 0.000004 |
| Opening tables | 0.000017 |
| init | 0.000005 |
| System lock | 0.000193 |
| updating | 39.470708 |
| end | 0.000009 |
| query end | 0.000004 |
| waiting for handler commit | 0.038291 |
| closing tables | 0.000039 |
| freeing items | 0.000024 |
| cleaning up | 0.000028 |
+--------------------------------+-----------+
14 rows in set, 1 warning (0.00 sec)
Although the tables were freshly created using mysqldump, I even tried optimising the table but it didn't help.
mysql> optimize table users;
+--------------+----------+----------+-------------------------------------------------------------------+
| Table | Op | Msg_type | Msg_text |
+--------------+----------+----------+-------------------------------------------------------------------+
| mesibo.users | optimize | note | Table does not support optimize, doing recreate + analyze instead |
| mesibo.users | optimize | status | OK |
+--------------+----------+----------+-------------------------------------------------------------------+
2 rows in set (2 min 3.07 sec)
Below is config file. Our server has 64GB ram, we allocated 16G to innodb, however changing this value or removing it from configuration does not help.
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysql/mysqld.log
pid-file=/run/mysqld/mysqld.pid
innodb_buffer_pool_size=16G
max_connections = 1024
Any clue what might be wrong here?
I have a strange problem with a particular database:
mysql> update person set last_name=null where first_name='john';
ERROR 1713 (HY000): Undo log record is too big.
What can be the problem?
(I've already googled quite a log)
mysql 5.7.20 installed with brew on Mac OSX 10.12.6.
show table status where name = 'person'
-> ;
+--------------------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+--------------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |
+--------------------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+--------------------+----------+----------------+---------+
| person | InnoDB | 10 | Dynamic | 650918 | 4322 | 2813853696 | 0 | 1369833472 | 6291456 | NULL | 2017-09-12 14:31:20 | 2017-10-24 12:37:03 | NULL | utf8mb4_unicode_ci | NULL | | |
+--------------------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+--------------------+----------+----------------+---------+
1 row in set (0.00 sec)
I have a table like this:
// banned
+----+---------+---------------+
| id | user_id | unix_time |
+----+---------+---------------+
| 1 | 32534 | 1467066745524 |
| 2 | 43535 | 1467066745541 |
| 3 | 24352 | 1467066745618 |
| 4 | 88734 | 1467066746093 |
+----+---------+---------------+
Actually I need to define a expire time for each row when I insert it. Is that possible in MySQL? (I heard it is possible in Redis, well what about MySQL?)
So I want something like this:
// banned
+----+---------+---------------+
| id | user_id | unix_time |
+----+---------+---------------+
| 1 | 32534 | 1467066745524 | -- removing this row automatically in 10 min
| 2 | 43535 | 1467066745541 | -- removing this row automatically in 1 hour
| 3 | 24352 | 1467066745618 | -- removing this row automatically 2 day
| 4 | 88734 | 1467066746093 | -- removing this row automatically 8 hours min
+----+---------+---------------+
As you see, each row has a arbitrary lifetime.
you can create a EVENT that running every minute and delete old records like this:
enable scheduler
SET GLOBAL event_scheduler = ON;
create EVENT runs every minute
CREATE EVENT cleanup
ON SCHEDULE EVERY 1 MINUTE
DO
DELETE
FROM yourTable
WHERE unix_time < UNIX_TIMESTAMP();
Sample
MariaDB [yourschema]> SET GLOBAL event_scheduler = ON;
Query OK, 0 rows affected (0.00 sec)
MariaDB [yourschema]> SHOW GLOBAL VARIABLES LIKE 'event_scheduler';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| event_scheduler | ON |
+-----------------+-------+
1 row in set (0.00 sec)
MariaDB [yourschema]> SHOW PROCESSLIST;
+-----+-----------------+-----------------+------------+---------+------+-----------------------------+------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+-----------------+-----------------+------------+---------+------+-----------------------------+------------------+----------+
| 28 | root | localhost:53255 | yourSchema | Sleep | 11 | | NULL | 0.000 |
| 29 | root | localhost:53256 | NULL | Sleep | 28 | | NULL | 0.000 |
| 34 | event_scheduler | localhost | NULL | Daemon | 6603 | Waiting for next activation | NULL | 0.000 |
| 316 | root | localhost | yourschema | Query | 0 | init | SHOW PROCESSLIST | 0.000 |
+-----+-----------------+-----------------+------------+---------+------+-----------------------------+------------------+----------+
4 rows in set (0.00 sec)
MariaDB [yourschema]>
MariaDB [yourschema]> SHOW events;
Empty set (0.01 sec)
MariaDB [yourschema]> DROP EVENT IF EXISTS cleanup;
Query OK, 0 rows affected, 1 warning (0.00 sec)
MariaDB [yourschema]> CREATE EVENT cleanup
-> ON SCHEDULE EVERY 5 SECOND ON COMPLETION PRESERVE ENABLE
-> DO
-> DELETE
-> FROM schedulerTable
-> WHERE expire_unix_time < UNIX_TIMESTAMP() LIMIT 10;
Query OK, 0 rows affected (0.00 sec)
MariaDB [yourschema]> INSERT INTO schedulerTable VALUES
-> (NULL,100,UNIX_TIMESTAMP(now() + INTERVAL 30 SECOND)),
-> (NULL,101,UNIX_TIMESTAMP(now() + INTERVAL 40 SECOND)),
-> (NULL,111,UNIX_TIMESTAMP(now() + INTERVAL 1 MINUTE));
Query OK, 3 rows affected (0.01 sec)
Records: 3 Duplicates: 0 Warnings: 0
MariaDB [yourschema]> SELECT * FROM schedulerTable;SELECT SLEEP(20);
+-------------+---------+------------------+
| id_rubrique | id_stat | expire_unix_time |
+-------------+---------+------------------+
| 1 | 100 | 1467074632 |
| 2 | 101 | 1467074642 |
| 3 | 111 | 1467074662 |
+-------------+---------+------------------+
3 rows in set (0.00 sec)
+-----------+
| SLEEP(20) |
+-----------+
| 0 |
+-----------+
1 row in set (20.00 sec)
MariaDB [yourschema]> SELECT * FROM schedulerTable;SELECT SLEEP(20);
+-------------+---------+------------------+
| id_rubrique | id_stat | expire_unix_time |
+-------------+---------+------------------+
| 2 | 101 | 1467074642 |
| 3 | 111 | 1467074662 |
+-------------+---------+------------------+
2 rows in set (0.00 sec)
+-----------+
| SLEEP(20) |
+-----------+
| 0 |
+-----------+
1 row in set (20.01 sec)
MariaDB [yourschema]> SELECT * FROM schedulerTable;SELECT SLEEP(20);
+-------------+---------+------------------+
| id_rubrique | id_stat | expire_unix_time |
+-------------+---------+------------------+
| 3 | 111 | 1467074662 |
+-------------+---------+------------------+
1 row in set (0.00 sec)
+-----------+
| SLEEP(20) |
+-----------+
| 0 |
+-----------+
1 row in set (20.01 sec)
MariaDB [yourschema]> SELECT * FROM schedulerTable;
Empty set (0.00 sec)
MariaDB [yourschema]>
You could create a view, something like this:
create view v_table as
select (case when unix_time < UNIX_TIMESTAMP() then id end) as id,
(case when unix_time < UNIX_TIMESTAMP() then user_id end) as user_id
from banned;
Or:
create view v_table as
select b.*
from banned b
where unixtime < UNIX_TIME();
This returns no information about a user after the timestamp. You can then schedule an event to periodically delete rows with expired information.
I'm trying to get an rough (order-of-magnitude) estimate of how long time the following query could take:
mysql> EXPLAIN SELECT t1.col1, t1_col4 FROM t1 LEFT JOIN t2 ON t1.col1=t2.col1 WHERE col2=0 AND col3 IS NULL;
+----+-------------+--------------------+------+---------------+------------+---------+-----------------------------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------------+------+---------------+------------+---------+-----------------------------+---------+--------------------------+
| 1 | SIMPLE | t1 | ref | foobar | foobar | 4 | const | 9715129 | |
| 1 | SIMPLE | t2 | ref | col1 | col1 | 4 | db2.t1.col1 | 42318 | Using where; Using index |
+----+-------------+--------------------+------+---------------+------------+---------+-----------------------------+---------+--------------------------+
2 rows in set (0.00 sec)
mysql>
This can be done when using SHOW PROFILES syntax.
When you open a MySQL session, you could set the variable "profiling" to 1 or ON.
mysql> SET profiling = 1;
So all the statements sent to the server will be profiled and stored in a historical and shown later by typing the command:
mysql> SHOW PROFILES;
See, from MySQL manual:
mysql> SET profiling = 1;
Query OK, 0 rows affected (0.00 sec)
mysql> DROP TABLE IF EXISTS t1;
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> CREATE TABLE T1 (id INT);
Query OK, 0 rows affected (0.01 sec)
mysql> SHOW PROFILES;
+----------+----------+--------------------------+
| Query_ID | Duration | Query |
+----------+----------+--------------------------+
| 0 | 0.000088 | SET PROFILING = 1 |
| 1 | 0.000136 | DROP TABLE IF EXISTS t1 |
| 2 | 0.011947 | CREATE TABLE t1 (id INT) |
+----------+----------+--------------------------+
3 rows in set (0.00 sec)
mysql> SHOW PROFILE;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| checking permissions | 0.000040 |
| creating table | 0.000056 |
| After create | 0.011363 |
| query end | 0.000375 |
| freeing items | 0.000089 |
| logging slow query | 0.000019 |
| cleaning up | 0.000005 |
+----------------------+----------+
7 rows in set (0.00 sec)
mysql> SHOW PROFILE FOR QUERY 1;
+--------------------+----------+
| Status | Duration |
+--------------------+----------+
| query end | 0.000107 |
| freeing items | 0.000008 |
| logging slow query | 0.000015 |
| cleaning up | 0.000006 |
+--------------------+----------+
4 rows in set (0.00 sec)
mysql> SHOW PROFILE CPU FOR QUERY 2;
+----------------------+----------+----------+------------+
| Status | Duration | CPU_user | CPU_system |
+----------------------+----------+----------+------------+
| checking permissions | 0.000040 | 0.000038 | 0.000002 |
| creating table | 0.000056 | 0.000028 | 0.000028 |
| After create | 0.011363 | 0.000217 | 0.001571 |
| query end | 0.000375 | 0.000013 | 0.000028 |
| freeing items | 0.000089 | 0.000010 | 0.000014 |
| logging slow query | 0.000019 | 0.000009 | 0.000010 |
| cleaning up | 0.000005 | 0.000003 | 0.000002 |
+----------------------+----------+----------+------------+
References (updated at: 2014-09-04):
- SHOW PROFILE Syntax
- The INFORMATION_SCHEMA PROFILING Table
- How To Use MySQL Query Profiling (The Digital Ocean recently published a great article concerning this issue.)