I'm using MySQL on a CentOS server from a website running Apache and Perl. I'm seeing this behavior (mocked up):
mysql> describe product;
+-------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(10) | YES | | NULL | |
+-------+------------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
mysql> insert into product (name) values ('Widget');
Query OK, 1 row affected (0.00 sec)
mysql> insert into product (name) values ('Thing');
Query OK, 1 row affected (0.00 sec)
mysql> select * from product;
+----+--------+
| id | name |
+----+--------+
| 1 | Widget |
| 2 | Widget |
| 3 | Thing |
+----+--------+
3 rows in set (0.00 sec)
My theory is that the Perl CGI script may be executed concurrently (perhaps due to double click, browser refresh, etc.) so the insert gets performed twice. This is fairly rare but causes problems when it happens.
In cases where this happens, all columns except 'id' have identical values. Other than column 'id' duplicate values are allowed.
Is there a way to prevent this behavior?
If you installed Perl, Apache, and MySQL from CentOS repositories and didn't make any radical configuration changes, it's highly unlikely you've found a bug in the platform. Are you using mod_perl or PSGI or is it definitely CGI? Have you installed any Perl modules from CPAN sources or has everything been installed through yum?
A stupid solution you might consider implementing is generating some kind of nonce (one-time-use string or number) in your script, adding a column for it and a unique index on that column to your table, and inserting it along with the rest of the form data. If you catch a duplicate key error, just disregard it. That won't explain what's happening but it will prevent it from happening.
Related
I have the following table (mariadb 10.4) called p:
+----------------+----------------------------------------------------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+----------------------------------------------------------------------------------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| description | text | YES | | NULL | |
| url | text | YES | | NULL | |
| source | enum('source_a','source_b','source_c','source_d','source_e') | YES | | NULL | |
I currently have a couple of million rows on this table with the sources a, b, c, and d. Just recently we applied a migration to add source_e and we started getting the error ERROR 1265 (01000): Data truncated for column 'source' at row 1 when trying to inset a row with the source_e. The used command that yields the error is the following:
INSERT INTO p (description, url, `source`) VALUES ('test', 'https://google.com.br', 'source_e');
Insertions with any of the other sources are still working.
The behavior changes when editing a row that is already on the db, the error is not shown:
UPDATE `p` SET `source`='source_e' WHERE `id`='3';
Yields:
Query OK, 1 rows affected (0.001 sec)
Is there a way to debug this scenario? I've tried changing the log level of the db to get a better insight on the problem (SET GLOBAL log_warnings=3;) but the error message did not change.
I also tried changing the source_e name to source_e_, the error persisted.
Btw, i did change the name of the fields to comply with company policies.
It turns out it was my bad. We happen to have a trigger on insertions of this table that feeds a materialized view kind of table. All I had to do was add 'source_e' to the source field on the other table.
I'm trying to store about 100 Million domain names in a MySQL database, but I can't figure out the right INDEX method to use on the domain names.
The issue being that LIKE queries will also be executed:
SELECT id FROM domains WHERE domain LIKE '%.example.com'
or
SELECT id FROM domains WHERE domain LIKE 'example.%'
If it makes it easier, '%example%' is not a requirement, but at best a nice to have / be able to.
What would be the proper index to use? Left to right (example.%) should be realitivly straight forward, but right to left (%.example.com) is problematic but the most common query.
I'm using MariaDB 10.3 on Linux. DB running on a PCI-e SSD, lookup times longer then 10 seconds should be coincided "unacceptable"
You can spend one virtual permanent column (rdomain) in your table where the virtual function stores the domainname in reverse order like REVERSE(domain). so it is possible to search from start of string i.e. search for '%.mydomain.com' -> WHERE rdomain like REVERSE('%.mydomain.com
the table
CREATE TABLE `myreverse` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`domain` varchar(64) CHARACTER SET latin1 DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_domain` (`domain`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
add the column
ALTER TABLE myreverse
ADD COLUMN rdomain VARCHAR(64) AS (REVERSE(domain)),
ADD KEY idx_rdomain (rdomain);
insert some data
INSERT INTO `myreverse` (`id`, `domain`)
VALUES
(2, 'img.google.com'),
(3, 'w3.google.com'),
(1, 'www.coogle.com'),
(4, 'www.google.de'),
(5, 'www.mydomain.com');
see the data
mysql> SELECT * from myreverse;
+----+------------------+------------------+
| id | domain | rdomain |
+----+------------------+------------------+
| 1 | www.google.com | moc.elgoog.www |
| 2 | img.google.com | moc.elgoog.gmi |
| 3 | w3.coogle.com | moc.elgooc.3w |
| 4 | www.google.de | ed.elgoog.www |
| 5 | www.mydomain.com | moc.niamodym.www |
+----+------------------+------------------+
5 rows in set (0.01 sec)
mysql>
now you can query with reverse order and MySQL can use the index.
query
mysql> select * from myreverse WHERE rdomain like REVERSE('%.google.com');
+----+----------------+----------------+
| id | domain | rdomain |
+----+----------------+----------------+
| 3 | w3.google.com | moc.elgoog.3w |
| 2 | img.google.com | moc.elgoog.gmi |
+----+----------------+----------------+
2 rows in set (0.00 sec)
mysql>
Here you can see that the optimizer use the index.
mysql> EXPLAIN select * from myreverse WHERE rdomain like REVERSE('%.google.com');
+----+-------------+-----------+------------+-------+---------------+-------------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-----------+------------+-------+---------------+-------------+---------+------+------+----------+-------------+
| 1 | SIMPLE | myreverse | NULL | range | idx_rdomain | idx_rdomain | 195 | NULL | 2 | 100.00 | Using where |
+----+-------------+-----------+------------+-------+---------------+-------------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.01 sec)
mysql>
I'm not sure an index would help you here. If you can't change the database, your options seem limited. One thing you could do, is if you're running both a subdomain and domain query back to back, to run the subdomain query first. That should help reduce the number of rows the domain query has to cover.
It would definitely help if you split the URL between subdomains and domains into different columsn in the database. Have indexes for both of them. Then you could query the subdomains only and the domains only. It should speed things up. And if there are a lot of repeating values, you should normalize those fields so to remove repetition and speed up queries even more.
I want to copy a table, including its indexes and triggers, from one database to another. This is not as straightforward as I had hoped. Here is a minimal working example (MWE) to demonstrate. First, my MariaDB version:
$ mysql --version
mysql Ver 15.1 Distrib 10.0.29-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
Next, the first table:
CREATE DATABASE db1;
USE db1;
CREATE TABLE tb1 (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
word VARCHAR(50) NOT NULL);
INSERT INTO tb1 (word) VALUES ('foo');
DELETE FROM tb1 WHERE word='foo';
DELIMITER $$
CREATE TRIGGER before_word_insert BEFORE INSERT ON tb1 FOR EACH ROW
BEGIN
SET NEW.word=TRIM(NEW.word);
IF NOT (NEW.word REGEXP '^[[:alpha:]]+$') THEN
SIGNAL SQLSTATE '12345' SET MESSAGE_TEXT = 'Invalid word';
END IF;
END$$
DELIMITER ;
INSERT INTO tb1 (word) VALUES ('foo');
DESCRIBE tb1;
SHOW TRIGGERS;
SHOW INDEX FROM tb1;
These last three lines give:
+-------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| word | varchar(50) | NO | | NULL | |
+-------+-------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
+--------------------+--------+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------+--------+---------+----------+----------------+----------------------+----------------------+--------------------+
| Trigger | Event | Table | Statement | Timing | Created | sql_mode | Definer | character_set_client | collation_connection | Database Collation |
+--------------------+--------+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------+--------+---------+----------+----------------+----------------------+----------------------+--------------------+
| before_word_insert | INSERT | tb1 | BEGIN SET NEW.word=TRIM(NEW.word); IF NOT (NEW.word REGEXP '^[[:alpha:]]+$') THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'Invalid word'; END IF; END | BEFORE | NULL | | root#localhost | utf8 | utf8_general_ci | latin1_swedish_ci |
+--------------------+--------+-------+----------------------------------------------------------------------------------------------------------------------------------------------------------+--------+---------+----------+----------------+----------------------+----------------------+--------------------+
1 row in set (0.04 sec)
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| tb1 | 0 | PRIMARY | 1 | id | A | 3 | NULL | NULL | | BTREE | | |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
1 row in set (0.00 sec)
So far, so good. Now, to copy tb1 to another database (on the same server):
CREATE DATABASE db2;
USE db2;
CREATE TABLE db2.tb1 AS SELECT * FROM db1.tb1;
DESCRIBE tb1;
SHOW TRIGGERS;
SHOW INDEX FROM tb1;
I had hoped these last three lines would give identical output as they did for db1, but they don't:
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id | int(11) | NO | | 0 | |
| word | varchar(50) | NO | | NULL | |
+-------+-------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
Empty set (0.00 sec)
Empty set (0.00 sec)
In other words, CREATE TABLE db2.tb1 AS SELECT * FROM db1.tb1;:
copied the table contents;
copied the column types;
did not copy the "Key" property;
did not copy the "Default" property;
did not copy the "Extra" property;
did not copy the triggers; and
did not copy the indexes.
My question: what would be a concise equivalent to CREATE TABLE db2.tb1 AS SELECT * FROM db1.tb1; that would copy all of those missing items?
CREATE TABLE newTable LIKE oldTable
should include the indexes and setup, although not the data and triggers.
After that, you need to copy the data:
INSERT INTO newTable SELECT * FROM oldTable
The triggers are not directly part of the table but rather part of the script management.
You can find the triggers in information_schema.triggers. There you can query every trigger set for your table. But I strongly recommend not to mess around with that table manually.
Instead, you can read the definition there and create a new trigger using SHOW CREATE TRIGGER and CREATE TRIGGER. That involves either dynamic sql or a client able to manipulate the sql (which should apply to every client).
If there is a client connected to the source database and to the target database, you can do something like that:
SELECT `TRIGGER_NAME` FROM information_schema.triggers WHERE `TRIGGER_SCHEMA` = database() AND `EVENT_OBJECT_TABLE` = oldTableName
As you stated in the comments, you can also use
SHOW TRIGGERS WHERE `Table` = "oldTable"
instead of reading the triggers table. As a side note, these backticks around Table are important, because Table is a reserved word which is used here in a manner like a column name.
With the trigger names, for each trigger you cast a
SHOW CREATE TRIGGER triggerNameFromQueryAbove
This gives you the create statement you can use to create the trigger in the new database. But careful: There might be the database name included and the definer, which may not exist in the new database, so you have to strip out that information manually.
For more information on the "triggers" table, read the mysql manual (which also should apply to mariadb): https://dev.mysql.com/doc/refman/5.7/en/triggers-table.html
I'm looking at this question, because I have the same question. I'm a bit dismayed, if there really are no more concise commands to do this. I want to point out a (perhaps obvious) procedure that will also work. You could dump the database to a file ("dumpdb1.sql", for example) which is already handy if you're using mysqldump for backups. You could then of course make a copy of just the relevant text out of dumpdb1.sql that creates tb1, sets its constraints and indexes, and adds its triggers and data, placing this excerpt into a file "buildtb1.sql". Finally, just run this latter file on db2 and you should get a perfect full copy of tb1 there.
Also, I see in the MariaDB documentation at: Tutorialspoint MariaDB Table Cloning they point out the usefulness of the SHOW CREATE TABLE statement in this regard. It will give a CREATE TABLE statement you can copy and use during this process. I'm just mentioning this as an alternative to the mysqldump approach, if you don't want to use that. Of course SHOW CREATE TABLE will not describe triggers, so you would still need to perform a SHOW TRIGGERS process to capture and reproduce those.
My database table structure is
mysql> desc webhotel.`first`;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| one | varchar(45) | NO | PRI | NULL | |
+-------+-------------+------+-----+---------+-------+
1 row in set (0.19 sec)
I want to add ('Harshal') as field value for one
mysql> select * from webhotel.`first`;
+-------------+
| one |
+-------------+
| ('Harshal') |
+-------------+
1 row in set (0.00 sec)
I want this output
My problem is that i don't know how to insert a string containing single quotes into the database. I can use the IDE to do it, but i want an insert query.
Are you looking for this?
INSERT INTO first (one) VALUES ('(''Harshal'')');
^^ ^^
Outcome:
| ONE |
|-------------|
| ('Harshal') |
Here is SQLFiddle demo
If the value contains single quotes, you can use double quotes in MySQL to enter the string:
INSERT INTO first (one)
VALUES ("('Harshal')");
or you can escape the quotes:
INSERT INTO first (one)
VALUES ('(\'Harshal\')');
depending on the server side software you are using, you should look into 'prepared statements'. here's the link for PHP, which provides easy to follow code. while the other answers technically work, prepared statements were created to specifically address this problem, and will help alleviate some of the issues the other answers will give you.
Below are the two ways you can use insert query.
INSERT into tablename values ('colValue1', 'colValue2')
INSERT into tablename (`colname1`, `colname2`) values ('colValue1', 'colValue2')
Visit http://www.w3schools.com/sql/sql_insert.asp for more sql tutorials
Hey guys I created a database column in my regular LAMP stack that seems to work great, the trouble is when migrating this into CPanel, it seems that my Default values in enum revert to ' ' or whitespace?
the command I used to create this column was
`status` ENUM('0','1','2') NOT NULL DEFAULT '0',
But it seems this doesn't actually happen.....
Is there an error in my syntax? A stupidity of CPanel?
What's going on here?
EDIT
It looks like it has something to do with the input button
submitting a blank value? Anyone heard of this before?
MariaDB [test]> create table settest(attrib set('bold','italic','underline') DEF
AULT 'bold',color enum('red','green','blue') DEFAULT 'blue');
MariaDB [test]> INSERT INTO settest VALUES('a','s');
Query OK, 1 row affected, 2 warnings (0.14 sec)
MariaDB [test]> SHOW WARNINGS;
+---------+------+---------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------+
| Warning | 1265 | Data truncated for column 'attrib' at row 1 |
| Warning | 1265 | Data truncated for column 'color' at row 1 |
+---------+------+---------------------------------------------+
2 rows in set (0.00 sec)
MariaDB [test]> SELECT * FROM settest;
+--------+-------+
| attrib | color |
+--------+-------+
| | |
| | |
+--------+-------+
Looks like the answer to get a default is NOT NULL DEFAULT 1 as per 1.3. ENUM