I have a weird hyphen in one of the mysql tables created by Racktables. I am trying to replace it with a normal one, but I seem to be missing something:
mysql> update Port set reservation_comment = replace(reservation_comment,'–','-');
Query OK, 0 rows affected (0.01 sec)
Rows matched: 2358 Changed: 0 Warnings: 0
As you can see the "bad" hyphen gets matched but not replaced.
I have tried changing single quotes to double quotes and escaping the hyphens, but there is no real change.
Here goes some sample data:
| 2690 | 767 | R131226-005-23Ha | 1 | 24 | NULL | C130527-059 | |
| 2691 | 768 | R131226-005-24Ha | 1 | 24 | NULL | C130527-036 | |
| 2692 | 770 | R131226-006-01Ha | 1 | 24 | NULL | C140305�001 | |
| 2693 | 773 | R131226-006-04Ha | 1 | 24 | NULL | C140305�004 | |
| 2694 | 784 | R131226-006-15Ha | 1 | 24 | NULL | C140305�015 | |
| 2695 | 785 | R131226-006-16Ha | 1 | 24 | NULL | C140305�016 | |
| 2696 | 793 | R131226-006-24Ha | 1 | 24 | NULL | C140305�024 | |
| 2697 | 771 | R131226-006-02Ha | 1 | 24 | NULL | C140305�002 | |
| 2698 | 772 | R131226-006-03Ha | 1 | 24 | NULL | C140305-003
The hyphen I am trying to replace is encoded differently it seems
EDIT:
SO, all syntax below is good. The problem is that I cannot match the bad hyphen. If I try to replace the "good" one all is fine:
mysql> UPDATE Port SET reservation_comment=REPLACE(reservation_comment,'-','[[good_hyphen]]');
Query OK, 367 rows affected (0.34 sec)
Rows matched: 2358 Changed: 367 Warnings: 0
but if I try to replace the bad one:
mysql> UPDATE Port SET reservation_comment=REPLACE(reservation_comment,'–','[[bad_hyphen]]');
Query OK, 0 rows affected (0.01 sec)
Rows matched: 2358 Changed: 0 Warnings: 0
I will look into the encoding of that character and into different ways to match it.
That is matching all rows, regardless of the hyphen as there is no WHERE clause. It is just not changing rows where there is no bad hyphen.
It may be that the bad hyphen is getting stored as an unknown character, so will not be targeted by your REPLACE. You may have to change the charset/collation on your column to get it to import correctly, before changing.. or adjust the way that Racktables outputs the data.
Ok, I solved it by doing something similar to this post once I identified the character:
mysql> select hex("–");
+------------+
| hex("–") |
+------------+
| E28093 |
+------------+
1 row in set (0.00 sec)
Then I verified that matched:
mysql> select x'E28093';
+-----------+
| x'E28093' |
+-----------+
| – |
+-----------+
1 row in set (0.00 sec)
And finally the substitution:
mysql> UPDATE Port SET reservation_comment=REPLACE(reservation_comment,x'E28093','-');
Query OK, 33 rows affected (1.55 sec)
Rows matched: 2358 Changed: 33 Warnings: 0
Related
Dropping or truncating a table resets the counter(AUTO_INCREMENT), but deleting selected rows (with a WHERE clause) doesn’t reset the counter.
I want it to continue it from the point where rows were deleted.
Is there any way to reset the counter? Please follow the example below for better understanding.
mysql> create table dummy(id int NOT NULL AUTO_INCREMENT PRIMARY KEY);
Query OK, 0 rows affected (2.09 sec)
mysql> insert into dummy values (),(),(),(),(),();
Query OK, 6 rows affected (0.19 sec)
Records: 6 Duplicates: 0 Warnings: 0
mysql> select * from dummy;
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
+----+
6 rows in set (0.00 sec)
mysql> delete from dummy where id>4;
Query OK, 2 rows affected (0.23 sec)
mysql> select * from dummy;
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
+----+
4 rows in set (0.00 sec)
mysql> insert into dummy values (),(),(),(),(),();
Query OK, 6 rows affected (0.18 sec)
Records: 6 Duplicates: 0 Warnings: 0
mysql> select * from dummy;
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 7 |
| 8 |
| 9 |
| 10 |
| 11 |
| 12 |
+----+
10 rows in set (0.00 sec)
What I want:
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
| 10 |
+----+
We can query alter table dummy AUTO_INCREMENT=1; after delete statement;
So, as soon as we add new values to the table, it will check in the existing table, what value should be assign to the id starting from 0, and will assign the one which is exactly 1 greater than the max id of the table.
So, it will be like:
mysql> delete from dummy where id>4;
Query OK, 2 rows affected (0.14 sec)
mysql> select * from dummy;
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
+----+
4 rows in set (0.00 sec)
mysql> alter table dummy AUTO_INCREMENT=1;
Query OK, 0 rows affected (0.21 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> insert into dummy values (),(),(),(),(),();
Query OK, 6 rows affected (0.18 sec)
Records: 6 Duplicates: 0 Warnings: 0
And we will get in sequential form:
select * from dummy;
+----+
| id |
+----+
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
| 10 |
+----+
Thanks #jspcal
I've changed my.cnf. and check sql_mode use below command
select ##global.sql_mode;
and It says,
and Also I've tried
set global sql_mode='';
SET sql_mode = 'NO_ENGINE_SUBSTITUTION';
But... when I query to insert null values into not null column, It returns error...
How Can I turn off strict mode????????
Please help me.....
You cannot set a not null column to null. the effect of turning ofF sql mode is described https://dev.mysql.com/doc/refman/5.7/en/constraint-invalid-data.html,
If you are not using strict mode, then whenever you insert an “incorrect” value into a column, such as a NULL into a NOT NULL column or a too-large numeric value into a numeric column, MySQL sets the column to the “best possible value” instead of producing an error ... If you try to store NULL into a column that doesn't take NULL values, an error occurs for single-row INSERT statements. For multiple-row INSERT statements or for INSERT INTO ... SELECT statements, MySQL Server stores the implicit default value for the column data type. In general, this is 0 for numeric types, the empty string ('') for string types, and the “zero” value for date and time types
This is easily demonstrated:-
MariaDB [sandbox]> SET SESSION SQL_MODE = '';
Query OK, 0 rows affected (0.00 sec)
MariaDB [sandbox]> SELECT ##sESSION.SQL_MODE;
+--------------------+
| ##sESSION.SQL_MODE |
+--------------------+
| |
+--------------------+
1 row in set (0.00 sec)
MariaDB [sandbox]> select * from t;
+----+------------+---------+
| id | dt | meeting |
+----+------------+---------+
| 1 | 2017-12-20 | 1 |
| 2 | 2017-12-20 | 1 |
| 3 | 2017-12-20 | 1 |
| 4 | 2017-12-22 | 1 |
| 5 | 2017-12-25 | 1 |
+----+------------+---------+
5 rows in set (0.00 sec)
MariaDB [sandbox]> INSERT INTO T (DT,MEETING) VALUES ('2018-01-01',NULL);
ERROR 1048 (23000): Column 'meeting' cannot be null
MariaDB [sandbox]>
MariaDB [sandbox]>
MariaDB [sandbox]> INSERT INTO T (DT,MEETING) VALUES ('2018-01-01',NULL),('2018-01-02',NULL);
Query OK, 2 rows affected, 2 warnings (0.41 sec)
Records: 2 Duplicates: 0 Warnings: 2
MariaDB [sandbox]>
MariaDB [sandbox]> SELECT * FROM T;
+----+------------+---------+
| id | dt | meeting |
+----+------------+---------+
| 1 | 2017-12-20 | 1 |
| 2 | 2017-12-20 | 1 |
| 3 | 2017-12-20 | 1 |
| 4 | 2017-12-22 | 1 |
| 5 | 2017-12-25 | 1 |
| 13 | 2018-01-01 | 0 |
| 14 | 2018-01-02 | 0 |
+----+------------+---------+
7 rows in set (0.00 sec)
For single inserts the ignore extension
MariaDB [sandbox]> INSERT ignore INTO T (DT,MEETING) VALUES ('2018-01-10',NULL);
Query OK, 1 row affected, 1 warning (0.02 sec)
MariaDB [sandbox]> select * from t;
+----+------------+---------+
| id | dt | meeting |
+----+------------+---------+
| 1 | 2017-12-20 | 1 |
| 2 | 2017-12-20 | 1 |
| 3 | 2017-12-20 | 1 |
| 4 | 2017-12-22 | 1 |
| 5 | 2017-12-25 | 1 |
| 13 | 2018-01-01 | 0 |
| 14 | 2018-01-02 | 0 |
| 15 | 2018-01-10 | 0 |
+----+------------+---------+
8 rows in set (0.00 sec)
Allows the insert but stores the implicit value as described above.(this also works for multiple row inserts AND doesn't care about strict mode)
Warning turning off sql mode and using insert ignore are very blunt tools - and may be harmful to your db.
The problem here is that MySQL 5.1 and before allowed INSERT NULL values into NOT NULL fields and a lot of apps were written with this precept.
For DBAs, it is not easy to say to all the developers to upgrade MySQL when they have to change so much code.
I have a table like this:
// banned
+----+---------+---------------+
| id | user_id | unix_time |
+----+---------+---------------+
| 1 | 32534 | 1467066745524 |
| 2 | 43535 | 1467066745541 |
| 3 | 24352 | 1467066745618 |
| 4 | 88734 | 1467066746093 |
+----+---------+---------------+
Actually I need to define a expire time for each row when I insert it. Is that possible in MySQL? (I heard it is possible in Redis, well what about MySQL?)
So I want something like this:
// banned
+----+---------+---------------+
| id | user_id | unix_time |
+----+---------+---------------+
| 1 | 32534 | 1467066745524 | -- removing this row automatically in 10 min
| 2 | 43535 | 1467066745541 | -- removing this row automatically in 1 hour
| 3 | 24352 | 1467066745618 | -- removing this row automatically 2 day
| 4 | 88734 | 1467066746093 | -- removing this row automatically 8 hours min
+----+---------+---------------+
As you see, each row has a arbitrary lifetime.
you can create a EVENT that running every minute and delete old records like this:
enable scheduler
SET GLOBAL event_scheduler = ON;
create EVENT runs every minute
CREATE EVENT cleanup
ON SCHEDULE EVERY 1 MINUTE
DO
DELETE
FROM yourTable
WHERE unix_time < UNIX_TIMESTAMP();
Sample
MariaDB [yourschema]> SET GLOBAL event_scheduler = ON;
Query OK, 0 rows affected (0.00 sec)
MariaDB [yourschema]> SHOW GLOBAL VARIABLES LIKE 'event_scheduler';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| event_scheduler | ON |
+-----------------+-------+
1 row in set (0.00 sec)
MariaDB [yourschema]> SHOW PROCESSLIST;
+-----+-----------------+-----------------+------------+---------+------+-----------------------------+------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+-----------------+-----------------+------------+---------+------+-----------------------------+------------------+----------+
| 28 | root | localhost:53255 | yourSchema | Sleep | 11 | | NULL | 0.000 |
| 29 | root | localhost:53256 | NULL | Sleep | 28 | | NULL | 0.000 |
| 34 | event_scheduler | localhost | NULL | Daemon | 6603 | Waiting for next activation | NULL | 0.000 |
| 316 | root | localhost | yourschema | Query | 0 | init | SHOW PROCESSLIST | 0.000 |
+-----+-----------------+-----------------+------------+---------+------+-----------------------------+------------------+----------+
4 rows in set (0.00 sec)
MariaDB [yourschema]>
MariaDB [yourschema]> SHOW events;
Empty set (0.01 sec)
MariaDB [yourschema]> DROP EVENT IF EXISTS cleanup;
Query OK, 0 rows affected, 1 warning (0.00 sec)
MariaDB [yourschema]> CREATE EVENT cleanup
-> ON SCHEDULE EVERY 5 SECOND ON COMPLETION PRESERVE ENABLE
-> DO
-> DELETE
-> FROM schedulerTable
-> WHERE expire_unix_time < UNIX_TIMESTAMP() LIMIT 10;
Query OK, 0 rows affected (0.00 sec)
MariaDB [yourschema]> INSERT INTO schedulerTable VALUES
-> (NULL,100,UNIX_TIMESTAMP(now() + INTERVAL 30 SECOND)),
-> (NULL,101,UNIX_TIMESTAMP(now() + INTERVAL 40 SECOND)),
-> (NULL,111,UNIX_TIMESTAMP(now() + INTERVAL 1 MINUTE));
Query OK, 3 rows affected (0.01 sec)
Records: 3 Duplicates: 0 Warnings: 0
MariaDB [yourschema]> SELECT * FROM schedulerTable;SELECT SLEEP(20);
+-------------+---------+------------------+
| id_rubrique | id_stat | expire_unix_time |
+-------------+---------+------------------+
| 1 | 100 | 1467074632 |
| 2 | 101 | 1467074642 |
| 3 | 111 | 1467074662 |
+-------------+---------+------------------+
3 rows in set (0.00 sec)
+-----------+
| SLEEP(20) |
+-----------+
| 0 |
+-----------+
1 row in set (20.00 sec)
MariaDB [yourschema]> SELECT * FROM schedulerTable;SELECT SLEEP(20);
+-------------+---------+------------------+
| id_rubrique | id_stat | expire_unix_time |
+-------------+---------+------------------+
| 2 | 101 | 1467074642 |
| 3 | 111 | 1467074662 |
+-------------+---------+------------------+
2 rows in set (0.00 sec)
+-----------+
| SLEEP(20) |
+-----------+
| 0 |
+-----------+
1 row in set (20.01 sec)
MariaDB [yourschema]> SELECT * FROM schedulerTable;SELECT SLEEP(20);
+-------------+---------+------------------+
| id_rubrique | id_stat | expire_unix_time |
+-------------+---------+------------------+
| 3 | 111 | 1467074662 |
+-------------+---------+------------------+
1 row in set (0.00 sec)
+-----------+
| SLEEP(20) |
+-----------+
| 0 |
+-----------+
1 row in set (20.01 sec)
MariaDB [yourschema]> SELECT * FROM schedulerTable;
Empty set (0.00 sec)
MariaDB [yourschema]>
You could create a view, something like this:
create view v_table as
select (case when unix_time < UNIX_TIMESTAMP() then id end) as id,
(case when unix_time < UNIX_TIMESTAMP() then user_id end) as user_id
from banned;
Or:
create view v_table as
select b.*
from banned b
where unixtime < UNIX_TIME();
This returns no information about a user after the timestamp. You can then schedule an event to periodically delete rows with expired information.
I'm trying to sort some results by time. I've gathered str_to_date is the way to go, but I appear to be using it wrong, and I can't tell for sure, but I think it's converting to NULL and then not sorting in a meaningful way:
mysql> SELECT member_id, result_result, str_to_date('result_result','%i:%s.%f') FROM results WHERE workout_id = '2' ORDER BY str_to_date('result_result','%i:%s.%f') LIMIT 5;
+-----------+---------------+-----------------------------------------+
| member_id | result_result | str_to_date('result_result','%i:%s.%f') |
+-----------+---------------+-----------------------------------------+
| 0 | 1:35.0 | NULL |
| 1 | 1:35.0 | NULL |
| 3 | 1:40 | NULL |
| 4 | 1:37.8 | NULL |
| 7 | 1:27.3 | NULL |
+-----------+---------------+-----------------------------------------+
5 rows in set, 5 warnings (0.00 sec)
but the two result types seem to convert fine if I do it manually:
mysql> select str_to_date('1:40','%i:%s.%f');
+--------------------------------+
| str_to_date('1:40','%i:%s.%f') |
+--------------------------------+
| 00:01:40 |
+--------------------------------+
1 row in set (0.00 sec)
mysql> select str_to_date('1:35.0','%i:%s.%f');
+----------------------------------+
| str_to_date('1:35.0','%i:%s.%f') |
+----------------------------------+
| 00:01:35 |
+----------------------------------+
1 row in set (0.00 sec)
Any ideas what's happening / how to fix it?
Thanks!
You don't need the quotes inside the function. Try
str_to_date( result_result, '%i:%s.%f' )
I was trying to measure the difference between TINYINT and INT when I came across something interesting. For tables with small numbers of columns, the choice of data type does not seem to affect the size of the table.
Server version: 5.1.41-3ubuntu12.10 (Ubuntu)
Example:
mysql> describe tinyint_test;
+----------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------+------+-----+---------+-------+
| id | int(11) | YES | | NULL | |
| test_int | tinyint(4) | YES | | NULL | |
+----------+------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
mysql> describe tinyint_id_test;
+-------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------+------+-----+---------+-------+
| id | tinyint(4) | YES | | NULL | |
+-------+------------+------+-----+---------+-------+
1 row in set (0.00 sec)
mysql> describe int_test;
+--------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------+---------+------+-----+---------+-------+
| not_id | int(11) | YES | | NULL | |
+--------+---------+------+-----+---------+-------+
1 row in set (0.00 sec)
mysql> select * from tinyint_test;
+------+----------+
| id | test_int |
+------+----------+
| 1 | 1 |
| 2 | 2 |
| 3 | 127 |
| 10 | 50 |
+------+----------+
4 rows in set (0.00 sec)
mysql> select * from tinyint_id_test;
+------+
| id |
+------+
| 1 |
| 2 |
| 127 |
| 50 |
+------+
4 rows in set (0.00 sec)
mysql> select * from int_test;
+--------+
| not_id |
+--------+
| 1 |
| 2 |
| 127 |
| 50 |
+--------+
4 rows in set (0.00 sec)
mysql> SELECT TABLE_NAME, DATA_LENGTH FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA like '%test%';
+-----------------+-------------+
| TABLE_NAME | DATA_LENGTH |
+-----------------+-------------+
| int_test | 28 |
| tinyint_id_test | 28 |
| tinyint_test | 28 |
+-----------------+-------------+
3 rows in set (0.00 sec)
I vaguely suspect that there might be an internal column in each row, or that the minimum data size for a given row must be at least the size of a full INT, but neither of these suspicions really account for what's happening here. What could be the case is my choice of DATA_LENGTH is the incorrect tool for measuring the true size of the tables, in which case an acceptable answer would point me in the right direction for actually measuring these tables.
EDIT:
I can generate a table of a different size by using two INTs:
mysql> describe int_id_test;
+----------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------+------+-----+---------+-------+
| id | int(11) | YES | | NULL | |
| test_int | int(11) | YES | | NULL | |
+----------+---------+------+-----+---------+-------+
2 rows in set (0.01 sec)
mysql> select * from int_id_test;
+------+----------+
| id | test_int |
+------+----------+
| 1 | 1 |
| 2 | 2 |
| 3 | 127 |
| 10 | 50 |
+------+----------+
4 rows in set (0.00 sec)
mysql> SELECT TABLE_NAME, DATA_LENGTH FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA like '%test%';
+-----------------+-------------+
| TABLE_NAME | DATA_LENGTH |
+-----------------+-------------+
| int_id_test | 36 |
| int_test | 28 |
| tinyint_id_test | 28 |
| tinyint_test | 28 |
+-----------------+-------------+
4 rows in set (0.01 sec)
the data_length column is how much hard drive space the operating system allocates
for a table.
mysql database page sizes configurable default is 16KB, the three table's data may used same pages, so the data_length are same!!
edit:
innodb engine default page size is 16KB, i don't know this size for other engines
I have found a work around for this problem as well as something of an explanation.
After looking at the table structure in a hex editor (on my linux machines these were located in /var/lib/mysql/[DATABASE NAME]/[TABLE NAME].MYD), I found that in all cases the records were created using a minimum of 7 bytes for a row, regardless of the actual data types involved. Any extra bytes that were not used by the table were zeroed out.
Here is an example with a smaller data set to illustrate:
mysql> describe int_test_2;
+-------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+---------+------+-----+---------+-------+
| id | int(11) | YES | | NULL | |
+-------+---------+------+-----+---------+-------+
1 row in set (0.00 sec)
mysql> select * from int_test_2;
+------+
| id |
+------+
| 1 |
| 2 |
+------+
2 rows in set (0.00 sec)
Looking at this guy in a hex editor, we see:
fd01 0000 0000 00fd 0200 0000 0000
Using information from Neo's link, I was able to decode this row:
fd Record header bits.
01000000 Integer value "1" (little endian)
0000 Wasted Space!
fd Record header bits.
02000000 Integer value "2" (little endian)
0000 Wasted Space!
However, notice the following:
mysql> alter table int_test_2 MAX_ROWS=50000000, AVG_ROW_LENGTH=4;
Query OK, 2 rows affected (0.01 sec)
Records: 2 Duplicates: 0 Warnings: 0
Now, the MYD file looks like this:
fd01 0000 00fd 0200 0000
That is, it uses the correct sizes.
One thing to note is that the number in brackets does not effect the size of that column, i.e an INT(4) is the same size as an INT(11) in terms of storage, all the number in brackets does is pad the returned value with spaces so that it fills 11 or 4 characters.
I suspect if you trully want to work out the size of the tables, you will need to look in the MySQL file itself and see how they are stored. All the data is stored in /var/lib/mysql/ - ibdata & ib_logfile are the main files. Open this in a text editor (Caution - this file may be HUGE depending on the sizes of your databases.. also DO NOT modify this file!!)
All the tables and cells are stored in here, however they are not delimeted, so its very difficult to see where one column ends and the next begins - it is all based on the data size which you are trying to establish. If you know the data in the table you should be able to work out the structure.
Edit: I think some of the data in these files may be stored in hex, so if it doesnt immediately make sense, try a hex editor.