Slow InfiniDB queries, what am I doing wrong? - mysql

I'm testing the InfiniDB community edition to see if it suits our needing.
I imported in a single table about 10 millions rows (loading of data was surprisingly fast), and I'm trying to do some query on it, but these are the results (with NON cached queries.. if query caching exists in InfiniDB):
Query 1 (very fast):
select * from mytable limit 150000,1000
1000 rows in set (0.04 sec)
Query 2 (immediate):
select count(*) from mytable;
+----------+
| count(*) |
+----------+
| 9429378 |
+----------+
1 row in set (0.00 sec)
Ok it seems to be amazingly fast.. but:
Query 3:
select count(title) from mytable;
.. still going after several minutes
Query 4:
select id from mytable where id like '%ABCD%';
+------------+
| id |
+------------+
| ABCD |
+------------+
1 row in set (11 min 17.30 sec)
I must be doing something wrong, it's not possible that it's performing this way with so simple queries. Any Idea?

That shouldn't be the case, there does appear to be something odd going on, see quick test below.
What is your server configuration: memory/OS/CPU and platform (dedicated, virtual, cloud).
Could I get the schema declaration and method to load the data?
Which version are you using? Version 4 community has significantly more features than prior versions, i.e. core syntax matches enterprise.
Cheers,
Jim T
mysql> insert into mytable select a, a from (select hex(rand() * 100000) a from lineitem limit 10000000) b;
Query OK, 10000000 rows affected (1 min 54.12 sec)
Records: 10000000 Duplicates: 0 Warnings: 0
mysql> desc mytable;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id | varchar(32) | YES | | NULL | |
| title | varchar(32) | YES | | NULL | |
+-------+-------------+------+-----+---------+-------+
2 rows in set (0.01 sec)
mysql> select * from mytable limit 150000,1000;
+-------+-------+
| id | title |
+-------+-------+
| E81 | E81 |
| 746A | 746A |
. . .
| DFC8 | DFC8 |
| 2C56 | 2C56 |
+-------+-------+
1000 rows in set (0.07 sec)
mysql> select count(*) from mytable;
+----------+
| count(*) |
+----------+
| 10000000 |
+----------+
1 row in set (0.06 sec)
mysql> select count(title) from mytable;
+--------------+
| count(title) |
+--------------+
| 10000000 |
+--------------+
1 row in set (0.09 sec)
mysql> select id from mytable where id like '%ABCD%' limit 1;
+------+
| id |
+------+
| ABCD |
+------+
1 row in set (0.03 sec)

Related

Search for a text containing backslash in MySQL

I have a mysql table with 2 entries (id ,title) (int, text)
A row of dummy data in the table looks like
id title
1 apple\"apple
My query intends to do a search on the field title
select * from table1 where title='apple\"apple';
I also tried using
select * from table1 where title like('apple\"apple');
The above query did not seem to work as well.
Both queries are returning an empty sets. Is it not possible to search for a string that has backslash in it?
PS :: The \ is added to the DB as part of escaping the double quotes.
Sure? It works as expected.
mysql> CREATE TABLE MyTbl (id INT, title TEXT);
Query OK, 0 rows affected (0.01 sec)
mysql> INSERT INTO MyTbl VALUES(1,'apple "apple');
Query OK, 1 row affected (0.00 sec)
mysql> SELECT * FROM MyTbl;
+------+-------------+
| id | title |
+------+-------------+
| 1 | apple"apple |
+------+-------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM MyTbl WHERE title = 'apple \"apple';
+------+-------------+
| id | title |
+------+-------------+
| 1 | apple"apple |
+------+-------------+
1 row in set (0.00 sec)
mysql> SELECT * FROM MyTbl WHERE title like('apple \"apple');
+------+-------------+
| id | title |
+------+-------------+
| 1 | apple"apple |
+------+-------------+
1 row in set (0.00 sec)
Maybe Is it something related to the space character?
mysql> SELECT * FROM MyTbl WHERE title = 'apple\"apple';
Empty set (0.00 sec)
mysql> SELECT * FROM MyTbl WHERE title = 'apple \"apple';
+------+--------------+
| id | title |
+------+--------------+
| 1 | apple "apple |
+------+--------------+
1 row in set (0.00 sec)
What about running this command? What It returns for you?
mysql> SELECT #NO_BACKSLASH_ESCAPES;
+----------------------------------------------+
| #NO_BACKSLASH_ESCAPES |
+----------------------------------------------+
| NULL |
+----------------------------------------------+
1 row in set (0.00 sec)

MARIADB - Can SQL convert NULL to an empty string if no default value is provided?

I have a table that has nullable columns:
+-------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| id | int(11) | YES | | NULL | |
| name | varchar(255) | YES | | NULL | |
+-------+--------------+------+-----+---------+-------+
I insert a row with name set to NULL;
INSERT INTO some_table (id, name) VALUES (1, NULL);
Query OK, 1 row affected (0.02 sec)
SELECT * FROM some_table;
+------+------+
| id | name |
+------+------+
| 1 | NULL |
+------+------+
1 row in set (0.01 sec)
If I alter the table's name column to be not-nullable it apparently converts NULL to an empty string:
ALTER TABLE some_table CHANGE COLUMN name name VARCHAR(255) NOT NULL;
Query OK, 1 row affected, 1 warning (0.02 sec)
Records: 1 Duplicates: 0 Warnings: 1
SELECT * FROM some_table;
+------+------+
| id | name |
+------+------+
| 1 | |
+------+------+
1 row in set (0.02 sec)
At this point I would expect an exception to be raised telling me that I have NULL in my dataset and I can not set the column name to NOT NULL.
Is this a configurable option in SQL/MariaDB?
Why is NULL being converted to an empty string?
There is a warning being invoked when altering the table:
SHOW WARNINGS;
+---------+------+-------------------------------------------+
| Level | Code | Message |
+---------+------+-------------------------------------------+
| Warning | 1265 | Data truncated for column 'name' at row 1 |
+---------+------+-------------------------------------------+
1 row in set (0.01 sec)
Version:
SELECT version();
+----------------+
| version() |
+----------------+
| 5.5.62-MariaDB |
+----------------+
1 row in set (0.02 sec)
Apparently, from the documentation for ALTER TABLE, enabling strict mode would prevent your alter statement from succeeding:
This conversion may result in alteration of data. For example, if you shorten a string column, values may be truncated. To prevent the operation from succeeding if conversions to the new data type would result in loss of data, enable strict SQL mode before using ALTER TABLE.
One way to enable strict mode from within MySQL:
SET GLOBAL sql_mode='STRICT_TRANS_TABLES';
See here for other options.
Using 10.3.15-MariaDB-1 on Debian Buster, I cannot reproduce the problem:
MariaDB [foo]> CREATE TABLE some_table(id int(11), name varchar(255));
Query OK, 0 rows affected (0.009 sec)
MariaDB [foo]> INSERT INTO some_table (id, name) VALUES (1, NULL);
Query OK, 1 row affected (0.003 sec)
MariaDB [foo]> SELECT * FROM some_table;
+------+------+
| id | name |
+------+------+
| 1 | NULL |
+------+------+
1 row in set (0.000 sec)
MariaDB [foo]> ALTER TABLE some_table CHANGE COLUMN name name VARCHAR(255) NOT NULL;
ERROR 1265 (01000): Data truncated for column 'name' at row 1
MariaDB [foo]> SELECT * FROM some_table;
+------+------+
| id | name |
+------+------+
| 1 | NULL |
+------+------+
1 row in set (0.000 sec)
MariaDB [foo]> SELECT version();
+-------------------+
| version() |
+-------------------+
| 10.3.15-MariaDB-1 |
+-------------------+
1 row in set (0.000 sec)
If possible, I suggest you update your MariaDB version. It seems very old to me.

what syntax to use to update a SET column in mysql?

I created a column called oilcompany that has SET data (Hunt, Pioneer, Chevron, BP)
I can enter any one of those into the oilcompany column and change from one to another one but I can not figure out how to change from one oilcompany to multiple oilcompany (eg. Hunt and BP)... any suggestion?
In the MySQL documentation there are not examples for UPDATE statements, but I normally use two ways to update these kind of columns:
Using text values
Using numeric values
Creating the test environment
mysql> CREATE TABLE tmp_table(
-> id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
-> oilcompany SET('Hunt', 'Pioneer', 'Chevron', 'BP')
-> );
Query OK, 0 rows affected (0.54 sec)
mysql> INSERT INTO tmp_table(oilcompany) VALUES ('Hunt'), ('Pioneer');
Query OK, 2 rows affected (0.11 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM tmp_table;
+----+------------+
| id | oilcompany |
+----+------------+
| 1 | Hunt |
| 2 | Pioneer |
+----+------------+
2 rows in set (0.00 sec)
Alternative#1: Using Text Values
As a SET is a collection of ENUM elements, and any ENUM element can be treated as a string, then we can do things like:
mysql> UPDATE tmp_table
-> SET oilcompany = 'Hunt,BP'
-> WHERE id = 1;
Query OK, 1 row affected (0.07 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> SELECT * FROM tmp_table;
+----+------------+
| id | oilcompany |
+----+------------+
| 1 | Hunt,BP |
| 2 | Pioneer |
+----+------------+
2 rows in set (0.00 sec)
Alternative#2: Using Numeric Values
Any SET element is stored internally as a 64bit number containing the combination of the bits that represent each SET element.
In our table: 'Hunt'=1, 'Pioneer'=2, 'Chevron'=4, 'BP'=8.
Also, mysql allows to use these numbers instead of text values. If we need to see the numeric value in the select, we need to use the SET column inside a numeric expression (E.g. adding zero).
Let's see the current values:
mysql> SELECT id, oilcompany+0, oilcompany FROM tmp_table;
+----+--------------+------------+
| id | oilcompany+0 | oilcompany |
+----+--------------+------------+
| 1 | 9 | Hunt,BP |
| 2 | 2 | Pioneer |
+----+--------------+------------+
2 rows in set (0.00 sec)
Here 9 = 'Hunt' (1) + 'BP' (8) and 2 = 'Pioneer' (2).
Now, let's change the Pioneer to 'Hunt' (1) + 'Chevron' (4):
mysql> UPDATE tmp_table
-> SET oilcompany = 5
-> WHERE id = 2;
Query OK, 1 row affected (0.08 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> SELECT id, oilcompany+0, oilcompany FROM tmp_table;
+----+--------------+--------------+
| id | oilcompany+0 | oilcompany |
+----+--------------+--------------+
| 1 | 9 | Hunt,BP |
| 2 | 5 | Hunt,Chevron |
+----+--------------+--------------+
2 rows in set (0.00 sec)

Find and replace a specific part of a string with a backslash in a field

For example this is myTable:
| i | data |
+---+---------+
| 1 | d\one |
| 2 | d\two |
| 3 | d\three |
But I want to change it to:
| i | data |
+---+-------+
| 1 | one |
| 2 | two |
| 3 | three |
I know how to find a specific part of a string with a backslash in a field:
SELECT * FROM myTable WHERE data LIKE 'd%\%'
I know how to find & replace a field a value, only this query won't work:
UPDATE myTable SET data=REPLACE(data,'d\','') WHERE data LIKE 'd%\%'
You need to escape the backslash
UPDATE myTable
SET data = REPLACE(data,'d\\','')
WHERE data INSTR('d\\') = 1
SQLFiddle demo
I hope you should put triple slash \\\ instead of double slash \\ in Like Clause. As I have tried I am unable to pick data from using \\
SELECT * FROM mytable WHERE data LIKE 'd\\%';
Empty set (0.00 sec)
mysql> SELECT * FROM mytable WHERE data LIKE 'd\\\%';
+---------+
| data |
+---------+
| d\one |
| d\two |
| d\three |
+---------+
3 rows in set (0.00 sec)
mysql> SELECT REPLACE(data,'d\\','') FROM mytable WHERE data LIKE 'd\\\%';
+------------------------+
| REPLACE(data,'d\\','') |
+------------------------+
| one |
| two |
| three |
+------------------------+
3 rows in set (0.00 sec)
mysql> UPDATE mytable SET data = REPLACE(data,'d\\','') WHERE data LIKE 'd\\\%';
Query OK, 3 rows affected (0.06 sec)
Rows matched: 3 Changed: 3 Warnings: 0
mysql> SELECT * FROM mytable;
+-------+
| data |
+-------+
| one |
| two |
| three |
+-------+
3 rows in set (0.00 sec)
try this one..
UPDATE myTable SET `data`=REPLACE(`data`,'d\','');

How to select from MySQL where a SET does not equal a value?

we have an issue because this is how the table was setup. How do we select from a table where a set does not contain a value? If we have a set with one,two and we do
SELECT *
FROM table
WHERE column NOT IN('one')
we will still get that row. What is the correct syntax for this?
Thanks
this is what worked:
SELECT *, FIND_IN_SET('hidden', props) As hidden
FROM gt_content
HAVING hidden IS NULL
Try:
SELECT *
FROM table
WHERE FIND_IN_SET('one',column) = 0
Some test I did:
mysql> CREATE TABLE setTest (s SET('a','b','c','d'));
Query OK, 0 rows affected (0.60 sec)
mysql> INSERT INTO setTest VALUES ('a'),('a,b'),('b'),('a,c'),('c,d');
Query OK, 5 rows affected (0.14 sec)
Records: 5 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM setTest;
+------+
| s |
+------+
| a |
| a,b |
| b |
| a,c |
| c,d |
+------+
5 rows in set (0.20 sec)
mysql> SELECT * FROM setTest WHERE FIND_IN_SET('a',s) = 0;
+------+
| s |
+------+
| b |
| c,d |
+------+
2 rows in set (0.12 sec)
mysql> SELECT * FROM setTest WHERE FIND_IN_SET('b',s) = 0;
+------+
| s |
+------+
| a |
| a,c |
| c,d |
+------+
3 rows in set (0.00 sec)
mysql> SELECT * FROM setTest WHERE FIND_IN_SET('b',s) > 0;
+------+
| s |
+------+
| a,b |
| b |
+------+
2 rows in set (0.04 sec)
From the mysql manual at http://dev.mysql.com/doc/refman/5.0/en/set.html:
Normally, you search for SET values using the FIND_IN_SET() function or the LIKE operator:
SELECT * FROM tbl_name WHERE FIND_IN_SET('value',set_col)>0;
SELECT * FROM tbl_name WHERE set_col LIKE '%value%';
The first statement finds rows where set_col contains the value set member. The second is similar, but not the same: It finds rows where set_col contains value anywhere, even as a substring of another set member.