Say you have the following query:
SELECT * FROM table1 WHERE table1.id IN (1, 2, 3, 4, 5, ..., 999999)
What is a reasonable maximum for the number of items in the IN clause? I'm using Sphinx to generate full-text search results and inserting the IDs into a MySQL query. Is this an acceptable way to do it?
You can also have the IN clause take the results of a query, such as:
SELECT * FROM table1
WHERE table1.id IN
(
SELECT id from table2
)
That way, you don't need to generate a text string with all the possible values.
In mysql, you should be able to put as many values in the IN clause as you want, only constrained by the value of "max_allowed_packet".
http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_max_allowed_packet
MariaDB (10.3.22 in my case) has a limit of 999 parameters to IN() before it creates a materialized temporary table, resulting in possibly much longer execution times. Depending on your indices. I haven't found a way to control this behaviour. MySQL 5.6.27 does not have this limit, at least not at ~1000 parameters. MySQL 5.7 might very well have the same "feature".
I ended up doing a series of where id = a or id = b ... but it also works fine by using series of where id in(a, b) or id in(c, d) ....
You have to add laravel row query and then add NOT IN condition into this:
$object->whereRaw('where id NOT IN (' . $array_list . ') ');
This will work for my code.
From my experience the maximum values is 1000 values in clause IN ('1',....,'1000'),
I have 1300 value in my excel sheet,I put them all into IN ,MySQL return only 1000 .
Related
Say you have the following query:
SELECT * FROM table1 WHERE table1.id IN (1, 2, 3, 4, 5, ..., 999999)
What is a reasonable maximum for the number of items in the IN clause? I'm using Sphinx to generate full-text search results and inserting the IDs into a MySQL query. Is this an acceptable way to do it?
You can also have the IN clause take the results of a query, such as:
SELECT * FROM table1
WHERE table1.id IN
(
SELECT id from table2
)
That way, you don't need to generate a text string with all the possible values.
In mysql, you should be able to put as many values in the IN clause as you want, only constrained by the value of "max_allowed_packet".
http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_max_allowed_packet
MariaDB (10.3.22 in my case) has a limit of 999 parameters to IN() before it creates a materialized temporary table, resulting in possibly much longer execution times. Depending on your indices. I haven't found a way to control this behaviour. MySQL 5.6.27 does not have this limit, at least not at ~1000 parameters. MySQL 5.7 might very well have the same "feature".
I ended up doing a series of where id = a or id = b ... but it also works fine by using series of where id in(a, b) or id in(c, d) ....
You have to add laravel row query and then add NOT IN condition into this:
$object->whereRaw('where id NOT IN (' . $array_list . ') ');
This will work for my code.
From my experience the maximum values is 1000 values in clause IN ('1',....,'1000'),
I have 1300 value in my excel sheet,I put them all into IN ,MySQL return only 1000 .
Let assume Following simple table
Col1
======
one
two
Let assume Following simple query
Select count(*) from TABLE_A where Col1 in ('one','two','three','four')
In above query it will produce following result
2
Now I want to find out what are the values in IN- condition which is not available in table_A.
How to find out that values which are not available in table?
like below result
three
four
Above queries only example. In my real time query in have 1000 values in IN-Condition.
Working Database : DB2
This is the one of the work around to achieve your expectation.
Instead of hard-coding the values in IN condition, you can move those values in to a table. If it done simply using LEFT JOIN with NULL check you can get the not matching values.
SELECT MR.Col1
FROM MatchingRecords MR -- here MatchingRecords table contains the IN condition values
LEFT JOIN Table_A TA ON TA.Col1 = MR.Col1
WHERE TA.Col1 IS NULL;
Working DEMO
If the values are to be listed in the statement string rather than stored in a table, then perhaps a revision to the syntax being used for that list of values currently being composed [apparently, from some other input than a TABLE] for the IN predicate can be effected? The following revised syntax for a list of values could be used both for the original aggregate query [shown immediately below as the first of two queries], and for the query for which the how-to-code is being asked [the second of the two queries below]:
Select count(*)
from TABLE_A
where Col1 in ( values('one'),('two'),('three'),('four') )
; -- report from above query follows:
COUNT ( * )
2
[Bgn-Edit 05-Aug-2016: adding this text and example just below]Apparently at least one DB2 variant balks at unnamed columns for the derived table, so the query just below names the column; I chose COL1, so as to match the name from the actual TABLE, but that should not be necessary. The (col1) is added to the original query that remains from the original pre-edit version; that version remains after this edit\insertion and is missing the (col1) added here:
select *
from ( values('one'),('two'),('three'),('four') ) as x (col1)
except ( select * from table_a )
; -- report from above query follows:
COL1
three
four
The following is the original query given, for which the comment below suggests a failure for an unnamed column when run on some unstated DB2 variant; I should have noted that this SQL query functions without error, on DB2 for i 7.1
[End-Edit 05-Aug-2016]
select *
from ( values('one'),('two'),('three'),('four') ) as x
except ( select * from table_a )
; -- report from above query follows:
VALUES
three
four
Is there a way to retrieve the column names of a query that returns no data?
The result of this query would be empty.
Is there a way how to find the column names when there's no result?
Please note that I'm aware of solutions using DESCRIBE and select column_name from information_schema.columns where table_name='person';
but I need a more flexible solution that will fit these multicolumn queries.
Please also note that I am still using the original PHP MySQL extention (so no MySQLi, and no PDO).
If you wrap your query with the following SQL, it will always return the column names from your query, even if it is an empty query result:
select myQuery.*
from (select 1) as ignoreMe
left join (
select * from myTable where false -- insert your query here
) as myQuery on true
Note: When the results of the subquery are empty, a single row of null values will be returned. If there is data in the subquery it won't affect the output because it creates a cross-product with a single row...and value x 1 = value
Execute following command if the result of your previous query is empty
SHOW columns FROM your-table;
For more details check this.
I'm not sure if it will satisfy you but you can do this
SELECT *, COUNT(*) FROM table;
It will return null values (except last column which you can ignore) if the query is empty and you will be able to access all columns. It's not proper way of doing it and selecting names from INFORMATION_SCHEMA would be much better solution.
Please note that result is aggregated and you need to use GROUP BY to get more results if there are any.
You should ,
Select COLUMN_NAME From INFORMATION_SCHEMA.COLUMNS
Where TABLE_SCHEMA='yourdb'
AND TABLE_NAME='yourtablename';
I have a field number of type varchar. Even though it is of type varchar, it stores integer values with optional leading zeros. A sort orders them lexicographically ("42" comes before "9"). How can I order by numeric values ("9" to come before "42")?
Currently I use the query:
SELECT * FROM table ORDER BY number ASC
Try this
SELECT * FROM table_name ORDER BY CAST(field_name as SIGNED INTEGER) ASC
There are a few ways to do this:
Store them as numeric values rather than strings. You've already discounted that as you want to keep strings like 00100 intact with the leading zeros.
Order by the strings cast as numeric. This will work but be aware that it's a performance killer for decent sized databases. Per-row functions don't really scale well.
Add a third column which is the numeric equivalent of the string and index on that. Then use an insert/update trigger to ensure it's set correctly whenever the string column changes.
Since the vast majority of databases are read far more often than written, this third option above amortises the cost of the calculation (done at insert/update) over all selects. Your selects will be blindingly fast since they use the numeric column to order (and no per-row functions).
Your inserts and updates will be slower but that's the price you pay and, to be honest, it's well worth paying.
The use of the trigger maintains the ACID properties of the table since the two columns are kept in step. And it's a well-known idiom that you can usually trade off space for time in most performance optimisations.
We've used this "trick" in many situations, such as storing lower-cased versions of surnames alongside the originals (instead of using something like tolower), lengths of identifying strings to find all users with 7-character ones (instead of using len) and so on.
Keep in mind that it's okay to revert from third normal form for performance provided you understand (and mitigate) the consequences.
Actually i've found something interesting:
SELECT * FROM mytable ORDER BY LPAD(LOWER(mycol), 10,0) DESC
This allows you to order the field like:
1
2
3
10
A
A1
B2
10A
111
SELECT * FROM table ORDER BY number + 0
Trick I just learned. Add '+0' to the varchar field order clause:
SELECT * FROM table ORDER BY number+0 ASC
I now see this answer above. I am wondering if this is typecasting the field and an integer. I have not compared performance. Working great.
For a table with values like Er353, ER 280, ER 30, ER36
default sort will give
ER280
ER30
ER353
ER36
SELECT fieldname, SUBSTRING(fieldname, 1, 2) AS bcd,
CONVERT(SUBSTRING(fieldname, 3, 9), UNSIGNED INTEGER) AS num
FROM table_name
ORDER BY bcd, num;
the results will be in this order
ER30
ER36
ER280
ER353
you can get order by according to your requirement my using following sql query
SELECT * FROM mytable ORDER BY ABS(mycol)
given a column username containing VARCHAR's like these:
username1
username10
username100
one could do:
SELECT username,
CONVERT(REPLACE(username, 'username', ''), UNSIGNED INTEGER) AS N
FROM users u
WHERE username LIKE 'username%'
ORDER BY N;
it is not cheap, but does the job.
SELECT * FROM table ORDER BY number ASC
Should display what you want it to display.. looks like you're sorting it by id or number is not defined as integer at the moment.
MySQL ORDER BY Sorting alphanumeric on correct order
example:
SELECT `alphanumericCol` FROM `tableName` ORDER BY
SUBSTR(`alphanumericCol` FROM 1 FOR 1),
LPAD(lower(`alphanumericCol`), 10,0) ASC
output:
0
1
2
11
21
100
101
102
104
S-104A
S-105
S-107
S-111
Another option to keep numerics at a top, then order by alpha.
IF(name + 0, name + 0, 9999999), name
Rough and ready: order by 1*field_name
So I have a data with format like ;1;;2; and then I need to use this number in a query so I thought I'd convert it to 1,2 and use that in a IN condition. In my table, the result should return 2 rows but instead it is returning only 1 row.
My query is like this. The subquery return 1,2 with no problem but only 1 row is retrieve.
select *
from wt_lists
where id IN ((select replace (replace(sendto, ';;',','),';','')
from wt_stats where statsid IN (1)))
But when I try it with this. It returns the correct result, which in my case is 2 rows.
select *
from wt_lists
where id IN (1,2)
What am I missing here?
Comma delimited strings need to be explicitly defined in the query in order to be used in the IN clause - there's countless examples on SO where people need to use dynamic SQL to incorporate user submitted comma delimited strings.
That said, I have a solution using the FIND_IN_SET function:
SELECT DISTINCT wl.*
FROM WT_LISTS wl
JOIN (SELECT REPLACE(REPLACE(ws.sendto, ';;',','),';','') AS ids
FROM WT_STATS ws
WHERE ws.statsid = 1) x ON FIND_IN_SET(wl.id, x.ids) > 0
You are replacing the string:
';1;;2;'
To:
'1,2'
So, you SQL query looks like:
select * from wt_lists where id IN ('1,2') from wt_stats where statsid IN (1)
To use IN clause you need select different values in different rows.
I found this store procedure that does exactly what you need.
http://kedar.nitty-witty.com/blog/mysql-stored-procedure-split-delimited-string-into-rows/
I have not tested, but it is the way.
Obs: Like David said in the comments above, parsing the data in your application is a better way to do this.