I do use InnoDB with MySQL
Hey,
I do have two tables:
First one is called adtells and contains very few elements and is rarely updated.
Second one contains a lot of elements and gets new inserts on a regular base.
The second table stores, which user has hidden an element out of the first table. I think I'm not able to join these tables.
My first query was something like this:
SELECT xy FROM firstTable WHERE 1 NOT IN (SELECT abc FROM secondTable)
My second approach are two queries:
The first selects all needed elements out of the second table. With those results, I access the first table.
What are your thoughts to those approaches? Is the first one or the second one more performant or do you have any other ideas?
Further explanation of the given example:
Table 1:
id | content
Table 2:
userId, tableOneId
The second table contains an assignment from a given user to an element of the first table. If there is an assignment, the affected tableOneId shall not be retrieved by the queries above.
Would love to hear from you!
Use NOT EXISTS
SELECT t1.* FROM firstTable T1
WHERE
NOT EXISTS
(SELECT 1 FROM secondTable T2 where t2.id = t1.id)
NOT EXISTS/EXISTS will terminate when the first match is found
IN will search all rows regardless of how many matches.. if there are 100 matches then 100 iterations.
Based on rule optimizer:
EXISTS is much faster than IN, when the sub-query results is very
large. IN is faster than EXISTS, when the sub-query results is very
small. Based on cost optimizer:
There is no difference.
Difference between EXISTS and IN in SQL?
Related
So I've been searching for a solution and reading books, and havent been able to figure it out, the question is rather simple, I have 2 tables. On one table I have 2 fields:
table_1:"chromosome" and "position" both of the being integers.
table_2:"chromosome" "start" and "end", all being integers as well.
I want a query that gives me back all rows from table_1 that are between the start and end of table_2. The query looks like this:
SELECT
table_1 . *
FROM
table_1,
table_2
WHERE
table_1.chromosome = table_2.chromosome
AND table_1.position > table_2.start
AND table_1.position < table_1.end;
So this query works fine, but my tables are many millions of rows (7092713) and (215909) respectvely. I indexed chromosome, pos and chromosome, start, end. The weird part is that if I do the query one by one (perl DBI, do one statement for every row of table_2), this runs a lot faster. Not sure where am I screwing up.
Any help would be appreciated.
Jorge Kageyama
For the sake of clarity, let's start by recasting your query using the standard JOIN syntax. The query is equivalent but easier to read.
SELECT table_1 . *
FROM table_1
JOIN table_2 ON ( table_1.chromosome = table_2.chromosome
AND table_1.position > table_2.start
AND table_1.position < table_1.end)
Second, it's smart when searching large tables (or any tables for that matter) to avoid * in your SELECT clauses. Using * denies useful data to the optimizer about what you do, or don't, need in your result set. So let us say
SELECT table_1.chromosome, table_1.position
for SELECT.
So, it becomes clear that your result set, and your join, need chromosome and position, and nothing else, from your larger table. Try creating a compound BTREE index on that table, as follows.
CREATE INDEX ON table_1(chromosome,position) USING BTREE
Similarly, try creating an index on table_2 as follows.
CREATE INDEX ON table_2(chromosome,start, end) USING BTREE
These are called covering indexes. They contain enough columns that the query can be satisfied from the index without having to bounce back to the original table.
BTREE indexes (the default by the way) are inherently ordered. Appropriate records in table_1 can be found by range scans on the index starting with (chromosome,start) and ending with (chromosome,end).
Third, it's possible you're getting a massive combinatorial explosion of rows from table_1 in your result set. You'll get a row for every combination of rows in the two tables that matches your ON() clause. It's hard to know whether that's the case without knowing a lot about your data.
You could try to reduce that combinatorial explosion using
SELECT DISTINCT table_1.chromosome, table_1.position
Give this a try. If you're still not getting anywhere, maybe another question with complete table definitions and the results of EXPLAIN will be helpful.
Interesting question. Without knowing more about the quantities contained in "position," I would still approach it generally in this way:
Select for position generally from table_1 (with 7.0mm entities) so that the resulting table is a bin of a smaller amount of data. Let's say, for instance, that the "position" quantity is a set of discrete integers from 2-9. Select from table_1 where position is equal to 2, then select from table_2 where "start" is less than 2 and "end" is greater than 2. Iterate over this query selection 8 times updating a new table_3 with results.
I am assuming here that table_2 is unique on chromosome, and table_1 is not. Therefore, you end up with chromosomes that could have multiple positions within the same range (a chromosome has one range, but can appear anywhere within that range). You also, then, can't tell how large the resulting join table is going to be, but it could be quite large as each of the 7mm entities in table_1 could be within all ranges in table_2.
Iterating would allow you to "grow" your results while observing the quality at each point experimentally before committing to the entire loop.
Here is an idea of the query I have in mind (untested):
SELECT table_1.chromosome, table_1.position, table_2.start, table_2.end
FROM
(SELECT table_1.chromosome, table_1.position
from table_1 where table_1.position = 2)
JOIN
(SELECT table_2.chromosome, table_2.start, table_2.end
from table_2 where table_2.start < 2 AND table_2.end > 2)
ON
table_1.chromosome = table_2.chromosome
Good luck, and I hope you find your answer!
Here is my setup:
Table records contains multiple (more than two) PKID columns along with some other columns.
Table cached_records only has two columns, which are the same as two of the PKIDs for records.
For instance, let's assume records has PKIDs 'keyA', 'keyB', and 'keyC' and cached_records only has 'keyA' and 'keyB'.
I need to pull the rows from the records table where the appropriate PKIDs (so, 'keyA' and 'keyB') are not in the cached_records table.
IF I was working with only ONE PKID, I know how simple this task would be:
SELECT
pkid
FROM
records
WHERE
pkid NOT IN (SELECT pkid FROM cached_records)
However, the fact that there is two PKIDs means I can't use a simple NOT IN. This is what I currently have:
SELECT
`keys`.`keyA` AS `keyA`,
`keys`.`keyB` AS `keyB`
FROM
(
SELECT DISTINCT
`keyA`,
`keyB`
FROM
`records`
) AS `keys`
LEFT JOIN
`cached_records` AS `cached`
ON
`keys`.`keyA` = `cached`.`keyA`
AND
`keys`.`keyB` = `cached`.`keyB`
WHERE
(
`cached`.`keyA` IS NULL
AND
`cached`.`keyB` IS NULL
)
(The DISTINCT is needed because since I am only grabbing two of the multiple PKIDs from the records table, there could be duplicates and I really don't need duplicates; 'keyC' is not being used and it helps determine uniqueness of the records).
This query above works just fine, however, as the cached_records table grows, the query takes longer and longer to process (we're talking minutes now, sometimes takes long enough that my code hangs and crashes).
So, I'm wondering what the most efficient way is to do this kind of operation (selecting rows from one table where the rows don't exist in another) with multiple PKIDS instead of just one...
This should be quicker:
SELECT DISTINCT
`records`.`keyA` AS `keyA`,
`records`.`keyB` AS `keyB`
FROM
`records`
LEFT JOIN
`cached_records` AS `cached`
ON
`records`.`keyA` = `cached`.`keyA`
AND
`records`.`keyB` = `cached`.`keyB`
WHERE
`cached`.`keyA` IS NULL -- one is enough here
Notes:
with the query as table, you lose a lot of performance. You can do the distinct in the outmost SELECT here.
it is enough to check one of the two keys if they are null, as none can be null
you should verify that the keyA and keyB columns are of the same type, and no conversion occurs (seen such in working live code...)
You should have proper indexes on the tables. Minutes for this query is the sign of something awful going on... (Or an insane amount of data)
I sort the rows on date. If I want to select every row that has a unique value in the last column, can I do this with sql?
So I would like to select the first row, second one, third one not, fourth one I do want to select, and so on.
What you want are not unique rows, but rather one per group. This can be done by taking the MIN(pk_artikel_Id) and GROUP BY fk_artikel_bron. This method uses an IN subquery to get the first pk_artikel_id and its associated fk_artikel_bron for each unique fk_artikel_bron and then uses that to get the remaining columns in the outer query.
SELECT * FROM tbl
WHERE pk_artikel_id IN
(SELECT MIN(pk_artikel_id) AS id FROM tbl GROUP BY fk_artikel_bron)
Although MySQL would permit you to add the rest of the columns in the SELECT list initially, avoiding the IN subquery, that isn't really portable to other RDBMS systems. This method is a little more generic.
It can also be done with a JOIN against the subquery, which may or may not be faster. Hard to say without benchmarking it.
SELECT *
FROM tbl
JOIN (
SELECT
fk_artikel_bron,
MIN(pk_artikel_id) AS id
FROM tbl
GROUP BY fk_artikel_bron) mins ON tbl.pk_artikel_id = mins.id
This is similar to Michael's answer, but does it with a self-join instead of a subquery. Try it out to see how it performs:
SELECT * from tbl t1
LEFT JOIN tbl t2
ON t2.fk_artikel_bron = t1.fk_artikel_bron
AND t2.pk_artikel_id < t1.pk_artikel_id
WHERE t2.pk_artikel_id IS NULL
If you have the right indexes, this type of join often out performs subqueries (since derived tables don't use indexes).
This non-standard, mysql-only trick will select the first row encountered for each value of pk_artikel_bron.
select *
...
group by pk_artikel_bron
Like it or not, this query produces the output asked for.
Edited
I seem to be getting hammered here, so here's the disclaimer:
This only works for mysql 5+
Although the mysql specification says the row returned using this technique is not predictable (ie you could get any row as the "first" encountered), in fact in all cases I've ever seen, you'll get the first row as per the order selected, so to get a predictable row that works in practice (but may not work in future releases but probably will), select from an ordered result:
select * from (
select *
...
order by pk_artikel_id) x
group by pk_artikel_bron
I join table A to table B and need to know if table B has 1 matching row or more than one.
Of course, I can do it with GROUP BY and COUNT, but it's an overkill, because it has to count all the matches and I don't need this info.
Is there a simple way to get the info I need (only one matching row or more) which short circuits the evaluation and stops when it knows the answer without scanning and counting all the remaining matches?
Or should I not care about this, becasue it's not a big performance hit and I should simply go with COUNT?
It really depends on the size of the DB, and your exact requirements. Generally a count()/Group By/Having combination is a pretty efficient query, with the right indexes. You could do it in a more complicated way, for example, having a trigger on after update that keeps a count table updated.
Are you seeing the count(*)/group/having combination giving you performance issues?
If you just need to know if there is one or more than one row for a certain join sql, meaning a matching row:
-- Without any sample SQL code, here's a return sample
SELECT B.SOMEJOINAPPLICABLECOLUMN
FROM A
LEFT OUTER JOIN B
ON A.SOMEJOINAPPLICABLECOLUMN = B.SOMEJOINAPPLICABLECOLUMN
WHERE
B.SOMEJOINAPPLICABLECOLUMN IS NOT NULL
LIMIT 2;
Naturally:
2 returned rows = more than one match
1 returned row = one match
0 returned rows = no matches
I have seen many queries with something as follows.
Select 1
From table
What does this 1 mean, how will it be executed and, what will it return?
Also, in what type of scenarios, can this be used?
select 1 from table will return the constant 1 for every row of the table. It's useful when you want to cheaply determine if record matches your where clause and/or join.
SELECT 1 FROM TABLE_NAME means, "Return 1 from the table". It is pretty unremarkable on its own, so normally it will be used with WHERE and often EXISTS (as #gbn notes, this is not necessarily best practice, it is, however, common enough to be noted, even if it isn't really meaningful (that said, I will use it because others use it and it is "more obvious" immediately. Of course, that might be a viscous chicken vs. egg issue, but I don't generally dwell)).
SELECT * FROM TABLE1 T1 WHERE EXISTS (
SELECT 1 FROM TABLE2 T2 WHERE T1.ID= T2.ID
);
Basically, the above will return everything from table 1 which has a corresponding ID from table 2. (This is a contrived example, obviously, but I believe it conveys the idea. Personally, I would probably do the above as SELECT * FROM TABLE1 T1 WHERE ID IN (SELECT ID FROM TABLE2); as I view that as FAR more explicit to the reader unless there were a circumstantially compelling reason not to).
EDIT
There actually is one case which I forgot about until just now. In the case where you are trying to determine existence of a value in the database from an outside language, sometimes SELECT 1 FROM TABLE_NAME will be used. This does not offer significant benefit over selecting an individual column, but, depending on implementation, it may offer substantial gains over doing a SELECT *, simply because it is often the case that the more columns that the DB returns to a language, the larger the data structure, which in turn mean that more time will be taken.
If you mean something like
SELECT * FROM AnotherTable
WHERE EXISTS (SELECT 1 FROM table WHERE...)
then it's a myth that the 1 is better than
SELECT * FROM AnotherTable
WHERE EXISTS (SELECT * FROM table WHERE...)
The 1 or * in the EXISTS is ignored and you can write this as per Page 191 of the ANSI SQL 1992 Standard:
SELECT * FROM AnotherTable
WHERE EXISTS (SELECT 1/0 FROM table WHERE...)
it does what it says - it will always return the integer 1. It's used to check whether a record matching your where clause exists.
select 1 from table is used by some databases as a query to test a connection to see if it's alive, often used when retrieving or returning a connection to / from a connection pool.
The result is 1 for every record in the table.
To be slightly more specific, you would use this to do
SELECT 1 FROM MyUserTable WHERE user_id = 33487
instead of doing
SELECT * FROM MyUserTable WHERE user_id = 33487
because you don't care about looking at the results. Asking for the number 1 is very easy for the database (since it doesn't have to do any look-ups).
Although it is not widely known, a query can have a HAVING clause without a GROUP BY clause.
In such circumstances, the HAVING clause is applied to the entire set. Clearly, the SELECT clause cannot refer to any column, otherwise you would (correct) get the error, "Column is invalid in select because it is not contained in the GROUP BY" etc.
Therefore, a literal value must be used (because SQL doesn't allow a resultset with zero columns -- why?!) and the literal value 1 (INTEGER) is commonly used: if the HAVING clause evaluates TRUE then the resultset will be one row with one column showing the value 1, otherwise you get the empty set.
Example: to find whether a column has more than one distinct value:
SELECT 1
FROM tableA
HAVING MIN(colA) < MAX(colA);
If you don't know there exist any data in your table or not, you can use following query:
SELECT cons_value FROM table_name;
For an Example:
SELECT 1 FROM employee;
It will return a column which contains the total number of rows & all rows have the same constant value 1 (for this time it returns 1 for all rows);
If there is no row in your table it will return nothing.
So, we use this SQL query to know if there is any data in the table & the number of rows indicates how many rows exist in this table.
If you just want to check a true or false based on the WHERE clause, select 1 from table where condition is the cheapest way.
This means that You want a value "1" as output or Most of the time used as Inner Queries because for some reason you want to calculate the outer queries based on the result of inner queries.. not all the time you use 1 but you have some specific values...
This will statically gives you output as value 1.
I see it is always used in SQL injection,such as:
www.urlxxxxx.com/xxxx.asp?id=99 union select 1,2,3,4,5,6,7,8,9 from database;
These numbers can be used to guess where the database exists and guess the column name of the database you specified.And the values of the tables.
it simple means that you are retrieving the number first column from table ,,,,means
select Emply_num,Empl_no From Employees ;
here you are using select 1 from Employees;
that means you are retrieving the Emply_num column.
Thanks
The reason is another one, at least for MySQL. This is from the MySQL manual
InnoDB computes index cardinality values for a table the first time that table is accessed after startup, instead of storing such values in the table. This step can take significant time on systems that partition the data into many tables. Since this overhead only applies to the initial table open operation, to “warm up” a table for later use, access it immediately after startup by issuing a statement such as SELECT 1 FROM tbl_name LIMIT 1
This is just used for convenience with IF EXISTS(). Otherwise you can go with
select * from [table_name]
Image In the case of 'IF EXISTS', we just need know that any row with specified condition exists or not doesn't matter what is content of row.
select 1 from Users
above example code, returns no. of rows equals to no. of users with 1 in single column