I have two large tables, table_1 and table_2. Each has columns id, name and number. I want to find the records in table_1 that have matching names and numbers in table_2:
select t1.* from table_1 t1, table_2 t2
where t1.name = t2.name and t1.number = t2.number;
Or using the join syntax:
select t1.* from table_1 join table_2 on
t1.name = t2.name and t1.number = t2.number;
Both are very slow.
I added indexes for name and number in both tables. Still very slow. I then added a multi column index (name, number) on both tables. Still very slow.
Finally, I tried creating a combined column in both tables and set its value to concat(name, number). I added indexes in both tables for the combined column and then ran the following:
select t1.* from table_1 t1, table_2 t2
where t1.combined = t2.combined
Super fast!
Isn't there a more elegant way to achieve the performance without the awkward workaround of creating the combined column?
Related
I am using mySQL 5.6. I have two tables: t1 and t2. Both have many columns. And many columns in t1 and t2 share the same name: for example, there is a "var1" column in t1 and in t2.
I want to join the tables, selecting (a) all columns from t1 and (b) only the columns in t2 that have names that don't appear in t1. For example, I would not select "var1" from t2.
Here is a valid mySQL command that does not work because some columns share the same name:
SELECT * FROM t1 LEFT JOIN t2 ON (t1.ID=t2.ID);
MySQL sensibly returns this error message:
ERROR 1060 (42S21): Duplicate column name 'var1'
So I want to run a command like
SELECT t1.*, DISTINCTCOLUMNS(t2.*) FROM t1 LEFT JOIN t2 ON (t1.ID=t2.ID);
Except, of course, that there is no DISTINCTCOLUMNS operation in SQL. But is there a similar (real) command that will achieve the same effect?
I see that common advice is to avoid SELECT * syntax, partly for efficiency purposes. I appreciate that, but I do not want to write out the names of all of the columns that I need.
The real issue here is you have the same column name for both tables. So you need to alias your columns.. This is also why you shouldn't just pull all columns out but the specific ones you need..
SELECT t1.ID as t1_id,
t1.var1 as t1_var1,
t2.ID as t2_id,
t2.var1 as t2_var1,
... Etc.
FROM t1
LEFT JOIN t2 on t1.ID = t2.ID
With two ID columns and no way to distinguish between the two an error will occur
You could qualify the columns like
SELECT t1.*, t2.* FROM t1 LEFT JOIN t2 ON (t1.ID=t2.ID);
But you should not do it unless the columns with identical names do actually hold/reference the same data.
try like following. this may help you
SELECT t1.*, t2.YourDesiredColumnName FROM t1 LEFT JOIN t2 ON (t1.ID=t2.ID);
I have a simple task in MySQL. I have two tables t1 and t2. Table t1 has one column in common with table t2.
What I want is to return all the columns from both tables (order is not important) but making sure that the rows of both tables are matching based on a common field, say fieldx. However, not all the rows in table t1 have a matching row in table t2 (i.e. table t1 has much more rows, but all rows in t2 have a matching row in t1 ).
I tried to use where as such:
SELECT t1.* t2.* from t1, t2 where t1.fieldx=t2fieldx
However this returns only the number of rows as there are in table t2.
I also tried right and left joins, but couldn't get that to work either (this way not all columns of both tables were included..)
How would I do this?
The LEFT JOIN will give you all the rows from t1 and if t2 does not have an entry for it, t2 columns will appear as NULL:
SELECT t1.*,
t2.*
FROM t1
LEFT JOIN t2
ON t1.fieldx = t2.fieldx
Here is an Live DEMO of the above said, as you can see in t2, I've only referenced the first 3 records of t1 hence why the rest of the t2 information comes out NULL, as there is no t2 record to that given t1 entry.
I want is to return all the columns from both tables (order is not
important) but making sure that the rows of both tables are matching
based on a common field, say fieldx.
This query solve this:
SELECT t1.* t2.* FROM t1 INNER JOIN t2 WHERE t1.fieldx = t2fieldx
Further, if the next phrase is true, you can try with LEFT OUTER JOIN too (or RIGHT).
but all rows in t2 have a matching row in t1
SELECT t1.* t2.* FROM t2 LEFT OUTER JOIN t1 WHERE t1.fieldx = t2fieldx
What I am trying to do is join two tables, lets call them t1 and t2 on two columns. id and name, for this example. t1 will always have id and name, but t2 won't always have id and name. t1 has more columns like viewes, reports, and t2 has other columns that need to be joined. My question is, how can I show 0's for t2's columns if they don't exist?
I hav something similar to this, that joins tables only if both tables' rows have some value.
SELECT
date(t1.start_time) date,
t1.name,
t1.viewes,
t1.reports,
t2.col5,
t2.col6
from
table1 t1
left outer join table2 t2
on t2.name = t1.name and date(t2.start_time) = date(t1.start_time)
group by
1,2
order by
1 desc,
2 asc
;
I have lot's of experience with MySQL, but sometimes find that things need to be hacked to work correctly. What's your suggestion for this problem?
I am trying to spot some broken records in a MS-SQL Database.
In a simplified example, the scenerio is this:
I have 2 tables, simply put:
Table_1 : Id,Date,OpId
Table_2 : Date,OpId,EventName
And I have this business rule: If there is a record in Table_1 THEN at least 1 row should exist in the Table_2 for the Table_1.Date and Table.OpId.
If there is a row in Table_1 and if there is no row matching with that row in Table_2 THEN there is a broken data -whatever the reason-.
To find out the incorrect data, I use:
SELECT *
FROM table_1 t1
LEFT JOIN table_2 t2 ON t1.Date = t2.Date AND t1.OpId = t2.OpId
WHERE t2.OpId IS NULL -- So, if there is no
-- matching row in table_2 then this is a mistake
But it takes too long to have the query completed.
Can there be a faster or better way to approach similar scenerios?
To do an anti semi join NOT EXISTS in SQL Server is usually better than or equal to in performance the other options (NOT IN, OUTER JOIN ... NULL, EXCEPT)
SELECT *
FROM table_1 t1
WHERE NOT EXISTS (SELECT *
FROM table_2 t2
WHERE t1.Date = t2.Date
AND t1.OpId = t2.OpId)
See Left outer join vs NOT EXISTS. You may well be missing a useful index though.
If you use proper indexing there is nothing to do with it (may be use NOT EXISTS instead of LEFT JOIN will be a little bit faster),
BUT
if the Table_1 is has relatively small amount of data and there is no any FKeys or other such a stuff, and this is a one time procedure, then you can use trick like this to drop incorrect lines:
SELECT table_1.*
INTO tempTable
FROM table_1 t1
WHERE EXISTS(SELECT * FROM table_1 t1 WHERE t1.Date = t2.Date AND t1.OpId = t2.OpId)
drop table Table_1
exec sp_rename 'tempTable', 'Table_1'
This may be faster
Say I've two tables - "Table1" and "Table2" in my MySQL database.
"id" primary key (auto_increment) in "Table1" is the reference key in "Table2" - "tab_id".
There could be zero or more "Table2" rows for one "Table1" row.
Now I'm trying to do a search on one of the column in "Table2" say "email" column OR on one of the column in "Table1" say "address" and print "Table1" row values.
I see there are 3 possibilities:
1. Join
2. Sub-Query
3. Union
1 Join
SELECT *
FROM Table1 t1, Table t2
WHERE t1.id = t2.tab_id
AND (t1.address like '%str%' OR t2.email like '%str%');
-- This works fine, but when there are no rows in "Table2" relevant to "Table1" .. the JOIN will fail, hence output is in-consistent.
2 Sub-Query
SELECT *
FROM Table1 t1
WHERE t1.address like '%str%'
OR t1.id IN (SELECT t2.tab_id
FROM Table2 t2
WHERE t2.email like '%str%');
-- This works fine, but when there are two manys rows in "Table2" (say 5K) the query goes very slow :(
3 Union
SELECT 'relevant_columns'
FROM Table1 t1, Table t2
WHERE t1.id = t2.tab_id
AND (t1.address like '%str%' OR t2.email like '%str%')
UNION
SELECT 'relevant_columns'
FROM Table1 t1
WHERE t1.address like '%str%'
ORDER BY relevant_column
-- This works fine, may be create a view with a similar UNION, does the job.
Now, my question what is the correct way ... is it okay to call a UNION always?
MySQL Engine: MyISAM
SELECT *
FROM Table1 t1
LEFT JOIN Table t2 ON t2.tab_id = t1.id
WHERE t1.address like '%str%'
OR t2.email like '%str%';
You need to do a LEFT JOIN. When you make a FROM from two tables as you did, it works as an INNER JOIN (or a CROSS JOIN if there is no WHERE clause), which means that the output shows only rows that have a match in both tables. With LEFT JOIN you said that you want all rows from the left table (t1) with the matched row on the right table (t2). If there is no match in t2, then null is used.
You can use sub-query, but as you can see it is not the best choice
An UNION here does not give you any advantage. An UNION is useful to merge together datasets with same columns.
Edit
If you have issues with JOIN, because some Table1 rows do not appear, then you need a LEFT JOIN. The fact that takes a long time, is another problem. Those tables are not big at all, so I guess you need to do some index work on those tables.
If you want help about the union you need to tell me which are those relevant_columns, because they must have the same number of columns, same type and same sequence.
You might optimize the union without joins, depending on what you want to output when t2.email has a match. Here is an example
SELECT t1.id, t1.address, null as email
FROM Table1 t1
WHERE t1.address like '%str%'
union
SELECT t2.tab_id as id, null as address, t2.email
FROM Table t2
WHERE t2.email like '%str%';
SELECT *
FROM Table1 t1
LEFT JOIN Table2 t2
ON t1.id = t2.tab_id
WHERE t1.address like '%str%' OR t2.email like '%str%';