Say I need to pull data from several tables like so:
item 1 - from table 1
item 2 - from table 1
item 3 - from table 1 - but select only max value of item 3 from table 1
item 4 - from table 2 - but select only max value of item 4 from table 2
My query is pretty simple:
select
a.item 1,
a.item 2,
b.item 3,
c.item 4
from table 1 a
left join (select b.key_item, max(item 3) from table 1, group by key_item) b on a.key_item = b.key_item
left join (select c.key_item, max(item 4) from table 2, group by key_item) c on c.key_item = a.key_item
I am not sure if my methodology of pulling just a single max item from a table is the most efficient. Assume both tables are over a million rows. my actual sql run forever using this sql setup.
EDIT: I changed the group by clause to reflect comments made. I hope it makes a bit of sense now?
Your best bet is to add an index on table1 and table2, as follows:
ALTER TABLE table1
ADD INDEX `GoodIndexName1` (`key_item`,`item3`)
ALTER TABLE table2
ADD INDEX `GoodIndexName2` (`key_item`,`item4`)
This will allow you to use queries as described in the MySQL documentation for finding the rows holding the group-wise maximum, which appears to be what you are looking for.
Your original (edited) query should work:
select
a.item1,
a.item2,
b.item3,
c.item4
from table1 a
LEFT OUTER JOIN (
SELECT
b.key_item,
MAX(item3) AS item3
FROM table1
GROUP BY key_item
) b
ON a.key_item = b.key_item
LEFT OUTER JOIN (
SELECT
c.key_item,
MAX(item4)
FROM table2
GROUP BY key_item
) c
ON c.key_item = a.key_item
and if that performs slowly after adding the indexes, try the following too:
SELECT
a.item1,
a.item2,
b.item3,
c.item4
FROM table1 a
LEFT OUTER JOIN table1 b
ON b.key_item = a.key_item
LEFT OUTER JOIN table1 larger_b
ON larger_b.key_item = b.key_item
AND larger_b.item3 > b.item_3
LEFT OUTER JOIN table2 c
ON c.key_item = a.key_item
LEFT OUTER JOIN table2 larger_c
ON larger_c.key_item = c.key_item
AND larger_c.item4 > c.item4
WHERE larger_b.key_item IS NULL
AND larger_c.key_item IS NULL
(I have modified the table and column names only slightly, so that they conform to correct MySQL syntax. )
I work with queries that use the above structure all the time, and they perform very efficiently with indexes like the one I provided.
That said, usually I am using INNER JOINs on the b and c tables, but I don't see why your query should have any issues.
If you do experience performance problems still, report the data types of the key_item columns for each table, as if you try to join on different data types, you will generally get poor performance.
Related
I am hoping someone can point me in the correct direction, I am just an amateur playing with a hobby.
I have 2 tables, first table 'tbl_wheel_diameter' has just 'wheel_diameter' and 'id'
Second table 'tbl_tyres' has 'wheel_diameter_front','wheel_diameter_rear' and 'vehicleId'
I need to write a query that selects the front and rear diameter together.
This is something I tried.
SELECT tbl_wheel_diameter.wheel_diameter_front,tbl_wheel_diameter.wheel_diameter_rear
FROM tbl_tyres
INNER JOIN tbl_wheel_diameter
ON tbl_wheel_diameter.id = tbl_tyres.wheel_diameter
WHERE tbl_tyres.vehicleId = 2
I have read that I should use left joins but I seem to get confused with how to write them?
Add an alias for the table so you can include the same table twice. Here is an example using an alias of t1 and t2.
SELECT t1.wheel_diameter,t2.wheel_diameter
FROM tbl_tyres
INNER JOIN tbl_wheel_diameter t1
ON t1.id = tbl_tyres.wheel_diameter_front
INNER JOIN tbl_wheel_diameter t2
ON t2.id = tbl_tyres.wheel_diameter_rear
WHERE tbl_tyres.vehicleId = 1
Here's my problem all :
I have 2 big table call it A n B.
If I join that's 2 table with a very simple query like this example :
SELECT COUNT(*) FROM lib_judul, lib_buku
Then mysql process is not over yet, I don't know why. Table A have 158,670 records (33,6 MB) and Table B have 130,028 records (34,6 MB). I think myquery is right, cause I've try before to join table A with table C (the very smaller table one) and it's run well.
What should I do to do this?
You have implicit CROSS JOIN in your code which creates full Cartesian Product of the two tables. It creates a new table with 158,670 times 130,028 rows. This is more than 20 billion (20,631,542,760) records.
It's because there is no common field for both of the tables. Try using Explicit Join just like below:
SELECT
COUNT(*)
FROM lib_judul A
JOIN lib_buku B ON A.id=B.id
The cost of your query maybe is too large. Your query have cost = 158,670 x 130,028 = 20,631,542,760 I/O.
The query execution plan will execute join first, then select the column.
Know your need. May be you can add some "where condition" before you join it. Example:
this query: SELECT
COUNT(*)
FROM lib_judul A, lib_buku B
WHERE B.id = 1 AND B.id = A.id
can be optimized like this:
SELECT * FROM
(SELECT * FROM lib_judul) A
JOIN
(SELECT * FROM lib_buku WHERE lib_buku.id = 1) B
ON B.id = A.id
So after helpful feedback from my original question, I now have this query:
SELECT sessions.id, sessions.title, sessions.abstract, sessions.presenters, sessions.proposal_id, proposals.outcomes, proposals.CategorySelection, proposals.research3, proposals.research4, proposals.research5, proposals.research6, proposals.innovation3, proposals.innovation4, proposals.innovation5,proposals.innovation6, proposals.application3, proposals.application4, proposals.application5, proposals.application6, proposals.integration3, proposals.integration4, proposals.integration5, proposals.integration6, proposals.references, proposals.organization
FROM sessions, proposals
INNER JOIN proposals ON proposals.id = sessions.proposal_id
WHERE sessions.id = '$id
LIMIT 1;)
that is getting me nowhere fast. What am I doing wrong?
Original question:
I need to pull several fields from one table and several more from a second table. The criteria is that a field called proposal_id match the id field of the second table. I am fairly new so this is what I have so far. It is not working, but not sure how to make it work.
(SELECT `title`,`abstract`,`presenters`,`proposal_id` FROM `sessions` WHERE `id`='$id')
UNION
(SELECT `outcomes`,`CategorySelection`,`research3`,`research4`,`research5`,`research6`,`innovation3`,`innovation4`,`innovation5`,
`innovation6`,`application3`,`application4`,`application5`,`application6`,`integration3`,`integration4`,`integration5`,`integration6`,`references`,`organization` FROM `proposals` WHERE `id`= `sessions`.`proposal_id`)
LIMIT 1;
You need to use JOIN not UNION
select
s.*,p.*
from `sessions` s
inner join `proposals` p on p.id = s.proposal_id
where s.id = '$id'
This is how you can join both the tables using the common key between.
You can select the specific fields instead of .* by specifying the column names as
s.col1,s.col2,p.col1,p.col2
etc
Try to use JOINS, where you can match the related fields from both the tables , this is the most convenient way to fetch records from multiple tables
UNION is used when you want to combine two queries
select a.id,b.some_field from table1 as a
INNER JOIN table2 as b ON b.prospal_id = a.id
I have a select query which selects all products from my inventory table and joins them with two other tables (tables l_products and a_products)
SELECT
i.*,
b.title,
ROUND((i.price/100*80) - l.price,2) AS margin,
l.price AS l_price,
a.price AS a_price,
ROUND((a.price/100*80) - l.price, 2) AS l_margin
FROM inventory i
LEFT JOIN products b ON i.id = b.id
LEFT JOIN a_products a ON i.id = a.id
LEFT JOIN l_products l ON i.id = l.id
WHERE
a.condition LIKE IF(i.condition = 'New', 'New%', 'Used%')
AND l.condition LIKE IF(i.condition = 'New', 'New%', 'Used%')
This select query will normally give me a table such as...
id, title, condition, margin, l_price, a_price ...
001-new ... new 10 20 10
001-used ... used 10 25 20
002....
Now I need a condition in the query which will ignore all used products that are more expensive (have a higher a_price) than their 'new' counterparts, such as in the example above you can see that 001-used has a higher a_price than 001-new.
How can I achieve this with out having to resolve to using php
FULL JOIN this query with it self on a column which has a uniquely same value for each id prefix.
You may achieve this effect by adding another field to your SELECT call which produces same unique value for 001-new and 001-used, 002-new and 002-used...
Such value generation can be done by defining your own SQL Routine to extract first 3 characters from a column.
I know I can change the way MySQL executes a query by using the FORCE INDEX (abc) keyword. But is there a way to change the execution order?
My query looks like this:
SELECT c.*
FROM table1 a
INNER JOIN table2 b ON a.id = b.table1_id
INNER JOIN table3 c ON b.itemid = c.itemid
WHERE a.itemtype = 1
AND a.busy = 1
AND b.something = 0
AND b.acolumn = 2
AND c.itemid = 123456
I have a key for every relation/constraint that I use. If I run explain on this statement I see that mysql starts querying c first.
id select_type table type
1 SIMPLE c ref
2 SIMPLE b ref
3 SIMPLE a eq_ref
However, I know that querying in the order a -> b -> c would be faster (I have proven that)
Is there a way to tell mysql to use a specific order?
Update: That's how I know that a -> b -> c is faster.
The above query takes 1.9 seconds to complete and returns 7 rows. If I change the query to
SELECT c.*
FROM table1 a
INNER JOIN table2 b ON a.id = b.table1_id
INNER JOIN table3 c ON b.itemid = c.itemid
WHERE a.itemtype = 1
AND a.busy = 1
AND b.something = 0
AND b.acolumn = 2
HAVING c.itemid = 123456
the query completes in 0.01 seconds (Without using having I get 10.000 rows).
However that is not a elegant solution because this query is a simplified example. In the real world I have joins from c to other tables. Since HAVING is a filter that is executed on the entire result it would mean that I would pull some magnitues more records from the db than nescessary.
Edit2: Just some information:
The variable part in this query is c.itemid. Everything else are fixed values that don't change.
Indexes are setup fine and mysql chooses the right ones for me
between a and b there is a 1:n relation (index PRIMARY is used)
between b and c there is a many to many relation (index IDX_ITEMID is used)
the point is that mysql should start querying table a and work it's way down to c and not the other way round. Any change to achive that.
Solution: Not exactly what I wanted but this seems to work:
SELECT c.*
FROM table1 a
INNER JOIN table2 b ON a.id = b.table1_id
INNER JOIN table3 c ON b.itemid = c.itemid
WHERE a.itemtype = 1
AND a.busy = 1
AND b.something = 0
AND b.acolumn = 2
AND c.itemid = 123456
AND f.id IN (
SELECT DISTINCT table2.id FROM table1
INNER JOIN table2 ON table1.id = table2.table1_id
WHERE table1.itemtype = 1 AND table1.busy = 1)
Perhaps you need to use STRAIGHT_JOIN.
http://dev.mysql.com/doc/refman/5.0/en/join.html
STRAIGHT_JOIN is similar to JOIN, except that the left table is always read before the right table. This can be used for those (few) cases for which the join optimizer puts the tables in the wrong order.
You can use FORCE INDEX to force the execution order, and I've done that before.
If you think about it, there's usually only one order you could query tables in for any index you pick.
In this case, if you want MySQL to start querying a first, make sure the index you force on b is one that contains b.table1_id. MySQL will only be able to use that index if it's already queried a first.
You can try rewriting in two ways
bring some of the WHERE condition into JOIN
introduce subqueries even though they are not necessary
Both things might impact the planner.
First thing to check, though, would be if your stats are up to date.