I have to get the names of the Departments and the number of Employees in it. Test is my schema.
So I come up with two queries that give me the same result -
First
SELECT Department.Departmentname,
(
SELECT COUNT(*)
FROM test.Employee
WHERE Employee.Departmentid = Department.idDepartment
) AS NumberOfEmployees
FROM test.Department;
Second
SELECT Department.Departmentname AS NAme,COUNT(Employee.idEmployee) AS Employee_COUNT
FROM test.Department
LEFT JOIN test.Employee
ON Employee.Departmentid = Department.idDepartment
GROUP BY Employee.Departmentid ;
Which of the two is the best and efficient way to get the required result? Any other solution is welcome.
Please explain why a particular solution is better
My preference for expressing the logic is the second query, which I would write as:
SELECT d.Departmentname AS Name, COUNT(e.idEmployee) AS Employee_COUNT
FROM test.Department d LEFT JOIN
test.Employee e
ON e.Departmentid = d.idDepartment
GROUP BY d.Departmentname;
Note the use of table aliases and the fact that the GROUP BY uses the same columns as the SELECT. However, in MySQL, this query will not use an index on DepartmentName for the group by. That means that the GROUP BY is doing a file sort, a relatively expensive operation.
When you write the query like this:
SELECT d.Departmentname,
(SELECT COUNT(*)
FROM test.Employee e
WHERE e.Departmentid = d.idDepartment
) AS NumberOfEmployees
FROM test.Department d;
No explicit group by is needed. With an index on Employee(DepartmentId) this will use the index for the count(*), so this version would normally perform better in MySQL.
The difference in performance is probably negligible until you start having thousands or ten of thousands of rows.
Related
I'm giving one example
-- sub-query
SELECT p.first_name, p.last_name,
d.department_count, s.total_sales
FROM persons as p
INNER JOIN
(
SELECT department_id,
COUNT(people) as department_count
FROM department as d
WHERE department_type = 'sales'
GROUP BY department_id
) as d ON d.department_id = p.department_id
LEFT OUTER JOIN
(
SELECT person_id,
SUM(sales) as total_sales
FROM orders
WHERE orders.department_id = d.department_id
GROUP BY person_id
) as s ON s.person_id = p.person_id
-- cte
WITH deps as
(
SELECT department_id,
COUNT(people) as department_count
FROM department as d
WHERE department_type = 'sales'
GROUP BY department_id
), sales as
(
SELECT person_id,
SUM(sales) as total_sales
FROM orders
WHERE orders.department_id = d.department_id
GROUP BY person_id
)
SELECT p.first_name, p.last_name,
d.department_count, s.total_sales
FROM persons as p
INNER JOIN deps as d
ON d.department_id = p.department_id
LEFT OUTER JOIN sales as s
ON s.person_id = p.person_id
but I'm also wanting the answer in overall case. In some cases it may depend on the dataset and objective? But usually, which one is better for optimization/performance when running the query? Moreover, if there's few less lines in any of these procedure compared to the other, will that make the execution faster?
Both examples you show will be executed by MySQL using temporary tables. That is, the result of both the subquery or the CTE will be stored in a temporary table that lives for the duration of the query, then automatically dropped when the query ends.
Temporary tables are used for other types of queries in MySQL. You can read more about them here: https://dev.mysql.com/doc/refman/8.0/en/internal-temporary-tables.html
Temporary tables are often associated with performance overhead. It takes time for the temporary table to be created and filled with rows from the result of the subquery or CTE. This is unavoidable.
If you can run a different query to get the result you want without creating a temporary table, that's almost always better for performance. But in the examples you show, I don't think it's possible to do in a single query.
Almost every general rule about performance has exceptions, so you really need to be careful to evaluate performance on a case by case basis. Performance optimization is a complex subject.
These indexes may help:
orders: INDEX(department_id, person_id)
p: INDEX(department_id, first_name, last_name, person_id)
s: INDEX(person_id, total_sales)
d: INDEX(department_type, department_id)
Typically COUNT(*) is better than COUNT(col)
I have this query I need to optimize further since it requires too much cpu time and I can't seem to find any other way to write it more efficiently. Is there another way to write this without altering the tables?
SELECT category, b.fruit_name, u.name
, r.count_vote, r.text_c
FROM Fruits b, Customers u
, Categories c
, (SELECT * FROM
(SELECT *
FROM Reviews
ORDER BY fruit_id, count_vote DESC, r_id
) a
GROUP BY fruit_id
) r
WHERE b.fruit_id = r.fruit_id
AND u.customer_id = r.customer_id
AND category = "Fruits";
This is your query re-written with explicit joins:
SELECT
category, b.fruit_name, u.name, r.count_vote, r.text_c
FROM Fruits b
JOIN
(
SELECT * FROM
(
SELECT *
FROM Reviews
ORDER BY fruit_id, count_vote DESC, r_id
) a
GROUP BY fruit_id
) r on r.fruit_id = b.fruit_id
JOIN Customers u ON u.customer_id = r.customer_id
CROSS JOIN Categories c
WHERE c.category = 'Fruits';
(I am guessing here that the category column belongs to the categories table.)
There are some parts that look suspicious:
Why do you cross join the Categories table, when you don't even display a column of the table?
What is ORDER BY fruit_id, count_vote DESC, r_id supposed to do? Sub query results are considered unordered sets, so an ORDER BY is superfluous and can be ignored by the DBMS. What do you want to achieve here?
SELECT * FROM [ revues ] GROUP BY fruit_id is invalid. If you group by fruit_id, what count_vote and what r.text_c do you expect to get for the ID? You don't tell the DBMS (which would be something like MAX(count_vote) and MIN(r.text_c)for instance. MySQL should through an error, but silently replacescount_vote, r.text_cbyANY_VALUE(count_vote), ANY_VALUE(r.text_c)` instead. This means you get arbitrarily picked values for a fruit.
The answer hence to your question is: Don't try to speed it up, but fix it instead. (Maybe you want to place a new request showing the query and explaining what it is supposed to do, so people can help you with that.)
Your Categories table seems not joined/related to the others this produce a catesia product between all the rows
If you want distinct resut don't use group by but distint so you can avoid an unnecessary subquery
and you dont' need an order by on a subquery
SELECT category
, b.fruit_name
, u.name
, r.count_vote
, r.text_c
FROM Fruits b
INNER JOIN Customers u ON u.customer_id = r.customer_id
INNER JOIN Categories c ON ?????? /Your Categories table seems not joined/related to the others /
INNER JOIN (
SELECT distinct fruit_id, count_vote, text_c, customer_id
FROM Reviews
) r ON b.fruit_id = r.fruit_id
WHERE category = "Fruits";
for better reading you should use explicit join syntax and avoid old join syntax based on comma separated tables name and where condition
The next time you want help optimizing a query, please include the table/index structure, an indication of the cardinality of the indexes and the EXPLAIN plan for the query.
There appears to be absolutely no reason for a single sub-query here, let alone 2. Using sub-queries mostly prevents the DBMS optimizer from doing its job. So your biggest win will come from eliminating these sub-queries.
The CROSS JOIN creates a deliberate cartesian join - its also unclear if any attributes from this table are actually required for the result, if it is there to produce multiples of the same row in the output, or just an error.
The attribute category in the last line of your query is not attributed to any of the tables (but I suspect it comes from the categories table).
Further, your code uses a GROUP BY clause with no aggregation function. This will produce non-deterministic results and is a bug. Assuming that you are not exploiting a side-effect of that, the query can be re-written as:
SELECT
category, b.fruit_name, u.name, r.count_vote, r.text_c
FROM Fruits b
JOIN Reviews r
ON r.fruit_id = b.fruit_id
JOIN Customers u ON u.customer_id = r.customer_id
ORDER BY r.fruit_id, count_vote DESC, r_id;
Since there are no predicates other than joins in your query, there is no scope for further optimization beyond ensuring there are indexes on the join predicates.
As all too frequently, the biggest benefit may come from simply asking the question of why you need to retrieve every single row in the tables in a single query.
I'm running two queries.
The first one gets unique IDs. This executes in ~350ms.
select parent_id
from duns_match_sealed_air_072815
group by duns_number
Then I paste those IDs into this second query. With >10k ids pasted in, it also executes in about ~350ms.
select term, count(*) as count
from companies, business_types, business_types_to_companies
where
business_types.id = business_types_to_companies.term_id
and companies.id = business_types_to_companies.company_id
and raw_score > 25
and diversity = 1
and company_id in (paste,ten,thousand,ids,here)
group by term
order by count desc;
When I combine these queries into one it takes a long time to execute. I don't know how long because I stopped it after minutes.
select term, count(*) as count
from companies, business_types, business_types_to_companies
where
business_types.id = business_types_to_companies.term_id
and companies.id = business_types_to_companies.company_id
and raw_score > 25
and diversity = 1
and company_id in (
select parent_id
from duns_match_sealed_air_072815
group by duns_number
)
group by term
order by count desc;
What is going on?
It's down to the way it processes the query - I believe it has to run your embedded query once for each row, whereas using two queries allows you to store the result.
Hope this helps!
The query has been re-written using JOIN, but particularly I've used EXISTS instead of IN. This is a short in the dark. It is possible that there may be many values generated in the sub-query causing the outer query to struggle while it goes through matching each item returned from the sub-query.
select term, count(*) as count
from companies c
inner join business_types_to_companies bc on bc.company_id = c.id
inner join business_types b on b.id = bc.term_id
where
raw_score > 25
and diversity = 1
and exists (
select 1
from duns_match_sealed_air_072815
where parent_id = c.id
)
group by term
order by count desc;
First, with respect, your subquery doesn't use GROUP BY in a sensible way.
select parent_id /* wrong GROUP BY */
from duns_match_sealed_air_072815
group by duns_number
In fact, it misuses the pernicious MySQL extension to GROUP BY. Read this. http://dev.mysql.com/doc/refman/5.6/en/group-by-handling.html . I can't tell what your application logic intends from this query, but I can tell you that it actually returns an unpredictably selected parent_id value associated with each distinct duns_number value.
Do you want
select MIN(parent_id) parent_id
from duns_match_sealed_air_072815
group by duns_number
or something like that? That one selects the lowest parent ID associated with each given number.
Sometimes MySQL has a hard time optimizing the WHERE .... IN () query pattern. Try a join instead. Like this:
select term, count(*) as count
from companies
join (
select MIN(parent_id) parent_id
from duns_match_sealed_air_072815
group by duns_number
) idlist ON companies.id = idlist.parent_id
join business_types_to_companies ON companies.id = business_types_to_companies.company_id
join business_types ON business_types.id = business_types_to_companies.term_id
where raw_score > 25
and diversity = 1
group by term
order by count desc
To optimize this further we'll need to see the table definitions and the output from EXPLAIN.
This is my code :
SELECT *
FROM Event_list
WHERE interest in
(
SELECT Interest_name
from Interest
where Interest_id in
(
SELECT Interest_id
FROM `User's Interests`
where P_id=Pid and is_canceled=0
)
)
order by count(Eid) desc
I don't use any GROUP BY clause but still only get one row. when removing the ORDER BY clause I get all the correct rows (but not in the right order).
I'm trying to return a view (named Event_list) sorted by most common Eid (Event id), but I want to see every row without any grouping.
COUNT() is a group function, so using it will automatically result in grouping of rows. This is why you get only one row in your result when you use it in your ORDER BY clause.
Unfortunately, it's not clear what you're trying to do, so I can't tell you how to rewrite your query to get your desired results.
I suspect the query you want is more like this:
SELECT el.*,
(select count(*)
from interest i join
UserInterests ui
on ui.is_canceled = 0 and ui.p_id = i.id
where el.interest = i.interest_name
) as cnt
FROM Event_list el
ORDER BY cnt desc;
It is a bit hard to tell without sample data and a better formed query. Some notes:
Don't use special characters in table and column names. Having to escape the names merely leads to queries that are harder to read, write, and understand.
Qualify column names, so you know what tables columns come from.
Use table aliases -- so queries are easier to write and to read.
The WHERE clause only does filtering. Your description of the problem doesn't seem to involve filtering, only ordering.
Any time you use an aggregation function, the query automatically becomes an aggregation query. Without a group by, exactly one row is returned.
Give foreign keys the same names as primary keys, where possible.
You may try:
SELECT L.* , C.Cnt
FROM Event_list L
LEFT JOIN (
SELECT E.EID, COUNT(*) AS Cnt
FROM Event_List E
JOIN Interest I
ON E.Interest = I.Interest_name
JOIN `User's Interests` U
ON U.Interest_id = I.Insert_Id
Where U.P_id=Pid and U.is_canceled=0
GROUP BY E.EID
) C
ON E.Eid = C.Eid
Order By Cnt DESC
I don't have the tables to test so you may want to correct column names and other conditions. Just provide you the idea.
So let's say I have the following tables Person and Wage. It's a 1-N relation, where a person can have more then one wage.
**Person**
id
name
**Wage**
id
person_id
amount
effective_date
Now, I want to query a list of all persons and their latest wages. I can get the results by doing the following query:
SELECT
p.*,
( SELECT w.amount
FROM wages a w
WHERE w.person_id = p.id
ORDER BY w.effective_date
LIMIT 1
) as wage_amount,
( SELECT w.effective_date
FROM wages a w
WHERE w.person_id = p.id
ORDER BY w.effective_date
LIMIT 1
) as effective_date
FROM person as p
The problem is, my query will have multiple sub-queries from different tables. I want to make it as efficient as possible. Is there an alternative to using sub-queries that would be faster and give me the same results?
Proper indexing would probably make your version work efficiently (that is, an index on wages(person_id, effective_date)).
The following produces the same results with a single subquery:
SELECT p.*, w.amount, w.effective_date
from person p left outer join
(select person_id, max(effective_date) as maxdate
from wages
group by personid
) maxw
on maxw.person_id = p.id left outer join
wages w
on w.person_id = p.id and w.effective_date = maxw.maxdate;
And this version might make better us of indexes than the above version:
SELECT p.*, w.amount, w.effective_date
from person p left outer join
wages w
on w.person_id = p.id
where not exists (select * from wages w2 where w2.effective_date > w.effective_date);
Note that these version will return multiple rows for a single person, when there are two "wages" with the same maximum effective date.
Subqueries can be a good solution like Sam S mentioned in his answer but it really depends on the subquery, the dbms you are using, and your indexes. See this question and answers for a good discussion on the performance of subqueries vs. joins: Join vs. sub-query
If performance is an issue for you, you must consider using the EXPLAIN command of your dbms. It will show you how the query is being built and where the bottlenecks are. Based on its results, you might consider rewriting your query some other way.
For instance, it was usually the case that a join would yield better performance, so you could rewrite your query according to this answer: https://stackoverflow.com/a/2111420/362298 and compare their performance.
Note that creating the right indexes will also make a big difference.
Hope it helps.
Subqueries are very efficient as long as you make sure you use indexes. Try running EXPLAIN on your query and see if it uses correct indexes
SELECT p.name, w.amount, MAX(w.effective_date) FROM Person p LEFT JOIN Wage
w ON w.person_id = p.id GROUP BY p.name
I didn't test this query.