Sorting before GROUP BY - not working - mysql

I need to perform a sort before the GROUP BY clause, but MySQL does not want to cooperate.
SELECT `article`, `date`, `aip`
FROM `official_prices`
WHERE `article` = 2003
GROUP BY `article`
ORDER BY `date` ASC
The row that should be picked is the one with the earliest date (2013-07-15) but instead it picks the date that comes first in table order. Changing to DESC does no difference.
First image shows both rows, ungrouped. Second image is them being grouped.
This table is being joined to by a main query, so (I think) any solutions involving LIMIT 1 won't be useful to me.
Full query:
SELECT `articles`.*, `official_prices`.`aip`
FROM `articles`
LEFT JOIN `official_prices`
ON (`official_prices`.`article` = `articles`.`id`)
GROUP BY `articles`.`id`, `official_prices`.`article`
ORDER BY `official_prices`.`date` ASC, `articles`.`name`

You can't use group by and order like that. The order will only apply to the complete record set being returned and not in the group itself. This will work:
select o1.*
from official_prices o1
inner join
(
SELECT `article`, min(`date`) as mdate
from `official_prices`
WHERE `article` = 2003
GROUP BY `article`
) o2 on o1.article = o2.article and o1.date = o2.mdate

What you are trying to do is simply incorrect. The ordering before the group by does not have a (guaranteed) effect on the results.
My guess is that you want to get the most recent date and aip for that date. Here is a better approach:
SELECT `article`, max(`date`),
substring_index(group_concat(`aip` order by date desc), ',', 1) as lastAip
FROM `official_prices`
WHERE `article` = 2003
GROUP BY `article`;
The only downside is that the group_concat() will convert any value to a string. If it is some other type (and a string poses problems), then convert it back to the desired type.
Actually, an even better approach is to skip the group by entirely, because you are already filtering down to one article:
select article, `date`, aip
from official_prices
where article = 2003
order by `date` desc
limit 1;
The first approach works for multiple articles.
EDIT:
Your full query is:
SELECT `articles`.*, `official_prices`.`aip`
FROM `articles` LEFT JOIN
`official_prices`
ON `official_prices`.`article` = `articles`.`id`
GROUP BY `articles`.`id`, `official_prices`.`article`
ORDER BY `official_prices`.`date` ASC, `articles`.`name`;
You are looking for more than one article, so the second approach won't work. So, use the first:
SELECT `articles`.*,
substring_index(group_concat(`official_prices`.`aip` order by `official_prices`.`date` desc),
',', 1) as lastAIP
FROM `articles` LEFT JOIN
`official_prices`
ON `official_prices`.`article` = `articles`.`id`
GROUP BY `articles`.`id`, `official_prices`.`article`
ORDER BY `articles`.`name`;

Related

How to make faster queries on my mysql table?

I have the following table
As you can see It has 1868155 rows. I am attempting to make a realtime graph, but It is impossible since almost any query lasts 1 or 2 seconds.
For example, this query
SELECT sensor.nombre, temperatura.temperatura
FROM sensor, temperatura
WHERE sensor.id = temperatura.idsensor
ORDER BY temperatura.fecha DESC, idsensor ASC
LIMIT 4
Is supposed to show this
Ive tried everything, using indexes(perhaps not correctly), using only the fields i need instead of *, etc. but the results are the same!
These are the indexes of the table.
Explain of the query
EDITED
This is the explain of the query after implementing
ALTER TABLE temperatura
ADD INDEX `sensor_temp` (`idsensor`,`fecha`,`temperatura`)
And using inner join syntax for the query
SELECT s.nombre, t.temperatura
FROM sensor s
INNER JOIN temperatura t
ON s.id = t.idsensor
ORDER BY t.fecha DESC, t.idsensor ASC
LIMIT 4
This is my whole sensor table
Try the following:
ALTER TABLE temperatura
ADD INDEX `sensor_temp` (`idsensor`,`fecha`,`temperatura`)
I also recommend using modern join syntax:
SELECT s.nombre, t.temperatura
FROM sensor s
INNER JOIN temperatura t
ON s.id = t.idsensor
ORDER BY t.fecha DESC, t.idsensor ASC
LIMIT 4
Report the EXPLAIN again after making the above changes, if performance is still not good enough.
Attempt #2
After looking closely at what it appears you are trying to do, I believe this next query may be more effective:
SELECT
s.nombre, t.temperatura
FROM temperatura t
LEFT OUTER JOIN temperatura later_t
ON later_t.idsensor = t.idsensor
AND later_t.fecha > t.fecha
INNER JOIN sensor s
ON s.id = t.idsensor
WHERE later_t.idsensor IS NULL
ORDER BY t.idsensor ASC
You can also try:
SELECT
s.nombre, t.temperatura
FROM temperatura t
INNER JOIN (
SELECT
t.idsensor,
MAX(t.fecha) AS fecha
FROM temperatura t
GROUP BY t.idsensor
) max_fecha
ON max_fecha.idsensor = t.idsensor
AND max_fecha.fecha > t.fecha
INNER JOIN sensor s
ON s.id = t.idsensor
ORDER BY t.idsensor ASC
In my experience, if you are trying to find the most recent record, one of the two queries above will work. Which works best depends on various factors, so try them both.
Let me know how those perform, and if they still get you the data you want. Also, any query you run, run at least 3 times, and report all 3 times. That will help get an accurate measure of how fast a given query is, since various external factors can affect the speed of a query.
It is not possible to optimize a mixture of ASC and DESC, as in
ORDER BY t.fecha DESC, t.idsensor ASC
You tried a covering index:
INDEX `sensor_temp` (`idsensor`,`fecha`,`temperatura`)
However, this covering index may be better:
INDEX `sensor_temp` (`fecha`,`idsensor`,`temperatura`)
Then, if you are willing to get the sensors in a different order, use
ORDER BY t.fecha DESC, t.idsensor DESC
This will give you up to 4 sensors for the last fecha:
sensor: PRIMARY KEY(id)
tempuratura: INDEX(fecha, idsensor, tempuratura)
SELECT
( SELECT nombre FROM sensor WHERE id = t.idsensor ) AS nombre,
t.temperatura
FROM
( SELECT MAX(fecha) AS max_fecha FROM tempuratura ) AS z
JOIN temperatura AS t ON t.fecha = z.max_fecha
ORDER BY t.idsensor ASC
LIMIT 4;

MYSQL results order is mixing

SELECT
GROUP_CONCAT(CONCAT(user_firstname,' ', user_lastname)) fullname,
workour_day,
GROUP_CONCAT(distinct timetable_start) starttime,
GROUP_CONCAT(timetable_end) endtime
FROM doctors_timetable
INNER JOIN doctors ON user_id = doctor_id
INNER JOIN workours ON timetable_day = workour_id GROUP BY workour_day
ORDER BY timetable_id ASC
before Thursday everything works good and than it's mixing:
http://s24.postimg.org/w827dxclh/Capture.png
MYSQL results order is mixing. How can I fix it?
As what i understand from your comment you need the starttime and endtitme values to be in ascending order if thats the case you can use ORDER BY in GROUP_CONCAT to tell in what order to concatenate values
SELECT
GROUP_CONCAT(CONCAT(user_firstname,' ', user_lastname)) fullname,
workour_day,
GROUP_CONCAT(distinct timetable_start ORDER BY timetable_start ASC) starttime,
GROUP_CONCAT(timetable_end ORDER BY timetable_end) endtime
FROM doctors_timetable
INNER JOIN doctors ON user_id = doctor_id
INNER JOIN workours ON timetable_day = workour_id
GROUP BY workour_day
ORDER BY timetable_id ASC
Note you are using GROUP_CONCAT function which has a default limit of 1024 characters set by default,if your output exceeds this limit the result will be truncated to increase the limit you can see steps mentioned in the manual
timetable_id is still a mystery, but it's what you're ordering by. Order by starttime, endtime if you want to enforce ordering on those.

MySQL Sub-Queries Simplifying

I have two tables: Races and RacesTimes, I want to extract all from Races and from RacesTimes only Finisher and Time, only the best RacesTimes.TotalTime (ordered ASC with LIMIT 1) from each RaceID (a column from RacesTimes).
So the result would be:
Races.*, RacesTimes.Finisher, RacesTimes.Time
This is what I made:
SELECT
Races.*,
(
SELECT
`TotalTime`
FROM
`RacesTimes`
WHERE
`RaceID` = Races.ID
ORDER BY
`TotalTime` ASC
LIMIT 1
) AS `BestTime`,
(
SELECT
`Time`
FROM
`RacesTimes`
WHERE
`RaceID` = Races.ID
ORDER BY
`TotalTime` ASC
LIMIT 1
) AS `BestTimeS`,
(
SELECT
`Finisher`
FROM
`RacesTimes`
WHERE
`RaceID` = Races.ID
ORDER BY
`TotalTime` ASC
LIMIT 1
) AS `BestFinisher`
FROM `Races`
It is extracting corectly all, but the query is way too long, can't it be simplified ? I think the simplified version uses LEFT JOIN or other thing like that, I don't know how to use queries with JOIN.
The approach here is to aggregate RaceTimes by race. The trick is to get the finisher with the minimum time.
MySQL offers a solution for this, by using group_concat() and substring_index() in a clever way. group_concat() takes an order by argument, so it can order the results by the time. Then the best finisher is in the first position.
The SQL looks like this:
select r.*, rtr.mintt as TotalTime, rtr.Finisher
from Races r join
(select RaceId, MIN(TotalTime) as mintt,
substring_inde(group_concat(finisher separator ',' order by totaltime), 1) as Finisher
from RaceTimes rt
group by RaceId
) rtr
on rtr.RaceId = r.id

MySQL sort grouped data

I have tried to program a inbox that display messages in the order they were received and then by if they have been read or not, it seemed to work for a while, but not it doesn't. It may have only worked under certain circumstances maybe..
Anyway here is my query;
SELECT `id`, `from_userid`, `read`, max(sent) AS sent
FROM (`who_messages`)
WHERE `to_userid` = '41'
GROUP BY `from_userid`
ORDER BY `read` ASC, `sent` DESC
I believe the problem is that the messages are being grouped in the wrong order.. as the inbox is always showing as read, when new messages exist. I get the right time of the new messages, but I am guessing this because I selected max(sent).
Is my logic wrong? or can I sort and then group as all my efforts have resulted in 'Every derived table must have its own alias'
Setup an SQL Fiddle - here's the best I came up with. Basically I do the ordering first in a sub-query then group them afterwards. That seemed to work with the (limited) test data I entered.
SELECT *
FROM (SELECT id, from_userid, is_read, sent
FROM who_messages
WHERE to_userid = 41
ORDER BY from_userid ASC, is_read ASC) m
GROUP BY m.from_userid
ORDER BY m.is_read ASC, m.sent DESC
See the fiddle to play around: http://sqlfiddle.com/#!2/4f63d/8
You are selecting non-grouping fields in a grouped query. It is not guaranteed which record of the group will be returned, and ORDER BY is processed after GROUP BY.
Try this:
SELECT m.*
FROM (
SELECT DISTINCT from_userid
FROM who_messages
WHERE to_userid = 41
) md
JOIN who_messages m
ON m.id =
(
SELECT mi.id
FROM who_message mi
WHERE (mi.to_userid, mi.from_userid) = (41, md.from_userid)
ORDER BY
mi.sent DESC, mi.id DESC
LIMIT 1
)
Create an index on who_message (to_userid, from_userid, sent, id) for this to work fast.
Update
The above query will return the record for the last message from any given user (including its read status). If you want to check that you have any unread messages from the user, use this:
SELECT m.*, md.all_read
FROM (
SELECT from_userid, MIN(read) AS all_read
FROM who_messages
WHERE to_userid = 41
GROUP BY
from_userid
) md
JOIN who_messages m
ON m.id =
(
SELECT mi.id
FROM who_message mi
WHERE (mi.to_userid, mi.from_userid) = (41, md.from_userid)
ORDER BY
mi.sent DESC, mi.id DESC
LIMIT 1
)
For this to work fast, create an index on who_message (to_userid, from_userid, read) (in addition to the previous index).
As Quassnoi said, you are using a GROUP BY query and ordering on 'read' which is not an aggregate function. Therefore you can't be certain of the value used by the MySQL engine (usually the last of the group but...)
I would suggest writing your query this way, as it doesn't involve any subquery and has some many other performance-friendly usage:
SELECT
from_userid,
COUNT(*) AS nb_messages,
SUM(NOT is_read) AS nb_unread_messages,
MAX(sent) AS last_sent
FROM who_messages
WHERE to_userid = 41
GROUP BY from_userid
ORDER BY nb_unread_messages DESC, last_sent DESC;
(I used Andy Jones' fiddle schema: http://sqlfiddle.com/#!2/4f63d/8.
By the way, many thanks Andy, this site is great !)
Hope this help !
"inbox that display messages in the order they were received and then by if they have been read or not ... however it is suppose to be the latest message" - assumes read is a nullable date/time column, and messages are stored in the order they are sent (newer have larger id than older - autoid)
SELECT wm.id, wm.from_userid, (wm.read IS NULL) as unread, wm.sent
FROM (SELECT MAX(id) AS id FROM who_messages WHERE to_userid = '41' GROUP BY from_userid) sub
INNER JOIN who_messages wm ON sub.id = wm.id
ORDER BY wm.sent DESC, wm.read

group by month and year, count from another table

im trying to get my query to group rows by month and year from the assignments table, and count the number of rows that has a certain value from the leads table. they are linked together as the assignments table has an id_lead field, which is the id of the row in the leads table.
d_new would be a count of the assignments for leads for the month whose website is newsite.com
d_subprime would be a count of the assignments for leads for the month whose website is not newsite.com
here are the tables being used:
`leads`
id (int)
website (varchar)
`assignments`
id_lead (int)
date_assigned (int)
heres my query which is not working:
SELECT
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
(select COUNT(*) from leads where website='newsite.com' ) as d_new,
(select COUNT(*) from leads where website!='newsite.com') as d_subprime
FROM assignments as a
left join leads as l on (l.id = a.id_lead)
where id_dealership='$id_dealership2'
GROUP BY
d_month,
d_year
ORDER BY
d_year asc,
MONTH(FROM_UNIXTIME(a.date_assigned)) asc
$id_dealership is a variable containing a id of the dealership im trying to view the count for.
any help would be greatly appreciated.
You can sort of truncate your timestamps to months and use the obtained values for grouping, then derive the necessary date parts from them:
SELECT
YEAR(d_yearmonth) AS d_year,
MONTHNAME(d_yearmonth) AS d_month,
…
FROM (
SELECT
LAST_DAY(FROM_UNIXTIME(a.date_assigned)) as d_yearmonth,
…
FROM assignments AS a
LEFT JOIN leads AS l ON (l.id = a.id_lead)
WHERE id_dealership = '$id_dealership2'
GROUP BY
d_yearmonth
) AS s
ORDER BY
d_year ASC,
MONTH(d_yearmonth) ASC
Well, LAST_DAY() doesn't really truncate a timestamp, but it does turn all the values belonging to the same month into the same value, which is basically what we need.
And I guess the counts should be related to the rows you are actually selecting, which is not what your subqueries are. Something like this might do:
…
COUNT(d.website = 'newsite.com' OR NULL) AS d_new,
/* or: COUNT(d.website) - COUNT(NULLIF(d.website, 'newsite.com')) AS d_new */
COUNT(NULLIF(d.website, 'newsite.com')) AS d_subprime
…
Here's the entire query with all the modifications mentioned:
SELECT
YEAR(d_yearmonth) AS d_year,
MONTHNAME(d_yearmonth) AS d_month,
d_new,
d_subprime
FROM (
SELECT
LAST_DAY(FROM_UNIXTIME(a.date_assigned)) as d_yearmonth,
COUNT(d.website = 'newsite.com' OR NULL) AS d_new,
COUNT(NULLIF(d.website, 'newsite.com')) AS d_subprime
FROM assignments AS a
LEFT JOIN leads AS l ON (l.id = a.id_lead)
WHERE id_dealership = '$id_dealership2'
GROUP BY
d_yearmonth
) AS s
ORDER BY
d_year ASC,
MONTH(d_yearmonth) ASC
This should do the trick:
SELECT
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
l.website,
COUNT(*)
FROM
assignments AS a
INNER JOIN leads AS l on (l.id = a.id_lead) /*are you sure, that you need a LEFT JOIN?*/
WHERE id_dealership='$id_dealership2'
GROUP BY
d_year, d_month, website
/*an ORDER BY is not necessary, MySQL does that automatically when grouping*/
If you really need a LEFT JOIN, be aware that COUNT() ignores NULL values. If you want to count those as well (which I can't imagine to make sense) write it like this:
SELECT
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
l.website,
COUNT(COALESCE(l.id, 1))
FROM
assignments AS a
LEFT JOIN leads AS l on (l.id = a.id_lead)
WHERE id_dealership='$id_dealership2'
GROUP BY
d_year, d_month, website
Start with
SELECT
MONTHNAME(FROM_UNIXTIME(a.date_assigned)) as d_month,
YEAR(FROM_UNIXTIME(a.date_assigned)) as d_year,
SUM(IF(l.website='newsite.com',1,0) AS d_new,
SUM(IF(l.website IS NOT NULL AND l.website!='newsite.com',1,0) AS d_subprime
FROM assignments AS a
LEFT JOIN leads AS l ON l.id = a.id_lead
WHERE id_dealership='$id_dealership2'
GROUP BY
d_month,
d_year
ORDER BY
d_year asc,
MONTH(FROM_UNIXTIME(a.date_assigned)) asc
and work from here: The field id_dealership is neither in leads nor in assignments, so you need more work.
If you edit your question to account for id_dealership we might be able to help you further.