I've mysql tables that looks like :
user_messages
id | user_id | phone_number | message | direction | created_at
users
id | name
I want to 'group by' user_messages two times and UNION the result. Why I want to do this? because user_id sometimes has a valid user id (anything but '-1') then I group by it, if it has -1, then group by phone_number.
I also want to left join the result with users table to get the user name in case user_id is set to a valid user
I'm almost done with the query, but the problem is:
- I want the result to have the record that results from group by to be the latest one, which means, the biggest created_at value
select * from (
(
select *, count(*) as `total` from
(select `user_id`, `message`, `created_at`, `phone_number`,`direction` from `users_messages` where `user_id` != -1 order by `created_at` desc)
as `t1` group by `user_id`
)
union
(
select *, count(*) as `total` from
(select `user_id`, `message`, `created_at`, `phone_number`,`direction` from `users_messages` where `user_id` = -1 order by `created_at` desc)
as `t2` group by `phone_number`
)
) as `t3`
left join (select `id`,`name` from `users`) as `t4` on `t3`.`user_id` = `t4`.`id` order by `created_at` desc
What this gets me is the results not sorted by created_at DESC
Update:
The query actually works in my local machine but not on the production server. In my local machine I have 5.5.42 - Source distribution and in server Ver 14.14 Distrib 5.7.17, for Linux (x86_64) using EditLine wrapper ... What could be wrong?
In local machine it correctly returns me the max created_at but in server it returns the FIRST created for the grouped by record
Something like this should work:
SELECT s.`user_id`, um.`phone_number`, s.msgCount
, um.`message`, um.`created_at`, um.`direction`
, u.`name` AS userName
FROM (
SELECT `user_id`, IF(`user_id` = -1, `phone_number`, '') AS altID, MAX(`created_at`) AS lastCreatedAt, COUNT(*) AS msgCount
FROM `users_messages`
GROUP BY user_id, altID
) AS s
INNER JOIN `users_messages` AS um
ON s.user_id = um.user_id
AND s.altID = IF(um.`user_id` = -1, um.`phone_number`, '')
AND s.lastCreatedAt = um.created_at
LEFT JOIN `users` AS u
ON s.user_id = u.user_id
ORDER BY um.created_at DESC
;
The s subquery gets the summary information for each user and userless phone number; the summary information calculated includes the most recent created_at value for use in the following....
The join to um gets the row data for their last messages (by including the lastCreatedAt value from s in the join criteria)
The final join to users is used to get the user.name for the known users (and assumes there will be no -1 user, or that such a user would have an appropriate 'unknown' name.)
Since you're grouping by user_id and phone_number, you can't keep message or direction. Add a max function for created_at in each subquery. I think this would work.
select * from (
(
select user_id
,'' as phone_number
,max('created_at') as 'created_at'
,count(*) as `total` from
(select `user_id`
,`created_at`
from `users_messages`
where `user_id` != -1)
as `t1` group by `user_id`
)
union
(
select '' as user_id
,phone_number
,max('created at') as 'created_at'
,count(*) as `total` from
(select `created_at`
,`phone_number'
from `users_messages`
where `user_id` = -1)
as `t2` group by `phone_number`
)
) as `t3`
left join (select `id`,`name` from `users`) as `t4`
on `t3`.`user_id` = `t4`.`id`
order by `created_at` desc
I have two tables:
users and users_img.
I need to create one query to select all users from users table that have zip column not empty AND have img column empty in users_images (both tables have user ids: users.id and users_img.user_id so those could be joined).
users
id
name
zip
last_time
users_ids
id
user_id (same as id in users)
img
I tried this:
SELECT * FROM `users` JOIN `users_ids` on users.id = users_ids.user_id
WHERE `zip` != '' AND `img` = '' ORDER BY `last_time` DESC
with no luck. I know is supposed to be quite simple.
it will also work for you
SELECT * FROM `users` as `u`
JOIN `users_ids` as `uid` on `u`.`id` = `uid`.`user_id`
WHERE `u`.`zip` IS NOT NULL
AND `uid`.`img` IS NULL
ORDER BY `u`.`last_time` DESC
I have modified your query to check for Null values also in OR
Try this:
SELECT * FROM `users`
LEFT JOIN `users_ids` on users.id = users_ids.user_id
WHERE (TRIM(`zip`) != '' OR `zip` is not null) AND
(TRIM(`img`) = '' OR `img` is null) ORDER BY `last_time` DESC
This worked:
SELECT * FROM `users` as `u`
LEFT JOIN `users_ids` as `uid` on `u`.`id` = `uid`.`user_id`
WHERE `u`.`zip` != ''
AND `uid`.`img` IS NOT NULL
ORDER BY `u`.`last_time` DESC
I need help optimising this MySQL statement that I whipped up. It does exactly what I want, however I have a great feeling that it'll be quite slow, since I do multiple selects within the statement, and I also query achievements_new multiple times. This is the first time I do some major statement like this, I'm used to the simple SELECT FROM WHERE style crap.
I might do some explaining, this is for a leaderboard style thing for my website.
--First variable output is a rank that is calculated according to the formula shown, (Log + Log + # of achievements).
--Wepvalue is the sum of the values of the weapons which that id has. playerweapons contains all the weapons, and weaponprices convert the type to the price, and then the SUM calculates the value.
--Achcount is simply the amount of achievements that's unlocked. Maybe this can be optimised somehow with the rank output?
--id in achievements_new and playerweapons are Foreign Keys to the id in playerdata
SELECT
(
IFNULL(LOG(1.5, cashearned),0) +
IFNULL(LOG(1.3, roundswon), 0) +
(
SELECT COUNT(*)
FROM achievements_new
WHERE `value` = -1 AND achievements_new.id = playerdata.id
)
) as rank,
nationality,
nick,
steamid64,
cash,
playtime,
damage,
destroyed,
(
SELECT SUM(price)
FROM weaponprices
WHERE weapon IN
(
SELECT class
FROM playerweapons
WHERE playerweapons.id = playerdata.id
)
) as wepvalue,
(
SELECT COUNT(*)
FROM achievements_new
WHERE `value` = -1 AND achievements_new.id = playerdata.id
) as achcount,
lastplayed
FROM playerdata
ORDER BY rank DESC
Table structures:
playerdata:
CREATE TABLE IF NOT EXISTS `playerdata` (
`id` int(11) unsigned NOT NULL,
`steamid64` char(17) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
`nick` varchar(32) NOT NULL DEFAULT '',
`cash` int(32) unsigned NOT NULL DEFAULT '0',
`playtime` int(32) unsigned NOT NULL DEFAULT '0',
`nationality` char(2) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
`damage` int(32) unsigned NOT NULL DEFAULT '0',
`destroyed` int(32) unsigned NOT NULL DEFAULT '0',
`cashearned` int(10) unsigned NOT NULL,
`roundswon` smallint(5) unsigned NOT NULL,
`lastplayed` datetime NOT NULL,
) ENGINE=InnoDB
achievements_new:
CREATE TABLE IF NOT EXISTS `achievements_new` (
`id` int(10) unsigned NOT NULL,
`achkey` enum(<snip - lots of values here>) NOT NULL,
`value` mediumint(8) NOT NULL DEFAULT '0'
) ENGINE=InnoDB
playerweapons:
CREATE TABLE IF NOT EXISTS `playerweapons` (
`id` int(10) unsigned NOT NULL,
`class` varchar(30) CHARACTER SET ascii NOT NULL
) ENGINE=InnoDB
weaponprices:
CREATE TABLE IF NOT EXISTS `weaponprices` (
`weapon` varchar(30) NOT NULL,
`price` int(10) unsigned NOT NULL
) ENGINE=InnoDB
Thanks in advance!
Try something like the query below.
I used LEFT JOIN instead of joins because there may be players without achievements or weapons. If you do not need these players you can use JOIN
SELECT
IFNULL(LOG(1.5, p.cashearned),0) +
IFNULL(LOG(1.3, p.roundswon), 0) +
SUM(CASE WHEN ac.id IS NOT NULL THEN 1 ELSE 0 END)/COUNT(pw.id) as rank
p.nationality,
p.nick,
p.steamid64,
p.cash,
p.playtime,
p.damage,
p.destroyed,
--SUM(CASE WHEN pw.id IS NOT NULL THEN pw.price ELSE 0 END) as wepvalue,
--wpn.price as wepvalue,
SUM(CASE WHEN pw.id IS NOT NULL THEN wp.price ELSE 0 END)/COUNT(ac.id) as wepvalue,
SUM(CASE WHEN ac.id IS NOT NULL THEN 1 ELSE 0 END)/COUNT(pw.id) as achcount,
lastplayed
FROM playerdata as p
JOIN playerweapons as pw ON pw.id = p.id
JOIN weaponprices as wp ON pw.class = wp.weapon
LEFT JOIN achievements_new as ac ON ac.id = p.id AND ac.value = -1
--LEFT JOIN playerweapons as pw ON pw.id = p.id
--LEFT JOIN weaponprices as wp ON pw.class = wp.weapon
--LEFT JOIN ( SELECT
--pw.id as player,
--SUM(wp.price) as price
--FROM weaponprices as wp
--JOIN playerweapons as pw ON pw.class = wp.weapon
--GROUP BY pw.id
--) as wpn ON wpn.player = p.id
GROUP BY
p.nationality,
p.nick,
p.steamid64,
p.cash,
p.playtime,
p.damage,
p.destroyed,
p.lastplayed
Your query is fairly reasonable, although I would rewrite the subqueries to use explicit joins rather than in and factor out the achievements subquery:
SELECT (IFNULL(LOG(1.5, cashearned),0) + IFNULL(LOG(1.3, roundswon), 0) +
coalesce(an.cnt, 0)
) as rank,
nationality, nick, steamid64, cash, playtime, damage, destroyed,
(SELECT SUM(wp.price)
FROM weaponprices wp JOIN
playerweapons pw
on pw.class = wp.weapons
WHERE pw.id = pd.id
) as wepvalue,
coalesce(an.cnt, 0) as achcount,
lastplayed
FROM playerdata pd left outer join
(SELECT id, count(*) as cnt
FROM achievements_new an
WHERE an.`value` = -1
GROUP BY an.id
) an
on an.id = pd.id
ORDER BY rank DESC;
For this query, create the following indexes:
playerweapons(id, weapon);
weaponprices(class, price);
achievements_new(value, id);
This does the following things:
It eliminates two redundant subqueries on achievements_new.
It should optimize the prices subquery to only use indexes.
It replaces the in with an explicit join, which is sometimes optimized better.
It does not require an outer group by.
I would try to remove all correlated subqueries
SELECT
( COALESCE(LOG(1.5, pd.cashearned), 0)
+ COALESCE(LOG(1.3, pd.roundswon), 0)
+ COALESCE(an.cnt, 0)) AS rank
, pd.nationality
, pd.nick
, pd.steamid64
, pd.cash
, pd.playtime
, pd.damage
, pd.destroyed
, COALESCE(pw.wepvalue, 0) AS wepvalue
, COALESCE(an.cnt, 0) AS achcount
, pd.lastplayed
FROM playerdata pd
LEFT JOIN (
SELECT
id
, COUNT(*) AS cnt
FROM achievements_new
WHERE value = -1
GROUP BY
id
) an
ON pd.id = an.id
LEFT JOIN (
SELECT
playerweapons.id
, SUM(price) AS wepvalue
FROM weaponprices
INNER JOIN playerweapons
ON weaponprices.weapon = playerweapons.class
GROUP BY
playerweapons.id
) pw
ON pd.id = pw.id
ORDER BY
rank DESC;
I have a barebones search system that just got more complicated and there are many weird issues going on with it. How can I rewrite this search query to be more accurate and efficient
The tbl_notes table only has the id of the course but to add it to the search I used a inner query to check it too. There are alot of issues with this and I'm not too strong in mysql any help would be appreciated.
TBL_NOTES:
[id][user_id][note_title][note_title][topic][course][note]
TBL_COURSES:
[id][user_id][course_name]
SELECT * FROM `tbl_notes`
WHERE `active` = '0'
AND `valid_note` = '0'
AND `user_id` = '33'
AND `note` LIKE '%biology%'
OR `topic` LIKE '%biology%'
OR `note_title` LIKE '%biology%'
OR `course` IN (SELECT `id` FROM tbl_courses WHERE `course_name` LIKE '%biology%' AND user_id = '33')
ORDER BY id DESC LIMIT 0, 6
Thanks
I would write it like this:
SELECT
DISTINCT n.*
FROM
`tbl_notes` n left join `tbl_courses` c
on n.course = c.id
WHERE
n.`user_id` = '33'
AND n.`active` = '0'
AND n.`valid_note` = '0'
AND (n.`note` LIKE '%biology%'
OR n.`topic` LIKE '%biology%'
OR n.`note_title` LIKE '%biology%'
OR c.`course_name` LIKE '%biology%')
ORDER BY n.id DESC LIMIT 0, 6
Maybe ?:
SELECT * FROM `tbl_notes`
WHERE `active` = '0'
AND `valid_note` = '0'
AND `user_id` = '33'
AND
(`note` LIKE '%biology%'
OR `topic` LIKE '%biology%'
OR `note_title` LIKE '%biology%'
OR `course` IN (SELECT `id` FROM tbl_courses WHERE `course_name` LIKE '%biology%' AND user_id = '33')
)
ORDER BY id DESC LIMIT 0, 6
I'm always getting a timed out on the following query:
select * FROM `products`
where
`active`=1
and `group`=6
and `id` not in (select `id` from `purcheased` where `userId`=14 and `price`>100
and `reversed`=0)
order by `price` asc limit 0,500
This takes 0.01s to execute, and in this particular case returns 0 results:
select `id` from `purcheased` where `userId`=14 and `price`>100 and `reversed`=0
This executes in .02s:
select * FROM `products`
where
`active`=1
and `group`= 6
order by `price` asc limit 0,500
The full query
select * FROM `products`
where
`active` = 1
and `group` = 6
and `id` not in (
select `id` from `purcheased`
where
`userId`=14
and `price` > 100
and `reversed`=0
)
order by `price` asc limit 0,500
executes it 60 seconds!
I think this is happening because select id from purcheased ... is being executed for each row of the products.
I'm running the queries in mysql.
How do I tell mysql to execute the select id from purcheased once and than to re-use the result?
MySQL misoptimizes IN and NOT IN with subqueries. You can rewrite the query as a correlated subquery:
select *
FROM `products`
where `active`=1 and `group`=6 and
not exists (select `id`
from `purchased`
where `userId`=14 and `price`>100 and `reversed`=0 and
purchased.id = products.id
)
order by `price` asc
limit 0,500
This will also work better if you have an index on purchased.id. Actually, if this is the form of your table, then an index on purchased for (userid, reversed, id, price) should make this go much faster.
A LEFT OUTER JOIN is probably your best bet here:
select p.*
from `products` p
left outer join (
select `id`
from `purcheased`
where `userId` = 14
and `price` > 100
and `reversed` = 0
) pu on p.id = pu.id
where p.`active` = 1
and p.`group` = 6
and pu.id is null
order by p.`price`
limit 0, 500