I need help to optimize my 3 queries into one.
I have 2 tables, the first has a list of image processing servers I use, so different servers can handle different simultaneous job loads at a time, so I have a field called quota as seen below.
First table name, "img_processing_servers"
| id | server_url | server_key | server_quota |
| 1 | examp.uu.co | X0X1X2XX3X | 5 |
| 2 | examp2.uu.co| X0X1X2YX3X | 3 |
The second table registers if there is a job being performed at this moment on the server
Second table, "img_servers_lock"
| id | lock_server | timestamp |
| 1 | 1 | 2020-04-30 12:08:09 |
| 2 | 1 | 2020-04-30 12:08:09 |
| 3 | 1 | 2020-04-30 12:08:09 |
| 4 | 2 | 2020-04-30 12:08:09 |
| 5 | 2 | 2020-04-30 12:08:09 |
| 6 | 2 | 2020-04-30 12:08:09 |
Basically what I want to achieve is that my image servers don't go past the max quota and crash, so the 3 queries I would like to combine are:
Select at least one server available that hasn't reached it's quota and then insert a lock record for it.
SELECT * FROM `img_processing_servers` WHERE
SELECT COUNT(timestamp) FROM `img_servers_lock` WHERE `lock_server` = id
! if the count is < than quota, go ahead and register use
INSERT INTO `img_servers_lock`(`lock_server`, `timestamp`) VALUES (id_of_available_server, now())
How would I go about creating this single query?
My goal is to keep my image servers safe from overload.
Join the two tables and put that into an INSERT query.
INSERT INTO img_servers_lock(lock_server, timestamp)
SELECT s.id, NOW()
FROM img_processing_servers s
LEFT JOIN img_servers_lock l ON l.lock_server = s.id
GROUP BY s.id
HAVING IFNULL(COUNT(l.id), 0) < s.server_quota
ORDER BY s.server_quota - IFNULL(COUNT(l.id), 0) DESC
LIMIT 1
The ORDER BY clause makes it select the server with the most available quota.
OK, so I encountered just a small addition that was giving me a bug and it was that the s.server_quota had to be added to GROUP BY for it to work in the HAVING
INSERT INTO img_servers_lock(lock_server, timestamp)
SELECT s.id, NOW()
FROM alpr_servers s
LEFT JOIN img_servers_lock l ON l.lock_server = s.id
GROUP BY s.id, s.server_quota
HAVING IFNULL(COUNT(l.id), 0) < s.server_quota
ORDER BY s.server_quota - IFNULL(COUNT(l.id), 0) DESC
LIMIT 1
Thanks again Barmar!
Related
I'm offering an experience leaderboard for a Discord bot I actively develop with stuff like profile cards showing one's rank. The SQL query I'm currently using works flawlessly, however I notice that this query takes a rather long processing time.
SELECT id,
discord_id,
discord_tag,
xp,
level
FROM (SELECT #rank := #rank + 1 AS id,
discord_id,
discord_tag,
xp,
level
FROM profile_xp,
(SELECT #rank := 0) r
ORDER BY xp DESC) t
WHERE discord_id = '12345678901';
The table isn't too big (roughly 20k unique records), but this query is taking anywhere between 300-450ms on average, which piles up relatively fast with a lot of concurrent requests.
I was wondering if this query can be optimized to increase performance. I've isolated this to this query, the rest of the MySQL server is responsive and swift.
I'd be happy about any hint and thanks in advance! :)
You're scanning 20,000 rows to assign "row numbers" then selecting exactly one row from it. You can use aggregation instead:
SELECT *, (
SELECT COUNT(*)
FROM profile_xp AS x
WHERE xp > profile_xp.xp
) + 1 AS rnk
FROM profile_xp
WHERE discord_id = '12345678901'
This will give you rank of the player. For dense rank use COUNT(DISTINCT xp). Create an index on xp column if necessary.
Not an answer; too long for a comment:
I usually write this kind of thing exactly the same way that you have done, because it's quick and easy, but actually there's a technical flaw with this method - although it only becomes apparent in certain situations.
By way of illustration, consider the following:
DROP TABLE IF EXISTS ints;
CREATE TABLE ints (i INT NOT NULL PRIMARY KEY);
INSERT INTO ints VALUES
(0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
Your query:
SELECT a.*
, #i:=#i+1 rank
FROM ints a
JOIN (SELECT #i:=0) vars
ORDER
BY RAND() DESC;
+---+------+
| i | rank |
+---+------+
| 3 | 4 |
| 2 | 3 |
| 5 | 6 |
| 1 | 2 |
| 7 | 8 |
| 9 | 10 |
| 4 | 5 |
| 6 | 7 |
| 8 | 9 |
| 0 | 1 |
+---+------+
Look, the result set isn't 'random' at all. rank always corresponds to i
Now compare that with the following:
SELECT a.*
, #i:=#i+1 rank
FROM
( SELECT * FROM ints ORDER by RAND() DESC) a
JOIN (SELECT #i:=0) vars;
+---+------+
| i | rank |
+---+------+
| 5 | 1 |
| 2 | 2 |
| 8 | 3 |
| 7 | 4 |
| 4 | 5 |
| 6 | 6 |
| 0 | 7 |
| 1 | 8 |
| 3 | 9 |
| 9 | 10 |
+---+------+
Assuming discord_id is the primary key for the table, and you're just trying to get one entry's "rank", you should be able to take a different approach.
SELECT px.discord_id, px.discord_tag, px.xp, px.level
, 1 + COUNT(leaders.xp) AS rank
, 1 + COUNT(DISTINCT leaders.xp) AS altRank
FROM profile_xp AS px
LEFT JOIN profile_xp AS leaders ON px.xp < leaders.xp
WHERE px.discord_id = '12345678901'
GROUP BY px.discord_id, px.discord_tag, px.xp, px.level
;
Note I have "rank" and "altRank". rank should give you a similar position to what you were originally looking for; your results could have fluctuated for "ties", this rank will always put tied players at their highest "tie". If 3 records tie for 2nd place, those (queried separately with this) will show 2nd place, the next xp down would should 5th place (assuming 1 in 1st, 2,3,4 in 2nd, 5 in 5th). The altRank would "close the gaps" putting 5 in the 3rd place "group".
I would also recommend an index on xp to speed this up further.
I have two tables
Accounts:
+------------+--------+
| accountsid | name |
+------------+--------+
| 1 | Bob |
| 2 | Rachel |
| 3 | Mark |
+------------+--------+
Sales Orders
+--------------+------------+------------+--------+
| salesorderid | accountsid | so_date | amount |
+--------------+------------+------------+--------+
| 1 | 1 | 2015-12-16 | 50 |
| 2 | 1 | 2016-01-13 | 20 |
| 3 | 2 | 2015-12-14 | 10 |
| 4 | 3 | 2016-02-14 | 35 |
+--------------+------------+------------+--------+
As you can see, is a 1-N relation where Accounts has many Salesorders and Salesorder has 1 Account.
I need to retrieve "old" Accounts where are not active anymore. For example, If some Account dont have Salesorder in 2016 is an inactive Account.
So, in this example the result will be ONLY Rachel.
How can i retrieve this? I think its the "opposite" of between but I cant figure how to do it...
Thanks.
PS. Despite the title I can get this without INNER JOIN.
You're looking to effect an anti-join, for which there are three possibilities in MySQL:
Using NOT IN:
SELECT a.*
FROM Accounts a
WHERE a.accountsid NOT IN (
SELECT so.accountsid
FROM `Sales Orders` so
WHERE so.so_date >= '2016-01-01'
)
Using NOT EXISTS:
SELECT a.*
FROM Accounts a
WHERE NOT EXISTS (
SELECT *
FROM `Sales Orders` so
WHERE so.accountsid = a.accountsid
AND so.so_date >= '2016-01-01'
)
Using an outer JOIN:
SELECT a.*
FROM Accounts a LEFT JOIN `Sales Orders` so
ON so.accountsid = a.accountsid
AND so.so_date >= '2016-01-01'
WHERE so.accountsid IS NULL
why do you need to use only inner join? inner join is for cases you have data matching on two tables but in this case you don't you need to be using a subquery with either "not in" or "not exists"
What you want is to get the ids that didn´t make any order, so get the ids that made some order and the rest of them are the ones that didn´t make orders.
It should be something like this SELECT * FROM Accounts WHERE accountsid NOT IN (SELECT accountsid FROM Sales Orders WHERE so_date > your_date)
I have two tables, one with ranges of numbers, second with numbers. I need to select all ranges, which have at least one number with status in (2,0). I have tried number of different joins, some of them took forever to execute, one which I ended with is fast, but it select really small number of ranges.
SELECT SQL_CALC_FOUND_ROWS md_number_ranges.*
FROM md_number_list
JOIN md_number_ranges
ON md_number_list.range_id = md_number_ranges.id
WHERE md_number_list.phone_num_status NOT IN (2, 0)
AND md_number_ranges.reseller_id=1
GROUP BY range_id
LIMIT 10
OFFSET 0
What i need is something like "select all ranges, join numbers where number.range_id = range.id and where there is at least one number with phone_number_status not in (2, 0).
Any help would be really appreciated.
Example data structure:
md_number_ranges:
id | range_start | range_end | reseller_id
1 | 000001 | 000999 | 1
2 | 100001 | 100999 | 2
md_number_list:
id | range_id | number | phone_num_status
1 | 1 | 0000001 | 1
2 | 1 | 0000002 | 2
3 | 2 | 1000012 | 0
4 | 2 | 1000015 | 2
I want to be able select range 1, because it has one number with status 1, but not range 2, because it has two numbers, but with status which i do not want to select.
It's a bit hard to tell what you want, but perhaps this will do:
SELECT *
from md_number_ranges m
join (
SELECT md_number_ranges.id
, count(*) as FOUND_ROWS
FROM md_number_list
JOIN md_number_ranges
ON md_number_list.range_id = md_number_ranges.id
WHERE md_number_list.phone_num_status NOT IN (2, 0)
AND md_number_ranges.reseller_id=1
GROUP BY range_id
) x
on x.id=m.id
LIMIT 10
OFFSET 0
Is this what you're looking for?
SELECT DISTINCT r.*
FROM md_number_ranges r
JOIN md_number_list l ON r.id = l.range_id
WHERE l.phone_num_status NOT IN (0,2)
SQL Fiddle Demo
I have a table from which I am trying to retrieve the latest position for each security:
The Table:
My query to create the table: SELECT id, security, buy_date FROM positions WHERE client_id = 4
+-------+----------+------------+
| id | security | buy_date |
+-------+----------+------------+
| 26 | PCS | 2012-02-08 |
| 27 | PCS | 2013-01-19 |
| 28 | RDN | 2012-04-17 |
| 29 | RDN | 2012-05-19 |
| 30 | RDN | 2012-08-18 |
| 31 | RDN | 2012-09-19 |
| 32 | HK | 2012-09-25 |
| 33 | HK | 2012-11-13 |
| 34 | HK | 2013-01-19 |
| 35 | SGI | 2013-01-17 |
| 36 | SGI | 2013-02-16 |
| 18084 | KERX | 2013-02-20 |
| 18249 | KERX | 0000-00-00 |
+-------+----------+------------+
I have been messing with versions of queries based on this page, but I cannot seem to get the result I'm looking for.
Here is what I've been trying:
SELECT t1.id, t1.security, t1.buy_date
FROM positions t1
WHERE buy_date = (SELECT MAX(t2.buy_date)
FROM positions t2
WHERE t1.security = t2.security)
But this just returns me:
+-------+----------+------------+
| id | security | buy_date |
+-------+----------+------------+
| 27 | PCS | 2013-01-19 |
+-------+----------+------------+
I'm trying to get the maximum/latest buy date for each security, so the results would have one row for each security with the most recent buy date. Any help is greatly appreciated.
EDIT: The position's id must be returned with the max buy date.
You can use this query. You can achieve results in 75% less time. I checked with more data set. Sub-Queries takes more time.
SELECT p1.id,
p1.security,
p1.buy_date
FROM positions p1
left join
positions p2
on p1.security = p2.security
and p1.buy_date < p2.buy_date
where
p2.id is null;
SQL-Fiddle link
You can use a subquery to get the result:
SELECT p1.id,
p1.security,
p1.buy_date
FROM positions p1
inner join
(
SELECT MAX(buy_date) MaxDate, security
FROM positions
group by security
) p2
on p1.buy_date = p2.MaxDate
and p1.security = p2.security
See SQL Fiddle with Demo
Or you can use the following in with a WHERE clause:
SELECT t1.id, t1.security, t1.buy_date
FROM positions t1
WHERE buy_date = (SELECT MAX(t2.buy_date)
FROM positions t2
WHERE t1.security = t2.security
group by t2.security)
See SQL Fiddle with Demo
This is done with a simple group by. You want to group by the securities and get the max of buy_date. The SQL:
SELECT security, max(buy_date)
from positions
group by security
Note, this is faster than bluefeet's answer but does not display the ID.
The answer by #bluefeet has two more ways to get the results you want - and the first will probably be more efficient than your query.
What I don't understand is why you say that your query doesn't work. It seems pretty fine and returns the expected result. Tested at SQL-Fiddle
SELECT t1.id, t1.security, t1.buy_date
FROM positions t1
WHERE buy_date = ( SELECT MAX(t2.buy_date)
FROM positions t2
WHERE t1.security = t2.security ) ;
If the problems appears when you add the client_id = 4 condition, then it's because you add it only in one WHERE clause while you have to add it in both:
SELECT t1.id, t1.security, t1.buy_date
FROM positions t1
WHERE client_id = 4
AND buy_date = ( SELECT MAX(t2.buy_date)
FROM positions t2
WHERE client_id = 4
AND t1.security = t2.security ) ;
select security, max(buy_date) group by security from positions;
is all you need to get max buy date for each security (when you say out loud what you want from a query and you include the phrase "for each x", you probably want a group by on x)
When you use a group by, all columns in your select must either be columns that have been grouped by or aggregates, so if, for example, you wanted to include id, you'd probably have to use a subquery similar to what you had before, since there doesn't seem to be any aggregate you can reasonably use on the ids, and another group by would give you too many rows.
I have this mysql statement but I receive LIMIT in subquery error
SELECT id
FROM articles
WHERE section=1 AND id NOT IN
(
SELECT id
FROM articles
WHERE is_top_story=1 ORDER BY timestamp DESC LIMIT 2
)
I want to select all id-s from table where section=1 and id-s not in my inner(second) statement
+--id--+section+-is_top_story-+--timestamp--+
| 54 | 1 | 1 | 130 |
| 70 | 2 | 0 | 129 |
| 98 | 3 | 1 | 128 |
| 14 | 1 | 1 | 127 |
| 58 | 4 | 0 | 126 |
| 13 | 3 | 1 | 125 |
| 64 | 1 | 1 | 124 |
| 33 | 1 | 1 | 123 |
My sql should return 64 and 33(they are with section=1 and is_top_story=1), because 54 and 14 (are in inner statment)
If any can give me some code I will be very grateful
Thanks
How about this:
SELECT a.id, a.times
FROM articles AS a
LEFT JOIN (
SELECT id
FROM articles
WHERE is_top_story =1
ORDER BY times DESC LIMIT 2) AS ax
USING (id)
WHERE section = 1 AND ax.id IS NULL;
Join is a usual workaround when you need limits in subqueries; need for excluding logic just adds these 'left join - joined.id IS NULL` parts to query. )
UPDATE: Got a bit confused by your example. The original query you've quoted was "take some articles with section = 1, then take out those that belong to the 2 most recent top_stories". But in your example the section should also be taken into account when selecting these stories-to-ignore-...
It's actually quite easy to update my query with that condition as well: just replace
WHERE is_top_story = 1
with
WHERE is_top_story = 1 AND section = 1
... but I think it might be even better solved at the client code. First, you send a simple query, without any joins whatsoever:
SELECT id, is_top_story
FROM articles
WHERE section = 1
ORDER BY times DESC;
Then you just walk through the fetched rowset, and just ignore two first elements with 'is_top_story' flag on, like that:
...
$topStoriesToIgnore = 2;
foreach ($rowset as $row) {
if ($row->is_top_story && $topStoriesToIgnore-- > 0) {
continue;
}
// actual processing code goes here
}
I don't know if this is that you want, the question is a little confuse. I don't understand why you use a subquery for the same table. Anyway LIMIT is the same of "TOP" for MSSQL, so LIMIT 2 should be only returns two records.
If this is not that you want please comment and I will edit my answer:
SELECT id
FROM articles
WHERE section=1 AND is_top_story != 1
ORDER BY timestamp DESC