I'm trying to create a SQL query that retrieves records based on the start and end dates. However, the system doesn't have a very nice Database schema, so I'm looking for a way to create a query that works with it.
Basically, there's a meta table that stores the events' dates using the "date_NUMBER_(start|end)" format:
event_id
meta_key
meta_value
1
date_0_start
2022-04-02
1
date_0_end
2022-04-03
-
-
-
2
date_0_start
2022-03-21
2
date_0_end
2022-03-22
2
date_1_start
2022-06-24
2
date_1_end
2022-06-30
So, Event 1 has one date span: 2022-04-02 to 2022-04-03. While Event 2 has two: 2022-03-21 to 2022-03-22 and 2022-06-24 to 2022-06-30.
After 2022-03-22, Event 2 is not active, starting again only on 2022-06-24.
To create a filter that works with this schema, I came up with the following SQL query. For example, to search for events happening between the 2022-04-01 - 2022-04-03 range (any event that there's an event day between the range):
SELECT events.id
FROM events INNER JOIN events_meta ON ( events.id = events_meta.event_id ) INNER JOIN events_meta AS evt1 ON (
events.id = evt1.event_id )
WHERE
(
(
events_meta.meta_key LIKE 'date_%_start' AND
CAST(events_meta.meta_value AS DATE) >= '2022-04-01'
)
AND
(
evt1.meta_key LIKE 'date_%_end' AND
CAST(evt1.meta_value AS DATE) <= '2022-04-03'
)
)
The problem is that in this way, I should get only Event 1, but Event 2 is also returned since date_1_start is >= '2022-04-01' and date_0_end is <= '2022-04-03'
I'm not being able to find a way where I can match the NUMBER in the "date_NUMBER_(start|end)" meta_key format, so the query doesn't compare different NUMBERs.
Any help is appreciated :)
I made a fiddle with the INNER JOIN query:
http://sqlfiddle.com/#!9/fb497e/2
Use a self-join to get different keys for the same event ID. See MYSQL Select from tables based on multiple rows
SELECT m1.event_id, m1.meta_value AS start, m2.meta_value AS end
FROM events_meta AS m1
JOIN events_meta AS m2
ON m1.event_id = m2.event_id
AND SUBSTRING_INDEX(m1.meta_key, '_', 2) = SUBSTRING_INDEX(m2.meta_key, '_', 2)
WHERE m1.meta_key LIKE 'date_%_start' AND m2.meta_key LIKE 'date_%_end'
AND CAST(m1.meta_value AS DATE) >= '2022-04-01'
AND CAST(m2.meta_value AS DATE) <= '2022-04-03'
The SUBSTRING_INDEX() calls will return the prefixes date_0, date_1, etc. Including this in the ON condition will pair up the corresponding start and end keys.
I have a stored procedure. A little problematic in terms of performance. I want to improve the performance of the stored procedure, but I could not figure out what to do. There are approximately 3 million records in my database. When I run this query one by one, it's good in performance. But when 150 people run this stored procedure at the same time, there are spikes in the CPU.
As an example, I created my procedure and table structures.
My Stored Procedure:
BEGIN
SELECT ss.car_route from person o
inner join car_time ss on ss.inst_id =o.inst_id
and ss.start_time<=DATE_FORMAT(CURTIME(),'%H:%i') AND ss.finish_time>= date_format(curtime() ,'%H:%i') AND ss.car_id=carid
and ss.days like concat('%',(select WEEKDAY(now())+1),'%')
where (o.car_id=carid or o.back_car_id=carid ) LIMIT 1 into #route_;
select sf.stop_service from car_comp sf
inner join cars s on s.inst_id = sf.id and s.id=carid and s.active=1 limit 1
into #stop_ser;
if #route_ = 1 and #stop_ser=0 THEN
select DISTINCT ss.start_time,ss.finish_time ,o.id,o.name,r.photo, oh.state ,oh.datee,ss.car_route,
ifnull(bh.id,0) AS called,
ifnull(mh.excuse_id,0) AS excuse_id,
ifnull(o.latitude_1,0) AS latitude_1,
ifnull(o.longitude_1,0) AS longitude_1,
ifnull(o.latitude_2,0) AS latitude_2,
ifnull(o.longitude_2,0) AS longitude_2,
case when (ifnull(o.call_notify,0)=1 or ifnull(o.mes_notify,0)=1) then 1 else 0 end AS call_notify ,
ifnull(o.rownumber,0) AS rownumber,
ifnull(o.number_1,0) AS number_1,
ifnull(o.number_2,0) AS number_2,
ifnull(o.brownumber,0) AS brownumber,
ifnull(ROUND(o.notify_meter_1/2),0) AS notify_meter_1,
ifnull(ROUND(o.notify_meter_2/2),0) AS notify_meter_2
from person o
inner join car_time ss on ss.inst_id =o.inst_id and o.car_id=ss.car_id
and ss.start_time<=DATE_FORMAT(CURTIME(),'%H:%i') AND ss.finish_time>= date_format(curtime() ,'%H:%i')
and ss.days like concat('%',(select WEEKDAY(now())+1),'%')
LEFT JOIN notify_records bh ON bh.table_id=o.id AND bh.car_route=#route_
and bh.table_name='person' AND bh.notify=4 AND bh.car_id=o.car_id and bh.date_ >= CURDATE() and bh.date_ < CURDATE() + INTERVAL 1 DAY
left join person_records oh on oh.person_id=o.id
and oh.car_id=o.car_id
and date_format(oh.datee,'%H:%i') >=ss.start_time
and date_format(oh.datee,'%H:%i') <=ss.finish_time
AND oh.car_route= #route_
and
oh.id in(select max(id) from person_records
where date_time >= CURDATE() and date_time < CURDATE() + INTERVAL 1 DAY and car_id = carid and car_id = carid
GROUP by person_id
)
left join inst ok on o.inst_id = ok.id and o.car_id=carid
left join excuse_records mh on mh.person_id=o.id and mh.date_time >= CURDATE() and mh.date_time < CURDATE() + INTERVAL 1 DAY and (mh.car_route=ss.car_route)
left join photo_ r on r.table_id = o.id and r.table_name = 'person'
where
(ss.car_route=o.cars_route_ or o.cars_route_=3) and
o.car_id = carid and o.active=1
AND o.work_time=ss.work_time;
elseif #route_ = 2 and #stop_ser=0 then
select DISTINCT ss.start_time,ss.finish_time ,o.id,o.name,r.photo, oh.state ,oh.datee,ss.car_route,
ifnull(bh.id,0) AS called,
ifnull(mh.excuse_id,0) AS excuse_id,
ifnull(o.latitude_1,0) AS latitude_1,
ifnull(o.longitude_1,0) AS longitude_1,
ifnull(o.latitude_2,0) AS latitude_2,
ifnull(o.longitude_2,0) AS longitude_2,
case when (ifnull(o.call_notify,0)=1 or ifnull(o.mes_notify,0)=1) then 1 else 0 end AS call_notify ,
ifnull(o.rownumber,0) AS rownumber,
ifnull(o.number_1,0) AS number_1,
ifnull(o.number_2,0) AS number_2,
ifnull(o.brownumber,0) AS brownumber,
ifnull(ROUND(o.notify_meter_1/2),0) AS notify_meter_1,
ifnull(ROUND(o.notify_meter_2/2),0) AS notify_meter_2
from person o
inner join car_time ss on ss.inst_id =o.inst_id and o.back_car_id=ss.car_id
and ss.start_time<=DATE_FORMAT(CURTIME(),'%H:%i') AND ss.finish_time>= date_format(curtime() ,'%H:%i')
and ss.days like concat('%',(select WEEKDAY(now())+1),'%')
LEFT JOIN notify_records bh ON bh.table_id=o.id AND bh.car_route=#route_
and bh.table_name='person' AND bh.notify=4 AND bh.car_id=o.back_car_id and bh.date_ >= CURDATE() and bh.date_ < CURDATE() + INTERVAL 1 DAY
left join person_records oh on oh.person_id=o.id
and oh.car_id=o.back_car_id and oh.car_route=2
and date_format(oh.datee,'%H:%i') >=ss.start_time
and date_format(oh.datee,'%H:%i') <=ss.finish_time
AND oh.car_route= #route_
and
oh.id in (select max(id) from person_records
where date_time >= CURDATE() and date_time < CURDATE() + INTERVAL 1 DAY and car_id = carid
GROUP by person_id
)
left join inst ok on o.inst_id = ok.id and o.car_id=carid
left join excuse_records mh on mh.person_id=o.id and mh.date_time >= CURDATE() and mh.date_time < CURDATE() + INTERVAL 1 DAY and (mh.car_route=ss.car_route)
left join photo_ r on r.table_id = o.id and r.table_name = 'person'
where
(ss.car_route=o.cars_route_ or o.cars_route_=3) and
o.back_car_id = carid and o.active=1
AND o.work_time=ss.work_time;
END IF;
end
I have a database example here.
I made my.cnf improvement but still have difficulties with performance. What is wrong with this query? What can I change?
Thank you from now.
Edit:
Server version: 10.1.41-MariaDB - MariaDB Server
I have indexes. I forgot to add indexes while creating test data.
What the heck is this?
ss.days like concat('%',(select WEEKDAY(now())+1),'%')
It can at least be sped up by changing to
ss.days like concat('%',WEEKDAY(now()),'%')
And, won't that lead to checking against 2, 21, 20, 12, ... if the WEEKDAY is "2"?
These might be useful for ss:
(car_id, inst_id, start_time)
(inst_id, car_id, finish_time)
LIMIT 1 without ORDER BY leads to some random row being returned? Is the LIMIT redundant? Or is an ORDER BY needed?
Suggest you get some timings -- It is not obvious which of the SELECTs is chewing up the most CPU.
If the PRIMARY KEY of cars is id, then why test for inst_id and active? Yikes! You don't seem to have a PK for cars! Please verify that every table has a PK.
Redundant:
and car_id = carid
and car_id = carid
Why twice? And what tables are those columns in? Please qualify columns so we can understand what is going on.
When #stop_ser=0, the procedure does nothing? In which case, perform that test first, so you can avoid computing #route.
Change start_time to datatype TIME; then you can get rid of DATE_FORMAT in
and ss.start_time<=DATE_FORMAT(CURTIME(),'%H:%i')
AND ss.finish_time>= date_format(curtime() ,'%H:%i')
Also, beware of the inequality tests, it may lead to some edge cases you did not want.
Don't use (m,n) on FLOAT (eg, float(11,7)); it does rounding that is unnecessary. Also, you can't get 7 decimal places for lat/lng except very near the equator and longitude=0. More on precision: http://mysql.rjweb.org/doc.php/latlng#representation_choices
After you have cleaned up those and provided the requested info, I will take another look.
Fairly new to mysql & the stack overflow community. - I have a question that's been bugging me.
I'm trying to return the results of individuals that have taken out a book on 2 specific dates. This is my effort so far;
SELECT borrower.fname, borrower.lname
FROM borrower, loan
WHERE borrower.cardno = loan.cardno
AND loan.dateout = DATE '2019-01-01'
AND loan.dateout = DATE '2018-02-01';
I'm not sure why I'm not getting any results - When I run the query with just one date it comes up with results. I can see from these results that there are definitely records that should satisfy the criteria, I'm not sure where I'm going wrong.
Can anyone pinpoint where I'm going wrong? I thought it looked quite straightforward but it's driving me crazy.
Thanks
One option uses exists twice:
select b.*
from borrower b
where
exists (select 1 from loan l where l.cardno = b.cardno and l.dateout = date '2019-01-01')
and exists (select 1 from loan l where l.cardno = b.cardno and l.dateout = date '2018-02-01')
With an index on loan(cardno, dateout), this should be an efficient solution.
Alternatively, you can use aggregation:
select b.fname, b.lname
from borrower b
inner join loan l on l.cardno = b.cardno
where l.dateout in (date '2019-01-01', date '2018-02-01')
group by b.fname, b.lname
having
max(case when l.dateout = date '2019-01-01' then 1 end) = 1
and max(case when l.dateout = date '2018-02-01' then 1 end) = 1
I have the following query that is quite complex and even though I tried to understand how to do using various sources online, all the examples uses simple queries where mine is more complex, and for that, I don't find the solution.
Here's my current query :
SELECT id, category_id, name
FROM orders AS u1
WHERE added < (UTC_TIMESTAMP() - INTERVAL 60 SECOND)
AND (executed IS NULL OR executed < (UTC_DATE() - INTERVAL 1 MONTH))
AND category_id NOT IN (SELECT category_id
FROM orders AS u2
WHERE executed > (UTC_TIMESTAMP() - INTERVAL 5 SECOND)
GROUP BY category_id)
GROUP BY category_id
ORDER BY added ASC
LIMIT 10;
The table orders is like this:
id
category_id
name
added
executed
The purpose of the query is to list n orders (here, 10) that belong in different categories (I have hundreds of categories), so 10 category_id different. The orders showed here must be older than a minute ago (INTERVAL 60 SECOND) and never executed (IS NULL) or executed more than a month ago.
The NOT IN query is to avoid treating a category_id that has already been treated less than 5 seconds ago. So in the result, I remove all the categories that have been treated less than 5 seconds ago.
I've tried to change the NOT IN in a LEFT JOIN clause or a NOT EXISTS but the switch results in a different set of entries so I believe it's not correct.
Here's what I have so far :
SELECT u1.id, u1.category_id, u1.name, u1.added
FROM orders AS u1
LEFT JOIN orders AS u2
ON u1.category_id = u2.category_id
AND u2.executed > (UTC_TIMESTAMP() - INTERVAL 5 SECOND)
WHERE u1.added < (UTC_TIMESTAMP() - INTERVAL 60 SECOND)
AND (u1.executed IS NULL OR u1.executed < (UTC_DATE() - INTERVAL 1 MONTH))
AND u2.category_id IS NULL
GROUP BY u1.category_id
LIMIT 10
Thank you for your help.
Here's a sample data to try. In that case, there is no "older than 5 seconds" since it's near impossible to get a correct value, but it gives you some data to help out :)
Your query is using a column which doesn't exist in the table as a join condition.
ON u1.domain = u2.category_id
There is no column in your example data called "domain"
Your query is also using the incorrect operator for your 2nd join condition.
AND u2.executed > (UTC_TIMESTAMP() - INTERVAL 5 SECOND)
should be
AND u2.executed < (UTC_TIMESTAMP() - INTERVAL 5 SECOND)
as is used in your first query
I've got a database with events and organizations that run those events. I am looking to create a query that shows any organization that has not created an event in 90 days or more.
So far, I have this query:
SELECT organizations.name, organizations.first_name, organizations.last_name,
organizations.email, events.created_at, events.start_date, events.end_date
FROM events
INNER JOIN organizations ON events.organizer_id = organizations.id
WHERE DATE_SUB(CURDATE(),INTERVAL 90 DAY) > events.created_at
GROUP BY events.organizer_id
ORDER BY events.created_at DESC
The problem is that this will just choose any event that is more than 90 days old, but not the latest event. How do I get the query to look at the event with the newest created_at and see if that is 90 days or older and include only that?
You can use aggregation. If you care about organizations, then you don't need event information in the select:
SELECT o.name, o.first_name, o.last_name, o.email
FROM events e INNER JOIN
organizations o
ON e.organizer_id = o.id
GROUP BY o.id
HAVING DATE_SUB(CURDATE(), INTERVAL 90 DAY) > MAX(e.created_at)
ORDER BY MAX(e.created_at) DESC;
This query will not select organizations that never have events. To do that, you need a left outer join. Here is one way:
SELECT o.name, o.first_name, o.last_name, o.email
FROM organizations o LEFT JOIN
events e
ON e.organizer_id = o.id AND
e.created_at >= DATE_SUB(CURDATE(), INTERVAL 90 DAY)
WHERE e.organizer_id is null
GROUP BY o.id
ORDER BY MAX(e.created_at) DESC;
Note that I also changed the query to use table aliases. These make the query easier to write and read.
Try this:
First create a query (call it "A" here) that selects all organiazations which created an event in the last 90 days.
SELECT organizer_id from events where DATE_SUB(CURDATE(),INTERVAL 90 DAY) <= events.created_at
Then write a query which lists all organizations which are not in the result set of query A:
SELECT * from organizations where id not in (A)
Which makes it to:
SELECT * from organizations where id not in (SELECT organizer_id from events where DATE_SUB(CURDATE(),INTERVAL 90 DAY) <= events.created_at)
This works because SQL is "orthogonal", i.e., you can embed queries into other queries.