I have a Mysql Table that holds dates that are booked (for certain holiday properties).
Example...
Table "listing_availability"
Rows...
availability_date (this shows the date format 2013-04-20 etc)
availability_bookable (This can be yes/no. "Yes" = the booking changeover day and it is "available". "No" means the property is booked for those dates)
All the other dates in the year (apart from the ones with "No") are available to be booked. These dates are not in the database, only the booked dates.
My question is...
I have to make a SQL Statement that first calls the Get Date Function (not sure if this is correct terminology)
Then removes the dates from "availability_date" WHERE "availability_bookable" = "No"
This will give me the dates that are available for bookings, for the year, for a property.
Can anyone help?
Regards M
Seems like you've almost written the query.
SELECT availability_date FROM listing_availability
WHERE availability_bookable <> 'NO'
AND availability_date >= CURDATE()
AND YEAR(CURDATE()) = YEAR(availability_date)
I think I understand, and you'll obviously confirm. Your "availability_booking" has some records in it, but not every single day of the year, only those that may have had something, and not all are committed, some could have yes, some no.
So, you want to simulate All dates within a given date range... Say April 1 - July 1 as someone is looking to book a party within that time period. Instead of pre-filling your production table, you can't say that April 27th is open and available... since no such record exists.
To SIMULATE a calendar of days for a date range, you can do it using MySQL variables and join to "any" table in your database provided it has enough records to SIMULATE the date range you want...
select
#myDate := DATE_ADD( #myDate, INTERVAL 1 DAY ) as DatesForAvailabilityCheck
from
( select #myDate := '2013-03-31' ) as SQLVars,
AnyTableThatHasEnoughRows
limit
120;
This will just give you a list of dates starting with April 1, 2013 (the original #myDate is 1 day before the start date since the field selection adds 1 day to it to get to April 1, then continues... for a limit of 120 days (or whatever you are looking for range based -- 30days, 60, 90, 22, whatever). The "AnyTableThatHasEnoughRows" could actually be your "availability_booking" table, but we are just using it as a table with rows, no join or where condition, just enough to get ... 120 records.
Now, we can use this to join to whatever table you want and apply your condition. You just created a full calendar of days to compare against. Your final query may be different, but this should get it most of the way for you.
select
JustDates.DatesForAvailabilityCheck,
from
( select
#myDate := DATE_ADD( #myDate, INTERVAL 1 DAY ) as DatesForAvailabilityCheck
from
( select #myDate := '2013-03-31' ) as SQLVars,
listing_availability
limit
120 ) JustDates
LEFT JOIN availability_bookable
on JustDates.DatesForAvailabilityCheck = availability_bookable.availability_date
where
availability_bookable.availability_date IS NULL
OR availability_bookable.availability_bookable = "Yes"
So the above uses the sample calendar and looks to the availability. If no such matching date exists (via the IS NULL), then you want it meaning there is no conflict. However, if there IS a record in the table, you only want those where YES, you CAN book it, the entry on file might not be committed and CAN be in your result query of available dates.
Related
Based on an advise from another developer here on stackoverflow, I have updated my query as below, but I still need to optimise it further. Can someone guide my on how best I can apply indexing to the query.
See Query Below:
SELECT a.id, a.user_unique_id, a.loan_location,
a.ippis, a.tel_no,
a.organisation, a.branch, a.loan_agree,
a.loan_type, a.appr, a.sold,
a.loan_status, a.top_up, a.current_loan,
a.date_created, a.date_updated, c.loan_id, c.user_unique_id AS tu_user_unique_id,
c.ippis AS tu_ippis, c.top_up_approved,
c.loan_type AS tu_loan_type, c.dse, c.status, c.current_loan AS tu_current_loan,
c.record_category, c.date_created AS tu_date_created,
c.date_updated AS tu_date_updated
FROM loan_applications_tbl a
LEFT JOIN topup_or_reapplication_tbl c
ON a.ippis=c.ippis
WHERE ((c.status IN ('pending', 'corrected', 'Rejected', 'Processing', 'Captured', 'Reviewed', 'top up')
AND MONTH(CURRENT_DATE) IN (MONTH(c.date_created), MONTH(c.date_updated)
AND YEAR(CURRENT_DATE) IN (YEAR(c.date_created), YEAR(c.date_updated))
AND c.current_loan='1' ))
OR ( a.loan_status IN ('pending', 'corrected', 'Rejected', 'Processing', 'Captured', 'Reviewed', 'top up')
AND MONTH(CURRENT_DATE) IN (MONTH(a.date_created), MONTH(a.date_updated)) )
AND YEAR(CURRENT_DATE) IN (YEAR(a.date_created), YEAR(a.date_updated))
AND (a.current_loan='1'
OR (a.current_loan='0'
AND a.loan_status IN('Approved','Closed')))))
Execution time: 53s
No of records: 11000
using mysql EXPLAIN gives the screenshot below: (How do I maximise the information in the possible_keys column
I HAVE UPDATED ADDITIONAL INFORMATION BELOW:
I am using the OR between c and a for the below reasons:
a is the parent table with 66 columns which gets populated with loan entries, if a new entry on a has a matching/existing ippis (Unique field on a) some columns in a are updated/overwriten with data from the new entry, while the remaining data in the entry are inserted as new rows in c (ippis is not unique in table c). This is to keep a history of all subsequent loan requests while not giving room for redundancy
While retrieving records I needed the big OR clause to enable me check both a and c tables for all instances of each loan records where the status, date and current_loan columns match the parameters in my WHERE clause.
a will ALWAYS have a complete record in it but c will NOT ALWAYS have a record in it, except there are more loan requests for the same unique ID. a contains the "who is the account person such as by unique ID", and the additional / supplemental status detail FOR THE FIRST LOAN, subsequently, after the first loan "c" will be additional / supplemental status detail of the actual loan applications with the same Unique ID.
if "A" is created Mar 12, and a new "c" record is created on Mar 16. The "A" record also gets the last updated stamped with Mar 16 since it has a child attachment that has some impact against it, while the new c record has it own created and updated time stamps. The Updated field will be blank/null for the a record until changes are made or there is a c record, The Updated field will be blank/null for c record until there's some changes made to c record
I hope this is understandable
I keep forgetting the term as it comes up very rarely to me, but anyhow, your indexes can not be optimized by using MONTH() and YEAR() as they are functions on the underlying data. By applying a date RANGE, they can. So you can keep your month/year such as if something was created in Jan 2021 and updated in Mar 2021, but in addition, adding an "and c.date_created >= current_date AND current_date <= c.date_updated", you CAN utilize the index if it has the created date in it (less important in this case for the date updated.
Similarly for your other table.
In addition, when you have your left-join from the "a" to the "c" table, then applying where, its almost like you are trying to force the join but remains left-join due to the OR.
I would move the "c" based condition to the left-join, then just test for the record found there as NULL or not.
Although not clear (was not clarified when I asked), I THINK that when a new "A" record is created, the system may actually put the creation date into both the date created and date updated. IF THIS IS THE CASE, then we only need to query/concern the last updated date field with the current month/year of activity. That is now the PRIMARY requirement for the where clause -- REGARDLESS of the underlying OR condition to the "C" table.
Additionally, since the month() and year() are not sargeable (Thanks Ollie), I am doing a prequery to get the beginning of the current month and next month so I can build out a
WHERE > beginning of this month and LESS than beginning of next month
As for indexes, I would start update to
loan_applications_tbl ( date_created, date_updated, loan_status, current_loan, ippis )
topup_or_reapplication_tbl ( ippis, status, current_loan, date_created, date_updated )
Final query to try with.
SELECT
a.id,
a.user_unique_id,
a.loan_location,
a.ippis,
a.tel_no,
a.organisation,
a.branch,
a.loan_agree,
a.loan_type,
a.appr,
a.sold,
a.loan_status,
a.top_up,
a.current_loan,
a.date_created,
a.date_updated,
c.loan_id,
c.user_unique_id tu_user_unique_id,
c.ippis tu_ippis,
c.top_up_approved,
c.loan_type tu_loan_type,
c.dse,
c.status,
c.current_loan tu_current_loan,
c.record_category,
c.date_created tu_date_created,
c.date_updated tu_date_updated
FROM
-- this creates inline mySQL variables I can use for the WHERE condition
-- by doing comma after with no explicit join, it is a single row
-- and thus no Cartesian result, just #variables available now
( select
-- first truncating any TIME portion by casting to DATE()
#myToday := date(curdate()),
#howFarBack := date_sub( #myToday, interval 6 month ),
-- now subtract day of month -1 to get first of THIS month
#beginOfMonth := date_sub( #myToday, interval dayOfMonth( #myToday ) -1 day ),
-- and now, add 1 month for beginning of next
#beginNextMonth := date_add( #beginOfMonth, interval 1 month ) ) SqlVars,
loan_applications_tbl a
LEFT JOIN topup_or_reapplication_tbl c
ON a.ippis = c.ippis
AND c.current_loan='1'
AND c.status IN ('pending', 'corrected', 'Rejected',
'Processing', 'Captured', 'Reviewed', 'top up')
AND
(
(#beginOfMonth <= c.date_created
AND c.date_created < #beginNextMonth)
OR
(#beginOfMonth <= a.date_updated
AND a.date_updated < #beginNextMonth )
)
WHERE
-- forces only activity for the single month in question
-- since the "a" table knows of any "updates" to the "C",
-- its updated basis will keep overall restriction to any accounts
-- updated within this month in question only
-- testing specifically for created OR updated within the
-- current month in question
a.date_created >= #howFarBack
AND
(
(#beginOfMonth <= a.date_created
AND a.date_created < #beginNextMonth)
OR
(#beginOfMonth <= a.date_updated
AND a.date_updated < #beginNextMonth )
)
-- and NOW we can easily apply the OR without requiring
-- to run against the ENTIRE set of BOTH tables.
AND (
c.ippis IS NOT NULL
OR
( a.loan_status IN ( 'pending', 'corrected', 'Rejected', 'Processing',
'Captured', 'Reviewed', 'top up')
AND (
a.current_loan = '1'
OR ( a.current_loan = '0'
AND a.loan_status IN ('Approved', 'Closed')
)
)
)
)
CLOSING COMMENTS FOR QUERY
I modified the query and also the primary index on the first table to INCLUDE (first position) the date created of the record. I also added an additional variable #howFarBack to be the maximum going back time to consider for a loan. I defaulted to 6 months back. Would you ever need to consider a given account older than 6 months for a loan? Or is the "a" account records something that could go back 10 years and want to include? My impression is that it is a new LOAN APPLICATION add date. If so, allowing to go 6 months back before it is approved, finalized, cancelled would still prevent going through as many months of data historically.
In the WHERE clause, I added explicit add for the CREATED_DATE >= #howFarBack. It would never be possible for a child record to be created, let alone updated any time prior to the original add date. This will force only current month activity OR FORWARD to qualify.
Ex: Create a loan on April 28th. So running the query, the beginning of the month is April 1st but LESS than May 1st (this allows inclusion of April 30 at 11:59:59pm)
Now, we get into May and a change on the loan is done on May 4th. We are in a new month and the #howFarBack still allows older applications as far as December 2020 to POSSIBLY qualify vs the entire table of applications that could go back as far as 2005 for all we know. You are always staying with the most current data and you can change the #howFarBack easily enough as the maximum going back time. This should help your performance needs.
We have a business that rents out international phone numbers to customers when traveling. When a customer makes an order We want to display to the customer the available phone numbers for his booking dates based on his start_date and end_date and numbers which is not occupied yet.
Since these phone numbers are rented out, I need to select from the table ONLY those numbers that are not rented out yet for dates that would interfere with the current customers dates.
I also don't want to rent out any phone number prior to 7 days after its end date. Meaning, If a customer booked a phone number for 1-1-2020 through 1-20-2020, I don't want this phone number to be booked by another customer before 1-27-2020. I want the phone number to have a 7 day window of being clear.
I have a table with the phone numbers and a table with the orders that is related to the phone numbers table via phone_number_id. The orders table has the current customers start_date and end_date for travel without the phone number id saved yet to it. The orders table also has the start_date and end_date for all other customers dates of travel as well as which phone_number_id was assigned/booked up for their travel dates.
How would the MySQL query look like when trying to select the phone numbers that are available for the current customers dates?
I build below query at the moment
SELECT x.id
, x.area_code
, x.phone_number
, y.start_date
, y.end_date
FROM vir_num_table x
LEFT
JOIN orderitemsdetail_table y
ON y.vn_id = x.id
WHERE y.start_date BETWEEN '2020-01-11' AND '2020-01-18'
OR y.start_date IS NULL
I've build this query but stuck here how can I add end_date logic.
Any help would be appreciated! Thanks in advance.
The way I'd approach the problem would be to look at conceptually, is as a cross product of the set of all phone numbers, along with the reservation timeframe, and then exclude those where there's a conflicting reservation.
A conflict would be an overlap, existing reservation that has a start_date before the end of the proposed reservation AND has an end_date on or after the start of the proposed reservation.
I'd do an anti-join pattern, something like this:
SELECT pn.phone_number
FROM phone_number pn
LEFT
JOIN reservation rs
ON rs.phone_number = pn.phone_number
AND rs.start_dt <= '2019-12-27' + INTERVAL +7 DAY
AND rs.end_dt > '2019-12-20' + INTERVAL -7 DAY
WHERE rs.phone_number IS NULL
That essentially says get all rows from phone number, along with matching rows from reservations (rows that overlap), but then exclude all the rows that had a match, leaving just phone_number rows that did not have a match.
We can make the < test a <= or , subtract 8 days, to tailor the "7 day" window before; we can tweak as we run the query through the test cases,
We can achieve an equivalent result using a NOT EXISTS and a correlated subquery. Some people find this easier to comprehend than the ant-join, but its essentially the same query, doing the same thing, get all rows from phone_number but exclude the rows where there is a matching (overlapping) row in reservation
SELECT pn.phone_number
FROM phone_number pn
WHERE NOT EXISTS
( SELECT 1
FROM reservation rs
WHERE rs.phone_number = pn.phone_number
AND rs.start_dt <= '2019-12-27' + INTERVAL +7 DAY
AND rs.end_dt > '2019-12-20' + INTERVAL -7 DAY
)
There are several questions on StackOverflow about checking for overlap, or no overlap, of date ranges.
See e.g.
How to check if two date ranges overlap in mysql?
PHP/SQL - How can I check if a date user input is in between an existing date range in my database?
MySQL query to select distinct rows based on date range overlapping
EDIT
Based on the SQL added as an edit to the question, I'd do the query like this:
SELECT pn.`id`
, pn.`area_code`
, pn.`phone_number`
FROM `vir_num_table` pn
LEFT
JOIN `orderitemsdetail_table` rs
ON rs.vn_id = pn.id
AND rs.start_date <= '2020-01-18' + INTERVAL +7 DAY
AND rs.end_date > '2020-01-11' + INTERVAL -7 DAY
WHERE rs.vn_id IS NULL
The two "tricky" parts. First is the anti-join, understanding how that works. (An outer join, to return all rows from vir_num_table but exclude any rows that have a matching row in reservations. The second tricky part is checking for the overlap, coming up with the conditions: r.start <= p.end AND r.end >= p.start, then tweaking whether we want to include the equals as an overlap, and tweaking the extra seven days (easiest to me to just subtract the 7 days from the beginning of the proposed reservation)
... now occurs to me like we need to add a guard period of 7 days on the end of the reservation period as well, doh!
Here's a query plus sorting algo to choose the optimal phone number selection for maximum utilization efficiency (i.e. getting as close as possible to exactly 7 days before and after each use).
I set it to give open ends a weight of 9, so that "near perfect" fits (7-8 days before or after) would be selected ahead of open-ended numbers. This will yield a slight efficiency improvement, as open numbers can accommodate any reservation. You can adjust this for your needs. If you set this to 0, for example, it would always select open numbers first.
SELECT ph.phone_number,
COALESCE(
MIN(
IF(res.end_date > res.start_date > '2020-01-18',
NULL, -- ignore before-comparison for reservations starting and ending after date range
DATEDIFF('2020-01-11', res.end_date)
), 9) AS open_days_before,
COALESCE(
MIN(
IF(res.start_date < res.end_date < '2020-01-11',
NULL, -- ignore after-comparison for reservations starting and ending before date range
DATEDIFF(res.start_date, '2020-01-18')
), 9) AS open_days_after
FROM phone_number ph
LEFT JOIN reservation res
ON res.phone_number = ph.phone_number
AND res.end_date >= CURRENT_DATE() - INTERVAL 6 DAY
GROUP BY ph.phone_number
HAVING open_days_before >= 7
AND open_days_after >= 7
ORDER BY open_days_before + open_days_after
LIMIT 1
Edit: updated to add grouping, because I realize this is an aggregate problem.
Edit 2: bug fix, changed MAX to MIN
Edit 3: added res.end_date >= CURRENT_DATE - INTERVAL 6 DAY to ignore past reservations, limiting aggregate data and treating phone number with no reservations between 6 days ago and the beginning of the new order as "open on the front-end"
Edit 4: added IF conditions to eliminate reservations outside the given before-or-after comparison ranges (e.g. comparing reservations after the selected range from influencing the "open days before" number), to prevent negative numbers, except when there's overlap with the selected range.
Based on the info you've added then you shouldn't need to check the start date of phone numbers which have been booked out.
You customer provides you with a start date and an end date.
You only rent out phone numbers 7 days after their last lease ended
All you need to do is fetch back phone numbers which either:
- Are not rented out and therefor aren't in the orderitems table
- OR have an end_date which is 7 days before the new customer's start date.
Here you go:
SELECT
`main_table`.`id`,
`main_table`.`area_code`,
`main_table`.`phone_number`,
`orderitemsdetail_table`.`start_date`,
`orderitemsdetail_table`.`end_date`
FROM
`vir_num_table` AS `main_table`
LEFT JOIN
`orderitemsdetail_table` AS `orderitemsdetail_table` ON main_table.id = orderitemsdetail_table.vn_id
WHERE
(DATE_ADD(orderitemsdetail_table.end_date, INTERVAL 7 DAY) < '<CUSTOMER START DATE>'
AND orderitemsdetail_table.start_date > '<CUSTOMER END DATE>')
OR orderitemsdetail_table.id IS NULL
I am using the Graph Reports for the select below. The MySQL database only has the active records in the database, so if no records are in the database from X hours till Y hours that select does not return anything. So in my case, I need that select return Paypal zero values as well even the no activity was in the database. And I do not understand how to use the UNION function or re-create select in order to get the zero values if nothing was recorded in the database in time interval. Could you please help?
select STR_TO_DATE ( DATE_FORMAT(`acctstarttime`,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', count(*) as `Active Paid Accounts`
from radacct_history where `paymentmethod` = 'PayPal'
group by DATE_FORMAT(`#date`,'%y-%m-%d %H')
When I run the select the output is:
Current Output
But I need if there are no values between 2016-07-27 07:00:00 and 2016-07-28 11:00:00, then in every hour it should show zero active accounts Like that:
Needed output with no values every hour
I have created such select below , but it not put to every hour the zero value like i need. showing the big gap between the 12 Sep and 13 Sep anyway, but there should be the zero values every hour
(select STR_TO_DATE ( DATE_FORMAT(acctstarttime,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', count(paymentmethod) as Active Paid Accounts
from radacct_history where paymentmethod <> 'PayPal'
group by DATE_FORMAT(#date,'%y-%m-%d %H'))
union ALL
(select STR_TO_DATE ( DATE_FORMAT(acctstarttime,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', 0 as Active Paid Accounts
from radacct_history where paymentmethod <> 'PayPal'
group by DATE_FORMAT(#date,'%y-%m-%d %H')) ;
I guess, you want to return 0 if there is no matching rows in MySQL. Here is an example:
(SELECT Col1,Col2,Col3 FROM ExampleTable WHERE ID='1234')
UNION (SELECT 'Def Val' AS Col1,'none' AS Col2,'' AS Col3) LIMIT 1;
Updated the post: You are trying to retrieve data that aren't present in the table, I guess in reference to the output provided. So in this case, you have to maintain a date table to show the date that aren't in the table. Please refer to this and it's little bit tricky - SQL query that returns all dates not used in a table
You need an artificial table with all necessary time intervals. E.g. if you need daily data create a table and add all day dates e.g. start from 1970 till 2100.
Then you can use the table and LEFT JOIN your radacct_history. So for each desired interval you will have group item (group by should be based on the intervals table.
I have attendance data for employees stored in the table attendance with the following column names:
emp_id (employee ID)
date
type (leave, absent, etc.)
(there are others but I'm omitting them for the sake of simplicity)
My objective is to retrieve all dates of the given month on which the employee was on leave (type = 'Leave') and the last leave taken in the last month, if any.
It's easy to do it using two queries (I'm using PHP to get process the data), but is there any way this can be done in a single query?
I'm answering my own question so as to close it. As #bpgergo pointed out in the comments, UNION will do the trick here.
SELECT * FROM table_name
WHERE type="Leave" AND
date <= (CURRENT_DATE() - 30)
Select the fields, etc you want then se a combined where clause using mysql's CURRENT_DATE() function. I subtracted 30 for 30 days in a month.
If date is a date column, this will return everyone who left 1 month or longer ago.
Edit:
If you want a specific date, change the 2nd month like this:
date <= (date_number - 30)
I want to get stats for each day in a given month. However, if a day has no rows in the table, it doesn't show up in the results. How can I include days with no data, and show all days until the current date?
This is the query I have now:
SELECT DATE_FORMAT(FROM_UNIXTIME(timestamp), '%d'), COUNT(*)
FROM data
WHERE EXTRACT(MONTH FROM FROM_UNIXTIME(timestamp)) = 6
GROUP BY EXTRACT(DAY FROM FROM_UNIXTIME(timestamp))
So if I have
Row 1 | 01-06
Row 2 | 02-06
Row 3 | 03-06
Row 4 | 05-06
Row 5 | 05-06
(i changed timestamp values to a day/month date just to explain)
It should output
01 | 1
02 | 1
03 | 1
04 | 0
05 | 2
06 | 0
...Instead of ignoring day 4 and today (day 6).
You will need a calendar table to do something in the form
SELECT `date`, count(*)
FROM Input_Calendar c
LEFT JOIN Data d on c.date=d.date
GROUP BY `date`
I keep a full copy of a calendar table in my database and used a WHILE loop to fill it but you can populate one on the fly for use based on the different solutions out there like http://crazycoders.net/2012/03/using-a-calendar-table-in-mysql/
In MySQL, you can use MySQL variables (act like in-line programming values). You set and can manipulate as needed.
select
dayofmonth( DynamicCalendar.CalendarDay ) as `Day`,
count(*) as Entries
from
( select
#startDate := date_add( #startDate, interval 1 day ) CalendarDay
from
( select #startDate := '2013-05-31' ) sqlvars,
AnyTableThatHasAsManyDaysYouExpectToReport
limit
6 ) DynamicCalendar
LEFT JOIN Input_Calendar c
on DynamicCalendar.CalendarDay = date( from_unixtime( c.date ))
group by
DynamicCalendar.CalendarDay
In the above sample, the inner query can join against as the name implies "Any Table" in your database that has at least X number of records you are trying to generate for... in this case, you are dealing with only the current month of June and only need 6 records worth... But if you wanted to do an entire year, just make sure the "Any Table" has 365 records(or more).
The inner query will start by setting the "#startDate" to the day BEFORE June 1st (May 31). Then, by just having the other table, will result in every record joined to this variable (creates a simulated for/next loop) via a limit of 6 records (days you are generating the report for). So now, as the records are being queried, the Start Date keeps adding 1 day... first record results in June 1st, next record June 2nd, etc.
So now, you have a simulated calendar with 6 records dated from June 1 to June 6. Take that and join to your "data" table and you are already qualifying your dates via the join and get only those dates of activity. I'm joining on the DATE() of the from unix time since you care about anything that happend on June 1, and June 1 # 12:00:00AM is different than June 1 # 8:45am, so matching on the date only portion, they should remain in proper grouping.
You could expand this answer by changing the inner '2013-05-31' to some MySQL Date function to get the last day of the prior month, and the limit based on whatever day in the current month you are doing so these are not hard-coded.
Create a Time dimension. This is a standard OLAP reporting trick. You don't need a cube in order to do OLAP tricks, though. Simply find a script on the internet to generate a Calendar table and join to that table.
Also, I think your query is missing a WHERE clause.
Other useful tricks include creating a "Tally" table that is a list of numbers from 1 to N where N is usually the max of the bigint on your database management system.
No code provided here, as I am not a MySQL guru.
Pseudo-code is:
Select * from Data left join TimeDimension on data.date = timedimension.date