Based on an advise from another developer here on stackoverflow, I have updated my query as below, but I still need to optimise it further. Can someone guide my on how best I can apply indexing to the query.
See Query Below:
SELECT a.id, a.user_unique_id, a.loan_location,
a.ippis, a.tel_no,
a.organisation, a.branch, a.loan_agree,
a.loan_type, a.appr, a.sold,
a.loan_status, a.top_up, a.current_loan,
a.date_created, a.date_updated, c.loan_id, c.user_unique_id AS tu_user_unique_id,
c.ippis AS tu_ippis, c.top_up_approved,
c.loan_type AS tu_loan_type, c.dse, c.status, c.current_loan AS tu_current_loan,
c.record_category, c.date_created AS tu_date_created,
c.date_updated AS tu_date_updated
FROM loan_applications_tbl a
LEFT JOIN topup_or_reapplication_tbl c
ON a.ippis=c.ippis
WHERE ((c.status IN ('pending', 'corrected', 'Rejected', 'Processing', 'Captured', 'Reviewed', 'top up')
AND MONTH(CURRENT_DATE) IN (MONTH(c.date_created), MONTH(c.date_updated)
AND YEAR(CURRENT_DATE) IN (YEAR(c.date_created), YEAR(c.date_updated))
AND c.current_loan='1' ))
OR ( a.loan_status IN ('pending', 'corrected', 'Rejected', 'Processing', 'Captured', 'Reviewed', 'top up')
AND MONTH(CURRENT_DATE) IN (MONTH(a.date_created), MONTH(a.date_updated)) )
AND YEAR(CURRENT_DATE) IN (YEAR(a.date_created), YEAR(a.date_updated))
AND (a.current_loan='1'
OR (a.current_loan='0'
AND a.loan_status IN('Approved','Closed')))))
Execution time: 53s
No of records: 11000
using mysql EXPLAIN gives the screenshot below: (How do I maximise the information in the possible_keys column
I HAVE UPDATED ADDITIONAL INFORMATION BELOW:
I am using the OR between c and a for the below reasons:
a is the parent table with 66 columns which gets populated with loan entries, if a new entry on a has a matching/existing ippis (Unique field on a) some columns in a are updated/overwriten with data from the new entry, while the remaining data in the entry are inserted as new rows in c (ippis is not unique in table c). This is to keep a history of all subsequent loan requests while not giving room for redundancy
While retrieving records I needed the big OR clause to enable me check both a and c tables for all instances of each loan records where the status, date and current_loan columns match the parameters in my WHERE clause.
a will ALWAYS have a complete record in it but c will NOT ALWAYS have a record in it, except there are more loan requests for the same unique ID. a contains the "who is the account person such as by unique ID", and the additional / supplemental status detail FOR THE FIRST LOAN, subsequently, after the first loan "c" will be additional / supplemental status detail of the actual loan applications with the same Unique ID.
if "A" is created Mar 12, and a new "c" record is created on Mar 16. The "A" record also gets the last updated stamped with Mar 16 since it has a child attachment that has some impact against it, while the new c record has it own created and updated time stamps. The Updated field will be blank/null for the a record until changes are made or there is a c record, The Updated field will be blank/null for c record until there's some changes made to c record
I hope this is understandable
I keep forgetting the term as it comes up very rarely to me, but anyhow, your indexes can not be optimized by using MONTH() and YEAR() as they are functions on the underlying data. By applying a date RANGE, they can. So you can keep your month/year such as if something was created in Jan 2021 and updated in Mar 2021, but in addition, adding an "and c.date_created >= current_date AND current_date <= c.date_updated", you CAN utilize the index if it has the created date in it (less important in this case for the date updated.
Similarly for your other table.
In addition, when you have your left-join from the "a" to the "c" table, then applying where, its almost like you are trying to force the join but remains left-join due to the OR.
I would move the "c" based condition to the left-join, then just test for the record found there as NULL or not.
Although not clear (was not clarified when I asked), I THINK that when a new "A" record is created, the system may actually put the creation date into both the date created and date updated. IF THIS IS THE CASE, then we only need to query/concern the last updated date field with the current month/year of activity. That is now the PRIMARY requirement for the where clause -- REGARDLESS of the underlying OR condition to the "C" table.
Additionally, since the month() and year() are not sargeable (Thanks Ollie), I am doing a prequery to get the beginning of the current month and next month so I can build out a
WHERE > beginning of this month and LESS than beginning of next month
As for indexes, I would start update to
loan_applications_tbl ( date_created, date_updated, loan_status, current_loan, ippis )
topup_or_reapplication_tbl ( ippis, status, current_loan, date_created, date_updated )
Final query to try with.
SELECT
a.id,
a.user_unique_id,
a.loan_location,
a.ippis,
a.tel_no,
a.organisation,
a.branch,
a.loan_agree,
a.loan_type,
a.appr,
a.sold,
a.loan_status,
a.top_up,
a.current_loan,
a.date_created,
a.date_updated,
c.loan_id,
c.user_unique_id tu_user_unique_id,
c.ippis tu_ippis,
c.top_up_approved,
c.loan_type tu_loan_type,
c.dse,
c.status,
c.current_loan tu_current_loan,
c.record_category,
c.date_created tu_date_created,
c.date_updated tu_date_updated
FROM
-- this creates inline mySQL variables I can use for the WHERE condition
-- by doing comma after with no explicit join, it is a single row
-- and thus no Cartesian result, just #variables available now
( select
-- first truncating any TIME portion by casting to DATE()
#myToday := date(curdate()),
#howFarBack := date_sub( #myToday, interval 6 month ),
-- now subtract day of month -1 to get first of THIS month
#beginOfMonth := date_sub( #myToday, interval dayOfMonth( #myToday ) -1 day ),
-- and now, add 1 month for beginning of next
#beginNextMonth := date_add( #beginOfMonth, interval 1 month ) ) SqlVars,
loan_applications_tbl a
LEFT JOIN topup_or_reapplication_tbl c
ON a.ippis = c.ippis
AND c.current_loan='1'
AND c.status IN ('pending', 'corrected', 'Rejected',
'Processing', 'Captured', 'Reviewed', 'top up')
AND
(
(#beginOfMonth <= c.date_created
AND c.date_created < #beginNextMonth)
OR
(#beginOfMonth <= a.date_updated
AND a.date_updated < #beginNextMonth )
)
WHERE
-- forces only activity for the single month in question
-- since the "a" table knows of any "updates" to the "C",
-- its updated basis will keep overall restriction to any accounts
-- updated within this month in question only
-- testing specifically for created OR updated within the
-- current month in question
a.date_created >= #howFarBack
AND
(
(#beginOfMonth <= a.date_created
AND a.date_created < #beginNextMonth)
OR
(#beginOfMonth <= a.date_updated
AND a.date_updated < #beginNextMonth )
)
-- and NOW we can easily apply the OR without requiring
-- to run against the ENTIRE set of BOTH tables.
AND (
c.ippis IS NOT NULL
OR
( a.loan_status IN ( 'pending', 'corrected', 'Rejected', 'Processing',
'Captured', 'Reviewed', 'top up')
AND (
a.current_loan = '1'
OR ( a.current_loan = '0'
AND a.loan_status IN ('Approved', 'Closed')
)
)
)
)
CLOSING COMMENTS FOR QUERY
I modified the query and also the primary index on the first table to INCLUDE (first position) the date created of the record. I also added an additional variable #howFarBack to be the maximum going back time to consider for a loan. I defaulted to 6 months back. Would you ever need to consider a given account older than 6 months for a loan? Or is the "a" account records something that could go back 10 years and want to include? My impression is that it is a new LOAN APPLICATION add date. If so, allowing to go 6 months back before it is approved, finalized, cancelled would still prevent going through as many months of data historically.
In the WHERE clause, I added explicit add for the CREATED_DATE >= #howFarBack. It would never be possible for a child record to be created, let alone updated any time prior to the original add date. This will force only current month activity OR FORWARD to qualify.
Ex: Create a loan on April 28th. So running the query, the beginning of the month is April 1st but LESS than May 1st (this allows inclusion of April 30 at 11:59:59pm)
Now, we get into May and a change on the loan is done on May 4th. We are in a new month and the #howFarBack still allows older applications as far as December 2020 to POSSIBLY qualify vs the entire table of applications that could go back as far as 2005 for all we know. You are always staying with the most current data and you can change the #howFarBack easily enough as the maximum going back time. This should help your performance needs.
Related
I have a number of stores where I would like to sum the energy consumption so far this year compared with the same period last year. My challenge is that in the current year the stores have different date intervals in terms of delivered data. That means that store A may have data between 01.01.2018 and 20.01.2018, and store B may have data between 01.01.2018 and 28.01.2018. I would like to sum the same date intervals current year versus previous year.
Data looks like this
Store Date Sum
A 01.01.2018 12
A 20.01.2018 11
B 01.01.2018 33
B 28.01.2018 32
But millions of rows and would use these dates as references to get the same sums previous year.
This is my (erroneous) try:
SET #curryear = (SELECT YEAR(MAX(start_date)) FROM energy_data);
SET #maxdate_curryear = (SELECT MAX(start_date) FROM energy_data WHERE
YEAR(start_date) = #curryear);
SET #mindate_curryear = (SELECT MIN(start_date) FROM energy_data WHERE
YEAR(start_date) = #curryear);
-- the same date intervals last year
SET #maxdate_prevyear = (#maxdate_curryear - INTERVAL 1 YEAR);
SET #mindate_prevyear = (#mindate_curryear - INTERVAL 1 YEAR);
-- sums current year
CREATE TABLE t_sum_curr AS
SELECT name as name_curr, sum(kwh) as sum_curr, min(start_date) AS
min_date_curr, max(start_date) AS max_date_curr, count(distinct
start_date) AS ant_timer FROM energy_data WHERE agg_type = 'timesnivå'
AND start_date >= #mindate_curryear and start_date <= #maxdate_curryear GROUP BY NAME;
-- also seems fair, the same dates one year ago, figured I should find those first and in the next query use that to sum each stores between those date intervals
CREATE TABLE t_sum_prev AS
SELECT name_curr as name_curr2, (min_date_curr - INTERVAL 1 YEAR) AS
min_date_prev, (max_date_curr - INTERVAL 1 YEAR) as max_date_prev FROM
t_sum_curr;
-- getting into trouble!
CREATE TABLE the_results AS
SELECT name, start_date, sum(kwh) as sum_prev from energy_data where
agg_type = 'timesnivå' and
start_date >= #mindate_prevyear and start_date <=
#maxdate_prevyear group by name having start_date BETWEEN (SELECT
min_date_prev from t_sum_prev) AND
(SELECT max_date_prev from t_sum_prev);
`
This last query just tells me that my sub query returns more than 1 row and throws an error message.
I assume what you have is a list of energy consumption figures, where bills or readings have been taken at irregular times, so the consumption covers irregular periods.
The basic approach you need to take is to regularise the consumption periods - by establishing which days each periods covers, and then breaking each reading down into as many days as it covers, and the consumption for each day being a daily average of the period.
I'm assuming the consumption periods are entirely sequential (as a bill or reading normally would be), and not overlapping.
Because of the volume of rows involved (you say millions even in its current form), you might not want to leave the data in daily form - it might suffice to regroup them into regular weekly, monthly, or quarterly periods, depending on what level of granularity you require for comparison.
Once you have your regular periods, comparison will be as easy as cake.
If this is part of a report that will be run on an ongoing basis, you'd probably want to implement some logic that calculates a "regularised consumption" incrementally and on a scheduled basis and stores it in a summary table, with appropriate columns and indexes, so that you aren't having to process many millions of historical rows each time the report is run.
Trying to work around the irregular periods (if indeed it can be done) with fancy joins and on-the-fly averages, rather than tackling them head on, will likely lead to very difficult logic, and particularly on a data set of this size, dire performance.
EDIT: from the comments below.
#Alexander, I've knocked together an example of a query. I haven't tested it and I've written it all in a text editor, so excuse any small syntax errors. What I've come up with seems a bit complex (more complex than I imagined when I began), but I'm also a little bit tired, so I'm not sure whether it could be simplified further.
The only point I would make is that the performance of this query (or any such query), because of the nature of what it has to do in traversing date ranges, is likely to be appalling on a table with millions of rows. I stand by my earlier remarks, that proper indexing of the source data will be crucial, and summarising the source data into a larger granularity will massively aid performance (at the expense of a one-off hit to summarise it). Even daily granularity, will reduce the number of rows by a factor of 24!
WITH energy_data_ext AS
(
SELECT
ed.name AS store_name
,YEAR(ed.start_date) AS reading_year
,ed.start_date AS reading_date
,ed.kwh AS reading_kwh
FROM
energy_data AS ed
)
,available_stores AS
(
SELECT ede.store_name
FROM energy_data_ext AS ede
GROUP BY ede.store_name
)
,current_reading_yr_per_store AS
(
SELECT
ede.store_name
,MAX(ede.reading_year) AS current_reading_year
FROM
energy_data_ext AS ede
GROUP BY
ede.store_name
)
,latest_reading_ranges_per_year AS
(
SELECT
ede.store_name
,ede.reading_year
,MAX(ede.start_date) AS latest_reading_date_of_yr
FROM
energy_data_ext AS ede
GROUP BY
ede.store_name
,ede.reading_year
)
,store_reading_ranges AS
(
SELECT
avs.store_name
,lryps.current_reading_year
,lyrr.latest_reading_date_of_yr AS current_year_latest_reading_date
,(lryps.current_reading_year - 1) AS prev_reading_year
,(lyrr.latest_reading_date_of_yr - INTERVAL 1 YEAR) AS prev_year_latest_reading_date
FROM
available_stores AS avs
LEFT JOIN
current_reading_yr_per_store AS lryps
ON (lryps.store_name = avs.store_name)
LEFT JOIN
latest_reading_ranges_per_year AS lyrr
ON (lyrr.store_name = avs.store_name)
AND (lyrr.reading_year = lryps.current_reading_year)
)
--at this stage, we should have all the calculations we need to
--establish the range for the latest year, and the range for the year prior to that
,current_year_consumption AS
(
SELECT
avs.store_name
SUM(cyed.reading_kwh) AS latest_year_kwh
FROM
available_stores AS avs
LEFT JOIN
store_reading_ranges AS srs
ON (srs.store_name = avs.store_name)
LEFT JOIN
energy_data_ext AS cyed
ON (cyed.reading_year = srs.current_reading_year)
AND (cyed.reading_date <= srs.current_year_latest_reading_date)
GROUP BY
avs.store_name
)
,prev_year_consumption AS
(
SELECT
avs.store_name
SUM(pyed.reading_kwh) AS prev_year_kwh
FROM
available_stores AS avs
LEFT JOIN
store_reading_ranges AS srs
ON (srs.store_name = avs.store_name)
LEFT JOIN
energy_data_ext AS pyed
ON (pyed.reading_year = srs.prev_reading_year)
AND (pyed.reading_date <= srs.prev_year_latest_reading_date)
GROUP BY
avs.store_name
)
SELECT
avs.store_name
,srs.current_reading_year
,srs.current_year_latest_reading_date
,lyc.latest_year_kwh
,srs.prev_reading_year
,srs.prev_year_latest_reading_date
,pyc.prev_year_kwh
FROM
available_stores AS avs
LEFT JOIN
store_reading_ranges AS srs
ON (srs.store_name = avs.store_name)
LEFT JOIN
current_year_consumption AS lyc
ON (lyc.store_name = avs.store_name)
LEFT JOIN
prev_year_consumption AS pyc
ON (pyc.store_name = avs.store_name)
I have a table that stores the statuses an applications goes through. Some applications go through the same status multiple times. Each time it goes through a status, the time of the status change is recorded.
How can I pull a list of applications based on the first time applications goes through a particular status within a specified date range. Below is what I have tried thus far:
SELECT d1.STATUS,
d1.APPL_ID
FROM APP_STATUS d1
LEFT JOIN APP_STATUS d2 ON d1.APPL_ID = d2.APPL_ID AND d1.STATUS = 'AT_CUSTOMER' AND d2.STATUS = 'AT_CUSTOMER'
WHERE DATE(d1.STATUS_CREATE_DT) >= '2014-10-26'
AND DATE(d1.STATUS_CREATE_DT) <= '2014-11-25'
AND d2.STATUS IS NULL
GROUP BY d1.APPL_ID;
To get the first time a status goes through, try this query:
select a.appl_id, min(status_create_dt) as first_dt
from ap_status
where d.STATUS_CREATE_DT >= '2014-10-26' and
d.STATUS_CREATE_DT < date('2014-11-25') + interval 1 day and
d2.STATUS IS NULL
group by a.appl_id;
I think this does what you need. If you want more columns, then you can join this back to ap_status.
Note that I changed the date logic a bit. The date functions are only on the constant side of the dates. This allows the query to take advantage of an index on STATUS_CREATE_DT, if appropriate.
I have a MySQL query that generates a table for my vehicle tracking 'in' and 'out' times.
The problem is that the 'in' time is not the same as the 'out' time so seconds or minutes are lost in between.
Is there a way to set the 'in' time equal to the 'out time' from the previous row, even if I need to embed my current select inside a new select?
you will see on the image below that the first rows out time is 15:45:14 and the in time for the next row is 15:46:14. so in this case a minute is lost
in reality if the vehicles has left one point, it is immediately on the road to the next point so I can set the in time equal to the out time of the previous row. This way, time is never lost
the sql for my query is:
select vehicle,InTime,OutTime from (select
PreQuery.callingname as vehicle,
PreQuery.geofence,
PreQuery.GroupSeq,
MIN( PreQuery.`updatetime` ) as InTime,
UNIX_TIMESTAMP(MIN( PreQuery.`updatetime`))as InSeconds,
MAX( PreQuery.`updatetime` ) as OutTime,
UNIX_TIMESTAMP(MAX( PreQuery.`updatetime`))as OutSeconds,
TIME_FORMAT(SEC_TO_TIME((UNIX_TIMESTAMP(MAX( PreQuery.`updatetime` )) - UNIX_TIMESTAMP(MIN( PreQuery.`updatetime`)))),'%H:%i:%s') as Duration,
(UNIX_TIMESTAMP(MAX( PreQuery.`updatetime` )) - UNIX_TIMESTAMP(MIN( PreQuery.`updatetime`))) as DurationSeconds
from
( select
v_starting.callingname,
v_starting.geofence,
v_starting.`updatetime`,
#lastGroup := #lastGroup + if( #lastAddress = v_starting.geofence
AND #lastVehicle = v_starting.callingname, 0, 1 ) as GroupSeq,
#lastVehicle := v_starting.callingname as justVarVehicleChange,
#lastAddress := v_starting.geofence as justVarAddressChange
from
v_starting,
( select #lastVehicle := '',
#lastAddress := '',
#lastGroup := 0 ) SQLVars
order by
v_starting.`updatetime` ) PreQuery
Group By
PreQuery.callingname,
PreQuery.geofence,
PreQuery.GroupSeq) parent
where (InTime> DATE_SUB('2013-03-23 15:00', INTERVAL 24 HOUR) or OutTime> '2013-03-23 15:00' ) and vehicle='TT08' order by InTime asc
The MySQL syntax is in depth so quite large but could be done on a much simpler query as well. like
select vehicle, intime,outtime from vehicletimes
My desired result is something like:
select vehicle, intime(outtime of row above),outtime from vehicletimes
The first rows in time can be as is and the last rows outtime can be as is. I just need to account for every second between the smallest in time and the largest out time.
Any help appreciated as always.
Thanks in advance
I think this will give you the latest in-time prior to each current out-time, for your existing records:
select
vt.vehicle, max(qGetMaxOut.outtime) as intime , vt.outtime
from
vehicle_times vt
inner join
(
select vehicle, outtime
from vehicle_times
) qGetMaxOut
on qGetMaxOut.vehicle = vt.vehicle
and qGetMaxOut.outtime <= vt.intime
group by
vt.vehicle, vt.outtime
The above query will also help you if you want to insert a new record, but need to find the previous in-time for a particular time (ie if you need to insert a new record who's in/out times are prior to the latest time - eg inserting a record that was somehow previously missed and where newer time entries have been added since). If you need this scenario, let me know and I'll elaborate if you can't work it out from the above.
The join basically joins the table "back on itself" to provide another "copy", but limits the results in the "copy" to only those rows for the current vehicle in the main table, and excludes those rows from the copy where the vehicle's out-time is more recent than the current in-time from the main table. This way you can do a MAX() over the copy, to find what the previous out time was.
I don't know your specific requirements, but I would recommend storing the most accurate information you can. So if "sythensising" a value is just for cosmetic purposes on a few reports, I would leave the data alone, and tidy up the report, rather than loosing data that might come in handy down the track. eg what happens if in the future, you suddenly have a requirement to tell your boss "how long are our vehicles 'in' and sitting idle for?"
But if you do just want to insert a new record with the actual out-time ignored, and replaced by the in-time from the most recent record, then this following query will find that value for you:
select
vt.vehicle, max(vt.outtime) as intime
from
vehicle_times vt
group by
vt.vehicle
Have I missed your requirement?
I have a Mysql Table that holds dates that are booked (for certain holiday properties).
Example...
Table "listing_availability"
Rows...
availability_date (this shows the date format 2013-04-20 etc)
availability_bookable (This can be yes/no. "Yes" = the booking changeover day and it is "available". "No" means the property is booked for those dates)
All the other dates in the year (apart from the ones with "No") are available to be booked. These dates are not in the database, only the booked dates.
My question is...
I have to make a SQL Statement that first calls the Get Date Function (not sure if this is correct terminology)
Then removes the dates from "availability_date" WHERE "availability_bookable" = "No"
This will give me the dates that are available for bookings, for the year, for a property.
Can anyone help?
Regards M
Seems like you've almost written the query.
SELECT availability_date FROM listing_availability
WHERE availability_bookable <> 'NO'
AND availability_date >= CURDATE()
AND YEAR(CURDATE()) = YEAR(availability_date)
I think I understand, and you'll obviously confirm. Your "availability_booking" has some records in it, but not every single day of the year, only those that may have had something, and not all are committed, some could have yes, some no.
So, you want to simulate All dates within a given date range... Say April 1 - July 1 as someone is looking to book a party within that time period. Instead of pre-filling your production table, you can't say that April 27th is open and available... since no such record exists.
To SIMULATE a calendar of days for a date range, you can do it using MySQL variables and join to "any" table in your database provided it has enough records to SIMULATE the date range you want...
select
#myDate := DATE_ADD( #myDate, INTERVAL 1 DAY ) as DatesForAvailabilityCheck
from
( select #myDate := '2013-03-31' ) as SQLVars,
AnyTableThatHasEnoughRows
limit
120;
This will just give you a list of dates starting with April 1, 2013 (the original #myDate is 1 day before the start date since the field selection adds 1 day to it to get to April 1, then continues... for a limit of 120 days (or whatever you are looking for range based -- 30days, 60, 90, 22, whatever). The "AnyTableThatHasEnoughRows" could actually be your "availability_booking" table, but we are just using it as a table with rows, no join or where condition, just enough to get ... 120 records.
Now, we can use this to join to whatever table you want and apply your condition. You just created a full calendar of days to compare against. Your final query may be different, but this should get it most of the way for you.
select
JustDates.DatesForAvailabilityCheck,
from
( select
#myDate := DATE_ADD( #myDate, INTERVAL 1 DAY ) as DatesForAvailabilityCheck
from
( select #myDate := '2013-03-31' ) as SQLVars,
listing_availability
limit
120 ) JustDates
LEFT JOIN availability_bookable
on JustDates.DatesForAvailabilityCheck = availability_bookable.availability_date
where
availability_bookable.availability_date IS NULL
OR availability_bookable.availability_bookable = "Yes"
So the above uses the sample calendar and looks to the availability. If no such matching date exists (via the IS NULL), then you want it meaning there is no conflict. However, if there IS a record in the table, you only want those where YES, you CAN book it, the entry on file might not be committed and CAN be in your result query of available dates.
Say I have a table with two columns
TimeStamp of type TIMESTAMP
A of type FLOAT
This table is created and updated by an external application, so inserts and updates are outside of my control. The table design can't be altered in any way.
What I need to do is select each entry closest to and before 10AM for each day during the entire past month.
Thanks in advance.
The inner pre-query should get on a per year/month basis, prior to the month you are currently in. This is forced by a SQLVariable created by the formatted 'YYYY-MM-01' date, such as today... 2012-03-19, keep just year/month but force 01. This also implies timestamp of 12:00:00 am (midnight). The NEXT # variable is to determine the first of the month PRIOR to the one just computed... thus 2012-02-01. That builds the variables for the WHERE clause queried against your table of timestamp/float values.
Now, you can get the maximum time, grouped by just the common date portion of the timestamp, but retaining the full actual date AND time of the entry where the HOUR() of the entry is before 10am...
From that, re-join back to the original table where the FINAL "LastPerDay" time matches the per-day basis. Now, you MAY get multiple entries if the actual last timestamp entry for the same day actually HAS multiple exact time entries to the granularity of hh:mm:ss (or whatever precision)
select
PreQuery.JustTheDate,
YT2.FloatColumnName
from
( select
Date_Format( YT.TimeStampColumn, '%Y-%m-%d' ) JustTheDate,
max( YT.TimeStampColumn ) as LastPerDay
from
( select #FirstOfThisMonth := Date_Format( '%Y-%m-01' ),
#FirstOfPriorMonth := Date_Sub( #FirstOfThisMonth, interval 1 month ) ) sqlvars,
YourTable YT
where
YT.TimeStampColumn >= #FirstOfPriorMonth
AND YT.TimeStampColumn < #FirstOfThisMonth
AND Hour( YT.TimeStampColumn ) < 10
group by
`JustTheDate`
order by
`JustTheDate` DESC ) PreQuery
JOIN YourTable YT2
ON PreQuery.LastPerDay = YT2.TimeStampColumn