Most efficient way to sum overlapping datetimes in MySQL - mysql

I'm in the process of evaluating the proposed solutions on SO related to the sum of overlapping datetimes in MySQL. I wasn't able to find out a silver-bullet solution, so would like to know if any classic/industrial-grade algorithmic procedure is available or if a custom-made needs to be developed.
Total should be 8 hours (4+4).
Proposed solution through MySQL
function final_balance($teacher_id, $aa, $teaching_id=0) {
$dbo = $this->Attendance->getDataSource();
$years=$this->Attendance->Student->Year->find('list', array('fields' => array('anno', 'data_from')));
$filteraa='attendances.start>="'.$years[$aa].'"';
$this->query('SET #interval_id = 0');
$this->query('SET #interval_end = \'1970-01-01 00:00:00\'');
$sql='SELECT
MIN(start) as start,
MAX(end) as end
FROM (
SELECT
#interval_id := IF(attendances.start > #interval_end, #interval_id + 1, #interval_id) AS interval_id,
#interval_end := IF(attendances.start < #interval_end, GREATEST(#interval_end, attendances.end), attendances.end) AS interval_end,
attendances.start,
attendances.end
FROM attendances
INNER JOIN attendance_sheets ON (
attendance_sheet_id = attendance_sheets.id AND
attendance_sheets.teacher_id='.$teacher_id.' AND '.$filteraa.' AND
attendance_sheet_status_id = 2 AND
attendance_status_id!=3'.
($teaching_id?' AND attendances.teaching_id IN ('.$teaching_id.')':'').'
)
ORDER BY attendances.start,attendances.end
) intervals GROUP BY interval_id';
// final query to sum in the temp table
$finalStatement =array(
'table' => $dbo->expression('('.$sql.')')->value,
'alias' => 'Attendance',
'fields' => array(
'DATE_FORMAT(start, \'%d/%m/%Y\') as data',
'DATE_FORMAT(start, \'%m-%Y\') as datamese',
'DATE(start) as datasql',
$teacher_id.' AS teacher_id',
'DAY(start) as giorno',
'MONTH(start) as mese',
'YEAR(start) as anno',
'SEC_TO_TIME(SUM((TIME_TO_SEC(end) - TIME_TO_SEC(start)))) as ore',
),
'conditions' => array(),
'limit' => null,
'group' => array('CONCAT(YEAR(start),MONTH(start))', 'DATE(start) WITH ROLLUP'),
'order' => null
);
$finalQuery= $dbo->buildStatement($finalStatement, $this->Attendance);
return $this->Attendance->query($dbo->expression($finalQuery)->value);
}
References
Sum amount of overlapping datetime ranges in MySQL
performs a different task
MySQL: sum time ranges exluding overlapping ones
and
MySQL: sum datetimes without counting overlapping periods twice
both seems to me like not considering all the cases
GeeksForCode: Merge Overlapping Intervals

Depending on the circumstances, the following might be useful and efficient.
Create another table that has one row per hour. Inner join that table with your table while selecting only the new column and dedupe the rows.
You can keep increasing the resolution (eg. to minutes or seconds), but that might make your code run slow.

Related

SQL query needed for a complex structure

I have a tricky SQL query that needs to be built to get the highest priority rule based on customer session and geo IP data.
I attached the following tables: rule, rule_attribute, rule_attribute_value.
rule - table where all rules are stored
Click here to see a screenshot of the 'rule' table
rule_attribute - table where all rule attributes are stored
Click here to see a screenshot of the 'rule_attribute' table
rule_attribute_value - table where all rule attribute values are stored
Click here to see a screenshot of the 'rule_attribute_value' table
When the customer logs in, I have access to all those attributes (customer_id, customer_group_id, country_id, subdivision_one_id, subdivision_two_id). Only customer_id and customer_group_id will always have values. The others are optional, but there is a dependency between them. We can't have subdivisions without selecting first a country. We can have a second subdivision without selecting a country and then the first subdivision.
What I would like to get is the highest priority rule that matches the session data in the most optimized way. I have a solution that involves some coding, but I want to see if it's possible directly through SQL.
Here are some examples of session data arrays:
Array
(
[customer_id] => 2
[customer_group_id] => 1
[current_store_id] => 0
[country_id] => 15
[subdivision_one_id] => 224
[subdivision_two_id] =>
)
Array
(
[customer_id] => 2
[customer_group_id] => 1
[current_store_id] => 0
[country_id] => 15
[subdivision_one_id] =>
[subdivision_two_id] =>
)
Array
(
[customer_id] => 3
[customer_group_id] => 2
[current_store_id] => 0
[country_id] =>
[subdivision_one_id] =>
[subdivision_two_id] =>
)
Without a better understanding of the rules and data this is the best I can come up with. It is based on your first array example -
SELECT `r`.*
FROM `rule_attribute_value` `rav`
INNER JOIN `rule` `r`
ON `rav`.`rule_id` = `r`.`rule_id`
INNER JOIN `rule_attribute` `ra`
ON `rav`.`attribute_id` = `ra`.`attribute_id`
WHERE
(`rav`.`store_id` = 0 AND `ra`.`attribute_code` = 'customer' AND `rav`.`value` = 2) OR
(`rav`.`store_id` = 0 AND `ra`.`attribute_code` = 'customer_group' AND `rav`.`value` = 1) OR
(`rav`.`store_id` = 0 AND `ra`.`attribute_code` = 'country' AND `rav`.`value` = 15) OR
(`rav`.`store_id` = 0 AND `ra`.`attribute_code` = 'subdivision_one' AND `rav`.`value` = 224)
GROUP BY `r`.`rule_id`
HAVING COUNT(DISTINCT `rav`.`attribute_id`) = 4 /* 4 IS THE NUMBER OF ATTRIBUTES BEING QUERIED */
ORDER BY `r`.`position` ASC
LIMIT 1;

Reducing queries Yii2

I`m really stuck on this one.
I have a table with contracts, and one with items.
What Im trying to achieve is to see the total quantity of an item on contracts in a date range for every day.
For example, item1 appears on 3 contracts on Monday and the total quantity is 30.
Currently, Im doing a while loop for each day in the selected date range and using this query
$itemQty = ContItems::find()
->where(['ITEMNO' => $itemNo])
->andWhere(['<=' , 'HIREDATE' , $today])
->andWhere(['>=' , 'ESTRETD' , $today])
->andWhere(['or',
['STATUS' => 0],
['STATUS' => 1]
])
->SUM('QTY');
It is working fine, but I want to check for every item I have for each day in the selected date range, and the number of queries goes up to 50k+ if I do that.
What I tried to do is selected every contract in a date range and stored in an array
Every contract has a start and end date and I want to select every contract where the start or end date is in the date range I specified.
$ContItemsArray = ContItems::find()
->where(['or',
['<=' , 'HIREDATE' , $dateFrom],
['<=' , 'HIREDATE' , $dateTo],
['>=' , 'HIREDATE' , $dateFrom],
['>=' , 'ESTRETD' , $dateTo],
['>=' , 'ESTRETD' , $dateFrom]
])
->andWhere(['or',
['STATUS' => 0],
['STATUS' => 1]
])
->asArray()
->all();
And use this code to count the quantities
$theQty = 0;
foreach ($ContItemsArray as $subArray)
{
if ($subArray['ITEMNO'] == $itemNo && ($subArray['HIREDATE'] <= $today && $subArray['ESTRETD'] >= $today) && ($subArray['STATUS'] <= 1))
{
$theQty += $subArray['QTY'];
}
else
{
$theQty += 0;
}
}
But it is returning incorrect numbers, contracts are missing for some reason and I can`t figure it out why.
I tried to change around the query for the $ContItemsArray but with no luck.
I hope I was clear enough with the question.
I would appreciate any help.
Edit:
ContItems table has all the contracts with the item number 'ITEMNO' start date 'HIREDATE' end date 'ESTRETD' and the quantity 'QTY' fields.
And I have an array with all the item numbers what I want to check.
The expected result would be, if the day I`m checking is between the contract start date and end date, sum the qty of the items.
I solved the problem, had to use date('Y-m-d') and strtotime on the dates coming from the table (HIREDATE, ESTRETD) and now it is working fine

CakePHP 3 - finder not loading correct data

I have a table called Bookings, with the following attributes:
id
artist_id - foreign key
status
amount
created
modified
It is associated with tables Artists, Payments and Sessions.
In the View, I have used the jQuery plugin DataTables to display Bookings which meet the following conditions:
Their status is equal to 'confirmed'
Associated table Session's attribute date_end must be greater than the current date.
However, the second condition does not seem to work in that it returns nothing when data entries which do meet that condition exist.
In my controller, the corresponding find for this particular table:
$bookingsConfirmed = $this->Bookings->find('all',[
'contain' => ['Sessions', 'Sessions.Studios', 'Sessions.Engineers', 'Artists'],
'conditions'=>['status' => 'confirmed', 'Sessions.date_end >=' => DATE(Time::now())],
'order'=>['Bookings.created'=>'ASC']
]);
I've attempted the following changes to the condition, all of which showed no notable difference:
date_end >=' => DATE(Time::now())
'Sessions.date_end >=' => Time::now()
'date(Sessions.date_end) >=' => DATE(Time::now())
'date(Sessions.date_end) >=' => date(DATE(Time::now()))
If I switch around the >= sign to a <, all my booking data appear, despite not all of them meeting that condition.
In the SQL log, this is the output for this particular table:
SELECT
Bookings.id AS `Bookings__id`,
Bookings.artist_id AS `Bookings__artist_id`,
Bookings.status AS `Bookings__status`,
Bookings.amount AS `Bookings__amount`,
Bookings.created AS `Bookings__created`,
Bookings.modified AS `Bookings__modified`,
Sessions.id AS `Sessions__id`,
Sessions.booking_id AS `Sessions__booking_id`,
Sessions.studio_id AS `Sessions__studio_id`,
Sessions.engineer_id AS `Sessions__engineer_id`,
Sessions.guestengineer_id AS `Sessions__guestengineer_id`,
Sessions.date_start AS `Sessions__date_start`,
Sessions.date_end AS `Sessions__date_end`,
Sessions.starttime AS `Sessions__starttime`,
Sessions.hours AS `Sessions__hours`,
Sessions.session_genre AS `Sessions__session_genre`,
Sessions.no_people AS `Sessions__no_people`,
Sessions.studio_usage AS `Sessions__studio_usage`,
Sessions.otherpeople_req AS `Sessions__otherpeople_req`,
Sessions.special_req AS `Sessions__special_req`,
Sessions.created AS `Sessions__created`,
Sessions.modified AS `Sessions__modified`,
Studios.id AS `Studios__id`,
Studios.name AS `Studios__name`,
Studios.description AS `Studios__description`,
Studios.created AS `Studios__created`,
Studios.modified AS `Studios__modified`,
Engineers.id AS `Engineers__id`,
Engineers.user_id AS `Engineers__user_id`,
Engineers.eng_firstname AS `Engineers__eng_firstname`,
Engineers.eng_lastname AS `Engineers__eng_lastname`,
Engineers.eng_email AS `Engineers__eng_email`,
Engineers.eng_phoneno AS `Engineers__eng_phoneno`,
Engineers.eng_status AS `Engineers__eng_status`,
Engineers.rate AS `Engineers__rate`,
Engineers.created AS `Engineers__created`,
Engineers.modified AS `Engineers__modified`,
Artists.id AS `Artists__id`,
Artists.name AS `Artists__name`,
Artists.cp_id AS `Artists__cp_id`,
Artists.user_id AS `Artists__user_id`,
Artists.genre AS `Artists__genre`,
Artists.created AS `Artists__created`,
Artists.modified AS `Artists__modified`
FROM
bookings Bookings
LEFT JOIN sessions Sessions ON Bookings.id = (Sessions.booking_id)
INNER JOIN studios Studios ON Studios.id = (Sessions.studio_id)
LEFT JOIN engineers Engineers ON Engineers.id = (Sessions.engineer_id)
INNER JOIN artists Artists ON Artists.id = (Bookings.artist_id)
WHERE
(
status = 'confirmed'
AND Sessions.date_end >= '20/2/17, 4:47 p02'
)
ORDER BY
Bookings.created ASC
As of today's date, there should be 3 entries that show. I did a print of Time::now and got the date:
Cake\I18n\Time Object
(
[time] => 2017-02-20T16:47:11+11:00
[timezone] =>
[fixedNowTime] =>
)
The odd thing is that this was working fine last week, and the submission forms still work. In MySQL for example, my latest entry that I inputted today shows Sessions.date_end filled in, in a YYYY-MM-DD format like all the entries that proceeded it.
Try this line instead
`date_end >=' => date('Y-m-d H:i:s', Time::now()->getTimestamp())`
Note : use the exact word case for functions, date instead of DATE

How to use the table columns instead of variables in QueryExpression::addCase()

In CakePHPs new ORM, you can use the QueryBuilder to build (in theory) any query.
I want to select the value of one of two columns, depending on another value. In a regular query, that can be done as follows:
SELECT IF(from_id = 1, to_id, from_id) AS other_id FROM messages;
I am trying to archive the same query using the QueryBuilder and QueryExpression::addCase()
$messagesQuery = $this->Messages->find('all');
$messagesQuery->select([
'other_id' => $messagesQuery->newExpr()->addCase(
$messagesQuery->newExpr()->add(['from_id' => $this->authUser->id]),
['to_id', 'from_id'],
['integer', 'integer']
)
]);
This does not work, as the passed values are not integers, but rather table columns containing integers.
Through trial and error (using the method add() again), I got the following:
$messagesQuery = $this->Messages->find('all');
$messagesQuery->select([
'other_id' => $messagesQuery->newExpr()->addCase(
$messagesQuery->newExpr()->add(['from_id' => $this->authUser->id]),
[
$messagesQuery->newExpr()->add(['to_id']),
$messagesQuery->newExpr()->add(['from_id'])
],
['integer', 'integer']
)
]);
This results in the following query:
SELECT (CASE WHEN from_id = 1 THEN to_id END) AS `other_id` FROM messages Messages
Now, the ELSE part is missing, although the CakePHP book states:
Any time there are fewer case conditions than values, addCase will automatically produce an if .. then .. else statement.
The examples in the CakePHP book are not very helpful in this case, as they only use static integers or strings as values, for example:
#SELECT SUM(CASE published = 'Y' THEN 1 ELSE 0) AS number_published, SUM(CASE published = 'N' THEN 1 ELSE 0) AS number_unpublished FROM articles GROUP BY published
$query = $articles->find();
$publishedCase = $query->newExpr()->addCase($query->newExpr()->add(['published' => 'Y']), 1, 'integer');
$notPublishedCase = $query->newExpr()->addCase($query->newExpr()->add(['published' => 'N']), 1, 'integer');
$query->select([
'number_published' => $query->func()->sum($publishedCase),
'number_unpublished' => $query->func()->sum($unpublishedCase)
])
->group('published');
Is there a way to get the method addCase to use the two table columns as values instead of just static values?
As it turns out, I was just one logical step short of the solution in my previous edit.
As the CakePHP book correctly states:
Any time there are fewer case conditions than values, addCase will automatically produce an if .. then .. else statement.
For that to work though, both the conditions and values have to be an array, even if there is only one condition. (This the CakePHP book does not state.)
This code:
$messagesQuery = $this->Messages->find('all');
$messagesQuery->select([
'other_id' => $messagesQuery->newExpr()->addCase(
[
$messagesQuery->newExpr()->add(['from_id' => $this->authUser->id])
],
[
$messagesQuery->newExpr()->add(['to_id']),
$messagesQuery->newExpr()->add(['from_id'])
],
['integer', 'integer']
)
]);
results in this query:
SELECT (CASE WHEN from_id = 1 THEN to_id ELSE from_id END) AS `other_id` FROM messages Messages
Eureka

How to create a select where count is not zero in MySQL

Here's what I'm trying to do. I'm trying to select from a forum views table all of the user_ids where there are 5 or more records. That's fairly easy (this is Zend):
$objCountSelect = $db->select()
->from(array('v' =>'tbl_forum_views'), 'COUNT(*) AS count')
->where('u.id = v.user_id')
->having('COUNT(user_id) >= ?', 5)
;
But I need to somehow connect this to my users table. I don't want to return a result if the count is greater than 5. I tried this:
$objSelect = $db->select()
->from(array('u' => 'tbl_users'), array(
'id as u_id',
'count' => new Zend_Db_Expr('(' . $objCountSelect . ')'),
))
;
But that returns a record for every user, leaving blank the count if it's less than or equal to 5. How do I exclude the rows where the count is less than or equal to 5?
I figured it out, but wanted to post the answer in case someone else had the same issue. I added:
->having('count > 0')
to the second select and now it works.