mysql split a string in a where clause - mysql
I have an event system and for my repeat events I am using a cron like system.
Repeat Event:
+----+----------+--------------+
| id | event_id | repeat_value |
+----+----------+--------------+
| 1 | 11 | *_*_* |
| 2 | 12 | *_*_2 |
| 3 | 13 | *_*_4/2 |
| 4 | 14 | 23_*_* |
| 5 | 15 | 30_05_* |
+----+----------+--------------+
NOTE: The cron value is day_month_day of week
Event:
+----+------------------------+---------------------+---------------------+
| id | name | start_date_time | end_date_time |
+----+------------------------+---------------------+---------------------+
| 11 | Repeat daily | 2014-04-30 12:00:00 | 2014-04-30 12:15:00 |
| 12 | Repeat weekly | 2014-05-06 12:00:00 | 2014-05-06 13:00:00 |
| 13 | Repeat every two weeks | 2014-05-08 12:45:00 | 2014-05-08 13:45:00 |
| 14 | Repeat monthly | 2014-05-23 15:15:00 | 2014-05-23 16:00:00 |
| 15 | Repeat yearly | 2014-05-30 07:30:00 | 2014-05-30 10:15:00 |
+----+------------------------+---------------------+---------------------+
Anyway I have a query to select the events:
SELECT *
FROM RepeatEvent
JOIN `Event`
ON `Event`.`id` = `RepeatEvent`.`event_id`
That produces:
+----+----------+--------------+----+------------------------+---------------------+---------------------+
| id | event_id | repeat_value | id | name | start_date_time | end_date_time |
+----+----------+--------------+----+------------------------+---------------------+---------------------+
| 1 | 11 | *_*_* | 11 | Repeat daily | 2014-04-30 12:00:00 | 2014-04-30 12:15:00 |
| 2 | 12 | *_*_2 | 12 | Repeat weekly | 2014-05-06 12:00:00 | 2014-05-06 13:00:00 |
| 3 | 13 | *_*_4/2 | 13 | Repeat every two weeks | 2014-05-08 12:45:00 | 2014-05-08 13:45:00 |
| 4 | 14 | 23_*_* | 14 | Repeat monthly | 2014-05-23 15:15:00 | 2014-05-23 16:00:00 |
| 5 | 15 | 30_05_* | 15 | Repeat yearly | 2014-05-30 07:30:00 | 2014-05-30 10:15:00 |
+----+----------+--------------+----+------------------------+---------------------+---------------------+
However, I want to select events within a month. I will only have certain conditions: daily, weekly, every two weeks, month and yearly.
I want to put in my where clause a way to divide the string of the repeat value and if it fits any of the following conditions to show it as a result (repeatEvent is row that is being interrogated, search is the date being looked for):
array(3) = string_divide(repeat_value, '_')
daily = array(0)
monthy = array(1)
dayOfWeek = array(2)
if(daily == '*' && month == '*' && dayOfWeek == '*') //returns all the daily events as they will happen
return repeatEvent
if(if(daily == '*' && month == '*' && dayOfWeek == search.dayOfWeek) //returns all the events on specific day
return repeatEvent
if(daily == search.date && month == '*' && dayOfWeek == '*') //returns all the daily events as they will happen
return repeatEvent
if (contains(dayOfWeek, '/'))
array(2) = string_divide(dayOfWeek,'/')
specificDayOfWeek = array(0);
if(specificDayOfWeek == repeatEvent.start_date.dayNumber)
if(timestampOf(search.timestamp)-timestampOf(repeatEvent.start_date)/604800 == (0 OR EVEN)
return repeatEvent
if(daily == search.date && month == search.month && dayOfWeek == '*') //returns a single yearly event (shouldn't often crop up)
return repeatEvent
//everything else is either an unknown format of repeat_value or not an event on this day
To summarise I want to run a query in which the repeat value is split in the where clause and I can interrogate the split items. I have looked at cursors but the internet seems to advise against them.
I could process the results of selecting all the repeat events in PHP, however, I imagine this being very slow.
Here is what I would like to see if looking at the month of April:
+----------+--------------+----+------------------------+---------------------+---------------------+
| event_id | repeat_value | id | name | start_date_time | end_date_time |
+----------+--------------+----+------------------------+---------------------+---------------------+
| 11 | *_*_* | 11 | Repeat daily | 2014-04-30 12:00:00 | 2014-04-30 12:15:00 |
+----------+--------------+----+------------------------+---------------------+---------------------+
Here is what I would like to see if looking at the month of May
+----------+--------------+----+------------------------+---------------------+---------------------+
| event_id | repeat_value | id | name | start_date_time | end_date_time |
+----------+--------------+----+------------------------+---------------------+---------------------+
| 11 | *_*_* | 11 | Repeat daily | 2014-04-30 12:00:00 | 2014-04-30 12:15:00 |
| 12 | *_*_2 | 12 | Repeat weekly | 2014-05-06 12:00:00 | 2014-05-06 13:00:00 |
| 13 | *_*_4/2 | 13 | Repeat every two weeks | 2014-05-08 12:45:00 | 2014-05-08 13:45:00 |
| 14 | 23_*_* | 14 | Repeat monthly | 2014-05-23 15:15:00 | 2014-05-23 16:00:00 |
| 15 | 30_05_* | 15 | Repeat yearly | 2014-05-30 07:30:00 | 2014-05-30 10:15:00 |
+----------+--------------+----+------------------------+---------------------+---------------------+
Here is what I would like to see if looking at the month of June
+----------+--------------+----+------------------------+---------------------+---------------------+
| event_id | repeat_value | id | name | start_date_time | end_date_time |
+----------+--------------+----+------------------------+---------------------+---------------------+
| 11 | *_*_* | 11 | Repeat daily | 2014-04-30 12:00:00 | 2014-04-30 12:15:00 |
| 12 | *_*_2 | 12 | Repeat weekly | 2014-05-06 12:00:00 | 2014-05-06 13:00:00 |
| 13 | *_*_4/2 | 13 | Repeat every two weeks | 2014-05-08 12:45:00 | 2014-05-08 13:45:00 |
| 14 | 23_*_* | 14 | Repeat monthly | 2014-05-23 15:15:00 | 2014-05-23 16:00:00 |
+----------+--------------+----+------------------------+---------------------+---------------------+
You could put a bandaid on this, but no one would be doing you any favors to tell you that that is the answer.
If your MySQL database can be changed I would strongly advise you to split your current column with underscores day_month_day of year to three separate columns, day, month, and day_of_year. I would also advise you to change your format to be INT rather than VARCHAR. This will make it faster and MUCH easier to search and parse, because it is designed in a way that doesn't need to be translated into computer language through complicated programs... It is most of the way there already.
Here's why:
Reason 1: Your Table is not Optimized
Your table is not optimized and will be slowed regardless of what you choose to do at this stage. SQL is not built to have multiple values in one column. The entire point of an SQL database is to split values into different columns and rows.
The advantage to normalizing this table is that it will be far quicker to search it, and you will be able to build queries in MySQL. Take a look at Normalization. It is a complicated concept, but once you get it you will avoid creating messy and complicated programs.
Reason 2: Your Table could be tweaked slightly to harness the power of computer date/time functions.
Computers follow time based on Unix Epoch Time. It counts seconds and is always running in your computer. In fact, computers have been counting this since, as the name implies, the first Unix computer was ever switched on. Further, each computer and computer based program/system, has built in, quick date and time functions. MySQL is no different.
I would also recommend also storing all of these as integers. repeat_doy (day of year) can easily be a smallint or at least a standard int, and instead of putting a month and day, you can put the actual 1-365 day of the year. You can use DAY_OF_YEAR(NOW()) to input this into MySQL. To pull it back out as a date you can use MAKEDATE(YEAR(NOW),repeat_doy). Instead of an asterisk to signify all, you can either use 0's or NULL.
With a cron like system you probably will not need to do that sort of calculation anyway.
Instead, it will probably be easier to just measure the day of year elsewhere (every computer and language can do this. In Unix it is just date "%j").
Solution
Split your one repeat_value into three separate values and turn them all into integers based on UNIX time values. Day is 1-7 (or 0-6 for Sunday to Saturday), Month is 1-12, and day of year is 1-365 (remember, we are not including 366 because we are basing our year on an arbitrary non-leap year).
If you want to pull information in your SELECT query in your original format, it is much easier to use concat to merge the three columns than it is to try to search and split on one column. You can also easily harness built in MySQL functions to quickly turn what you pull into real, current, days, without a bunch of effort on your part.
To implement it in your SQL database:
+----+----------+--------------+--------------+------------+
| id | event_id | repeat_day | repeat_month | repeat_doy |
+----+----------+--------------+--------------+------------+
| 1 | 11 | * | * | * |
| 2 | 12 | * | * | 2 |
| 3 | 13 | * | * | 4/2 |
| 4 | 14 | 23 | * | * |
| 5 | 15 | 30 | 5 | * |
+----+----------+--------------+--------------+------------+
Now you should be able to build one query to get all of this data together regardless of how complicated your query. By normalizing your table, you will be able to fully harness the power of relational databases, without the headaches and hacks.
Edit
Hugo Delsing made a great point in the comments below. In my initial example I provided a fix to leap years for day_of_year in which I chose to ignore Feb 29. A much better solution removes the need for a fix. Split day_of_year to month and day with a compound index. He also has a suggestion about weeks and number of weeks, but I will just recommend you read it for more details.
Try to write where condition using this:
substring_index(repeat_value,'_', 1)
instead of daily
substring_index(substring_index(repeat_value,'_', -2), '_', 1)
instead of monthly
and
substring_index(substring_index(repeat_value,'_', -1), '_', 1)
instead of dayOfWeek
I think you are overthinking the problem if you only want the events per month and not per day. Assuming that you always correctly fill the repeat_value, the query is very basic.
Basically all event occur every month where the repeat_value is either LIKE '%_*_%' or LIKE '%_{month}_%'.
Since you mentions PHP I'm assuming you are building the query in PHP and thus I used the same.
<?php
function buildQuery($searchDate) {
//you could/should do some more checking if the date is valid if the user provides the string
$searchDate = empty($searchDate) ? date("Y-m-d") : $searchDate;
$splitDate = explode('-', $searchDate);
$month = $splitDate[1];
//Select everything that started after the searchdate
//the \_ is because else the _ would match any char.
$query = 'SELECT *
FROM RepeatEvent
JOIN `Event`
ON `Event`.`id` = `RepeatEvent`.`event_id`
WHERE `Event`.`start_date_time` < \''.$searchDate.'\'
AND
(
`RepeatEvent`.`repeat_value` LIKE \'%\_'.$month.'\_%\'
OR `RepeatEvent`.`repeat_value` LIKE \'%\_*\_%\'
)
';
return $query;
}
//show querys for all months on current day/year
for ($month = 1; $month<=12; $month++) {
echo buildQuery(date('Y-'.$month.'-d')) . '<hr>';
}
?>
Now if the repeat_value could be wrong, you could add a simple regex check to make sure the value is always like *_*_* or *_*_*/*
You can use basic regular expressions in MySQL:
http://dev.mysql.com/doc/refman/5.0/en/pattern-matching.html
For a monthly event in May (first day) you can use a pattern like this (not tested):
[0-9\*]+\_[5\*]\_1
You can generate this pattern via PHP
Related
Mysql - Get season from current month
I have the following table of seasons: | id | name | start_month | end_month | ------------------------------------------ | 101 | Summer | 12 | 2 | | 102 | Winter | 6 | 8 | | 103 | Spring | 9 | 11 | | 104 | Fall | 3 | 5 | I need to get the season by month. Say current month is 2 (February), I want Summer to be the output. I can get other seasons to work by simply having the where condition start_month >= 4 and end_month <= 4. But this won't work with Summer since the season crosses into next year. What do I have to do to handle the case of Summer? One solution I thought was to use dates instead of month number like 1980-12-01 and use between function but it gets a bit complicated for the user end. It'd be great if it could work with just month numbers.
You could do: (month(d) between start_month and end_month) or (start_month>end_month and (month(d)>=start_month or month(d)<=end_month)) See db-fiddle
SQL - select x entries within a timespan
I'm creating a database (in MySQL) with a table of measurements. For each measurement I want to store the DateTime it came in. For showing plots within an app for different intervals (measurements of the day/week/month/year) I want sample the data points I have, so I can return e. g. 30 data points for the whole year as well as for the day/hour. This is the same as done with stock price graphs: stock price plot for 1 day vs stock price plot for 1 month As you can see, the amount of data points is the same in both pictures. So how can I select x entries within a timespan in MySQL via SQL? My data looks like this: +====+====================+=============+==========+ | id | datetime | temperature | humidity | +====+====================+=============+==========+ | 1 | 1-15-2016 00:30:00 | 20 | 40 | +----+--------------------+-------------+----------+ | 2 | 1-15-2016 00:35:00 | 19 | 41 | +----+--------------------+-------------+----------+ | 3 | 1-15-2016 00:40:00 | 20 | 40 | +----+--------------------+-------------+----------+ | 4 | 1-15-2016 00:45:00 | 20 | 42 | +----+--------------------+-------------+----------+ | 5 | 1-15-2016 00:50:00 | 21 | 42 | +----+--------------------+-------------+----------+ | 6 | 1-15-2016 00:55:00 | 20 | 43 | +----+--------------------+-------------+----------+ | 7 | 1-15-2016 01:00:00 | 21 | 43 | +====+====================+=============+==========+ Let's say, I always want two data points (in reality a lot more). So for the last half hour I want the database to return data point 1 and 4, for the last ten minutes I want it to return 6 and 7. Thanks for helping! PS: I'm sorry for any errors in my English
OK, assuming a very simple systematic approach, you can get the first and last entry for any defined period: select * from table where mydatetime = (select max(mydatetime) from table where mydatetime between '2017-03-01' and '2017-03-15' ) OR mydatetime = (select min(mydatetime) from table where mydatetime between '2017-03-01' and '2017-03-15' )
I believe your answer can be found at the following location: https://stackoverflow.com/a/1891796/7176046 If you are looking to filter out any items not within your date/time your query would use: Select * from table where Date/Time is (What you want to sort by)
mysql 5.5.50 Calculate one day multiple events to one
I have a problem with my thinking and I must ask someone to help. I create simply working time tracking system and I can manage all other things with that code, but I cannot now figure out how I can calculate following data. Fist, I have couple tables and one store four things. index | who | timestamp | what 1 | 1 | 2016-09-21 08:00:00 | Work 2 | 2 | 2016-09-21 08:01:00 | Work 3 | 1 | 2016-09-21 10:00:00 | Bin 4 | 2 | 2016-09-21 10:00:00 | Bin 5 | 1 | 2016-09-21 10:15:00 | Bout 6 | 2 | 2016-09-21 10:17:00 | Bout 7 | 2 | 2016-09-21 13:00:00 | Bin 8 | 1 | 2016-09-21 13:00:00 | Bin 9 | 1 | 2016-09-21 13:30:00 | Bout 10 | 2 | 2016-09-21 13:30:00 | Bout 11 | 2 | 2016-09-21 15:58:00 | Home 12 | 1 | 2016-09-21 16:05:00 | Home I can nicely calculate times between Work and Home and got right value to the right person. But I'm stuck now with those break times. I need calculate all possible breaks together per person and that way get a total time what is spend to breaks per person. So I need something like following answer when ask person 1 info: Who | Time | Breaktime | Working time 1 | 08:05:00 | 00:45:00 | 07:20:00 Or maybe all persons can came to same page when ask specific day... Who | Time | Breaktime | Working time 1 | 08:05:00 | 00:45:00 | 07:20:00 2 | 07:57:00 | 00:47:00 | 07:10:00 There is always pairs with events. Work -> Home and Bin -> Bout. And yes, there is much more persons and could be much more brake times per person. That might be a bad presentation, sorry about that (NooB). I hope I give that much information as someone can help. But ask if there is something to ask. That is code what I use when I solve one day total time at working place. SELECT TIMEDIFF( (SELECT timestamp FROM `stamps` WHERE (who like '1' and DATE (timestamp) like '2016-09-21' and what like 'Home')), (SELECT timestamp FROM `stamps` WHERE (who like '1' and DATE (timestamp) like '2016-09-21' and what like 'Work')) ) But I cannot use it with multiple events.
That is what I found. Maybe not nice, but it works =). $break = mysqli_query($conn,"SELECT what, timestamp FROM stamps WHERE (who like '$id' and DATE (timestamp) like '$day' and what like 'B%') ORDER BY timestamp"); while($row = mysqli_fetch_array($break)) { if ( $row['what'] == "Bin" ) $start = strtotime( $row['timestamp'] ); else { $stop = strtotime( $row['timestamp'] ); $hours += ( $stop - $start ); } } $btime = gmdate('H:i:s', floor($hours)); $btime gives 00:00:00 style result.
SQL COUNT query similar to UNION but with results in multiple columns
I have a single-table SQL database built from DHCPD logs, structured as below: +------+-------+------+----------+---------+-------------------+-----------------+ | id | Month | Day | Time | Type | MAC | ClientIP | +------+-------+------+----------+---------+-------------------+-----------------+ | 9305 | Nov | 24 | 03:20:00 | DHCPACK | 00:04:f2:4b:dd:51 | 10.123.246.116 | | 9307 | Nov | 24 | 03:20:07 | DHCPACK | 00:04:f2:99:4c:ba | 10.123.154.176 | | 9310 | Nov | 24 | 03:20:08 | DHCPACK | 00:19:bb:cf:cd:28 | 10.99.107.3 | | 9311 | Nov | 24 | 03:20:08 | DHCPACK | 00:19:bb:cf:cd:28 | 10.99.107.3 | Every DHCP event from the log will eventually make its way into this database, so events from any point in time will be potentially used in the construction of graphs. To make use of the data for graphing, I need to be able to create an output table with multiple columns, but with values derived from a count of those in a single column matching a specific pattern. The closest thing I've managed to come up with is this query: select 'Data' as ClientIP, count(*) from Log where ClientIP like '10.99%' and MAC like '00:04:f2%' union select 'Voice' as ClientIP, count(*) from Log where ClientIP like '10.123%' and MAC like '00:04:f2%'; Which yields the following result: +-----------+-------+ | ClientIP | Count | +-----------+-------+ | Data | 4618 | | Voice | 13876 | +-----------+-------+ Fine for a one-off query, but I want to take those two rows, turn them into two columns, and run the same query with one row per hour (for instance). I want something like this: +------+-------+------+ | Hour | Voice | Data | +------+-------+------+ | 03 | 22 | 4 | | 04 | 123 | 23 | | 05 | 45 | 5 | Any advice is greatly welcomed. Thanks
You can group by hour and use conditional computation to count Data and Voice traffic. For example: SELECT HOUR(time) AS `Hour`, SUM(CASE WHEN ClientIP like '10.99%' and MAC like '00:04:f2%' THEN 1 ELSE 0 END) AS `Data`, SUM(CASE WHEN ClientIP like '10.123%' and MAC like '00:04:f2%' THEN 1 ELSE 0 END) AS `Voice` FROM log GROUP BY HOUR(time)
Create a separate table for (as you want) : +------+-------+------+ | Hour | Voice | Data | +------+-------+------+ and update it every hour using triggers.
Selecting max price from every 5 rows
I've been trying and trying and haven't been able to figure this out. In stock charts that have Open, High, Low, Close, you can always chart every minute, 5 min, 10 min, hour, etc. I have data for every minute and I'm trying to select out the Open, High, Low, Close from that minute-by-minute data, but for every 5 minutes. I have data similar to this: __________________________________________________ | Date | TIME | TICKER | Open | High | Low | Close | | 20121203 | 090000 | QQQQ | 23.54 | 24.12 | 23.01 | 23.24 | | 20121203 | 090100 | QQQQ | 23.24 | 24.14 | 22.98 | 24.13 | | 20121203 | 090200 | QQQQ | 24.13 | 25.88 | 23.75 | 25.81 | | 20121203 | 090300 | QQQQ | 25.81 | 25.83 | 24.63 | 24.99 | | 20121203 | 090400 | QQQQ | 24.99 | 25.21 | 23.89 | 24.12 | | 20121203 | 090500 | QQQQ | 24.12 | 24.19 | 21.94 | 22.03 | | 20121203 | 090600 | QQQQ | 22.03 | 22.97 | 20.68 | 21.44 | | 20121203 | 090700 | QQQQ | 21.44 | 24.06 | 19.32 | 23.56 | | 20121203 | 090800 | QQQQ | 23.56 | 25.48 | 23.07 | 25.01 | | 20121203 | 090900 | QQQQ | 25.01 | 28.00 | 24.18 | 27.21 | | 20121203 | 091000 | QQQQ | 27.21 | 27.55 | 24.31 | 24.31 | I need to grab the max(high) for the rows that have a time >= 090000 (that's 9 a.m.) 09 hours 00 minutes 00 seconds. Similar to this, I need min(low), and then I'll grab the close price when time is 090400 because I'm getting every 5 minutes. I could also use the open of the next 5 minute increment, so that's flexible. I've used nested SELECT statements, multiple joins, etc. The problem is that the MySQL execution time is about 1 second per row returned. That's crazy slow when you figure there are 12 rows per hour (60 minutes / 5 minutes = 12), and then because I'm actually doing FOREX, the trading is around the clock, so 24 hours. That gives me 288 per day, or just under 5 minutes per day. To do 1 year of data (~ 250 trading days) would be about 20 hours. There has to be a faster way. I've got some solutions for this with the ID being continuous, and though that might be easiest, I'm not 100% sure my data would be correct in doing that. The reason why is that on Fridays the trading day ends at normal business hours in NY and opens up with the first trading in Tokyo (about mid-afternoon in the United States). I've looked at GROUP BY, but I'm not sure how I can group the data to get a group of 5 where the time is within 5 minutes of each 5 minute group. Thanks for your thoughts and discussion. Jarod
This should show max(high) and min(low) for every 5 minutes SELECT Max(high), Min(low) FROM tbl GROUP BY ROUND(Unix_timestamp(Date(Concat(`date`, `time`))) / ( 5 * 60 )) In the Group by clause we concat your date time column. So it forms something like 20121203090000. This is one the format that is recognized as date in mysql. So we pass it to date() function. Then its converted to UNIX_TIMESTAMP. Its divided by 5 mins timespan. The result will be a float value. But we require same value for a specific time span. Hence the ROUND(). It makes the floating value to the nearest integer. To understand how its working run this query. SELECT high, low Unix_timestamp(Date(Concat(`date`, `time`))) / ( 5 * 60 ) `5-min span` ROUND(Unix_timestamp(Date(Concat(`date`, `time`))) / ( 5 * 60 )) `5-min span rounded` FROM tbl
try this: SELECT CONCAT(DATE,SUBSTRING(Time,1,2),"[" ,IF(SUBSTRING(Time,4,1)<5,CONCAT(SUBSTRING(Time,3,1),"0"),CONCAT(SUBSTRING(Time,3,1),"5")) ,"-" ,IF(SUBSTRING(Time,4,1)<5,CONCAT(SUBSTRING(Time,3,1),"5"),CONCAT(SUBSTRING(Time,3,1)+1,"0")) ,"]") AS timeStr ,MAX(High) ,MIN(LOW) FROM tb1 GROUP BY timeStr; http://sqlfiddle.com/#!2/6b748/1