MySQL nested JSON column search and compare to with today's day - mysql

I am using MySQL Version 5.7.28. I am having json data like below.
CREATE TABLE `week2` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`type` smallint(1),
`json` text ,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
INSERT INTO week2(id,type,json)
VALUES
(121,1,'[{"weekdays":"Sunday"},{"weekdays":"Monday"},{"weekdays":"Tuesday"},{"weekdays":"Wednesday"},{"weekdays":"Thursday"},{"weekdays":"Friday"},{"weekdays":"Saturday"}]'),
(122,1,'[{"weekdays":"Sunday"},{"weekdays":"Monday"}]'),
(123,2,'[{"start_time":"08:00 AM","end_time":"10:00 PM"}]');
As you see, the json column has nested JSON data. so here i am looking to compare today's day ( which is Saturday and we are currently in between start time and end time)
Expected Result:
(121,1,'[{"weekdays":"Sunday"},{"weekdays":"Monday"},{"weekdays":"Tuesday"},{"weekdays":"Wednesday"},{"weekdays":"Thursday"},{"weekdays":"Friday"},{"weekdays":"Saturday"}]'),
(123,2,'[{"start_time":"08:00 AM","end_time":"10:00 PM"}]');

You need to detect day name (1) and time period (2), and combine those two conditions by OR operator at the end.
For (1) : Detect the current day's name by using DAYNAME() function and search whether exists in the JSON data containing weekday keys through use of JSON_CONTAINS function.
For (2) : TIME() function and CASTing strings to TIME data type might be used with a trick to add 12 hours iterations for the cases of PM type times.
So, consider using :
SELECT *
FROM `week2`
WHERE JSON_CONTAINS(`json`->>'$[*].weekdays', CONCAT('"',DAYNAME( NOW() ),'"')) = 1
OR
(
TIME(ADDTIME(NOW(),"8:00:00")) >=
CASE WHEN INSTR(REPLACE(`json`->>'$[0].start_time',"12:00 AM","00:00 AM"),"PM")>0
THEN
CAST(CONCAT(MOD((TIME_FORMAT(REPLACE(`json`->>'$[0].start_time',"12:00 AM","00:00 AM"), "%T")+12),24),":00 AM")
AS TIME)
ELSE
CAST((REPLACE(`json`->>'$[0].start_time',"12:00 AM","00:00 AM")) AS TIME)
END
AND
TIME(ADDTIME(NOW(),"8:00:00")) <=
CASE WHEN INSTR(REPLACE(`json`->>'$[0].end_time',"12:00 AM","00:00 AM"),"PM")>0
THEN
CAST(CONCAT(MOD((TIME_FORMAT(REPLACE(`json`->>'$[0].end_time',"12:00 AM","00:00 AM"), "%T")+12),24),":00 AM")
AS TIME)
ELSE
CAST(REPLACE(`json`->>'$[0].end_time',"12:00 AM","00:00 AM") AS TIME)
END)
Demo

Perhaps, something like this:
SELECT *,CURRENT_TIME
FROM
(SELECT *,JSON_UNQUOTE(JSON_EXTRACT(`json`,'$[0].start_time')) AS st,
JSON_UNQUOTE(JSON_EXTRACT(`json`,'$[0].end_time')) AS et FROM week2) V
WHERE (JSON_SEARCH(`json`, 'one', DAYNAME(CURDATE())) IS NOT NULL
OR
CURRENT_TIME hour >=
CASE WHEN st LIKE '%AM%' THEN REPLACE(st,' AM',':00')
WHEN st LIKE '%PM%' THEN SEC_TO_TIME(TIME_TO_SEC(REPLACE(st,' PM',':00'))+43200)
END
AND
CURRENT_TIME hour <=
CASE WHEN et LIKE '%AM%' THEN REPLACE(et,' AM',':00')
WHEN et LIKE '%PM%' THEN SEC_TO_TIME(TIME_TO_SEC(REPLACE(et,' PM',':00'))+43200)
END);
First query is using JSON_EXTRACT to extract the time and JSON_UNQUOTE to remove the (") then make it as a sub-query.
On the outer-query, find the weekdays value using JSON_SEARCH compare with today's DAYNAME(CURDATE()); if JSON_SEARCH doesn't find it, it will return NULL, hence the IS NOT NULL condition is used.
Append OR to compare CURRENT_TIME with the time value extracted from the json field. But first, using CASE expression to find out if it's AM or PM; if it's AM, just REPLACE the AM with seconds hand (:00). If it's PM, do the REPLACE like AM then convert the time value to seconds using TIME_TO_SEC then add 43200 seconds (12 hours) and convert again to standard hour:minute:second format using SEC_TO_TIME.
P/S: It was my attempt to answer but I didn't post it because I get wrong result with the time.. Only when I put CURRENT_TIME in the fiddle that I realized that the fiddle time zone is different that mine. Also, I was testing on my local DB; which is MariaDB and the results of converting string to time using STR_TO_TIME is different. Honestly, in my opinion, if this is MariaDB, I think there's a chance that the query is much shorter.

Related

Mysql returns some records null in field of date as date have multiple format

I am trying to analyze order_Date column and column have multiple date format i want to convert all those date in same format which wull make be easier to analyze the order_date.
I am trying to analyze the order_date however this column have multiple date format 2019/07/15 and 1/13/2014
Howeever, while converting different format date with one format yyyy/mm/dd with query.
select date_format(order_date, '%y/%m/%d'),orderid from superstore;
it shows null values like this.
i have tried to use `CAST as well but it shows every single value as null.
select case when order_date like '%Y' then date_format(order_date, '%Y/%m/%d') else null end as newdate from superstore;
date_format funtion is used to format a date datatype you should use https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_str-to-date any null values returned by str_to_date either failed or started as null. You will need to examine these and adjust the str_to_date parameters appropriately. There is a catch though is 20/2/20 y/m/d or d/m/y (for example) and how can you differentiate month and day where both are <=12?
For example
drop table if exists t;
create table t
(dt varchar(10));
insert into t values
('1/1/2020'),('2020/1/12'),('12/12/12'),(null),('13-14-15');
select dt,
case when length(substring_index(dt,'/',-1)) = 4 then str_to_date(dt,'%d/%m/%Y')
when length(substring_index(dt,'/',1)) = 4 then str_to_date(dt,'%Y/%m/%d')
when length(substring_index(dt,'/',1)) = 2 then str_to_date(dt,'%y/%m/%d')
else str_to_date(dt,'%y/%m/%d')
end dateformatted
from t;
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=553219f33ad9e9a4404fc4c0cb6571c9
note in no case can I identify month and day and sometimes year..

Change Time Portion Only Of A DateTime Field With Case Statement

I am wanting to use a case statement to change the time piece only of a datetime field in SQL Server 2008 I have a query that will change the time piece, but I need to know how to keep the date portion intact so only the time piece is altered. Meaning if the datetime is
01/01/2015 08:45:10.863
with my syntax I would want to alter it to
01/01/2015 08:30:00.000
This is my syntax which as I said will change the time portion but it does not retain the date. How can I keep the date and change the time portion only?
Create Table #Test
(
[charactername] varchar(100)
,[lefttabletime] datetime
)
Insert Into #Test Values
('Bob Goblin', '01/01/2015 08:14:23.000'),
('Grab Crab', '01/01/2015 08:30:56.023'),
('Mike Knight', '01/01/2015 08:45:10.863')
Select
[charactername]
,case when CAST([lefttabletime] As TIME) > '08:40:00.000' THEN '08:30:00.000'
else [lefttabletime]
FROM #Test
Drop Table #Test
EDIT Additional Syntax Attempted
This threw an error of:
Msg 241, Level 16, State 1, Line 10
Conversion failed when converting date and/or time from character string.
And I tried this syntax
Select
[charactername]
,case when CAST([lefttabletime] As TIME) > '08:40:00.000'
THEN CAST(CAST(CONVERT(DATE, [lefttabletime],101) AS VARCHAR)
+ '08:40:00.000' AS DATETIME) else [lefttabletime] end
FROM #Test
I find it easier if you split the leftabletime into a date and a time component first, the recombine them with the new time portion:
;WITH cte AS
(
SELECT charactername,
[date] = CAST(lefttabletime as date),
[time] = CAST(lefttabletime as time)
FROM #test
)
SELECT charactername,
CAST([date] as datetime)
+ CAST(CASE WHEN [time] > '08:40:00' THEN '08:30:00' ELSE [time] END as datetime)
FROM cte
You can merge the two statement together but I like the simplicity and clarity that a separate CTE provides.
I usually try to make datetime rounding problems fit the DATEADD/DATEDIFF pattern, and I've managed to do that here:
Create Table #Test
(
[charactername] varchar(100)
,[lefttabletime] datetime
)
Insert Into #Test Values
('Bob Goblin', '2015-01-01T08:14:23.000'),
('Grab Crab', '2015-01-01T08:30:56.023'),
('Mike Knight', '2015-01-01T08:45:10.863')
select charactername,
CASE WHEN DATEDIFF(minute,'20010101',lefttabletime)%60 >= 40 THEN
DATEADD(minute,((DATEDIFF(minute,'20010101',lefttabletime)/30)*30),'20010101')
ELSE lefttabletime END
from #Test
Drop Table #Test
The expression DATEADD(minute,((DATEDIFF(minute,'20010101',lefttabletime)/30)*30),'20010101') rounds the time of a datetime down to the nearest 30 minutes (20010101 is an arbitrary date and doesn't need to be adjusted in any way).
I also just use a separate DATEDIFF to find the required matching condition. Where possible, with datetime data, I try to keep it in datetime variables, or, at worst, ints. As soon as you convert to strings you have to start worrying about formats, etc, which I'd usually rather avoid.
Result:
charactername
--------------------- -----------------------
Bob Goblin 2015-01-01 08:14:23.000
Grab Crab 2015-01-01 08:30:56.023
Mike Knight 2015-01-01 08:30:00.000

How to convert a dateformat from SQL Server to MySQL

I've been searching the internet for almost 6 hours now for the fastest solution to my problem.
I've got a SQL Server database where some of the tables have a DATETIME format for columns, but the values in these columns are in the following format:
DD-MM-YYYY HH:MM
eg.
18-12-2012 00:00
From my research MySQL only accepts the following format for its DATETIME values:
YYYY-MM-DD HH:MM:SS
Now, I'm actually trying to make the conversion in the Scripts themselves, not the database.
basically I have over 250,000 records in multiple tables in SQL Server Script Format.
I've already changed all the syntax to MySQL except this part.
Any help or advice would be appreciated. URGENT.
thank you
This is the Table
CREATE TABLE Price(
AirlineCode char(2) NOT NULL
, FlightNumber varchar(6) NOT NULL
, ClassCode char(3) NOT NULL
, TicketCode char(1) NOT NULL
, StartDate DateTime NOT NULL
, EndDate DateTime NOT NULL
, Price decimal NOT NULL
, PriceLeg1 decimal NULL
, PriceLeg2 decimal NULL
, PRIMARY KEY (
AirlineCode,
FlightNumber,
ClassCode,
TicketCode,
StartDate
)
, FOREIGN KEY (AirlineCode) REFERENCES Airlines (AirlineCode)
, FOREIGN KEY (ClassCode) REFERENCES TicketClass (ClassCode)
, FOREIGN KEY (TicketCode) REFERENCES TicketType (TicketCode));
and this is a sample insertion:
INSERT INTO Price VALUES ( 'QF', 'QF67', 'ECO', 'E', '18-12-2012 00:00', '04-01-2013 00:00', 3427.82, 1636.14, 2045.20 );
You can use MySQL STR_TO_DATE() function to convert dates from a known format to a datetime value. Assuming that you have dates in dd-mm-yyyy hh:mm (all numbers 0 padded and hh:mm is a 24-hour time) you can use:
SELECT STR_TO_DATE('18-12-2012 00:00', '%d-%m-%Y %k:%i')
-- 2012-12-18 00:00:00
You can use this function in your insert queries like so:
INSERT INTO Price VALUES (
'QF',
'QF67',
'ECO',
'E',
STR_TO_DATE('18-12-2012 00:00', '%d-%m-%Y %k:%i'),
STR_TO_DATE('04-01-2013 00:00', '%d-%m-%Y %k:%i'),
3427.82,
1636.14,
2045.20
);
Edit
If editing your insert queries is not possible, you can let MySQL do all the conversion. Temporarily change the datatype of datetime columns to VARCHAR and import the data. Then add a temporary field e.g. StartDate_Temp DATETIME and run an update query:
UPDATE Price SET StartDate_Temp = STR_TO_DATE(StartDate, '%d-%m-%Y %k:%i')
Once you've verified that your data is OK, delete the temporary columns.
Edited answer:
Also, you may use RegEx in SQL Server Management Studio:
Find what: {[0-9][0-9]}\-{[0-9][0-9]}\-{[0-9][0-9][0-9][0-9]} {[0-9][0-9]}\:{[0-9][0-9]}
Replace with:\3-\2-\1
First of all I would like to say thank you to all for your efforts. This was the first question I've ever asked at StackOverflow.com and am really impressed by the level of support.
After a little bit more research, I remembered the good old toolkit of RegEx that I learnt last year. So I decided to put it in some good use.
in Notepad++ thanks to its Regex search capabilities, I managed to trace down every date value in my 250000 records and replace them with the required ones.
([0-9][0-9])-([0-9][0-9])-([0-9][0-9][0-9][0-9]) ([0-9][0-9]:[0-9][0-9])
this was the search I put through for the date format that I had, and then i swapped the year and the day values and added the seconds in there as well by having the following in the Replace with box:
\3-\2-\1 \4:00
Problem solved.

How do I get the average time from a series of DateTime columns in SQL Server 2008?

Lets say I have a table that contains the following - id and date (just to keep things simple).
It contains numerous rows.
What would my select query look like to get the average TIME for those rows?
Thanks,
Disclaimer: There may be a much better way to do this.
Notes:
You can't use the AVG() function against a DATETIME/TIME
I am casting DATETIME to DECIMAL( 18, 6 ) which appears to yield a reasonably (+- few milliseconds) precise result.
#1 - Average Date
SELECT
CAST( AVG( CAST( TimeOfInterest AS DECIMAL( 18, 6 ) ) ) AS DATETIME )
FROM dbo.MyTable;
#2 - Average Time - Remove Date Portion, Cast, and then Average
SELECT
CAST( AVG( CAST( TimeOfInterest - CAST( TimeOfInterest AS DATE ) AS DECIMAL( 18, 6 ) ) ) AS DATETIME )
FROM dbo.MyTable;
The second example subtracts the date portion of the DATETIME from itself, leaving only the time portion, which is then cast to a decimal for averaging, and back to a DATETIME for formatting. You would need to strip out the date portion (it's meaningless) and the time portion should represent the average time in the set.
SELECT CAST(AVG(CAST(ReadingDate AS real) - FLOOR(CAST(ReadingDate as real))) AS datetime)
FROM Rbh
I know that, in at least some of the SQL standards, the value expression (the argument to the AVG() function) isn't allowed to be a datetime value or a string value. I haven't read all the SQL standards, but I'd be surprised if that restriction had loosened over the years.
In part, that's because "average" (or arithmetic mean) of 'n' values is defined to be the sum of the values divided by the 'n'. And the expression '01-Jan-2012 08:00' + '03-Mar-2012 07:53' doesn't make any sense. Neither does '01-Jan-2012 08:00' / 3.
Microsoft products have a history of playing fast and loose with SQL by exposing the internal representation of their date and time data types. Dennis Ritchie would have called this "an unwarranted chumminess with the implementation."
In earlier versions of Microsoft Access (and maybe in current versions, too), you could multiply the date '01-Jan-2012' by the date '03-Mar-2012' and get an actual return value, presumably in units of square dates.
If your dbms supports the "interval" data type, then taking the average is straightforward, and does what you'd expect. (SQL Server doesn't support interval data types.)
create table test (
n interval hour to minute
);
insert into test values
('1:00'),
('1:30'),
('2:00');
select avg(n)
from test;
avg (interval)
--
01:30:00

Converting a date string which is before 1970 into a timestamp in MySQL

Not a very good title, so my apologies.
For some reason, (I wasn't the person who did it, i digress) we have a table structure where the field type for a date is varchar. (odd).
We have some dates, such as:
1932-04-01 00:00:00 and 1929-07-04 00:00:00
I need to do a query which will convert these date strings into a unix time stamp, however, in mySQL if you convert a date which is before 1970 it will return 0.
Any ideas?
Thanks so much!
EDIT: Wrong date format. ooops.
Aha! We've found a solution!
The SQL to do it:
SELECT DATEDIFF( STR_TO_DATE('04-07-1988','%d-%m-%Y'),FROM_UNIXTIME(0))*24*3600 -> 583977600
SELECT DATEDIFF( STR_TO_DATE('04-07-1968','%d-%m-%Y'),FROM_UNIXTIME(0))*24*3600 -> -47174400
This could be useful for future reference.
You can test it here: http://www.onlineconversion.com/unix_time.htm
I've adapted the DATEDIFF workaround to also include time not just days. I've wrapped it up into a stored function, but you can just extract the SELECT part out if you don't want to use functions.
DELIMITER |
CREATE FUNCTION SIGNED_UNIX_TIMESTAMP (d DATETIME)
RETURNS BIGINT
DETERMINISTIC
BEGIN
DECLARE tz VARCHAR(100);
DECLARE ts BIGINT;
SET tz = ##time_zone;
SET time_zone = '+00:00';
SELECT DATEDIFF(d, FROM_UNIXTIME(0)) * 86400 +
TIME_TO_SEC(
TIMEDIFF(
d,
DATE_ADD(MAKEDATE(YEAR(d), DAYOFYEAR(d)), INTERVAL 0 HOUR)
)
) INTO ts;
SET time_zone = tz;
return ts;
END|
DELIMITER ;
-- SELECT UNIX_TIMESTAMP('1900-01-02 03:45:00');
-- will return 0
-- SELECT SIGNED_UNIX_TIMESTAMP('1900-01-02 03:45:00');
-- will return -2208888900
convert these date strings into a
unix time stamp
Traditional Unix timestamps are an unsigned integer count of seconds since 1-Jan-1970 therefore can't represent any date before that.
At best you will have mixed results depending on the system you are using to represent the timestamp.
From wikipedia
There was originally some controversy
over whether the Unix time_t should
be signed or unsigned. If unsigned,
its range in the future would be
doubled, postponing the 32-bit
overflow (by 68 years). However, it
would then be incapable of
representing times prior to 1970.
Dennis Ritchie, when asked about this
issue, said that he hadn't thought
very deeply about it, but was of the
opinion that the ability to represent
all times within his lifetime would be
nice. (Ritchie's birth, in 1941, is
around Unix time −893 400 000.) The
consensus is for time_t to be signed,
and this is the usual practice. The
software development platform for
version 6 of the QNX operating system
has an unsigned 32-bit time_t, though
older releases used a signed type.
It appears that MySQL treats timestamps as an unsigned integer, meaning that times before the Epoc will all resolve to 0.
This being the case, you always have the option to implement your own unsigned timestamp type and use that for your calculations.
If its feasible for your problem, you could shift all your mysql times by, say 100 years, and then work with those adjusted timestamps or re calculate the negative timestamp value.
As some have said, make sure your system is using 64bits to represent the timestamp otherwise you'll hit the year 2038 problem.
I have not tried the above solutions but this might in case you are not able to retrieve the date value from the MySQL database in the form of timestamp, then this operation can also be tried
SELECT TIMESTAMPDIFF(second,FROM_UNIXTIME(0),'1960-01-01 00:00:00');
To get the max. Range +/- wise use this query on your birthday field, in my case "yyyy-mm-dd" but you can change it to your needs
select name, (#bday:=STR_TO_DATE(birthday,"%Y-%m-%d")),if(year(#bday)<1970,UNIX_TIMESTAMP(adddate(#bday, interval 68 year))-2145916800,UNIX_TIMESTAMP(#bday)) from people
I feel like we're making this much too difficult...
Use my functions below so you can convert anything to and from unix timestamps, much like you do in a browser.
Call functions like this:
select to_unix_time('1776-07-04 10:02:00'),
from_unix_time(-6106024680000);
By compiling these:
delimiter $$
create function to_unix_time (
p_datetime datetime
) returns bigint
deterministic
begin
declare v_ret bigint;
select round(timestampdiff(
microsecond,
'1970-01-01 00:00:00',
p_datetime
) / 1000, 0)
into v_ret;
return v_ret;
end$$
create function from_unix_time (
p_time bigint
) returns datetime(6)
deterministic
begin
declare v_ret datetime(6);
select '1970-01-01 00:00:00' +
interval (p_time * 1000) microsecond
into v_ret;
return v_ret;
end$$
Use Date instead of timestamps. Date will solve your Problems.
check this link