I am working on a WordPress site where I have to create a table using values from MySQL, something like 'https://alerts.flexoffers.com/v2/PublisherLog'.
What I have achieved right now shows all the entries on the base of warning or error. This means there will be multiple entries each day telling if it is an error or a warning.
What I need is that there is only one entry for a specific date in the date column. And then in respective warning and error columns, all the entries from that day add up and show the number of errors/warnings there were.
SELECT bctw_error_logs.`id`,
SUM(if(bctw_error_logs.`severity_level` = '1', 1, 0)) as Warnings,
SUM(if(bctw_error_logs.`severity_level` = '2', 1, 0)) as Errors,
DAYNAME(`log_date`) as Day,
DATE_FORMAT(`log_date`, "%M %d %Y") as Date
FROM bctw_error_logs group by bctw_error_logs.`id`;
This what I am using right now.
Below are the table I've created and its records:
CREATE TABLE students (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
gender TEXT NOT NULL,
created_at datetime
);
INSERT INTO students VALUES (1, 'Ryan', 'M', '2012-11-03 00:00:00');
INSERT INTO students VALUES (2, 'Joanna', 'F', '2013-11-03 00:00:00');
To fetch the records by time, I use the following SQL statement:
SELECT * FROM students WHERE created_at > 1620489600000;
and both records can be returned, which confuses me because 1620489600000 (Sat May 08 2021 16:00:00 GMT+0000) should be a timestamp way later than the create_at fields of both records.
Indeed I know this can also be achieved by specifying a time formatted as 2012-11-03 00:00:00, but I just wonder:
whether we can effectively query a datetime column in where clause with unix timestamp?
if not, why does the above select statement return both records?
To compare two different data types MySQL automatically converts one of them to the other. In this case it tries to make numbers of the dates. What is the result? See here
The date gets converted to
20121103000000
which is bigger than
1620489600000
So the result in your WHERE clause is true and returns all records.
If you need to go from a Unix Timestamp you could use this:
WHERE created_at > FROM_UNIXTIME(1620489600000 / 1000)
Notice that I divided by 1000 because Unix time has to be in seconds and not milli seconds.
`ID`,`NAMES`, `TIMESTAMP`
'3', 'Sharon', '2020-02-17 21:11:12'
'4', 'RALPH', '2020-02-18 01:50:19'
Above is an extraction from my table.I would want to find time difference between the two timestamps(i.e for ID=3 and ID=4). Also i would like to create a column for the time difference.Please someone help
Thanks
You can use TIMESTAMPDIFF() function
I have a column(varchar) with date values, I need to find those dates which are expiring in next 30 days.
ExpiringDate
===================
20171208,
20171215,samples
20171130,tested
N/A
No
(empty row)
So, First I need to get values before comma. On the resultset, I need to filter out rows that has only numbers(no 'N/A' or 'No' or empty rows) & then I need to filter those dates which are expiring in next 30 days.
Edited
I have tried the following & resultset seems to be inappropriate
SELECT
DocName,
CategoryName,
AttributeName,
CAST(SUBSTRING_INDEX(AttributeValue, ',', 1) AS DATE) AS ExpiredDate
FROM myDB
WHERE (AttributeName = 'Date of last vessel OVID' OR AttributeName = 'Next Statutory docking' OR
AttributeName = 'Last statutory docking') AND AttributeValue LIKE '%[^0-9]%' AND
DATEDIFF(now(), AttributeValue) <= 30;
Because you are not only storing dates as text, but mixing those dates with entirely non date information, this complicates things. In this case, we can do two checks, one to ensure that the record starts with an actual expected date, and the second to make sure that the date diff is within 30 days from now.
SELECT ExpiringDate
FROM
(
SELECT ExpiringDate
FROM yourTable
WHERE ExpiringDate REGEXP '^[0-9]{8}'
) t
WHERE
DATEDIFF(LEFT(ExpiringDate, 8), NOW()) BETWEEN 0 AND 30;
Note that I use a subquery to first remove rows that do not even have a parseable date. The reason for this is that DATEDIFF will error out if not passed valid dates for both parameters.
Demo
I've been looking at several other SO questions but I could not make out a solution from these. First, the description, then what I'm missing from the other threads. (Heads up: I'm very well aware of the non-normalised structure of our database, which is something I have addressed in meetings before but this is what we have and what I have to work with.)
Background description
We have a machine that manufactures products in 25 positions. These products' production data is being logged in a table that among other things logs current and voltage for every position. This is only logged when the machine is actually producing products (i.e. has a product in the machine). The time where no product is present, nothing is being logged.
This machine can run in two different production modes: full production and R&D production. Full production means that products are being inserted continuously so that every instance has a product at all times (i.e. 25 products are present in the machine at all times). The second mode, R&D production, only produces one product at a time (i.e. one product enters the machine, goes through the 25 instances one by one and when this one is finished, the second product enters the machine).
To clarify: every position logs data once every second whenever a product is present, which means 25 instances per second when full production is running. When R&D mode is running, position 1 will have ~20 instances for 20 consecutive seconds, position 2 will have ~20 instances for the next 20 consecutive seconds and so on.
Table structure
Productiondata:
id (autoincrement)
productID
position
time (timestamp for logged data)
current (amperes)
voltage (volts)
Question
We want to calculate the uptime of the machine, but we want to separate the uptime for production mode and R&D mode, and we want to separate this data on a weekly basis.
Guessed solution
Since we have instances logged every second I can count the amount of DISTINCT instances of time values we have in the table to find out the total uptime for both production and R&D mode. To find the R&D mode, I can safely say that whenever there is a time instance that has only one entry, I'm running in R&D mode (production mode would have 25 instances).
Progress so far
I have the following query which sums up all distinct instances to find both production and R&D mode:
SELECT YEARWEEK(time) AS YWeek, COUNT(DISTINCT time) AS Time_Seconds, ROUND(COUNT(DISTINCT time)/3600, 1) AS Time_Hours
FROM Database.productiondata
WHERE YEARWEEK(time) >= YEARWEEK(curdate()) - 21
GROUP BY YWeek;
This query finds out how many DISTINCT time instances there are in the table and counts the number and groups that by the week.
Problem
The above query counts the amount of instances that exist in the table, but I want to find ONLY the UNIQUE instances. Basically, I'm trying to find something like IF count(time) = 1, then count that instance, IF count(time) > 1 then don't count it at all (DISTINCT still counts this).
I looked at several other SO threads, but almost all explain how to find unique values with DISTINCT, which only accomplishes half of what I'm looking for. The closest I got was this which uses a HAVING clause. I'm currently stuck at the following:
SELECT YEARWEEK(time) as YWeek, COUNT(Distinct time) As Time_Seconds, ROUND(COUNT(Distinct time)/3600, 1) As Time_Hours
FROM
(SELECT * FROM Database.productiondata
WHERE time > '2014-01-01 00:00:00'
GROUP BY time
HAVING count(time) = 1) as temptime
GROUP BY YWeek
ORDER BY YWeek;
The problem here is that we have a GROUP BY time inside the nested select clause which takes forever (~5 million rows only for this year so I can understand that). I mean, syntactically I think that this is correct but it takes forever to exectue. Even EXPLAIN for this times out.
And that is where I am. Is this the correct approach or is there any other way that is smarter/requires less query time/avoids the group by time clause?
EDIT: As a sample, we have this table (apologies for formatting, don't know how to make a table format here on SO)
id position time
1 1 1
2 2 1
3 5 1
4 19 1
... ... ...
25 7 1
26 3 2
27 6 2
... ... ...
This table shows how it looks like when there is a production run going on. As you can see, there is no general structure for which position gets the first entry when logging the data in the table; what happens is that the 25 positions gets logged during every second and the data is then added to the table depending on how fast the PLC sends the data for every position. The following table shows how the table looks like when it runs in research mode.
id position time
245 1 1
246 1 2
247 1 3
... ... ...
269 1 25
270 2 26
271 2 27
... ... ...
Since all the data is consolidated into one single table, we want to find out how many instances there are when COUNT(time) is exactly equal to 1, or we could look for every instance when COUNT(time) is strictly larger than 1.
EDIT2: As a reply to Alan, the suggestion gives me
YWeek Time_Seconds Time_Hours
201352 1 0.0
201352 1 0.0
201352 1 0.0
... ... ...
201352 1 0.0 (1000 row limit)
Whereas my desired output is
Yweek Time_Seconds Time_Hours
201352 2146 35.8
201401 5789 96.5
... ... ...
201419 8924 148.7
EDIT3: I have gathered the tries and the results so far here with a description in gray above the queries.
You might achieve better results by eliminating your sub select:
SELECT YEARWEEK(time) as YWeek,
COUNT(time) As Time_Seconds,
ROUND(COUNT(time)/3600, 1) As Time_Hours
FROM Database.productiondata
WHERE time > '2014-01-01 00:00:00'
GROUP BY YWeek
HAVING count(time) = 1)
ORDER BY YWeek;
I'm assuming time has an index on it, but if it does not you could expect a significant improvement in performance by adding one.
UPDATE:
Per the recently added sample data, I'm not sure your approach is correct. The time column appears to be an INT representing seconds while you're treating it as a DATETIME with YEARWEEK. Below I have a working example in SQL that does exactly what you asked IF time is actually a DATETIME column:
DECLARE #table TABLE
(
id INT ,
[position] INT ,
[time] DATETIME
)
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -1, GETDATE()) )
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -2, GETDATE()) )
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -2, GETDATE()) )
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -2, GETDATE()) )
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -2, GETDATE()) )
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -3, GETDATE()) )
INSERT INTO #table
VALUES ( 1, 1, DATEADD(week, -3, GETDATE()) )
SELECT CAST(DATEPART(year, [time]) AS VARCHAR)
+ CAST(DATEPART(week, [time]) AS VARCHAR) AS YWeek ,
COUNT([time]) AS Time_Seconds ,
ROUND(COUNT([time]) / 3600, 1) AS Time_Hours
FROM #table
WHERE [time] > '2014-01-01 00:00:00'
GROUP BY DATEPART(year, [time]) ,
DATEPART(week, [time])
HAVING COUNT([time]) > 0
ORDER BY YWeek;
SELECT pd1.*
FROM Database.productiondata pd1
LEFT JOIN Database.productiondata pd2 ON pd1.time=pd2.time AND pd1.id<pd2.id
WHERE pd1.time > '2014-01-01 00:00:00' AND pd2.time > '2014-01-01 00:00:00'
AND pd2.id IS NULL
You can LEFT JOIN to the same table and leave only the rows with no related
UPDATE The query works using the SQL fiddle
SELECT pd1.* From productiondata pd1
left Join productiondata pd2
ON pd1.time = pd2.time and pd1.id < pd2.id
Where pd1.time > '2014-01-01 00:00:00' and pd2.id IS NULL;