So i had a mysql query which used to work as intended but now i think something has happened and it seems to not work anymore.
What i have is 2 tables that i want to join. "
users
and
logs
I want a list of all the user id's(UID) in the "users" table that have not logged in today. So i used this query.
SELECT users.UID
FROM users
LEFT JOIN logs
ON users.UID = logs.UID
AND DATE(logs.SCANTIME )=DATE(SYSDATE())
WHERE logs.UID is null
The above query used to work but now returns with a list of UID's that are not supposed to be there.
If i look into the log today i can see multiple rows, i have omitted all but one for the sake of saving space.
mysql> SELECT UID, SCANTIME FROM logs WHERE DATE(scantime)=DATE(SYSDATE());
+------------+---------------------+
| UID | SCANTIME |
+------------+---------------------+
.............
| AA9B351B | 2017-08-02 06:13:21 |
.............
+------------+---------------------+
63 rows in set (0.00 sec)
So this guy AA9B351B is clearly logged in today. Yet he showes up when i run the query above. Thanks for any replies
The server time may have become incorrect, so SYSDATE is returning the wrong value. Check the system time and also run a SQL query with SYSDATE function replaced with literals (Just to test). Also related to this check the time zone has not changed. If server time wrong then reset.
Related
I have a table setup for member logins. Right now the last_login field is stamped with MySQL's NOW(). But I want to also track their last active time on the site. And the only way I can think of is to create a new query, insert it into every procedure I have on every page, and update a timestamped last_activity field for the current login. Is there a better way to do this?
Example:
MariaDB [master]> select logintime, last_activity
from memberlogins
where memberid = "1"
order by loginid
desc limit 1;
+---------------------+---------------------+
| logintime | last_activity |
+---------------------+---------------------+
| 2017-02-11 22:28:54 | 2017-02-11 23:48:14 |
+---------------------+---------------------+
That's what I want, to add the last_activity to the table. And the only way I can think of to accomplish this is to add this query:
$stmt = $dbsotp->prepare('UPDATE memberlogins
SET last_activity = NOW()
WHERE memberid = :memberid');
And then the rest of the PDO here. So all I'm asking is if there's a better way to do this than inserting this query into every procedure I have on every page. I have 67 pages with a several hundred procedures, that's why I ask if this is the only way or if there's a better way to go.
I have a dataset from which I need to count all row occurrences grouping by each day and sum them into a dataset of following format:
| date | count |
| 2001-01-01 | 11 |
| 2001-01-02 | 0 |
| 2001-01-03 | 4 |
The problem is, that some of the data is missing from certain periods of time and new dates should be created to have the count of zero. I have searched various topics considering this same issue and from them I've learned that it's possible to solve by creating a temporary calendar table to hold all the dates and join the result dataset with the date table.
Though, I have only a read access to the database I'm using, so it's not possible for me to create a separate calendar table. So could this be possible to solve in a single query only? If not, I could always do this in PHP but I would prefer a more straighforward way to do this.
EDIT: Just to clarify based on the questions asked in the comments: The missing dates are required for a spesific, user given time frame. E.g. the query could be:
SELECT date(timestamp), count(distinct(id))
FROM 'table'
WHERE date(timestamp) BETWEEN date("2001-01-01") AND date("2001-12-31")
GROUP BY date(timestamp)
SQL is really not made for this kind of job :/
That's possible but really really messy and I strongly discourage you from doing it.
The easiest way was to have a separate calendar table but as you said you only have a read access to your database.
The other one is to generate the sequence using this kind of trick:
SELECT #rownum:=#rownum+1 rownum, t.*FROM (SELECT #rownum:=0) r, ("yourquery") t;
I won't get into it, as I already told you, it's really ugly :(
try this...
SELECT Date, COUNT(*) Count
FROM yourtable
GROUP BY Date
This works for sure!!!
Let me know, if it helped!
On a MySQL 5.1.34 server, I have the following perplexing situation:
mysql> explain select * FROM master.ObjectValue WHERE id IN ( SELECT id FROM backup.ObjectValue ) AND timestamp < '2008-04-26 11:21:59';
+----+--------------------+-------------+-----------------+-------------------------------------------------------------+------------------------------------+---------+------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+-------------+-----------------+-------------------------------------------------------------+------------------------------------+---------+------+--------+-------------+
| 1 | PRIMARY | ObjectValue | range | IX_ObjectValue_Timestamp,IX_ObjectValue_Timestamp_EventName | IX_ObjectValue_Timestamp_EventName | 9 | NULL | 541944 | Using where |
| 2 | DEPENDENT SUBQUERY | ObjectValue | unique_subquery | PRIMARY | PRIMARY | 4 | func | 1 | Using index |
+----+--------------------+-------------+-----------------+-------------------------------------------------------------+------------------------------------+---------+------+--------+-------------+
2 rows in set (0.00 sec)
mysql> select * FROM master.ObjectValue WHERE id IN ( SELECT id FROM backup.ObjectValue ) AND timestamp < '2008-04-26 11:21:59';
Empty set (2 min 48.79 sec)
mysql> select count(*) FROM master.ObjectValue;
+----------+
| count(*) |
+----------+
| 35928440 |
+----------+
1 row in set (2 min 18.96 sec)
How can it take 3 minutes to examine 500000 records when it only
takes 2 minutes to visit all records?
How can a subquery on a
separate database be classified dependent?
What can I do to speed up
this query?
UPDATE:
The actual query that took a long time was a DELETE, but you can't do explain on those; DELETE is why I used subselect. I have now read the documentation and found out about the syntax "DELETE FROM t USING ..." Rewriting the query from:
DELETE FROM master.ObjectValue
WHERE timestamp < '2008-06-26 11:21:59'
AND id IN ( SELECT id FROM backup.ObjectValue ) ;
into:
DELETE FROM m
USING master.ObjectValue m INNER JOIN backup.ObjectValue b ON m.id = b.id
WHERE m.timestamp < '2008-04-26 11:21:59';
Reduced the time from minutes to .01 seconds for an empty backup.ObjectValue.
Thank you all for good advise.
The dependent subquery slows you outer query down to a crawl (I suppose you know it means it's run once per row of found in the dataset being looked at).
You don't need the subquery there, and not using one will speedup your query quite significantly:
SELECT m.*
FROM master.ObjectValue m
JOIN backup.ObjectValue USING (id)
WHERE m.timestamp < '2008-06-26 11:21:59'
MySQL frequently treats subqueries as dependent even though they are not. I've never really understood the exact reasons for that - maybe it's simply because the query optimizer fails to recognize it as independent. I never bothered looking more in details because in these cases you can virtually always move it to the FROM clause, which fixes it.
For example:
DELETE FROM m WHERE m.rid IN (SELECT id FROM r WHERE r.xid = 10)
// vs
DELETE m FROM m WHERE m.rid IN (SELECT id FROM r WHERE r.xid = 10)
The former will produce a dependent subquery and can be very slow. The latter will tell the optimizer to isolate the subquery, which avoids a table scan and makes the query run much faster.
Notice how it says there is only 1 row for the subquery? There is obviously more than 1 row. That is an indication that mysql is loading only 1 row at a time. What mysql is probably trying to do is "optimize" the subquery so that it only loads records in the subquery that also exist in the master query, a dependent subquery. This is how a join works, but the way you phrased your query you have forced a reversal of the optimized logic of a join.
You've told mysql to load the backup table (subquery) then match it against the filtered result of the master table "timestamp < '2008-04-26 11:21:59'". Mysql determined that loading the entire backup table is probably not a good idea. So mysql decided to use the filtered result of the master to filter the backup query, but the master query hasn't completed yet when trying to filter the subquery. So it needs to check as it loads each record from the master query. Thus your dependent subquery.
As others mentioned, use a join, it's the right way to go. Join the crowd.
How can it take 3 minutes to examine 500000 records when it only takes 2 minutes to visit all records?
COUNT(*) is always transformed to COUNT(1) in MySQL. So it doesn't even have to enter each record, and also, I would imagine that it uses in-memory indexes which speeds things up. And in the long-running query, you use range (<) and IN operators, so for each record it visits, it has to do extra work, especially since it recognizes the subquery as dependent.
How can a subquery on a separate database be classified dependent?
Well, it doesn't matter if it's in a separate database. A subquery is dependent if it depends on values from the outer query, which you could still do in your case... but you don't, so it is, indeed, strange that it's classified as a dependent subquery. Maybe it is just a bug in MySQL, and that's why it's taking so long - it executes the inner query for every record selected by the outer query.
What can I do to speed up this query?
To start with, try using JOIN instead:
SELECT master.*
FROM master.ObjectValue master
JOIN backup.ObjectValue backup
ON master.id = backup.id
AND master.timestamp < '2008-04-26 11:21:59';
The real answer is, don't use MySQL, its optimizer is rubbish. Switch to Postgres, it will save you time in the long run.
To everyone saying "use JOIN", that's just a nonsense perpetuated by the MySQL crowd who have refused for 10 years to fix this glaringly horrible bug.
I am working with MySQL and using stored procedures. I have a profiling tool that I am using to profile the code that communicates with MySQL through the stored procedures and I was wondering if there was a tool or capability within MySQL client to profile stored procedure executions. What I have in mind is something that's similar to running queries with profiling turned on. I am using MySQL 5.0.41 on Windows XP.
Thanks in advance.
There is a wonderfully detailed article about such profiling: http://mablomy.blogspot.com/2015/03/profiling-stored-procedures-in-mysql-57.html
As of MySQL 5.7, you can use performance_schema to get informations about the duration of every statement in a stored procedure. Simply:
1) Activate the profiling (use "NO" afterward if you want to disable it)
UPDATE performance_schema.setup_consumers SET ENABLED="YES"
WHERE NAME = "events_statements_history_long";
2) Run the procedure
CALL test('with parameters', '{"if": "needed"}');
3) Query the performance schema to get the overall event informations
SELECT event_id,sql_text,
CONCAT(TIMER_WAIT/1000000000,"ms") AS time
FROM performance_schema.events_statements_history_long
WHERE event_name="statement/sql/call_procedure";
| event_id | sql_text | time |
|2432 | CALL test(...) | 1726.4098ms |
4) Get the detailed informations of the event you want to profile
SELECT EVENT_NAME, SQL_TEXT,
CONCAT(TIMER_WAIT/1000000000,"ms") AS time
FROM performance_schema.events_statements_history_long
WHERE nesting_event_id=2432 ORDER BY event_id;
| EVENT_NAME | SQL_TEXT | time |
| statement/sp/stmt | ... 1 query of the procedure ... | 4.6718ms |
| statement/sp/stmt | ... another query of the procedure ... | 4.6718ms |
| statement/sp/stmt | ... another etc ... | 4.6718ms |
This way, you can tell which query takes the longest time in your procedure call.
I don't know any tool that would turn this resultset into a KCachegrind friendly file or so.
Note that this should not be activated on production server (might be a performance issue, a data size bump, and since performance_schema.events_statements_history_long holds the procedure's parameters values, then it might be a security issue [if procedure's parameter is a final user email or password for instance])
You can turn on the slow query logging within MySQL.
Take a look at this other SO question:
MYSQL Slow Query
Depending on which version, you may actually be able to set the value to zero, so every single query in the DB is shown in the slow query log.
See here for additional details:
http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html#sysvar_long_query_time
So I have a table where I collect data for the jobs that I do. Each time I create a job I assign it a date. The problem with this is the days I don't have jobs aren't stored in the database therefore when I graph my data I never see the days that I had zero jobs.
My current query looks like this:
SELECT job_data_date, SUM(job_data_invoice_amount) as job_data_date_income
FROM job_data
WHERE job_data_date >= '2010-05-05'
GROUP BY job_data_date
ORDER BY job_data_date;
The output is:
| job_data_date | job_data_date_income |
| 2010-05-17 | 125 |
| 2010-05-18 | 190 |
| 2010-05-20 | 170 |
As you can see from the example output the 2010-05-19 would not show up in the results because it was never stored there.
Is there a way to show the dates that are missing?
Thank you,
Marat
One idea is that you could have a table with all of the dates in it that you want to show and then do an outer join with that table.
So if you had a table called alldates with one column (job_data_date):
SELECT ad.job_data_date, SUM(job_data_invoice_amount) as job_data_date_income
FROM alldates ad left outer join job_data jd on ad.job_data_date = jd.job_data_date
WHERE ad.job_data_date >= '2010-05-05'
GROUP BY ad.job_data_date
ORDER BY ad.job_data_date;
The down side is that you would need to keep this table populated with all of the dates you want to show.
There's no reasonable way to do this using pure SQL, on MySQL at least, without creating a table with every date ever devised. Your best option is to alter the application that's using the results of that query to fill in the holes itself. Rather than graphing only the values it received, construct its own set of values with a simple loop; counting up one day at a time, filling in values from the query wherever they're available.