MySQL - How to optimize long running query - mysql

Question

I have a query that takes a very long time to complete. I am wondering if there is anyway to optimize it so it would run quicker. Currently the table has around 15 million row and has indexing on expirationDate. This query has been running for 8000+ seconds.
UPDATE LOW_PRIORITY items
SET expire = 1
WHERE expirationdate < Curdate()
AND expirationdate != '000-00-00'
AND expire = '0'
AND ( `submit_id` != '742457'
OR submit_id IS NULL )

Answer 1

Never mind your table structure (I requested it before this edit), I have a solution for you. I've done this before with my DBAs several times on our production mySQL databases even during peak hours of the day. The approach you need to take is taking your one massive UPDATE query (that will lock the table while executing) and turn it into several individual updates. Those individual updates will not have a material impact on your database, especially if you run them in smaller batches and break it down.
To generate those individual queries, run this (assuming the items table's primary key is named "id"):
SELECT CONCAT(
‘UPDATE items SET expire = 1 WHERE id = ‘,
items.id,
‘;’) AS ''
FROM items
WHERE expirationdate < Curdate()
AND expirationdate != '000-00-00'
AND expire = '0'
AND ( `submit_id` != ‘742457'
OR submit_id IS NULL )
This will generate SQL that looks like this:
UPDATE items SET expire = 1 WHERE id = 1234;
UPDATE items SET expire = 1 WHERE id = 2345;
.....
What I personally like to do is put the above query in a text file and run this from the command line:
cat theQuery.sql | mysql -u yourusername [+ command line args with db host, etc] -p > outputSQLCommands.sql
This will pass the query to mySQL and dump the results to an output file. If you notice, we named the column in our output to '' (nothing). That makes the output very nicely formatted in the text file so it can be easily dumped back into mySQL to run the commands. In fact you could do it all in one fell swoop like this:
cat theQuery.sql | mysql -u yourusername [+ command line args with db host, etc] -p | mysql -u yourusername [+ command line args with db host, etc] -p
Then you are definitely ballin' SQL style.
After you have the output SQL file though, if it's too many queries to run one after another, you can use a command line tool like "split" to split the file into a bunch of smaller files.
I would recommend testing to find the optimum batch size based on your DB's hardware/performance/etc. For example, try running 1000 updates first, if that works with no material impact try 5000, etc. Then you can split the rest of your file into chunks you are comfortable with and run them through.
Good luck!