MySQL Query with LARGE number of records gets Killed - mysql

Question

I run the following query from my shell :
mysql -h my-host.net -u myuser -p -e "SELECT component_id, parent_component_id FROM myschema.components comp INNER JOIN my_second_schema.component_parents related_comp ON comp.id = related_comp.component_id ORDER BY component_id;" > /tmp/IT_component_parents.txt
The query runs for a LONG time and then gets KILLED.
However if I add LIMIT 1000, then the query runs till the end and output is written in file.
I further investigated and found (using COUNT(*)) that the total number of records that would be returned are 239553163.
Some information about my server is here:
MySQL 5.5.27
+----------------------------+----------+
| Variable_name | Value |
+----------------------------+----------+
| connect_timeout | 10 |
| delayed_insert_timeout | 300 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| slave_net_timeout | 3600 |
| wait_timeout | 28800 |
+----------------------------+----------+
Here's STATE of the query as I monitored :
copying to tmp table on disk
sorting results
sending data
writing to net
sending data
writing to net
sending data
writing to net
sending data ...
KILLED
Any guesses what's wrong here ?

Answer 1

The mysql client probably runs out of memory.
Use the --quick option to not buffer results in memory.

Answer 2

What is wrong is that you are returning 239 553 163 rows of data! Don't be surprised it it takes a lot of time to process. Actually, the longest part might very well be sending the result set back to your client.
Reuduce the result set (do you really need all these rows?). Or try to output the data in smaller batches:
mysql -h my-host.net -u myuser -p -e "SELECT ... LIMIT 10000, 0" >> dump.txt
mysql -h my-host.net -u myuser -p -e "SELECT ... LIMIT 10000, 10000" >> dump.txt

Answer 3

Assuming you mean 8 hours when you say a long time, the value 28800 for your wait_timeout causes the connection to drop with no further activity in 28,800 seconds, i.e. 8 hours. If you can't optimize the statement to run in less than 8 hours, you should increase this value.
See this page for further information on the wait_timeout variable.
The interactive_timeout variable is used for interactive client connections, so if you run long queries from an interactive session, that's the one you need look at.

Answer 4

You may want to utilize OUTFILE mechanizm if you are going to dump large amounts of data. That or mysql_dump will be much more efficient (and OUTFILE got the benefit of not locking-down the table).

Answer 5

You said in a comment that your MySQL instance is on RDS. This means you can't be running the query from the same host, since you can't log into an RDS host. I guess you might be doing this query over the WAN from your local network.
You're most likely having trouble because of a slow network. Your process state frequently showing "writing to net" makes me think this is your bottleneck.
Your bottleneck might also be the sorting. Your sort is writing to a temp table, and that can take a long time for a result set that large. Can you skip the ORDER BY?
Even so, I wouldn't expect the query to be killed even if it runs for 3100 seconds or more. I wonder if your DBA has some periodic job killing long-running queries, like pt-kill. Ask your DBA.
To reduce network transfer time, you could try using the compression protocol. You can use the --compress or -C flags to the mysql client for this (see https://dev.mysql.com/doc/refman/5.7/en/mysql-command-options.html#option_mysql_compress)
On a slow network, compression can help. For example, read about some comparisons here: https://www.percona.com/blog/2007/12/20/large-result-sets-vs-compression-protocol/
Another alternative is to run the query from an EC2 spot instance running in the same AZ as your RDS instance. The network between those two instances will be a lot faster, so it won't delay your data transfer. Save the query output to a file on the EC2 spot instance.
Once the query result is saved on your EC2 instance, you can download it to your local machine, using scp or something, which should be more tolerant of slow networks.