After upgrading to MariaDB 5.5 connections are being dropped after a period of inactivity - mysql

We have upgraded our database server from MySQL 5.1 to MariaDB 5.5 (5.5.40-MariaDB-1~wheezy-log).
After this upgrade, some long running processes mysql connection si being dropped.
Common scenario for those processes is:
Connect to MySQL
Run some queries
Do some heavy lifting without connecting to MySQL for at least one minute
Try to query against the original connection
Exception giving a 2600 error - MySQL server has gone away
This does happen in PHP CLI scripts (php 5.3), but also in a Ruby application (Redmine 2.5.1). It was not happening with MySQL 5.1 and there were no changes on the applications side, so it should not be app-related.
The %timeout% variables in MariaDB are:
+----------------------------+----------+
| Variable_name | Value |
+----------------------------+----------+
| connect_timeout | 5 |
| deadlock_timeout_long | 50000000 |
| deadlock_timeout_short | 10000 |
| delayed_insert_timeout | 300 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| slave_net_timeout | 3600 |
| thread_pool_idle_timeout | 60 |
| wait_timeout | 28800 |
+----------------------------+----------+
We are not using thread pooling:
+---------------------------+---------------------------+
| Variable_name | Value |
+---------------------------+---------------------------+
| thread_cache_size | 128 |
| thread_concurrency | 10 |
| thread_handling | one-thread-per-connection |
| thread_pool_idle_timeout | 60 |
| thread_pool_max_threads | 500 |
| thread_pool_oversubscribe | 3 |
| thread_pool_size | 12 |
| thread_pool_stall_limit | 500 |
| thread_stack | 294912 |
+---------------------------+---------------------------+
When the thing happens, there is also an event logged in syslog, everytime looking the same:
Dec 16 13:00:14 DB01 mysqld: 141216 13:00:14 [Warning] Aborted connection 9202885 to db: 'some_db_name' user: 'user' host: 'app' (Unknown error)
Besides that, there are also weird root account disconnection messages:
Dec 16 13:05:02 DB01 mysqld: 141216 13:05:02 [Warning] Aborted connection 9225621 to db: 'unconnected' user: 'root' host: 'localhost' (Unknown error)
Dec 16 13:10:00 DB01 mysqld: 141216 13:10:00 [Warning] Aborted connection 9218291 to db: 'unconnected' user: 'root' host: 'localhost' (Unknown error)
Dec 16 13:10:12 DB01 mysqld: 141216 13:10:12 [Warning] Aborted connection 9232561 to db: 'unconnected' user: 'root' host: 'localhost' (Unknown error)
Dec 16 13:17:01 DB01 /USR/SBIN/CRON[41343]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 16 13:20:02 DB01 mysqld: 141216 13:20:02 [Warning] Aborted connection 9248777 to db: 'unconnected' user: 'root' host: 'localhost' (Unknown error)
Dec 16 13:20:02 DB01 mysqld: 141216 13:20:02 [Warning] Aborted connection 9248788 to db: 'unconnected' user: 'root' host: 'localhost' (Unknown error)
Dec 16 13:20:12 DB01 mysqld: 141216 13:20:12 [Warning] Aborted connection 9248798 to db: 'unconnected' user: 'root' host: 'localhost' (Unknown error)
Out of those settings is there any, that should be changed to fix the weird server has gone away errors?

In the end we have found out that the DB is not the cause of exceptions dropping, as the drops are appearing also in other unrelated systems.

Related

Hikari CP SSL Exception closing inbound before receiving peer's close_notify

Since switching from Tomcat CP (spring boot 1 default) to Hikari (spring boot 2 default) we've started seeing many instances of:
EXCEPTION STACK TRACE:
** BEGIN NESTED EXCEPTION **
javax.net.ssl.SSLException
MESSAGE: closing inbound before receiving peer's close_notify
STACKTRACE:
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:645)
at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:624)
at com.mysql.cj.protocol.a.NativeProtocol.quit(NativeProtocol.java:1312)
at com.mysql.cj.NativeSession.quit(NativeSession.java:182)
at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1750)
at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:720)
at com.zaxxer.hikari.pool.PoolBase.quietlyCloseConnection(PoolBase.java:135)
at com.zaxxer.hikari.pool.HikariPool.lambda$closeConnection$1(HikariPool.java:441)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Environment
Spring Boot 2.1.1.RELEASE
Java 11
mysql-connector-java 8.0.13
HikariCP 3.2.0
Database:
RDS Aurora MySql 5.7.12 (default param group)
Configuration (Spring Boot)
Settings:
spring.datasource.hikari.transactionIsolation=TRANSACTION_REPEATABLE_READ
spring.datasource.hikari.minimumIdle=10
spring.datasource.hikari.idleTimeout=300000
spring.datasource.hikari.maximumPoolSize=20
spring.datasource.hikari.connectionTimeout=5000
spring.datasource.hikari.maxLifetime=900000
spring.datasource.hikari.validationTimeout=1000
Is there a setting which I'm missing, perhaps my idle times should be set much lower?
We have not (yet) experienced any obvious bad side effects of this, i.e. the application appears to continue running without issue, but this stacktrace appears frequently (perhaps every 4 seconds.
Database Settings
If I connect to mysql via the cli and run show variables; and grep for timeout related values, I see:
| connect_timeout | 10 |
| delayed_insert_timeout | 300 |
| have_statement_timeout | YES |
| innodb_flush_log_at_timeout | 1 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| rpl_stop_slave_timeout | 31536000 |
| slave_net_timeout | 60 |
| wait_timeout | 28800 |

postfix not substituting %s

Postfix 3.1.4 on Debian Stretch appears to be failing to substitute the address it's attempting to resolve with mysql_table.
I have /etc/postfix/sasl/smtpd.conf configured (below), but it never authenticates. When I check the MySQL log table, I see:
| 2017-08-28 23:19:54 | postfix[postfix] # localhost [127.0.0.1] | 834 | 0 | Query | START TRANSACTION |
| 2017-08-28 23:19:54 | postfix[postfix] # localhost [127.0.0.1] | 834 | 0 | Query | SELECT password FROM mailbox WHERE username = '%s' |
| 2017-08-28 23:19:54 | postfix[postfix] # localhost [127.0.0.1] | 834 | 0 | Query | SELECT password FROM mailbox WHERE username = '%s' |
| 2017-08-28 23:19:54 | postfix[postfix] # localhost [127.0.0.1] | 834 | 0 | Query | COMMIT
When %s is a postfix template, not a MySQL / MariaDB one. It should be filling in the email address, or the user here, but it appears to just be copying this from the configs verbatim.
Here is the content of /etc/postfix/sasl/smtpd.conf:
autheck_method: saslauthd
mech_list: plain login
allow_plaintext: true
auxprop_plugin: sql
sql_hostnames: 127.0.0.1
sql_user: [redacted]
sql_passwd: [redacted]
sql_database: [redacted]
sql_select: SELECT password FROM mailbox WHERE username = '%s'
log_level:10
Seems this works: select password from users where email = '%u#%r', but %s does not.

Sequel Pro or other Non-Socket connections refused in MySql 5.6.26

Ok, I have looked at about 15 answers to this question on here, and none of them work.
I am on a Mac, running OSX 10.10, MySql 5.6.26
Whenever I try to connect with Sequel Pro I get the following:
Unable to connect to host 127.0.0.1 because access was denied.
My users are in my MySql dump, with grant on all of them to .
+----------------------+----------+
| Host | User |
+----------------------+----------+
| % | macfelon |
| 127.0.0.1 | macfelon |
| 127.0.0.1 | root |
| ::1 | root |
| localhost | |
| localhost | macfelon |
| localhost | root |
| robert-macbook.local | |
| robert-macbook.local | root |
+----------------------+----------+
My port is open at 3306. It connects with the Socket option, but not on Standard.
Nothing I can find seems to resolve this for me.
Are you absolutely certain it's actually listening on port 3306? There's an answer here: https://dba.stackexchange.com/questions/108522/mac-os-x-10-10-mysql-5-6-26-accessible-from-cli-but-not-port-3306 where others are reporting that the configuration in my.cnf is being ignored, and for some reason MySQL 5.6.26 configures /Library/LaunchDaemons/com.oracle.oss.mysql.mysqld.plist to use port 3307 by default. Modifying that .plist to port 3306 solved this issue for me.

MySQL - SSL - with TLS1.2 cipher AES256-SHA256 / DHE-RSA-AES256-SHA256

I'm using MySQL with SSL with TLS1.2 cipher AES256-SHA256 / DHE-RSA-AES256-SHA256.
I have compiled MySQL with openssl. I am able to connect to MySQL over
SSL with TLS1.0 ciphers. But when I tried to connect with TLS1.2 ciphers
connection fails with error.
MySQL server version :- 5.6.23-log Source distribution
Custom OpenSSL version :- OpenSSL 1.0.1j 15 Oct 2014
Java version :- 1.8.0_40
Error thrown with TLS1.2 cipher connect
> mysql -umysql --ssl-cipher=DHE-RSA-AES256-SHA256 -T -v
ERROR 2026 (HY000): SSL connection error:
error:00000001:lib(0):func(0):reason(1)
User time 0.00, System time 0.00
Maximum resident set size 2664, Integral resident set size 0
Non-physical pagefaults 777, Physical pagefaults 0, Swaps 0
Blocks in 0 out 0, Messages in 0 out 0, Signals 0
Voluntary context switches 2, Involuntary context switches 5
Snippet of my.cnf
[client]
default-character-set=utf8
ssl=ON
ssl-ca=/home/mysql-cert/ca.pem
ssl-cert=/home/mysql-cert/client-cert.pem
ssl-key=/home/mysql-cert/client-key.pem
[mysql]
default-character-set=utf8
[mysqld]
general_log=1
ssl-cipher=DHE-RSA-AES256-SHA256
ssl-cipher=AES256-SHA256
ssl-cipher=AES256-SHA
ssl-ca=/home/mysql-cert/ca.pem
ssl-cert=/home/mysql-cert/server-cert.pem
ssl-key=/home/mysql-cert/server-key.pem
MySQL prompt snipeet with TLS1.0 cipher connected
mysql> \s
--------------
mysql Ver 14.14 Distrib 5.6.23, for Linux (x86_64) using EditLine wrapper
Connection id: 6
Current database:
Current user: root#localhost
SSL: Cipher in use is AES256-SHA
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.6.23-log Source distribution
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: latin1
Db characterset: latin1
Client characterset: utf8
Conn. characterset: utf8
UNIX socket: /tmp/mysql.sock
Uptime: 1 hour 32 min 40 sec
Threads: 1 Questions: 11 Slow queries: 0 Opens: 67 Flush tables: 1
Open tables: 60 Queries per second avg: 0.001
--------------
mysql> SHOW STATUS LIKE 'ssl%';
+--------------------------------+--------------------------+
| Variable_name | Value |
+--------------------------------+--------------------------+
| Ssl_accept_renegotiates | 0 |
| Ssl_accepts | 6 |
| Ssl_callback_cache_hits | 0 |
| Ssl_cipher | AES256-SHA |
| Ssl_cipher_list | AES256-SHA |
| Ssl_client_connects | 0 |
| Ssl_connect_renegotiates | 0 |
| Ssl_ctx_verify_depth | 18446744073709551615 |
| Ssl_ctx_verify_mode | 5 |
| Ssl_default_timeout | 7200 |
| Ssl_finished_accepts | 3 |
| Ssl_finished_connects | 0 |
| Ssl_server_not_after | Jan 23 10:29:20 2025 GMT |
| Ssl_server_not_before | Mar 17 10:29:20 2015 GMT |
| Ssl_session_cache_hits | 0 |
| Ssl_session_cache_misses | 0 |
| Ssl_session_cache_mode | SERVER |
| Ssl_session_cache_overflows | 0 |
| Ssl_session_cache_size | 128 |
| Ssl_session_cache_timeouts | 0 |
| Ssl_sessions_reused | 0 |
| Ssl_used_session_cache_entries | 0 |
| Ssl_verify_depth | 18446744073709551615 |
| Ssl_verify_mode | 5 |
| Ssl_version | TLSv1 |
+--------------------------------+--------------------------+
25 rows in set (0.00 sec)
mysql> SHOW VARIABLES LIKE '%ssl%';
+---------------+----------------------------------+
| Variable_name | Value |
+---------------+----------------------------------+
| have_openssl | YES |
| have_ssl | YES |
| ssl_ca | /home/mysql-cert/ca.pem |
| ssl_capath | |
| ssl_cert | /home/mysql-cert/server-cert.pem |
| ssl_cipher | AES256-SHA |
| ssl_crl | |
| ssl_crlpath | |
| ssl_key | /home/mysql-cert/server-key.pem |
+---------------+----------------------------------+
9 rows in set (0.00 sec)
MySQL compiled as
> cmake . -DCMAKE_PREFIX_PATH=/opt/scr-openssl/ssl/
-DWITH_SSL=/opt/scr-openssl/ssl/
-DWITH_OPENSSL=/opt/scr-openssl/ssl/bin/
-DWITH_OPENSSL_INCLUDES=/opt/scr-openssl/ssl/include/
-DWITH_OPENSSL_LIBS=/opt/scr-openssl/ssl/lib/ -DENABLE_DOWNLOADS=1
>make
>make install
Please help me out to configure MySQL to work with TLS1.2 cipher.
MySQL v5.6.23 can only support TLS 1.0. To get support for TLS 1.2, you need to upgrade to a later MySQL version and ensure that both client and server have been compiled to use OpenSSL.
You might be able to use MySQL 5.6.46, according to the MySQL documentation.
When compiled using OpenSSL 1.0.1 or higher, MySQL supports the TLSv1, TLSv1.1, and TLSv1.2 protocols as of MySQL 5.6.46, and TLS1v1 prior to 5.6.46.

How can I find the Unix process that owns a local Sleeping MySQL connection?

I'm fighting a 'Too many connections' problem with my MySQL process and I've got to the point when
mysqladmin processlist -uroot -pXXXXX results in:
+------+------------+-----------+------------+---------+------+-------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+------+------------+-----------+------------+---------+------+-------+------------------+
| 842 | svcControl | localhost | svcObjects | Sleep | 1772 | | |
| 875 | svcControl | localhost | svcObjects | Sleep | 1773 | | |
| 884 | svcControl | localhost | svcObjects | Sleep | 1770 | | |
| 896 | svcControl | localhost | svcObjects | Sleep | 1540 | | |
| 951 | svcControl | localhost | svcObjects | Sleep | 709 | | |
| 1023 | svcControl | localhost | svcObjects | Sleep | 708 | | |
| 1063 | svcControl | localhost | svcObjects | Sleep | 708 | | |
| 1069 | svcControl | localhost | svcObjects | Sleep | 708 | | |
| 1075 | svcControl | localhost | svcObjects | Sleep | 707 | | |
| 1083 | svcControl | localhost | svcObjects | Sleep | 707 | | |
| 1091 | svcControl | localhost | svcObjects | Sleep | 706 | | |
| 1097 | svcControl | localhost | svcObjects | Sleep | 706 | | |
| 1107 | svcControl | localhost | svcObjects | Sleep | 705 | | |
| 1112 | svcControl | localhost | svcObjects | Sleep | 702 | | |
| 1120 | svcControl | localhost | svcObjects | Sleep | 704 | | |
| 1127 | svcControl | localhost | svcObjects | Sleep | 704 | | |
| 1136 | svcControl | localhost | svcObjects | Sleep | 704 | | |
| 1141 | svcControl | localhost | svcObjects | Sleep | 644 | | |
| 1202 | svcControl | localhost | svcObjects | Sleep | 268 | | |
| 1222 | svcView | localhost | svcObjects | Sleep | 8 | | |
| 1224 | svcControl | localhost | svcObjects | Sleep | 280 | | |
| 1225 | svcControl | localhost | svcObjects | Sleep | 8 | | |
| 1265 | root | localhost | | Query | 0 | | show processlist |
+------+------------+-----------+------------+---------+------+-------+------------------+
So, clearly, I'm leaking connections somewhere, but I can't figure out where. I've been looking for sources that tell me how to get from a connection ID to a Unix process ID, but most of the approaches expect the connections to be from remote servers, allowing you to use netstat to debug them. Either that, or they require you to modify your codebase to log each attempted connection, which seems a little like overkill.
So... are there any approaches that anyone knows of? Any files that this sleeping connection will have open, so that I can use fuser, for instance? Or... is it possible that these are zombie connections that have no owning process?
In response to answers below
If I run ps -ef | grep mysql, I see:
root 5960 1 0 Oct14 ? 00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --user=mysql
mysql 6007 5960 0 Oct14 ? 00:24:12 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
Running the suggest commands, I get:
# fuser -u /var/lib/mysql/mysql.sock
/var/lib/mysql/mysql.sock: 6007(mysql)
# lsof -p 5960
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
mysqld_sa 5960 root cwd DIR 8,1 4096 2 /
mysqld_sa 5960 root rtd DIR 8,1 4096 2 /
mysqld_sa 5960 root txt REG 8,1 735004 117216 /bin/bash
mysqld_sa 5960 root mem REG 8,1 50848 90123 /lib/libnss_files-2.5.so
mysqld_sa 5960 root mem REG 8,1 129832 87812 /lib/ld-2.5.so
mysqld_sa 5960 root mem REG 8,1 1689388 90858 /lib/libc-2.5.so
mysqld_sa 5960 root mem REG 8,1 20668 90874 /lib/libdl-2.5.so
mysqld_sa 5960 root mem REG 8,1 13276 91815 /lib/libtermcap.so.2.0.8
mysqld_sa 5960 root mem REG 253,18 56418144 656494 /usr/lib/locale/locale-archive
mysqld_sa 5960 root mem REG 253,18 25462 1573763 /usr/lib/gconv/gconv-modules.cache
mysqld_sa 5960 root 0r CHR 1,3 1432 /dev/null
mysqld_sa 5960 root 1w CHR 1,3 1432 /dev/null
mysqld_sa 5960 root 2w CHR 1,3 1432 /dev/null
mysqld_sa 5960 root 255r REG 253,18 13077 1181028 /usr/bin/mysqld_safe
lsof -p 6007
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
mysqld 6007 mysql cwd DIR 253,19 4096 33005 /var/lib/mysql
mysqld 6007 mysql rtd DIR 8,1 4096 2 /
mysqld 6007 mysql txt REG 253,18 7413588 1998921 /usr/libexec/mysqld
mysqld 6007 mysql mem REG 8,1 1296932 91720 /lib/libcrypto.so.0.9.8e
mysqld 6007 mysql mem REG 253,18 612180 1514570 /usr/lib/libkrb5.so.3.3
mysqld 6007 mysql mem REG 8,1 293108 91722 /lib/libssl.so.0.9.8e
mysqld 6007 mysql mem REG 8,1 3200288 87758 /lib/libnss_ldap-2.5.so
mysqld 6007 mysql mem REG 8,1 21948 90121 /lib/libnss_dns-2.5.so
mysqld 6007 mysql mem REG 8,1 129832 87812 /lib/ld-2.5.so
mysqld 6007 mysql mem REG 8,1 1689388 90858 /lib/libc-2.5.so
mysqld 6007 mysql mem REG 8,1 216544 90877 /lib/libm-2.5.so
mysqld 6007 mysql mem REG 8,1 20668 90874 /lib/libdl-2.5.so
mysqld 6007 mysql mem REG 8,1 137908 90866 /lib/libpthread-2.5.so
mysqld 6007 mysql mem REG 253,18 75284 1514584 /usr/lib/libz.so.1.2.3
mysqld 6007 mysql mem REG 8,1 48156 90867 /lib/librt-2.5.so
mysqld 6007 mysql mem REG 8,1 50848 90123 /lib/libnss_files-2.5.so
mysqld 6007 mysql mem REG 8,1 245376 90881 /lib/libsepol.so.1
mysqld 6007 mysql mem REG 8,1 93508 91676 /lib/libselinux.so.1
mysqld 6007 mysql mem REG 8,1 46636 90847 /lib/libgcc_s-4.1.2-20080825.so.1
mysqld 6007 mysql mem REG 253,18 936908 1514611 /usr/lib/libstdc++.so.6.0.8
mysqld 6007 mysql mem REG 8,1 109740 90873 /lib/libnsl-2.5.so
mysqld 6007 mysql mem REG 8,1 80636 90879 /lib/libresolv-2.5.so
mysqld 6007 mysql mem REG 8,1 7748 91684 /lib/libcom_err.so.2.1
mysqld 6007 mysql mem REG 253,18 190712 1514583 /usr/lib/libgssapi_krb5.so.2.2
mysqld 6007 mysql mem REG 253,18 157304 1514569 /usr/lib/libk5crypto.so.3.1
mysqld 6007 mysql mem REG 8,1 8072 90878 /lib/libkeyutils-1.2.so
mysqld 6007 mysql mem REG 253,18 33712 1509918 /usr/lib/libkrb5support.so.0.1
mysqld 6007 mysql mem REG 8,1 45432 90876 /lib/libcrypt-2.5.so
mysqld 6007 mysql 0r CHR 1,3 1432 /dev/null
mysqld 6007 mysql 1w REG 253,19 251337 163863 /var/log/mysqld.log
mysqld 6007 mysql 2w REG 253,19 251337 163863 /var/log/mysqld.log
mysqld 6007 mysql 3u IPv4 26825288 TCP monstermunch.ssd.hursley.ibm.com:59850->hurgsa.hursley.uk.ibm.com:ldap (CLOSE_WAIT)
mysqld 6007 mysql 4uW REG 253,19 161480704 33142 /var/lib/mysql/ibdata1
mysqld 6007 mysql 5u REG 253,16 0 13 /tmp/ibmydRNj (deleted)
mysqld 6007 mysql 6u REG 253,16 20 14 /tmp/ib59yuxj (deleted)
mysqld 6007 mysql 7u REG 253,16 0 15 /tmp/ib0Vp8gj (deleted)
mysqld 6007 mysql 8u REG 253,16 0 16 /tmp/ibjkCG1i (deleted)
mysqld 6007 mysql 9uW REG 253,19 5242880 33143 /var/lib/mysql/ib_logfile0
mysqld 6007 mysql 10uW REG 253,19 5242880 33144 /var/lib/mysql/ib_logfile1
mysqld 6007 mysql 11u IPv4 26825294 TCP *:mysql (LISTEN)
mysqld 6007 mysql 12u REG 253,16 0 17 /tmp/ibKpCJ1i (deleted)
mysqld 6007 mysql 13u unix 0xea432900 26825295 /var/lib/mysql/mysql.sock
mysqld 6007 mysql 14u REG 253,19 30720 65557 <DB Table>
mysqld 6007 mysql 15u unix 0xf5188c80 30430332 /var/lib/mysql/mysql.sock
mysqld 6007 mysql 16u REG 253,19 3072 98468 <DB Table>
mysqld 6007 mysql 17u unix 0xf5925680 30433174 /var/lib/mysql/mysql.sock
mysqld 6007 mysql 18u REG 253,19 1024 65605 <DB Table>
mysqld 6007 mysql 19u unix 0xf5188880 30430336 /var/lib/mysql/mysql.sock
mysqld 6007 mysql 20u REG 253,19 40 98469 <DB Table>
mysqld 6007 mysql 21u unix 0xf5925280 30433176 /var/lib/mysql/mysql.sock
mysqld 6007 mysql 22u REG 253,19 39344 65558 <DB Table>
mysqld 6007 mysql 23u REG 253,19 1024 65623 <DB Table>
mysqld 6007 mysql 24u REG 253,19 0 65624 <DB Table>
mysqld 6007 mysql 25u unix 0xf51f6880 30430339 /var/lib/mysql/mysql.sock
In the output above, <DB Table> is my edit.
I may be missing something, but I don't see anything there that is a process of mine holding open a connection... unless those deleted /tmp files are key.
If the connections were made over a local socket, you might have some luck with sudo fuser -u /tmp/mysql.sock. You could also find mysql's process ID (let's call it $MYSQL_PID) and then run sudo lsof -p $MYSQL_PID, which will include all open sockets.
If you can, change the way you are connecting to the mysql server by using 127.0.0.1 instead of localhost.
By doing that, a tcp/ip connection will be used and you will have in the processlist the port number that you can use to trace back to the process.
A tcp/ip connection is slower than a unix file socket connection, but it'll do the trick.
references :
http://dev.mysql.com/doc/refman/5.1/en/can-not-connect-to-server.html
http://dev.mysql.com/doc/refman/5.1/en/connecting.html
Check which application uses "svcObjects" database with user "svcControl". This will lead you to the problem root I guess.