I have an error that has plagued for quite some time and still no solution
Whenever I receive :
2018-07-16 11:21:27,815 [Thread-4] WARN spi.SqlExceptionHelper - SQL
Error: 0, SQLState: 08S01 2018-07-16 11:21:27,815 [Thread-4] ERROR
spi.SqlExceptionHelper - Communications link failure
The last packet successfully received from the server was 197,301
milliseconds ago. The last packet sent successfully to the server was
197,301 milliseconds ago.
The application never seems to reconnect. Even though this is my datasource configuration (it's an external configuration):
dataSource.dbCreate=none
dataSource.driverClassName=com.mysql.jdbc.Driver
dataSource.url=jdbc:mysql://****/*****?autoReconnect=true&failOverReadOnly=false&maxReconnects=10
dataSource.username=***
dataSource.password=******
dataSource.properties.maxActive = 50
dataSource.properties.maxIdle = 25
dataSource.properties.minIdle = 1
dataSource.properties.initialSize = 1
dataSource.properties.numTestsPerEvictionRun = 3
dataSource.properties.maxWait = 10000
dataSource.properties.testOnBorrow = true
dataSource.properties.testWhileIdle = true
dataSource.properties.testOnReturn = true
dataSource.properties.validationQuery = "select now()"
dataSource.properties.minEvictableIdleTimeMillis = 300000
dataSource.properties.timeBetweenEvictionRunsMillis = 300000
I saw several questions on this issue, but nothing that actually solved it for me.
NOTE : I just want to emphasis that this occurs rarely, so i'm not trying to solve the issue of link connection failure, I just want that when it happens to reconnect after retry so the system won't hang (another C process handles this adequately so I suspect the issue is with Hibernate or Grails)
These are my mysql config
[client]
port=3306
socket=/data/db/mysql/mysql.sock
[mysqld]
port=3306
datadir=/data/db/mysql
socket=/data/db/mysql/mysql.sock
log-bin=/data/db/mysql/mysql-bin
binlog_format=row
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
server-id=1
max_connections=250
[mysqld_safe]
log-bin=/data/db/mysql/mysql-bin
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
server-id=1
max_connections=250
Some other info
| wait_timeout | 28800
| tx_isolation | REPEATABLE-READ
| thread_handling | one-thread-per-connection
| Table_locks_immediate | 15090998 |
| Table_locks_waited | 7453 |
| table_definition_cache | 256
| table_lock_wait_timeout | 50
| table_open_cache | 64
| table_type | MyISAM
|Bytes_received | 2844964922 |
| Bytes_sent | 5623597485 |
| Com_select | 15003437 |
| Com_set_option | 35905 |
| Handler_commit | 10842183 |
| Handler_delete | 76 |
| Handler_discover | 0 |
| Handler_prepare | 410224 |
| Handler_read_first | 90118 |
| Handler_read_key | 47411397 |
| Handler_read_next | 6354573941 |
| Handler_read_prev | 2900331388 |
| Handler_read_rnd | 449 |
| Handler_read_rnd_next | 2183682054024 |
| Handler_rollback | 44 | |
This may cause because of initial datasource pool size and your mysql timeout config.
I use following configuration:
dataSource {
pooled = true
jmxExport = true
dbCreate = 'validate'
driverClassName = 'com.mysql.jdbc.Driver'
dialect = 'org.hibernate.dialect.MySQL5InnoDBDialect'
properties {
jmxEnabled = true
maxActive = 50
maxIdle = 25
minIdle = 5
initialSize = 5
minEvictableIdleTimeMillis = 60000
timeBetweenEvictionRunsMillis = 5000
maxWait = 10000
maxAge = 10 * 60000
numTestsPerEvictionRun = 3
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
ignoreExceptionOnPreLoad = true
validationQuery = "SELECT 1"
validationQueryTimeout = 3
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
abandonWhenPercentageFull = 100
removeAbandonedTimeout = 120
removeAbandoned = true
logAbandoned = false
dbProperties {
autoReconnect = true
jdbcCompliantTruncation = false
zeroDateTimeBehavior = 'convertToNull'
}
}
}
along with:
dataSource {
url = "jdbc:mysql://localhost:3306/dbname?useSSL=false&useUnicode=yes&characterEncoding=UTF-8&autoReconnect=true"
username = "user"
password = "xxxxx"
logSql = true
}
Does the application work fine on another database?
Kindly check your mysql settings and share what is in the following lines:
sudo nano /etc/mysql/my.cnf
wait_timeout = 28800
interactive_timeout = 28800
I am thinking this might not be a grails issue.
Related
I have MySQL 5.7 installed on two servers and I configured both servers' mysqld.cnf file as shown below
Server1 configuration.
[mysqld]
bind-address = 192.168.0.1
server_id = 1
log_bin = /var/log/mysql/mysql-bin.log
log_bin_index = /var/log/mysql/mysql-bin.log.index
relay_log = /var/log/mysql/mysql-relay-bin
relay_log_index = /var/log/mysql/mysql-relay-bin.index
expire_logs_days = 10
max_binlog_size = 100M
log_slave_updates = 1
auto-increment-increment = 2
auto-increment-offset = 1
Server2 configuration.
[mysqld]
bind-address = 192.168.0.2
server_id = 2
log_bin = /var/log/mysql/mysql-bin.log
log_bin_index = /var/log/mysql/mysql-bin.log.index
relay_log = /var/log/mysql/mysql-relay-bin
relay_log_index = /var/log/mysql/mysql-relay-bin.index
expire_logs_days = 10
max_binlog_size = 100M
log_slave_updates = 1
auto-increment-increment = 2
auto-increment-offset = 2
Then when I get the out put from Server1 with the below:
mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000006 | 459 | | | |
+------------------+----------+--------------+------------------+-------------------+
Then I go to run and get the output as per below at Server2:
mysql> STOP SLAVE;
mysql> CHANGE MASTER TO master_host='192.168.0.1', master_port=3306,
master_user='replicauser',
master_password='somestrongpassword', master_log_file='mysql-bin.000006',
master_log_pos=459;
mysql> START SLAVE;
mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000007 | 154 | | | |
+------------------+----------+--------------+------------------+-------------------+
So what is the missing to complete my solution ?
Edit:
The missing part was just restart for both servers
We run a CentOS DirectAdmin install with MariaDB 10.2.14 where on Magento is installed.
Currently our DB locks very often when a process runs, so all other processes are waiting until the current process finishes. This is quite a problem, because for example, also the adding to cart process is waiting in that case and people can not order.
How can we prevent the DB from being locked so long and solve this issue?
Server:
6x Intel Xeon
32GB RAM
500GB SSD
My.cnf:
[mysqld]
bind-address = 127.0.0.1
local-infile=0
innodb_file_per_table=1
innodb_file_format=barracuda
slow_query_log = 1
slow_query_log_file=/var/log/mysql-log-slow-queries.log
key_buffer = 250M
key_buffer_size = 250M
max_allowed_packet = 128M
table_cache = 512
sort_buffer_size = 7M
read_buffer_size = 7M
read_rnd_buffer_size = 7M
myisam_sort_buffer_size = 64M
tmp_table_size = 190M
query_cache_type = 1
query_cache_size = 220M
query_cache_limit = 512M
thread_cache_size = 150
max_connections = 225
wait_timeout = 300
innodb_buffer_pool_size = 7G
max_heap_table_size =180M
innodb_log_buffer_size = 36M
join_buffer_size = 32M
innodb_buffer_pool_instances = 7
long_query_time = 15
table_definition_cache = 4K
open_files_limit = 60K
table_open_cache = 50767
innodb_log_file_size= 128M
innodb_lock_wait_timeout = 700
Suggestions to consider for your my.cnf [mysqld] section
The following lead with # to disable or REMOVE to allow defaults to serve your requirements
Some of these are already mentioned by Rick James in earlier comment.
. key_buffer
. key_buffer_size
. table_cache
. sort_buffer_size
. read_buffer_size
. read_rnd_buffer_size
. MyISAM_sort_buffer_size
. join_buffer_size
. long_query_time
. innodb_lock_wait_timeout
make these changes or add lines to your my.cnf for
query_cache_type=0 # from 1 to turn OFF QC and conserve CPU cycles
query_cache_size=0 # from 220M to conserve RAM for more useful work
query_cache_limit=0 # from 512M to conserve RAM for more useful work
thread_cache_size=100 # from 150 V8 refman suggested CAP to avoid OOM
innodb_lru_scan_depth=100 # from 1024 to minimum to conserve CPU every SECOND
innodb_flush_neighbors=0 # from 1 no need to waste CPU cycles when using SSD
innodb_io_capacity_max=10000 # from 2000 since you have SSD
innodb_io_capacity=5000 # from 200 to use more of your SSD capability
for additional assistance, please check my profile, clk Network profile for contact info.
MySQL will wait a certain amount of time for the lock to be removed before it gives up and throws that error. If you are able to track when you are seeing these error messages down to any consistent time of the day, you should look at what else the server is doing at that time - for instance is a database backup running. By doing this you should be able to narrow down possibilities for what processes could be creating the lock although it's not always that straight forward to do - likely to be a bit of trial and error.
Sometimes deadlock issues can be caused on the database.The reason behind this issue is if you are running a lot of custom scripts and killing the scripts before the database connection gets chance to close.
If you can login to MySQL from CLI and run the following command
SHOW PROCESSLIST;
you will get the following output
+———+—————–+——————-+—————–+———+——+——-+——————+———–+—————+———–+
| Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined | Rows_read |
+———+—————–+——————-+—————–+———+——+——-+——————+———–+—————+———–+
| 6794372 | db_user| 111.11.0.65:21532 | db_name| Sleep | 3800 | | NULL | 0 | 0 | 0 |
| 6794475 | db_user| 111.11.0.65:27488 | db_name| Sleep | 3757 | | NULL | 0 | 0 | 0 |
| 6794550 | db_user| 111.11.0.65:32670 | db_name| Sleep | 3731 | | NULL | 0 | 0 | 0 |
| 6794797 | db_user| 111.11.0.65:47424 | db_name | Sleep | 3639 | | NULL | 0 | 0 | 0 |
| 6794909 | db_user| 111.11.0.65:56029 | db_name| Sleep | 3591 | | NULL | 0 | 0 | 0 |
| 6794981 | db_user| 111.11.0.65:59201 | db_name| Sleep | 3567 | | NULL | 0 | 0 | 0 |
| 6795096 | db_user| 111.11.0.65:2390 | db_name| Sleep | 3529 | | NULL | 0 | 0 | 0 |
| 6795270 | db_user| 111.11.0.65:10125 | db_name | Sleep | 3473 | | NULL | 0 | 0 | 0 |
| 6795402 | db_user| 111.11.0.65:18407 | db_name| Sleep | 3424 | | NULL | 0 | 0 | 0 |
| 6795701 | db_user| 111.11.0.65:35679 | db_name| Sleep | 3330 | | NULL | 0 | 0 | 0 |
| 6800436 | db_user| 111.11.0.65:57815 | db_name| Sleep | 1860 | | NULL | 0 | 0 | 0 |
| 6806227 | db_user| 111.11.0.67:20650 | db_name| Sleep | 188 | | NULL | 1 | 0 | 0 |
| 6806589 | db_user| 111.11.0.65:36618 | db_name| Query | 0 | NULL | SHOW PROCESSLIST | 0 | 0 | 0 |
| 6806742 | db_user| 111.11.0.75:38717 | db_name| Sleep | 0 | | NULL | 0 | 0 | 0 |
| 6806744 | db_user| 111.11.0.75:38819 | db_name| Sleep | 0 | | NULL | 61 | 61 | 61 |
+———+—————–+——————-+—————–+———+——+——-+——————+———–+—————+———–+
15 rows in set (0.00 sec)
You can see as an example
6794372 the command is sleep and time is 3800. This is preventing other operations.
These processes should be killed 1 by 1 using the command.
KILL 6794372;
Once you have killed all the sleeping connections, things should start working as normal again
These are deprecated; their names have changed. Remove them:
key_buffer = 250M
table_cache = 512
These are higher than they should be:
key_buffer_size = 250M
query_cache_size = 220M
thread_cache_size = 150
long_query_time = 15
table_definition_cache = 4K
table_open_cache = 50767
innodb_lock_wait_timeout = 700
The last one may be the villain. It implies that you have some loooong transactions. This is a design flaw in your code. Find a way to make the transactions shorter. If you need help, describe what you are doing to us.
I feel that 5 is plenty long for a transaction.
Do you sometimes get this?
ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
I recently upgraded MySQL 5.1 to 5.7.8rc.
I have a unique issue with "sending data" status during profiling. It takes more time than expected for every complex or union queries (some time for simple queries also). I tried with best optimized as well as original query. Googled and tried all the possible configurations but no luck.
Query performs supper fast in MySQL 5.1, but not in 5.7.
Also tried with table optimization, analyze, repair etc..
some details for reference:
OS: Centos 6.9 64 bit
MySQL: 5.7.8 rc
CPU: 4
RAM: 64 GB
Data volume: 450 GB
Type: Dedicated VM
Query Profiling:
Status Duration
starting 0.000515
checking permissions 0.000023
checking permissions 0.000016
checking permissions 0.000014
checking permissions 0.000014
checking permissions 0.000014
checking permissions 0.000014
checking permissions 0.000014
checking permissions 0.000014
checking permissions 0.000016
checking permissions 0.000014
checking permissions 0.000019
Opening tables 0.000079
init 0.000325
System lock 0.000068
optimizing 0.000079
statistics 0.001888
preparing 0.000151
Creating tmp table 0.000128
Sorting result 0.000027
optimizing 0.000034
statistics 0.000064
preparing 0.000047
Creating tmp table 0.000047
Sorting for group 0.000030
optimizing 0.000018
statistics 0.000022
preparing 0.000022
Creating tmp table 0.000043
Sorting for order 0.000023
executing 0.000016
Sending data 4.015596
Creating sort index 0.004766
executing 0.000010
Sending data 0.000159
executing 0.000008
Sending data 0.000025
Creating sort index 0.000349
end 0.000010
query end 0.000024
removing tmp table 0.000013
query end 0.000011
removing tmp table 0.000008
query end 0.000009
removing tmp table 0.000011
query end 0.000009
removing tmp table 0.000007
query end 0.000010
removing tmp table 0.000008
query end 0.000006
closing tables 0.000026
freeing items 0.000039
removing tmp table 0.000009
freeing items 0.000067
logging slow query 0.000047
cleaning up 0.000042
Execution Plan:
+----+--------------+-------------+------------+--------+-----------------------------------------------------+--------------------+---------+--------------------------------------+------+----------+------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------+-------------+------------+--------+-----------------------------------------------------+--------------------+---------+--------------------------------------+------+----------+------------------------------------+
| 1 | PRIMARY | TIGM_GRP | NULL | index | PRIMARY,FLD_PARENT_GROUP_ID | PRIMARY | 2 | NULL | 87 | 0.33 | Using where |
| 1 | PRIMARY | TIPLD | NULL | range | FLD_PRICE_LEVEL_ID | FLD_PRICE_LEVEL_ID | 3 | NULL | 26 | 10.00 | Using index condition; Using where |
| 1 | PRIMARY | TIPLM | NULL | eq_ref | PRIMARY | PRIMARY | 2 | TIPLD.FLD_PRICE_LEVEL_ID | 1 | 10.00 | Using where |
| 1 | PRIMARY | TIGL | NULL | ref | FLD_GROUP_ID,fld_item_id | FLD_GROUP_ID | 2 | TIGM_GRP.FLD_GROUP_ID | 404 | 100.00 | NULL |
| 1 | PRIMARY | TIM | NULL | eq_ref | PRIMARY,FLD_ITEM_TYPE,FLD_ADDON_ID,FLD_PRODUCT_TYPE | PRIMARY | 4 | TIGL.FLD_ITEM_ID | 1 | 50.00 | Using where |
| 1 | PRIMARY | TSIAM | NULL | eq_ref | PRIMARY | PRIMARY | 4 | TIM.FLD_ADDON_ID | 1 | 10.00 | Using where |
| 1 | PRIMARY | TSIARM | NULL | ref | FLD_ADDON_ID | FLD_ADDON_ID | 5 | TIM.FLD_ADDON_ID | 1 | 10.00 | Using index condition; Using where |
| 1 | PRIMARY | TIPD | NULL | ref | FLD_ITEM_ID,FLD_PRICE_LEVEL_ID | FLD_ITEM_ID | 2 | TIM.FLD_ITEM_ID | 3 | 1.35 | Using index condition; Using where |
| 1 | PRIMARY | TIGM_PARENT | NULL | eq_ref | PRIMARY | PRIMARY | 2 | TIGM_GRP.FLD_PARENT_GROUP_ID | 1 | 100.00 | Using index |
| 2 | UNION | TIGM_GRP | NULL | index | PRIMARY,FLD_PARENT_GROUP_ID | PRIMARY | 2 | NULL | 87 | 0.06 | Using where |
| 2 | UNION | TIGM_PARENT | NULL | eq_ref | PRIMARY | PRIMARY | 2 | TIGM_GRP.FLD_PARENT_GROUP_ID | 1 | 100.00 | Using index |
|NULL| UNION RESULT | <union1,2> | NULL | ALL | NULL | NULL | NULL | NULL | NULL | NULL | Using temporary; Using filesort |
+----+--------------+-------------+------------+--------+-----------------------------------------------------+--------------------+---------+--------------------------------------+------+----------+------------------------------------+
my.cnf:
[mysql]
# CLIENT #######################################################################
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
# GENERAL ######################################################################
user = mysql
port = 3306
socket = /var/lib/mysql/mysql.sock
server_id = 32108
default_storage_engine = InnoDB
pid_file = /var/run/mysqld/mysqld.pid
optimizer_prune_level = 0
optimizer_search_depth = 0
max_length_for_sort_data = 8388608 #New
net_buffer_length = 1048576 #New
back_log = 80
symbolic-links = 0
log_bin_trust_function_creators = 1
net_read_timeout = 10 #90
net_write_timeout = 10 #120
net_retry_count = 30
thread_stack = 512K #192K
long_query_time = 10
tmpdir = /tmp
# MyISAM #######################################################################
key_buffer_size = 64M
read_buffer_size = 32M
read_rnd_buffer_size = 32M
bulk_insert_buffer_size = 16M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 1G
myisam_repair_threads = 1
memlock
max_allowed_packet = 32M
max_connect_errors = 100
sql_mode = STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
sysdate-is-now = 1
explicit_defaults_for_timestamp = 1
innodb = FORCE
# Password policy disabled as per communication 05-Sep2017
validate_password = OFF
# DATA STORAGE ##################################################################
datadir = /var/lib/mysql/
# BINARY LOGGING ################################################################
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 14
sync-binlog = 1
# REPLICATION ###################################################################
skip-slave-start = 1
relay-log = /var/log/mysql-realy-logs/relay-bin
slave-skip-errors = 1062 #,1053,1032,1237,1146
slave-net-timeout = 60
relay_log_purge = 1
# CACHES AND LIMITS #############################################################
tmp-table-size = 256M
max-heap-table-size = 256M
query_cache_min_res_unit = 12288 #8192 #New
query-cache-type = 1
query-cache-size = 32M #102400 #0 #256M
max-connections = 150
thread-cache-size = 10 #-1 #Auto resized
open-files-limit = 65535
table_definition_cache = 2000
table_open_cache = 4096 #3092
table_open_cache_instances = 4
sort_buffer_size = 128M
join_buffer_size = 512M
binlog_cache_size = 16M
query_cache_limit = 4M
# INNODB ########################################################################
innodb_fast_shutdown = 1
innodb_flush_method = O_DIRECT
innodb_log_group_home_dir = /mysql_redo_logs/mysql_redo_logs
innodb_log_files_in_group = 2
innodb_log_file_size = 1G
innodb_flush_log_at_trx_commit = 2
innodb_file_per_table = 1 #ON
innodb_buffer_pool_size = 32G #20G
innodb_buffer_pool_instances = 32
innodb_log_buffer_size = 64M
innodb_adaptive_hash_index = 1 #ON
innodb_thread_concurrency = 300 # "0" is default and means infinite (as and when needed). #250 #32
innodb_thread_sleep_delay = 1
innodb_flush_neighbors = 1
innodb_sync_array_size = 832 # Default is 768
skip-innodb_doublewrite #New
innodb_page_cleaners = 32 # Must be =innodb_buffer_pool_instances
innodb_sort_buffer_size = 512M
innodb_read_io_threads = 64
innodb_write_io_threads = 16
#innodb_concurrency_tickets = 429496729
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 10 #80
innodb_compression_level = 0 #New
innodb_lru_scan_depth = 512 #1024 #Default
# LOGGING #######################################################################
log_error = /var/lib/mysql/mysql-error.log
log_queries_not_using_indexes = 1
slow_query_log = 1
log_error_verbosity = 3
[mysqld_safe]
open-files-limit = 4096
malloc-lib = /usr/lib64/libtcmalloc_minimal.so.4
[mysqldump]
quick
max_allowed_packet = 64M
So far I have tried most of the options, currently I started the service in caching mode for quick response.
Can you please help me fix this "sending data" delay issue
Everyone
I think I figured out the issue.
1) 5.6 the optimization engine and plan generated is a result mixture of meta-data and user query (non relational joins). engine trusts user and get best plan.
But 5.7 does not trust soft relations between the tables though they are indexed.
2) 5.7 expects explicit data integrity and do not believe on user.
POC:
I created the same tables involved with hard FK references, wala ! found a miraculous response. the same query which executed without FK took 6.085 Sec (sending data),
and the query with data integrity enforced took 0.05 Sec (with no change in indexing).
So I think 5.7 expects strong data integrity constraints explicitly.
A lot of Optimizer changes happened between 5.1 and 5.7. For one thing, ICP is new; I see it several times in the EXPLAIN. We need to see the query to discuss further. If possible, get the EXPLAIN from 5.1.
Meanwhile, here are some minor comments on my.cnf:
thread_stack = 512K #192K
Generally, it is not useful to increase this setting.
optimizer_prune_level = 0
optimizer_search_depth = 0
What prompted you to set to 0?
innodb = FORCE
deprecated in 5.7.5; suggest removing before it becomes an error.
slave-skip-errors = 1062 #,1053,1032,1237,1146
Sweeping gremlins under the rug -- you will stub your toes on they lump.
innodb_buffer_pool_size = 32G #20G
Unless you have some large apps on the same machine, 44G might be a little better.
innodb_page_cleaners = 32 # Must be =innodb_buffer_pool_instances
re the comment -- Technically, it is auto-limited to pool_instances; see https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_page_cleaners (Thanks, WilsonHauck)
log_queries_not_using_indexes = 1
In my opinion, this clutters the slowlog. The interesting entries are those that exceed long_query_time. Do you have any interesting queries in the slowlog?
I am trying to load a file in to database using Ruby. Its a large file about 15 Mb ... it copied the records properly for some time .... but after copying few records, there is no error but it does not insert records in to database ......... and when I connect to Msql prompt in a separate console ... i get an error :
mysql> desc testdb2.test_descriptions;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 52
after this i am able to connect to Mysql database ....... and it is now again the scripts starts writing the records to the database .....
Is there any way to maintain the connection with the database while the app is running ?
I am not sure if its a kind of time out issue or something .... please correct me ....
def simulate_datasets
Log.initialize
data_folders = ["Jun_06_2013"];
data_folders.each do |data_folder|
add(data_folder);
end
render :text => Log.dump
end
def add (data_folder)
#dataset = Dataset.initialize
#dataset.created_at=Date.new(2013,06,06)
#dataset.save
current_root = "script/datasets/"+data_folder+"/"
strip_string = "/development/A/"
population_time = {}
total_time = 0
clusters = Cluster.find(:all, :order=>"created_at DESC");
if clusters.empty?
Log.info "No Clusters found"
Cluster.initialize
clusters = Cluster.find(:all, :order=>"created_at DESC");
end
clusters.each do |cluster|
cluster_path = cluster.path
root = current_root + cluster.name+'/'
total_time += populate_file_or_folder(root+"fileListWithMLintMetrics.txt", cluster_path)
end
end
I am using populate_file_or_folder method to populate to the database
mysql> show variables like '%time%';
+----------------------------+-------------------+
| Variable_name | Value |
+----------------------------+-------------------+
| connect_timeout | 10 |
| datetime_format | %Y-%m-%d %H:%i:%s |
| delayed_insert_timeout | 300 |
| flush_time | 0 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lc_time_names | en_US |
| long_query_time | 10.000000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| slave_net_timeout | 3600 |
| slow_launch_time | 2 |
| system_time_zone | EDT |
| table_lock_wait_timeout | 50 |
| time_format | %H:%i:%s |
| time_zone | SYSTEM |
| timed_mutexes | OFF |
| timestamp | 1372869659 |
| wait_timeout | 28800 |
+----------------------------+-------------------+
20 rows in set (0.00 sec)
def self.populate_file_or_folder(fileName, cluster_path)
counter = 0
# Reading directly from the CSV library
CSV.foreach(fileName) do |line|
counter = counter+1
completePath = line[0]
completePath = cluster_path+ '/'+completePath
newStructure = FileOrFolder.new
newStructure.fullpath = path
pathbits = path.split('/')
newStructure.name = pathbits.last
newStructure.save
end
end
I was running benchmark test in mysql server using mysqlslap. But sometimes the last thread in the pool is not executing and closing the connection. It keeps in sleep state.
mysql> show full processlist;
+-----+------+-----------------------+-----------+---------+------+-------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-----+------+-----------------------+-----------+---------+------+-------+-----------------------+
| 2 | root | 122.178.231.xxx:15832 | mysqlslap | Sleep | 740 | | NULL |
| 106 | root | localhost | NULL | Query | 0 | NULL | show full processlist |
+-----+------+-----------------------+-----------+---------+------+-------+-----------------------+
2 rows in set (0.00 sec)
Any idea? It happens often :(
This was the command I executed.
mysqlslap -h IP_address -u root --auto-generate-sql --concurrency=100
-pPASSWORD -vvv --number-of-queries=2500
my.cnf file is as follows,
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
skip-bdb
max_connections = 500
query_cache_type = 1
query_cache_size = 128M
sort_buffer_size = 64M
key_buffer_size = 425M
query_cache_limit = 1024M
thread_cache_size = 4M
innodb_buffer_pool_size = 1360M
innodb_log_file_size = 512M
long_query_time = 5
innodb_thread_concurrency = 8
innodb_lock_wait_timeout = 300
max_allowed_packet = 128M
net_read_timeout = 240
net_write_timeout = 240
table_lock_wait_timeout =240
connect_timeout = 30
log-warnings = 2
log = /var/log/mysqld.log
log-error = /var/log/mysqld.error.log
log-slow-queries = /var/log/mysql_slow_queries.log