How can i improve mysql query if include point, comma etc..
My data size is 3GB and 12,000,000 row.
My System Information
Time of this report: 4/20/2018, 13:55:16
Operating System: Windows Server 2008 R2 Standard 64-bit (6.1, Build 7600) (7600.win7_gdr.120401-1505)
Language: Turkish (Regional Setting: Turkish)
System Manufacturer: VMware, Inc.
System Model: VMware Virtual Platform
BIOS: PhoenixBIOS 4.0 Release 6.0
Processor: Intel(R) Xeon(R) CPU E3-1245 V2 # 3.40GHz (8 CPUs), ~3.4GHz
Memory: 16384MB RAM
Available OS Memory: 16384MB RAM
Page File: 8402MB used, 24362MB available
Windows Dir: C:\Windows
DirectX Version: DirectX 11
DX Setup Parameters: Not found
User DPI Setting: Using System DPI
System DPI Setting: 96 DPI (100 percent)
DWM DPI Scaling: Disabled
DxDiag Version: 6.01.7600.16385 32bit Unicode
MY.ini file is:
[client]
port=3306
[mysql]
no-beep=
[mysqld]
port=3306
datadir=C:/ProgramData/MySQL/MySQL Server 5.7/Data
default-storage-engine=INNODB
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
log-output=FILE
general-log=0
general_log_file="WIN-FFJIGKKSAPK.log"
slow-query-log=1
slow_query_log_file="WIN-FFJIGKKSAPK-slow.log"
long_query_time=10
log-error="WIN-FFJIGKKSAPK.err"
server-id=1
secure-file-priv="C:/ProgramData/MySQL/MySQL Server 5.7/Uploads"
max_connections = 2000
query_cache_size = 0
table_open_cache=39390
open_files_limit=78780
tmp_table_size = 20M
thread_cache_size = 100
myisam_max_sort_file_size=100G
myisam_sort_buffer_size = 64M
key_buffer_size = 32M
read_buffer_size=64K
read_rnd_buffer_size=256K
innodb_flush_log_at_trx_commit = 0
innodb_log_buffer_size=8M
innodb_buffer_pool_size = 6000M
innodb_log_file_size = 256M
innodb_thread_concurrency=0
innodb_autoextend_increment=64
innodb_write_io_threads = 16
innodb_read_io_threads = 16
innodb_io_capacity=500
innodb_buffer_pool_instances=8
innodb_concurrency_tickets=5000
innodb_old_blocks_time=1000
innodb_open_files=300
innodb_stats_on_metadata=0
innodb_file_per_table=1
innodb_checksum_algorithm=0
back_log=80
flush_time=0
join_buffer_size=256K
max_allowed_packet = 16M
query_cache_type=0
sort_buffer_size=256K
table_definition_cache=1400
binlog_row_event_max_size=8K
sync_master_info=10000
sync_relay_log=10000
sync_relay_log_info=10000
ft_min_word_len = 3
max_connect_errors = 1000000
expire_logs_days = 14
max_heap_table_size = 32M
myisam-recover-options = FORCE
slave-net-timeout = 60
sync_binlog = 1
skip-name-resolve
please help me i want to speed up mysql queries. thank you
Related
I am facing issue the kernal is again and agin killing mysql
And due to this tomcat stops arbitrarily.
My db size is approx 700 MB.
My application is fetching much more data on regular basis.
JVM memory usages
=================
-Xms3000m
-Xmx12000m
-XX:MaxMetaspaceSize=4000m
my.cnf
======
[mysqld]
performance-schema=0
max_connections = 150
max_user_connections = 100
query_cache_size = 64M
skip-external-locking
key_buffer_size = 64M
max_allowed_packet=268435456
table_open_cache = 256
sort_buffer_size = 1M
net_buffer_length = 16K
read_buffer_size = 1M
read_rnd_buffer_size = 1M
myisam_sort_buffer_size = 32M
innodb_file_per_table
tmp_table_size = 64M
max_heap_table_size = 64M
thread_cache_size = 8
query_cache_type = on
#innodb_force_recovery = 1
performance-schema=0
# This setting allows the use of asynchronous I/O in InnoDB.
# The following files track usage of this resource:
# - /proc/sys/fs/aio-max-nr
# - /proc/sys/fs/aio-nr
# Default limit is 65536, of which a single instance of mysql uses 2661 out of the box
innodb_use_native_aio = 1
default_storage_engine = MyISAM
open_files_limit=50000
default-storage-engine=MyISAM
Ehcache enabled in my application.
Any solution will be appreciated.
Since we are rookies on this, we are having a hard time trying to setup a Wordpress server to support around 200K monthly visiting users. We have two boxes (details below).
The issue here is when the users start accessing the server, MySQL CPU goes to 149%. Here is a copy of the apache2.conf and the mysql.cnf files. Our MySQL database is InnoDB. Any suggestions on how to fix it?
1. Apache Server
Configuration
8 GB Memory / 25 GB Disk + 200 GB / Ubuntu 16.04
apache.conf excerpt
Timeout 40
KeepAlive On
MaxKeepAliveRequests 500
KeepAliveTimeout 2
2. MySQL Server
Configuration
8 GB Memory / 25 GB Disk + 50 GB / Ubuntu 16.04
my.cnf file
root#mundo-mysql01-c-2-4gib-nyc3-01:/etc/mysql# cat my.cnf
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
general_log_file = /var/log/mysql/mysql.log
general_log = 0
# CACHES AND LIMITS #
open_files_limit = 65535
table_definition_cache = 1024
table_open_cache = 2048
# MyISAM Settings
max_connections = 300
key_buffer_size = 256M
myisam_sort_buffer_size = 32M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 16M
thread_cache_size = 32
wait_timeout = 300
connect_timeout = 300
tmp_table_size = 32M
max_heap_table_size = 128M
max_allowed_packet = 128M
max_connect_errors = 10000
read_rnd_buffer_size = 1M
bulk_insert_buffer_size = 1M
query_cache_limit = 4M
query_cache_size = 128M
query_cache_type = 1
query_prealloc_size = 65536
query_alloc_block_size = 131072
low_priority_updates = 1
max_write_lock_count = 1
# * InnoDB
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 256M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 2048M
Hi are trying to optimize MySQL for a huge traffic web server, we are using Apache 2.4 , and MySQL 5.6 version.
We have put these parameters in MySQL configuration file
cat /etc/my.cnf
[mysqld]
skip-external-locking
key_buffer = 500M
max_allowed_packet=64M
max_connections = 2000
max_user_connections = 50000
wait_timeout=180
connect_timeout=120
table_open_cache=4096
table_definition_cache=4096
myisam_sort_buffer_size = 64M
thread_cache_size = 128
query_cache_size = 190M
query_cache_limit = 3M
open_files_limit=10000
log_error = /var/log/mysql/error.log
default-storage-engine=MyISAM
[mysqld]
log-bin=mysql-bin
binlog-do-db=******
server-id=1
[mysql]
no-auto-rehash
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
replicate-do-db=*****
auto_increment_increment = 2
auto_increment_offset = 1
replicate-same-server-id = 0
slave-skip-errors=all
[mysqld]
slave-skip-errors=1062
skip-slave-start
query_cache_type=1
[mysqld]
innodb_lock_wait_timeout=120
slave_compressed_protocol=1
query_cache_size = 190M
The QC is inefficient when it is big; limit to about 50M.
slave-skip-errors=1062
This is a flavor of "sweeping bugs under the rug".
default-storage-engine=MyISAM
passé.
key_buffer = 500M
If you do stick with MyISAM, and this server is running just MySQL, then use 1000M.
server-id=1
Make sure each server has a different value.
But that is only configuration. If you have slow queries, let's see them and try to improve them.
what best config for my.cnf file i have big size sql
Total processors: 4
Processor #1
Vendor
GenuineIntel
Name
Intel(R) Core(TM) i7-3930K CPU # 3.20GHz
Speed
3200.058 MHz
Cache
12288 KB
Memory Information
Memory: 14243308k/15728640k available
There is no straight forward answer of this question as one configuration can be suitable for one but not for other..
If you are using innodb engine then You can start with below configuration initially and can optimize as per your requirement/problems-
default_storage_engine = InnoDB
innodb_file_per_table
lower_case_table_names = 1
wait_timeout = 120
innodb_buffer_pool_size = 10G
key_buffer_size = 20M
max_allowed_packet = 64M
max_connections = 100
query_cache_type = 1
query_cache_size = 20M
query_cache_limit = 2M
read_buffer_size = 2M
sort_buffer_size = 2M
join_buffer_size = 2M
tmp_table_size = 512M
max_heap_table_size = 512M
for the mysql optimization, I will suggest you please try with MySQL tuner script. You can try it with the following command
wget https://launchpad.net/mysql-tuning-primer/trunk/1.6-r1/+download/tuning-primer.sh
chmod 755 tuning-primer.sh
./tuning-primer.sh
To start with, I'm relatively new to web development.
Under reasonable load (500-700 Q/s) the server is restarting frequently (sometimes every hour), and I get the follow in my syslog each time it happens. I don't get any output to mysql.err or mysql/error.log
This is my first real mysql setup, so I'm sure there is lots that could be improved, but I'm surprised that the server is actually going down so often. The machine has 4GB of ram. Thanks for any suggestions or help!
Ubuntu 10.04, mysql is from the lucid packages.
Ver 14.14 Distrib 5.1.41, for debian-linux-gnu (x86_64) using readline 6.1
Oct 10 01:45:33 host init: mysql main process (1199) terminated with status 1
Oct 10 01:45:33 host init: mysql main process ended, respawning
Oct 10 01:45:42 host /etc/mysql/debian-start[1644]: Upgrading MySQL tables if necessary.
Oct 10 01:45:42 host /etc/mysql/debian-start[1648]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Oct 10 01:45:42 host /etc/mysql/debian-start[1648]: Looking for 'mysql' as: /usr/bin/mysql
Oct 10 01:45:42 host /etc/mysql/debian-start[1648]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Oct 10 01:45:42 host /etc/mysql/debian-start[1648]: This installation of MySQL is already upgraded to 5.1.41, use --force if you still need to run mysql_upgrade
Oct 10 01:45:42 host /etc/mysql/debian-start[1656]: Checking for insecure root accounts.
Oct 10 01:45:43 host /etc/mysql/debian-start[1672]: Triggering myisam-recover for all MyISAM tables
Here's my my.cnf:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld]
user = mysql
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
skip-external-locking
bind-address = host
back_log = 50
max_connections = 100
max_connect_errors = 10
table_open_cache = 2048
max_allowed_packet = 16M
binlog_cache_size = 1M
max_heap_table_size = 64M
sort_buffer_size = 8M
join_buffer_size = 8M
thread_cache_size = 8
thread_concurrency = 8
query_cache_size = 64M
query_cache_limit = 2M
ft_min_word_len = 4
default-storage-engine = innodb
default-character-set = utf8
thread_stack = 192K
transaction_isolation = REPEATABLE-READ
tmp_table_size = 32M
log-bin=trent-bin
binlog_format=mixed
slow_query_log
long_query_time = 2
server-id = 1
key_buffer_size = 32M
read_buffer_size = 2M
read_rnd_buffer_size = 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 128M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover
innodb_additional_mem_pool_size = 16M
innodb_buffer_pool_size = 2800M
innodb_data_file_path = ibdata1:100M:autoextend
innodb_file_per_table
innodb_file_io_threads = 4
innodb_thread_concurrency = 8
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 4M
innodb_log_file_size = 256M
innodb_log_files_in_group = 3
innodb_max_dirty_pages_pct = 90
innodb_flush_method=O_DIRECT
innodb_lock_wait_timeout = 50
ssl-ca=/etc/mysql/cacert.pem
ssl-cert=/etc/mysql/server-cert.pem
ssl-key=/etc/mysql/server-key.pem
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[myisamchk]
key_buffer_size = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M
[mysqlhotcopy]
interactive-timeout
[mysqld_safe]
open-files-limit = 8192
I have seen mysql behave this way a few times, there are many potential causes.. : to investigatem I would check
1) are there any custom functions that you have written that might be memory leaks? I once saw a median function, custom written on mysql 5, lead to restarts if it was run by two simultaneous queries...
2) if you optimize the tables, does this reduce the restart frequency?
3) are therea ny corrupt tables in the db ?