How to set up group_concat_max_length permanently in Mysql? - mysql

How to set up group concat max length permanently in Mysql?
While setting group concat max length:
SET group_concat_max_len=102400
At the time it is set, but after restarting the server, it shows its default value 1024 again.

Edit my.cnf file and add group_concat_max_len=102400 under [mysqld] section. Then restart mysql service and that's all.

You can pass it to MySQL deamon via param:
--group_concat_max_len=xxx

Related

How to use init_connect in azure database server for mysql

Have created a mysql server which has somehow log_bin set to ON and enforce_gtid_consistency is also set to ON. Now this is causing few issues in my case and i want to turn OFF those variables.
Process i have tried
1. Create a .my.cnf file in user root (~/.my.cnf) and added
[mysqld]
enforce_gtid_consistency=OFF
Result: No changes. It doesnt take effect.
2. In azure portal in init_connect variable i have set "SET enforce_gtid_consistency=OFF;" (because there is no mention of this variable in server parameters).
Result: Cant connect to mysql after setting up value in this parameter.
Is there any way we can fix this?
May be the following doc is helpful:
https://learn.microsoft.com/en-us/azure/mysql/concepts-read-replicas#global-transaction-identifier-gtid
Once GTID is enabled, you cannot turn it back off. If you need to turn GTID OFF, please contact support.

Why mysql max_allowed_packet reset to 1m automatically

I set
SET GLOBAL max_allowed_packet=16777216;
and also
[mysqld]
max_allowed_packet = 16M
I checked the max_allowed_packet through below command
SHOW VARIABLES LIKE 'max_allowed_packet';
and the value is = 16777216
But after some days max_allowed_packet automatically reset to 1M.
i am pretty sure that your are hacked. i had the same problem for months. i opened general_log and finally found some codes:
connect root#someipaddress on
Query select 0x4D5A900..........(verylong)
Query select sys_exe('cmd /c c:/windows/nbvqc4.vbs')
.........
set global max_allowed_packet 1024
........
suggestion: change your root password.
MySQL has both GLOBAL variables and SESSION variables, as well as the my.cnf.
GLOBAL variables are initialised on startup from my.cnf, and many variables are taken from the GLOBAL value at connection time and copied into the SESSION. If you change the GLOBAL value, the SESSION keeps it's own value -- but any new sessions will take the new GLOBAL default.
It seems that you did the right thing in terms of setting the GLOBAL variable and updating my.cnf, but in your example you ran "SHOW VARIABLES" which returns the SESSION value. So it is possible you were not checking the correct value in that case. I would recommend for all future checks that you check both the global and session values to help get an idea of what is changing when.
SHOW GLOBAL VARIABLES LIKE 'max_allowed_packet';
SHOW SESSION VARIABLES LIKE 'max_allowed_packet';
For the value to change itself later, the following explanations are possible
(1) You are re-using a session that still has the old value
(2) Another connection has run SET GLOBAL max_allowed_packet, it is possible even that some uncourteous application or script is doing this
(3) The server was restarted, and the my.cnf change is not being applied as expected -- perhaps the file is in the wrong path, or the setting exists more than once in the configuration file. I would check the current Uptime in SHOW GLOBAL STATUS to understand if the server was restarted
I cannot think of any other reasons this would occur. I did check to see if the client negotiates the server-side value when you pass --max_allowed_packet but that does not seem to be the case.
Yes, someone hack the system.I changed the root password and everything working fine.
By default value of max_allowed_packet is 1M in MySQL. I believe size of your "max_allowed_packet" is exceed its upper limit. So, when you check "SHOW VARIABLES LIKE 'max_allowed_packet';" its showing some negative value.
Additionally,
You have two values of max_allowed_packet in MySQL :
one on the client side : [mysql] section, [mysqldump], [client] and more.
one on the server side : [mysqld] section.
Try setting 'Super' privilege of all users to 'N', except 1 admin user. This prevents users from changing max_allowed_packet.
We just ran into this issue and the root cause is we were hacked. Some 3rd party was running a script that was changing the value down.
Tip for those trying to figure out if a hack is a root cause for them - temporarily change your MySQL logging to include all queries. That's how we ended up finding the issue.

Intermittent mySQL crash. phpmyadmin logs me out when I click on variables tab [duplicate]

I'm running a server at my office to process some files and report the results to a remote MySQL server.
The files processing takes some time and the process dies halfway through with the following error:
2006, MySQL server has gone away
I've heard about the MySQL setting, wait_timeout, but do I need to change that on the server at my office or the remote MySQL server?
I have encountered this a number of times and I've normally found the answer to be a very low default setting of max_allowed_packet.
Raising it in /etc/my.cnf (under [mysqld]) to 8 or 16M usually fixes it. (The default in MySql 5.7 is 4194304, which is 4MB.)
[mysqld]
max_allowed_packet=16M
Note: Just create the line if it does not exist
Note: This can be set on your server as it's running.
Note: On Windows you may need to save your my.ini or my.cnf file with ANSI not UTF-8 encoding.
Use set global max_allowed_packet=104857600. This sets it to 100MB.
I had the same problem but changeing max_allowed_packet in the my.ini/my.cnf file under [mysqld] made the trick.
add a line
max_allowed_packet=500M
now restart the MySQL service once you are done.
I used following command in MySQL command-line to restore a MySQL database which size more than 7GB, and it works.
set global max_allowed_packet=268435456;
It may be easier to check if the connection exists and re-establish it if needed.
See PHP:mysqli_ping for info on that.
There are several causes for this error.
MySQL/MariaDB related:
wait_timeout - Time in seconds that the server waits for a connection to become active before closing it.
interactive_timeout - Time in seconds that the server waits for an interactive connection.
max_allowed_packet - Maximum size in bytes of a packet or a generated/intermediate string. Set as large as the largest BLOB, in multiples of 1024.
Example of my.cnf:
[mysqld]
# 8 hours
wait_timeout = 28800
# 8 hours
interactive_timeout = 28800
max_allowed_packet = 256M
Server related:
Your server has full memory - check info about RAM with free -h
Framework related:
Check settings of your framework. Django for example use CONN_MAX_AGE (see docs)
How to debug it:
Check values of MySQL/MariaDB variables.
with sql: SHOW VARIABLES LIKE '%time%';
command line: mysqladmin variables
Turn on verbosity for errors:
MariaDB: log_warnings = 4
MySQL: log_error_verbosity = 3
Check docs for more info about the error
Error: 2006 (CR_SERVER_GONE_ERROR)
Message: MySQL server has gone away
Generally you can retry connecting and then doing the query again to solve this problem - try like 3-4 times before completely giving up.
I'll assuming you are using PDO. If so then you would catch the PDO Exception, increment a counter and then try again if the counter is under a threshold.
If you have a query that is causing a timeout you can set this variable by executing:
SET ##GLOBAL.wait_timeout=300;
SET ##LOCAL.wait_timeout=300; -- OR current session only
Where 300 is the number of seconds you think the maximum time the query could take.
Further information on how to deal with Mysql connection issues.
EDIT: Two other settings you may want to also use is net_write_timeout and net_read_timeout.
In MAMP (non-pro version) I added
--max_allowed_packet=268435456
to ...\MAMP\bin\startMysql.sh
Credits and more details here
If you are using xampp server :
Go to xampp -> mysql -> bin -> my.ini
Change below parameter :
max_allowed_packet = 500M
innodb_log_file_size = 128M
This helped me a lot :)
This error is occur due to expire of wait_timeout .
Just go to mysql server check its wait_timeout :
mysql> SHOW VARIABLES LIKE 'wait_timeout'
mysql> set global wait_timeout = 600 # 10 minute or maximum wait time
out you need
http://sggoyal.blogspot.in/2015/01/2006-mysql-server-has-gone-away.html
I was getting this same error on my DigitalOcean Ubuntu server.
I tried changing the max_allowed_packet and the wait_timeout settings but neither of them fixed it.
It turns out that my server was out of RAM. I added a 1GB swap file and that fixed my problem.
Check your memory with free -h to see if that's what's causing it.
On windows those guys using xampp should use this path xampp/mysql/bin/my.ini and change max_allowed_packet(under section[mysqld])to your choice size.
e.g
max_allowed_packet=8M
Again on php.ini(xampp/php/php.ini) change upload_max_filesize the choice size.
e.g
upload_max_filesize=8M
Gave me a headache for sometime till i discovered this. Hope it helps.
It was RAM problem for me.
I was having the same problem even on a server with 12 CPU cores and 32 GB RAM. I researched more and tried to free up RAM. Here is the command I used on Ubuntu 14.04 to free up RAM:
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
And, it fixed everything. I have set it under cron to run every hour.
crontab -e
0 * * * * bash /root/ram.sh;
And, you can use this command to check how much free RAM available:
free -h
And, you will get something like this:
total used free shared buffers cached
Mem: 31G 12G 18G 59M 1.9G 973M
-/+ buffers/cache: 9.9G 21G
Swap: 8.0G 368M 7.6G
In my case it was low value of open_files_limit variable, which blocked the access of mysqld to data files.
I checked it with :
mysql> SHOW VARIABLES LIKE 'open%';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| open_files_limit | 1185 |
+------------------+-------+
1 row in set (0.00 sec)
After I changed the variable to big value, our server was alive again :
[mysqld]
open_files_limit = 100000
This generally indicates MySQL server connectivity issues or timeouts.
Can generally be solved by changing wait_timeout and max_allowed_packet in my.cnf or similar.
I would suggest these values:
wait_timeout = 28800
max_allowed_packet = 8M
If you are using the 64Bit WAMPSERVER, please search for multiple occurrences of max_allowed_packet because WAMP uses the value set under [wampmysqld64] and not the value set under [mysqldump], which for me was the issue, I was updating the wrong one. Set this to something like max_allowed_packet = 64M.
Hopefully this helps other Wampserver-users out there.
There is an easier way if you are using XAMPP.
Open the XAMPP control panel, and click on the config button in mysql section.
Now click on the my.ini and it will open in the editor. Update the max_allowed_packet to your required size.
Then restart the mysql service. Click on stop on the Mysql service click start again. Wait for a few minutes.
Then try to run your Mysql query again. Hope it will work.
It's always a good idea to check the logs of the Mysql server, for the reason why it went away.
It will tell you.
MAMP 5.3, you will not find my.cnf and adding them does not work as that max_allowed_packet is stored in variables.
One solution can be:
Go to http://localhost/phpmyadmin
Go to SQL tab
Run SHOW VARIABLES and check the values, if it is small then run with big values
Run the following query, it set max_allowed_packet to 7gb:
set global max_allowed_packet=268435456;
For some, you may need to increase the following values as well:
set global wait_timeout = 600;
set innodb_log_file_size =268435456;
For Vagrant Box, make sure you allocate enough memory to the box
config.vm.provider "virtualbox" do |vb|
vb.memory = "4096"
end
This might be a problem of your .sql file size.
If you are using xampp. Go to the xampp control panel -> Click MySql config -> Open my.ini.
Increase the packet size.
max_allowed_packet = 2M -> 10M
The unlikely scenario is you have a firewall between the client and the server that forces TCP reset into the connection.
I had that issue, and I found our corporate F5 firewall was configured to terminate inactive sessions that are idle for more than 5 mins.
Once again, this is the unlikely scenario.
uncomment the ligne below in your my.ini/my.cnf, this will split your large file into smaller portion
# binary logging format - mixed recommended
# binlog_format=mixed
TO
# binary logging format - mixed recommended
binlog_format=mixed
I found the solution to "#2006 - MySQL server has gone away" this error.
Solution is just you have to check two files
config.inc.php
config.sample.inc.php
Path of these files in windows is
C:\wamp64\apps\phpmyadmin4.6.4
In these two files the value of this:
$cfg['Servers'][$i]['host']must be 'localhost' .
In my case it was:
$cfg['Servers'][$i]['host'] = '127.0.0.1';
change it to:
"$cfg['Servers'][$i]['host']" = 'localhost';
Make sure in both:
config.inc.php
config.sample.inc.php files it must be 'localhost'.
And last set:
$cfg['Servers'][$i]['AllowNoPassword'] = true;
Then restart Wampserver.
To change phpmyadmin user name and password
You can directly change the user name and password of phpmyadmin through config.inc.php file
These two lines
$cfg['Servers'][$i]['user'] = 'root';
$cfg['Servers'][$i]['password'] = '';
Here you can give new user name and password.
After changes save the file and restart WAMP server.
I got Error 2006 message in different MySQL clients software on my Ubuntu desktop. It turned out that my JDBC driver version was too old.
I had the same problem in docker adding below setting in docker-compose.yml:
db:
image: mysql:8.0
command: --wait_timeout=800 --max_allowed_packet=256M --character-set-server=utf8 --collation-server=utf8_general_ci --default-authentication-plugin=mysql_native_password
volumes:
- ./docker/mysql/data:/var/lib/mysql
- ./docker/mysql/dump:/docker-entrypoint-initdb.d
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
I also encountered this error. But even with the increased max_allowed_packet or any increase of value in the my.cnf, the error still persists.
What I did is I troubleshoot my database:
I checked the tables where the error persists
Then I checked each row
There are rows that are okay to fetch and there are rows where the error only shows up
It seems that there are value in these rows that is causing this error
But even by selecting only the primary column, the error still shows up (SELECT primary_id FROM table)
The solution that I thought of is to reimport the database. Good thing is I have a backup of this database. But I only dropped the problematic table, then import my backup of this table. That solved my problem.
My takeaway of this problem:
Always have a backup of your database. Either manually or thru CRON job
I noticed that there are special characters in the affected rows. So when I recovered the table, I immediately changed the collation of this table from latin1_swedish_ci to utf8_general_ci
My database was working fine before then my system suddenly encountered this problem. Maybe it also has something to do with the upgrade of the MySQL database by our hosting provider. So frequent backup is a must!
Just in case this helps anyone:
I got this error when I opened and closed connections in a function which would be called from several parts of the application.
We got too many connections so we thought it might be a good idea to reuse the existing connection or throw it away and make a new one like so:
public static function getConnection($database, $host, $user, $password){
if (!self::$instance) {
return self::newConnection($database, $host, $user, $password);
} elseif ($database . $host . $user != self::$connectionDetails) {
self::$instance->query('KILL CONNECTION_ID()');
self::$instance = null;
return self::newConnection($database, $host, $user, $password);
}
return self::$instance;
}
Well turns out we've been a little too thorough with the killing and so the processes doing important things on the old connection could never finish their business.
So we dropped these lines
self::$instance->query('KILL CONNECTION_ID()');
self::$instance = null;
and as the hardware and setup of the machine allows it we increased the number of allowed connections on the server by adding
max_connections = 500
to our configuration file. This fixed our problem for now and we learned something about killing mysql connections.
For users using XAMPP, there are 2 max_allowed_packet parameters in C:\xampp\mysql\bin\my.ini.
This error happens basically for two reasons.
You have a too low RAM.
The database connection is closed when you try to connect.
You can try this code below.
# Simplification to execute an SQL string of getting a data from the database
def get(self, sql_string, sql_vars=(), debug_sql=0):
try:
self.cursor.execute(sql_string, sql_vars)
return self.cursor.fetchall()
except (AttributeError, MySQLdb.OperationalError):
self.__init__()
self.cursor.execute(sql_string, sql_vars)
return self.cursor.fetchall()
It mitigates the error whatever the reason behind it, especially for the second reason.
If it's caused by low RAM, you either have to raise database connection efficiency from the code, from the database configuration, or simply raise the RAM.
For me it helped to fix one's innodb table's corrupted index tree. I localized such a table by this command
mysqlcheck -uroot --databases databaseName
result
mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ...
as followed I was able to see only from the mysqld logs /var/log/mysqld.log which table was causing troubles.
FIL_PAGE_PREV links 2021-08-25T14:05:22.182328Z 2 [ERROR] InnoDB: Corruption of an index tree: table `database`.`tableName` index `PRIMARY`, father ptr page no 1592, child page no 1234'
The mysqlcheck command did not fix it, but helped to unveil it.
Ultimately I fixed it as followed by a regular mysql command from a mysql cli
OPTIMIZE table theCorruptedTableNameMentionedAboveInTheMysqld.log

SET GLOBAL max_allowed_packet doesn't work [duplicate]

This question already has answers here:
How to change max_allowed_packet size
(14 answers)
Closed 1 year ago.
I found out how to change the default value of max_allowed_packet in MySQL using SET GLOBAL. However, each time I used this command, the default value stayed untouched!
I used these commands:
mysql --user=root --password=mypass
mysql> SET GLOBAL max_allowed_packet=32*1024*1024;
Query OK, 0 rows affected (0.00 secs)
mysql> SHOW VARIABLES max_allowed_packet;
And then the result is max_allowed_packet = 1048576. What am I missing?
Hmmmm.. You have hit this NOT-A-BUG it seems. :)
If you change a global system variable, the value is remembered and used for new
connections until the server restarts. (To make a global system variable setting
permanent, you should set it in an option file.) The change is visible to any client that
accesses that global variable. However, the change affects the corresponding session
variable only for clients that connect after the change. The global variable change does
not affect the session variable for any client that is currently connected (not even that
of the client that issues the SET GLOBAL statement).
Refer this too. Read Shane Bester explanation.
You should change from the my.ini/my.cnf file and restart the server for the max_allowed_packet setting to take effect.
After running
set global max_allowed_packet=1000000000;
you have to restart mysql before
SHOW VARIABLES LIKE 'max_allowed_packet'
will show the new value.
I have this issue when restarting mysql through the MAC OSX system preferences and the value hadn't changed. So by logging into mysql via console
mysql -u root -p
changing it and then restarting mySql seemed to work. Might have been a OS X quirk though.
For those with a MariaDb configuration the problem could be that the max_allowed_packet variable is overwritten by a configuration file called later.
In my case I tried to import a database and the server answered me:
ERROR 2006 (HY000) at line 736: MySQL server has gone away
I discovered that the file:
/etc/mysql/mariadb.conf.d/50-server.cnf
is called later
/etc/mysql/conf.d/mysql.cnf
I tried continuously changing in the "mysql.cnf" file but the value was overwritten in "50-server.cnf".
So the solution is to enter the file
/etc/mysql/mariadb.conf.d/50-server.cnf
and instead of
"max_allowed_packet = 16M"
put the desired value as an example
"max_allowed_packet = 64M"
I came across this problem as well and in my case I have multiple versions of MySql installed.
Adding this note for anyone who might have setup MySql using homebrew on mac and are having trouble setting max_allowed_packet value in your my.cnf file.
The most key information that helped was that the my.cnf file can be present in different locations (excerpt from https://github.com/rajivkanaujia/alphaworks/wiki/Install-MySQL-using-Homebrew) -
/usr/local/etc/my.cnf --> Global Access
/usr/local/etc/my.cnf --> Global Access
/usr/local/Cellar/mysql/5.7.18/my.cnf --> Sever Level Access
~/.my.cnf --> User Level Access
Since I installed MySql 5.6 via Home brew I found it at -
/usr/local/Cellar/mysql\#5.6/5.6.43/my.cnf
Steps followed -
Update the /usr/local/Cellar/mysql\#5.6/5.6.43/my.cnf file under [mysqld] group with the necessary max_allowed_packet value -
[mysqld]
max_allowed_packet=5G
Restart mysql using brew services -
brew services restart mysql#5.6
Connect/Reconnect to the mysql shell and verify that the configuration has taken effect using -
show variables like 'max_allowed_packet';
Just a quick way to see the variable for anybody who comes across this. To get the value back you need to run
SHOW VARIABLES LIKE 'max_allowed_packet'

When using rails/activerecord to talk to a MySQL database, can I set max_allowed_packet?

MySQL has a setting called 'max_allowed_packet' which is set separately for clients and servers. On the server side it's in a config file, is it possible to set this to a non-default value on the client side when the client is rails ActiveRecord?
From the documnentation
On the client side, max_allowed_packet has a default of 1GB.
So, you just have to change it on the server side. Depending what version of MySQL you are using, you can do this on the fly on a global basis by running
SET GLOBAL max_allowed_packet=16777216
or
SET max_allowed_packet=16777216
where I assumed a 16MB max size. As you noted, you can also set the global variable in your my.cnf.
Currently, you cannot change this on a per-session basis.
mysql> SET SESSION max_allowed_packet=102400000;
ERROR 1621 (HY000): SESSION variable 'max_allowed_packet' is read-only. Use SET GLOBAL to assign the value
I dont know if could help , but you can set mysql variable at runtime .
ActiveRecord::Base.connection.execute("set global max_allowed_packet=1000000000;")