Related
I have a mysql log file that regularly goes over 30gb, this sucks when you realise that your server is full because of this file. I need a simple solution to limit this file to about 1gb, i don't need logs that run that long, and i'd rather avoid this problem in the future.
Any ideas? Thanks
To specify it in the my.cnf file, backup your current my.cnf file (always recommended), stop slave, stop the MySQL server and place the following option:
# relay log restrictions
relay-log-space-limit=15G
Then save and quit the file and start MySQL. Unless you configured differently, MySQL will automatically start the slave thread.
I tried to import a large sql file through phpMyAdmin...But it kept showing error
'MySql server has gone away'
What to do?
As stated here:
Two most common reasons (and fixes) for the MySQL server has gone away
(error 2006) are:
Server timed out and closed the connection. How to fix:
check that wait_timeout variable in your mysqld’s my.cnf configuration file is large enough. On Debian: sudo nano
/etc/mysql/my.cnf, set wait_timeout = 600 seconds (you can
tweak/decrease this value when error 2006 is gone), then sudo
/etc/init.d/mysql restart. I didn't check, but the default value for
wait_timeout might be around 28800 seconds (8 hours).
Server dropped an incorrect or too large packet. If mysqld gets a packet that is too large or incorrect, it assumes that something has
gone wrong with the client and closes the connection. You can increase
the maximal packet size limit by increasing the value of
max_allowed_packet in my.cnf file. On Debian: sudo nano
/etc/mysql/my.cnf, set max_allowed_packet = 64M (you can
tweak/decrease this value when error 2006 is gone), then sudo
/etc/init.d/mysql restart.
Edit:
Notice that MySQL option files do not have their commands already available as comments (like in php.ini for instance). So you must type any change/tweak in my.cnf or my.ini and place them in mysql/data directory or in any of the other paths, under the proper group of options such as [client], [myslqd], etc. For example:
[mysqld]
wait_timeout = 600
max_allowed_packet = 64M
Then restart the server. To get their values, type in the mysql client:
> select ##wait_timeout;
> select ##max_allowed_packet;
For me this solution didn't work out so I executed
SET GLOBAL max_allowed_packet=1073741824;
in my SQL client.
If not able to change this with MYSql service running, you should stop the service and change the variable in "my.ini" file.
For example:
max_allowed_packet=20M
If you are working on XAMPP then you can fix the MySQL Server has gone away issue with following changes..
open your my.ini file
my.ini location is (D:\xampp\mysql\bin\my.ini)
change the following variable values
max_allowed_packet = 64M
innodb_lock_wait_timeout = 500
If you are running with default values then you have a lot of room to optimize your mysql configuration.
The first step I recommend is to increase the max_allowed_packet to 128M.
Then download the MySQL Tuning Primer script and run it. It will provide recommendations to several facets of your config for better performance.
Also look into adjusting your timeout values both in MySQL and PHP.
How big (file size) is the file you are importing and are you able to import the file using the mysql command line client instead of PHPMyAdmin?
If you are using MAMP on OS X, you will need to change the max_allowed_packet value in the template for MySQL.
You can find it at: File > Edit template > MySQL my.cnf
Then just search for max_allowed_packet, change the value and
save.
I had this error and other related ones, when I imported at 16 GB SQL file. For me, editing my.ini and setting the following (based on several different posts) in the [mysqld] section:
max_allowed_packet = 110M
innodb_buffer_pool_size=511M
innodb_log_file_size=500M
innodb_log_buffer_size = 800M
net_read_timeout = 600
net_write_timeout = 600
If you are running under Windows, go to the control panel, services, and look at the details for MySQL and you will see where my.ini is. Then after you edit and save my.ini, restart the mysql service (or restart the computer).
If you are using HeidiSQL, you can also set some or all of these using that.
I solved my issue with this short /etc/mysql/my.cnf file :
[mysqld]
wait_timeout = 600
max_allowed_packet = 100M
The other reason this can happen is running out of memory. Check /var/log/messages and make sure that your my.cnf is not set up to cause mysqld to allocate more memory than your machine has.
Your mysqld process can actually be killed by the kernel and then re-started by the "safe_mysqld" process without you realizing it.
Use top and watch the memory allocation while it's running to see what your headroom is.
make a backup of my.cnf before changing it.
I got same issue with
$image_base64 = base64_encode(file_get_contents($_FILES['file']['tmp_name']) );
$image = 'data:image/jpeg;base64,'.$image_base64;
$query = "insert into images(image) values('".$image."')";
mysqli_query($con,$query);
In \xampp\mysql\bin\my.ini file of phpmyadmin we get only
[mysqldump]
max_allowed_packet=110M
which is just for mysqldump -u root -p dbname . I resolved my issue by replacing above code with
max_allowed_packet=110M
[mysqldump]
max_allowed_packet=110M
I updated "max_allowed_packet" to 1024M, but it still wasn't working. It turns out my deployment script was running:
mysql --max_allowed_packet=512M --database=mydb -u root < .\db\db.sql
Be sure to explicitly specify a bigger number from the command line if you are donig it this way.
If your data includes BLOB data:
Note that an import of data from the command line seems to choke on BLOB data, resulting in the 'MySQL server has gone away' error.
To avoid this, re-create the mysqldump but with the --hex-blob flag:
http://dev.mysql.com/doc/refman/5.7/en/mysqldump.html#option_mysqldump_hex-blob
which will write out the data file with hex values rather than binary amongst other text.
PhpMyAdmin also has the option "Dump binary columns in hexadecimal notation (for example, "abc" becomes 0x616263)" which works nicely.
Note that there is a long-standing bug (as of December 2015) which means that GEOM columns are not converted:
Back up a table with a GEOMETRY column using mysqldump?
so using a program like PhpMyAdmin seems to be the only workaround (the option noted above does correctly convert GEOM columns).
If it takes a long time to fail, then enlarge the wait_timeout variable.
If it fails right away, enlarge the max_allowed_packet variable; it it still doesn't work, make sure the command is valid SQL. Mine had unescaped quotes which screwed everything up.
Also, if feasible, consider limiting the number of inserts of a single SQL command to, say, 1000. You can create a script that creates multiple statements out of a single one by reintroducing the INSERT... part every n inserts.
i got a similar error.. to solve this just open my.ini file..here at line no 36 change the value of maximum allowed packet size ie. max_allowed_packet = 20M
Make sure mysqld process does not restart because of service managers like systemd.
I had this problem in vagrant with centos 7. Configuration tweaks didn't help. Turned out it was systemd which killed mysqld service every time when it took too much memory.
I had similar error today when duplicating database (MySQL server has gone away...), but when I tried to restart mysql.server restart I got error
ERROR! The server quit without updating PID ...
This is how I solved it:
I opened up Applications/Utilities/ and ran Activity Monitor
quit mysqld
then was able to solve the error problem with
mysql.server restart
I am doing some large calculations which involves the mysql connection to stay long time and with heavy data. i was facing this "Mysql go away issue". So i tried t optimize the queries but that doen't helped me then i increased the mysql variables limit which is set to a lower value by default.
wait_timeout
max_allowed_packet
To the limit what ever suits to you it should be the Any Number * 1024(Bytes). you can login to terminal using 'mysql -u username - p' command and can check and change for these variable limits.
For GoDaddy shared hosting
On GoDaddy shared hosting accounts, it is tricky to tweak the PHP.ini etc files. However, there is another way and it just worked perfectly for me. (I just successfully uploaded a 3.8Mb .sql text file, containing 3100 rows and 145 cols. Using the IMPORT command in phpMyAdmin, I was getting the dreaded MySQL server has gone away error, and no further information.)
I found that Matt Butcher had the right answer. Like Matt, I had tried all kinds of tricks, from exporting MySQL databases in bite-sized chunks, to writing scripts that break large imports into smaller ones. But here is what worked:
(1) CPANEL ---> FILES (group) ---> BACKUP
(2a) Under "Partial Backups" heading...
(2b) Under "Download a MySQL Database Backup"
(2c) Choose your database and download a backup (this step optional, but wise)
(3a) Directly to the right of 2b, under heading "Restore a MySQL Database Backup"
(3b) Choose the .SQL import file from your local drive
(3c) True happiness will be yours (shortly....) Mine took about 5 seconds
I was able to use this method to import a single table. Nothing else in my database was affected -- but that is what step (2) above is intended to protect against.
Notes:
a. If you are unsure how to create a .SQL import file, use phpMyAdmin to export a table and modify that file structure.
SOURCE:
Matt Butcher 2010 Article
If increasing max_allowed_packet doesn't help.
I was getting the same error as you when importing a .sql file into my database via Sequel Pro.
The error still persisted after upping the max_allowed_packet to 512M so I ran the import in the command line instead with:
mysql --verbose -u root -p DatabaseName < MySQL.sql
It gave the following error:
ASCII '\0' appeared in the statement, but this is not allowed unless option --binary-mode is enabled
I found a couple helpful StackOverflow questions:
Enable binary mode while restoring a Database from an SQL dump
Mysql ERROR: ASCII '\0' while importing sql file on linux server
In my case, my .sql file was a little corrupt or something. The MySQL dump we get comes in two zip files that need to be concatenated together and then unzipped. I think the unzipping was interrupted initially, leaving the file with some odd characters and encodings. Getting a fresh MySQL dump and unzipping it properly worked for me.
Just wanted to add this here in case others find that increasing the max_allowed_packet variable was not helping.
None of the solutions regarding packet size or timeouts made any difference for me. I needed to disable ssl
mysql -u -p -hmyhost.com --disable-ssl db < file.sql
https://dev.mysql.com/doc/refman/5.7/en/encrypted-connections.html
What I want to do is pretty simple. I would like to have sync for www and mysql/data folder sync.
I have successful make the change, first I point www folder to dropbox, and change the my.ini
datadir=C:/Dropbox/web/mysql/data
(note: I know some ppls might comment, it is not good to have the mysql data sync, and try to use remote mysql database, but my problem is I do not have internet connection all the time, or my internet connection is pretty bad / slow), I just like localhost speed :)
Now my problem is, inside my C:/Dropbox/web/mysql/data there is a lot mysql-bin.0000xy file. How I can remove it, and totally disable the log in mysql.
I had tried.
login to root in mysql run
SHOW VARIABLES;
SET sql_log_bin=0;
SET GLOBAL expire_logs_days=10;
FLUSH LOGS;
PURGE BINARY LOGS TO 'mysql-bin.000001';
PURGE BINARY LOGS BEFORE '2011-12-31 23:59:59';
The log files still appear inside C:/Dropbox/web/mysql/data
Next I tried this
SHOW BINARY LOGS;
Error Code : 1381
You are not using binary logging
What I want just get rid for the log file inside my mysql data folder, so I can save the space, and dropbox do not need to sync it.
http://systems.takizo.com/2009/08/23/how-to-remove-mysql-binary-log/
To clean up Binary Log on Master Server
shell> mysql -u username -p
mysql> RESET MASTER;
A client that has the SUPER privilege can disable binary logging of its own statements by using a
SET sql_log_bin = 'OFF'
statement.
I've installed MySQL on Windows 7. When I'm trying to start MySQL service I'm getting error 1067: The process terminated unexpectedly. Log message:
101111 22:27:11 [Note] Plugin 'FEDERATED' is disabled.
C:\Program Files\MySQL\MySQL Server 5.1\bin\mysqld: Table 'mysql.plugin' doesn't exist
101111 22:27:11 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
InnoDB: Log scan progressed past the checkpoint lsn 0 37356
101111 22:27:11 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 0 44233
101111 22:27:11 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
101111 22:27:12 InnoDB: Started; log sequence number 0 44233
101111 22:27:12 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.host' doesn't exist
What can I do about it?
The solution was to install MySQL without spaces in installation path.
Windows 7, 64-bit
One more thing that prevents the mysqld windows service from running is if you have mysqld.exe already running (but not as a service) and occupying port 3306. When the service tries to start and sees that port 3306 is already taken, it fails.
Just open up the windows task manager and look for "mysqld.exe" under the Processes tab. If you see it, kill it and then try to start the service again.
c:\> net start [servicename]
example: c:\> net start MySQL
My issue happened right after a power failure. I got the error 1067 The process terminated unexpectedly. MySQL needless to say did not start. The answer was simple
Open mysql path\data
Remove (delete) both ib_logfile0 and ib_logfile1.
Start the service
The solution to the problem for me was looking in my install directory, finding the /data folder, and copying it's content to the data folder that was specified in my .ini/.cnf configuration file.
I just encountered a similar issue and I found that it is a permission problem. When I copy the database "data", I have to add the permission for "NETWORK SERVICE"
I had the same problem. In my case, it was "user error" (although the Windows installer should have been smarter about it and prevented me from committing such an error).
During installation, if you make changes to the default installation paths, make sure you use the same paths for both the "Server data files" on the Custom Setup screen and then later in the "InnoDB Tablespace Settings" during the "MySQL Server Instance Configuration Wizard"
I had a problem changing the datadir in my.ini for Windows 7.
I wanted the data to be stored on a different drive and I was moving this data from another PC by copying the whole folder. I changed datadir to desired drive and saved the my.ini file with no problems.
But mysql would not start. I opened my.ini file again and it appeared to have been changed.
Then, I noticed the date on the my.ini had not changed. So I had to change the security privileges to give me write access to it.
This time when I saved it, the date changed and mysql started up access to all the correct data.
Before messing with too much things, please check the user the service is trying to run as. In my case it was NETWORK this one did not have write permissions to some locations where it was needed. Changing the user to Local System Account did the trick. If the event viewer shows any error like "Can't create test file C:\Program Files\MySQL\MySQL Server 5.6\data\XXX.lower-test", there is a high probability for this solution to work.
Good luck!
I have mysql data folder replaced by a windows directory junction.
I suspect ib_logfile0/1 and/or ibdata1 is corrupted.
Just try to delete those files and computername.err. Then restart mysql service.
That's what I did, with success.
Copying ibdata1 files, after a full reinstallation of mysql, to the junction dir and replacing dir by the junction, restarting mysql, was not enough.
You have to let mysql rebuild those files.
Also check if all dirs which you wrote in the my.ini exists.
My problem was that tmpdir doeesn`t exist so MySQL daemon falls with error 1067.
[mysqld]
port= 3306
tmpdir = "C:/tmp"
In this case C:/tmp must exists.
Check the file "C:\Program Files\MySQL\MySQL Server 5.1\my.ini"
The datadir line in my.ini should specify a path. Check the contents of that datadir path. Does it contain a folder named "mysql" and another folder named "test"?
If not, here are two choices:
Change the datadir line in my.ini to the correct location. This will probably be C:\ProgramData\MySQL\MySQL Server 5.1\data
Clean out the existing contents of your datadir path. Copy the contents of the C:\ProgramData\MySQL\MySQL Server 5.1\data to your datadir path. Restarting the mysql service should rebuild your empty database.
In my instance it had nothing to do with spaces in the file name. I used the MSI installer custom configuration and opted to exclude the default databases, assuming it was just something like Northwind/Adventureworks. Nope, it includes the core MySql system database... once I added that to the installation it worked.
in my case innodb_data_home_dir was no longer correct because I had shuffled some drive letters around when I added a new drive to my system
In my case, I have unzipped XAMPP ina a second disk on my installation, "F". When I tried to start as a service, the return was "1067". The solution was to edit my.ini, inserting the "f:" drive in the files lines. It solved the problem.
The problem look like there is no data in the directory.
copy at least the mysql directory either from your previous directory or from C:\Program Files\MySQL\MySQL Server 5.5\data .
And then try again.
I had the same error and it was caused by non standard characters in the log files path.
In order to fix that I found my.ini config file (in my case C:\ProgramData\MySQL\MySQL Server 5.6\my.ini) and modified keys slow_query_log_file and log-error.
After that I managed to start MySQL service succesfully.
In my case, I had chosen a custom path for my log-files in the MySQL installer. I had put the log-files in my user-folder C:\Users\%MY_USERACCOUNT%\Documents\mysql-logs, and by default, NETWORK SERVICE (or any other non-administrator useraccount in Windows) does not have access to a user's folder.
I fixed this by changing security options for the log-folder, giving modify rights read/write/modify rights to NETWORK SERVICE.
I also get log with Table 'mysql.plugin' doesn't exist
if install MYSQL Server 5.1 by 'msiexec.exe'
DataDir I put as C:\MYSQL\MySQL_Server_5_1\data\
but to my surprise was create data in a C:\MYSQL\MySQL_Server_5_1\data\data
There are was add word data . So I change my.ini file from
datadir="C:/MySQL/MySQL_Server_5_1/Data/" .
to the
datadir="C:/MySQL/MySQL_Server_5_1/Data/data"
and then I can use
net start MYSQL51
and then mysqld.exe run and appear in a Task Manager
when looked at mysql log (.err file under data folder), i could see the following
21:41:47 UTC - mysqld got exception 0xc0000005 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
I realized i was starting the service while i plugged usb. To be honest, the problem was resolved after i restarted my machine followed by restarting the service. In addition i removed ib_logfile0 and ib_logfile1 files before my restart. Though the event logger indicated "InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html for information
Blockquote
about forcing recovery.For more information, see Help and Support Center at http://www.mysql.com", i do not think so because i never changed any configurations.
Experienced the same error, below is the reason and solution that worked for me for mysql-5.7.14-winx64
reason: DATA folder to have some default folders and files which were missing
solution: delete everything from DATA folder, i assume its a fresh installation so backup anything that you need if at all. Then run this from the command prompt and it will create required files and folders
"mysqld --initialize --console"
now run "mysqld" and it should work well.
In my case, I had another MySQL version installed and running.
I found this by going into the mysql_error.log file.
I fix this by going to services and stopping the running MySQL version and setting up to a manual, and starting the mysql needed.
...an old one... anyway I had the same issue with MariaDB
In my case most pathes contain special characters like: #
Wrapping pathes in my.ini in double quotes made the trick - e.g.
datadir="C:/#windata64/db/MariaDB/data"
In my case, in order to delete a heavy schema from mysql server, just went to C:\ProgramData\MySQL\MySQL Server 5.7\Data and deleted relevant folder. But it was not being deleted because mysqld.exe was preventing it. so I stopped mysqld.exe, deleted the folder and then all the schemas went disappeared from the list in mysql workbench. No matter how much I tried to restart mysql service, it didnt unless I restored that folder from junk.
Hope it helps someone who tried the same shortcut as I did.
I run MariaDB (MySQL compatible) on two machines locally. I'm not sure what prompted the error and nothing I tried worked. So I stopped the service, deleted everything in MariaDB's directory (except the data directory) and copied the files from my secondary machine and everything is working well enough as far as I can tell.
For a live server it'd be a bit different and a super-guru might be able to add an insight comment (e.g. something outside of the data directory might have something to do with preventing data corruption or indexes in example?). I would just stop the service and copy the entire directory once every month or so and then start the service again.
I ran into the same errors. Similar approach for me. From what I can tell, there is something weird going on with the reference to the datadir in the my.ini file. Even when I manually edited it I could not seem to have any effect on it, until I blew EVERYTHING AWAY. Wish I had better news...do a DB backup first.
For me the key to getting this to work was:
1) Remove the previous installation from settings->control panel. Restart your machine.
2) Once machine comes back up, forcefully delete the previous installation directory.
[mine is C:\apps\MySQL\MySQLServer-5.5\, as I REFUSE to use c:\program files\..]
3) Forcefully delete the previous datadir directory [mine was c:\data\mysql].
4) Forcefully delete the previous default data directory [C:\Documents and Settings\All Users\Application Data\MySQL].
5) Re-run the install, selected the same installation directory. Skip the instance configurator/wizard at the end of the install.
6) Make sure the ../bin directory gets added to the path. Verify it.
7) Manually run the instance configurator/wizard.
Set the root password, port [3306].
It will try to start it. Again, mine FAILED to start
[duh! nothing new there!!!]
8) Now, manually edit the my.ini file in the install directory, and correct the datadir setting to be [datadir="C:/Data/MySQL/"] MATCH CAPITALIZATION !!!!
9) Verify the service is setup correctly via the command-prompt [sc qc mysql <enter>].
Should look like:
C:\dev\cmdz>sc qc mysql
[SC] GetServiceConfig SUCCESS
SERVICE_NAME: mysql
TYPE : 10 WIN32_OWN_PROCESS
START_TYPE : 2 AUTO_START
ERROR_CONTROL : 1 NORMAL
BINARY_PATH_NAME : "C:\apps\MySQL\MySQLServer-5.5\bin\mysqld" --defaults-file="C:\apps\MySQL\MySQLServer-5.5\my.ini" MySQL
LOAD_ORDER_GROUP :
TAG : 0
DISPLAY_NAME : MySQL
DEPENDENCIES :
SERVICE_START_NAME : LocalSystem
10) Copy the contents of the default data-directory created under C:\Documents and Settings\All Users\Application Data\MySQL [basically everything in this directory to your desired data directory c:\data\mysql]. Make sure you get the C:\Documents and Settings\All Users\Application Data\MySQL\mysql directory. This has host.frm file, and others.
You should end up with a directory now of c:\data\MySQL\mysql...
11) Rename the default directory
C:\Documents and Settings\All Users\Application Data\MySQL
To
C:\Documents and Settings\All Users\Application Data\MySQLxxx
So it cannot find it...
12) Say a quick prayer...
13) Give it a kick start from command line with [net start mysql]
That got it working for me...
Best of Luck!
Something I just discovered was that in the setup I choose InnoDB, whereas in fact all my existing databases were in MyIsam format. I uninstalled and then re-installed choosing MyIsam and everything worked just fine.
If you are trying to run MySql on a Windows 10 installation with a GPT partition, please try this:
Todays systems are often running on a disk which is formatted in GPT. This is because Windows 10 needs such a partition to be installed. The problem is that MySql cant deal with this partition style and so it crashes. Here is my workaround:
Create a virtual disk: manual here
Format this VHD with MBR
Install MySql custom on your new VHD (dont forget to create a user for DB)
This should be it! :)
Reference Link:
Issue with MySql installation on Windows 10
This question is about MySQL binary logs.
We need to move a binary log to a different hard disk.
What is the configuration change required in MySQL?
Currently binary logs go into the same folder as the ibdata and there is a replication slave running which needs the binary logs.
Changing the log-bin works. BUT you need to copy the log-bin files first to the new location and modify the index file. If you dont'do this you will break your remote slave with the error:
Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
More info here:
http://freebsd.down-to-details.com/sys-admin/relocating-moving-mysql-binlogs/
From the way I read the doc, you can specify a path on the log-bin config.
To enable the binary log, start the
server with the --log-bin[=base_name]
option. If no base_name value is
given, the default name is the value
of the pid-file option (which by
default is the name of host machine)
followed by -bin. If the basename is
given, the server writes the file in
the data directory unless the basename
is given with a leading absolute path
name to specify a different directory.
It is recommended that you specify a
basename; see Section C.5.8, “Known
Issues in MySQL”, for the reason.