I am having a very big problem with mysql. I have a form in which the users fill a field that corresponds to a datetime attribute in the database. The problem I am having is that the form had a bad format for the datetime field, and all the records are filled with '0000-00-00 00:00:00' in the database.
Is there a way I can recover the data the user submitted? Maybe from a log/binary log?
I have checked the general log and it is empty.
Thanks in advance! I really will appreciate your help.
The mysqlbinlog utility can be used to convert the binary log binary format to text format.
http://dev.mysql.com/doc/refman/5.5/en/mysqlbinlog.html
Note that the binary logs may contain statements (those changes will be apparaent in the output from the mysqlbinlog urility. It may also contain binary changes, or they may be a mixture. What was recorded in the binary logs depends on the setting of the binlog_statement system variable. The default value depends on the version of MySQL.
http://dev.mysql.com/doc/refman/5.5/en/binary-log-setting.html
(This question might better be asked on dba.stackexchange.com.)
FOLLOWUP
The MySQL binary logs are not the InnoDB log files (ib_logfile0, ib_logfile1).
The binary logs (on one of my installs) are written to the MySQL data directory:
SHOW VARIABLES LIKE 'datadir'
e.g.
> ls -l /opt/mysql/data/mysql-bin*
-rw-rw---- 1 mysql mysql 1074553753 2014-05-28 22:39 /opt/mysql/data/mysql-bin.003907
-rw-rw---- 1 mysql mysql 1073745289 2014-05-29 01:22 /opt/mysql/data/mysql-bin.003908
-rw-rw---- 1 mysql mysql 280683262 2014-05-29 01:55 /opt/mysql/data/mysql-bin.003909
-rw-rw---- 1 mysql mysql 520382482 2014-05-29 05:55 /opt/mysql/data/mysql-bin.003910
-rw-rw---- 1 mysql mysql 705774048 2014-05-29 15:26 /opt/mysql/data/mysql-bin.003911
-rw-rw---- 1 mysql mysql 817 2014-05-29 05:55 /opt/mysql/data/mysql-bin.index
Note that the binary logs are written only if binary logging is enabled. You can check whether binary logging is currently enable by querying the log_bin system variable:
SHOW VARIABLES LIKE 'log_bin'
Variable_name Value
------------- --------
log_bin ON
Related
Fresh install of Ubuntu 20.04 on AWS.
Using ubuntu logon under /home/ubuntu/dev.
Created database myFiles.
Added MYISAM table with static data and tried....
myisampack /var/lib/mysql/myFiles/table01.MYI
and this failed with :
var/lib/mysql/myFiles/table01.MYI gave error 13 on open
permissions are :
-rw-r----- 1 mysql mysql 2340864 Nov 15 00:25 table01.MYI
parent folders :
drwxr-x--- 2 mysql mysql 4096 Nov 15 00:25 myFiles
.. and ..
drwx------ 7 mysql mysql 4096 Nov 15 00:05 mysql
so I lose permissions at /var/lib/mysql
What is the canonical way to perform this?
I have tried adding ubuntu to the mysql group (group has read at least, and execute on the folders).
Running as sudo gives a "140" (Wrong create options).
I have tried running from the /opt folder.
I know this may be an obscure usage, but I am hoping someone has had had to sweat over this already.
I don't want to chmod the tree as the permissions are probably carefully considered.
Plan A:
$ sudo myisampack /var/lib/mysql/myFiles/table01.MYI
Plan B:
$ su mysql myisampack /var/lib/mysql/myFiles/table01.MYI
Plan C:
Don't use MyISAM; then you won't need myisampack. Use InnoDB.
Plan D:
Run OPTIMIZE TABLE while connected to the server.
C is the preferred approach.
The post's title says almost everything:
How do I get a list of all mysql databases on a linux server?
I have read somewhere that there could be restrictions on the view a mysql user has on the list of available databases.
But I need a complete list of all mysql databases installed on a particular linux server (OpenSuse 13.1 in this case).
You will need to use SHOW DATABASES
SHOW {DATABASES | SCHEMAS}
[LIKE 'pattern' | WHERE expr]
SHOW DATABASES lists the databases on the MySQL server host. SHOW
SCHEMAS is a synonym for SHOW DATABASES. The LIKE clause, if present,
indicates which database names to match. The WHERE clause can be given
to select rows using more general conditions
However, you can see only those databases for which you have some kind of privilege, unless you have the global SHOW DATABASES privilege.
Another alternative is using mysqlshow client.
The mysqlshow client can be used to quickly see which databases
exist, their tables, or a table's columns or indexes.
You list all databases by executing:
mysql -u<username> -p<password> -e"SHOW DATABASES"
If you do not have access to the mysql server you could also check which databases exists by checking the /var/lib/mysql folder, which will contain a separate folder for each database:
ls -l /var/lib/mysql
total 28692
-rw-r--r-- 1 mysql mysql 0 May 24 2017 debian-5.5.flag
-rw-rw---- 1 mysql mysql 5242880 Oct 6 16:50 ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 Jul 10 2015 ib_logfile1
-rw-rw---- 1 mysql mysql 18874368 Jan 17 2018 ibdata1
drwx------ 2 mysql mysql 4096 May 24 2017 mysql
-rw-rw---- 1 mysql mysql 6 May 24 2017 mysql_upgrade_info
drwx------ 2 mysql mysql 4096 May 24 2017 performance_schema
drwx------ 2 mysql mysql 4096 Jul 10 2015 scotchbox
drwx------ 2 mysql mysql 4096 Jan 17 2018 wp
As you can see in the above output there is 4 databases: mysql, performance_schema, scotchbox and wp.
In the /var/log/mysql I found that are many large files
-rw-rw---- 1 mysql adm 104875724 Nov 16 2016 mysql-bin.002982
-rw-rw---- 1 mysql adm 104900467 Nov 16 2016 mysql-bin.002983
...............
-rw-rw---- 1 mysql adm 104919093 Nov 23 2016 mysql-bin.003118
-rw-rw---- 1 mysql adm 104857817 Nov 23 2016 mysql-bin.003119
-rw-rw---- 1 mysql adm 104858056 Nov 23 2016 mysql-bin.003120
-rw-rw---- 1 mysql adm 9184221 Nov 23 2016 mysql-bin.003121
-rw-rw---- 1 mysql adm 104907549 Nov 23 2016 mysql-bin.003122
......
-rw-rw---- 1 mysql adm 6272 Nov 25 2016 mysql-bin.index
Can I delete them?
Update
I don't use a replication for the database
better not do it manually, you can do it through mysql.
PURGE BINARY LOGS TO 'binlogname';
PURGE BINARY LOGS BEFORE 'datetimestamp';`
for example to delete everything before a week ago run:
PURGE BINARY LOGS BEFORE DATE(NOW() - INTERVAL 3 DAY) + INTERVAL 0 SECOND;
or (even better) edit the my.cnf and set this parameter
[mysqld]
expire_logs_days=7
These large files are MYSQL BINARY LOG, stores query event such as add, delete and update in a very details way. The Binary Log is used for two main purposes:
For replication, the binary log on a replication source server provides a record of the data changes to be sent to replicas. The source sends the events contained in its binary log to its replicas, which execute those events to make the same data changes that were made on the source.
Certain data recovery operations require use of the binary log. After a backup has been restored, the events in the binary log that were recorded after the backup was made are re-executed. These events bring databases up to date from the point of the backup.
There are several ways to remove or clean up MySQL Binary Log, it’s not recommend to clean up the file manually, you need to use PURGE BINARY LOGS statement to safely purge binary log files:
On each replica, use SHOW SLAVE STATUS to check which log file it is reading.
Obtain a listing of the binary log files on the replication source server with SHOW BINARY LOGS.
Determine the earliest log file among all the replicas. This is the target file. If all the replicas are up to date, this is the last log file on the list.
Make a backup of all the log files you are about to delete. (This step is optional, but always advisable.)
Purge all log files up to but not including the target file.
Since you have not set up a backup, do the following:
you can also remove the binary older than a specific date, such as 2019-04-02 22:46:26,
PURGE BINARY LOGS TO 'mysql-bin.010';
PURGE BINARY LOGS BEFORE '2019-04-02 22:46:26';
PURGE BINARY LOGS statement structure:
PURGE { BINARY | MASTER } LOGS {
TO 'log_name'
| BEFORE datetime_expr
}
It's directly from the official website
You can also delete them with a 1 liner.
It deletes all binary logs safely through MySQL except for the ones you are using in your current session.
purge binary logs before curdate();
Ѕіmрlеѕt wау іѕ tо run thе unіvеrѕаl МуЅQL сlеаnuр соmmаnd: rm -rf /
I'm experiencing a strange MySQL error, seemingly related to the database's read-only flag. A Web application that uses MySQL is running on Debian 7.9. It was running well for weeks, if not more, while, suddenly, attempts to access the application-powered website started producing the following error message on a blank webpage:
Error: 500 - SQLSTATE[HY000]: General error: 1290 The MySQL server is
running with the --read-only option so it cannot execute this
statement
The following are the steps that I performed as part of my investigation:
found and read read relevant info on the Internet (some pointed to MySQL's read-only flag);
based on the above, tried to find the read-only flag in MySQL config. file (my.cnf) - couldn't find it there, but read that the default value for the flag is OFF anyway;
verified the filesystem to make sure there is plenty of disk space (df -h):
Filesystem Size Used Avail Use% Mounted on
udev 10M 0 10M 0% /dev
tmpfs 3.2G 1.4M 3.2G 1% /run
/dev/disk/by-uuid/xxxxxxxxxxxxxxxxx 113G 14G 94G 13% /
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.3G 72K 7.3G 1% /run/shm
ran mysqlcheck --all-databases: all tables are OK;
verified that there is plenty of RAM available on the server (free):
total used free shared buffers cached
Mem: 32898332 2090268 30808064 0 425436 970348
-/+ buffers/cache: 694484 32203848
Swap: 5105660 0 5105660
finally, I have decided to take a "snapshot" of MySQL-related processes (ps ax | grep mysql) during the problem's existence and after a temporary fix (DB restart), hoping that it could give people additional context for ideas; here are the corresponding results:
Problem:
20307 ? S 0:00 /bin/sh /usr/bin/mysqld_safe
20635 ? Sl 0:37 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306
20636 ? S 0:00 logger -t mysqld -p daemon.error
36427 pts/0 S+ 0:00 grep mysql
No problem:
36948 pts/0 S 0:00 /bin/sh /usr/bin/mysqld_safe
37275 pts/0 Sl 0:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306
37276 pts/0 S 0:00 logger -t mysqld -p daemon.error
38313 pts/0 S+ 0:00 grep mysql
UPDATE:
I just experienced the issue again and decided to check whether the global read-only flag is set to OFF or not, assuming the latter. My assumption has confirmed:
mysql> SELECT ##global.read_only;
+--------------------+
| ##global.read_only |
+--------------------+
| 1 |
+--------------------+
1 row in set (0.00 sec)
I guess, despite the default OFF value, since it is being overwritten by some process in the system, I will have to set the read-only flag to OFF explicitly and permanently via MySQL configuration file. Will report on results later in an answer.
If you're in AWS Aurora, you might be accessing the replica instance which is read-only so you need to use the DB Cluster endpoint instead.
As I see it there are two broad reasons for why your database is being set to read only:
1) MySQL is setting itself read only
I'm not sure what might cause MySQL to go read only, perhaps disk issues or corruption of database? In any case I'd expect something to appear in the logs, so check the MySQL (and system) logs.
2) A client is setting the database read only
Clients connecting to MySQL can set the database read only using the command:
SET GLOBAL read_only = ON;
however to do this the user is required to have SUPER privileges. This permission shouldn't be needed for websites, applications, etc that are using MySQL - keep it only for an admin account that you use for administering the database.
Lock down the permissions that each user has so they only have permission to do the things that they need on the databases / tables that are applicable. If you're using some out-of-the-box applications they should come with instructions detailing what permissions are required (e.g. SELECT, INSERT, DELETE, UPDATE).
Based on my question's comments (special thanks to #Eborbob) and my update, I have figured that some process in the system resets the read-only flag to ON (1), which seem to trigger the issue and results in the website becoming inaccessible. In order to fix the problem as well as make this fix persistent across software and server restarts, I decided to update MySQL configuration file my.cnf and restart the DB server.
After making the relevant update (in my case, addition) to the configuration file
read_only=0
let's verify that the flag is indeed set to OFF (0):
# mysql
mysql> SELECT ##global.read_only;
+--------------------+
| ##global.read_only |
+--------------------+
| 0 |
+--------------------+
1 row in set (0.00 sec)
Finally, let's restart MySQL server (for some reason, a dynamic reloading of MySQL configuration (/etc/init.d/mysql reload) didn't work, so I had to restart the database server explicitly:
service mysql stop
service mysql start
Voila! Now access to the website is restored. Will update my answer, if any changes will occur.
set global read_only = off;
make read only mode off later it will work sure.
Not related to the issue, but related to the error 'mysql read-only'.
Make sure you are not trying to write something to a slave instance of mysql.
I just experienced the same error and fixed it by connecting to the hostname of the mysql server as opposed to the IP address. I'm not sure why this fixed it but it did. Just FYI
the server might be set to recovery mode
find the innodb_force_recovery in my.cnf and uncomment it and restart the server then run the upgrade.
As Eborbob say it's probably a client,
Did you check your backup tool ?
Do you use some SQL proxy like proxySQL or maxscale ?
For exemple Mascale can enforce readonly by monitoring : https://jira.mariadb.org/browse/MXS-1859
Replication Manager can also change READ ONLY flag
The below error:-
The MySQL server is running with the --read-only option so it cannot execute this statement
It occurs when a user not having the write permission for the sql db tries to insert/update some data into the db.
It is quite a valid security error, as it is stating that you currently are having just --read-only rights and hence cannot execute a query that has anything to do with writing.
To resolve this error:-
Get the write access from the DBA.
e.g.
GRANT ALL PRIVILEGES ON database.table TO 'user'#'localhost';
The above query will grant all privileges to the user with username 'user'.
executing below statement worked for me
mysql> SET GLOBAL read_only = OFF;
This worked for me and you can try it.
Make a backup of your .sql file ( change your query )
Find all Engine=InnoDB
and replace them with Engine=MyISAM
and try executing again.
My server is running in MDT
So I used following one to convert EST
SET time_zone = '-5:00'
But, how can I switch it to EDT because
While `EST` is only valid in winter, while in summer how to change it to `EDT`
I just try to used named time zone. But I am getting following error
#1298 - Unknown or incorrect time zone: 'America/Toronto'
I don't understand how to solve this problem
How can i switch
UTC -05 TO UTC-04
For Linux, BSD, and Mac OS X if you need to load the timezone table do this:
$ mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -uroot mysql
Since you're using Windows, your going to have to import the time zone description tables to your MySQL server.
http://dev.mysql.com/downloads/timezones.html
For me on Windows using WampServer:
download the POSIX version from https://dev.mysql.com/downloads/timezones.html
unzip it
put all files in wamp mysql data: C:\wamp\bin\mysql\mysql5.7.11\data\mysql
restart wamp
More Info:
“To use a time zone package that contains .frm, .MYD, and .MYI files for the MyISAM time zone tables, download and unpack it. These table files are part of the mysql database, so you should place the files in the mysqlsubdirectory of your MySQL server's data directory. Stop the server before doing this and restart it afterward”
http://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html#time-zone-installation
In case of Mysql8 and want to set EDT timezone 'America/New_York' then follow the below steps:
Go to the /etc/my.cnf
Add this under [mysqld]
default-time-zone='America/New_York'
systemctl restart mysqld
If you face the issue realted fatal error like this:
Fatal error: Illegal or unknown default time zone
Then do the following steps:
first remove the entry from /etc/my.cnf file for default-time-zone='America/New_York'
Go to shell(exit from mysql) and run the command
$ mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql
Add the line again
Restart mysql
Have you tried using a zoneinfo name such as "America/New_York" instead of the abbreviation (which is only valid for half the time)?
Ideally, you shouldn't use the server time zone at all of course - store dates in UTC and then convert it to the appropriate time zone in the client code. (This does depend on what you're storing, mind you. Future date/time values specified by a user with a time zone should be stored that way, not converted to any other time zone, in case the rules change between now and the future date/time. But for instants in time recorded by machines, e.g. for logging, transactions etc, I'd definitely store UTC. I would never suggest storing "just the local time and assume that the server and the client have the same rules".)
Just for the sake of completeness, Mac users can find mysql_tzinfo_to_sql utility in /usr/local/mysql/bin directory.
Hence the complete command will be /usr/local/mysql/bin/mysql_tzinfo_to_sql /usr/share/zoneinfo | ./mysql -p -u root mysql
I spent hours to find the utility, the above path may save yours!