I am facing problem, where mysqldump(ing) my database behaves strange, with just one table in database. Table has nothing unusual configuration in it and all the columns are just basic stuff, expect it has two columns with longtext/text datatype, if that does matter somehow. Longtext column contains valid xml strings and text column contains valid json strings.
Problem is that produced dump file contains no inserts at all for that specific table. Dump file has sql lines for everything else. Dump file even has create table lines for that problematic table and all other mysql stuff, but insert lines are totally missing for that table. Instead of inserts, there is just huge amount of empty lines in the middle of dump file. Dump is done like this:
mysqldump -u root -ppassword dbname > dump.sql
I did notice, that when I add
--extended-insert=false
to my dump command, dump contains some insert rows. Rows with ids 1-7 are present, 8-9 are missing, 10 is present, 11 is missing, 12-13 are present etc... missing row insert lines are replaced by empty line in dump file.
Could anyone have any clue, what is happening here? For me, data seems not corrupted and it can be browsed via phpmyadmin interface.
Some facts about the case
mysql Ver 14.14 Distrib 5.5.31, for debian-linux-gnu (x86_64) using
readline 6.2
Problem table engine is InnoDB
Dump file size is around 1.7G
Problem table has around 230.000 rows
Problem was solved by inspecting dump file closer with different program. LogExpert has known bug of not showing lines with too many characters in it. Dump file was just fine and problem did not actually even exist.
Importing from dump file failed because of too small innodb buffer size in php.ini file. So there was two separated problems that confused me.
For me, after long hours of troubleshooting, getting a blank dump file, it was disk space issues on the server where I am trying to take db dump.
Seems silly but posting may it helped.
Related
So I have an exported database database.sql.gz from a database I can't access personally. Simply uploading it into PHPMyAdmin gives errors, and uploading it with zcat {directory} | mysql -u {user} -p {database} also gives an error "Row size too large (> 8126)". After reading through the file with Vi, I realize I can't simply change some row file formats around to make it fit, as the specific table has 67 rows. I also found out this table (along with others) don't get filled at all and get dropped at the end of the document. I tried commenting out the CREATE TABLE of the too large table, and there don't seem to have been any related errors from that, but I did run into some different errors at CREATE ALGORITHM commands.
Long story short, is there a better way to remove this giant table from my file that doesn't include exporting it again (as I don't have direct access to the database) or commenting out every bit that has to do with that table.
I am trying to convert a MySQL UTF8mb4 database which contains both Thai and English to Postgresql. This appears to go
well until I try and add tsearch. Let me outline the steps taken.
Install this Thai parser https://github.com/zdk/pg-search-thai
I restore a copy of production locally from a dump file into MariaDB
Fix some enum values that trip up Postgresql due to them being
missing. MariaDB is happy with them :(
Convert some polygons to
text format as pgloader does not deal with them gracefully.
-Run pgloader against a fresh postgresql database, testdb
pgloader mysql://$MYSQL_USER:$MYSQL_PASSWORD#localhost/$MYSQL_DB postgresql://$PG_USER:$PG_PASSWORD#localhost/testdb
This appears to work, the site, a Laravel one, appears to function although with some bugs to fix due to differences
between MariaDB and Postgresql constraint behavior. However when I try and create text vectors for tsearch, I run into
encoding issues. This is where I need advice.
-- trying to create minimal case, dumping Thai names into a temporary table
CREATE EXTENSION thai_parser;
CREATE TEXT SEARCH CONFIGURATION thai_unstemmed (PARSER = thai_parser);
ALTER TEXT SEARCH CONFIGURATION thai_unstemmed ADD MAPPING FOR a WITH simple;
-- to test the parser is working, which it is
SELECT to_tsvector('thai_unstemmed', 'ข้าวเหนียวส้มตำไก่ย่าง ต้มยำกุ้ง in thailand');
-- to recreate my error I did this
CREATE TABLE vendor_names AS SELECT id,name from vendors_i18n;
ALTER TABLE vendor_names ADD COLUMN tsv_name_th tsvector;
-- this fails
UPDATE vendor_names SET tsv_name_th=to_tsvector('thai_unstemmed', coalesce(name, ''));
The error I get is ERROR: invalid byte sequence for encoding "UTF8": 0x80
If I dump that table and restore into a new Postgresql database I do not get the encoding error.
Questions:
What is the correct encoding to use for UTF8mb4 to Postgresql for pgloader?
Is there any way, other than the above, of checking the data being correct UTF8 or not?
Is the problem in the Thai parser tool?
Any suggestions as to how to solve this would be appreciated.
Cheers,
Gordon
PS I'm an experienced developer but not an experienced DBA.
Have you tried manually importing the dataset row-by-row to see which rows are successfully imported and which ones fail? If some imports succeed but others fail it would seem to be a data integrity problem.
If none of the records are successfully imported it's obviously an encoding problem.
I am moving a MySQL database from a now inaccessible server to a new one. The dump contains tables which in turn contain binary blobs, which seems to cause trouble with the MySQL command line client. When trying to restore the database, I get the following error:
ERROR at line 694: Unknown command '\''.
I inspected the line at which the error is occurring and found that it is a huge insert statement (approx. 900k characters in length) which seems to insert binary blobs into a table.
Now, I have found these two questions that seem to be connected to mine. However, both answers proved to not solve my issue. Adding --default-character-set=utf8 or even --default-caracter-set=latin1 didn't change anything and creating a dump with --hex-dump is not possible because the source database server is no longer accessible.
Is there any way how I can restore this backup via the MySQL command line client? If yes, what do I need to do?
Please let me know if you need any additional information.
Thanks in advance.
EDIT: I am using MySQL 5.6.35. Also, in addition to the attempts outlined above, I have already tried increasing the max_allowed_packet system variable to its maximum value - on both server and client - but to no avail.
If I remember correctly, you need to set the max_allowed_packet in your my.cnf to a large enough value to accommodate the largest data blob in your dump file, and restart the MySQL server.
Then, you can use a restore command like this one :
mysql --max_allowed_packet=64M < your_dumpfile.sql
More info here :
[https://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_max_allowed_packet]
No solution, just confirming that I had seen the same behavior with a "text" field type that contains a long JSON string. The SQL (backup) file that MySQLdump generates has an INSERT statement and it truncates the length of that particular text field to "about" 64K (there are many escaped quotes/double-quotes and various UTF-8 characters) - without issuing a warning that such truncation had occurred.
Naturally the restore into a JSON column fails because of the premature termination of the JSON formatted string.
What was odd in this case, that the column in the backed-up table was defined as TEXT, which indeed should have been limited to 64 KB. On a hunch, I changed the schema for the backed up table to MEDIUMTEXT. After THAT MySQLdump no longer truncated that string in the INSERT statement somewhere beyond 64K.
It appears as if MySQLdump doesn't just output the entire column, but truncates to whatever it thinks the maximum string length should be based on schema information, and does NOT issue warnings when it does truncate.
I am having the weirdest error of all.
Sometimes, when creating or altering tables, I get the 'table already exists' error. However, DROP TABLE returns '#1051 - unknown table'. So I got a table I cannot create, cannot drop.
When I try to drop the database, mysqld crashes. Sometimes it helps to create another db with different name, sometimes it does not.
I use a DB with ~50 tables, all InnoDB. This problem occurs with different tables.
I experienced this on Windows, Fedora and Ubuntu, MySQL 5.1 and 5.5. Same behaviour, when using PDO, PHPMyAdmin or commandline. I use MySQL Workbench to manage my schema - I saw some related errors (endlines and stuff), however none of them were relevant for me.
No, it is not a view, it is a table. All names are lowercase.
I tried everything I could google - flushing tables, moving .frm files from db to db, reading mysql log, nothing helped but reinstalling the whole damn thing.
'Show tables' reveals nothing, 'describe' table says 'table doesn't exist,' there is no .frm file, yet 'create table' still ends with an error (and so does 'create table if not exists') and dropping database crashes mysql
Related, yet unhelpful questions:
Mysql 1050 Error "Table already exists" when in fact, it does not
MySQL Table does not exist error, but it does exist
Edit:
mysql> use askyou;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql> create table users_has_friends (id int primary key);
ERROR 1050 (42S01): Table '`askyou`.`users_has_friends`' already exists
mysql> drop table users_has_friends;
ERROR 1051 (42S02): Unknown table 'users_has_friends'
And such, all the same: table doesn't exist, yet cannot be created;
mysql> drop database askyou;
ERROR 2013 (HY000): Lost connection to MySQL server during query
Names change, this is not the only table / database I've run into problems with
I've seen this issue when the data file is missing in the data directory but the table definition file exists or vise-versa. If you're using innodb_file_per_table, check the data directory to make sure you have both an .frm file and .ibd file for the table in question. If it's MYISAM, there should be a .frm, .MYI and a .MYD file.
The problem can usually be resolved by deleting the orphaned file manually.
Going on a wild guess here, but it seems like innodb still has an entry for your tables in a tablespace, probably in ibdata. If you really don't need any of the data, or if you have backups, try the following:
Delete all schemas (excluding mysql)
shut down the database
Make sure that all folders in your data directory have been removed properly (again, excluding mysql)
delete ibdata and log files
restart the database. It should recreate the tablespace and logs from scratch.
The fix turns out to be easy; at least what I worked out, worked for me.
Create a table "zzz" on another MySQL instance, where zzz is the problem table name.
(i.e. if the table is called schrodinger, substitute that for zzz whever written.)
It does not matter what the definition of the table is. It's a temporary dummy;
Copy the zzz.frm file to the database directory on server where the table should be,
making sure file ownership and permissions are still correct on the file.
On MySQL, you can now do "show tables;", and the table zzz will be there.
mysql> drop table zzz;
...should now works. Clear any zzz.MYD or ZZZ.MYI files in the directory if necessary.
I doubt this is a direct answer to the question case here, but here is how I solved this exact perceived problem on my OS X Lion system.
I frequently create/drop tables for some analytics jobs I have scheduled. At some point, I started getting table already exists errors half-way through my script. A server restart typically solved the issue, but that was too annoying of a solution.
Then I noticed in the local error log file this particular line:
[Warning] Setting lower_case_table_names=2 because file system for /usr/local/mysql/data/ is case insensitive
This gave me the idea that maybe if my tables contained capital letters, MySQL would be fooled into thinking they are still there even after I had dropped them. That turned out to be the case and switching to using only lowercase letters for table names made the problem go away.
It is likely the result of some misconfiguration in my case, but hopefully this error case will help someone waste less time trying to find a solution.
This is an old question but I just hit the same issue and an answer in one of the related issues linked at the top was just what I needed and far less drastic than deleting files, tables, shutting down the server etc.
mysqladmin -uxxxxxx -pyyyyy flush-tables
If will are stock with this error 1051 and you only want to delete the database and import this again do this steps and all gonna be just fine....
in Unix envoriment AS root:
rm -rf /var/lib/mysql/YOUR_DATABASE;
OPTIONAL -> mysql_upgrade --force
mysqlcheck -uUSER -pPASS YOUR_DATABASE
mysqladmin -uUSER -pPASS drop YOUR_DATABASE
mysqladmin -uUSER -pPASS create YOUR_DATABASE
mysql -uUSER -pPASS YOUR_DATABASE < IMPORT_FILE
Regards,
Christus
In my case the problem was solved by changing the ownership of the mysql data directory to the user that ran the application. (In my case it was a Java application running Jetty webserver.)
Even though mysql was running and other apps could use it properly, this app had a problem with that. After changing the data directory ownership and resetting the user's password, everything worked properly.
I was having this problem with one particular table. Reading the possible solutions i've did some steps like:
Search for orphan files: didn't exist anyone;
execute: show full tables in database;: didn't see the problematic one;
execute: describe table;: returned table doesn't exist;
execute: SELECT * FROM information_schema.TABLES WHERE TABLE_NAME='table';: returned Empty set;
Search by the phpMyAdmin manually the query above: didn't exist;
And, after those steps, i check again with the show tables; and... vualá! the problematic table was gone. I could create it and drop it with the same problematic name with no problem, and i didn't have even to restart the server! Weird...
I had this problem and hoped deleting the IBD file would help as posted above but it made no difference . MySQL only recreated a new IBD file . In my case, there are actually similar tables in other databases in the same MySQL instance . Since the FRM file was missing , I copied the FRM file from the similar table in another database , restarted MySQL and the table worked correctly .
I ran into this error after I created a table and deleted it, then wanted to create it again.
In my case, I had a self-contained dump file so I dropped my schema, recreated it and imported tables and data using the dump file.
It happens at our site (but rarely) usually when an "event" happens while running certain scripts that do a lot of rebuilding. Events include network outages or power problems.
What I do for this on the very rare occasions it happens - I use the heavy-handed approach:
I needed to simply get rid-of-and-rebuild the particular table. I'm usually in a position that this is OK since the table is being built. (Your situation may be different if you need to recover data)
As an admin, go into the mysql installation (on windows its may be "...program files/mysql/MySQL Server xx/data/<schemaname>
Find the offending file with the table name in the <schemaname> folder - and delete it.
Check for orphaned temporary files and delete them too. #...frm files if they happen to be there.
MySQL will let you CREATE the table again
I've had this problem on a couple of different databases over a long time (years). It was a stumper because the contradicting messages. The first time I did a variation of the deleting/rebuilding/renaming database as described in the other answers and managed to get things going, but it definitely takes longer that way. Lucky for me it's always happened to reference tables that are being rebuilt - DROP'd and CREATEd - typically in the morning. Rarely got the problem but came to recognize it as a special quirky case. (I'll restate : if you need to recover the data look to the other solutions.)
it isn't a table belonging to another user, or in another database
it isn't the upper/lower case issue, I use all lower-case, but that was an interesting issue!
it was extra extra frustrating seeing responses with variations of "it definitely was <there/not-there/some-other-user-table-case> and you're just not doing it right" :)
the table didn't show on "show tables"
the table was (always was/had been) an INNODB table.
trying to DROP the table gave the error message that the table doesn't exist.
but trying to CREATE the table gave the error message that the table already exists.
using mysql 5.0 or 5.1
REPAIR is ineffective for this problem
I am having the weirdest error of all.
Sometimes, when creating or altering tables, I get the 'table already exists' error. However, DROP TABLE returns '#1051 - unknown table'. So I got a table I cannot create, cannot drop.
When I try to drop the database, mysqld crashes. Sometimes it helps to create another db with different name, sometimes it does not.
I use a DB with ~50 tables, all InnoDB. This problem occurs with different tables.
I experienced this on Windows, Fedora and Ubuntu, MySQL 5.1 and 5.5. Same behaviour, when using PDO, PHPMyAdmin or commandline. I use MySQL Workbench to manage my schema - I saw some related errors (endlines and stuff), however none of them were relevant for me.
No, it is not a view, it is a table. All names are lowercase.
I tried everything I could google - flushing tables, moving .frm files from db to db, reading mysql log, nothing helped but reinstalling the whole damn thing.
'Show tables' reveals nothing, 'describe' table says 'table doesn't exist,' there is no .frm file, yet 'create table' still ends with an error (and so does 'create table if not exists') and dropping database crashes mysql
Related, yet unhelpful questions:
Mysql 1050 Error "Table already exists" when in fact, it does not
MySQL Table does not exist error, but it does exist
Edit:
mysql> use askyou;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql> create table users_has_friends (id int primary key);
ERROR 1050 (42S01): Table '`askyou`.`users_has_friends`' already exists
mysql> drop table users_has_friends;
ERROR 1051 (42S02): Unknown table 'users_has_friends'
And such, all the same: table doesn't exist, yet cannot be created;
mysql> drop database askyou;
ERROR 2013 (HY000): Lost connection to MySQL server during query
Names change, this is not the only table / database I've run into problems with
I've seen this issue when the data file is missing in the data directory but the table definition file exists or vise-versa. If you're using innodb_file_per_table, check the data directory to make sure you have both an .frm file and .ibd file for the table in question. If it's MYISAM, there should be a .frm, .MYI and a .MYD file.
The problem can usually be resolved by deleting the orphaned file manually.
Going on a wild guess here, but it seems like innodb still has an entry for your tables in a tablespace, probably in ibdata. If you really don't need any of the data, or if you have backups, try the following:
Delete all schemas (excluding mysql)
shut down the database
Make sure that all folders in your data directory have been removed properly (again, excluding mysql)
delete ibdata and log files
restart the database. It should recreate the tablespace and logs from scratch.
The fix turns out to be easy; at least what I worked out, worked for me.
Create a table "zzz" on another MySQL instance, where zzz is the problem table name.
(i.e. if the table is called schrodinger, substitute that for zzz whever written.)
It does not matter what the definition of the table is. It's a temporary dummy;
Copy the zzz.frm file to the database directory on server where the table should be,
making sure file ownership and permissions are still correct on the file.
On MySQL, you can now do "show tables;", and the table zzz will be there.
mysql> drop table zzz;
...should now works. Clear any zzz.MYD or ZZZ.MYI files in the directory if necessary.
I doubt this is a direct answer to the question case here, but here is how I solved this exact perceived problem on my OS X Lion system.
I frequently create/drop tables for some analytics jobs I have scheduled. At some point, I started getting table already exists errors half-way through my script. A server restart typically solved the issue, but that was too annoying of a solution.
Then I noticed in the local error log file this particular line:
[Warning] Setting lower_case_table_names=2 because file system for /usr/local/mysql/data/ is case insensitive
This gave me the idea that maybe if my tables contained capital letters, MySQL would be fooled into thinking they are still there even after I had dropped them. That turned out to be the case and switching to using only lowercase letters for table names made the problem go away.
It is likely the result of some misconfiguration in my case, but hopefully this error case will help someone waste less time trying to find a solution.
This is an old question but I just hit the same issue and an answer in one of the related issues linked at the top was just what I needed and far less drastic than deleting files, tables, shutting down the server etc.
mysqladmin -uxxxxxx -pyyyyy flush-tables
If will are stock with this error 1051 and you only want to delete the database and import this again do this steps and all gonna be just fine....
in Unix envoriment AS root:
rm -rf /var/lib/mysql/YOUR_DATABASE;
OPTIONAL -> mysql_upgrade --force
mysqlcheck -uUSER -pPASS YOUR_DATABASE
mysqladmin -uUSER -pPASS drop YOUR_DATABASE
mysqladmin -uUSER -pPASS create YOUR_DATABASE
mysql -uUSER -pPASS YOUR_DATABASE < IMPORT_FILE
Regards,
Christus
In my case the problem was solved by changing the ownership of the mysql data directory to the user that ran the application. (In my case it was a Java application running Jetty webserver.)
Even though mysql was running and other apps could use it properly, this app had a problem with that. After changing the data directory ownership and resetting the user's password, everything worked properly.
I was having this problem with one particular table. Reading the possible solutions i've did some steps like:
Search for orphan files: didn't exist anyone;
execute: show full tables in database;: didn't see the problematic one;
execute: describe table;: returned table doesn't exist;
execute: SELECT * FROM information_schema.TABLES WHERE TABLE_NAME='table';: returned Empty set;
Search by the phpMyAdmin manually the query above: didn't exist;
And, after those steps, i check again with the show tables; and... vualá! the problematic table was gone. I could create it and drop it with the same problematic name with no problem, and i didn't have even to restart the server! Weird...
I had this problem and hoped deleting the IBD file would help as posted above but it made no difference . MySQL only recreated a new IBD file . In my case, there are actually similar tables in other databases in the same MySQL instance . Since the FRM file was missing , I copied the FRM file from the similar table in another database , restarted MySQL and the table worked correctly .
I ran into this error after I created a table and deleted it, then wanted to create it again.
In my case, I had a self-contained dump file so I dropped my schema, recreated it and imported tables and data using the dump file.
It happens at our site (but rarely) usually when an "event" happens while running certain scripts that do a lot of rebuilding. Events include network outages or power problems.
What I do for this on the very rare occasions it happens - I use the heavy-handed approach:
I needed to simply get rid-of-and-rebuild the particular table. I'm usually in a position that this is OK since the table is being built. (Your situation may be different if you need to recover data)
As an admin, go into the mysql installation (on windows its may be "...program files/mysql/MySQL Server xx/data/<schemaname>
Find the offending file with the table name in the <schemaname> folder - and delete it.
Check for orphaned temporary files and delete them too. #...frm files if they happen to be there.
MySQL will let you CREATE the table again
I've had this problem on a couple of different databases over a long time (years). It was a stumper because the contradicting messages. The first time I did a variation of the deleting/rebuilding/renaming database as described in the other answers and managed to get things going, but it definitely takes longer that way. Lucky for me it's always happened to reference tables that are being rebuilt - DROP'd and CREATEd - typically in the morning. Rarely got the problem but came to recognize it as a special quirky case. (I'll restate : if you need to recover the data look to the other solutions.)
it isn't a table belonging to another user, or in another database
it isn't the upper/lower case issue, I use all lower-case, but that was an interesting issue!
it was extra extra frustrating seeing responses with variations of "it definitely was <there/not-there/some-other-user-table-case> and you're just not doing it right" :)
the table didn't show on "show tables"
the table was (always was/had been) an INNODB table.
trying to DROP the table gave the error message that the table doesn't exist.
but trying to CREATE the table gave the error message that the table already exists.
using mysql 5.0 or 5.1
REPAIR is ineffective for this problem