I'm currently attempting to create a script that will:
process a mysql schema and dump the descriptions of tables in the schema,
translate the MySQL-specific parts of the resulting DDL to another database system's equivalent (currently targeting H2, but any database that runs in-process in a Java environment and supports memory tables would be appropriate if I need to switch), and
then recreates them in a schema in the new database system.
Currently, I'm using mysqldump to perform the first part of the operation, and a sequence of string/regexp substitutions to perform the second.
The problem I have is that mysqldump is producing output with tables that have foreign keys before the tables they reference. This isn't a problem for mysql because in mysql you can just disable constraint checking and then create them -- unfortunately, while H2 does have an option for disabling constraints, it only applies to data changes and not table creation.
I'd really rather not have to parse the SQL enough to identify the correct order of the tables myself, as it seems at least vaguely tricky to do it right.
Therefore, is there either (1) any way to get mysqldump to produce the tables in the correct order or (2) an alternative approach to produce the correct order?
(I'm aware that it is at least theoretically possible to have a schema where no correct order is possible, but I know none of the schemas I'm likely to work with have this problem, so this isn't something I need to worry about)
Related
I'd like a snapshot of a live MySQL DB to work with on my development machine. The problem is that the DB is too large, so my thought was to execute:
mysqldump [connection-info-here] --no-autocommit --where="1 limit 1000" mydb > /dump.sql
I think this will give me the first thousand rows of every table in database mydb. I anticipate that the resulting dataset will break a lot of foreign key constraints since some records will be missing. As a result the application I mean to run on the dev machine will fail.
Is there a way to mysqldump a sample of the database while ensuring that all records dumped abide by key constraints? (for instance if a foreign key is dumped, the matching record in the foreign table will also be dumped).
If that isn't possible, how do you guys deal with this problem?
No, there's no option for mysqldump to dump only rows that match in foreign key relationships. You already know about the --where option, and that won't do it.
I've had the same task as you, to dump a subset of data but only data that is related. For example, for creating a test instance.
I've been using MySQL for many years, I've worked as a MySQL consultant and trainer, and I try to keep up with current tools. I have never heard of any MySQL tool that does this operation.
The only solution I can suggest is to write your own script to dump table by table using SELECT...INTO OUTFILE.
It's sometimes easier to write a custom script just for your specific schema, than for someone to write a general-purpose tool that works for everyone's schema.
How I have dealt with this problem in the past is I don't copy data from the live database. I find some other way to create a subset of fake data for testing. It's probably better to create synthetic data anyway, because then you don't risk accidentally using live data in your dev/test environment, in case some of it is private data.
This may sounds like a stupid question but can't find anything on google, probably using the wrong key words.
Anyway, I have been working on a project - version 1 which has a MySQL Database. I ready to release to version 2 but there are changes to the database tables, e.g. extra columns.
If I backup the current database with the data and create a database with the new structure. How can I add the data from the old database into the new database.
I know there won't be any problems with the existing data being added to the new database structure as the existing fields haven't changed, its just extra columns.
Thanks for your help.
I use mysqldump with some addition keys in this case, something like
mysqldump --host=localhost --user=root --no-create-db --no-create-info --complete-insert --extended-insert
That will produce the complete inserts with column names, so you may not to worry about the final table structure, if you did not change the column names, even the order of columns may change in this case.
Consider using ALTER TABLE to resolve this issue.
The key is to take the new fields in your database and append them to the end of your entities, like so:
ALTER TABLE myTable ADD COLUMN myColumn (... further specification ...)
MySQL will expand the table and set the new fields to the defaults you specify. You can then layer any new data on top of the old, as long as there are no conflicts, as you describe.
Option B, when the online solution is expensive, is to use mysqldump, then alter the output to fit the new table specification. As long as the columns align properly (this may require a simple regular expression to parse, in the worst case), you should be able to recreate the data by importing it into the new schema.
See also, this answer.
I am facing a problem for a task I have to do at work.
I have a MySQL database which holds the information of several clients of my company and I have to create a backup/restore procedure to backup and restore such information for any single client. To clarify, if my client A is losing his data, I have to be able to recover such data being sure I am not modifying the data of client B, C, ...
I am not a DB administrator, so I don't know if I can do this using standard mysql tools (such as mysqldump) or any other backup tools (such as Percona Xtrabackup).
To backup, my research (and my intuition) led my to this possibile solution:
create the restore insert statement using the insert-select syntax (http://dev.mysql.com/doc/refman/5.1/en/insert-select.html);
save this inserts into a sql file, either in proper order or allowing this script to temporarily disable the foreign key checks to meet foreign keys' constraint;
of course, I do this for all my clients on a daily base, using a file for each client (and day).
Then, in the case I have to restore the data for a specific client:
I delete all his data left;
I restore the correct data using his sql file I created during the backup.
This way I believe I may recover the right data of client A without touching the data of client B. Is my solution eventually working? Is there any better way to achieve the same result? Or do you need more information about my problem?
Please, forgive me if this question is not well-formed, but I am new here and this is my first question so I may be unprecise...thanks anyway for the help.
Note: we will also backup the entire database with mysqldump.
You can use the --where parameter, you could provide a condition like *client_id=N* . Of course I am making an assumption since you don't provide any information on your schema.
If you have a Star schema , then you could probably write a small script that backups all lookup tables (considering they are adequately small) by using this parameter --tables and use the --where condition for your client data table. For additional performance, perhaps you could partition the table by the client_id.
How do you view all the constraints (Primary keys, Secondary keys, Assertions , Triggers, etc) in a MySQL Database / Table in the command line environment.
Just dump the database without data using
mysqldump --no-data (other options)
As if you were taking a backup. Use the same options as you do when taking a backup (maybe --lock-tables=0 - you don't need a lock when dumping the schema)
Without the data, you get just the schema which includes all those things you said above.
DESCRIBE is an alias for SHOW COLUMNS, like a shortcut. If you want all the other stuff then you need to use SHOW commands for your stuff. And SHOW COLUMNS for the rest.
Describe
Show
I have a MySQL database running on a deployment machine which also contains data. Then I have another MySQL database which has evolved in terms of STRUCTURE + DATA for some time. I need a way to merge the changes (ONLY) for both structure and data to the DB in deployment machine without disturbing the existing data. Does anyone know of a tool available which can do this safely. I have had a look at a few comparison tools but I need a tool which can automate the merge operation. Note also that most of the data in the tables is in BINARY so I can't use many file comparison tools. Does any one know of a solution to this?
I doubt you can get around implementing your own diff & merge without paying a lot.
Read the structures on both databases, execute a few alter table [table] add column [foo] statements to update the structure, then port data line by line (SELECT * on old Database, UPDATE [new columns] WHERE [primary key conditions]).
There is no easier way to my knowledge.