This may sounds like a stupid question but can't find anything on google, probably using the wrong key words.
Anyway, I have been working on a project - version 1 which has a MySQL Database. I ready to release to version 2 but there are changes to the database tables, e.g. extra columns.
If I backup the current database with the data and create a database with the new structure. How can I add the data from the old database into the new database.
I know there won't be any problems with the existing data being added to the new database structure as the existing fields haven't changed, its just extra columns.
Thanks for your help.
I use mysqldump with some addition keys in this case, something like
mysqldump --host=localhost --user=root --no-create-db --no-create-info --complete-insert --extended-insert
That will produce the complete inserts with column names, so you may not to worry about the final table structure, if you did not change the column names, even the order of columns may change in this case.
Consider using ALTER TABLE to resolve this issue.
The key is to take the new fields in your database and append them to the end of your entities, like so:
ALTER TABLE myTable ADD COLUMN myColumn (... further specification ...)
MySQL will expand the table and set the new fields to the defaults you specify. You can then layer any new data on top of the old, as long as there are no conflicts, as you describe.
Option B, when the online solution is expensive, is to use mysqldump, then alter the output to fit the new table specification. As long as the columns align properly (this may require a simple regular expression to parse, in the worst case), you should be able to recreate the data by importing it into the new schema.
See also, this answer.
Related
So I don't want this thread to be marked as spam, as a previous thread was on this topic was, so I will explain what I have done so far and my issue is and ask if there are any solutions.
I have a MySQL database on my laptop that I need to migrate to DB2 on iSeries. I'm using a tool, I won't say which one because of the spam issue, which allows me to "copy" a table in my MySQL database and "paste" it into my DB2 database.
The issue that I'm having is because the table names and column names contain spaces in the MySQL db, the tool is failing on the paste. I confirmed this by altering one table by replacing the spaces with underscores and the copy worked perfectly. I have over a hundred tables I need to copy over and don't want to have to manually edit every table and column name.
Is there a way to globally replace spaces with underscores in MySQL table names and columns?
Any other ideas? I'm also researching a way to force the query the tool creates to enclose the object names in quotes, but have had no luck so far.
Thanks for any help and suggestions you can provide.
Since Stack Overflow is about helping to solve programming problems, I'm going to ignore the issue of deficiencies in the chosen tool and propose a programming solution to the larger problem - DB2 does not allow spaces in table and column names. You did ask for any suggestions...
Write code that reads the MySQL catalog tables. In DB2 they'd be SYSTABLES, SYSVIEWS, SYSINDEXES, SYSCOLUMNS, etc. Read SYSTABLES and use that as the 'primary' source for the rest of the code. Check the table name; if it has an embedded space, replace it with an underscore. Use SYSCOLUMNS to generate a CREATE TABLE statement that will create the new table (in a new MySQL database?) - also performing space to underscore replacement. After issuing the CREATE TABLE, generate an SQL statement that will INSERT INTO the new table the columns from the old table; again doing the space to underscore substitutions. Once the new table is populated, generate SQL statements to CREATE VIEW, CREATE PROCEDURE, CREATE FUNCTION, etc.
The general idea is that you will completely re-create your MySQL database with table, view and column names that are immediately compatible with DB2 for i so that your tool can do it's thing.
Of course, if you go to that much trouble it'll probably be just as easy to directly CREATE TABLE, etc on the IBM i side rather than go through an intermediate tool that isn't quite what you need.
We will migrate the database from mysql to postgresql in our product(through java). So we need to change the mysql query to postgresql query in java application. How to create the table i.e., databasename.tablename in postgresql.
For mysql, we can directly create the table e.g create table information.employee.
Here database name is "information" and table name is "employee" . Is it possible to achieve same query in postgresql.
I searched google it says cross database reference is not possible. Please help me.
I saw pg_class table it contains the table names in the specific database, like wise databse and tables relationships are stored in any other table.
This is normally done using schemas rather than databases, which is more or less like how MySQL organizes it anyway.
Instead of
create database xyz
use
create schema xyz
When you create tables, create them:
create table xyz.myTable
you will need to update your search path to see them on the psql command line tool, or if you want to query them without using the schema explicitly. The default schema is public, so when you create a table without a schema name, it ends up in public. If you modify your search_path as below, the default schema becomes the first in the list: xyz.
set search_path=xyz,public,pg_catalog;
and you must not have spaces in that statement. You can do it globally for a user/role too:
alter role webuser set search_path=xyz,public,pg_catalog;
Also, don't forget that postgresql string matches are case sensitive by default (this one catches people out a lot).
If you want to have different physical locations for the files for each schema, you can do that with tablespaces. If you have a look at the postgresql documentation page, they have info on how to do it, it's pretty easy.
database in MySQL == schema in PostgreSQL. So you will most probably want to migrate all your mysql dbs into one postgres db. Then you will be able to do "cross-database" queries.
See my answer to this question: Relationship between catalog, schema, user, and database instance
I am using mysql. Some of the tables contain sensitive data like user names, email addresses, etc. I want to dump the data but with these columns in the table removed or modified to some fake data. Is there any way to do it easily?
I'm using this approach:
Copy contents of sensitive tables to a temporary table.
Clear/encrypt the sensitive columns.
Provide --ignore-table arguments to mysqldump.exe to leave the original tables out.
It preserves foreign key contraints, and you can keep columns that are not sensitive.
The first two actions are contained in a stored procedure that I call before doing the dump. It looks something like this:
BEGIN
truncate table person_anonymous;
insert into person_anonymous select * from person;
update person_anonymous set Title=null, Initials=mid(md5(Initials),1,10), Midname=md5(Midname), Lastname=md5(Lastname), Comment=md5(Comment);
END
As you can see, I'm not clearing the contents of the fields. Instead, I keep a hash. That way, you can still see which rows have the same value, and between exports you can see if something changed or not, without anyone being able to read the actual values.
There is a tool called Jailer that is typically used to export a subset of a database. We use this at work to create a smaller test database from a production backup, with all sensitive data obfuscated.
The GUI is a bit crude, but Jailer is the best alternative I have found so far.
You can simply unselect the sensitive tables or columns and get a full copy of the rest. Jailer also supports obfuscating data during export - you could for instance md5 hash all user names or change all email addresses to user#example.org.
There is a tutorial to get you started.
ProxySQL is another approach.
Here is an article explaining how to obfuscate data with proxysql.
https://proxysql.com/blog/obfuscate-data-from-mysqldump
If I have a MySQL database with several tables on a live server, now I would like to migrate this database to another server. Of course, the migration I mean here involves some database tables, for example: add some new columns to several tables, add some new tables etc..
Now, the only method I can think of is to use some php/python(two scripts I know) script, connect two databases, dump the data from the old database, and then write into the new database. However, this method is not efficient at all. For example: in old database, table A has 28 columns; in new database, table A has 29 columns, but the extra column will have default value 0 for all the old rows. My script still needs to dump the data row by row and insert each row into the new database.
Using MySQLDump etc.. won't work. Here is the detail. For example: I have FOUR old databases, I can name them as 'DB_a', 'DB_b', 'DB_c', 'DB_d'. Now the old table A has 28 columns, I want to add each row in table A into the new database with a new column ID 'DB_x' (x to indicate which database it comes from). If I can't differentiate the database ID by the row's content, the only way I can identify them is going through some user input parameters.
Is there any tools or a better method than writing a script yourself? Here, I dont need to worry about multithread writing problems etc.., I mean the old database will be down (not open to public usage etc.., only for upgrade ) for a while.
Thanks!!
I don't entirely understand your situation with the columns (wouldn't it be more sensible to add any new columns after migration?), but one of the arguably fastest methods to copy a database across servers is mysqlhotcopy. It can copy myISAM only and has a number of other requirements, but it's awfully fast because it skips the create dump / import dump step completely.
Generally when you migrate a database to new servers, you don't apply a bunch of schema changes at the same time, for the reasons that you're running into right now.
MySQL has a dump tool called mysqldump that can be used to easily take a snapshot/backup of a database. The snapshot can then be copied to a new server and installed.
You should figure out all the changes that have been done to your "new" database, and write out a script of all the SQL commands needed to "upgrade" the old database to the new version that you're using (e.g. ALTER TABLE a ADD COLUMN x, etc). After you're sure it's working, take a dump of the old one, copy it over, install it, and then apply your change script.
Use mysqldump to dump the data, then echo output.txt > msyql. Now the old data is on the new server. Manipulate as necessary.
Sure there are tools that can help you achieving what you're trying to do. Mysqldump is a premier example of such tools. Just take a glance here:
http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html
What you could do is:
1) You make a dump of the current db, using mysqldump (with the --no-data option) to fetch the schema only
2) You alter the schema you have dumped, adding new columns
3) You create your new schema (mysql < dump.sql - just google for mysql backup restore for more help on the syntax)
4) Dump your data using the mysqldump complete-insert option (see link above)
5) Import your data, using mysql < data.sql
This should do the job for you, good luck!
Adding extra rows can be done on a live database:
ALTER TABLE [table-name] ADD [row-name] MEDIUMINT(8) default 0;
MySql will default all existing rows to the default value.
So here is what I would do:
make a copy of you're old database with MySql dump command.
run the resulting SQL file against you're new database, now you have an exact copy.
write a migration.sql file that will modify you're database with modify table commands and for complex conversions some temporary MySql procedures.
test you're script (when fail, go to (2)).
If all OK, then goto (1) and go live with you're new database.
These are all valid approaches, but I believe you want to write a sql statement that writes other insert statements that support the new columns you have.
How do you view all the constraints (Primary keys, Secondary keys, Assertions , Triggers, etc) in a MySQL Database / Table in the command line environment.
Just dump the database without data using
mysqldump --no-data (other options)
As if you were taking a backup. Use the same options as you do when taking a backup (maybe --lock-tables=0 - you don't need a lock when dumping the schema)
Without the data, you get just the schema which includes all those things you said above.
DESCRIBE is an alias for SHOW COLUMNS, like a shortcut. If you want all the other stuff then you need to use SHOW commands for your stuff. And SHOW COLUMNS for the rest.
Describe
Show