Often I need to pull the production database of some project to my local sql server to add features, test out stuff etc.
Today my procedure is to create a backup on the production server, somehow get that to my local machine - and then create a new database locally and restore the backup on top of that.
It is a pain, and takes more time than I like - and I would like to think, there must be a better way.
I have access via SQL Server Management Studio to the production database - isn't there an easier way, that requires fewer manual steps?
How do you do it?
Can't think of a quicker way using SQL Sever Management Studio. I'd recommend SQL Compare from Red Gate for synchronising the schema, SQL Data Compare can sync the data, but it's not quick for large databases over the internet.
You can use SSIS and copy objects between these environments, assuming you have a direct connection to the production server.
Another way, it's a must to make regular backups of the production databases, a simple maintenance plan can make a full backup at night, by example. If this is the case here, just request an early backup to the administrators and then mount it into your environment.
One more way, because production data disclosure can involve legal issues, you can just extract the database schema and then use testing data to make any development. This is also the fastest way to get a database.
A new option is to clone the database. Red Gate SQL Clone is one solution, and Windocks provides SQL Server containers with built-in DB cloning support. Full disclosure, I work for Windocks.
Backup and Restore is slow for my databases, so what I do is:
1. Detach production database
2. Copy files to my dev machine
3. Attach database to dev server.
but no body should work on the production database.
My backup procedure is similar - I am detaching the database and archive the files, it is faster than backing up.
Related
I am working at a company that has some CRM software running in a remote Windows XP server that uses a SQLAnywhere 9 db to store its data; I have access to this remote server with an administrator account.
I would like to extract the db into a .sql file so that I can run the db locally on my machine without affecting the running db in the server (since it is key for the company's day to day operation).
The reason I need this is that we are going to test some BI Software and we need data from this database to test it, but we don't know the structure of the database since the developers of the CRM software didn't give us any documentation on it. So we need to have the database locally so that, without affecting the running CRM, we can:
understand the structure by looking at the DDL
make queries to it to get sample data
I researched a bit, and the most common solution to my problem was to use dbunload on the remote server to unload the db into a reload.sql file that contained what I needed. But most tutorials on the subject mention that I have to stop the db first (which would be catastrophic). If this is the only option, then I guess I am willing to do it on the weekend when the CRM is not used, but I wanted to know if there was another solution first.
If there is no other solution, can you point me to where I can find the proper and safer way to do this?
I have researched a lot, but prior to this day I have never even heard of SQLAnywhere, so I really need all the help I can get. My main concern is doing something that impacts negatively the CRM software.
Thank you.
You can run dbunload across the network, you just have to tell it to do an "external" unload. The default is to do an internal unload which would only work from the machine where the database server is running.
I don't have SQL Anywhere 9 documentation right now to look up the exact switch, but dbunload -? should show you all the possible switches.
Edit:
-an will create a new database and load the data and schema from another data
-xi switch will do external unload and internal reload.
-c parameters to connect to your remote database
I have a situation where I would like a desktop application to be useable whether an internet connection is present or not.
I have a choice of MySQL on a web server and I could work with a local MySQL database or maybe MS Access database on the local drive and then just update data when connection is restored. My issues are as follows.
Sync local changes to remote server. Multi site / multi user scenario so how to keep db in sync when connection restored without loosing changes from other users in server data.
Sync remote changes to local. Multi site / multi user scenario so how to keep db in sync when connection restored without loosing changes made locally while updating with server data.
Currently I am using XML files and LingtoXML querying but it is unsatisfactory to continue with these files so a better solution is required.
Any help would be appreciated to identify what technology would work best and how to keep them in sync.
Thanks in Advance.
"Jet Replication Objects (JRO)", the replication features of the Access Database Engine, have been deprecated (ref: here). I believe that the related management features have also been completely removed from Access 2013. Therefore, native Access replication should no longer be considered a viable option.
Microsoft's recommendation would be to use SQL Server and its replication features. However, SQL Server Express has limitations on how much it can do (e.g., it can be a "Subscriber" but not a "Publisher" or "Distributor", ref: here) so presumably there would have to be a non-free copy of SQL Server involved somehow.
I haven't yet had the occasion to use MySQL replication myself, but it is probably worth considering. Chances are good that you could still use Access as a front-end (via ODBC linked tables).
I'm working with MySQL databases.
To simplify the problem, let's say I have two environments : the local one (development) and the remote one (production mode).
In the database, I have some tables that contain configuration data.
How can I automate cleanly the delivery from the development mode to the production mode when I modify the database schema and the configuration tables content ?
For instance, I dot it manually by doing a diff between the local and remote databases. But, I find that method not so clean and I believe there is good practice allowing that.
This might be helpful in cases where you have multiple environments and multiple developers making schema changes very often and using php.. https://github.com/davejkiger/mysql-php-migrations
Introduce parameter "version" for your database. This version should be written somewhere in your code and somewhere in your database. Your code will work with database only if they have equal versions
Create a wrapper around your MySQL connection. This wrapper should check versions and if versions are not compatible, it should start upgrade.
"Upgrade" is a process of sequential applying the list of *.sql files with SQL commands, which will move your database from one state to another. It can be schema changes or data manipulation commands.
When you do something with database, do it only through adding new *.sql file and incrementing version.
As a result, when you deploy your database from development enviroment to production, your database will be upgraded automatically in the same way as it was upgraded during development.
I've seen LiquiBase http://www.liquibase.org/ a lot in Java environments.
In most of my projects I use sqlalchemy(a Python tool to manage db plus an ORM). If you have some experience(little more than beginner) with Python I highly recommend using it.
You can check this tool with a little help of that. This is also very useful for migrating your db to other rdbms(for example mysql to postgres or oracle).
I need to migrate data from SQL Server 2000 to MySQL. Currently I am using MySQL workbench. Can any one tell how to do this?
If you are searching for a tool MySQL has a tool called "MySQL Migration Toolkit" that can be used to migrate the data from SQL to mysql. But the first thing as mentioned above is to do a backup. The next thing to check would be whether there are any datatypes that cannot be converted?
Exactly, what have you tried? You can quickly migrate data from MSSQL to MySQL if they are in any of the following data file formats:
Paradox (.db)
DBase (.dbf)
Delimited Text (.txt)
Excel (.xls)
XML (.xml)
MS Access Database (.mdb)
ODBC
If its a one-to-one, exact same database architecture on both the new and the old servers, you might want to try using database tools meant to make this an easier process for GUI based administrators. Just go to download.com and find some software that may assist you in that migration, Navicat is a good one. The most important thing is to always BACK IT UP! BACK IT UP! BACK IT UP! Never do anything without mirroring drives and doing whatever it takes to make sure if you don't destroy any data, but if you do you'll have backup copies of it. Also how fast your machines are will be a sizable factor when it comes to migrating very large databases.
All in all you have many options to choose from, yet the most important thing is to back it up! Can't stress that enough, yeah it might seem like meaningless extra work especially on humongous database systems, but trust me, its better to be safe than sorry. Also, I always like rebooting machines prior to making database changes to them, cutting off any connection to the outside world or any processes depending or updating its data. Turning off the ODBC will do much of that for you on Windows as well, but as always better safe than sorry. Many a corruption can be avoided by simply booting the machine and having all data in memory finalized on the active database, before backing it up or appending to it.
Check out etlalchemy. It is a free, open-sourced Python tool capable of migrating between any of the following SQL databases: PostgreSQL, MySQL, Oracle, SQL Server, and SQLite.
To install: pip install etlalchemy
To run:
from etlalchemy import ETLAlchemySource, ETLAlchemyTarget
# Migrate from SQL Server onto MySQL
src = ETLAlchemySource("mssql+pyodbc://user:passwd#DSN_NAME")
tgt = ETLAlchemyTarget("mysql://user:passwd#hostname/dbname",
drop_database=True)
tgt.addSource(src)
tgt.migrate()
Our Java server application logs data to a SQL database, which may or may not be on the same machine. Currently we use MS SQL Server, and we're now porting to MySQL. A user configures database backup parameters on our app server, e.g. time of day to run a backup, and the app server executes SQL Server's BACKUP DATABASE command at the appropriate time, via a sproc. It does incremental backups daily and full backups weekly.
MySQL lacks an equivalent feature to tell the database from a client connection to back itself up. Options we're considering are:
Create a UDF to shell out to mysqldump (or copy database files), which can be called from our app server via a sproc. Essentially we'd be implementing a version of BACKUP DATABASE for MySQL.
Create a service to run on the MySQL box that can get the backup settings from the app server and run mysqldump (or file copy) locally.
Create a backup sproc to mimic mysqldump, e.g. SHOW CREATE TABLES and SELECT INTO OUTFILE for each table.
Setting up a cron job, Perl script, third-party app or other tricks that'd work great in a data center aren't preferred; this is a shrink-wrap package that needs to be pretty robust and hands off.
Database sizes can range from roughly 10MB to 10GB.
I'm aware of the binary logs for the incremental piece. I figure the general solution will probably apply to them as well, if we decide to use them.
This is all on Windows 2003 32-bit or 2008R2 64-bit, MySQL 5.1.
The UDF option seems the best to me. The UDF Repository (http://www.mysqludf.org/) has mysqludf_sys, which may be all we need, but I thought I'd ask for opinions since after extensive googling it doesn't seem like others have reached the same conclusion, or maybe our needs are just out of the ordinary. Our app is the only thing in MySQL, so I'm not worried about other users having access to our UDF.
Any solutions I'm overlooking? Any experience with using UDFs in such a way?
Thanks,
Eric
For this an other reasons we decided to collocate our application with the database, so this problem became moot.