I have a SQL Server 2008 db. It contains some great test data...a proposal that has a lot of child records. I want to script this Proposal so as not to re-enter the data for each new environment. Is there a way to script the data in the proposal as well as all the dependent tables. I can run a query to get the data but what I really need is a set of SQL Insert Statements to insert the data into the new environments. At least that's what I think I need. Any ideas?
Several options for this exist.
If you're a DIY person and want a manual option with no support, check out Vyas' script:
http://vyaskn.tripod.com/code.htm#inserts
CodeProject article:
http://www.codeproject.com/Articles/5598/Generating-INSERT-statements-in-SQL-Server
Red Gate SQL Compare (to build the schema in the destination) and Data Compare (to pump data over there after the tables are created):
http://www.red-gate.com/products/sql-development/sql-compare/
SSMS Tools Pack:
http://ssmstoolspack.com/Features
If you dont want to use 3rd party stuff, you can pretty much achieve this with SSMS out of the box, simply right click the database, hit "Generate Scripts", follow the wizard, on the options step, set "Script Data" to true (or, if your using 2012, set "Types of data to script" to "Schema and Data").
http://msdn.microsoft.com/en-us/library/ms178078.aspx
http://msdn.microsoft.com/en-us/library/ee210523.aspx
Related
I have two databases: Sybase and MySQL. I need to export records to MySql when these are inserted in Sybase or export in some scheduled event.
I've tried with output statement but this can not be used in triggers or procedures.
Any suggestion to solve this problem?
(disclaimer, I've done similar things previously, but by no means would I consider the answer below the state of the art - just one possible approach
google around something like 'cross-database replication' or 'cross rdbms replication' to see who's done this before.
).
I would first of all see if you can't score an ETL tool do the job without too much work. There are free open source ones and even things like Microsoft SSIS might work on non-MS databases.
If not, I would split this into different steps.
Find an appropriate Sybase output command that exports a subset of rows from one or more tables. By subset I mean you need to be able to add a WHERE clause, not just do a full table dump.
Use an appropriate MySQL import script/command to load the data gotten out of step #1. You may need to cycle back and forth between the 2 till you have something that works manually.
Write a Sybase trigger to insert lookup keys into a to-export table. You want to store at least the tablename & source Sybase table's keys for each inserted row. Use column names like key1_char, key2_char, not the actual column names, that makes it easier to extend to other source tables as needed. keep trigger processing as light as possible. What about updates btw?
Write a scheduled batch on Sybase side to run step #1 for the rows flagged in #3.
Write a scheduled batch on Mysql to import ,via #2, the results of #4. Or kick it off from #4.
Another approach is to do the #3 flagging bit as needed, but use to drive one scheduled batch that SELECTs data from Sybase and INSERTs it into mysql directly.
You'll have to pick up the data from Sybase's SELECT and bind it manually to the INSERT of mysql. But you probably get finer control over whats going on and you don't have to juggle 2 batches. That's what I think a clever ETL would already be doing on your behalf. Any half clever scripting language like php, python or ruby ought to handle it easily. Especially important if you have things like surrogate/auto-generated keys.
Keep in mind that in both cases you'll have to either delete the to-export rows that you've successfully inserted or flag them as done.
I have a fairly straight way to copy selective data using SQL like so:
# Courses
DROP TABLE db_node.courses;
CREATE TABLE db_node.courses LIKE db_prod.course_sis;
INSERT INTO db_node.courses SELECT
*
FROM
db_prod.course_sis
WHERE
db_prod.course_sis.enabled = 1
AND db_prod.course_sis.hidden <> 1;
This is easy when I am on the same server with the same db, however I want to run this SQL to get put the final data on the SQL Server.
This isn't just a once off thing, it would need to be handled every hour or so. I am unable to change db's, one will always be MySQL and SQL Server because the data is used in a sharepoint app.
Thanks
There are several third-party tools to make this transition easier.
Check out SSMA here and here.
I'd also look at SQL Studio scheduled tasks to automate the process.
I have two databases DB1 and DB2. Now there are some stored procedure and views in DB1 which are related to DB2 also like this:
SELECT ....
FROM DB1.TableA INNER JOIN DB2.TableB
.....
Now, as I have to move these two database to the other server with scripts. I have to make sure that if I am running the script of DB2 I would first run script forthat DB1 object. I am not sure if I can run the script one by one or do I need to prepare one script which will be made based on the dependency?
Any suggestion ?
If you want to generate scripts for moving data, I would suggest you use the tasks -> generate scripts option on the database you want to move. You can choose whether you to script data and schema, or just schema, whether to script indexes, foreign key relationships, and so forth. Quick guide here:
http://blog.sqlauthority.com/2011/05/07/sql-server-2008-2008-r2-create-script-to-copy-database-schema-and-all-the-objects-data-schema-stored-procedure-functions-triggers-tables-views-constraints-and-all-other-database-objects/
If I were you though, I would download the trial version of RedGate SQL Compare. It has 2 tools called Schema Compare and Data Compare, which allows you to compare 2 databases, and synchronize them schema and data wise. It is a brilliant tool, and we use it everyday to make sure our databases are in sync.
Also, if you have linked servers set up like you say above (DB1 referencing DB2), then it might be worthwhile to take both databases offline, when the traffic on the website is low.
Alternatively, you can create a replica to DB1 and DB2, using SQL Compare/Generate Scripts, and then switch your connection strings to point to the replicas, once they have been ported to the new servers. That way, the switch is instantaneous, and you do not have to take the databases offline.
I have a reporting database and have to transfer data from that to another server where we run some other reports or functions on Data. What is the best way to transfer data periodically like months or by-weekly. I can use SSIS but is there anyway I can put some where clause on what rows should be extracted from the source database? like i only want to extract data for a current month. Please do let me know.
Thanks,
Vivek
For scheduling periodic extractions, I'd leave to that SQL Agent.
As for restricting the results by some condition, that's an easy thing. Instead of this (and you should always use SQL Command or SQL Command From Variable over Table Name/Table Name From Variable as they are faster)
Add a parameter. If you're use OLE DB connection manager, your indicator for a variable is ?. ADO.NET will be #parameterName
Now, wire the filter up by clicking the Parameters... button. With OLE DB, it's ordinal position starting at 0. If you wanted to use the same parameter twice, you will have to list it each time or use the ADO.NET connection manager.
The biggest question you will have to answer is how do I identify what row(s) need to go. Possibilities are endless: query into the target database and find most recent modified date for a table or highest key value. You could create a local table that tracks what's been sent and query that. You could perform an incremental load / ETL Instrumentation to identify new/updated/unchanged rows, etc.
I need to convert data that already exists in a MySQL database, to a SQL Server database.
The caveat here is that the old database was poorly designed, but the new one is in a proper 3N form. Does any one have any tips on how to go about doing this? I have SSMS 2005.
Can I use this to connect to the MySQL DB and create a DTS? Or do I need to use SSIS?
Do I need to script out the MySQL DB and alter every statement to "insert" into the SQL Server DB?
Has anyone gone through this before? Please HELP!!!
See this link. The idea is to add your MySQL database as a linked server in SQL Server via the MySQL ODBC driver. Then you can perform any operations you like on the MySQL database via SSMS, including copying data into SQL Server.
Congrats on moving up in the RDBMS world!
SSIS is designed to do this kind of thing. The first step is to map out manually where each piece of data will go in the new structure. So your old table had four fields, in your new structure fileds1 and 2 go to table a and field three and four go to table b, but you also need to have the autogenerated id from table a. Make notes as to where data types have changed and you may need to make adjustments or where you have required fileds where the data was not required before etc.
What I usually do is create staging tables. Put the data in the denormalized form in one staging table and then move to normalized staging tables and do the clean up there and add the new ids as soon as you have them to the staging tables. One thing you will need to do if you are moving from a denormalized database to a normalized one is that you will need to eliminate the duplicates from the parent tables before inserting them into the actual production tables. You may also need to do dataclean up as there may be required fileds in the new structure that were not required in the old or data converstion issues becasue of moving to better datatypes (for instance if you stored dates in the old database in varchar fields but properly move to datetime in the new db, you may have some records which don't have valid dates.
ANother issue you need to think about is how you will convert from the old record ids to the new ones.
This is not a an easy task, but it is doable if you take your time and work methodically. Now is not the time to try shortcuts.
What you need is an ETL (extract, transform, load) tool.
http://en.wikipedia.org/wiki/Extract,_transform,_load#Tools
I don't really know how far an 'ETL' tool will get you depending on the original and new database designs. In my career I've had to do more than a few data migrations and we usually always had to design a special utility which would update a fresh database with records from the old database, and yes we coded it complete with all the update/insert statements that would transform data.
I don't know how many tables your database has, but if they are not too many then you could consider going the grunt root. That's one technique that's guaranteed to work after all.
If you go to your database in SSMS and right-click, under tasks should be an option for "Import Data". You can try to use that. It's basically just a wizard that creates an SSIS package for you, which it can then either run for you automatically or which you can save and then alter as needed.
The big issue is how you need to transform the data. This goes into a lot of specifics which you don't include (and which are probably too numerous for you to include here anyway).
I'm certain that SSIS can handle whatever transformations you need to do to change it from the old format to the new. An alternative though would be to just import the tables into MS SQL as-is into staging tables, then use SQL code to transform the data into the 3NF tables. It's all a matter of what your most comfortable with. If you go the second route, then the import process that I mentioned above in SSMS could be used. It will even create the destination tables for you. Just be sure that you give them unique names, maybe prefixing them with "STG_" or something.
Davud mentioned linked servers. That's definitely another way that you can go (and got my upvote). Personally, I prefer to copy the tables over into MS SQL first since linked servers can sometimes have weirdness, especially when it comes to data types not mapping between different providers. Having the tables all in MS SQL will also probably be a bit faster and saves time if you have to rerun or correct portions of the data. As I said though, the linked server method would probably be fine too.
I have done this going the other direction and SSIS works fine, although I might have needed to use a script task to deal with slight data type weirdness. SSIS does ETL.