I have two sql 2008 R2 servers, which are not linked. I need to read data from server A and write them to server B. Best way is to use SSIS.
But the data insert has to be generic, that means, I do not know the table structure. I have sql queries ready (when I run them on linked server, it works), but SSIS with oledb source and oledb destination needs table structure to do the column mapping.
how can I run dynamic sql task using SSIS and be able to read data from server A and insert them to server B (and then perform rollback on error)?
Your best bet is probably BiML, which dynamically creates SSIS packages based on meta data.
Related
I think I can use a for each container for doing this. There are multiple examples for SQL servers but not for other types. Can anyone suggest me how to do this in SSIS? (CRM doesn't have SQL database).
I am new to SSIS is there is any component to load data from MYSQL to SQL server using SSIS. Currently am loading data using ODBC connection it is really slow and it around the speed of 30000 rows/Minute. Is there any way to make the load run fast.
Thanks in Advance...
You can install the .NET Connector for MySQL: http://dev.mysql.com/downloads/connector/net/
Then you can create a script task to act as a data source, import MySql.Data.MySqlClient, and query MySQL directly in C#. The data will then enter your Data Flow and you can map it to a SQL Server destination the same as normal.
I find that when using the SSIS connection manager with .Net Providers I get malformed SQL errors, but this way you write all the SQL yourself.
To improve the performance, we can add Conditional Split Transformation build some parallelism into the common data flow that directly load data from ODBC Source to OLE DB Destination.
For more information about speeding up SSIS Bulk Inserts into SQL Server, please see the following blog:
http://henkvandervalk.com/speeding-up-ssis-bulk-inserts-into-sql-server
In DataFlowTask property, Increase buffer size and no of row commit
I have two databases with the same schema.
Database A : Production database
Database B : Test database
Database B contains outdated data and I want to synchronize it with the Database A.
I want all the data from Database A in DataBase B .
How can I do that without having to create a new database or detach the production database.
You can create a backup on the source server and then you can restore it on the destination server. (Please note, that all custom settings will be reseted on the destination).
If you do not want to restore all tables, then you can either build some SSIS packages which packages syncronize the tables. (Build one package per table based on a template package) or you can create a Linked Server on the destination server and write some stored procedures to sync the tables.
If you are familiar with SPs and dynamic queries, you can write an SP, which dynamically builds the syncronization query based on the table's meta data.
Another option is to use a third party software like Red Gate's SQL Data Compare
It is easy to schedule SSIS packages and SPs so if you want to run the sync based on a schedule, choose these options.
I need to read a column from a database table depending upon some parameter. If the database table has two columns, status and ID, then I have to read the ID if the status is true. Then I have to pass this ID to a C# method.
How can I achieve this in SSIS? So basically my database package will read the data from SQL Server and pass it to a C# method.
SSIS is an ETL Tool for moving and transforming large quantities of data. If you need to do a lot of C# work and you only have one record, or a few records, SSIS may not be the right tool for this purpose. You might do better writing an ASP.NET web application or a Windows application. These applications can also use SQL to get data for processing in C#.
If you are determined to do this in SSIS and C#, here are two possible approaches:
You could use an Execute SQL Task to perform your query and save the rowset into a variable. Then you would use a C# Script Task to do something with the contents of the variable.
You could create a Data Flow Task. The dataflow should have the structure Source -> Transformation -> Destination, and can include several transformation components.
You would use, for example, an OLE DB Source Component to perform your query. Then you would use a C# Script Component to transform each record that is returned by the query. Finally, you would use a OLE DB Destination Component to do something with the output for each record.
Problem Definition: I would like to export all the data from all the tables from QuickBooks into SQL Server 2008 database. I tried to create a SSIS and using RssBus .NET Data Provider for QuickBooks - http://www.rssbus.com/ado/quickbooks/. I am able to export data from QuickBooks Customers table into SQL Server 2008 database table by using ADO.NET (source data reader) and OLE DB Connection (for destination SQL Server).
Question: I would like to loop through all the tables on source side and then export one by one. Is it possible through SSIS package? I will really appreciate if someone can point good resource links or example or detailed instruction will be much appreciated!
Then I would like to do import from SQL Server to QuickBooks. I am expecting SSIS can be used to accomplish the import process.
Thanks in anticipation!
You can't loop over the tables, you have to define each source and target table in advance in the package.
Of course, you always have the option of creating packages dynamically from .NET code, so you could read the list of tables in QuickBooks then generate a package. If the table structures change a lot or if there is a very large number of tables it could be worth it (I know nothing about QuickBooks).