I am working with a SQL Server 2008 R2 BIDS 2008 solution.
I need to transfer data from an Oracle(11g) source database to a SQL Server database. I need to copy all data from multiple tables in the Oracle database. I have a ForEach container in which there is one Data Flow Task.
I want to loop round a list of table names, select all columns out of in the Oracle source and copy to dest_ in the SQL Server.
I have added a Property Expression to the Data Flow Task for [ADO NET Source].[SqlCommand] with an expression of "SELECT * FROM " + #[User::ImportPath]
As the table is undefined at this point it isn't clear if I can map columns or setup the ADO NET Destination task correctly.
Is there steps I am missing? Is what I am attempting to do even possible?
You cant loop through tables and dinamically map columns on your source and destinations components. You would need one set of Source -> Ddestination per table.
If that's not feasible, you may want to lokk at the Transfer SQL Server Objects Task
Related
I have a current package which basically copies an Excel template sheet, writes data into this from a table in the SQL database and then sends to Excel destination (ishare drive and server drives). We have repeated rows with specific data which need to be removed from the Excel sheet before it is sent out. We have a VBA code which can be run in Excel as a macro in order to achieve this result. I am wondering how I can automate this in SSIS in the data flow?
I have a current package which basically copies a an excel template
sheet, writes data into this from a table in the sql db
You can use a Data Flow Task where you import data from the SQL Server database:
Create a connection manager :
When you click on the New Connection Manager option, an Add SSIS Connection Manager window form will open to select the connections managers from the given list.
Then select the Provider, Server Name, and Database Name (you will point to a SQL Server database you can either select the table or use a query)
If you want to remove duplicate entries you can use Drag Transformation and Connect your OLE DB Source to it. Double Click on Sort Transformation and Choose the columns to Sort. Also Check the Check Box : Remove rows with duplicate sort values and then click OK.
If you have other specific rules, you can implement a T-SQL query when importing data from the begining (when using the OLE DB Source)
Then you can link it to an Excel destination.
Monthly, I need to get "new" data from our SQL Server business application into an Access 2010 database that contains all our "Management Reports". The Access database has a "staging table" that is to contain the raw data on which the reports are based.
I have no Access experience, but I suggested that we:
Write a query (stored proc?) on our SQL Server that returns the required raw data (...this bit was easy.)
At the end of each month, call the SQL Server stored proc from within Access 2010 (...click a button?)
Save the results of the stored proc into the staging table within Access.
But I'm finding it harder than I expected. I think I can get something ugly working using ADODB in a code-behind, looping through rows in a recordset one by one, and then setting column values one by one. But there must be a better way :)
How should I go about getting SQL Server data from Access 2010? (ADODB? DAO? QueryDesigner? other?)
Is there a "Insert Recordset Into Table" (or similar) mechanism that I can leverage?
Link the relevant sql server table or view to MS Access. Run a query against the linked table using MS Access syntax and update the staging table.
It is also possible to update an MS Access table using a connection string for SQL Server in-line in your query.
SELECT *
INTO newtable
FROM [odbc;filedsn=Z:\DSN\test.dsn].table1
Working from the SQL Server end, you can use MS Access as a linked server or run a query and update from there.
INSERT INTO
OPENDATASOURCE(
'Microsoft.ACE.OLEDB.12.0', 'Data Source=z:\docs\test.accdb')...[table1]
( atext )
SELECT atext FROM table1 WHERE id=2
I have some requirements as explained below:
I need to have a SSIS data data flow task,which will accept the input parameters as
--SourceServer Connection String
--DestinationServer Connection String
--Source Table Name
--Destination Table Name
Then It should do the column mapping of source and destination tables dynamically at run time.
The source and destination tables schema would be same always.
I want to call this package by passing the above parameters from C#.net.
One main thing is here I will have to pass different sets of source and destination tables.
just answered this on a previous question. You cant loop through tables and dinamically map columns on your source and destinations components. You would need one set of Source -> Ddestination per table.
If that's not feasible, you may want to lokk at the Transfer SQL Server Objects Task
Create SSIS Packages parameters. Set web.config file for passing that parameters.
First you deploy the package in SQL Server.
Create one job for execute the package.
Create one sp using SQL Server.
& execute the job.
using sp_start_job.
I think it solve ur problem.
I have a reporting database and have to transfer data from that to another server where we run some other reports or functions on Data. What is the best way to transfer data periodically like months or by-weekly. I can use SSIS but is there anyway I can put some where clause on what rows should be extracted from the source database? like i only want to extract data for a current month. Please do let me know.
Thanks,
Vivek
For scheduling periodic extractions, I'd leave to that SQL Agent.
As for restricting the results by some condition, that's an easy thing. Instead of this (and you should always use SQL Command or SQL Command From Variable over Table Name/Table Name From Variable as they are faster)
Add a parameter. If you're use OLE DB connection manager, your indicator for a variable is ?. ADO.NET will be #parameterName
Now, wire the filter up by clicking the Parameters... button. With OLE DB, it's ordinal position starting at 0. If you wanted to use the same parameter twice, you will have to list it each time or use the ADO.NET connection manager.
The biggest question you will have to answer is how do I identify what row(s) need to go. Possibilities are endless: query into the target database and find most recent modified date for a table or highest key value. You could create a local table that tracks what's been sent and query that. You could perform an incremental load / ETL Instrumentation to identify new/updated/unchanged rows, etc.
I am trying to retieve data from a MySQL table and insert into a SQL Server table using ADO .NET connections in SQL Server 2008 SSIS. In my data flow task I have an ADO .NET Source which queries the MySQL table (select all invoices) and an ADO .NET Destination which inserts the data in my SQL Server table. Now, I would like to add a parameter in my data source so that I only select the max(invoiceNumber) retrieved from my SQL Server table. I have performed a similar task using "OLE DB Command" but the problem is that I need to query a MySQL database. Any ideas how I can achieve this?
I found that the only way to use parameter with the ADO.NET Data Source is this workaround:
Go to the Flow Control and selenct the Activity Flow containing your ADO.NET source.
In the properties window you can see the ADO.NET source Sql Command
Go to expression and select the properties: [YOU SOURCE NAME].[SqlCommand] and then edit the expression using variables to simulate parameters
Set Data Access Mode in ADO.NET Source to SQL Command and write the query.
You shoudn't have to add a parameter:
select *
from invoices
where invoiceNumber = (select max(invoiceNumber) from invoices)
The above works in SQL Server. I'm assuming that the same query will work in MySQL