I need to read a column from a database table depending upon some parameter. If the database table has two columns, status and ID, then I have to read the ID if the status is true. Then I have to pass this ID to a C# method.
How can I achieve this in SSIS? So basically my database package will read the data from SQL Server and pass it to a C# method.
SSIS is an ETL Tool for moving and transforming large quantities of data. If you need to do a lot of C# work and you only have one record, or a few records, SSIS may not be the right tool for this purpose. You might do better writing an ASP.NET web application or a Windows application. These applications can also use SQL to get data for processing in C#.
If you are determined to do this in SSIS and C#, here are two possible approaches:
You could use an Execute SQL Task to perform your query and save the rowset into a variable. Then you would use a C# Script Task to do something with the contents of the variable.
You could create a Data Flow Task. The dataflow should have the structure Source -> Transformation -> Destination, and can include several transformation components.
You would use, for example, an OLE DB Source Component to perform your query. Then you would use a C# Script Component to transform each record that is returned by the query. Finally, you would use a OLE DB Destination Component to do something with the output for each record.
Related
I want migrate data from salesforce to SQL server and I am using SSIS connectors for salesforce. I am creating single SSIS package which fetch data for all objects and insert into SQL server. I tried using following connectors for salesforce.
Connector 1 : Kingswaysoft
https://www.kingswaysoft.com/
Connector 2 : CData
https://www.cdata.com/kb/articles/ado-ssistask-sf.rst
Connector 3 : SSIS PowerPack -
https://zappysys.com/onlinehelp/ssis-powerpack/index.htm
https://zappysys.com/products/ssis-powerpack/ssis-salesforce-source-connector/
In all the connectors I am unable to provide different columns(salesforce fields) dynamically in SOQL query using SSIS variables.
I agree with the comments that SSIS is for static ETL & you can work around with C# script task for dynamic metadata.
As an Alternative, you can try conditional branches and run two different tasks based on expression. Read Add expressions to precedence constraints.
Not sure how many dynamic columns we are talking here, but for discussion sake let’s take 2 different columns has to be filled in salesforce destination based on source column, then have 2 branch.
Is there any way in SSIS to send mail dynamically with dynamic subject, dynamic recipients, dynamic body itself? It is possible with database tables but I dont want to use database tables and stored procedures in my package.
SSIS is an in-memory ETL (Extract, Transform and Load) tool that is powered by the .NET framework (and some COM hackery). It does have a Send Mail Task which is capable of sending emails.
To make them as dynamic as you describe, you'll want to use a combination of Expressions, as Aryadev linked, coupled with Variables.
Most every thing in SSIS exposes a set of Properties that support configuration so what I generally suggest is that people create variables and perform all their expressions on the variables and then assign a variable to a property's expression. While it seems like more work, when you have to troubleshoot a package you'll be grateful. You can set a breakpoint in a package and inspect the current value of a Variable but you can't examine the run-time value of a property.
Finally, unless you're doing ETL stuff in addition to email, I'd skip the overhead of SSIS in favor of just using sp_send_dbmail in a TSQL space or call into the .NET smtp library.
Yes. Look into SSIS Expressions.
First, a little background:
We have been using SSRS in LocalMode for processing. We have an Object Data Source we've been using. This Object is serialized from an XML column in the database.
We're now forced to move to Server Mode due to memory leaks in Local Mode. As such, I'm rebuilding the reports. The problem I've run into is that I want to pull the XML data out of the database as an object but can't seem to figure out how to do this. I know I could create a web service or URL to provide it to the SSRS, but I want to avoid this if possible.
Is there a way to pull an XML column from a DB Data Source and get all the elements of it as the "Fields"?
Are you asking how to query xml from on the SQL Server? There's plenty of query examples online. It still returns as a dataset but you can modify it in custom assembly to return a String that can become the query for your xml dataset inside the report.
I have an Oracle query and I want to export the query results to an excel file daily. I've looked into both SSRS and SSIS and am not sure which would be better to use.
The query is a normal select that returns 10-20 fields. It is pretty straight forward with a couple joins and where clauses. It selects DISTINCT to get rid of duplicate rows.
It's a straight mapping from the query to the excel file.
Does SSIS have performance advantages over SSRS?
I was leaning toward SSRS because it's very simple to set up and there are added benefits of being able to easily run our extract/report with different dates through the SSRS web UI.
SSIS seems like it will be more complex to set up, but still simple. However, it seems I would have to handle how to rename the extracts without using the main excel "template" so there are more steps involved. Also having issues getting parameters to work with Oracle queries.
Even though I am a big fan of SSIS, I would go with SSRS in this scenario.
Your requirement is that you simply need data in an Excel file. Though both SSIS and SSRS can do this task. SSRS has slight advantage in what you are trying to achieve.
You can format the Excel file in SSRS report however you would like to.
Similar to SSIS package that is easy to configure, SSRS also has easier development process. You can design and populate however you would like.
SSIS requires a SQL job to schedule it in order to run the package and then send you the Excel file or save it to some location. However in SSRS, you can simply create a subscription and export the Excel file to particular folder or send it to you in an email.
If you ever want to change the file export format, SSRS already does that for you.
Some of the points that I could think of.
This isn't a report, so don't use Reporting Services.
The SSIS package necessary for this is a single Data Flow task with two components: an "OLE DB Source" for your Oracle query, and an "Excel Destination". Draw a connection between the two components, configure them, press F5 and you're done.
Almost any property in SSIS can be set to the value of an expression. This includes the "ExcelFilePath" property of the Excel Connection Manager. Simply set that to an expression that appends the date to the file path, and you'll be set as long as you only run the package daily.
If you need to run it more than once a day, then simply precede the Data Flow task with a File System task to delete any previous version of the file.
Just tried this quickly myself and found one small issue. The data source I used included VARCHAR columns. The Excel Destination wanted Unicode, so I had to place a Data Conversion component between source and destination.
This link has a nice evaluation of the case you are presenting:
http://theruntime.com/blogs/gscarfone/archive/2009/07/15/data-dump-to-excel-through-ssis-and-ssrs.aspx
Basically it depends on your particular scenerio.
Hope it helps...
We are considering migrating to Sql Server 2005 Reporting Services. Many of our existing reports require pre-processing of the data before rendering it.
For example, we have a query for a report that returns GPS coordinates (Latitude and Longitude) from a stored procedure, but before passing off the DataSet to our reporting engine (currently Crystal) we call out to a Web Service to reverse geocode the coordinates and get an address string. We push this into the DataSet object.
I've read a bit about Data Processing extensions but I'm not sure that's what I want, as then (if I understand correctly) I would need to implement the entire processing flow (including retrieving the data from the stored proc) just to massage it a bit at the end.
How can I interject and pre-process the dataset after retrieving it from the data source, but before passing it on to the renderer?
You could create a .NET assembly with a method in it that processes an individual record. Then, include that assembly in the report, and call that method when each row is being rendered. This would do the processing and display the result.
For instance, after creating the assembly and adding it to the report, you could have a table where one of the cell expressions looks like this:
=Code.ReverseGeocode(Fields!Latitude, Fields!Longitude)
See http://msdn.microsoft.com/en-us/library/ms155798.aspx for some guidance.
You can also use the results of an SSIS package as a data source (the package is executed on demand when the report runs).
This can be very helpful if you are trying to solve the problem of joining data from multiple data sources into a single dataset for example: a geocode web service and a relational result set.
I don't think a lot of people realize that this exists and how useful it is for handling EII type reports
One option would be to use a ReportViewer control in local mode.
The data is loaded into ASP-NET, you can the pre-process it, then pass it to the control for presentation.
I'm not sure of how SSRS would deal with custom code/DLL when you want to process all data before thinking about presentation (as in the =Code.ReverseGeocode solution mentioned). RBAR feels wrong at the text box or cell level.
I guess I don't understand why you just don't write the SQl to provide the data the way you want it. Can't you calculate the field through a function or use a where clause to filter the data further. You can do quite extensive data manipulation in a stored proc or query data source.