I am in a process of generating reports using SSRS. I have multiple servers with multiple oracle databases on each server. I am wondering if I can create multiple shared data sources, and shared data sets, and create one reports by switching data sets and data sources.
Otherwise I will have to create multiple reports for each data source, which can run into 100's of reports.
Any suggestion help would be highly appreciated..
Thanks
Nirmal
Almost everything in Reporting Services is an expression, including the SQL Statement of the dataset. This means it can be altered on the fly. Assuming the datasource credentials you use can access the databases you want to get to, then you just supply the database as a parameter and you're good to go. Of course, for databases on other servers you will need to use linked servers so the server you connect to can link across to the other server to access the database.
We have a table with a nice user readable name for the database such as "End of Financial Year 2009" which holds the database name for that data. Create a dataset to use this as a parameter - display the nice name as the label and get the server+databasename connection string from the value.
Then your dataset just looks like:
="SELECT * FROM " & Parameters!Database.Value & "TableName"
This assumes the databases have the same structures as far as the report's needs are concerned.
You have to set the fields manually but it gives you flexibility.
Related
My Application's data is stored in both MongoDB and MYSQL. Is there any way to write queries in Metabase that will help in fetching data from both MYSQL and Mongo Databases?
I'm assuming here that by "My application's data" you are refering to the data sources, not to the Metabase internal databasa (aka metadata).
As of now (0.32.10), there is no way to query data from two different data sources.
What you can do, though, is to setup two (or more, depending on your need) different questions, and add them to a dashboard - that way, you will be able to show the data in a "data source agnostic" way.
If I didn't not get it wrong, I think this will be somewhat possible in the incoming version (0.33), scheduled for the next weeks (I believe), using the new join system (which will allow you to join data from different data sources, given a certain common key - say, "order_id" or something like that).
You can get more info, and even test the new version (which is currently in the RC2) at
this link.
I am implementing a SSIS package and currently trying to do the following.
Truncate the destination table
Fetch the data by executing the stored procedure and insert it into the destination table.
I have created an Execute SQL task to address step 1 and dataflow with oledb source and oledb destination to address the second point. It been working successfully so far but isn't working for one my stored procedure that uses temp tables.
When I edit the oledb source and click the preview button, I get the error no column returned
I know that SSIS has an issue with generating column while executing stored procedures that depend on temp tables. I have converted the stored proc to use temporary table variables and its now able to return columns in SSIS when I do a preview. The only downside is that the stored procedure is taking longer time to execute. Its taking 1 hour 15 mins as compared to 15 mins while using temp tables.
I did see a suggestion to use SET FMTONLY before executing the stored procedure as an alternate solution to changing to temp table variables but that didn't seem to work as I am getting syntax or permission denied error.
Could somebody tell me a solution to my problem which does not compromise on the performance.
Sounds like you've already read all the approaches to using Temp tables in SSIS, including the IF 1=0... trick? If you haven't seen that one yet, google it.
You say that using Table Variables causes your stored procedure to take about 5 times longer than using Temp Tables. The most likely reason for that is that you are indexing your temp tables but not your table variables. If you didn't know that table variables can be indexed, they can. You might try that.
Finally, a solution that you haven't mentioned is that you can replace your temporary table with a real table that gets truncated when you're done using it.
Short comment:
Try EXEC WITH RESULT SETS and specify the metadata yourself for a proc with temp tables; or use the Script Component as a source and specify the Output columns yourself.
Long comment:
Technically speaking, it is the driver/database you are using in SSIS that would decide the behavior when working with temp tables.
Metadata is an important factor when using SSIS's pipeline components. By metadata, I mean the names of the columns, their data types etc that a pipeline component uses. When designing a data flow, someone/something should provide this metadata to the components that require it.
In most cases, SSIS automatically retreives the metadata. Components that do not connect to a external data source, like Conditional Split etc, get their metadata from the other components they are connected to. For the pipeline components that connect to a external data source (like Oledb source, oledb destination, Lookup etc.), SSIS provides a mechanism to get this metadata without human involvement. This mechanism involves the driver connecting to the database and retrieving the metadata of the output. If the driver/database is capable of returning the metadata, then that metadata is used. If the driver/database is incapable, then you get the errors you are seeing. The rest of my comments are based on the assumption that you are using a SQL Server database in your question.
When working with a SQL Server database in SSIS, typically, we use the native client drivers provided by Microsoft. When trying to get the metadata, these drivers try to get the metadata without actually executing the SQL Statement (actual execution can have side effects; and also, might take more than a few seconds/minutes/hours; and you dont want side effects and long wait times during package design time.) So to get the metadata, the driver relies on the metadata of the actual objects used in the sql command. If the command uses a physical table or view, SQL Server already has the metadata available and can supply it to the driver. If it is a temp table, SQL Server does not have the metadata until it can create the temp table. If using FMT ONLY option, you can use it in such a way to create the temp tables, but avoid any heavy processing/side affects and thus be able to retrieve metadata without penalties. Post 2012, these native client drivers rely on some newer functionality to retrieve metadata than the drivers before 2012. In 2012 and after, the driver uses the sp_describe_first_result_set proc to retrieve metadata. So, whether you can get metadata or not is determined by the ability of the sp_describe_first_result_set proc.
So while SSIS can automatically get the metadata (because of the driver/database), it does not automatically get the metadata in some cases (again because of the driver/database). In cases involving the second scenario, some other process (typically a human) can help the driver infer metadata or provide the metadata to the component directly.
To help the driver, in case of SQL Server 2012 and after, you can use the WITH RESULTSETS clause to specify the output metadata. When this clause is present, the driver will use it and doesnt try to query the metadata from system objects; and thus avoid the error which you would otherwise get. If you are using the drivers that came with SQL Server 2008, you can use FMT ONLY. This option is at the driver/database level.
Another option could be to use a Script Component as the Source and in the Output columns, you can specify the columns/metadata. SSIS would not try to retrieve metadata from the datasource in this case, but would rely on the definitions you provided in the Output section of the Script Component.
As you can see, both options involve a human (or some other process) specifying the metadata instead of SSIS trying to retrieve the metadata in an automated fashion. I would prefer the first option if working with SQL Server and the second option if working with databases like MySql.
I have an SQL Database (SQL Server 2008 r2) that I link to Tableau via a Custom SQL Query (connected to a view in SQL). Instead of adding the new fields in the SQL Database, I am creating calculated fields in Tableau. Thedata source connected to Tableau is a viewI created by joining other views (there is a pivot I join to the main table). I want to now add the parameters from tableau as new fields in SQL and remove them from Tableau since it is bogging the tool down. What do you recommend to be the best approach in this? I have never added fields to a view and curious how this can be achieved. All of the parameters work in tableau only via the custom SQL query (the view created), but would not if I created them from the main data table in SQL.. I hope this makes sense. Thank you so much for the help! Have a great day -p
its always a good practice to create all your required fields calculation in sql table which will improve your performance. and create a extract connection with your table instead of live.
I'm new to Talend. I want to make a script that transform my data (using an sql query) from a database and insert them to an other one.
Could you help me please with the list of components that I should use ? any tutorial ?
Thank you
Talend has many database specific components. You will find input and output components for all databases. input components are those which enables you to read data from database and bring it to talend and output components are those which allows you to write/insert data to database.
for example if your source and target are both oracle then -
toracleinput will be for reading from oracle db - (you have to mention connection details, sql query and schema for this component).
toracleoutput will be for inserting this data back to oracle database table. Here again you have to mention connection details, target table name, type of operation etc..
finally you have to connect flow of data from input to output component and if you need any intermediate transformations, joins etc you have to user other talend components in between like tMap etc..
I have a reporting database and have to transfer data from that to another server where we run some other reports or functions on Data. What is the best way to transfer data periodically like months or by-weekly. I can use SSIS but is there anyway I can put some where clause on what rows should be extracted from the source database? like i only want to extract data for a current month. Please do let me know.
Thanks,
Vivek
For scheduling periodic extractions, I'd leave to that SQL Agent.
As for restricting the results by some condition, that's an easy thing. Instead of this (and you should always use SQL Command or SQL Command From Variable over Table Name/Table Name From Variable as they are faster)
Add a parameter. If you're use OLE DB connection manager, your indicator for a variable is ?. ADO.NET will be #parameterName
Now, wire the filter up by clicking the Parameters... button. With OLE DB, it's ordinal position starting at 0. If you wanted to use the same parameter twice, you will have to list it each time or use the ADO.NET connection manager.
The biggest question you will have to answer is how do I identify what row(s) need to go. Possibilities are endless: query into the target database and find most recent modified date for a table or highest key value. You could create a local table that tracks what's been sent and query that. You could perform an incremental load / ETL Instrumentation to identify new/updated/unchanged rows, etc.