I'm trying to create a query that grabs records that fall within a specified time frame, hence BETWEEN. However, I need to do this from an interactive report which user can specify the endDate and startDate parameters.
I've read that RedShift may not necessarily support variables or parameters and that I may need to use temp tables, but my requirements are that users can pass in values. I'm confused on how using temp tables with pre-defined values allows for in-determinant values to be passed to the base query...
Here's my initial attempt (dont laugh)
prepare prep_select_plan(date)
AS select TOP 10 * from table WHERE date BETWEEN $1 AND $2;
EXECUTE prep_select_plan(#startDate);
EXECUTE prep_select_plan(#endDate);
DEALLOCATE prep_select_plan;
Is there another platform that would allow me to create a web based, interactive report with the ability to have end users enter parameter values?
Update*
I've attached the dataset properties window to elicit feedback on how to pass value to the RedShift query.
When connecting to a database like RedShift or MySQL using ODBC, you have to use a more generic syntax for parameters. Rather than using the # symbol and the parameter name, you just use a ? in the query. For example, one line would look like this:
EXECUTE prep_select_plan(?);
You can map which report parameters go to the query parameters in the parameters tab of the dataset properties. The parameters are simply mapped in the order they appear, you can't reference the same name in multiple places in the query. It's not as user friendly, but SSRS is primarily designed to work with SQL Server.
Related
I am creating an ETL in SSIS in which I which I want my data source to be a restricted query, like select * from table_name where id='Variable'. This variable is what I defined as User created variable.
I do not understand how I can have my source query interact with the SSIS scoped Variable.
The only present options are
Table
Table from variable
SQL Command
SQL command from a variable
What I want is to have a SQL statement having a variable as parameter
Simple. Choose SQL command as the Data Access Mode. Enter your query with a question mark as a parameter placeholder. Then click the Parameters button and map your variable to Parameter0 in the Set Query Parameters dialog:
More information is available on MSDN.
An inferior alternative to #Edmund's approach is to use an Expression on another Variable to build your string. Assuming you have #[User::FirstName] already defined, you would then create another variable, #[User::SourceQuery].
In the properties for this variable, set EvaluateAsExpression to True and then set an Expression like "SELECT FirstName, LastName, FROM Person.Person WHERE FirstName = '" + #[User::FirstName] +"'" The double quotes are required because we are building an SSIS String.
There are two big reasons this approach should not be implored.
Caching
This approach is going to bloat your plan cache in SQL Server with N copies of essentially the same query. The first time it runs and the value is "Edmund" SQL Server will create an execution plan and save it (because it can be expensive to build them). You then run the package and the value is "Bill". SQL Server checks to see if it has a plan for this. It doesn't, it only has one for Edmund and so it creates another copy of the plan, this time hard coded to Bill. Lather-rinse-repeat and watch your available memory dwindle until it unloads some plans.
By using the parameter approach, when the plan is submitted to SQL Server, it should be creating a parameterized version of the plan internally and assumes that all parameters supplied will result in equal costing executions. Generally speaking, this is the desired behaviour.
If your database is optimized for ad-hoc workload (it's a setting turned off by default), that should be mitigated as every plan is going to get parameterized.
SQL Injection
The other big nasty you will run into with building your own string is that you open yourself up to SQL Injection attacks or at the least, you can get runtime errors. It's as simple as having a value of "d'Artagnan." That single quote will cause your query to fail resulting in package failure. Changing the value to "';DROP TABLE Person.Person;--" will result in great pain.
You might think it's trivial to safe quote everything but the effort of implementing it consistently everywhere you query is beyond what your employer is paying you. All the more so since there is native functionality provided to do the same thing.
When using OLEDB Connection manager (with SQL Server Native Client 11.0 provider in my case) you can catch an error like this:
Parameters cannot be extracted from the SQL command. The provider
might not help to parse parameter information from the command. In
that case, use the "SQL command from variable" access mode, in which
the entire SQL command is stored in a variable.
So you need to explicitly specify database name in OLEDB Connection manager properties. Otherwise SQL Server Native Client can use different database name then you mean (e.g. master in MSSQL Server).
For some cases you can explicitly specify database name for each database object used in query, e.g.:
select Name
from MyDatabase.MySchema.MyTable
where id = ?
I have an input csv file with columns eid,ename,designation. Next i use Lookup transformation, inside look up am using query like
select * from employee where ename=?
i need to pass parameter ? from csv file. That is ename which is in csv file has to be passed into the query using Lookup transformation.
Inside Lookup i have changed mode to Partial cache, and inside Advanced tab, i selected Modify the SQL Statement and placed my query, and clicke on paramters tab. But i don't know like how to pass the parameter.
you cant add parameters to your lookup query. If by adding the parameters your goal is to reduce the amount of data read from the database, you don't have to worry, the "partial cache" will do that for you.
Partial cache means that the lookup query is not executed on the validation phase (like the full cache option) and that rows are being added to the cache as they are being queried from the database one by one. So, if you have one million rows on your lookup cache and your query only have reference to 10 of those rows, your lookup will do 10 selects to your database and end up with 10 rows only.
My table contains the following fields, Name,Age,Salary,Phone,DOB. Based on a settings table, I have to select only some fields. For example, I say in settings, only Name and Phone is required. How can I do it using stored procedure ?
EDIT :
Which one is good.
Select the required fields from the table.
Select all columns and in ASP.NET page, use .Visibility property to hide or show columns
SQL is a fixed column language: columns can not be added or removed "on the fly"
You would need to use dynamic SQL to build a SELECT statement, or use IF statements to execute different ones. However, you open up caching, security and injection issues.
Personally, I'd ignore columns in the client code and have a simple, single efficient SQL query. The contract or API between SQL Server and the client should be static and predictable. If the settings table is applied in SQL Server, your client doesn't know what columns to expect. If your client does know, then it can ignore them.
After your edit, option 2, kind of.
But the data should be removed before being rendered in the page.
Keep it simple: don't try to optimise anything yet
You would need to have multiple different selects - based on your settings table - in your stored proc to return the different sets of data.
CREATE PROCEDURE dbo.YourProcedure(...)
AS BEGIN
DECLARE #Setting INT -- ?? whatever it is
SELECT #Setting = Choice FROM dbo.YourSettingsTable WHERE ....... ???
IF #Setting = 1
SELECT Name, Phone
FROM dbo.YourDataTable
ELSE
SELECT Name, Age, DOB, Phone, Salary
FROM dbo.YourDataTable
END
Using this approach, however, has its dangers - since the stored proc might return one set of data or quite another, your SQL Server query optimizer might make a very good decision on how to access the data for one setting - but when your setting changes, that execution plan will be totally outdated, thus potentially leading to horrible performance......
On the other hand - it might be easier to determine that setting before calling your stored proc - and then just pass in that setting as a stored proc parameter.
Or even better yet: have separate stored procs for each "scenario" - and then from the caller, call the appropriate stored proc depending on the value of your setting....
Create the sql you want dynamically then execute it with exec.
declare #sql varchar(500)
set #sql = 'select 123'
exec (#sql)
The above code should help you understand what you need to know.
Have a stored procedure for each set of fields you want to select
Allow the list of field names to be passed in as a parameter
I have a reporting database and have to transfer data from that to another server where we run some other reports or functions on Data. What is the best way to transfer data periodically like months or by-weekly. I can use SSIS but is there anyway I can put some where clause on what rows should be extracted from the source database? like i only want to extract data for a current month. Please do let me know.
Thanks,
Vivek
For scheduling periodic extractions, I'd leave to that SQL Agent.
As for restricting the results by some condition, that's an easy thing. Instead of this (and you should always use SQL Command or SQL Command From Variable over Table Name/Table Name From Variable as they are faster)
Add a parameter. If you're use OLE DB connection manager, your indicator for a variable is ?. ADO.NET will be #parameterName
Now, wire the filter up by clicking the Parameters... button. With OLE DB, it's ordinal position starting at 0. If you wanted to use the same parameter twice, you will have to list it each time or use the ADO.NET connection manager.
The biggest question you will have to answer is how do I identify what row(s) need to go. Possibilities are endless: query into the target database and find most recent modified date for a table or highest key value. You could create a local table that tracks what's been sent and query that. You could perform an incremental load / ETL Instrumentation to identify new/updated/unchanged rows, etc.
I am currently trying to automate reports using SSRS in SQL Server 2008, requiring little to no user input at all.
I have the queries already to acquire the data, but they require datetime parameters which are retrieved from an invoices table in the database.
Is there any way to automate this, without requiring user input? I'd like to have these reports fire off every Monday morning without prompting. We also have an internal web site which is used for administrative work, written in ASP and C#, that I can use in conjunction in need be.
Regards and thanks
You should be able to do this. In each report you can set a query to retrieve parameter values and you can also specify a query for the default values.
So
Add a parameter and have that parameter be passed to your stored proc that gets your data
Set the available and default values for that parameter to a sql query that returns the value you need for the parameter.
You'll need to do this for each parameter. Then as long as all parameters in the report have default values the report will run without prompting for parameters.