I have a situation where I am trying to lookup a value in one table based on values in another table, using a BETWEEN operator and not an = operator.
In one table, I have a value "EffectiveDate". I want to get a Weight number from another table, but the other table has two fields: "Inception" and "Termination". What I want to do is extract the Weight from that table for use where the EffectiveDate is between Inception and Termination.
SSIS doesn't seem to provide a way to do this. It's good at matching one column to another column, but doesn't seem to allow one to many-column comparison/operations.
Am I missing anything? Is this possible to do somehow?
Have a look on this URL as it could be a performance killer
http://blogs.msdn.com/mattm/archive/2008/11/25/lookup-pattern-range-lookups.aspx
A script component could be the direction forward
JUst put two conditions on it mydatefield >= some date and mydatefield <= some date
Take a look at:
http://www.julian-kuiters.id.au/article.php/ssis-lookup-with-range
http://sqlblog.com/blogs/jamie_thomson/archive/2009/11/28/debunking-kimball-effective-dates.aspx
(The second link is a controversial argument for how [not] to respresent type 2 SCDs).
If you have the time and patience, writting a script component is by far the way to go from a performance perspective.
Related
I have a two table scenario with a typical parent / child relational setup:
tblGroup - idGroup (autonumber, PK); GroupName (Short Text), Rank (Number)
tblRange - idRange (autonumber, PK), idGroup (FK -> tblGroup.idGroup), RangeName (Short Text), Rank (Number)
What I am doing on the parent (tblGroup) table is using a data macro to add the rank using the BeforeChange event:
IF
IsInsert
SetField - Rank
- DMAX(Rank, [tblGroup])+1
This works nicely and I can happily use a parametized INSERT query to add rows to the table and not have to worry about duplicate ranks and so forth.
What I would like to be able to do but cannot figure out how, is to have a data macro do the same thing for the child (tblRange) table, with the rank being set to the new highest for the parent group the child record belongs to.
If I use the same DMAX approach as I have above I am supposed to be able to set a criteria as a third option, acting like a where clause, to limit the lookup / calculation. How can I refer to the specific idGroup I am working with in tblRange in the macro? I cannot seem to figure out how to reference the new records value for this in the macro.
Something like DMAX(Rank, [tblRange], ???How_to_refer_to_idGroup_Properly???)+1
Any help greatly appreciated
Cheers
The Frog
I figured out a way to do this. Thankyou caffeinated beverages!
The reason for the strange error messages is due to limitations in the Data Macro processing, specifically in the BeforeChange event. The solution is as follows:
Create a query that selects MAX rank (MaxRank) and GROUP BY for the idGroup (ParentID)
The resultant query produces two columns of data: [MaxRank] and [ParentID]
There will be a row for every idGroup with the maximum Rank for each
Create a BeforeChange data macro
Set the following:
IF IsInsert
LookupRecord
Lookup Record In - qryGetMaxRank (or whatever you called your query)
WHERE - [qryGetMaxRank].[ParentID] = [tblRange].[idGroup]
Set Field
Name - [tblRange].[Rank]
Value - [MaxRank] + 1
The BeforeChange event cannot handle parameters for a query, and I am guessing that this applies in some form the to DMAX function here too. The use of a query that does not use any parameters, and then using the LookupRecord WHERE clause to do the filtering provided the single row result needed. The [MaxRank] value from the returned result is then able to be used to set a new value for the field.
Bit of a workaround but it does allow someone to work with the data either through a form or through the datasheet view and not create a problem.
**In answer to if this is a multi-user DB - it is not. It is just me working with it. If / when the solution is scaled up to something requiring multi-user I will likely recreate the BE in SQL Server or MySQL and use stored procedures for all data I/O. Happy to keep Access as the FE and compile into an application (using the runtime for clients), but I am a fair way off from having to do that yet. Very early stages of development at this time.
Cheers to everyone for the pointers. They helped me figure this out. Hopefully this will be of use to someone else in the future.
PS: If you need to use a parametrized query in a data macro it looks like the best bet is with the AfterInsert event or AfterUpdate event as they can support parameters.
I have job in Talend that is designed to bring together some data from different databases: one is a MySQL database and the other a MSSQL database.
What I want to do is match a selection of loan numbers from the MySQL database (about 82,000 loan numbers) to the corresponding information we have housed in the MSSQL database.
However, the tables in MSSQL to which I am joining the data from MySQL are much larger (~ 2 million rows), are quite wide, and thus cost much more time to query. Ideally I could perform an inner join between the two tables based on the loan number, but since they are in different databases this is not possible. The inner join that is performed inside a tMap occurs after the Lookup input has already returned its data set, which is quite large (especially since this particular MSSQL query will execute a user-defined function for each loan number).
Is there any way to create a global variable out of the output from the MySQL query (namely, the loan numbers selected by the MySQL query) and use that global variable as an IN clause in the MSSQL query?
This should be possible. I'm not working in MySQL but I have something roughly equivalent here that I think you should be able to adapt to your needs.
I've never actually answered a Stackoverflow question and while I was typing this the page started telling me I need at least 10 reputation to post more than 2 pictures/links here and I think I need 4 pics, so I'm just going to write it out in words here and post the whole thing complete with illustrations on my blog in case you need more info (quite likely, I should think!)
As you can see, I've got some data coming out of the table and getting filtered by tFilterRow_1 to only show the rows I'm interested in.
The next step is to limit it to just the field I want to use in the variable. I've used tMap_3 rather than a tFilterColumns because the field I'm using is a string and I wanted to be able to concatenate single quotes around it but if you're using an integer you might not need to do that. And of course if you have a lot of repetition you might also want to get a tUniqueRows in there as well to save a lot of unnecessary repetition
The next step is the one that does the magic. I've got a list like this:
'A1'
'A2'
'B1'
'B2'
etc, and I want to turn it into 'A1','A2','B1','B2' so I can slot it into my where clause. For this, I've used tAggregateRow_1, selecting "list" as the aggregate function to use.
Next up, we want to take this list and put it into a context variable (I've already created the context variable in the metadata - you know how to do that, right?). Use another tMap component, feeding into a tContextLoad widget. tContextLoad always has two columns in its schema, so map the output of the tAggregateRows to the "value" column and enter the name of the variable in the "key". In this example, my context variable is called MyList
Now your list is loaded as a text string and stored in the context variable ready for retrieval. So open up a new input and embed the variable in the sql code like this
"SELECT distinct MY_COLUMN
from MY_SECOND_TABLE where the_selected_row in ("+
context.MyList+")"
It should be as easy as that, and when I whipped it up it worked first time, but let me know if you have any trouble and I'll see what I can do.
I am trying to find a way to sort the results in a view. I know the Order by command does not work. The reason for this is that we are using a BI tool 'Board' to drill through to the raw data. Unfortunately there is no way in the program to sort a drill through as I would a regular query since the Where clause is dynamic and the program always puts it at the end of the query.
Hence an Order by would appear prior to a Where which results in an error.
Any suggestions would be greatly appreciated.
Thanks,
Scott
So there's a couple of ways to achieve this, one is to make sure that the data the view is reading is already sorted, to do this, you make sure that the indexes on the table/s that the view is reading from are sorted the way you want, if this isn't possible, then you could also create a clustered index on the view itself and specify the sort order there.
If creating indexes is not possible, there is another way that I have found to work, but is not always ideal as it creates an extra column in your result set that doesn't mean anything, you can do this:
SELECT ROW_NUMBER() OVER ( ORDER BY [col1] ASC ) AS [sort_col] ,
[col1] ,
[col2] ,
[col3]
FROM [dbo].[table];
If you do this, you will end up with the extra column giving the row number, but the data will be sorted by [col1]. One thing to mention with this is that if you do put this into the view definition, you must still select that column in your query for the sorting to take place.
I have two tables and from that I am generating a query. I have columns in my query with field type yes/no. These are basically columns to determine the race of a person. The user enters information in yes/no for various races. I want another calculated column in the query which checks for all other race columns and calculates values in it.
I have to check for a few conditions in for the values in columns
For example:
1) If Hispanic is chosen, the new column should say hispanic(no matter what other options are selected. This is like a trump card)
2) If more than one is selected, then new column entry should say "multi"
3) If none of the options are selected, it should say "unknown"
4) If exactly one of them is selected, then that race should be displayed
Can anyone help me with this? I am new to Access
I can't code it for you but I can point you in the right direction. What you want to do is take all the tests you explained above and put them in a coded format :
iif ( condition, value_if_true, value_if_false )
Since you have a lot of possible outputs i'd use something like a Case Statement where you can test for all the possibilities.
Follow this link if you need any info on how to code both type of statements (iif and case).
Once you have tried something like this, you can comeback with a specific question if you encountered a problem in the process.
Good luck with your database.
I have an rdlc report file, and I am trying to make a sum which can only include the last item in each group. I have a table kind of like this:
Place = ? (Group header 1)
User = ? (Group header 2)
Date =Last(Fields!Number.Value) (Group header 3)
Number =Fields!Number.Value (Detail row)
So, in other words, in User there, I want a sum of Date... if that made sense...
The Numberrows contain many numbers per Date. But Date shows only the last number for that day, because the rest doesn't count (but must be displayed) In User I want to sum up those last numbers for all the dates for that user. And same with Place (which would be the sum of every last number for every day for every user).
Could anyone help me with this? I tried the obvious (to me at least) =Sum(Last(Fields!Number.Value)), but (also tried to specify the group in those functions, but didn't make a difference because) I get an error when I try to compile which says:
The Value expression for the textbox 'numberTextbox' contains an aggregate function (or RunningValue or RowNumber functions) in the argument to another aggregate function (or RunningValue). Aggregate functions cannot be nested inside other aggregate functions.
Which I guess kind of makes sense... but how do I do this then?
Update: I have solved the issue by adding another column, and copying those last numbers into that column. This way I can display all the numbers, and do the summing on the column that only contains the ones that is going to be in the sum. I am still very curious to if anyone have a solution to my original problem though... so please post an answer if you do!
Not sure I understand exactly what you're trying to do. Maybe something like =Last(Fields!Number.Value,"Group 1") + Last(Fields!Number.Value,"Group 2") + Last(Fields!Number.Value,"Group 3"), instead using a sum function?
the easiest way to do this would be to modify your dataset to only include the records your are displaying in the date field, that way you could just use a simple sum() instead of trying to do something weird and screwy and might not work.
Modifying the data set may really be the simplest solution, but if you really wanted to do this without complicating the query you could try "custom aggregation".
The exact techniques for this depend on the version of SSRS, and my understanding is it didn't really work that well before SSRS 2008.
The idea is, you write some custom code to maintain an array containing the last value for each date. One function updates the "last value" for a date, and another sums the values in the array. Your header calls the latter function; you conspire to cause the former function to be called once for each detail row before the header is processed.
Here's a blog with a write-up that explains the technique in SSRS 2008.
It also gives some insight into how you can try to make this work in SRSS 2005, but again apparently that's not as reliable.