I'm using a derived column to concatenate two WSTR columns, either of which could be NULL.
Alert (3000 width column) and customAlert (125 wide column)
The problem I'm having is that when I run this with REPLACENULL so that the concatenation happens properly if one of the columns is NULL, it's doubling the calculated destination column width for each column, which is pushing the derived column outside the limit for a WSTR column (so for the expression as written, it wants 6254 column width, which is not possible without moving to NTEXT, which causes its own problems when I have to convert it back.) The column size requirements don't change if I don't use REPLACENULL, but that comes with other problems.
REPLACENULL(Alert,"") + REPLACENULL(customAlert,"")
I could work around this using a C# script in the data flow to do the work, but I'd rather understand why this is happening.
Congratulations, it is not often that I get to learn something new about SSIS.
It sure feels like there's a bug in the REPLACENULL expression but I was also unaware of its existence
Set up
I defined a data flow with an OLE DB Source using the following query
SELECT
REPLICATE(N'A', 3000) AS Alert
, REPLICATE(N'C', 125) AS customAlert;
That results in pipeline metadata of
Alert - DT_WSTR 3000
customAlert - DT_WSTR 125
I can concur that as provided the expression fails out with an error about the invalid length of the resulting string. What's really weird to my brain is that the computed string length, is as you say, 6254. That's not a perfect doubling so ... I have no idea what is going on there.
I advocate for breaking things down in SSIS to smaller pieces as they make debugging easier.
I started by splitting your 3 operations into 2 separate ones with an eye towards the concatenation in a later step.
Right off the bat, this alone errors out.
REPLACENULL(Alert, "")
Error approximately
the length is not valid. Must be between 0 and 4000. Computed length 6002
6002? I really have no idea what's going on behind the scenes. We'll pin that and come back to it.
I added
REPLACENULL(customAlert, "")
and that worked by itself but once again the length is doubled + 2 for an expected width of 252.
So, we can't use replacenull with Alert and using it with customAlert doubles our space.
What else can we do?
Invent a time machine and cut your teeth on SSIS 2005. ;) A different way of solving the problem is to use the ternary operator test condition ? true clause : false clause
ISNULL(Alert) ? "" : Alert
and
ISNULL(customAlert) ? "" : customAlert
That results in a column of length 3000 and 125 respectively. Personally, I'd add a derived column after the one that makes these two new columns and then the concatenation because the next error you run into, is going to make the all-in-one expression more difficult to diagnose
(ISNULL(Alert) ? "" : Alert ) + (ISNULL(customAlert) ? "" : customAlert )
This has a correct length of 3125
Nowhere in the documentation for REPLACENULL does it identify this unexpected behaviour. Tested using SQL SERVER 2017
SQL Server Integration Services
Microsoft SQL Server Integration Services Designer
Version 14.0.3002.92
Related
I am using Below code in my derived column of SSIS to remove title in Name column such as Mr,Mrs,Ms and Dr.
Ex:-
Mr ABC
MS XYZX
Mrs qwrer
DR ADCS
SO I am removing the title of the name.
SUBSTRING( [Name] , 1, 3)=="Mr" && LEN( [Name] ) >2 ? RIGHT([Name],LEN([Name])-2)
But getting Error as incomplete token or invalid statement.
Please help.
any other suggestion to remove the prefixes are also welcome but need to Use transformation.
A different way to think about the problem is that you want to look at the first "word" in the column Name where "word" is the collection of characters from the start of the string to the first space.
String matching in SSIS is case-sensitive so you'll want to force the first word to lower/upper case - however your master list of titles is cased (and then ensure the title list is all lower/upper case).
I am an advocate of making life easier on yourself so I'll add a Derived Column, actually lots of derived columns, that will identify the position of the first space in Name and I'll call this FirstSpace
DER GetFirstSpace
Add a new column, called FirstSpace. The expression we want to use is FINDSTRING
FINDSTRING([Name], " ", 1)
That will return the position of the first instance of a space (or zero if no space was found).
DER GetFirstWord
Add another derived column after the DER GetGetFirstSpace. We need to do this so we can inspect the values we're passing to get the first word. Do it all in a single Derived column and when you get something wrong, you won't be able to debug it and the real cost of development is maintenance. New column, FirstWord will be type DT_WSTR 4000 because that's what happens when you use the string manipulation expressions. I am going to force this to upper case as I'll need it later on.
UPPER(SUBSTRING([Name], 1, [FirstSpace]))
TODO: Verify whether that will be "DR" or "DR " with trailing space as I'm currently coding this by memory.
TODO: What happens if FirstSpace is 0 - we might need to make use of ternary operator ?:
At this point in the data flow, we have a single word in a column named FirstWord what we need to do is compare that to our list of known titles and if it exists, strip it from the original. And that's an interesting problem.
DER GetIsTitleMatched
Add yet another Derived column, this time to solve whether we've matched our list of titles. This will be a boolean type column named IsTitleMatched
[FirstWord] == "DR" || [FirstWord] == "MRS" || [FirstWord] == "MR" || [FirstWord] == "MS"
Following that pattern "FirstWord is exactly equal literal text OR..." when this derived column evaluates, we'll know whether the first word is something to be removed (finally)
DER SetFinalName
Here we're going to add yet another column, NameFinal The magic of stripping out the bad word will be that we use the RIGHT expression starting at the position of that opening space and going to the end of the word. You might need to add a left TRIM in there based on whether the RIGHT operation is inclusive of the starting point or not. Again, free handing at the moment so good but no guarantee of perfection.
(IsTitleMatched) ? RIGHT([Name], [FirstSpace]) : [Name]
I do violate my own rule here as I have a quasi complex expression there in the positive case. Feel free to insert a derived column task that computes the stripped version of the name.
At this point, you've got 4 to 5 derived columns in your data flow but you can add a data viewer between each to ensure you're getting the expected result. You're welcome to solve this in your own manner but that's the easiest approach I can think of.
I created a MS Access database for tracking some items at work. Users enter data which is then aggregated in queries (using count or sum) to calculate the actual value for each area, joined using UNION, then compared to goals for that area. I attempted to enter an IIF statement to conditionally calculate the percentage of [Act]/[Goal], leaving it zero if [Act] is blank or 1 if [Act] is greater than [Goal] so there is nothing over 100%. The issue is that it works most of the time, but other times fails with no obvious logic error or reason why that I can figure out.
Sometimes it can't tell that [Act] > [Goal] even though looking at it, it's obvious. The numbers are all integers, nothing crazy or formatting differences. The formula in [Met] is what I hope to achieve. I added the [TEST] field to trace back where it might not be working, which shows Access just isn't always returning the correct answer to [Act] > [Goal].
My Query:
What comes out (just the broken part):
As you can see, it works correctly for most rows, but then thinks 149 is less than 52, and 128 is less than 3. Because of this, it generates [Met] values over 100%.
Has anyone had this happen before or have any suggestions? I have tried using refresh, clicking in the cell to hit enter, everything I can think of.
I think that although your columns are Ints, they are being converted to strings in the variables (or at least one of the variables) Goal and Met.
If you look at the data, you'll see that if you compare them the results of Test are correct for a string comparison.
E.g. "3" > "128" (Because the first character has a higher char value).
In your query, try surrounding the variables with Val() when you are comparing them, as follows:
IIf(IsNull([Act]),0,IIf(Val([Act])>Val([Goal]),1,Round([Act]/[Goal],2)))
I have two equations on the same dataset. One returns an error, the other does not. The equations are as follows:
1) =Sum(IIF(Fields!Service.Value="Dispatch Only", 0, Fields!Charge.Value))
2) =Sum(IIF(Fields!Service.Value="DispatchOnly", 0, Fields!Charge.Value))
Note that the only difference between the two is the fact that I took a space out of the field I'm looking for.
I want a total sum of the entire data set. But then I also need to know what that same sum is, without the "Dispatch Only" values. Obviously, equation #2 doesn't produce any useful information (it's just the sum again), but I added it just to prove I wasn't insane with some very basic typo, like a comma out of place.
What am I doing wrong with equation #1??
This commonly occurs when applying an aggregate on differing data types, which you're unlikely to get unless you have an IIf statement, like in your example.
You're probably getting a mismatch between Charge and 0 (i.e. between the underlying data type of Charge and the integer 0).
To get around this, make sure that 0 is the same data type as Charge, i.e. use something like CDec(0) or CDbl(0) as required.
One other option to consider is to use Nothing in place of 0 - by using the SSRS null value you avoid any data type clashes, but it does have the disadvantage of returning Nothing if there are no matching Dispatch Only rows.
SQL Server 2008 R2, using BIDS to design the report.
I have a table and I am trying to only show a certain row. Maybe there are better ways to do this, but I am coming across an error with the filter expression and regardless of how I achieve my initial task, I'd like to understand the filtering.
I started with the filter expression (set to type "Integer"):
RowNumber(Nothing) = 1
This gave the error:
Cannot compare data of types System.String and System.Int32.
I found the solution to this is to change the 1 to "=1" as 1 is evaluated as a string.
So I then had:
RowNumber(Nothing) = =1
That changed nothing, I got the same error.
Then I tried to do that to the first part of the expression:
=RowNumber(Nothing) = =1
This changed the error to a deployment problem (still builds, which is frustrating):
Error pvInvalidDefinition : The definition of the report '/ReportName' is invalid.
I then tried using CInt on RowNumber:
CInt(RowNumber(Nothing) = =1
Then I can deploy it, but the error just changes back to the first one:
Cannot compare data of types System.String and System.Int32.
It seems no matter what I try here I either can't deploy the report or I get an error that I'm comparing a string to an int.
RowNumber returns an integer, so it seems like this should work. I've tried using the name of the dataset in place of "Nothing" but that doesn't change what I'm seeing.
I realize there are many ways to solve my initial problem, but I am curious as to why the filter expression is invalid.
Its better to hide a row with visibilty property. Just click on any text box and go to visibily tab . You can now click on show or hode and go to expression.
That default to Hide . So write an expression there to hide the row.
=IIf(NOT(RowNumber = 1),TRUE,FALSE)
Let me know if you get any error
RowNumber is not available to use in a Tablix Filter.
Using RowNumber(Nothing) <> 1 as a Row visibility property fixed the issue.
Using BIDS you are not given any error that indicates what the problem is, but importing the report to Report Builder and deploying it from there will give a more descriptive error that, in the end, helped me to solve my problem.
In a report, I've a dataset with a filter(based on a MultiValue parameter).
This dataset contains two field: Id and Name.
I need to display somewhere the concatenation of all names:
Name1 / Name2 / Name3
The problem is that the join method works only on array, and then I cannot specify a dataset as value.
I looked in custom code too, but I didn't found anything working.
How should I do this ?
I may be a bit late for this but for anyone that's interested in this, there is a rather easy way of doing this in SSRS:
=Join(LookupSet(1,1,Fields!Name.Value, "DatasetName")," / ")
SSRS-2008 R2 and higher...
1. Using LookupSet
If you're beyond the 2008 version OP has, there exists a good solution:
=Join(LookupSet(1, 1, Fields!Name.Value, "DatasetName"), " / ")
Credit for this answer using the LookupSet solution goes entirely to #urbanhusky's answer.
SSRS-2008 and lower...
I'm keeping this answer though because it aggregates #urbanhusky's solution with the solutions available to poor souls stuck with OP's version of SSRS and below.
In SSRS 2008 there's only three "options" as far as I can see, each with its own downside. The first one's probably the least hackish.
2. Extra parameter
Create an internal parameter (e.g. "NameParameter", see this SO answer or MSDN) with Allow Multiple Values. Set the default value of the parameter to the Name field from your dataset. Then use the function =Join(Parameters!NameParameter.Value, " / ") to show the joined names in a textbox.
This may be your best bet, but if there are a lot of values the parameter may not work very well.
3. Use a List
Create a List and drag/drop the Name field to it. If necessary, group on the Name as well.
The disadvantage here is that (AFAIK) the list can't be made to show horizontally.
4. Use a Matrix
Oh boy, this one's real ugly. Nonetheless, here goes: create a matrix, drag the Name field to the column header, and hide the first column as well as the second row (for displaying the data).
The main disadvantage is that it's a hack (and quite some overkill), plus you'll have to trim the last seperator character manually with an expression.