SSIS Equality Operator not working as expected - mysql

I have an SSIS package that is hooked into a MySQL database that has a tinyint(1) field which is set to the value 4 and I'm checking to see if that value is 4 in the database because it should not change based on the incoming flat file if it is already set to 4, but it doesn't seem to be working.
It should not be updating the status because the field is set to 4 but I guess I just don't understand why the expression isn't working. I have tried all the different int types for SSIS and it doesn't matter because it always sets the status to 1. Based on a quick google search a tinyint should be an unsigned 1 byte int which is why I am casting it to DT_UI1
Incoming values from flat file the first being the product and the last being the status
"039414","*****","*****","*****","*****","*****","*****","*****"," "
Existing value from the database the first being the product and the second being the status
039414 4
I handle the status by the following derived column expression, basically if there is no value it should be 1 otherwise it should be 4
LEN(TRIM(Status)) > 0 ? (DT_UI1)4 : (DT_UI1)1
Here is the derived column expression that I'm using to set the product_status (products_status being the actual field from the database and prod_status the status from the incoming flat file):
((DT_UI1)products_status == (DT_UI1)4) || ISNULL(prod_status) ? (DT_UI1)4 : (DT_UI1)prod_status
What I'm expecting to happen is a few products should keep their status of 4 but are being updated to 1, does anyone have an idea why?
Edit:
I tried changing the expression to the following with no success.
(DT_UI1)products_status == (DT_UI1)4 || ISNULL(prod_status) ? (DT_UI1)4 : (DT_UI1)prod_status
and
((DT_UI1)products_status == (DT_UI1)4 || ISNULL(prod_status)) ? (DT_UI1)4 : (DT_UI1)prod_status
and just the base test
(DT_UI1)products_status == (DT_UI1)4 ? (DT_UI1)4 : (DT_UI1)prod_status
Solution:
I ended up casting the field in the query to an integer because it was being pulled in as a boolean instead of an int
CAST(products_status AS SIGNED)
Then changed my expression to the following and it worked as expected
(products_status == 4 || ISNULL(prod_status)) ? (DT_I4)4 : prod_status

Not sure about the precedence here, but don't you think you want extra parentheses around the condition in the immediate if:
( ((DT_UI1)products_status == (DT_UI1)4) || ISNULL(prod_status) ) ? (DT_UI1)4 : (DT_UI1)prod_status
Looks like it might otherwise return "true", which is sometimes rendered as 1.
UPDATE
Here is a link to Data Viewer which lets you see what values SSIS is assigning to the columns and helps you analyze situations like this.

Your original expression
((DT_UI1)products_status == (DT_UI1)4) || ISNULL(prod_status) ? (DT_UI1)4 : (DT_UI1)prod_status
returns a boolean. Let's assume products_status is 4 and prod_status is also set to 4.
((DT_UI1)products_status == (DT_UI1)4)
Evaluates as TRUE.
ISNULL(prod_status) ? (DT_UI1)4 : (DT_UI1)prod_status
Evaluates as 4.
Your original expression now evaluates as
TRUE || 4
Which evaluates to
TRUE
Which when you try to stick it in an integer column like products_status becomes 1.
Your first revision makes no substantive change to the evaluation chain and also returns TRUE (i.e. 1).
Your second revision is the correct formulation of the ternary operator, as is the final base test. However, it appears that ISNULL(prod_status) always returns FALSE in your case since you are explicitly setting it to 4 or 1 in your other derived column, so let's ignore your second revision and just go to your final base test.
With regards to your final base test, I would definitely:
Attach a Data Viewer before your Derived Column component to ensure that the products_status column being passed in is indeed set to 4
Set a Data Viewer after your Derived Column component to see that products_status has been set (or not set) properly;
Make sure that you properly mapped the products_status field to your database destination and not the prod_status field by mistake.

Related

SSIS Conditional split handling nulls and other values in column

I have an SSIS package in 2010
I want to return any nulls in a column or a value in the same column
This using the conditional split
I have the first part
ISNULL([Data Conversion].ID)== TRUE
this will bring me all the null ID rows
but I want to add a OR part
I am looking for something like :
ISNULL([Data Conversion].ID)== TRUE && [Data Conversion].ID =="ABC"
what replaces the && to achieve the OR condition please - or how to amend the code to achieve the OR condition
|| (Logical OR) (SSIS Expression) : Performs a logical OR operation. The expression evaluates to TRUE if one or both conditions are TRUE.
(ISNULL([Data Conversion].ID) == TRUE || [Data Conversion].ID =="ABC" ) && [Data Conversion].Sex =="Male"

SSIS Derived Column Expression - removing leading zeros

I have a package that is failing due to a negative sign with a numerical string such as:
000000000-25.00
The Derived column expression is:
ISNULL(wstr_Payment_Amount) || TRIM(wstr_Payment_Amount) == "" ? (DT_CY)0 : (DT_CY)wstr_Payment_Amount
The data type to column is money. I did a redirect row on the derived column to confirm it was the negative sign that was causing the failure.
Here is what I have..
FINDSTRING(wstr_Payment_Amount, "-", 1) >0 ? TRIM(SUBSTRING(wstr_Payment_Amount, FINDSTRING(wstr_Payment_Amount, "0-", 1), 8)) : wstr_Payment_Amount
I am using SSIS 2008.
If you are using two derived columns then leave it as a string in the first one...
ISNULL(wstr_Payment_Amount) || TRIM(wstr_Payment_Amount) == "" ? (DT_WSTR,100)"0" : (DT_WSTR,100)wstr_Payment_Amount
I think you were really close but consider the following once you identify the negative sign...
FINDSTRING(wstr_Payment_Amount, "-", 1) >0 ? (DT_CY) ("-" + replace(wstr_Payment_Amount,"-","")) : (DT_CY)wstr_Payment_Amount
This will just move the "-" to the front and all the zeroes will just go away.

SSIS expression fails to validate

I have set a following expression in SSIS derived column component:
TRIM(xCOL) == "" ? (DT_STR,7,1252)xCOL : NULL(DT_STR,7,1252)
However, this fails to validate - I get the following error:
Error at PKG: For operands of the conditional operator, the data type
DT_STR is supported only for input columns and cast operations. The
expression "TRIM(xCOL) == "" ? (DT_STR,7,1252)xCOL :
NULL(DT_STR,7,1252)" has a DT_STR operand that is not an input column
or the result of a cast, and cannot be used with the conditional
operation. To perform this operation, the operand needs to be
explicitly cast with a cast operator.
What is wrong here?
Unicode - the bane of all SSIS things that you would think just work.
TRIM(xCOL) == "" ? (DT_STR,7,1252)xCOL : NULL(DT_WSTR, 7)
By default, expressions on strings have a return type of Unicode (DT_WSTR). You can prove this yourself by replacing your last expression with an empty string "". Look at at the type that the derived column is assigned as - it's a DT_WSTR
TRIM(xCOL) == "" ? (DT_STR,7,1252)xCOL : ""
The resolution then is to cast the result of the entire expression to your desired type. (I also think you have a logic issue since I'm assuming you are inspecting the trimmed string to determine if it's empty/zero length and then casting it to NULL). Also note, that if you ever do get a NULL in for your xCOL field, this will blow up.
(DT_STR, 7, 1252)((TRIM(xCOL) == "") ? NULL(DT_WSTR, 7) : TRIM(xCOL))
Source data
SELECT 'Hello' AS xCOL
UNION ALL SELECT ''
Results
Like the error says, you must explicitly cast with a cast operator.
TRIM(xCOL) == "" ? (DT_STR,7,1252)xCOL : (DT_STR,7,1252)NULL(DT_STR,7,1252)

How to avoid inserting a field that is not null from csv to mysql using Perl?

I am inserting a csv file into MySQL using Perl.
I have successfully inserted some data into MySQL.
However, some fields in the csv file are null and the code cannot continue to run.
What is the correct way to express if statement to check if $rttjitter is not null, else print that row number, then execute?
i.e. 2 criteria: $rtt is not equal to 0 and $rttjitter is not null.
my $empty = q{};
my #nulls = ();
if ($rtt !=0 && $rttjitter ne undef)
^
These do not work quite well.
You can use the built-in defined function to explicitly check if a variable is defined, so e.g.
if( defined( $rttjitter ) and $rttjitter ) { ...
Will check that it's defined and has a value that is not 0 or an empty string.
It seems likely though you should take another look at how you're reading in your CSV (you are using a module like Text::CSV, right?), and/or how you're doing the DB insertion, as generally with this sort of thing the undef value will cleanly insert as a NULL into the DB and should not cause any problems with your code, unless you're doing something strange.
if (defined($rttjitter) && defined($rtt) && $rtt != 0)
{
... # $rttjitter is not null (undef) and $rtt is not null (undef) and nonzero
}
else
{
... # at least one column fails the test
}

Is there a nesting levels limit with expressions used in SSIS packages?

Working in SQL Server 2008. My first stab at an SSIS script and I need to emulate some if/then conditional logic written in VB.net. I couldn't find any previous questions dealing with nested conditions in expressions and believe I'm following what I've been able to uncover via google on nested conditions in a derived column.
I'm receiving an error while attempting to use nested conditions in the derived column transformation editor. The error I'm receiving indicates that SSIS could not parse my expression. The actual exception: "Exception from HRESULT: 0xC0204006 (Microsoft.SqlServer.DTSPipelineWrap)"
Questions for which the answers might immediately answer my question (and create a new problem):
is there a nesting levels limit?
can nesting be performed in the condition1 portion of [expression] ? [condition1] : [condition2]
I'll give two snippets, the first is what I'm actually inserting, the second is a more reader-friendly version. Hopefully someone can point out my error.
Not sure that it has bearing, but please note that [BusArea] is a column derived in a previous step.
actual expression:
[BusArea] == "CCC" || [BusArea] == "NBU" || [BusArea] == "CA" ? (ISNULL([CASE_MORG]) or TRIM([CASE_MORG]) == "" ? ( ISNULL([TRX_MORG]) or TRIM([TRX_MORG]) == "" ? NULL(DT_WSTR,50) : [TRX_MORG]) : [CASE_MORG]) : (ISNULL([CASE_AGT]) or TRIM([CASE_AGT]) == "" ? ( ISNULL([TRX_AGT]) or TRIM([TRX_AGT]) == "" ? NULL(DT_WSTR,50) : [TRX_AGT]) : [CASE_AGT])
formatted for easier reading:
[BusArea] == "CCC" || [BusArea] == "NBU" || [BusArea] == "CA" ?
(ISNULL([CASE_MORG]) or TRIM([CASE_MORG]) == "" ?
( ISNULL([TRX_MORG]) or TRIM([TRX_MORG]) == "" ?
NULL(DT_WSTR,50)
: [TRX_MORG]
)
: [CASE_MORG]
)
: (ISNULL([CASE_AGT]) or TRIM([CASE_AGT]) == "" ?
( ISNULL([TRX_AGT]) or TRIM([TRX_AGT]) == "" ?
NULL(DT_WSTR,50)
: [TRX_AGT]
)
: [CASE_AGT]
)
I don't think there is any limit with nesting conditions. Even if there is one, I don't think we will reach the limit in the packages that we create handle our business processes.
You almost got everything correct. The issue with your conditional statement is that you have used or instead of ||
I copied your exact statement and pasted in Derived Transformation within a Data Flow task and got an error because the package couldn't parse the expression. I replaced all the or's with correct Logical OR (||) operator and the expression evaluated correctly.