I have a variable with a value of '1617'. It is a DT_WSTR datatype currently. Sometimes I need a string, others an integer.
I am using a derived column to replace the ' values so that I can cast this value as an integer.
My replace function is not working.
REPLACE([User::schoolYear],"'","")
What am I doing wrong?
The problem with your supplied expression, is that you are not referencing the variable schoolYear. Sometimes, you can address a variable as #schoolYear but the consistent, explicit syntax I would encourage is #[User::schoolYear] That way, you can identify the namespace in case someone like me has used a custom namespace.
Your Derived Column expression then becomes
REPLACE(#[User::schoolYear],"'","")
You are having space before and after the single quote, which is causing the replace to fail.
Modify the expression as given below. I have tested it. It is working fine.
REPLACE([User::schoolYear],"'","")
Related
I have created an SSIS package where two columns of type varchar(1) have to be mapped to columns of Integer. I have this working using a Derived Column and giving both fields a type cast of (DT_I4). However, I discovered in the complete data set there are records with no value in these two fields and so I have to Type Cast AND add a condition in expression to default to "0" if null.
So far I have tried the following but are not valid
(IsNull[Notes Taken])?(DT_I4)"0":[Notes Taken]
(DT_I4)(IsNull[Notes Taken])?"0":[Notes Taken]
How do I create this expression properly
The most simple solution is to use REPLACENULL function like:
REPLACENULL([Notes Taken], "0")
And then - cast it to DT_I4. This function replaces the logic you are devising with conditional operator.
Your both formulas have errors. The most prominent - ISNULL is a function and needs parenthesis around its arguments, ISNULL([Notes Taken]), brackets only define a dataflow column. See MS Docs.
Then, your first expression
(IsNull[Notes Taken])?(DT_I4)"0":[Notes Taken]
Possibly the field [Notes Taken] is not matching data type of the DT_I4 which is the datatype of the first argument of ? : operator.
Your second expression
(DT_I4)(IsNull[Notes Taken])?"0":[Notes Taken]
Applies the data cast to the logical function ISNULL, not to the complete expression. You should put the parenthesis around the complete ? : operator like:
(DT_I4)(IsNull([Notes Taken])?"0":[Notes Taken])
I'm using SSIS to separate good data from unusable date. In order to do that I used derived columns, script task and conditional split where I assigned certain conditions. One of the conditions I need to apply is that none of the numbers in one column cannot be negative. I'm guessing that the best way to solve this would be using conditional split, but I cannot get it to work. I'm new to SSIS, so any help would be appreciated.
You'd have an Expression like
[MyCaseSensitiveColumnName] < 0
and then name the output path something like BadData_NegativeValue
From the comments
that is what I did before, but I'm getting an error saying that The data types "DT_WSTR" and "DT_I4" are incompatible for binary operator ">"
That error message indicates that you are attempting to compare a unicode string (DT_WSTR) and an integer (DT_I4) and that the expression language does not allow it.
To resolve this type incompatibility, you would need to first convert the value of MyCaseSensitiveColumnName from DT_WSTR to an integer.
I'd likely add a Derived Column Component to my data flow and create a new column called MyCaseSensitiveColumnNameAsInteger with an expression like
(DT_I4) [MyCaseSensitiveColumnName]
Now, that may be perilous depending on the quality of your source data. I don't know why you are pulling numeric data in as a string. If there could be non whole numbers in the data set, then we will need to check before making the cast. If there are NULLs in that dataset, those too may cause issues.
That would result in our conditional split check becoming
[MyCaseSensitiveColumnNameAsInteger] < 0
This code gives an error. I have looked similar type questions and couldn't find the answer.
sum(COALESCE(((rpt.report_target_data::json->>'itemQuantity')::int)::int),0) as itemQuantity,
report_target_data is a json object and 'itemQuantity' is an element of that json. Sometimes that field contains an empty value. So when I try to get the sum it gives an error because postgres cannot get the sum if a column had a empty value. What is the wrong with the above code. It there a way to walk around that matter? Is there a way to calculate sum even if some rows contain empty values?
Here is the error of the above code ->
No function matches the given name and argument types. You might need to add explicit type casts.
In my case, it was not a COALESCE problem but I ended up in this question.
I noticed that my column values were characters (the varchar type) so what I did is:
select sum(cast(num_suf as int)) as total from results;
Just in case someone lands in this question again :)
what is equivalent's of R ('NA'|'NULL'|'None') value in RAPIDMINER? Or what is some other value for denoting that this value should not be taken into consideration while making calculations, or that it is missing?
Missing values show up as '?' in the data view of example sets and they are counted in the meta-data view. Imported data can obviously have them and you can generate your own using the Declare Missing Values operator or within the Generate Attributes operator using a code fragment containing an invalid mathematical operation like 0/0. You can test for the presence of a missing value by using the missing() function within the Generate Attributes operator.
It is probabbly the question mark: ?
So I am currently working on a migration from an old Advantage database server to SQL 2005 using SSIS 2008. One of the columns in the old Advantage database is a MEMO type. By default this translates to a DT_TEXT column. Well in the new database I do not need this large of field, but can limit it to something such as VARCHAR(50). I successfully set up a derived column transformation to convert this with the following expression:
(DT_STR,50,1252)[ColumnName]
Now I want to go a step further and replace all NULL values with an empty string. This would seem easy enough using an ISNULL([ColumnName])?"":(DT_STR,50,1252)[ColumnName] expression, but the problem is that the OLE DB Destination contains the following error
Cannot convert between unicode and non-unicode strings...
So apparently the whole ISNULL expression converts the data type to Unicode string [DT-WSTR]. I have tried a variety of casts upon the whole expression or different parts, but I cannot get the data type to match what I need it.
First, is it possible to convert the DT_TEXT type directly to unicode? From what I can tell, the casts don't work that way. If not, is there a way to get an expression to work so that NULL values get converted to empty strings?
Thank you for all your help!
Give this a try in your derived column.
(DT_STR,50,1252) (ISNULL(ColumnName) ? "" : (DT_STR,50,1252) ColumnName)
It includes an additional type cast with the Conditional (?:) in parentheses to ensure the desired processing sequence. I think your original expression was implicitly casting to DT_WSTR because the "" defaults to DT_WSTR. With this new version, you force the cast to DT_STR after the expression is evaluated.
I figured something out that works. It may not be the best solution, but it will work for my situation.
From my OLE DB source I first did a Derived Column. This I used the ISNULL which ended up converting it to a DT_WSTR unicode type. although I could not get any casts to get it back to the type required, I then added a Data Conversion transformation in-between the Derived Column and the OLE DB Destination. This would take the input string and convert it back to a DT_STR. This all feels a little annoying converting so many times, but the column does not contain any funky information that I should have to worry about, so I suppose it will work.
Thanks for all those who pondered the solution, and if you find some awesome way to tackle it, I would be more than interested.