In SSIS, how would you parse the values between the " " in this example:
KeyFieldForRecord:"ORD101300"}
So that the derived column value only returns: ORD101300 ?
I think the fact that the source wraps what I need in "" will make it challenging to use any of the SSIS string functions
You can simply use REPLACE in your derived column:
REPLACE(KeyFieldForRecord, "\"", "")
Related
I am trying to remove [" from beginning of the string and "] end of the string by using REPLACE function in derived column. But it is giving an error.
I have used the below formula
REPLACE(columnanme,"["","")
is used in the to remove [" in the beginning of the string. But not working.
Can someone help me on this.
Note: Data is in table and datatype is NTEXT
Regards,
Khatija
I believe you just need to escape the " value
so
\”
REPLACE(columnanme,"[\"","")
otherwise it sees the " in the middle as the closing quote and you have an invalid statement.
I am trying to remove [" from beginning of the string and "] end of the string
Supposing that we reliably have brackets and quotes wrapping the data, the simplest approach would be to use substring. This would be easier to do in SQL:
UPDATE myTable SET columnname = SUBSTRING(columnname, 3, LEN(columnname) -4)
WHERE columnname LIKE '["%"]'
If you want to do this in SSIS, you'll need to use a script component transformation to avoid data loss when converting the value to a string. Select the column you want to work with and set the usage type to ReadWrite:
In the script, I have added a method GetNewString, which converts the blob to a string and strips the unwanted characters. You can also use Replace or Regex.Replace if that makes more sense.
In the Input0_ProcessInputRow method, we convert the columns data, reset the blob and then add the new value:
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
var input = GetNewString(Row.columname);
Row.columname.ResetBlobData();
Row.columname.AddBlobData(System.Text.Encoding.Unicode.GetBytes(input));
}
public string GetNewString(Microsoft.SqlServer.Dts.Pipeline.BlobColumn blobColumn)
{
if (blobColumn.IsNull)
return string.Empty;
var blobData = blobColumn.GetBlobData(0, (int)blobColumn.Length);
var stringData = System.Text.Encoding.Unicode.GetString(blobData);
stringData = stringData.Substring(2, stringData.Length - 4);
return stringData;
}
I'm working on a SSIS package where I have a text file with 5 columns. I need to check if all the rows for 5th column are NULL values.
If all the rows in 5th column are NULL then all the data should go for invalid file table.
If any row in 5th column have non NULL value then all the data should go to valid table.
You will need to read the entire file before being able to make the decision of where to write it to, so introduce a third table where you can stage the data first
Next part would be to build the logic that checks the staging table for all NULLS. Below query would return 0 if all was NULL and more than 0 if any record had a value
SELECT COUNT(*) FROM dbo.StagingTable ST WHERE ST.Column5 IS NOT NULL
Once you feed the answer into a variable you can use precedence constraints to fire the dataflow copy [staging to active] if the result was more than 0 or [staging to faulty] if the result was 0
personally if i had to perform this task I would use a script task to do it all:
Load into a data table
Use linq to check column to determine destination .Where(x => x[4]!=null).Count()
Load to destination via bulk Copy
You can check if the file is empty with C# using an OleDbDataAdapter and search the file, then determine where to load the file using SSIS Precedence Constraints. This example uses a CSV file without column names. If the columns do have names add the replacement code noted in the comments below. You will also need the using statements listed.
Add an SSIS Boolean variable. This is IsColumnNull in the following example. Next add a C# Script Task with IsColumnNull variable in the ReadWriteVariables field, and (optionally) a variable holding the file path ReadOnlyVariables pane.
Next set Precedence Constraints to check for both a true condition (has null rows) or false condition (does not have null records). Since the IsColumnNull variable is a Boolean, use just the variable itself as the expression to check for all null rows, but add ! for non-nulls, i.e. !#[User::IsColumnNull].
Connect the appropriate Data Flow Tasks with each destination table to the corresponding Precedence Constraint. For example, add the Data Flow Task with the "invalid file table" as the destination after the Precedence Constraint checking for a true value in the IsColumnNull variable.
Precedence Constraint For Rows with Nulls:
Precedence Constraint for Rows without Nulls:
Script Task Example:
using System;
using System.Data;
using System.IO;
using System.Data.OleDb;
using System.Linq;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
string fullFilePath = Dts.Variables["User::FilePath"].Value.ToString();
string fileName = Path.GetFileName(fullFilePath);
string filePath = Path.GetDirectoryName(fullFilePath);
string connStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filePath
+ ";Extended Properties=\"text;HDR=No;FMT=Delimited\";";
//add filter for NOT NULL on given column to only return non-nulls
string sql = "SELECT F2 FROM " + fileName + " WHERE F2 IS NOT NULL";
//if file has column names replce "connStr" and "sql" as shown below
/*
string connStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filePath
+ ";Extended Properties=\"text;HDR=Yes;FMT=Delimited\";";
string sql = "SELECT ID FROM " + fileName + " WHERE ID IS NOT NULL";
*/
using (OleDbDataAdapter oleAdpt = new OleDbDataAdapter(sql, connStr))
{
DataTable dt = new DataTable();
oleAdpt.Fill(dt);
//if emtpy set IsColumnNull SSIS variable to true
if (dt.Select().Count() < 1)
{
Dts.Variables["User::IsColumnNull"].Value = true;
}
else
{
Dts.Variables["User::IsColumnNull"].Value = false;
}
}
My situation is
I have a parameter, this is a list, allowing multi values. That mean the first record in the list is 'Select All'
When user select All I need to include in my report all records that match with the list plus those that are blank. (My dataset is returning these)
When user select only 1 or a few I want to include only these records. No those that are blank
My problem:
I have a filter in my dataset that evaluate a parameter in a list, but I need to add a conditional filter to include a blank records when the selection will be "Select All"
I tried to use expression but this doesn't work
Filter expression
Fields!NAME.Value in = Parameters!List.Value !!!!!!!!!!!! Work Fine
But I need to change it like as
If Parameters!List.Value = 'Select All' Then
Fields!NAME.Value in = Parameters!List.Value or Fields!NAME.Value = " "
Else
Fields!NAME.Value in = Parameters!List.Value
End
Can you give an advice who can I resolve it please !!!
I'm working in SSRS R2
Thanks!!
This worked for me
Expression: =IIF(Parameters!pLocation.Value <> " All Locations", Fields!LOCATION.Value, FALSE)
Operator: =
Value: =IIF(Parameters!pLocation.Value <> " All Locations", Parameters!pLocation.Value, FALSE)
If you use Filter on your Dataset, try this:
Expression: [NAME]
Operator: IN
Value (fx): =Split(Replace(Join(Parameters!List.Value, ","), "Select All", " "), ",")
Try to work along this path. Basically you can reconstruct the multi value items into a string with Join(), and deconstruct it again into array by using Split(); where in between, you can manipulate them, for modifying (e.g. converting "Select All" into " "), adding (imitating "OR"), or removing extra items.
There is an alternative for this.
Add one more item to the paramater dataset values say "Not Available" as Label and value with the null. then there will be no change in the stored procedure and you can retrieve the data.
If the user select the specific item then he will get those values only. If he selects all then he will get the data for the null also with the all the others.
Hope this will help
You can put the logic in just one location if you do it this way.
You filter on the parameter, unless it's all values then the filter always matches.
Just a little cleaner.
Expression: =IIF(Parameters!pLocation.Value <> " All Locations", Fields!LOCATION.Value, " All Locations")
Operator: =
Value: =Parameters!pLocation.Value
I have the following:
Textfield called: WoNr
Table column called: Workorder
= DMax("[WoNr]","[Workorder]","[Workorder]") + 1
In the text field named WoNr I have entered the code above, I get an error.
Why is this?
Why are you using "[Workorder]" as the criterion (i.e., the last parameter)? Try the following:
= DMax("WoNr", "Workorder")
If this works, continue reading.
Now about the "+ 1" thing. You say that WoNr is a text field (rather than a numeric field). So, what do you want to get? Do you want to append "1" to the string (WoNr = "D1" => Result = "D11") or is WoNr actually a numeric value and you want to add 1? In any case, you should make your intention clear. For string concatenation, use &:
= DMax("WoNr", "Workorder") & "1"
for arithmetic operations, convert your text into an appropriate numeric data type first:
= CLng(DMax("WoNr", "Workorder")) + 1
I have an optional field in a database that I'm pulling out using a DAO Record Set. I need to check whether or not the field is set before I concatenate it with other fields. So far I have the following code snippet which I've tried with both Is and = (that's the obviously wrong syntax [[Is | =]]) to no avail. It appears that if I use = it will not correctly compare with Null and if I use Is then it complains that it's not comparing with an Object.
While Not rs.EOF
If rs.Fields("MiddleInitial") [[Is | =]] Null Then thisMiddleInitial = "" Else thisMiddleInitial = rs.Fields("MiddleInitial")
If prettyName(myLastName, myFirstName, myMiddleInitial) = prettyName(rs.Fields("LastName"), rs.Fields("FirstName"), thisMiddleInitial) Then
MsgBox "Yay!"
End If
rs.MoveNext
Wend
If there's a simpler way to do this I'm totally open to it. prettyName takes 3 Strings as parameters and initially I was just trying to pass rs.Fields("MiddleName") directly but it threw up at a Null value. I'd prefer to do something more direct like that but this is the best I could come up with.
How about:
IsNull(rs.Fields("MiddleInitial").Value)
You could also have a look at this article which has some explanation about Null values in Access VBA apps and how to handle them.
For the example you show, Nz would work:
thisMiddleInitial = Nz(rs!MiddleInitial,"")
Or simply concatenating the string with an empty string:
thisMiddleInitial = rs!MiddleInitial & ""
Your question has been answered by Remou, seems to me, but it occurs to me that you may just be trying to get proper concatenation of the name fields. In that case, you could use Mid() and Null propagation in VBA to get the result.
I don't use separate middle initial fields, so my usual name concatenation formula is:
Mid(("12" + LastName) & (", " + FirstName), 3)
The "12" string at the beginning is going to be tossed away if LastName is Not Null and ignored if it is null, because the + concatenation operator propagates Nulls.
To extend this to include middle intials would look like this:
Mid(("12" + LastName) & (", " + FirstName) & (" " + MiddleInitial), 3)
Assuming your UDF is not doing some kind of complicated cleanup of nicknames/abbreviations/etc., this could replace it entirely, seems to me.
If rst.Fields("MiddleInitial").Value = "Null" Then
This works for me. I use MS SQL Database.
I think the NoMatch option might work in this situation:
If rs.NoMatch = True Then
I prefer using the below to account for both Null and Empty string values. It's a good check to use you have forms collecting values from users.
If Trim(rs.Fields("MiddleInitial") & "") = "" then