I have a text file with more than one table's data in it (different column counts). I import the whole row as one column. Based on a conditional split, the rows are dispersed to their correct flow. I use a script component to split the single column values (row) into the correct columns for that table and give it as output columns. All of this is working fine, and data looks fine.
My problem comes in with some numeric fields. When a numeric field has no values in it, it ends up in the table with another column's numeric value.
I have put data viewers everywhere, in not one of them there is data for the column that should be empty. When I look in the table itself, there it is... data from another column.
It is not the mappings, I checked it a dozen times.
It is not the names that are the same or something like that.
There is no data according to dataviewers anywhere in the load process.
There is no hidden code anywhere.
I droped and recreated the table.
I displayed a messagebox with the column's (that is supposed to be empty) assigned "column value", and no data, like expected.
I used a derived column, same result, no data in dataviewers, but eventualy data in the table.
I also created another test table with those numeric fields as varchar. When I do this, the column is empty (like expected). When I change it to numeric, the field is populated again. (If it was the other way around I could understand).
What can be te reason for this? It is driving me insane.
EDIT
Script code:
//C#
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
ASCIIEncoding enc = new System.Text.ASCIIEncoding();
char[] seperator = { '|' };
Byte[] ByteBlob;
String[] ColumnValue;
ByteBlob = Row.Column0.GetBlobData(0, (int)(Row.Column0.Length));
ColumnValue = enc.GetString(ByteBlob).Split(seperator);
Row.OutputColumn0 = ColumnValue[0];
Row.OutputColumn1 = ColumnValue[1];
///etc
Just to give an example of what it does, this is what a row would look like in a sence.
Column names:
Source|Tablename|Value1|Value2|Description|Value3|Description2|Value4
Actual Data:
ABC|Revenue|123,456|729,537|MisterX||None|
Data in Table:
ABC|Revenue|123,456|729,537|MisterX|729,537|None|729,537
try using Row.ColumnX_IsNull , for example if (Row.Column0_IsNull) {youroutputcolumn=null} else {...}
Related
I am working on a SSIS(2017) solution to read and load data from these 3 excel file names:
message_EDF_100420202.csv
message_UltaBIO_10042020.csv
message_SEIDV_10042020.csv
What I need to do is get only EDF or UltraBIO or SEIDV as a new column (derived column task)
so I need some help to set up correctly the substring function inside the derived column task.
any suggestion?
It appears your pattern is message_ Stuff-I-Want _junk (spaces not present in actual pattern). It's delimited by underscores and since the starting text is constant, that makes life easier.
Create a new column called MessageLessName
Remove the message_ portion with an expression
REPLACE([SourceFile], "message_", "")
Now, we want to take the left N most characters where N corresponds to the location of the underscore in our new column MessageLessName. For ease of debugging, I propose you add a second Derived Column Task to the output of the first one (where we defined MessageLessName). Here, we're going to create FirstUnderscore column
findstring([MessageLessName], "_", 1)
Finally, we'll add a third Derived Column Task and here-in is where we'll get to the final file name.
LEFT([MessageLessName], [FirstUnderscore])
Now that may be off by one due to my being lazy but because you can check each step along the way, you can verify MessageLessName is exactly what you think it should be and that FirstUnderscore is N characters in from our MessageLessName column.
script component using Split.
Row.ColumnName.ToString().Split('_')[1];
You are taking the column value and casting to string. (current value is the whole string)
Next is splitting based on '_' (current value is an array of three strings)
Finally you are taking the second value (0 based) (current value is the string you want)
Here's a little bonus. Getting the date as well:
string[] breakdown = Row.fileNames.Split('_');
Row.Type = breakdown[1];
string dateToFix = breakdown[2].Replace(".csv", "");
Row.Date = DateTime.Parse(dateToFix.Substring(0,2) +"/"
+ dateToFix.Substring(2,2) + "/" + dateToFix.Substring(4,4));
I have a store procedure which brings the data as shown below . I'm new to SSRS reporting, I would like to show only those row where "email" column is null. How can i achieve it in SSRS ? As i mentioned I'm very new to this , any screenshot will help me a lot. Thank you for your time.
For this problem, you'll want to change the row visibility to hide rows with a value in that column. I assume you're using a table or matrix to layout this data. You'll want to right click on the row where your data fields are entered. Specifically, the grey box at the left of the row.
From there, you'll need to select the option to Show or hide based on an expression.
And finally, you'll need to enter an expression that finds the values in the email field. I'm not exactly sure what the field names are called but something like the following expression should do it.
= Not IsNothing(Fields!EmailField.Value)
This will check the field where you get the email value with a built-in function of IsNothing. Additionally, since you want fields that do not contain values, the Not keyword reverses the results. If the function evaluates to true and a value is present, the row will be hidden and vice versa.
I'm using the "phpsoda" library and trying to query the city of Seattle's permits dataset through the Socrata API (SODA). Data: https://data.seattle.gov/Permitting/Land-Use-Permits/ht3q-kdvx
They have several columns such as the ones I'm selecting below, but in the "AppliedDate" and "IssuedDatae" columns, there may or may not be data in that column.
So when I try to look at the array and arrange this into an HTML table, I'm getting some arrays (rows) that have fewer elements (columns) than other rows. This turns out to make it difficult to display since I don't know which columns are missing in the array (row).
I'm wondering if when I make the query, that those empty fields will look they seem in the visualized table on their site or when I export a CSV. Those columns in the query will return into the array element an empty string ("") instead so my rows and columns will come out all filled with values.
$soql->select("PermitNum", "AppliedDate", "IssuedDate", "Description", "OriginalAddress1")
->where("PermitClass = 'Multifamily' OR PermitClass = 'Commercial')
->limit(20);
$results = $ds->getDataset($soql);
Data would look something like...
print_r($results);
Array[0] -> [Description]=>"XXXXXXX", [PermitNum]=>"123456"
Array[1] -> [Description]=>"XXXXXXX", [PermitNum]=>"234567", [AppliedDate]=>"XX/XX/XXXX"
So the first row is missing the "AppliedDate" column just because it's not in the data.
Will I need to just go through this manually in the results array using a loop and checking column names and inserting an empty string if the loop doesn't find a column?
Following my own advice, I was able to just check for each row key if a specific key was missing (array_key_exists function), then I'd fill it in with "" if it returned false.
This seemed to work.
I have same problem. Socrata API will skip empty field or null value. Means, if field is null or empty, the result will not show field-name:'', instead, the result will just missing this, that cause your shorter row.
This is annoying bug, I have to fix it by my own. If I found it missing field, I will have to add field-name:'' to the result json, that will fix your shorter row problem. Make equal length row.
I have an RDLC report with a table in it. I want to hide a string column if there is no data present in any of the rows(supress if blank sort of thing). I have been googling for the last 2 hrs and trying with different suggestions but i can not make it work.
I tried the following so far.
Set the expression for the Hidden attribute of the column to
=IIf(Fields!Category.Value = "", True, False)
But it is checking only the first row but not the entire row set.
Trying to create a concatenated string with the field values, so if the final string is empty i'll hide the table column. But i can't find a way of concatenating a string column from a table. Runningtotal() works with only numbers it seems.
Can some one point me to the right direction.
initially i thought it is very easy, but doesn't seem so.
Did you try to use CountDistinct?
I think something like this should do the trick
=(CountDistinct(Fields!Category.Value) > 1) Or (Fields!Category.Value != "")
Or you can try to make a custom string aggregate function
String aggregation in SSRS 2005
I have a couple of questions about the task on which I am stuck and any answer would be greatly appreciated.
I have to extract data from a flat file (CSV) as an input and load the data into the destination table with a specific format based on position.
For example, if I have order_id,Total_sales,Date_Ordered with some data in it, I have to extract the data and load it in a table like so:
The first field has a fixed length of 2 with numeric as a datatype.
total_sales is inserted into the column of total_sales in the table with a numeric datatype and length 10.
date as datetime in a format which would be different than that of the flat file, like ccyy-mm-dd.hh.mm.ss.xxxxxxxx (here x has to be filled up with zeros).
Maybe I don't have the right idea to solve this - any solution would be appreciated.
I have tried using the following ways:
Used a flat file source to get the CSV file and then gave it as an input to OLE DB destination with a table of fixed data types created. The problem here is that the columns are loaded, but I have to fill them up with zeros in case the date when it is been loaded or in most of the columns if I am not utilizing the total length then it has to preceded with zeros in it.
For example, if I have an Orderid of length 4 and in the flat file I have an order id like 201 then it has to be changed to 0201 when it is loaded in the table.
I also tried another way of using a flat file source and created a variable which takes the entire row as an input and tried to separate it with derived columns. I was to an extent successful in getting it, but at last the data type in the derived column got fixed to Boolean type explicitly, which I am not able to change to the data type I want.
Please give me some suggestions on how to handle this issue...
Assuming you have a csv file in the following format
order_id,Total_sales,Date_Ordered
1,123.23,01/01/2010
2,242.20,02/01/2010
3,34.23,3/01/2010
4,9032.23,19/01/2010
I would start by creating a Flat File Source (inside a Data Flow Task), but rather than having it fixed width, set the format to Delimited. Tick the Column names in the first data row. On the column tab, make sure row delimiter is set to "{CR}{LF}" and column delimiter is set to "Comma(,)". Finally, on the Advanced tab, set the data types of each column to integer, decimal and date.
You mention that you want to pad the numeric data types with leading zero's when storing them in the database. Numeric data types in databases tend not to hold leading zero's. So you have two options; either hold the data as the type they are in the target system (int, decimal and dateTime) or use the Derived Column control to convert them to strings. If you decide to store them as strings, adding an expression like
"00000" + (DT_WSTR, 5) [order_id]
to the Derived Column control will add up to 5 leading zeros to order id (don't forget to set the data type length to 5) and would result in an order id of "00001"
Create your target within a Data Flow Destination and make the table/field mappings accordingly (or let SSIS create a new table / mappings for you).