I have a database(Source) which has a column named "country name" and few cells in it are empty & when I am transferring its data to another database (destination) it is also empty
I have tried to use derived column in which I had used REPLACE() but is didn't work I thought it would have been any "", " " or "\t" but it was neither of these then I thought may be it is NULL & I used ISNULL() but this also failed.
I assume you are trying to work out what the "empty" data in the source column actually is? If thats the case then just convert the value to its ascii value and look that value up in an ascii table to find out what it is.
If the empty data is actually more than one character you may need to extract each character individually using, for example, the MID function.
You did not specify which DB is in the source and which is in the target, and it has meaning.
Anyway, add this code to the Derived Column component:
(DT_WSTR,100)country_name == "" || ISNULL(country_name) == TRUE ? "new_value" : country_name
Results:
Related
I am having " (single quotes) in my data ,all the corresponding column values clubbed into one column even though I have used the delimiter value. In my case '|' is my delimiter.
Actual Data:
a|"b|c|d|
Expected Output:
a|"b|c|d
Actual Output Came:
a|"b**|c|d|**null|null| ( here 3rd & 4th column coming as single column, in place of actual 3rd & 4th col.. getting values as null)
I have tried below approach:
Approach 1:
df=spark.read.csv(filepath,header=True,sep='|',quote='')
Above approach gives particular column data correctly but empty columns coming values as """" but we need empty column as it is.
Approach 2:
df=spark.read.csv(filepath,header=True,sep='|',quote='',escape='\"')
Above approach gives values clubbing into single column as like actual output.
After some heads & trails
found solution
read the file with both below options :
quote='',escape='\"'
In my SSIS project I have to retrieve my data from a flat csv file. The data itself looks something like this:
AccountType,SID,PersonID,FirstName,LastName,Email,Enabled
NOR,0001,0001,Test,Test0001,Test1#email.com,TRUE
NOR,1001,NULL,Test,Test1002,Test2#email.com,FALSE
TST,1002,NULL,Test,Test1003,Test3#email.com,TRUE
I need to read this data and make sure it has the correct datatypes for future checks. Meaning SID and PersonID should have a numeric datatype, Enabled should be a boolean. But I would like to keep the same columns and names as my source file.
It seems like the only correct way to read this data trough the 'Flat File Source'-Task is as String. Otherwise I keep getting errors because "NULL" is literally a String and not a NULL value.
Next I perform a Derived Column transformation to get rid of all "NULL" values. For example, I use the following expression for PersonId:
(TRIM(PersonID) == "" || UPPER(PersonID) == "NULL") ? (DT_WSTR,50)NULL(DT_WSTR,50) : PersonID
I would like to immediatly convert it to the correct datatype by adding it in the expression above, but it seems impossible to select another datatype for the same column when I select 'Replace 'PersonId'' in the Derived Column dropdown box.
So next up I thought of using the Data Conversion task next to change the datatypes of these columns, but when I use this it only creates new columns, even when I enter the output alias to remain the same.
How could I alter my solution to efficiently and correctly read this data and convert its values to the correct datatypes?
I'm trying to ascertain the best way to test for blank/empty/null text records.
By this I mean text records which are either:
Null
An Empty String ("")
Any number of spaces (e.g. " ")
I've experimented with varying query criteria using the following table (named Table1):
Here, the Spaces record contains an arbitrary number number of spaces, the Empty String record contains an empty string (""), and the Null record is just that.
My first thought to achieve my goal was to use a query such as:
SELECT * FROM Table1 WHERE TextField IS NULL OR Trim(TextField)=""
And this indeed returns the three target records: Spaces, Empty String & Null.
However somewhat oddly, using:
SELECT * FROM Table1 WHERE TextField = ""
Returns both the Empty String record and the Spaces record:
Which leads me to think that my query can omit the Trim function and become:
SELECT * FROM Table1 WHERE TextField IS NULL OR TextField=""
But is this reliable?
Is this best practice when selecting empty text records?
Alternatively I considered using:
SELECT * FROM Table1 WHERE Nz(TextField)=""
Are there drawbacks to this approach?
Are there better ways to achieve this?
EDIT: To be specific, my question is ultimately:
What is the best way to select blank text records?
For normal text, Access handles strings with only spaces as empty strings (e.g. SELECT " " = "" returns -1 = True).
This means that the solution introduced by June7, WHERE TextField & "" = "", is likely the most efficient solution. Another contender is the Nz function, which is handled by the database engine and somewhat optimized.
When using indexes, however, both string concatenation and function calls invalidate the index. WHERE TextField IS NULL OR TextField="" doesn't, and will be the fastest by far if TextField is indexed.
If you bring rich text into the mix, you're not going to get away with anything but casting it to normal text first. In Access 2016, when you enter a space in a rich text field, it actually contains the following: <div> </div> (you can see this by using RichTextField & "").
For a rich text field, indexes are not going to work anyway, so you can use the following:
WHERE PlainText(RichTextField) & "" = ""
Nz(TextField)=""
Is the approach most often used when dealing with nulls and empty strings.
In Access, The Long Text data type will automatically be trimmed to save space, this is why Trim(TextField) is the same as TextField. If you ever convert it to a Rich Text data type, these will be different. In that case:
TRIM(Nz(TextField))=""
Should cover all your bases.
I have a pipe delimited file.
I want to store the records of this file in sql server table.
Schema of my table is already decided.I want to change the datatypes of the columns retrieved from file before storing data into table.
There are some ways I know , we can do
Data Conversion tool but problem this component is if column is blank and I want to store it into numeric(18,0) column then it not working.
Any suggestion how I can achieve this task?
First use a derived column. and within the derived column expression you check the input value, if its length is zero, or the content isNull then set the value to 0 (with a decimal type). ELSE you retrive the value of the input column as it is with numeric values conditions that you want.
LEN([Column 1]) == 0 || ISNULL([Column 1]) ? 0 : (DT_NUMERIC,18,0)[Column 1]
It is important to note: if all your input data are already NULL (BLANK) then you just have to do do the following:
NULL(DT_NUMERIC, 18, 0)
And here we go, in your case there is no need for Data Conversion component ;)
Best Regards,
S.ANDOURA
I have table named "city" with column named "city_name" with about 200 records.
I have created another colum named slugs where I want to copy all the records from "city_name" in the same row with spaces replaced with dash - and lowercase.
How can I achieve this via phpmyadmin.
Thanks
You should be able to do this via the following query:
UPDATE city SET slugs=LOWER(REPLACE(city_name, " ", "-"))
Breaking this down, we're using REPLACE to swap all instances of " " with "-" in the existing city_name column, and then passing the result of this to the LOWER function (to convert the data to lower case) before setting the slugs field with this value.
Depending on how "clean" your data is, you might also want to TRIM the data (to remove any leading or trailing spaces in the city_name field) before you apply the REPLACE as such:
UPDATE city SET slugs=LOWER(REPLACE(TRIM(city_name), " ", "-"))
Incidentally, if you've not used (My)SQL much I'd recommend a read of the String Functions manual page - the time you spend on this now will more than repay itself in the future.
Here is SQLFiddle
MYSql documentation for function LOWER(str) and REPLACE(str,from_str,to_str)
UPDATE city SET slugs = LOWER(REPLACE(city_name," ", "-"));