Derived column expression give truncation error - ssis

I have the following expression:
IsNull is column Name-- all the columns are numeric
ExpectedPremium:
([Isnull] < ExpectedMinPremium) ? ("Min" + ((DT_WSTR,10)(exp_pct_min * 100)) + "% = " + (DT_WSTR,10)ExpectedMinPremium) : ([Isnull] > ExpectedMaxPremium) ? ("Max" + ((DT_WSTR,10)(exp_pct_max * 100)) + "% = " + (DT_WSTR,10)ExpectedMaxPremium) : ""
The result of the column should look like : MAX 60% = 389727.600
I am not getting an error while creating the derived column but once I run the package I get the following message:
The "Derived Column" failed because truncation occurred, and the truncation row disposition on "Derived Column.Outputs[Derived Column Output].Columns[ExpectedPremium]" specifies failure on truncation.
A truncation error occurred on the specified object of the specified component.
It writes some rows (less than the input), I increased the destination length and it didn't work.
How to fix this expression?

Related

Warning: (1366, "Incorrect string value: '\\xB4\\xEB\\xC7\\xD1\\xB9\\xCE...' for column 'VARIABLE_VALUE' at row 1") result = self._query(query)

This error occurs whenever I try to insert below table in mysql schema.
I used code like this
self.collect.allday_list.to_sql(name=code_name, con=self.collect.engine_allday, if_exists='append')
and self.collect.egine_allday is
self.engine_allday = create_engine("mysql+mysqldb://" + info.db_id + ":" + info.db_passwd + "#"
+ info.db_ip + ":" + info.db_port + "/allday_list", encoding = 'utf-8')
and Warning: (1366, "Incorrect string value: '\xB4\xEB\xC7\xD1\xB9\xCE...' for column 'VARIABLE_VALUE' at row 1") result = self._query(query)
it doesn't have any emoji or special letter, but error is occurred
how do i get this table into schema in mysql?
Somewhere in the data you are trying to insert there is text encoded in euc-kr:
>>> bytes.decode(b'\xB4\xEB\xC7\xD1\xB9\xCE', encoding='euc-kr')
'대한민'
These bytes do not represent valid text in utf-8. You can try changing the database connection encoding to euc-kr, or converting the data from euc-kr to utf-8 when you read it.
On the other hand, the screenshot of the data has only numbers, no text. This suggests you may be trying to load the wrong file or that perhaps there is a portion of the file that you didn't want to insert in the database.

Why is setting SQL variable is creating an error?

Following SQL statement is creating an error with message :
"Message: Fatal error encountered during command execution."
"Inner exception: Parameter '#LastUserID' must be defined."
If I directly use LAST_INSERT_ID() instead of LastUserID, it always returns zero (hence fails at second insert) when executed like this.
I don't see my syntax is different than in mySQL document.
Could some one help me ?
string Query = #"INSERT INTO login (" +
"LOGIN_EMAIL," +
"LOGIN_PASSWORD," +
"LOGIN_SALT," +
"LOGIN_LAST_LOGIN_DATE," +
// "LOGIN_LAST_LOGIN_LOCATION," +
"LOGIN_ACCOUNT_STATUS," +
"LOGIN_LOGIN_ATTEMPTS," +
"LOGIN_CREATED_DATE) " +
"VALUES (" +
"#Parameter2," +
"#Parameter3," +
"#Parameter4," +
"#Parameter5," +
// "#Parameter6," +
"#Parameter6," +
"#Parameter7," +
"#Parameter8); " +
"SET #LastUserID = LAST_INSERT_ID(); " +
"INSERT INTO user_role (" +
"USER_ROLE_USER_ID," +
"USER_ROLE_ROLE," +
"USER_ROLE_STATUS," +
"USER_ROLE_CREATED_DATE) " +
"SELECT " +
"#LastUserID," +
"#Parameter9," +
"#Parameter10," +
"#Parameter11 " +
"FROM dual WHERE NOT EXISTS (SELECT USER_ROLE_USER_ID FROM user_role " +
"WHERE USER_ROLE_USER_ID = #LastUserID AND USER_ROLE_ROLE = #Parameter9)";
MySqlCommand oCommand = new MySqlCommand(Query, oMySQLConnecion);
oCommand.Transaction = tr;
Create a procedure in which you first do your insert, cache the last inserted id, do the other insert and let it print out your parameters with a bool if your last insert worked or not. That way you can debug it properly.
In general you should avoid concatinating strings to generate sql-commands or you might get troubles with parameters containing unexpected characters or be hit by a injection.
Simple fix : Replace "$LastUserID" with "$'LastUserID'". The ephostophy makes the difference.

Convert yymmdd to datetime using ssis derived column

Below are the strings representing date of birth (yymmdd). I want to convert it into datetime format and load into a database using ssis. How can I do it? In my derived column I have (DT_DATE)(SUBSTRING([Drv DOB],1,2) + "-" + SUBSTRING([Drv DOB],3,2) + "-" + SUBSTRING([Drv DOB],5,2)), but it's not working.
I GET THESE ERRORS:
[Derived Column [2]] Error: An error occurred while attempting to perform a type cast.
[Derived Column [2]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "Derived Column" failed because error code 0xC0049064 occurred, and the error row disposition on "Derived Column.Inputs[Derived Column Input].Columns[Drv DOB]" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
•470324
•470324
•470209
•101
•0
YEAR have 4 digits not 2, so it would be:
(DT_DATE)("20" + SUBSTRING([Drv DOB],1,2) + "-" +
SUBSTRING([Drv DOB],3,2) + "-" +
SUBSTRING([Drv DOB],5,2))
This works for me! Thanks everyone
([Drv DOB] == "0") || ([Drv DOB] == "101") ? NULL(DT_DBTIMESTAMP) : (DT_DBTIMESTAMP)(SUBSTRING([Drv DOB],1,2) + "-" + SUBSTRING([Drv DOB],3,2) + "-" + SUBSTRING([Drv DOB],5,2))

Load Data Infile errors

In the syntax of load infile data i saw that the fields and line clauses are optional. So I used only character set clause for utf8
Here my sql:
cmd = new MySqlCommand("LOAD DATA INFILE " + filename + " INTO TABLE " + tblname + " CHARACTER SET 'UTF8'", conn);
filename is the addresse it's format is: "E:\Macdata\20131228\atelier.sql"
table name is directly taken from database is as : "atelier"
But I get the error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'E:\Macdata\20131228\atelier.sql INTO TABLE atelier CHARACTER SET 'UTF8'' at line 1
What is the mistake in my query command ?
MYSQLversion is 5.0.10 with XAMPP
After changing the query I begin to receive fatal error number 0 (enclosed filename with ')
cmd = new MySqlCommand("LOAD DATA LOCAL INFILE '" + filename + "' IGNORE INTO TABLE " + tblname + " CHARACTER SET UTF8", conn);
My data file has this form which works on phpmyadmin
INSERT INTO `atelier` VALUES(1, 'Chateau Carbonnieux -1', '2013-12-26', 23, 10, 0, '4 macarons differents', 'mamie', '2013-12-15 11:09:14', 'sabrina', '2013-12-18 05:29:26');
As the error says, your statements is wrong. Quotes are missing in your first statement (see second statement). Check the syntax here:
http://dev.mysql.com/doc/refman/5.6/en/load-data.html
Some sparse notes:
0 is not a fatal error, it's the code for success.
IGNORE handles duplicate rows, not syntax errors.

SSIS expression to convert string to date

Using SQL Server 2008, Visual Studio 2008.
Trying to convert the data from a string column into a derived Date column.
The string can be in two formats, either : "dd/mm/yy" (where century 20 can be assumed), or "dd/mm/yyyy" . e.g. "02/05/12", "23/12/02", "13/08/2012".
So I tried an expression like this for a Derived column :
DateReceived == "" ? NULL(DT_DATE) : (DT_DATE)((LEN(RTRIM(DT_DATE)) == 10 ? SUBSTRING(DateReceived,7,4) : ("20" + SUBSTRING(DateReceived,7,2))) + "-" + SUBSTRING(DateReceived,4,2) + "-" + SUBSTRING(DateReceived,1,2))
but Visual Studio gives error :
"The expression was not valid, or there is an out-of-memory error".
It accepts a simpler expression like :
DateReceived == "" ? NULL(DT_DATE) : (DT_DATE)("20" + SUBSTRING(DateReceived,7,2) + "-" + SUBSTRING(DateReceived,4,2) + "-" + SUBSTRING(DateReceived,1,2))
but this expression will only work for "dd/mm/yy" format.
What am I doing wrong ? Thanks.
Never mind, the error was due to a typo. (Was calling RTRIM on the column type, not column name).
The following works :
DateReceived == "" ? NULL(DT_DATE) : (DT_DATE)((LEN(RTRIM(DateReceived)) == 10 ? SUBSTRING(DateReceived,7,4) : ("20" + SUBSTRING(DateReceived,7,2))) + "-" + SUBSTRING(DateReceived,4,2) + "-" + SUBSTRING(DateReceived,1,2))