I'm getting an error from OrmLiteWriteCommandExtensions in my log, and I can't seem to find where this happens. I have no loss of data; when I have the logfactiory set to null, it breaks on no errors; I have set it to allow zero date, but that didn't help. I not sure how to debug this or get around it. It clogs up my logg with statements that I can't track.
This occurs on both linux and windows.
Doesn anyone have an idea how to get at this?
OrmLiteWriteCommandExtensions System.FormatException: String was not recognized as a valid DateTime.
at System.DateTimeParse.ParseExact(String s, String format, DateTimeFormatInfo dtfi, DateTimeStyles style)
at System.DateTime.ParseExact(String s, String format, IFormatProvider provider, DateTimeStyles style)
at ServiceStack.Text.Common.DateTimeSerializer.ParseShortestXsdDateTime(String dateTimeStr)
at ServiceStack.OrmLite.Converters.DateTimeConverter.FromDbValue(Type fieldType, Object value)
at ServiceStack.OrmLite.OrmLiteWriteCommandExtensions.PopulateWithSqlReader[T](T objWithProperties, IOrmLiteDialectProvider dialectProvider, IDataReader reader, Tuple`3[] indexCache, Object[] values)
UPDATE:
The logging occurs in the correct file set in the nlog config file, BUT for this exception only it does not conform to formatting in the layout set for NLOG (all other exceptions do!).
Layout string:
${longdate} ${machinename} ${logger} ${level} ${message} ${onexception:EXCEPTION OCCURRED:${exception:format=type,message,method:maxInnerExceptionLevel=5:innerFormat=shortType,message,method}}
UPDATE2 SOLVED:
So this was fun... Before I start to explain how I solved it I have to say that I have no idea why it writes in the correct log file, but ignore all formatting for logging errors - that be another dragon to slay for another time.
So the problem was that a varchar field in the mysql database had birthdate as string and the implicit conversion between DateTime in dotnet to string thre an error.
After changing the field to datetime in thet database the error went away.
Funny thing is that I hade to set minlevel to trace to get the error in context with its call to the database - but it still ignored all error formatting, had no stacktrace, no sql statement, and no line number associated with the error in the log.
So the problem was that a varchar field in the mysql database had birthdate as string and the implicit conversion between DateTime in dotnet to string thre an error.
After changing the field to datetime in thet database the error went away.
Funny thing is that I had to set minlevel to trace to get the error in context with its call to the database - but it still ignored all error formatting, had no stacktrace, no sql statement, and no line number associated with the error in the log.
Related
So I want to import a datetime from a txt:
2015-01-22 09:19:59
into a table using a data flow. I have my Flat Source File and my destination DB set up fine. I changed the data type for the txt input for that column in the advanced settings and the input and output properties to:
database timestamp [DT_DBTIMESTAMP]
This is the same data type as the DB used for the table so this should work.
However, when I execute the package I get a error saying the data conversion failed... How do I make this possible?
[Import txt data [1743]] Error: Data conversion failed. The data conversion for column "statdate" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
[Import txt data [1743]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "output column "statdate" (2098)" failed because error code 0xC0209084 occurred, and the error row disposition on "output column "statdate" (2098)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
[Import txt data [1743]] Error: An error occurred while processing file "C:\Program Files\Microsoft SQL Server\MON_Datamart\Sourcefiles\tbl_L30T1.txt" on data row 14939.
On the row he is giving the error the datetime is filled up with spaces. that is why on the table the "allow nulls" is checked but my SSIS package gives the error for some reason... can I somewhere tell the package to allow nulls aswell?
I suggest you import the data in to a character field and then parse it after entry.
The following function should help you:
SELECT IsDate('2015-01-22 09:19:59')
, IsDate(Current_Timestamp)
, IsDate(' ')
, IsDate('')
The IsDate() function returns a 1 when it thinks the value is a date and a 0 when it is not.
This would allow you to do something like:
SELECT value_as_string
, CASE WHEN IsDate(value_as_string) = 1 THEN
Cast(date_as_string As datetime)
ELSE
NULL
END As value_as_datetime
FROM ...
I solved it Myself. Thank you for your suggestion gvee but the way I did it is way easier.
In the Flat File Source when making a new connection in the advanced tab I fixed all the data types according to the table in the database EXCEPT the column with the timestamp (in my case it was called "statdate")! I changed this data type to a STRING because otherwise my Flat File Source would give me a conversion error even before any scripts would have been able to be executed and the only way arround this was setting the error output to ignore failure wich I don't want. (You still have to change the data type after you set it to a string in the advanced settings by right clicking the flat file source -> show advanced editor -> going to the output colums and changing the data type there from Date to string.)
After the timestamp was set to a string I added a Derived Column with this expression to delete all the spaces and give it then "NULL" value:
TRIM(<YourColumnName>) == "" ? (DT_STR,4,1252)NULL(DT_STR,4,1252) : <YourColumnName>
Next I added a Data Conversion to set the string back to a timestamp. The Data conversion is finally connected to the OLE DB Destination.
I hope this helps anyone with the same problem in the future.
End result: Picture of data flow
I have completed migrating database from sqlserver to mysql but it seems to have problems in rails. When i fetch record in rails console for eg:
<AuthAdmin:0xb6e506f8 #attributes={"Status"=>"1", "LastUpdateSeqNo"=>nil, "CreationDate"=>"2005-08-03 22:53:57", "AuthAdminID"=>"8987", "PropertyID"=>nil, "Password"=>"trustbss", "LastUpdate"=>"2012-07-12 05:15:02", "UserType"=>"0", "LoginName"=>"dev"}
Now attributes such as CreationDate and AuthAdminID suppose to be date and integer are displaying as string.But when I do
AuthAdmin.find(:first).AuthAdminID.class output is Fixnum
You can check record given above,it shows string. Now when i do arithmatic operation in my views caught issue string can't be coerced into Fixnum.Explicitly changing everything to their own TYPE is a very bad idea.
Hope that explain my problem.
Look like no one come across this problem.
I'm using a reflected sqlalchemy mapped class into a SQL Server table.
I DBSession.add() instances of Activities class (the mapped class) with data I get from a different source.
and then I called transaction.commit() (since I'm calling from tg2 I can't use session.commit())
the error traceback:
DataError: (DataError) ('22001', '[22001] [Microsoft][ODBC SQL Server Driver][SQ
L Server]String or binary data would be truncated. (8152) (SQLExecDirectW); [010
00] [Microsoft][ODBC SQL Server Driver][SQL Server]The statement has been termin
ated. (3621)') u'INSERT INTO [NOAR_LOADEVENTS] ([EVENTCODE]) VALUES (?)' ((u'210
401',), (u'210402',), (u'210602',), (u'210603',), (u'000010',), (u'000102',), (u
'000206',), (u'000107',) ... displaying 10 of 49 total bound parameter sets ...
(u'211302',), (u'210403',))
be glad to any help about this since I'm clueless how to continue/debug this from here
edit: I had a suspicion that this has something to do with unicode so I changed the sqlalchemy column to unicode.
maybe the DataError is stuck somehow and I need to call a rollback, but I don't know hot to call a rollback in transaction in tg2
edit:the EVENTCODE column in the mssql is:datatype:PK,nvarchar(6), not null
hope this helps
another edit: the insert code (the relevant part)
event = #json with data
ac['EVENTCODE']=event.get('code')#for example u'210602' - from the failing data
...
...
e = Activities(**ac) # this is the class mapped with sqlalchemy to the NOAR_LOADEVENTS in sqlalchemy.
DBSession2.add(e)
transaction.commit()
solved. #beargle comment led me in the right direction. seems the problem strangly enough was trying to insert to nvarchar field with length of 6. seems that a unicode string with a length of 6 like: u'110110' would produce an error while a string '110110' passes fine. so i use
variable.encode('utf-8') and everything work.
what ever.
I jumped into this error and I solved it by changing the definition of a column (DB side) from NVARCHAR(100) to NVARCHAR(200).
I have an oledb connection to mssql and an ado.net destination (with odbc driver used) to mysql. The tables are exectly the same and all the columns are working bar one.
The error message received is:
[ADO NET Destination [325]] Error: An exception has occurred during data insertion, the message returned from the provider is: Unable to cast object of type 'System.DateTime' to type 'System.Char[]'.
I've seen similar questions on other data types but the resolution of changing to string does not work here. If I convert to string (has to be length 29 otherwise the conversion step fails) I get the following error message:
[ADO NET Destination [325]] Error: An exception has occurred during data insertion, the message returned from the provider is: ERROR [HY000] [MySQL][ODBC 5.1 Driver][mysqld-5.5.15]Incorrect datetime value: '2011-03-21 11:23:48.573000000' for column 'LastModificationDate' at row 1
Other potentially relevant details:
connection driver- {MySQL ODBC 5.1 Driver}
script run before dataflow - set sql_mode='STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,ANSI_QUOTES'
Other datetime columns are working
This column has a reasonably high proportion of nulls
mssql spec: [LastModificationDate] [datetime] NULL
mysql spec: LastModificationDate datetime NULL
Has anyone had experience with this issue and could provide some advice on resolving it?
Can you try converting it to string on sql server side in your query using:
convert(char(10),LastModificationDate,111)+' '+convert(char(8),LastModificationDate,108)
This works for me all the time.
I got the same big headache this week. I tried many ways. Thanks God, finnally, one of them worked. Hope it could help you a little bit.
For some columns with the data type of Int, datetime, decimal....,here, I identified as ColumnA, and I used it as datetime type.
1.in Data Flow Source, use SQL Command to retrieve data. Sth like select isnull(ColumnA,'1800-01-01') as ColumnA, C1, C2, ... Cn from Table
Make sure to use Isnull function for all columns with the datatype mentioned before.
2.Excute the SSIS pkg. It should work.
3.Go back to Control Flow, under the data flow task, add SQL Task control to replace the data back. I mean, update the ColumnA from '1800-01-01' to null again.
That works for me. In my situation, I cannot use ignore failure option. Because if I do, I will lose thousands rows of data.
Request Execute is failed if one of fields to be mapped has DateTime field and corresponding value in DB has '0000-00-00' or '0001-01-01'. The following error is returned
Unable to convert MySQL date/time value to System.DateTime
Is there any possibility to fetch such value?
I've tried to specify the 'DateTime?' value as property type - it doesn't help too (actually, I didn't expect that to be helpful).
P.S. I use MySql 5.1
I came across a similar problem using NHibernate with the same error in an exception.
It's due to MySQL's unique "feature" of allowing invalid dates in a DATE field, especially using 0000-00-00 as a default value for DATE NOT NULL columns. When such a date is encountered, it throws an exception when converting itself to a DateTime.
The suggested solution for this was to add
Allow Zero Datetime=True;
to the connection string, however in practice this did not work for me. I eventually solved the problem by altering the connection string adding
Convert Zero DateTime=true;
so your app.config section would look something like this
<connectionStrings>
<add
name="ConnectionString.MySql"
connectionString="Server=localhost;Port=3306;Database=BLT;Uid=someuser;Convert Zero DateTime=true;"
providerName="MySql.Data.MySqlClient"/>
Have you tried the MapValue attribute? I'm not sure if this will work but...
[MapValue(null, "0000-00-00")]
[MapValue(null, "0001-01-01")]
public DateTime? theDate;
i think you have to control it by another property.
[MapField("the_date")]
public DateTime? theDate; // Map
[MapIgnore]
public DateTime theDateControl
{
set {
if(theDate.HasValue)
{
....
}
}
}