I need your help in this issue,
I have a talend job which load data from a table to another with a simple tmap.
I called it a mysterious error because it happended just for a specific datetimes
java.sql.SQLException: Could not parse column as timestamp, was: "2009-06-01 00:00:00"
Thousands of rows before the row containing this line doesn't generate the error
When I modify this date 2009-06-01 00:00:00 to another or just changing the day part or the month or even the hour, It goes without error.
the datasource is a mariadb and the destination is a Mysql database
thnks for your help
and this is the part of code which contain the error generated
if (colQtyInRs_tMysqlInput_5 < 6) {
row5.created_at = null;
} else {
if (rs_tMysqlInput_5.getString(6) != null) {
String dateString_tMysqlInput_5 = rs_tMysqlInput_5
.getString(6);
if (!("0000-00-00")
.equals(dateString_tMysqlInput_5)
&& !("0000-00-00 00:00:00")
.equals(dateString_tMysqlInput_5)) {
row5.created_at = rs_tMysqlInput_5
.getTimestamp(6);
} else {
row5.created_at = (java.util.Date) year0_tMysqlInput_5
.clone();
}
} else {
row5.created_at = null;
}
}
Since you provided no further information in
How the source data looks like, e.g. is it a date field or a string field in the source?
Why parsing would happen, this seems to be connected to the source data being a string
How the parsing pattern looks like
I am going to assume a bit here.
1st: I assume you provide a string in the source. Since this is the case, you'd need to make sure that the date in the column is always formatted the same way. Also, you'd need to show us the timestamp format for parsing.
2nd: You said you'd need to change the values of the date for it to work. This seems to me to be an issue with parsing, so for example you have switched by accident the month and day field, e.g. yyyy-dd-mm HH:mm:ss or something alike. Again, this depends on your parsing string.
Since there is often a bit of confusion about this I created a blog post for date handling in Talend which you could consult as well.
This error is due to Timezone, after trying many solutions, I thought about changing the timezone because My Laptop is in UTC and the database timezone is UTC+01 so Talend generate this error in local environment.
Hope it will help someone else
Related
I have a database table containing dates
(`date` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00').
I'm using MySQL. From the program sometimes data is passed without the date to the database. So, the date value is auto assigned to 0000-00-00 00:00:00
when the table data is called with the date column it gives error
...'0000-00-00 00:00:00' can not be represented as java.sql.Timestamp.......
I tried to pass null value to the date when inserting data, but it gets assign to the current time.
Is there any way I can get the ResultSet without changing the table structure?
You can use this JDBC URL directly in your data source configuration:
jdbc:mysql://yourserver:3306/yourdatabase?zeroDateTimeBehavior=convertToNull
Whether or not the "date" '0000-00-00" is a valid "date" is irrelevant to the question.
"Just change the database" is seldom a viable solution.
Facts:
MySQL allows a date with the value of zeros.
This "feature" enjoys widespread use with other languages.
So, if I "just change the database", thousands of lines of PHP code will break.
Java programmers need to accept the MySQL zero-date and they need to put a zero date back into the database, when other languages rely on this "feature".
A programmer connecting to MySQL needs to handle null and 0000-00-00 as well as valid dates. Changing 0000-00-00 to null is not a viable option, because then you can no longer determine if the date was expected to be 0000-00-00 for writing back to the database.
For 0000-00-00, I suggest checking the date value as a string, then changing it to ("y",1), or ("yyyy-MM-dd",0001-01-01), or into any invalid MySQL date (less than year 1000, iirc). MySQL has another "feature": low dates are automatically converted to 0000-00-00.
I realize my suggestion is a kludge. But so is MySQL's date handling.
And two kludges don't make it right. The fact of the matter is, many programmers will have to handle MySQL zero-dates forever.
Append the following statement to the JDBC-mysql protocol:
?zeroDateTimeBehavior=convertToNull&autoReconnect=true&characterEncoding=UTF-8&characterSetResults=UTF-8
for example:
jdbc:mysql://localhost/infra?zeroDateTimeBehavior=convertToNull&autoReconnect=true&characterEncoding=UTF-8&characterSetResults=UTF-8
Instead of using fake dates like 0000-00-00 00:00:00 or 0001-01-01 00:00:00 (the latter should be accepted as it is a valid date), change your database schema, to allow NULL values.
ALTER TABLE table_name MODIFY COLUMN date TIMESTAMP NULL
As an exteme turnaround, when you cannot do an alter to your date column or to update the values, or while these modifications take place, you can do a select using a case/when.
SELECT CASE ModificationDate WHEN '0000-00-00 00:00:00' THEN '1970-01-01 01:00:00' ELSE ModificationDate END AS ModificationDate FROM Project WHERE projectId=1;
you can try like This
ArrayList<String> dtlst = new ArrayList<String>();
String qry1 = "select dt_tracker from gs";
Statement prepst = conn.createStatement();
ResultSet rst = prepst.executeQuery(qry1);
while(rst.next())
{
String dt = "";
try
{
dt = rst.getDate("dt_tracker")+" "+rst.getTime("dt_tracker");
}
catch(Exception e)
{
dt = "0000-00-00 00:00:00";
}
dtlst.add(dt);
}
I wrestled with this problem and implemented the URL concatenation solution contributed by #Kushan in the accepted answer above. It worked in my local MySql instance. But when I deployed my Play/Scala app to Heroku it no longer would work. Heroku also concatenates several args to the DB URL that they provide users, and this solution, because of Heroku's use concatenation of "?" before their own set of args, will not work. However I found a different solution which seems to work equally well.
SET sql_mode = 'NO_ZERO_DATE';
I put this in my table descriptions and it solved the problem of
'0000-00-00 00:00:00' can not be represented as java.sql.Timestamp
There was no year 0000 and there is no month 00 or day 00. I suggest you try
0001-01-01 00:00:00
While a year 0 has been defined in some standards, it is more likely to be confusing than useful IMHO.
just cast the field as char
Eg: cast(updatedate) as char as updatedate
I know this is going to be a late answer, however here is the most correct answer.
In MySQL database, change your timestamp default value into CURRENT_TIMESTAMP. If you have old records with the fake value, you will have to manually fix them.
You can remove the "not null" property from your column in mysql table if not necessary. when you remove "not null" property no need for "0000-00-00 00:00:00" conversion and problem is gone.
At least worked for me.
I believe this is help full for who are getting this below Exception on to pumping data through logstash
Error: logstash.inputs.jdbc - Exception when executing JDBC query {:exception=>#}
Answer:jdbc:mysql://localhost:3306/database_name?zeroDateTimeBehavior=convertToNull"
or if you are working with mysql
My purpose is since the time I login my page, I want my web to show how many updated data in the database. My code is like this
$current = $_SESSION['date'];
$query2 = "SELECT * FROM gmaptracker1 WHERE datetime >= '$current'";
When I echo the $current, it showed 27/09/14 : 06:53:24, so the $current is correct, however, when I request the number of database where date>='$current', I get zero, although I have inserted to the database the data with datetime 28/09/14 : 06:53:24 and 29/09/14 : 06:53:24.
Can anyone help me to get out of this, please?
Few things,
It seems like your code is vulnerable to SQL Injection. Just because you retrieve the content of the date from a session, it doesn't mean that it's safe.
Also, why do you need it to be in a session variable? If you always want to retrieve dates bigger than NOW() you can just write your query this way:
SELECT * FROM gmaptracker1 WHERE datetime >= NOW()
The part that caught my attention was the format you're storing the dates.
You said that when you echo'ed $_SESSION['date'] the value was: 27/09/14 : 06:53:24
Now, that does not look like the date format at all. Is your column actually a datetime or timestampcolumn?
If it's a VARCHAR or any other type other than datetime or timestamp, then there's no way for MySQL to know that you're trying to retrieve dates that occur in the future.
If you already have data stored, then it isn't going to be as easy as changing the data type because you already have data, and your data is in the wrong format. The format that MySQL stores datetime information is as follows:
YYYY-MM-DD HH:MM:SS
Based on the comments you left, you don't need the time > NOW(), you need the time when you log in. Now it makes sense why you're storing that time in a variable.
The problem is the format you're storing it.
Since you're using PHP, then you have to store the time this way:
$time = new DateTime();
$_SESSION['date'] = $time->format("Y-m-d H:i:s");
In my model, I have a field named timestamp (MySQL type: timestamp) defined in safe validator and I'm unable to write it manually. Each time I call:
$model->timestmap = time();
$model->save();
model is saved (row created / updated), that is -- it passes validation without errors, but timestamp field is filled with default value of 0000-00-00 00:00:00 (I decided to remove default on update CURRENT_TIMESTAMP attribute, I don't want MySQL to handle this).
However, when I remove above code and attach CTimestampBehavior instead, field is being filled with correct values on both update and create, without any problems.
What can be happening or what am I missing? How can behavior update field without problems, while my manual attempt fails? Is this because column type is timestamp, not datetime or int?
I was always told, that first clue, why some attribute isn't saved, is because it is not listed among safe validator list or because it is listed on unsafe validator list. But this one is listed on safe list.
Your database field is a datetime field (i assume, looking at your default value), but your filling it with a unix timestamp. Try it with an CDbExpression instead:
$model->timestamp = new CDbExpression('NOW()');
$model->save();
I usually use $model->timestamp = date('Y-m-d H:i:s'), it works perfectly
As the other answers already suggested:
The timestamp needs to be converted into a MySQL compatible date format string somehow upon saving and the other way around when loading. Now you already discovered that the CTimestampBehavior does this for you but unfortunately it doesn't care about loading.
IMO the best solution for you is something along the way of:
public function beforeSave()
{
$this->timestamp = date('Y-m-d H:i:s', $this->timestamp);
return parent::beforeSave();
}
public function afterSave()
{
// Turn it back into a unix timestamp in case you want to continue working with the record
$this->timestamp = CDateTimeParser::parse($this->timestamp, 'yyyy-MM-dd hh:mm:ss');
parent::afterSave();
}
public function afterFind()
{
$this->timestamp = CDateTimeParser::parse($this->timestamp, 'yyyy-MM-dd hh:mm:ss');
return parent::afterFind();
}
It's a lot of work for a stupid timestamp and so for myself I have an auto type conversion behaviour that I link to my models. This behaviour uses the table metadata to take care of everything automatically. Might be a good idea to invest in that. I've thought about making mine open source but it's a bit of a mess atm ;)
But the code above will give you unix times to work with during executing and whilst saving it will temporary convert into a mysql datetime string
Hope that helps.
In my application am using JPA entity manager to persist data/fetch data.
em.executeQuery("select * from file_calender_mapping where start_date between :start and :end");
em.setParameter("start",startDate)//startDate is an date object
em.setParameter("end",endDate)//endDate is an date object
List fmlist=em.execute();
The proble is just like this,
"select * from file_calender_mapping where start_date between start and end"
when am passing some date as start= "2011-08-03 05:08:00",and end="2011-08-04 06:08:00"
then the mysql return one row having the start time ="2011-08-03 05:30:00",its good,But
when my application executing such query it dose not returning any row.Actually what i have seen that my application returning value for two different date,but not for same date different time,thats the main problem.
One another thing is my "start" field for Table "file_calender_mapping" datatype is "timestamp".
So what i was thinking that ther may be some problem on JPA/Hibernate
You can try to specify the exact types of parameters as follows:
em.setParameter("start", startDate, TemporalType.TIMESTAMP);
em.setParameter("end",endDate, TemporalType.TIMESTAMP);
I have the strong feeling that you're confusing EntityManager.createQuery() with EntityManager.createNativeQuery() and you're somehow capturing all the exceptions, which somehow makes you don't receive anything back.
I'm assuming that, because I don't think you have a class named file_calender_mapping.
edit
The documentation will explain it better than I do, but a JPA QL query is transformed to the navite sql of the DB using the mapping, while a native query is send as it's to the DB.
Again, I suggest you to read the documentation, it's quite useful.
I have a simple table with few date fields.
Whenever I run following query:
var docs = ( from d in base.EntityDataContext.document_reviews
select d ).ToList();
I get following exception:
Unable to convert MySQL date/time value to System.DateTime.
MySql.Data.Types.MySqlConversionException: Unable to convert MySQL date/time value to System.DateTime
The document reviews table has two date/time fields. One of them is nullable.
I have tried placing following in connection string:
Allow Zero Datetime=true;
But I am still getting exception.
Anyone with a solution?
#effkay - if you solved this it would be great if you could post the answer.
Also if anyone else has a solution that would be great too :).
Edit:
The solution can be found in the http://dev.mysql.com/doc/refman/5.1/en/connector-net-connection-options.html connector documentation.
I needed to set "Convert Zero Datetime" to true, and now it works.
hth.
You need to set Convert Zero Datetime=True in connection string of running application