Dealing with different datetime formats in the DB? - mysql

I'm writing a Ruby program using Sequel which is running on the legacy database. There is an issue dealing with different date time formats.
DB has a table which has a column start_date. In Sequel's migration script I set it to DateTime which is a timestamp type in SQLite, however, the legacy data has a different time format:
Some are using an ISO8601, like 2013-09-01T08:28:00+10:00.
Some are using a different one, which I don't know if it has a name, like 2013-09-01 08:28:00.000000+1000.
The problem is, when I run a query against the table and try to filter by start_date, the difference between two date time formats will cause incorrect results.
The query I'm using is:
current = Time.now
MyModel.where { start_date < current }
Sequel will convert it into SQL like this:
SELECT * FROM `my_model` WHERE `start_date` < '2013-09-01 08:28:00.000000'
From my local testing, Sequel looks like it's comparing the date as a string, so 2013-09-01 08:28:00.000000+1000 is less than 2013-09-01T01:28:00+10:00. Because whitespace is less than T this is not what I want.
I could use an iso8601 time like:
current_iso8601 = Time.now.iso8601
MyModel.where { start_date < current_iso8601 }
But it won't solve the problem because the database has two different datetime formats.
My questions are:
Does Ruby/Sequel support querying the database by Date/Time not as a string?
Does it work for different date time formats?
SQLite is just for local testing, in production it will be MySQL. So, the solution should be using general Sequel methods as a adaptor and should not have any database specific methods.
NOTE: the program is not a Rails application.
Thank you very much!

SQLite does not have date/time types (see http://sqlite.org/datatype3.html). It stores datetime values as strings.
The best solution is to use the same database in development/testing that you use in production. If you don't want to do that, you need to convert all the SQLite datetime values so that they all use the same ISO8601 format. That way the comparison operators will work correctly (as they do in MySQL).

Related

How to export data from Cloud SQL to BigQuery on a daily basis?

I have created a connection to Cloud SQL and used EXTERNAL_QUERY() to export the data to Bigquery. My problem is that I do not know a computationally efficient way to export a new days data since the Cloud SQL table is not partitioned; however, it does have a date column date_field but it is of the datatype char.
I have tried running the following query with the view of scheduling a similar type so that it inserts the results: SELECT * FROM EXTERNAL_QUERY("connection", "SELECT period FROM table where date_field = cast(current_date() as char);") but it takes very long to run, whereas: SELECT * FROM EXTERNAL_QUERY("connection", "SELECT period FROM table where date_field = '2020-03-20';") is almost instant.
Firstly, it’s highly recommended to convert the ‘date_field’ column to the datatype DATE. This would improve simplicity and performance in the future.
When comparing two strings, MySQL will make use of indexes to speed up the queries. This is executed successfully when defining the string as ‘2020-03-20’ for example. When casting the current date to a string, it’s possible that the characters set used in the comparison aren’t the same, so indexes can’t be used.
You may want to check the characters set once current_datetime has been casted compared to the values in the ‘date_field’ column. You could then use this command instead of cast:
CONVERT(current_date() USING enter_char_sets_here)
Here is the documentation for the different casting functions.

Proper way to use datetime in Codeigniter queries

I wonder if there is some type of common method that would help me write query with date/time field in it. For example: I am developing a very small project utilizing MySQL database. However, my client is considering switching to his existing SQL server.
Example (datetime column):
SELECT DATE_FORMAT(contract_date, '%d.%m.%Y') FROM `employees`
Question: Can query below become usable in SQL in case I replace database driver (currently) mysqli to sqlsrv?
I understand I can use some type of config variable for date format... Would it be the best way? Is there something that Codeigniter 3 has in place?
feel free to use your own query sample

Create datafield with gmt-timezone timestamp in mysql?

I'm trying to convert a postgresql sql-query to mysql. Using a translator.
this is the query in postgres:
comment_date_gmt timestamp without time zone DEFAULT timezone('gmt'::text, now()) NOT NULL,
it's converted to
comment_date_gmt timestamp DEFAULT timezone('gmt',
The none-closed parenthesis is a sign that everything isn't right. I'm trying to figure out what this query should look like. Any suggestions?
The only reliable SQL query dialect converter is the human brain.
Tools can be useful for the basics, like data type renaming, but lots of that sort of thing can be avoided by just writing the queries using standard types in the first place.
You'll have a very hard time converting a MySQL query that uses query variables to a PostgreSQL query, or converting a PostgreSQL (well, SQL-standard) recursive common table expression to something MySQL understands. The two have totally different stored procedure languages, different built-in functions, and all sorts of things. array_agg, unnest, etc ... most of that stuff would require translation to queries using MySQL variables where it's possible to do it at all. Then you've got window functions like row_number, lead, lag, and aggregates used as running windows like sum(blah) OVER (...). A generic converter would need to "understand" the query to actually do the job.
A specific answer for the named problem isn't really possible since you haven't identified the converter tool.
At a guess, if you change the PostgreSQL query to:
comment_date_gmt timestamp without time zone DEFAULT (current_timestamp AT TIME ZONE 'utc') NOT NULL,
which is the standard phrasing understood by PostgreSQL and other compliant databases.

Problem with Date field after migration from mysql to oracle

I have lately migrate my JIRA database from mysql to oracle,
my problem is the field "created" exists in the jiraissue and changegroup tables,on this field I effectuate many calculation but I was surprised by the difference of the format of the fields.
in mysql database the field creation has the type timeStamp so it has the follwing format:
and in Oracle database it has the type date and the format like the following:
How can I resolve this problem?
The format of your displayed Oracle DATE column is due to your IDE that you are viewing it through.
Oracle stores all portions of a date, to display the full date stored use this:
SELECT TO_CHAR(created, 'DD-MON-YYYY HH24:MI:SS')
FROM jiraissue;
This will show you the full date that has been stored including the time portion. To store timestamps you need the column to be designated as a timestamp datatype column.
Ollie.
EDIT: You could change the NLS Date Format of your IDE to always show the full date format in it's settings somewhere.
Dates are not held as formatted text in Oracle. What you are seeing is a tool's (Toad's?) default formatting of the date to display it to you. This can be changed via a preference somewhere. It could be that the time component has been lost in migration, but that is unlikely. Try running this SQL to see:
select to_char (created, 'YYYY-MM-DD HH24:MI:SS') from jiraissue;
That should show the dates just as they appeared in MySQL.

Ruby convert Mysql timestamp to Mysql datetime

I am trying to import data from a DB to another. The source DB has TIMESTAMP (mysql) the destination has DATETIME (mysql). I am trying something like that:
start_at = DateTime.at(row['QA_CONF_START_STAMP']) #start_at
But it is not working
I'm not sure if conversion is expressly required in this case, as the two values should be equivalent.
Since you're not retrieving the original data using a model, it's coming through as a raw string. The easiest way to interpret that is:
DateTime.parse(row['QA_CONF_START_STAMP'])