Setup :
Angular 8 + Spring boot 2.2.3 + (Oracle 12.1 / PostgreSQL 12.3)
We are building a approval System where User can fill online form like google forms and can submit for approval. Rather than normalizing form structure , we'll be storing metadata in JSON format in our DB.
Values that are filled in form would also be going as JSON format in DB.
One point come up as a concern , in DB we can store date in particular format like 12-May-2020 which would be consistent across all inserted data as this data might be used to construct reports in near future.
Based on pros/cons of this approach need to decide on DB / data model as well.
So,
Is there any way I can enforce date format in JSON
If this cannot be done in JSON , what options do i have at Angular 8 / Spring boot application level which would enforce all developers / date components / date fields to use same date format.
If these cannot be done , how can I handle different formats in Query over JSON data that would be used in reporting or otherwise , both in Oracle and PostgreSQL
Regards
The proper solution to your problem is to create a real, normalized date column.
If for some reason you can't or don't want to do that, I would create a check constraint that validates the date format by trying to cast it to a real date value.
The following is for Postgres, but you can create something similar for Oracle as well:
create table the_table
(
form_data jsonb,
constraint check_valid_date check ( (form_data ->> 'entry_date')::date is not null)
);
Obviously you will need to adjust the expression that gets the date value from the JSON to match the key and path inside your json value.
The cast to date will require that the date value is entered using the ISO standard format, yyyy-mm-dd which is the only "consistent" way to store a date as a string.
Alternatively you can use to_date() with a format mask:
check ( to_date(form_data ->> 'entry_date', 'yyyy-mm-dd) is not null)
in DB we can store date in particular format like 12-May-2020
You are mistaken, Oracle doesn't store date in that format. It is internally stored in TYPE12/13 data type. Each bit represents different parts of the date. What you see is a human readable format displayed according to your locale-specific NLS settings or using TO_CHAR with format mask.
To keep it aligned across all platforms, use the globally accepted ANSI standard date literal which uses a fixed format 'YYYY-MM-DD'.
For example:
DATE '2020-05-21'
Related
I have a table where users used to enter random date formats like
dd/MM/Year or MM/dd/Year.
But I want to update all those dates to this format Year/MM/dd
I have made a cursor and changed data by CHAR INDEX and sub string..
but is there any easy way to change the date format to the one i want ?
and also how can I make my default database date format = Year/mm/dd
only database not server?
You can convert it into different styles.
for example:
select convert(varchar(15),getdate(),103)
will give today's date in dd/mm/yyyy.
Full list of converting styles supported by MSSQL is here
I have a MySql table with a String 2014-02-21 16:53:01 stored in a varchar column. I need to use Talend Data Integration to convert it into a date for calculations in the format dd/mm/yyyy hh:mm:ss and to store it in that format in another MySql as a date column after the calculations.
I have a talend tmap component but I get parsing errors and not sure how to solve it and the general sequence of steps needed.
In the tmap I have a variable with the expression
row1.date !=null ? TalendDate.parseDate("dd/mm/yyyy hh:mm:ss",row1.date):null
Which I hope will load the variable wiIs that the best way to
Convert the string in the source table to a date in the format I need?
How do you handle null dates as I need to get the difference between 2 dates but handle the situation where 1 or both may be blank or null
How do I get the destination table to store the date in the format I need? I have the Date Pattern supplied in the destination tmap schema but is that enough if the date format is yyyy-mm-dd ?
Any Talend experts able to help a novice out?
For better control on null check I recommend you to use below function.
!Relational.ISNULL(row1.date) && !"".equalsIgnoreCase(row1.date) ?TalendDate.parseDate("MM/dd/yyyy HH:mm",row1.date):(null or default date);
if you are doing any operation on date? then don`t supply null as a result because you may get error on next component.
always pass default value so that you can check in next component and ignore it or store it based on business need.
You can try below condition which will take care of blank string also;-
row1.date !=null && !"".equalsIgnoreCase(row1.date)? TalendDate.parseDate("MM/dd/yyyy HH:mm",row1.date) : null
Thanks for the help. I decided to configure the schema for the database input to identify the dates as the date datatype instead of string which got Talend to do the work of converting
The pointers on validating the date were useful as well. Thanks for the help UmeshR
A silly question maybe but I wanted clarification. I've created a script that has a date parameter like so:
DECLARE #dateparam as datetime
SET #dateparam = '01-01-2013 00:00:00'
This looks like it is working when I test it even if the date string is not in "correct" format yyyy-MM-dd hh:mm:ss. I changed my computer regional settings to English and the script still did what it was supposed to do.
Is this because of SQL Server 2008 R2 that I have in my computer that it knows how to convert the date or can I ran into trouble with using a dateformat like I have used?
Converting 01-01-2013 won't expose issues such as which 01 is the month, and which is the day.
It's not a safe format.
The safe formats (for converting to datetime, rather than to datetime2) are:
YYYYMMDD 20121201
YYYY-MM-DD'T'hh:mm:ss 2012-12-01T10:43:29
YYYY-MM-DD'T'hh:mm:ss.mil 2012-12-01T10:43:29.337
Stick to those and only those. (The examples all represent the 1st December 2012)
Or, better yet, don't treat dates as strings at all, if you can avoid it. If you're, for example, calling SQL Server from .NET code, keep that dates as DateTimes in your code, and let ADO.NET and SQL Server deal with any required translations to make them become datetimes - without translating them to and from strings.
You're making an implicit conversion from something that looks like a date, but inf fact is a string ( '01-01-2013 00:00:00'). Rather than trusting on SQL Server to make the correct guess in what format the string is in, you should make the conversion explicit by specifying the format.
This can be done by using CONVERT (not CAST) and specify a 'style'. The different styles are listed here: http://msdn.microsoft.com/en-us/library/ms187928.aspx.
If I have a date such as mm/dd/yyyy. How can I get mysql to actually store the date in that format. Is this possible or do have to store it in the form of yyyy/mm/dd and convert it later?
What I want to do is insert a date in the mm/dd/yyyy format but the database will not allow it. It wants yyyy/mm/dd
What is your reason for doing this? I can't see any reasonable use for it.
You cannot change the way MySQL stores dates, no. But you can of course format them when reading/writing them. You can do it in SQL query like this:
For example you can use STR_TO_DATE function to format the date when inserting it:
INSERT INTO mytable (mydate) VALUES (STR_TO_DATE('12/31/2009', '%m/%d/%Y'))
And vice versa:
SELECT DATE_FORMAT(mydate, '%m/%d/%Y') FROM mytable /* returns 12/31/2009 */
But as FactalizeR pointed out, it is not a good practice to do it in the query and it should be moved to script, like this (considering you are using PHP).
$date = '12/31/2009';
$date = date('Y-m-d', strtotime($date));
mysql_query("INSERT INTO mytable (mydate) VALUES ({$date})");
And vice versa
$date = mysql_result(mysql_query("SELECT mydate FROM mytable"), 0, 0);
$date = date('m/d/Y', strtotime($date)); //returns 12/31/2009
There is a built-in DATE type in MySQL. It allows to store a date. And later in your programming language you extract this date and convert to whatever format you like. Also, conversion can be done directly in MySQL via DATE_FORMAT(date,format) function.
MySQL DATE fields must store the date in YYYY-MM-DD (including dashes) format. If you attempt to store it any other way you will have problems with date comparisons and ordering.
Conversion later is a trivial task. Is there a compelling reason why you are trying to avoid doing this? What other technology are you using to talk to the MySQL database? (i.e. PHP, C# etc...)
I think you're mixing up what you are storing - the date itself - with how that date is subsequently referred to.
Why do you want to store in a particular format? Do you want to insert in that format? In which case you might be able to get away with it, depending on the localisation of your install, but otherwise convert - insert would be the way to do it (depending, of course, on how you're inserting).
Are you creating a SQL command from strings?
I have a csv file that has a date field in a format like (among other fields):
17DEC2009
When I do a mysqlimport, the other fields are imported properly, but this field remains 0000-00-00 00:00:00
How can I import this date properly? Do I have to run a sed/awk command on the file first to put it into a proper format? If so, what would that be like? Does the fact that the month is spelled out instead of a number matter?
STR_TO_DATE() enables you to convert a string to a proper DATE within the query. It expects the date string, and a format string.
Check the examples in the manual entry to figure out the correct format.
I think it should be along the lines of %d%b%Y (However the %b is supposed to produce Strings like Dec instead of DEC so you will have to try out whether it works).
I had this issue in the past. What I had to do was to utilize LOAD DATA and set the appropriate expression here -
[SET col_name = expr,...]
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Here is the approach I took to solve similar problem. My use case was bit complex with so many columns, but making here simple to present the solution.
I have Persons table with (Id int autogen, name varchar(100),DOB date), and few million of data(name,DOB) needs to be populated from CSV file.
Created additional column in persons table with name like (varchar_DOB varchar(25)).
Imported data using mysqlimport utility into columns(name,varchar_DOB).
Executed update query that updated DOB column using str_to_date(varchar_DOB,'format') function.
Now, I have expected data populated DOB column.
The same logic could be applied in doing even other kind of data formatting like double,time_stamp etc.