I am trying to create a Json schema regex to have a timestamp validation in milliseconds. Eg.20140508172846.127.
I tried like this Pattern:^[0-9] it's not working. please suggest any other way to perform timestamp validation.
You should use a regex. The problem is the pattern you provided does not match floating point numbers.
You could try with:
(\d+(?:\.\d*)?|\.\d+)\s+(\S+)
But you would need to specify exactly which timestamps values are allowed (and you will find for sure an answer in stackoverflow)
Related
I have a large dataset with employees' time entries. The current date format is MM/dd/yyyy. However, I need to convert all the dates into yyyy-MM-dd format.
I have tried the following:
Update human_resources.timekeeping
Set Actual_Date = str_to_date(Actual_Date,'%d-%m-%Y');
Got the errror messsage Error Code: 1411. Incorrect datetime value: '' for function str_to_date.
My SQL version is 5.7.18-log.
I tried to view SQL mode using SELECT ##sql_mode; and I got NO ENGINE SUBSTITUTION.
I have tried to retrieve the value like shown below and it was working fine.
Converting varchar mm/dd/yy to date format yyyy-mm-dd
However, updating the data would not work. I need to update the actual records, not insert new records.
Hope someone can help me regarding this. Thank you in advance!
EDIT: The data type for Actual_Date is VARCHAR.
Apologies if my explanation may be a bit confusing. But I am using this data set to display and filter time entries in a gridview. When I am filtering dates, say for example (01/15/2022-01/25/2022), data from 2021 is also being displayed. When I tried to manually change the format of some of my data in sql to yyyy-MM-dd, my code seemed to be working fine. The problem is there are a lot of data in this table, which is why manually updating the format is impossible. What is the first thing that I need to do? I'm sorry this is all still a bit confusing for me.
My apologies if you have already taken the following things into consideration but I thought them worth mentioning.
Given that you say this is a "large dataset" I assume this is a table that is currently in use. Does the existing application rely on the Actual_Date being in that string format? Does it rely on a fixed number of columns in the table? Some poorly written applications can be very brittle when it comes to changing underlying data structure.
You may want to consider creating a copy of the current table, modifying the structure of the copy, and replacing the original with a view with the same columns and formats as the original. This way you get improved data but reduce risk to existing application.
In the title and first line of your question you state that the current format is MM/dd/yyyy
Update human_resources.timekeeping Set Actual_Date = str_to_date(Actual_Date,'%m/%d/%Y');
Your separator is / not -
%d-%m-%Y >> %d/%m/%Y
I'm trying to query this API, filtering by the RptDt field, which is type esriFieldTypeDate.
The basic query looks like this:
https://dhsgis.wi.gov/server/rest/services/DHS_COVID19/COVID19_WI_V2/MapServer/11/query?where=1%3D1&outFields=*&outSR=4326&f=json
It's easy to filter a numeric variable like POS_CUM_SUM like so:
"https://dhsgis.wi.gov/server/rest/services/DHS_COVID19/COVID19_WI_V2/MapServer/11/query?where=%20(POS_CUM_CP%20%3D%200%20OR%20POS_CUM_CP%20%3D%2010)%20&outFields=*&outSR=4326&f=json"
I can't figure out how to format the minimum and maximum arguments for the date field RptDt.
because the attributes of RptDt are formatted as unix timestamps: 1642255200000, 1642428000000. But that returns error code 400.
https://dhsgis.wi.gov/server/rest/services/DHS_COVID19/COVID19_WI_V2/MapServer/11/query?where=%20(RptDt%20%3D%20'1642255200000'%20OR%20RptDt%20%3D%20'1642428000000')%20&outFields=*&outSR=4326&f=json
Then, I noticed that the field length for RptDt is 8, so I tried rounding the unix timestamp to 8 digits (16422552, 16424280), but that also gives error code 400.
I tried using YYYYMMDD format (20210101 to 20211231), this didn't give an error, but the response has no features. Most of the dates do fall this period.
https://dhsgis.wi.gov/server/rest/services/DHS_COVID19/COVID19_WI_V2/MapServer/11/query?where=%20(RptDt%20%3D%20%2720210101%27%20OR%20RptDt%20%3D%20%2720211231%27)%20&outFields=*&outSR=4326&f=json
I haven't been able to find a solution in the ArcGIS REST APIs documentation. Does anyone understand what's missing from my queries?
The correct format is 'YYYY-MM-DD'. This query works. The relevant bit is where=RptDt>'2022-01-01'.
https://dhsgis.wi.gov/server/rest/services/DHS_COVID19/COVID19_WI_V2/MapServer/11/query?where=RptDt>'2022-01-01'&outFields=*&outSR=4326&f=json
Thanks to JamieKelly1 at the Esri Community Forum for answering this question.
For anyone else stumbling upon this one after getting the -2147220985 (which btw means bad SQL query), the full url I used had to contain Date before the actual date like so Date'2015-02-02' not just '2015-02-02'. Also, you cannot use epoch timestamp in miliseconds in the query builder (at least not for open data dc).
Example url:
https://maps2.dcgis.dc.gov/dcgis/rest/services/DCGIS_DATA/ServiceRequests/MapServer/6/query?where=%20(ADDDATE%20%3D%20Date'2015-02-02'%20OR%20ADDDATE%20%3D%20Date'2015-02-03')%20&outFields=*&outSR=4326&f=json
To search between two dates:
https://maps2.dcgis.dc.gov/dcgis/rest/services/DCGIS_DATA/ServiceRequests/MapServer/6/query?where=ADDDATE BETWEEN DATE '2015-01-09 00:00:00' AND DATE '2015-01-09 12:00:00'&outFields=*&outSR=4326&f=json
We use presto JSON capabilities quit heavily and on thing the is missing for us is to be able to null when JSON is not a valid on this way SQL statement that use JSON functions will not break if there is a problem with the JSON format.
Initially I thought it can be done with some combination of JSON_PARSE and NULLIF but couldn't manage to pull this of..
is there a way to do make this kind of validation?
Thanks
You can use the try function to prevent the json functions from failing the query. For example, SELECT try(json_parse('bad json')) will return null instead of failing the query.
I am doing a series of updates on some tables after I import them from tab-separated values. The data comes with dates in a format I do not like. I bring them in as strings, manipulate them so that they are in the same format as MySQL dates and then convert the column. Or sometimes not, but I want them to be like MySQL dates even if they are strings.
They start out like '1/4/2013 12:00:00 AM' or '11/4/2012 2:37:45 PM'.
I turn these into '2013-01-04' (usually, since times are present even when the original schema clearly specifies dates only) and '2012-11-04 14:37:45'.
I am using rlike. And this does not use indexes? Wow. That sucks.
But already, for each column, I have to use 4 updates to handle the different cases ('1/7', '2/13', '11/2', '12/24'). If I did these using like, it might take 16 different updates for each column....
And, if I am seeing it right, I cannot even get positional parameters out of the rlike expression, yes? You know, the part of the expression wrapped in parentheses that becomes $1 or $2....
So, it seems as though it is going to be quicker to pre-process the tsv file with perl. Really? Wow. Again, this sucks.
Any other suggestions? I cannot have this taking 3 hours every time I need to pull in the data.
Recall the classic 1997 quote from Jamie Zawinski:
Some people, when confronted with a problem, think "I know, I'll use regular expressions."
Now they have two problems.
Have you tried using STR_TO_DATE()? This is exactly for parsing nonstandard date/time strings into canonical datetime values.
If you try parsing with STR_TO_DATE() and the string doesn't match the expected format, the function returns NULL.
So you could try parsing in different formats, and return the first one that gives a non-null result.
UPDATE mytable
SET datecolumn = COALESCE(
STR_TO_DATE(stringcolumn, '%m/%d'),
STR_TO_DATE(stringcolumn, '%d/%m/%Y'),
...etc.
);
I can't tell what your different cases are. It might or might not be possible to cover all cases in one pass.
Another alternative is as you say, preprocess the raw data with Perl before you load it into MySQL. But even then, don't fight with regular expressions, use Date::Parse instead.
I have been working with Symfony2 and doctrine2 recently and have realized a peculiar datatype called DC2Type:array that certain Symfony2 Roles get saved as. To me it just looks like a serialized PHP array where a signifies the total number of elements, i is the array index.
The value looks like this:
a:15:{i:0;s:32:"ROLE_SONATA_USER_ADMIN_USER_EDIT";i:1;s:32:"ROLE_SONATA_USER_ADMIN_USER_LIST";i:2;s:34:"ROLE_SONATA_USER_ADMIN_USER_CREATE";i:3;s:32:"ROLE_SONATA_USER_ADMIN_USER_VIEW";i:4;s:34:"ROLE_SONATA_USER_ADMIN_USER_DELETE";i:5;s:36:"ROLE_SONATA_USER_ADMIN_USER_OPERATOR";i:6;s:34:"ROLE_SONATA_USER_ADMIN_USER_MASTER";i:7;s:33:"ROLE_SONATA_USER_ADMIN_GROUP_EDIT";i:8;s:33:"ROLE_SONATA_USER_ADMIN_GROUP_LIST";i:9;s:35:"ROLE_SONATA_USER_ADMIN_GROUP_CREATE";i:10;s:33:"ROLE_SONATA_USER_ADMIN_GROUP_VIEW";i:11;s:35:"ROLE_SONATA_USER_ADMIN_GROUP_DELETE";i:12;s:37:"ROLE_SONATA_USER_ADMIN_GROUP_OPERATOR";i:13;s:35:"ROLE_SONATA_USER_ADMIN_GROUP_MASTER";i:14;s:10:"ROLE_ADMIN";}
I want to know what this datatype is?
And what do the following identifier signifies:
s:
I have searched the internet but haven't got any useful data.
I also bumped upon this cookbook entry - http://readthedocs.org/docs/doctrine-orm/en/2.0.x/cookbook/mysql-enums.html but didn't figure out the origin.
This is not a data type. You might have noticed that the column type is LONGTEXT. DC2Type:array is a comment of the field.
Doctrine uses the field's comment as column's metadata storage place. Since Mysql does not allow you to store an array, Doctrine use DC2Type:array as comment in order to know how to unserialize the content.
Take a look at the link below.
https://github.com/doctrine/dbal/issues/1614
From the link you mentioned, you can see that the comment DC2Type:enumvisibility indicates that the content of the field is a flag, indicating that the record is visible or not. It is not a new data type at all. It should be considered an helper strategy in the database level. For Doctrine, it's a custom data type.
This is simply a string. Its format is a serialized PHP array. The s: refers to the size or length of each item value in the array.
e.g. s:32:"ROLE_SONATA_USER_ADMIN_USER_EDIT"
If you count the characters in the ROLE string, there are 32.
Hope this helps.