I have a getdate() field and I want to convert it into 20210211T172650Z this format how do I do it in SSIS expression?
In SSIS, we have data types for strings, numbers and dates. Dates have no format and when it is converted to a string value, you're getting whatever format the localization rules dictate.
If you have a particular format you want, then you need to control that and the only way you can control it, is by using a string data type.
The pattern we're going to use here, for each element,
extract the digit(s)
convert the digits to string
left pad/prepend a leading zero
extract the last 2 characters from our string
When we extract digits, they're numbers and numbers don't have leading zeroes. We convert to string which will allow us to then add the character zero in front of it because we're just concatenating strings. If the number was less than 10, then this prepending of a zero will result in exactly what we want. 9 -> 09 If it was greater than 9, then we have an extraneous value in there. 11 -> 011. We don't care that we went too big because we're then going to take the right 2 most characters making 09 -> 09 and 011 -> 11. This is the shortest logic to making a leading zero string in SSIS.
Using that logic, we're going to create a variable for each element of our formatted string: year, month, day, hour, minute, second.
What's the starting date?
I created a variable called StartDate of type DateTime and hard coded it to a starting point. This is going to allow me to test various conditions. If I used getdate, then I'd either have to adjust my computer's clock to ensure my code works on 2001-01-01 at 01:01:01 as well as 2021-12-31 at 23:59:59. When you're satisfied your code passes all the tests, you can then specify that StartDate property EvaluateAsExpression is True and then use GetDate(). But I wouldn't use GetDate().
GetDate is going to evaluate every time you inspect it. When your package starts, it will show 2021-02-12 # 11:16 AM But your package takes 5 minutes to run, so when you go to re-use the value that is built on GetDate, you will now get 2021-02-12 # 11:21 AM.
In your case, those keys won't match if you send it more than once to your Amazon thing. Instead, use a System scoped variable like #[System::StartTime] That is updated to the time the package starts executing and remains constant for the duration of the SSIS package execution. So when you're satisfied the expression you've build matches the business rules, then change #[User::StartDate] over to use #[System::StartTime]. It provides the updated time but without the challenges of drifting time.
Extract the digit(s)
The SSIS expression language has YEAR, MONTH and DAY defined but no shorthand methods for time components. But, it does have the DATEPART function in which you can ask for any named date part. I'm going to use that for all of my access methods as it makes it nice and consistent.
As an example, this is how I get the Hour. String literal HOUR and we use our variable
DATEPART("HOUR",#[User::StartDate])
Convert the digits to string
The previous step gave us a number but we've got that leading zero problem to solve so convert that to a string
(DT_WSTR, 2)DATEPART("HOUR",#[User::StartDate])
Cast to string, two characters wide max, the number we generated
left pad/prepend a leading zero
String concatenation is the + operator and since we can't concatenate a string to a number, we make sure we have the correct operand types on both sides
"0" + (DT_WSTR, 2)DATEPART("HOUR",#[User::StartDate])
extract the last 2 characters from our string
Since we might have a 2 or 3 character string at this point, we're going to use the RIGHT function to only get the last N characters.
RIGHT("0" + (DT_WSTR, 2)DATEPART("HOUR",#[User::StartDate]), 2)
Final concatenation
Now that we have our happy little variables and we've checked our boundary conditions, the only thing left is to make one last variable, DateAsISO8601 type of string, EvaulateAsExpression = True
#[User::Year] + #[User::Month] +#[User::Day] + "T" +#[User::Hour] +#[User::Minute] +#[User::Second] + "Z"
Related
I have two fields, both have the size set to double in the table properties. When I subtract one field from the other some of the results are displayed as scientific notation when I click in the cell and others just show regular standard format to decimal places.
The data in both fields was updated with Round([Field01],2) and Round([Filed2],2) so the numbers in the fields should not be any longer than 2 decimal places.
Here's an example:
Field1 = 7.01
Field2 = 7.00
But when I subtract Field1 from Field2 the access display shows 0.01 but when I click on the result it displays, -9.99999999999979E-03. So of course, when I try to filter on all results that have 0.01 the query comes back empty because it thinks the result is -9.99999999999979E-03.
Even stranger is if Field1 = 1.02 and Field2 = 1.00, the result is 0.02 and when I click on the result the display still shows 0.02 and I can filter on all results that equal 0.02.
Why would MS Access treat numbers in the same query differently? Why is it displaying in Scientific Notation and not filtering?
Thanks for any support.
Take this simple code in Access (or even Excel) and run it!
Public Sub TestAdd()
Dim MyNumber As Single
Dim I As Integer
For I = 1 To 10
MyNumber = MyNumber + 1.01
Debug.Print MyNumber
Next I
End Sub
Here is the output of the above:
1.01
2.02
3.03
4.04
5.05
6.06
7.070001
8.080001
9.090001
10.1
You can see that after just 7 additions rounding is occurring!
Note how after JUST 7 simple little additions Access is now spitting out wrong numbers and has rounding errors!
More amazing? The above code runs the SAME in Excel!
Ok, I am sure I have your attention now!
If I recall, the FIRST day and first class in computing science? Computers don't store exact numbers when using floating point numbers.
So, then how is it possible that the WHOLE business community using Excel, or Access, or in fact your desktop calculator not come crashing down?
You mean Access cannot add up 7 simple little numbers without having errors?
How can I even do payroll then?
The basic concept and ALL you need to know here is that computers store real (floating) numbers only as approximate.
And integer values are stored exact.
so, there are several approaches here, and in fact if you writing ANY business software that needs to work with money values? And not suffer rounding errors?
Then you better off to choose what we called some kind of "scaled" integer. Behind the scenes, the computer does NOT use floating numbers, but uses a integer value, and the also has a "decimal" position.
In fact, in a lot of older business BASIC languages, or others? We often had to do the scaling on our own. (so, we would choose a large integer format). In fact, this "scaling" feature still exists in Access!!! (and you see it in the format options).
So, two choices here. If you don't want "tiny" rounding errors, then use "currency" data type. This may, or may not be sufficient for you, since it only allows a max of 4 decimal places. But in most cases, it should suffice. And if you need "more" decimal places, then you can multiply the values by 1000, and then divide by 1000 when done the calculations.
however, try changing the column type to currency and that should work. (this type of data is how your desktop calculator also works - and thus you not see funny rounding errors as a result (in most cases).
but, the FIRST rule of the day? First computer course?
Computers do not store exact numbers for floating point numbers - they are approximations, and are subject to rounding errors. Now, if you really are using double for the table, then I don't think these rounding errors should show up - since you have "so many decimal places" available.
But, I would try using currency data type - it is a scaled integer, or so called packed decimal.
You can ALSO choose to use a packed decimal in Access, and it supports out to 28 digits, and you can set the "scale" (the decimal point location). However, since you can't declare a decimal type in VBA, then I would suggest that in the table (and in VBA code, use currency data types).
If you need more then 4 decimal points, then consider scaling the currency in your code, or perhaps at that point, you consider using a packed decimal type in the table, but values in VBA will have to use the "variant" type, and they will correctly take on the data column setting if used in code and assigned a value from the table(s) in question.
Needless to say, the first day you start dealing with computers, and that first day ANYTHING beyond being a "end user"? Well, this is your first lesson of the day!
"The data in both fields was updated with Round([Field01],2) and Round([Filed2],2) so the numbers in the fields should not be any longer than 2 decimal places." instead of rounding up(which i think is the reason for the scientific notation) you can use number field as data type , then under field size choose double, then under decimal places choose 2.
Question: how do i remove the T in:
Question 2 : How do i add seconds
<input type="datetime-local"/>
Output: 1985-04-12T23:20:50
I want to remove the T and add seconds to the end 00:00:00
You cannot, as long as the content should stay legal for a <input type="datetime-local">. As per 4.10.5.1.12, the contents of a local date and time input element must be parseable:
4.10.5.1.12. Local Date and Time state (type=datetime-local)
When an input element’s type attribute is in the Local Date and Time state, the rules in this section apply.
The input element represents a control for setting the element’s value to a string representing a Local Date and Time, with no time-zone offset information.
If the element is mutable, the user agent should allow the user to change the Date and Time represented by its value, as obtained by parsing a date and time from it. User agents must not allow the user to set the value to a non-empty string that is not a valid normalized global date and time string.
A normalized local date and time always includes a T:
2.4.5.7. Global dates and times
…
A string is a valid normalized global date and time string representing a date, time, and a time-zone offset if it consists of the following components in the given order:
A valid date string representing the date converted to the UTC time zone
A U+0054 LATIN CAPITAL LETTER T character (T)
A valid time string representing the time converted to the UTC time zone and expressed as the shortest possible string for the given time (e.g., omitting the seconds component entirely if the given time is zero seconds past the minute)
A U+005A LATIN CAPITAL LETTER Z character (Z)
Therefore, you cannot expect your browser to remove the additional information. You can, however, copy the information to a hidden <input> tag with JavaScript and send that instead. Alternatively, have the server remove the superfluous information.
Keep in mind that a normalized time is really helpful, though, as local times without timezone information are ambigous.
Although you should save the reference to the value provided, because it is a standardize method of saving dates and times, if you need to modify it for displaying purposes, you can do this...
var modified_value = dateString.replace('T', ' ') + ' seconds';
Where dateString is a variable with the output of your input field.
I am having trouble with a sequence of code that is not reading the NVARCHAR length of my variables (they are barcode strings). We have two different barcodes and the inventory system I have set up measures only the format of the original one (has 7 characters). The new barcode has 9 characters. I need to run a loop value through each barcode input, hence how I have set up this line of script.
I originally thought that a DATALENGTH or LEN function would suffice but it seems that it is only measuring the variable as an integer, not the 7 characters in the string. If anybody has any input of how to manipulate my code sequence or a function that will measure a variables nvarchar length, it would more than appreciated!
CASE WHEN #BarcodeID = LEN(7)
THEN UPPER(LEFT(#BarcodeID,2))+CONVERT(nvarchar,RIGHT(#BarcodeID,5)+#LoopValue-1)
ELSE UPPER(LEFT(#BarcodeID,3))+CONVERT(nvarchar,RIGHT(#BarcodeID,6)+#LoopValue-1) END
Once again, the LEN(7) function in the beginning seems to be my issue.
Perhaps what you're trying to do is actually
CASE WHEN LEN(#BarcodeID) = 7
By using #BarcodeID = LEN(7) you are basically testing to see if the #BarcodeID variable is equal to 1 because the LEN() function, "Returns the number of characters of the specified string expression." It is implicitly converting 7 to a one-character string.
I am using an external program to run a simulation which returns to me a csv file containing output data. I need to read the data from this file into my fortran program, which analyses and optimizes the input conditions to rerun the external program.
The CSV file has say 20 columns and 70 rows. Each column contains output data for a specific parameter. Now since that program is not written by me, I cannot control the precision of the output values. So in many cases the external program truncates the number of digits after the decimal it they are zero. So it is possible in run number 1, a certain field has 3 digits after the decimal, but has only 2 digits after the decimal in run number 2.
What am I supposed to do for this? I cannot use the read command since in that I need to specify in advance the number of digits my program has to read.
I basically need a way for my program to identify data between commas and read a value or varying precision between the commas.
For input, the decimal part of a format specifier is only used if the input field does not contain a decimal point.
For the last few decades (since the demise of punched cards), users typically expect that a numeric value that doesn't contain a decimal point is an integer value. Consequently, for input, format specifications for real numbers should always have .0 for their decimal part.
For example, after:
CHARACTER(4) :: input
REAL :: a, b
input = '1 '
READ (input, "(F4.0)") a
READ (input, "(F4.1)") b
a will have the value 1.0, and b will have the value 0.1.
(For input, it doesn't particularly matter which particular real data descriptor is used (F, E, D, or G) - they all behave the same regardless of the nature of the input.)
So, for input, all you have to worry about is getting the field width right. Once you have read a record into a string this is easy enough to do by using the INDEX intrinsic.
I'm currently storing various metadata about videos and one of those bits of data is the length of a video.
So if a video is 10 minutes 35 seconds long, it's saved as "10:35" in the database.
But what I'd like to do is retrieve a listing of videos by length (longest first, shortest last).
The problem I'm having is that if a video is "2:56", it's coming up as longest because the number 2 is more than the number 1 in.
So, how can I order data based on that length field so that "10:35" is recognized as being longer than "2:56" (as per my example)?
SELECT * FROM table ORDER BY str_to_date(meta_time,'%l:%i')
You can find the specific formatters on the MySQL Website.
For example:
%k -> Hour (0..23)
%l -> Hour (1..12)
The easiest choice is to store a integer (seconds) or a float (minutes) instead of a string. So 10:35 would be 635 in seconds or 10.583 in minutes. You can sort by these numerically very easily. And you can output them in the format you'd like with some simple math and string functions.
Some options:
Save it as an integer representing the total number of seconds. "10:35" => 635
Save it as a timestamp object with no date component. "10:35" => MAKETIME(0, 10, 34)
Save it with leading decimals or spaces. "2:25" => " 2:25"
My preference would be for the first option.
You could try to see if
ORDER BY TIME_TO_SEC(timefield)
would parse it correctly, however it is not an optimal approach to store time as strings in the database, and I suggest that you store them as TIME if you are able to. Then you can use standard formatting functions to present them as you like.
I had the same problem - storing videos length in database.
I solved it by using TIME mysql type - it solves all ordering and converting issues.