A customer of mine is looking to mass create some customizing data related the routes. and as such I have a small program which reads in a CSV file with all of the fields as they would be in the customizing transaction.
I'm having a particular problem wrapping my head around a field TVRO-TRAZTD for a couple of reasons.
The user is only filling in a number which represents a number of days.
There is a conversion exit on TRAZTD, except it's obsolete, use CONVERT TIMESTAMP they say
I don't have a timestamp, I have a decimal number representing a part of a day
For example, TRAZTD would be entered as 0,58 from the CSV file, so why is it represented in the table as 135.512?
I tried it the old fashion way and multiplied 0,58 * 24 which gives me 13,92. if I take 13,92 * 10 I get 139.200, which isn't the same but it's the closest I can get, but I don't get it why 10?
Using the conversion exit even though it's obsolete doens't give me a result either, no matter number I give it I always get 0 back. I can't use the convert timestamp either because well, it's not a timestamp or I didn't look up carefully enough how to use it (I didn't see anything other than strings and characters).
The other thing I tried too was just saying "screw it" and placed the data from the CSV directly into the field and hoping the conversion routine will take care of the work, but that doesn't happen either.
Is there anybody out here that can maybe shed some light on where the number after the conversion comes from?
everybody I came to a solution, just incase anybody stumbles upon this same problem.
I took the value from the excel document and multiplied it by 24 to get the amount of hours, and then multipled it 10000 because I don't know, I picked it randomly.
Related
I need to import a CSV file in SAS and it gets stuck on empty fields with a date format. My log shows this field is properly recognised as DATETIME. and ANYDTDTM40. just like other datetime fields. In The first records this field is empty and the LOG then gives a NOTE invalid data. When I enter a date in the first rows with empty fields the message moves along. So it clearly has to do with missing values. Can someone help me out?
In the future please make sure to show your actual code and the log - feel free to omit the data part of the log if it's confidential information.
PROC IMPORT is a guessing procedure and guesses at types. For production processes it's not a good idea to use PROC IMPORT.
You can add the GUESSINGROWS=MAX; option to your code to force SAS to scan the entire file before guessing at types. This will increase the run time of the process but will likely fix your issue. Also, ensure your datetime fields are consistent and correct. If the data does has mixed date types, ie MMDDYY and DDMMYY then it can be bit of a pain to manage. Or if it has DDMMYY and SAS guesses MMDDYY (or vice versa) you'll get a bunch of errors. In that case you need to write your own data step code to read in the data. You can use the code from the log as a starting point.
I am trying to import a file with order numbers that have values in excess of 9 billion into Access.
I have already figured out that I needed to switch from Access 2013 to Access 2016 so that I could enable "Support Bigint Data Type for Linked/Imported Tables". This enabled the Large Number data type in the Access Database.
At this point, I can manually enter Large Numbers into Access, and everything works okay, but when I try to import a .csv with Large Numbers, I get data conversion errors for the Large Numbers (and only the Large Numbers).
In the past, I encountered the issue where text columns starting with what appeared to be a number in the first row encountered conversion errors for actual text values, so I tried putting the Large Number order numbers on the top of the file, and that didn't work.
Can anyone confirm that the Access 2016 Import Wizard can't import Large Number values?
If not, has anyone successfully used import wizard to get a Large Number into Access? How did you achieve that?
Actually, you can use a 20 year old version of access and it can handle numbers with 28 digits.
And if you can enter such numbers manually, the the data is messed up, has extra junk, or extra characters. So no, upgrading will not by magic fix bad formatted data.
I would consider perhaps importing the data, and using say a text column for that data, and then perhaps then see if you can remove junk.
but, if you want to try a larger column, then try currency. and if you want REALLY HUGE WHOPPER then you can use this:
As you can see in above, that is 18 digits long!!!!
So, if the import is failing with above, then attempting to use a MUCH smaller big int will not help you at all here - not at all.
So, try above - see if that works.
However, the fact that you can type in the larger values by hand means that the issue here is messy data - and not that the numbers are too large.
You may well have to settle for importing that column as text, and then running some update query, or even some VBA code to clean out or clear out bad junk and characters or even perhaps extra spaces etc.
So, you don't need to change access to get and allow you to work with larger numbers. The "main" goal and idea behind that larger number is really for better SQL server support, and it not really much if at all a new feature designed to allow you to work with larger numbers (that's more of a by-product of the first goal).
I think people are safer importing using a text file. Access' import specification just seems to fail. I can't rely on it for production systems - the comment by James Marshall above is correct. you can tell Access to import as text, but before it does, if it sees a phone number, say, it will try to convert to a long int first and then just not import numbers exceeding memory, like phone numbers with area codes > 212
I have about 300 measurements (each stored in a dat file) that I would like to read using MATLAB or Python. The files can be exported to text or csv using a proprietary program, but this has to be done one by one.
The question is: what would be the best approach to crack the format of the binary file using the known content from the exported file?
Not sure if this makes any difference to make the cracking easier, but the files are just two columns of (900k) numbers, and from the dat files' size (1,800,668 bytes), it appears as if each number is 16 bits (float) and there is some other information (possible the header).
I tried using HEX-Editor, but wasn't able to pick up any trends from there.
Lastly, I want to make sure to specify that these are measurements I made and the data in them belongs to me. I am not trying to obtain data that I am not supposed to.
Thanks for any help.
EDIT: Reading up a little more I realized that there my be some kind of compression going on. When you look at the data in StreamWare, it gives 7 decimal places, leading me to believe that it is a single precision value (4 bytes). However, the size of the files suggests that each value only takes 2 bytes.
After thinking about it a little more, I finally figured it out. This is very specific, but just in case another Dantec StreamWare user runs into the same problem, it could save him/her a little time.
First, the data is actually only a single vector. The time column is calculated from the length of the recorded signal and the sampling frequency. That information is probably in the header (but I wasn't able to crack that portion).
To obtain the values in MATLAB, I skipped the header bytes using fseek(fid, 668, 'bof'), then I read the data as uint16 using fread(fid, 900000, 'uint16'). This gives you integers.
To get the float value, all you have to do is divide by 2^16 (it's a 16 bit resolution system) and multiply by ten. I assume the factor of ten depends on the range of your data acquisition system.
I hope this helps.
I've got a field that expresses time as:
1900-01-01 07:00:00.000
But I'd like to convert it so it just 07:00 AM. The closest I can find is:
Convert(Varchar(20), DT.EarlyShiftStart, 114)
That gives me:
10:30:00:000
But I'd like to drop the milliseconds and add AM/PM. Anyone know the correct value?
Data and the display of the data are best kept separate. That makes it easier to tweak what the user sees without having to delve into the insides of the programming. For example, if you had your database output datetimes as strings, then it would be more difficult to have the display of those datetimes in different formats - if you wanted to display it in local time then it would have to be converted back to a time, adjusted, and then converted back to a string. If you've thrown away the date information by removing it at the database layer then that might be impossible. Even changing from 12- to 24-hour format would be hassle.
As the data is (likely) to be used in SSRS, it is better to take advantage of the formatting capabilities present in that. For example, you could do what you want with something along the lines of
=Format(yourTime, "hh:mm tt")
in SSRS. Then, if you wanted to show 24-hour time in one particular part of the report, it would just be a case of using something like
=Format(yourTime, "HH:mm")
and anywhere else in the report which needed 12-hour time could stay as it is.
Good day everyone.
I spent a whole day looking for an answer to my question but unfortunately I did not the answer I'm looking for.
I am trying to automate some processes that we are currently using by creating a web application in PHP and a MySQL database.
I would like to prepare the staff schedule in excel then upload it to MySQL using php. Most of the database fields are straight forward: an employee ID that is INT, date that is in DATE format.
The issue I am facing is in the time. When an employee is working, I would like to upload their shift start time and end time in TIME, however, if they are not working, I would like the database to store their status in letters: OFF for a day off, AL for annual leave, ML for medical leave, EL for emergency leave and so on.
I do not know what data type will support this. ie: accept either time or an entry out of a specific list of outcome codes.
Any assistance will be appreciated.
Mohamed.
Side Note:
I asked this question in a time where I was beginning to learn how to code, one of the mistakes that I have learned to avoid is mixing business logic and application logic. This question may be useful to someone in the future, and my advise would be to any new developer is to make sure that the business model logic may not go hand-in-hand with your application logic, but there is always a way to make it work.
Mohamed.
Here is an idea, put another column for status, one of the status will be working status and every other status will be as described. Only working status will have data in the "start time" and "end time" columns. In my opinion this is the best solution and allows for better search capabilities, cleaner database and more comprehensive readability.
However, if you absolutely want to, and/or have any reason on why you can't have an additional column, you can always store your time as text.
PS: Another tip for your database is to drop the date column and store both times in DATETIME format, it may range from unlikely to nearly impossible depending on what job shifts you are storing in the database, but it is possible to start a shift on one day and end it the next day, and even if you think you won't ever need it, it is good practice and makes the database more resilient. If you had to change it in the future it would be a pain to do so.
There is no such datatype in MySQL.
A VARCHAR field would accept both dates and your custom codes but if would be a pain writing the queries: you will always have to check the data before converting to times for reports etc.
I would just create three nullable fields: startTime, endTime and absenceReason and fill absenceReason if and only if both times are NULL.
You could use a VarChar or Char field to store either a time or text values. However, I would recommend using different fields for different things. Such as a start time and end time of data type time. And a separate field for the type of leave, etc. of type VarChar or Char.