I have a tab-delimited output from an outside program. One of the fields is (non-negotiable) is of #/#, that is 1/1, 1/2, 3/5 etc (represents part one of two, three of five etc).
Access is wrongly interpreting this as a date, where it needs to interpret it as a text. If I tell access to store it as a date (view the Type of field in the table) it still does not import properly.
How do I tell access, in the import stages, "interpret this field as TEXT"?
Assuming you are using the Import Wizard you should be able to specify the data type for each field in the source file. If it is a text file you can even create an "Import Specification" for the file where you indicate which fields to import and their data types. The import spec can be saved for future use if this will be a file format you need to work with in the future. You can also use the import spec via VBA if you use the Docmd.TransferText method.
Related
I am trying to load a big file which basically is a json format flat file from my local drive to SQL Server by using SSIS. It's a one line file and I don't need to specify columns and rows as I am going to parse it as soon as it's in SQL Server by OPENJSON.
but when I tried to create Flat File Source in Visual Studio SSIS, I was not able to do that as even I used 'fixed width' format according to the solution here: import large flat file with very long string as SSIS package, as the max width seems to be 32000, while the json file could be much bigger.
here are my settings:
There are other options of loading the data by t-sql like OPENROWSET but we have SQL Server instance installed on another server rather than the same one we are doing our dev work. So there are some security limits between them.
So just wondering if this is the limitation of Flat File Source in SSIS or I didn't do it right?
You're likely looking for the Import Column transformation. https://learn.microsoft.com/en-us/sql/integration-services/data-flow/transformations/import-column-transformation?view=sql-server-ver15
Define a Data Flow as OLE Source -> Import Column -> OLE Destination.
OLE Source
Really, any source but this is the easiest to reproduce
SELECT 'C:\curl\output\source_data.txt' AS SourceFilePath;
That will add a column named SourceFilePath with a single row.
Import Column
Reference the article on Import Column Transformation but the summary is
Check the column that will provide the path
Add a column to the Import Column Collection to hold the file content. Change the data type to DT_TEXT/DT_NTEXT depending on your unicode-ness and note the LineageID value
Click back to Import Column Input and find the column name. Scroll down to the Custom Properties and use the LineageID above for FileDataColumnID where it says 0. Otherwise, you have an error of
The "Import Column.Outputs[Import Column Output].Columns[FileContent]" is not referenced by any input column. Each output column must be referenced by exactly one input column.
OLE DB Destination
Any data sink will do but the important thing will be to map our column from the previous step to a n/varchar(max) in the database.
I need to skip the first three lines of a CSV file when loading into MS Access. The default CSV import does not drop the first three lines. I am thinking of writing a macro to ignore the first three lines.
My research has shown the DoCmd object but its methods do not cater to drop the first x lines.
Any thoughts?
The DoCmd.TransferText method takes a SpecificationName as one of its arguments. Follow the instructions at How to Create an Import Specification to create and save an import specification.
Now go into Access Options, Navigation Options, and show hidden/system objects. You should now be able to open a table named 'mSysIMEXSpecs'. Find the import spec you created earlier based on the SpecName column. Change the 'StartRow' column for that import spec to 3 (the StartRow is zero-based).
Now you should be able to use the DoCmd.TransferText method passing the name of the import spec you created and it will skip your first three lines.
In Access VBA I would use two TextStream objects, one to .ReadLine the original CSV file line-by-line and the other to .WriteLine to a temporary file, skipping the first three lines of the input file. Do some web searches on Scripting.FileSystemObject and I'm sure you'll find some sample code for this.
Then I would use DoCmd.TransferText acImportDelim to import the temporary file into Access.
The Docmd.TransferText method is a good option to go with but as an alternative to modifying your import spec, you could consider importing everything and doing some validation after the import.
So you could, import the entire file and then use a delete query to delete data from the table.
As mentioned in a comment above, modifying the spec via a system table could be tricky for someone else to find where-as a delete query with a nice little comment in your code could work a treat. If it's required use it, if not it could be commented out.
I have been trying to import CSV into a process node which doesnt care about rest of the fields (dynamic in count of fields as well) but for 2 or 3. But in those other fields I have date fields that is being imported in a wrong way. The field gets automatically assigned as Date20. while it is actually datetime. Also another field that is supposed to be a 16 digit character is being imported as number and is getting truncated (shows in the form 9.401153E15). After processing, this node exports the data into CSV and I see all these errors there.
I checked few links like http://www2.sas.com/proceedings/sugi30/038-30.pdf which is relevent to the topic but irrelevent in the context. How can I solve this?
PROC IMPORT for CSV simply generates datastep code, so I would recommend simply copying the datastep code into your program (it should be visible in the log) and editing it to reflect your needs.
I'm importing data from a CSV file into an Access table. The number is something like
-21000000 (-2.1E7). I'm using TransferText to do the import.
DoCmd.TransferText acImportDelim, , "matching report temp", Source_folder & "\" & Source
In the "matching report temp" table, the field is set up as Double. The import generates a type conversion failure. However, when I open the CSV file in Excel and copy the offending row, I can successfully use Paste Append to add it to the table manually - so the number doesn't exceed the capacity of the field.
As far as I can tell, this only happens with large negative numbers. Smaller numbers, and positive numbers of the same magnitude seem to import fine. I know I can specify an Import Specification in TransferText, but I don't see a way to set the field type to Double.
How can I get around this problem? I don't want to have to manually track down the import errors and append them by hand.
Don't let TransferText create the Access table for you. Create the Access destination table first and assign the field types you need.
When you run TransferText, the CSV data will be appended to that existing table. It should work without error as long as you choose compatible data types for the Access fields.
I examined your CSV file. The header row is troublesome because some field names are missing:
Reporting Unit,,$ Dollars,Offset Unit,,$ Dollars,Variance
That seemed to be a complication for DoCmd.TransferText. So I imported manually from the Access UI. And that gave me an import error on row 49 with the 3rd and 6th columns. In Access, the destination fields were both created as Long Integer. However the values for those fields in the CSV row are 2262169190 and -2262169190 ... both beyond the capacity of Access' Long Integer, -2,147,483,648 to 2,147,483,647.
As a cheap workaround, I selected Text for the data type of those 2 columns when I imported. That allowed the import to work without error. Once you get the data into Access successfully, you could cast those (string) values to a numeric type when you need to use them.
It also worked when I chose Double as the type for those 2 columns. It sounds like that's what you actually want.
If you want to get the import working with DoCmd.TransferText, I think you'll need to create an import specification. If possible, I would also first modify the header line of the CSV file so that all the fields have names.
I actually tested that approach without altering the CSV file. After creating an import specification in the Access UI (see screen capture below), this TransferText operation worked in Access 2007 without import errors.
DoCmd.TransferText acImportDelim, _
"IC_Y1301_Specification", _
"IC_Y1301_LD10279_F25210001", _
"C:\Users\hans\Downloads\IC_Y1301_LD10279_F25210001.CSV", _
True
b'h
i will be programmatically importing a text file into a table. i have saved the import steps. i will use this:
DoCmd.TransferText(TransferType, SpecificationName, TableName, FileName, HasFieldNames, HTMLTableName, CodePage)
to do the import.
this is what SpecificationName means:
Specification Name The specification name for the set of options that determines how a text file is imported, exported, or linked. For a fixed-width text file, you must either specify a specification name or use a schema.ini file, which should be stored in the same directory as the imported, linked, or exported text file.
i do need a specific specification; however i do not know how to specify one. when i manually imported a table, i did save the specifications for my import, however i do not know how to call this in the TransferText.
does anyone know how to specify SpecificationName ?
When you do the import manually, you can save the import options as a specification. You assign a name to the specification at the time you create it. In Access 2003, click the Advanced button on the Import Text Wizard. Then you will get a dialog for <filename> Import Specification; click the Save As button and assign a Specification Name ... for example "MySpec".
Use that name (in quotes) as the SpecificationName option for your DoCmd.TransferText command.