SSIS Scientific Notation Desirable - ssis

I am working on a SSIS project that scans a directory and loops through each excel files that will then be loaded into MSSQL. Currently, I am having an issue with 2966171 being represented as 2.966171e+006. Here is what I have:
1) The Excel Connection String is passing IMEX=1; (Import Export Mode)
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\\UNC\PATH\TO\Excel.xls;Extended Properties="EXCEL 8.0;HDR=NO;IMEX=1";
2) The has confirmed the data type for this field is DT_WSTR of length 255.
Things I have tried:
1) Changing the datatype in Excel to Text
2) Creating a script component that explicitly converts a string to a decimal back to a string. (Terrible Approach)
3) Casting in a derived column component.
EDIT: I must keep this column a DT_WSTR type, some other rows contain alphanumeric values.

use For-each loop controls to loop over excel files
and map the filepath into a variable (ex: #[User::strExcelFile])
in the foreach container you must use 2 dataflow tasks;
the first one contains an excel source and a script component as a destination, the second DataFlowTask is your task
if the excel files have the same structure you must follow the steps below:
open an excel file and change the entirecolumn type to number
in the excel connection manager choose this file
in the second dataflow task in the Excel Source set the delay validation property to true
in the first dataflow task int the script component
properties (script tab) put the variable "strExcelFile" in the read
only variables, And in the script you must do the following steps:
First Add Microsoft.Office.Interop.Excel.dll as a reference
Second read the ExcelFile path from the variable using the following code:
Imports Microsoft.Office.Interop.Excel
Dim strExcelFiles As String = String.Empty
Public Overrides Sub PreExecute()
MyBase.PreExecute()
strExcelFiles = Variables.strExcelFile
End Sub
Third, in The Main Sub write create an Excel.application set the Visible Property to false
Open the ExcelFile and Change the EntireColumnType to Number and save The Excel File and Close the application using the following Code:
Dim ColIdx As Integer = 0 'Number Column index
Dim appExcel As New Excel.Application
appExcel.Visible = False
Dim wrkbExcel As New Excel.Workbook
wrkbExcel = appExcel.Workbooks.Open(strExcelFile)
Dim wrkshExcel As Excel.Worksheet = wrkbExcel.Worksheets(0)
wrkshExcel.Cells(1, ColIdx).EntireColumn.NumberFormat = "0"
'this will change the EntireColumn Type to Number and eliminate scientific character E+
wrkbExcel.Close(True)
appExcel.Quit()
Brief, every Excel file must be edited before importing data from it because the scientific sign appear when a number is stored in a cell having a datatype different from number

Related

How to loop through full result set using foreach container in SSIS

I encounter an issue when trying to use SSIS to retrieve a full result set from excel file.
I got two variables
Here is the dashboard in control flow
#CriteriaResult returns the ADO Recordset
#CriteriaID is the variable I try to loop through under the object I just returned
I encounter the error:The type of the value being assigned to variable "User::CriteriaID" differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object.
My data is a column: 1,2,3A,3B,3C...
Online reference told me to use script task to change the convert object(#CriteriaResult) so I can use the data inside of it.
My VBA code is:
Public Sub Main()
Dim dt As Data.DataTable
Dim ds As Data.DataSet = CType(Dts.Variables("User::CriteriaID").Value, DataSet)
dt = ds.Tables(0)
Dts.TaskResult = ScriptResults.Success
End Sub
May I know how should I revise my code to make it works?
You can try this approach to iterate with an ADO enumerator:
1. Import to recordset with Data Flow Task
2. Iterate in ForEach with ADO Enumarator to recordset object variable
3. Each iteration saves record value to a variable
This is the package screenshot

Access: display .vsd from attachments

I have an Access table where each item has attached a Visio file (.vsd).
In my Access form, I would like to see the file. I don't care if it is an editable Visio file, a preview or just an image.
I have built a VBA code that let me load the Visio file from a Directory. But I need to load the file from a table.
Here my VBA code.
Private Sub Carica_Dati()
Dim path As String
path = "C:\Users\VisioFlow_001.vsd"
With Me.VisioObject ' name of the OLE Object where I want to put the Visio file
.Class = "Visio.Drawing.11"
.OLETypeAllowed = acOLELinked
.SourceDoc = path ' HERE I WANT TO LOAD THE FILE FROM A TABLE OF THE DB
.Enabled = True
.Locked = False
.Action = acOLECreateLink
.SizeMode = acOLESizeZoom
End With
End Sub
Here a preview of the form.
UPDATE
Here a picture to show how the file is attached to the table.
Since attachment fields in Access aren't very consistent, directly loading them into an OLE object is not an option, unless you're willing to do sophisticated things
Microsofts documentation on attachments can be found here
My observations on attachments: the binary data field contains one of the following:
Some characters I can't identify + the file type + the file data appended to it
Some characters I can't identify + the file type + a compressed version of the file data appended to it
Microsoft, in all it's wisdom, has supplied us with a way to save the original file to the disk, but hasn't supplied us with a way to remove those initial characters and the file type from the actual file data, or an easy way to identify if the file is compressed or not (you can check the file type with the table supplied in the link to check if it should be).
In conclusion, you're probably off best either replacing your attachment field with an OLE object in the database, or writing the attachment files to disk before displaying them.
If you use an OLE object field, and load them in as long binary data (not through the GUI), you can easily achieve the behaviour you seek without writing the file to disk, since the binary data is available without any extra characters.
To write an attachment file to disk:
Dim rsForm As DAO.Recordset2
Dim rsFiles As DAO.Recordset2
Set rsForm = Me.Recordset
Set rsFiles = rsForm.Fields("attachment_column").Value
If Not rsFiles.EOF Then
Dim fileLocation As String
fileLocation = Environ("TEMP") & rsFiles.Fields("FileName").Value
rsFiles.Fields("FileData").SaveToFile fileLocation
'Your existing code to display the OLE object here
End If
You do not want to use the Attachment feature. Its purpose is different than what you are attempting.
Put the images into their own stand alone folder outside of the database.
In the table that holds the records for your main form - you need a new field which holds the path & image file name. This is a text field. (If the path segment is uniform for all one can insert that elsewhere via code rather than store it in this field.)
Then in form design - use the image control. This control (all controls) have a source property - that will change with each record using that field that holds the path & file name.
Do a bing/google on the topic of changing an image with every record - the set up isn't intuitive necessarily. Note that older editions did things differently so be sure you get relatively recent advice.
Then when you are using the form and change records - the image will change.
Note after having typed all this.... I have no idea if the visio file type works - I know that jpg and bmp do... so first sanity check a simple fixed image with that file type to see if it works ...

Schema.ini doesn't work properly

I'm using Access and I would like to export a table to a .csv file.
I'm using the DoCmd.TransferText command to do it.
Since my table gets often updated, I don't want to use export specification, that's why I created a schema.ini to export my data.
The start of my schema.ini file looks like this :
ColNameHeader = True
TextDelimiter = None
Format = Delimited(;)
The exported csv file looks like this :
FIELD1;FIELD2;FIELD3
"data","data","data"
"data","data","data"
The formatting is only applied on the first row (which is the column headers) but not on the next rows (which are the actual data).
Can anyone tell me what's wrong please ?
Just an idea: I use only one data source for different kinds of mail merge documents, so I have to allow Access to create a new Schema file each time, so I delete any existing Schema file before proceeding.
Dim objFSO As FileSystemObject
Dim FileNm As String
Set objFSO = New FileSystemObject
FileNm = conAddrPth & "\Schema.ini"
If objFSO.FileExists(FileNm) = True Then
Kill FileNm
End If

Exporting data from MS Access to Excel using VBA

I have a table in MS Access, which has the following data to be exported to excel
Release numbers
Test cases
Results
After exporting to Excel I want to have distinct release numbers as rows starting from A2 and distinct test case name as columns starting from B1. There might be couple thousands records. Then each cell will be set to result tag. Additionally will need some fancy coloring/bordering stuff.
The question - is it possible to do this using VBA in Access and if yes what is the way to go? Any hint, sample, example, resource would be appreciated... I've googled but the most thing I came accross is DoCmd.TransferSpreadsheet or DoCmd.OutputTo which I believe will not do what I want. Saw some examples with CreateObject("Excel.Application") but not sure what are limitations and performance using this way.
I don't know if it would work for your case, but you might try adding the VBA code to an Excel document rather than the Access database. Then you could refresh the data from the Excel file and add the formatting there much easier. Here is one example:
http://www.exceltip.com/st/Import_data_from_Access_to_Excel_%28ADO%29_using_VBA_in_Microsoft_Excel/427.html
(Or see other examples at http://www.exceltip.com/exceltips.php?view=category&ID=213)
Again, it may not work for your case, but it may be an option to consider. Essentially, instead of pushing from Access, you would pull from Excel.
Yes, there are many cases when the DoCmd.TransferSpreadsheet command is inadaquate.
The easiest way is to reference the Excel xx.x Object model within Access (Early Binding). Create and test your vba export function that way. Then once you are satisfied with your output, remove the Excel object model reference, then change your objects to use use Late Binding using CreateObject. This allows you to easily have other machines that are using different versions of Excel/Access to use it just the same.
Here is a quick example:
Sub ExportRecordsetToExcel(outputPath As String, rs As ADODB.Recordset)
'exports the past due report in correct formattig to the specified path
On Error GoTo handler:
Const xlUP As Long = -4162 'excel constants if used need to be referenced manually!
Dim oExcel As Object
Dim oBook As Object
Dim oSheet As Object
Dim row As Long
If rs.BOF And rs.EOF Then
Exit Sub 'no data to write
Else
rs.MoveFirst
End If
row = 1
Set oExcel = CreateObject("Excel.Application")
oExcel.Visible = False 'toggle for debugging
Set oBook = oExcel.Workbooks.Add 'default workbook has 3 sheets
'Add data to cells of the first worksheet in the new workbook.
Set oSheet = oBook.worksheets(1)
Do While rs.EOF = False
oSheet.range("A" & row).value = rs.Fields("MyField").value
'increase row
row = row + 1
Loop
oBook.SaveAs (outputPath)
'tidy up, dont leave open excel process
Set oSheet = Nothing
Set oBook = Nothing
oExcel.Quit
Set oExcel = Nothing
Exit Sub
handler:
'clean up all objects to not leave hanging processes
End Sub

How can I modify a saved Microsoft Access 2007 or 2010 Import Specification?

Does anyone know how to modify an existing import specification in Microsoft Access 2007 or 2010? In older versions there used to be an Advanced button presented during the import wizard that allowed you to select and edit an existing specification. I no longer see this feature but hope that it still exists and has just been moved somewhere else.
I am able to use this feature on my machine using MS Access 2007.
On the Ribbon, select External Data
Select the "Text File" option
This displays the Get External Data Wizard
Specify the location of the file you wish to import
Click OK. This displays the "Import Text Wizard"
On the bottom of this dialog screen is the Advanced button you referenced
Clicking on this button should display the Import Specification screen and allow you to select and modify an existing import spec.
For what its worth, I'm using Access 2007 SP1
I don't believe there is a direct supported way. However, if you are desparate, then under navigation options, select to show system objects. Then in your table list, system tables will appear. Two tables are of interest here: MSysIMEXspecs and MSysIMEXColumns. You'll be able edit import and export information. Good luck!
Tim Lentine's answer seems to be true even in the full release. There is just one other thing I would like to mention.
If you complete your import without going into "Advanced..." and saving the spec, but you do save the import for reuse at the end of the wizard (new feature AFAIK), you will not be able to go back and edit that spec. It is built into the "Saved Import". This may be what Knox was referring to.
You can, however, do a partial work around:
Import a new file (or the same one all over again) but,
This time choose to append, instead of making a new
Click OK.
Go into "advanced" All your column heading and data-types will be there.
Now you can make the changes you need and save the spec inside that dialog. Then cancel out of that import (that is not what you wanted anyway, right?)
You can then use that spec for any further imports. It's not a full solution, but saves some of the work.
Below are three functions you can use to alter and use the MS Access 2010 Import Specification. The third sub changes the name of an existing import specification. The second sub allows you to change any xml text in the import spec. This is useful if you need to change column names, data types, add columns, change the import file location, etc.. In essence anything you want modify for an existing spec. The first Sub is a routine that allows you to call an existing import spec, modify it for a specific file you are attempting to import, importing that file, and then deleting the modified spec, keeping the import spec "template" unaltered and intact. Enjoy.
Public Sub MyExcelTransfer(myTempTable As String, myPath As String)
On Error GoTo ERR_Handler:
Dim mySpec As ImportExportSpecification
Dim myNewSpec As ImportExportSpecification
Dim x As Integer
For x = 0 To CurrentProject.ImportExportSpecifications.Count - 1
If CurrentProject.ImportExportSpecifications.Item(x).Name = "TemporaryImport" Then
CurrentProject.ImportExportSpecifications.Item("TemporaryImport").Delete
x = CurrentProject.ImportExportSpecifications.Count
End If
Next x
Set mySpec = CurrentProject.ImportExportSpecifications.Item(myTempTable)
CurrentProject.ImportExportSpecifications.Add "TemporaryImport", mySpec.XML
Set myNewSpec = CurrentProject.ImportExportSpecifications.Item("TemporaryImport")
myNewSpec.XML = Replace(myNewSpec.XML, "\\MyComputer\ChangeThis", myPath)
myNewSpec.Execute
myNewSpec.Delete
Set mySpec = Nothing
Set myNewSpec = Nothing
exit_ErrHandler:
For x = 0 To CurrentProject.ImportExportSpecifications.Count - 1
If CurrentProject.ImportExportSpecifications.Item(x).Name = "TemporaryImport" Then
CurrentProject.ImportExportSpecifications.Item("TemporaryImport").Delete
x = CurrentProject.ImportExportSpecifications.Count
End If
Next x
Exit Sub
ERR_Handler:
MsgBox Err.Description
Resume exit_ErrHandler
End Sub
Public Sub fixImportSpecs(myTable As String, strFind As String, strRepl As String)
Dim mySpec As ImportExportSpecification
Set mySpec = CurrentProject.ImportExportSpecifications.Item(myTable)
mySpec.XML = Replace(mySpec.XML, strFind, strRepl)
Set mySpec = Nothing
End Sub
Public Sub MyExcelChangeName(OldName As String, NewName As String)
Dim mySpec As ImportExportSpecification
Dim myNewSpec As ImportExportSpecification
Set mySpec = CurrentProject.ImportExportSpecifications.Item(OldName)
CurrentProject.ImportExportSpecifications.Add NewName, mySpec.XML
mySpec.Delete
Set mySpec = Nothing
Set myNewSpec = Nothing
End Sub
When I want to examine or change an import / export specification I query the tables in MS Access where the specification is defined.
SELECT
MSysIMEXSpecs.SpecName,
MSysIMexColumns.*
FROM
MSysIMEXSpecs
LEFT JOIN MSysIMEXColumns
ON MSysIMEXSpecs.SpecID = MSysIMEXColumns.SpecID
WHERE
SpecName = 'MySpecName'
ORDER BY
MSysIMEXSpecs.SpecID, MSysIMEXColumns.Start;
You can also use an UPDATE or INSERT statement to alter existing columns or insert and append new columns to an existing specification. You can create entirely new specifications using this methodology.
Another great option is the free V-Tools addin for Microsoft Access. Among other helpful tools it has a form to edit and save the Import/Export specifications.
Note: As of version 1.83, there is a bug in enumerating the code pages on Windows 10. (Apparently due to a missing/changed API function in Windows 10) The tools still works great, you just need to comment out a few lines of code or step past it in the debug window.
This has been a real life-saver for me in editing a complex import spec for our online orders.
Why so complicated?
Just check System Objects in Access-Options/Current Database/Navigation Options/Show System Objects
Open Table "MSysIMEXSpecs" and change according to your needs - its easy to read...
Tim Lentine's answer works IF you have yours specs saved. Your question did not specify that, it only stated you had imported the data. His method would not save your specs that way.
The way to save the spec of that current import is to re-open the import, hit "apend" and that will allow you to use your current import settings that MS Access picked up. (This is useful if your want to keep the import specs from an Excel format you worked on prior to importing into MS ACCESS.)
Once you're in the apend option, use Tim's instructions, which is using the advanced option and "Save As." From there, simply click cancel, and you can now import any other similar data to various tables, etc.
I have just discovered an apparent bug in the whole Saved Import/XML setup in Access. Also frustrated by the rigidity of the Saved Import system, I created forms and wrote code to pick apart the XML in which the Saved Import specs are stored, to the point that I could use this tool to actually create a Saved Import from scratch via coded examination of a source Excel workbook.
What I've found out is that, while Access correctly imports a worksheet per modifications of default settings by the user (for example, it likes to take any column with a header name ending with "ID" and make it an indexed field in the resulting table, but you can cancel this during the import process), and while it also correctly creates XML in accordance to the user changes, if you then drop the table and use the Saved Import to re-import the worksheet, it ignores the XML import spec and reverts back to using its own invented defaults, at least in the case of the "ID" columns.
You can try this on your own: import an worksheet Excel with at least one column header name ending with "ID" ("OrderID", "User ID", or just plain "ID"). During the process, be sure to set "Indexed" to No for those columns. Execute the import and check "Save import steps" in the final dialog window. If you inspect the resulting table design, you will see there is no index on the field(s) in question. Then delete the table, find the saved import and execute it again. This time, those fields will be set as Indexed in the table design, even though the XML still says no index.
I was pulling my hair out until I discovered what was going on, comparing the XML I built from scratch with examples created through the Access tool.
I used Mike Hansen's solution, it is great. I modified his solution in one point, instead of replacing parts of the string I modified the XML-attribute. Maybe it is too much of an effort when you can modify the string but anyway, here is my solution for that.
This could easily be further modified to change the table etc. too, which is very nice imho.
What was helpful for me was a helper sub to write the XML to a file so I could check the structure and content of it:
Sub writeStringToFile(strPath As String, strText As String)
'#### writes a given string into a given filePath, overwriting a document if it already exists
Dim objStream
Set objStream = CreateObject("ADODB.Stream")
objStream.Charset = "utf-8"
objStream.Open
objStream.WriteText strText
objStream.SaveToFile strPath, 2
End Sub
The XML of an/my ImportExportSpecification for a table with 2 columns looks like this:
<?xml version="1.0"?>
<ImportExportSpecification Path="mypath\mydocument.xlsx" xmlns="urn:www.microsoft.com/office/access/imexspec">
<ImportExcel FirstRowHasNames="true" AppendToTable="myTableName" Range="myExcelWorksheetName">
<Columns PrimaryKey="{Auto}">
<Column Name="Col1" FieldName="SomeFieldName" Indexed="NO" SkipColumn="false" DataType="Double"/>
<Column Name="Col2" FieldName="SomeFieldName" Indexed="NO" SkipColumn="false" DataType="Text"/>
</Columns>
</ImportExcel>
</ImportExportSpecification>
Then I wrote a function to modify the path. I left out error-handling here:
Function modifyDataSourcePath(strNewPath As String, strXMLSpec As String) As String
'#### Changes the path-name of an import-export specification
Dim xDoc As MSXML2.DOMDocument60
Dim childNodes As IXMLDOMNodeList
Dim nodeImExSpec As MSXML2.IXMLDOMNode
Dim childNode As MSXML2.IXMLDOMNode
Dim attributesImExSpec As IXMLDOMNamedNodeMap
Dim attributeImExSpec As IXMLDOMAttribute
Set xDoc = New MSXML2.DOMDocument60
xDoc.async = False: xDoc.validateOnParse = False
xDoc.LoadXML (strXMLSpec)
Set childNodes = xDoc.childNodes
For Each childNode In childNodes
If childNode.nodeName = "ImportExportSpecification" Then
Set nodeImExSpec = childNode
Exit For
End If
Next childNode
Set attributesImExSpec = nodeImExSpec.Attributes
For Each attributeImExSpec In attributesImExSpec
If attributeImExSpec.nodeName = "Path" Then
attributeImExSpec.Value = strNewPath
Exit For
End If
Next attributeImExSpec
modifyDataSourcePath = xDoc.XML
End Function
I use this in Mike's code before the newSpec is executed and instead of the replace statement. Also I write the XML-string into an XML-file in a location relative to the database but that line is optional:
Set myNewSpec = CurrentProject.ImportExportSpecifications.item("TemporaryImport")
myNewSpec.XML = modifyDataSourcePath(myPath, myNewSpec.XML)
Call writeStringToFile(Application.CurrentProject.Path & "\impExpSpec.xml", myNewSpec.XML)
myNewSpec.Execute