VBS DTEXEC - SSIS Package fails to fully execute before continuing on - ssis

I have a vbs script that's failing to finish loading a file before moving on to the next steps. I've been using this code for 2+ years so this is likely due to my poor code and error handling - If the SSIS package fails to fully load, the procedure should kick out and alert me - Can someone point me in the right direction to make sure this package fully loads or if it fails, truncate the stage table and try again?
The file can range from 50mb to 1.2Gb
'******************************
Sub ReloadTable(strTableName)
'******************************
Dim wsh
Set wsh = CreateObject("WScript.Shell")
Dim waitOnReturn
waitOnReturn = True
Dim windowStyle
windowStyle = 0
Dim errorCode
Dim DTEXECStatus
'Truncate the stage table
ExecOnSQL "TRUNCATE TABLE essstage." & strTableName
'Run the SSIS package and wait until complete
errorCode = wsh.Run("dtexec /File ""\\server.dev.local\Data\SSIS SQL16\" & strTableName & ".dtsx"" ", windowStyle, waitOnReturn)
If errorCode = 0 Then
DTEXECStatus = "Success! " & strTableName
'MsgBox "Success! " & strTableName
Else
'Should exit the sub if this fails and notify me
DTEXECStatus = "FAILED!! " & strTableName
Create_NOTICE_Email
MsgBox "Failed! " & strTableName
'It'd be better if this repeated the steps to clear/attempt to reload again but who knows how to do that
Exit Sub
'ML: Added this END to the script to see if it'll stop erroring out
End If
End Sub
'******************************
Sub ExecOnSQL(cmdTxt)
'******************************
Dim strConnSQL
strConnSQL = "Provider=SQLOLEDB; Server=SQLSERVER.dev.LOCAL; Database=goldmouse; Trusted_Connection=Yes"
'Open the connection to the database
Dim cn
Set cn = CreateObject("ADODB.Connection")
cn.Open strConnSQL
'Set the command
Dim cmd
Set cmd = CreateObject("ADODB.Command")
Set cmd.ActiveConnection = cn
'Set the record Set
Dim rs
Set rs = CreateObject("ADODB.recordSet")
'Prepare the command
cmd.CommandText = cmdTxt
cmd.CommandType = 1 'adCmdText
cmd.CommandTimeout = 3000 '50 minutes
'cmd.CommandType = 4 'adCmdStoredProc
'Execute the command
Set rs = cmd.Execute
Set cmd = Nothing
'Close connections
cn.Close
Set cn = Nothing
End Sub

Related

Exporting data from SCADA system

I am trying to create VBscript to export process data from a SCADA system (WinCC RT Professional) to periodically archive all process variables. The data are stored in SQL table that can be accessed through a connectivity pack. I managed to make the script working when exporting one tag (process variable), but I would like to loop over all tags in the system (about 60), collect them in another recordset and then all data from this recordset save in one csv-file. I have created RecSet that collects all variables (fields) of one tag (Time, Process Variable etc.), I only need values from Field 4 (the same field for all tags). I would then like to copy this field in another recordset - RecSetColl which collects all required data (Field 4) from all tags and finally save them in the CSV file. Thank you very much for any help.
Sub DataExport()
Dim fso 'FileSystemObject
Dim f 'File
Dim ts 'TextStream
Dim path 'Path
Dim ArchiveDate 'Archive date
'Name of CSV-file
ArchiveDate = ArchiveDate & Now
ArchiveDate = Replace(ArchiveDate,"/","")
ArchiveDate = Replace(ArchiveDate," ","")
ArchiveDate = Replace(ArchiveDate,":","")
ArchiveDate = "MDF_" & ArchiveDate
'Path to the csv-file
path = "D:\Historical_data\" & ArchiveDate & ".csv"
'Create Filesystemobject and CSV-file if not exists:
Set fso = CreateObject("Scripting.FileSystemObject")
If Not fso.FileExists(path) Then
fso.CreateTextFile(path)
Else
MsgBox "File already exists!"
Exit Sub
End If
'Create object and open it for writing
Set f = fso.GetFile(path)
Set ts = f.OpenAsTextStream(2,-2)
ts.WriteLine("Tag-Name;ValueID;Date/Time;Process-Value") 'Header
'Generate String for the CSV-Filename
Dim Pro 'Provider
Dim DSN 'Data Source Name
Dim DS 'Data Source
Dim ConnString 'Connection String
Dim MachineNameRT 'Name of the PC from WinCC-RT
Dim DSNRT 'Data Source Name from WinnCC-RT
Dim Conn 'Connection to ADODB
Dim RecSet 'RecordSet
Dim RecSetColl 'RecordSet storing data to be saved to the CSV-file
Dim Command 'Query
Dim CommandText 'Command-Text
Dim i
'Read the name of the PC-Station and the DSN-Name from WinCC-RT
Set MachineNameRT = HMIRuntime.Tags("#LocalMachineName")
Set DSNRT = HMIRuntime.Tags("#DatasourceNameRT")
'Preparing the Connection-String
Pro = "Provider=WinCCOLEDBProvider.1;" 'First instance of WinCCOLEDB
DSN = "Catalog=" & DSNRT.Read & ";" 'Name of Runtime-Database
DS = "Data Source=" & MachineNameRT.Read & "\WinCC" 'Data Source
'Build the complete String:
ConnString = Pro + DSN + DS
'Make Connection
Set Conn = CreateObject("ADODB.Connection")
Conn.ConnectionString = ConnString
Conn.CursorLocation = 3
Conn.open
Set RecSetColl = CreateObject("ADODB.Recordset")
With RecSetColl.Fields
.Append "Time1", adChar
.Append "AHU_RUN", adChar
.Append "Time2", adChar
.Append "TT01", adChar
.Append "TT02", adChar
End With
For i = 0 To 4
Set RecSet = CreateObject("ADODB.Recordset")
Set Command = CreateObject("ADODB.Command")
Command.CommandType = 1
Set Command.ActiveConnection = Conn
'Building the complete string
CommandText = "Tag:R," & i & ",'0000-00-00 12:00:00.000','0000-00-00 00:00:00.000'"
Command.CommandText = CommandText
Set RecSet = Command.Execute
RecSet.MoveFirst
RecSetColl.Fields(i) = RecSet.Fields(4) 'RecSet.Fields(4) stores a proces value
RecSet.Close
Set RecSet = Nothing
Set Command = Nothing
Next
'Writing recordsets to CSV-file
Do While Not RecSetColl.EOF
ts.WriteLine (RecSetColl.Fields(0).Value & ";" & RecSetColl.Fields(1).Value & ";" & RecSetColl.Fields(2).Value & ";" & RecSetColl.Fields(3).Value & ";" & RecSetColl.Fields(4).Value & ";" & RecSetColl.Fields(5).Value)
RecSetColl.MoveNext
Loop
RecSetColl.Close
Set RecSetColl = Nothing
Conn.close
Set Conn = Nothing
ts.Close
Set fso = Nothing
Set f = Nothing
Set ts = Nothing
End Sub
I do not really know whats not working, but a guess;
Does ValueID = 0 , (the "i" in the "for 0 to 4" ) exist in your project?
In the table "Archive" you will find the valid ValueIDs, starts with "1" in all my projects. It's simple to see in SQL Management Studio, perhaps sometimes 0 exist.
To get all the values exported, query the "Archive" table first and then ask for data in a loop using whatever ValueID's is returned.
//PerD

Recordset writing ÿþ to CSV file instead of query results

I have a VBS file that queries an access database and writes the results to a .CSV file. The only thing being written to the .CSV file is ÿþ.
I am using the same VBS file with a more complex query and it runs fine. Both queries return 2 fields of the same type. The only change between the 2 files is the SQL query. Also if I paste the query I'm trying to use into Access, it runs the query as expected.
Here is the VBS file:
Dim connStr, objConn, getNames, objFSO, rs
Const ForWriting = 2
'Make and open progress window
On Error Resume Next
Set objExplorer = CreateObject("InternetExplorer.Application")
objExplorer.Navigate "about:blank"
objExplorer.ToolBar = 0
objExplorer.StatusBar = 0
objExplorer.Width = 400
objExplorer.Height = 200
objExplorer.Visible = 1
objExplorer.Silent = 1
objExplorer.Document.Title = "Script in progress"
objExplorer.Document.Body.InnerHTML = "Your Script is being processed. " & "This might take several minutes to complete. Closing this window will stop the script. So don't close this window ^_^"
'Create csv file
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objLogFile = objFSO.CreateTextFile("\\192.168.100.4\data\IT\Scripts\testData.csv", ForWriting, True)
'Define Db String
connStr = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=\\192.168.100.16\Sage\2016\Connex\Sage100Shadow_CHRIS.mdb"
'Define object type
Set objConn = CreateObject("ADODB.Connection")
'Open Connection
objConn.Open connStr
'Define recordset and SQL query
Set rs = objConn.Execute("SELECT [BillNo],[InStock] FROM qryKitInvNums;")
MsgBox(rs.Fields(0).Type & ", " & rs.Fields(1).Type)
'While loop, loops through all available results
Do While Not rs.EOF
'add values seperated by comma to getNames
getNames = rs.Fields(0) & "," & rs.Fields(1)
'Write current record to LogFile
objLogFile.Write getNames
'Line return for next record in LogFile
objLogFile.Writeline
'move to next result before looping again
rs.MoveNext
'continue loop
Loop
MsgBox(getNames)
'Close connection and release objects
objLogFile.Close
objConn.Close
Set rs = Nothing
Set objConn = Nothing
'Closes progress window
objExplorer.Document.Body.InnerHTML = "Your Script is now complete."
WScript.Sleep 2000
objExplorer.Quit
Here is a copy of the working VBS file. The only difference is the SQL query which still returns 2 fields of same type as the first VBS file.
Dim connStr, objConn, getNames, objFSO
Const ForWriting = 2
'Make and open progress window
On Error Resume Next
Set objExplorer = CreateObject("InternetExplorer.Application")
objExplorer.Navigate "about:blank"
objExplorer.ToolBar = 0
objExplorer.StatusBar = 0
objExplorer.Width = 400
objExplorer.Height = 200
objExplorer.Visible = 1
objExplorer.Silent = 1
objExplorer.Document.Title = "Script in progress"
objExplorer.Document.Body.InnerHTML = "Your Script is being processed. " & "This might take several minutes to complete. Closing this window will stop the script. So don't close this window ^_^"
'Create csv file
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objLogFile = objFSO.CreateTextFile("\\192.168.100.4\data\IT\Scripts\testData.csv", ForWriting, True)
'Define Db String
connStr = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=\\192.168.100.16\Sage\2016\Connex\Sage100Shadow_CHRIS.mdb"
'Define object type
Set objConn = CreateObject("ADODB.Connection")
'Open Connection
objConn.Open connStr
'Define recordset and SQL query
Set rs = objConn.Execute("SELECT IM_ItemWarehouse.ItemCode, IIf([InStock] Is Null Or [InStock]="""",[IM_ItemWarehouse].[QuantityOnHand],[InStock]) AS EvalInstock FROM IM_ItemWarehouse LEFT JOIN qryInStockNum4ItemsWPO ON IM_ItemWarehouse.ItemCode = qryInStockNum4ItemsWPO.ItemCode WHERE (((IM_ItemWarehouse.ItemCode) Is Not Null) AND ((IM_ItemWarehouse.WarehouseCode)=""000""));")
'While loop, loops through all available results
Do While Not rs.EOF
'add values seperated by comma to getNames
getNames = rs.Fields(0) & "," & rs.Fields(1)
'Write current record to LogFile
objLogFile.Write getNames
'Line return for next record in LogFile
objLogFile.Writeline
'move to next result before looping again
rs.MoveNext
'continue loop
Loop
'Close connection and release objects
objLogFile.Close
objConn.Close
Set rs = Nothing
Set objConn = Nothing
'Closes progress window
objExplorer.Document.Body.InnerHTML = "Your Script is now complete."
WScript.Sleep 2000
objExplorer.Quit

Connect to an SQL database from EXCEL

I have gone through a few tutorials already and my connection keeps failing, I have tried a lot of different ways of connecting.
I have a connection to mySQL through the mySQL workbench. I am using the IP address and the Port number and then my credentials to login. This works well and I am able to do the queries I need.
I am now trying to access this database through Excel, preferably through VBA. I tried to create a new connection but nothing I do seems to work. I am not sure what to put into my strConn string.
I am currently using:
Options Explicit
Private Sub CommandButton2_Click()
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim strConn As String
Set cn = New ADODB.Connection
strConn = "DRIVER={MySQL ODBC 5.3.7 Driver};" & _
"SERVER=XXX.XXX.X.X;" & _
"PORT=3306" & _
"DATABASE=cahier_de_lab;" & _
"UID=xxx;" & _
"PWD=xxx;" & _
"Option=3"
cn.Open strConn
' Find out if the attempt to connect worked.
If cn.State = adStateOpen Then
MsgBox "Welcome to Pubs!"
Else
MsgBox "Sorry. No Pubs today."
End If
' Close the connection.
cn.Close
End Sub
Thanks for your help!
Export from Excel to SQL Server.
Sub InsertInto()
'Declare some variables
Dim cnn As adodb.Connection
Dim cmd As adodb.Command
Dim strSQL As String
'Create a new Connection object
Set cnn = New adodb.Connection
'Set the connection string
cnn.ConnectionString = "Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog=Northwind;Data Source=Excel-PC\SQLEXPRESS"
'cnn.ConnectionString = "DRIVER=SQL Server;SERVER=Excel-PC\SQLEXPRESS;DATABASE=Northwind;Trusted_Connection=Yes"
'Create a new Command object
Set cmd = New adodb.Command
'Open the Connection to the database
cnn.Open
'Associate the command with the connection
cmd.ActiveConnection = cnn
'Tell the Command we are giving it a bit of SQL to run, not a stored procedure
cmd.CommandType = adCmdText
'Create the SQL
strSQL = "UPDATE TBL SET JOIN_DT = '2013-01-22' WHERE EMPID = 2"
'Pass the SQL to the Command object
cmd.CommandText = strSQL
'Execute the bit of SQL to update the database
cmd.Execute
'Close the connection again
cnn.Close
'Remove the objects
Set cmd = Nothing
Set cnn = Nothing
End Sub
OR . . . .
Import from SQL Server into Excel . . . . .
Sub Create_Connectionstring()
Dim objDL As MSDASC.DataLinks
Dim cnt As ADODB.Connection
Dim stConnect As String 'Instantiate the objects.
Set objDL = New MSDASC.DataLinks
Set cnt = New ADODB.Connection
On Error GoTo Error_Handling 'Show the Data-link wizard
stConnect = objDL.PromptNew 'Test the connection.
cnt.Open stConnect 'Print the string to the VBE Immediate Window.
Debug.Print stConnect 'Release the objects from memory.
exitHere:
cnt.Close
Set cnt = Nothing
Set objDL = Nothing
Exit Sub
Error_Handling: 'If the user cancel the operation.
If Err.Number = 91 Then
Resume exitHere
End If
End Sub

how to use access VBA to store SQL output as XML

My apologies for the description, I have no idea how to name it properly
I have a few stored procedures (running on SQL2000) that provide me XML data as output. This all works fine and as expected, and I am using a VB module within my DTS to export and store the data. The reduced version of this file is as follows
set objStream = CreateObject( "ADODB.Stream" )
set objXML = CreateObject( "MSXML2.DOMDocument" )
set objConn = CreateObject( "ADODB.Connection" )
objConn.ConnectionTimeout = 3600
objConn.CommandTimeOut = 0
ObjConn.open(DTSGlobalVariables("ConnStringConso").Value)
set objComm = CreateObject( "ADODB.Command" )
objComm.ActiveConnection = objConn
objComm.CommandType = adCmdStoredProc
'Create the Stream object
set objStream = CreateObject( "ADODB.Stream" )
objStream.Open
objComm.Properties( "Output Stream" ) = objStream
' Execute the command
objComm.CommandText = "my_stored_procedure"
objComm.CommandTimeout = 3600
objComm.Execute ,, adExecuteStream
' Read the info returned adding the header and footer info
objStream.Position = 0
objXML.LoadXML("<?xml version='1.0'?>" & objStream.ReadText)
' Create the output XML file
xmlfile = filePath & "myfile.xml"
objXML.save( xmlfile )
So basically what this does is call the procedure, store the output in a stream and save the file, fairly straightforward and up till here no issue.
However, I would also need to be able to do this from my access front end, with VBA. So calling the procedure, wrapping the content and saving it as an XML file. Unfortunately my VBA knowledge is a bit rusty...
This is what I currently have :
Dim cnn As ADODB.Connection
Dim cmd As New ADODB.Command, rs As New ADODB.Recordset
Set cnn = New ADODB.Connection
cnn.ConnectionString = "DRIVER=SQL Server;SERVER=myserver;DATABASE=myDatabase;uid=myID;pwd=myPW;Trusted_Connection=Yes"
cnn.Open cnn.ConnectionString
Set cmd = New ADODB.Command
With cmd
.ActiveConnection = cnn
.CommandType = adCmdStoredProc
.CommandText = "my_stored_procedure"
End With
rs.CursorType = adOpenStatic
rs.CursorLocation = adUseClient
rs.LockType = adLockOptimistic
rs.Open cmd
Debug.Print rs(0)
so far so good, I can open the procedure, I get data returned, but that's where my luck ends. The returned data seems to be limited to a certain number of characters (around 2000) and I'm struggling to save it as I want.
I tried below for quick and dirty test, but it appears to screw up my XML (all my attributes are suddenly double-double quoted) and as stated only a part of the content is exported, so any advice on how to do it properly would be highly appreciated
Dim myFile As String
myFile = "d:\output.xml"
Open myFile For Output As #1
Write #1, rs(0)
Close #1
Never mind, I found it myself in the meantime. I had 2 issues, the VBA ADODB works different as the VB variant, and as I was using FOR XML on my SQL Query the output apperantly was chopped in different rows. Hence why I only had partial output. Fiddling a bit with the streaming part in VBA and going through all records made it work. Not too sure this is the most optimal way so if there are ways to improve I'd be interested to know.
Dim cmd As New ADODB.Command, rs As New ADODB.Recordset
Dim cnn As ADODB.Connection
Set cnn = New ADODB.Connection
cnn.ConnectionString = "DRIVER=SQL Server;SERVER=myServer;DATABASE=myDB;uid=myID;pwd=myPW;Trusted_Connection=Yes"
cnn.Open cnn.ConnectionString
Set cmd = New ADODB.Command
With cmd
.ActiveConnection = cnn
.CommandType = adCmdStoredProc
.CommandText = myProc
End With
rs.CursorType = adOpenStatic
rs.CursorLocation = adUseClient
rs.LockType = adLockOptimistic
rs.Open cmd
Dim myXML As Variant
If Not (rs.EOF And rs.BOF) Then
rs.MoveFirst
Do Until rs.EOF = True
myXML = myXML & rs.Fields(0)
rs.MoveNext
Loop
End If
'Create the Stream object
Set objStream = CreateObject("ADODB.Stream")
With objStream
.Type = 2 'Text
.Charset = "utf-8"
.Open
.WriteText myXML
.SaveToFile (filePath & myXMLFile)
.Close
End With

Upload files to SQL Server 2008 database using MS Access 2010

I need to load files eg excel, word etc documents to SQL Server 2008 so that they can be viewable/downloadable from a website. I need to be able to load and open the files from the access database and website.
Can anyone help, thank you.
Here is how I did it in an old project. You can strip away the progress bar stuff and some of the other stuff but you get the idea
Public Sub Upload_file_OLD(lMaterial_ID As Long, strFile_name As String)
'upload the file to the selected material ID.
Dim adStream As ADODB.Stream
Dim rst As ADODB.Recordset
On Error GoTo Error_trap
'check if we have an open connection, if we do use it
Select Case dbCon.State
Case adStateOpen
'connection is open, do nothing
Case adStateConnecting
'still conecting wait
Do Until dbCon.State = adStateOpen
Application.Echo True, "Connection to DB"
Loop
Case adStateClosed
'connection closed, try to open it
If Len(strSQL_con_string) = 0 Then
Set_SQL_con "MCTS"
End If
dbCon.ConnectionString = strSQL_con_string
dbCon.Provider = "sqloledb"
dbCon.Open
End Select
Me.acxProg_bar.Value = 0
Me.acxProg_bar.Visible = True
Me.Repaint
Set adStream = New ADODB.Stream
adStream.Type = adTypeBinary
adStream.Open
Me.acxProg_bar.Value = 10
Me.Repaint
adStream.LoadFromFile strFile_name
Me.acxProg_bar.Value = 50
Me.Repaint
Set rst = New ADODB.Recordset
rst.Open "SELECT Material_FS,Material_file_name, Material_size FROM tblMaterials WHERE Material_ID=" & lMaterial_ID, dbCon, adOpenKeyset, adLockOptimistic
Me.acxProg_bar.Value = 60
Me.Repaint
Me.txtFile_size = adStream.Size
rst.Fields("Material_FS").Value = adStream.Read
rst.Fields("Material_file_name").Value = GetFileName(strFile_name)
rst.Fields("Material_size").Value = adStream.Size
Me.acxProg_bar.Value = 90
Me.Repaint
rst.Update
rst.Close
dbCon.Close
Me.acxProg_bar.Value = 0
Me.acxProg_bar.Visible = False
Me.Repaint
Exit Sub
Error_trap:
If dbCon Is Nothing = False Then
If dbCon.State = adStateOpen Then dbCon.Close
End If
DoCmd.Hourglass False
MsgBox "An error happened in sub Upload_file, error description, " & Err.Description, vbCritical, "MCTS"
End Sub
I could not get the above code to work, but I did get this to work. SQL Server blob field is varbinary(max).
Upload:
Sub TestDocUpload()
Dim cmd As New ADODB.Command
Dim st As New ADODB.Stream
st.Type = adTypeBinary
st.Open
st.LoadFromFile "c:\temparea\18572.pdf"
With cmd
.CommandText = "Insert into tbldocuments(docblob, doctype) values (?, ?)"
.CommandType = adCmdText
.Parameters.Append .CreateParameter("#P1", adLongVarBinary, adParamInput, st.Size, st.Read)
.Parameters.Append .CreateParameter("#P2", adVarChar, adParamInput, 50, "CustPO")
End With
If cnlocal.State = 0 Then OpenNewLocalConnection
cmd.ActiveConnection = cnlocal
cmd.Execute
End Sub
Retrieve:
Sub TestReadDoc()
Dim myblob() As Byte
Dim rs As New ADODB.Recordset
If cnlocal.State = 0 Then OpenNewLocalConnection
rs.Open "tblDocuments", cnlocal, adOpenForwardOnly, adLockReadOnly
rs.MoveFirst
myblob = rs!DocBlob
Open "c:\temparea\output.pdf" For Binary Access Write As 1
Put #1, , myblob
Close #1
End Sub