Execute Script task throws error "The package execution returned DTSER_FAILURE (1)" - ssis

I have a SSIS package that will load data from a set of excel files then archive those files to a specified folder.
The excel files are stored in a folder and inside that folder I have the archive folder.
Below is my script task code for reference.
public void Main()
{
// TODO: Add your code here
string sourceDir = Dts.Variables["User::strFilePath"].Value.ToString();
string destDir = Dts.Variables["User::strArchivePath"].Value.ToString();
DirectoryInfo di = new DirectoryInfo(sourceDir);
string[] sDirFiles = Directory.GetFiles(sourceDir);
FileInfo[] fi = di.GetFiles("*.xls");
int fileCnt = fi.Length;
for (int i = 0; i < fileCnt; i++)
{
String filename = fi[i].Name;
string[] splitFilename = filename.Split('.');
DateTime dt = DateTime.Now;
string ArchiveDate = String.Format("{0:ddMMMyyyy}", dt);
string sourceFileName = filename;
string sourceFilePath = sourceDir + filename;
string destinationFileName = splitFilename[0] + '_' + ArchiveDate + '.' + splitFilename[1];
string destinationPath = destDir + destinationFileName;
//MessageBox.Show("Source File " + sourceFilePath + " to destination " + destinationPath);
if (File.Exists(destinationPath))
File.Delete(destinationPath);
// To move a file or folder to a new location:
System.IO.File.Move(sourceFilePath, destinationPath);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
The sourceDir and destDir are variables that provides the path of source files folder and archive folder. The package works fine when I run it from visual studio.
I have deployed it to run as a job using deployment utility by creating manifest file. When I run the job I'm getting error in my archive script task. Below is the screenshot of it.
I searched for solution in stackoverflow however the provided solutions does not solve my problem.
Problem using SQL Agent to run SSIS Packages - fails with “DTSER_FAILURE(1)”
DTSX package runs in Visual Studio but not when called from a Database Job
I have granted read write access to SQL server for both the folders. Still getting the same error?
Any help is highly appreciated.

Try choosing 64bit on your Project --> Properties --> Dubugging--> Run64BitTime this could solve the problem.

I replaced the script task with a file system task and it worked. I also created a proxy account to run the package with my system credentials so that it will have the access to the folder specified.
Putting the steps in creating a proxy account for running the SQL job so that it will be helpful for users looking out for it..
Create a User account in SQL under which you need to run the Job
Create a credentials for the user account created.
--Script #1 - Creating a credential to be used by proxy
USE MASTER
GO
--Drop the credential if it is already existing
IF EXISTS (SELECT 1 FROM sys.credentials WHERE name = N'SSISProxyCredentials')
BEGIN
DROP CREDENTIAL [SSISProxyCredentials]
END
GO
CREATE CREDENTIAL [SSISProxyCredentials]
WITH IDENTITY = N'<Insert the Username>',
SECRET = N'abcd#0987'
GO
Create Proxy account and associate the credentials created
--Script #2 - Creating a proxy account
USE msdb
GO
--Drop the proxy if it is already existing
IF EXISTS (SELECT 1 FROM msdb.dbo.sysproxies WHERE name = N'SSISProxyDemo')
BEGIN
EXEC dbo.sp_delete_proxy
#proxy_name = N'SSISProxyDemo'
END
GO
--Create a proxy and use the same credential as created above
EXEC msdb.dbo.sp_add_proxy
#proxy_name = N'SSISProxyDemo',
#credential_name=N'SSISProxyCredentials',
#enabled=1
GO
--To enable or disable you can use this command
EXEC msdb.dbo.sp_update_proxy
#proxy_name = N'SSISProxyDemo',
#enabled = 1 --#enabled = 0
GO
Granting proxy account to SQL Server Agent Sub-systems
USE msdb
GO
--You can view all the sub systems of SQL Server Agent with this command
--You can notice for SSIS Subsystem id is 11
EXEC sp_enum_sqlagent_subsystems
GO
--Grant created proxy to SQL Agent subsystem
--You can grant created proxy to as many as available subsystems
EXEC msdb.dbo.sp_grant_proxy_to_subsystem
#proxy_name=N'SSISProxyDemo',
#subsystem_id=11 --subsystem 11 is for SSIS as you can see in the above image
GO
--View all the proxies granted to all the subsystems
EXEC dbo.sp_enum_proxy_for_subsystem
Granting proxy access to security principals
USE msdb
GO
--Grant proxy account access to security principals that could be
--either login name or fixed server role or msdb role
--Please note, Members of sysadmin server role are allowed to use any proxy
EXEC msdb.dbo.sp_grant_login_to_proxy
#proxy_name=N'SSISProxyDemo'
,#login_name=N'<Insert the Username>'
--,#fixed_server_role=N''
--,#msdb_role=N''
GO
--View logins provided access to proxies
EXEC dbo.sp_enum_login_for_proxy
GO
Finally associate the proxy account to the package step. This can be also done through job wizard.
EXEC msdb.dbo.sp_add_jobstep #job_id=#jobId, #step_name=N'SSISPackageCall',
#step_id=1,
#cmdexec_success_code=0,
#on_success_action=1,
#on_success_step_id=0,
#on_fail_action=2,
#on_fail_step_id=0,
#retry_attempts=0,
#retry_interval=0,
#os_run_priority=0, #subsystem=N'SSIS',
#command=N'/FILE "C:\Package.dtsx" /CHECKPOINTING OFF /REPORTING E',
#database_name=N'master',
#flags=0,
#proxy_name = N'SSISProxyDemo';
Thanks for the valuable replies for my post..

Related

Visual Studio configuration for database access

I have the following in my initialization routine to connect to a database.
String server = "localhost";
String database = "jeniferdb";
String uid = "poalogin";
String password = "oracle";
string connectionString;
connectionString = "SERVER=" + server + ";" + "DATABASE=" + database + ";" + "UID=" + uid + ";" + "PASSWORD=" + password + ";";
connection = new MySqlConnection(connectionString);
CloseConnection();
Can this be put into the Visual studio Configuration file so that I can change the executable from test to prod by only changing a config file.
I once remember (but forgot) in another visual product this could be put in a VS file so changing the database to access was simple changing the xml file.
or what is the best method to secure the connection string.
Normally you put this values in a configuration file like appsettings.json or web.config and read from it. And also you can have a build variant like Debug and Release for the debugging and release version of your product.
NOTE: if you put the configuration in separate file it helps a lot after deploying your product because no need to open the source code and deploy it again you can actually change the value in the configuration file. and also this will helps in development.
this tutorial show how to read values from configuration file

Ms Access: Record(s) cannot be read; no read permission on [table]

i wrote script for downloading mdb files and reading them due OLEDB provider. All works fine, but if i try to read from table, it throws an exception:
Ms Access: Record(s) cannot be read; no read permission on tblMytable
var cmd = new OleDbCommand("SELECT * FROM tblMytable", conn);
var reader = cmd.ExecuteReader();
I changed permissions directly in Ms Access for user "administrator" and it works. But the problem is, that this script musst run twice a day and it downloads about 20 files. So its impossible manually changing permissions.
Is it possible to change read rights for a table programatically?
Thanks a lot for any ideas!
I solved this by using system.mdw file. I copied this file from "c:\Users\Administrator\AppData\Roaming\Microsoft\Access\" (in Win7) to application directory (App_Data) and modified connection string.
string connectionString = #"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=|DataDirectory|\Database.MDB;Persist Security Info=True;Jet OLEDB:System Database=|DataDirectory|\System.MDW;
var conn = new OleDbConnection(connectionString);
If still it's impossible to read data, i execute grant command:
"GRANT SELECT ON TABLE tblTable TO PUBLIC"
And it works :)

SSIS: Accessing a network drive using a different username and password

Is there a way to connect to a network drive that requires a different username/password than the username/password of the user running the package?
I need to copy files from a remote server. Right now I map the network drive in Windows Explorer then do I filesystem task. However, eventually this package will be ran automatically, from a different machine, and will need to map the network drive on its own. Is this possible?
You can use the Execute Process task with the "net use" command to create the mapped drive. Here's how the properties of the task should be set:
Executable: net
Arguments: use \Server\SomeShare YourPassword /user:Domain\YourUser
Any File System tasks following the Execute Process will be able to access the files.
Alternative Method
This Sql Server Select Article covers the steps in details but the basics are:
1) Create a "Execute Process Task" to map the network drive (this maps to the z:)
Executable: cmd.exe
Arguments: /c "NET USE Z: "\\servername\shareddrivename" /user:mydomain\myusername mypassword"
2) Then run a "File System Task" to perform the copy. Remember that the destination "Flat File Connection" must have "DelayValidation" set to True as z:\suchandsuch.csv won't exist at design time.
3) Finally, unmap the drive when you're done with another "Execute Process Task"
Executable: cmd.exe
Arguments: /c "NET USE Z: /delete"
Why not use an FTP task to GET the files over to the local machine? Run SSIS on the local machine. When transferring using FTP in binary, its real fast. Just remember that the ROW delimter for SSIS should be LF, not CRLF, as binary FTp does not convert LF (unix) to CRLF (windows)
You have to map the network drive, here's an example that I'm using now:
profile = "false"
landingPadDir = Dts.Variables("strLandingPadDir").Value.ToString
resultsDir = Dts.Variables("strResultsDir").Value.ToString
user = Dts.Variables("strUserName").Value.ToString
pass = Dts.Variables("strPassword").Value.ToString
driveLetter = Dts.Variables("strDriveLetter").Value.ToString
objNetwork = CreateObject("WScript.Network")
CheckDrive = objNetwork.EnumNetworkDrives()
If CheckDrive.Count > 0 Then
For intcount = 0 To CheckDrive.Count - 1 Step 2 'if drive is already mapped, then disconnect it
If CheckDrive.Item(intcount) = driveLetter Then
objNetwork.RemoveNetworkDrive(driveLetter)
End If
Next
End If
objNetwork.MapNetworkDrive(driveLetter, landingPadDir, profile, user, pass)
From There just use that driveLetter and access the file via the mapped drive.
I'm having one issue (which led me here) with a new script that accesses two share drives and performs some copy/move operations between the drives and I get an error from SSIS that says:
This network connection has files open or requests pending.
at Microsoft.VisualBasic.CompilerServices.LateBinding.InternalLateCall(Object o, Type objType, String name, Object[] args, String[] paramnames, Boolean[] CopyBack, Boolean IgnoreReturn)
at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateCall(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack, Boolean IgnoreReturn)
at ScriptTask_3c0c366598174ec2b6a217c43470f581.ScriptMain.Main()
This is only on the "2nd run" of the process and if I run it a 3rd time it all works fine so I'm guessing the connection isn't being properly closed or it is not waiting for the copy/move to complete before moving forward or some such, but I'm unable to find a "close" or "flush" command that prevents this error. If you have any solution, please let me know, but the above code should work for getting the drive mapped using your alternate credentials and allow you to access that share.
Zach
To make the package more robust, you can do the following;
In the first Execute Process Task, set - FailTaskIfReturnCodeNotSuccessValue = False
This will let the package run if the last disconnect has not worked.
This is an older question but more recent versions of SQL Server with SSIS databases allow you to use a proxy to execute SQ Server jobs.
In SSMS Under Security<Credentials set up a credential in the database mapped to the AD account you want to use.
Under SQL Server Agent create a new proxy giving it the credential from step 1 and permissions to execute SSIS packages.
Under the SQL Server Agent jobs create a new job that executes your package
Select the step that executes the package and click EDIT. In the Run As dropdown select the Proxy you created in step 2

Scripting setup of database mail

I've used the SQL Server 2008 GUI to set up database mail profiles & accounts on my test server, and I'd now like to duplicate those to our production database.
Is there a way to generate a script to do this?
AFAIK, there isn't a way to necessarily script this from SSMS but you can create a transportable script in TSQL once and reuse it on all the servers. Here is a good example to get you started with this:
USE [master]
GO
sp_configure 'show advanced options',1
GO
RECONFIGURE WITH OVERRIDE
GO
sp_configure 'Database Mail XPs',1
GO
RECONFIGURE
GO
-- Create a New Mail Profile for Notifications
EXECUTE msdb.dbo.sysmail_add_profile_sp
#profile_name = 'DBA_Notifications',
#description = 'Profile for sending Automated DBA Notifications'
GO
-- Set the New Profile as the Default
EXECUTE msdb.dbo.sysmail_add_principalprofile_sp
#profile_name = 'DBA_Notifications',
#principal_name = 'public',
#is_default = 1 ;
GO
-- Create an Account for the Notifications
EXECUTE msdb.dbo.sysmail_add_account_sp
#account_name = 'SQLMonitor',
#description = 'Account for Automated DBA Notifications',
#email_address = 'email#domain.com', -- Change This
#display_name = 'SQL Monitor',
#mailserver_name = 'smtp.domain.com' -- Change This
GO
-- Add the Account to the Profile
EXECUTE msdb.dbo.sysmail_add_profileaccount_sp
#profile_name = 'DBA_Notifications',
#account_name = 'SQLMonitor',
#sequence_number = 1
GO
The other option would be to leverage SMO, either through .NET or powershell to generate the scripts. The SMO reference for this would be:
SqlMail Class
UPDATE:
Here is how easy it turned out to be to script this with Powershell and SMO:
[void][reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo");
#Set the server to script from
$ServerName = "ServerName";
#Get a server object which corresponds to the default instance
$srv = New-Object -TypeName Microsoft.SqlServer.Management.SMO.Server $ServerName
#Script Database Mail configuration from the server
$srv.Mail.Script();

connection of MATLAB 7.0 and MYSQL

I want to connect MATLAB with MYSQL.I dont know the procedure.In MATLAB help it says about some drivers which confuses me alot.Can someone please guide me!please tell me the complete process.I shall be very very thankful!!!
I use JDBC to connect from MATLAB to mySQL database. Works seamlessly.
First download JDBC driver for mySQL from here:
http://www.mysql.com/downloads/connector/j/
Unpack mysql-connector-java-x.x.xx-bin.jar (the latest version) file from the archive into a folder
In the beginning of your script add the path to this jar file, then you can connect to a database and so on.
Here is an example of connecting to and querying public human genome database:
%# add path to the JAR file you just installed to Java dynamic classpath
javaaddpath('h:\Documents\MATLAB\myJavaClasses\mysql-connector-java-5.1.12-bin.jar')
%# connection parameteres
host = 'genome-mysql.cse.ucsc.edu';
user = 'genome';
password = '';
dbName = 'hg18';
%# JDBC parameters
jdbcString = sprintf('jdbc:mysql://%s/%s', host, dbName);
jdbcDriver = 'com.mysql.jdbc.Driver';
%# Create the database connection object
conn = database(dbName, user , password, jdbcDriver, jdbcString);
gene = 'NF1';
if isconnection(conn) % check to make sure that we successfully connected
qry = sprintf('SELECT geneName, chrom, txStart, txEnd FROM refFlat WHERE geneName=''%s''',gene);
rs = fetch(exec(conn, qry));
rsdata = get(rs, 'Data');
end