i am having a problem in executing the SSIS package through an asp.net application. i'm using c#. every time i run the package through web, i think the package is not disposed properly so until and unless i stop the application the package won't execute...i am not able to dispose or unload the package...
i'm geeting errors like
'WebDev.WebServer.EXE' (Managed): Loaded 'C:\WINDOWS\assembly\GAC_32\CustomMarshalers\2.0.0.0__b03f5f7f11d50a3a\CustomMarshalers.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
The result of the expression ""EXEC " + #[User::vRowData]" on property "SqlStatementSource" cannot be written to the property. The expression was evaluated, but cannot be set on the property.
: Package Execution results: {0}
here is my code
Application app = new Application();
//Package package = app.LoadPackage("D:\ChipRewards_WorkingFolder\ChipRewards\Codebase\HeathPlan\Common\Trunk\SSIS Package\ETL_WorkFlow\ETL_WorkFlow\DataImport.dtsx", null);
//package.ImportConfigurationFile("D:\ChipRewards_WorkingFolder\ChipRewards\Codebase\HeathPlan\Common\Trunk\SSIS Package\ETL_WorkFlow\ETL_WorkFlow\ETLConfig.dtsConfig");
using (Package package = app.LoadPackage("D:\\ChipRewards_Test_Environment\\Test\\ETL_WorkFlow\\ETL_WorkFlow\\DataImport.dtsx", null, true))
{
package.ImportConfigurationFile("D:\\ChipRewards_Test_Environment\\Test\\ETL_WorkFlow\\ETL_WorkFlow\\ETLConfig.dtsConfig");
DTSExecResult result = package.Execute();
if (result == Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure)
{
foreach (Microsoft.SqlServer.Dts.Runtime.DtsError local_DtsError in package.Errors)
{
Debug.WriteLine("Package Execution results: {0}", local_DtsError.Description.ToString());
}
}
package.Dispose();
}
i don't know what's happening...i would be very thankful if anyone could help me solve the problem...
Thanks,
Antiflag
Looking at this bit:-
The result of the expression ""EXEC " + #[User::vRowData]" on property "SqlStatementSource" cannot be written to the property.
It sounds like there is a problem with the configuration file and an expression you are trying to load.
Related
Can a Java application be loaded in a separate process using its name, as opposed to its location, in a platform independent manner?
I know you can execute a program via ...
Process process = Runtime.getRuntime().exec( COMMAND );
... the main issue of this method is that such calls are then platform specific.
Ideally, I'd wrap a method into something as simple as...
EXECUTE.application( CLASS_TO_BE_EXECUTED );
... and pass in the fully qualified name of an application class as CLASS_TO_BE_EXECUTED.
This is a synthesis of some of the other answers that have been provided. The Java system properties provide enough information to come up with the path to the java command and the classpath in what, I think, is a platform independent way.
public final class JavaProcess {
private JavaProcess() {}
public static int exec(Class klass, List<String> args) throws IOException,
InterruptedException {
String javaHome = System.getProperty("java.home");
String javaBin = javaHome +
File.separator + "bin" +
File.separator + "java";
String classpath = System.getProperty("java.class.path");
String className = klass.getName();
List<String> command = new LinkedList<String>();
command.add(javaBin);
command.add("-cp");
command.add(classpath);
command.add(className);
if (args != null) {
command.addAll(args);
}
ProcessBuilder builder = new ProcessBuilder(command);
Process process = builder.inheritIO().start();
process.waitFor();
return process.exitValue();
}
}
You would run this method like so:
int status = JavaProcess.exec(MyClass.class, args);
I thought it made sense to pass in the actual class rather than the String representation of the name since the class has to be in the classpath anyways for this to work.
Two hints:
System.getProperty("java.home") + "/bin/java" gives you a path to the java executable.
((URLClassLoader) Thread.currentThread().getContextClassLoader()).getURL() helps you to reconstruct the classpath of current application.
Then your EXECUTE.application is just (pseudocode):
Process.exec(javaExecutable, "-classpath", urls.join(":"), CLASS_TO_BE_EXECUTED)
This might be an overkill for you, but Project Akuma does what you want and more.
I found it via this entry at Kohsuke's (one of Sun's rock start programmers) fabulously useful blog.
Expanding on #stepancheg's answer the actual code would look like so (in the form of a test).
import org.junit.Test;
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Arrays;
import java.util.stream.Collectors;
public class SpinningUpAJvmTest {
#Test
public void shouldRunAJvm() throws Exception {
String classpath = Arrays.stream(((URLClassLoader) Thread.currentThread().getContextClassLoader()).getURLs())
.map(URL::getFile)
.collect(Collectors.joining(File.pathSeparator));
Process process = new ProcessBuilder(
System.getProperty("java.home") + "/bin/java",
"-classpath",
classpath,
MyMainClass.class.getName()
// main class arguments go here
)
.inheritIO()
.start();
int exitCode = process.waitFor();
System.out.println("process stopped with exitCode " + exitCode);
}
}
Did you check out the ProcessBuilder API? It's available since 1.5
http://java.sun.com/javase/6/docs/api/java/lang/ProcessBuilder.html
public abstract class EXECUTE {
private EXECUTE() { /* Procedural Abstract */ }
public static Process application( final String CLASS_TO_BE_EXECUTED ) {
final String EXEC_ARGUMENT
= new StringBuilder().
append( java.lang.System.getProperty( "java.home" ) ).
append( java.io.File.separator ).
append( "bin" ).
append( java.io.File.separator ).
append( "java" ).
append( " " ).
append( new java.io.File( "." ).getAbsolutePath() ).
append( java.io.File.separator ).
append( CLASS_TO_BE_EXECUTED ).
toString();
try {
return Runtime.getRuntime().exec( EXEC_ARGUMENT );
} catch ( final Exception EXCEPTION ) {
System.err.println( EXCEPTION.getStackTrace() );
}
return null;
}
}
Do you really have to launch them natively? Could you just call their "main" methods directly? The only special thing about main is that the VM launcher calls it, nothing stops you from calling main yourself.
Following on what TofuBeer had to say: Are you sure you really need to fork off another JVM? The JVM has really good support for concurrency these days, so you can get a lot of functionality for relatively cheap by just spinning off a new Thread or two (that may or may not require calling into Foo#main(String[])). Check out java.util.concurrent for more info.
If you decide to fork, you set yourself up for a bit of complexity related to finding required resources. That is, if your app is changing frequently and depends upon a bunch of jar files, you'll need to keep track of them all so that they can be passed out to the classpath arg. Additionally, such an approach requires to to infer both the location of the (currently executing) JVM (which may not be accurate) and the location of the current classpath (which is even less likely to be accurate, depending upon the way that the spawning Thread has been invoked - jar, jnlp, exploded .classes dir, some container, etc.).
On the other hand, linking into static #main methods has its pitfalls as well. static modifiers have a nasty tendency of leaking into other code and are generally frowned upon by design-minded folks.
A problem that occurs when you run this from a java GUI is it runs in the background.
So you cannot see the command prompt at all.
To get around this, you have to run the java.exe through "cmd.exe" AND "start".
I dont know why, but if you put "cmd /c start" infront it shows the command prompt as it runs.
However, the problem with "start" is that if there is a space in the path to the application
(which the path to the java exe usually have as it is in
C:\Program Files\Java\jre6\bin\java.exe or similar),
then start just fails with "cannot find c:\Program"
So you have to put quotes around C:\Program Files\Java\jre6\bin\java.exe
Now start complains about parameters that you pass to java.exe:
"The system cannot find the file -cp."
Escaping the space in "Program Files" with a backslash also does not work.
So the idea is to not use space.
Generate a temporary file with the bat extension and then put your command with spaces in there
and run the bat.
However, running a bat through start, does not exit when done,
so you have to put "exit" at the end of the batch file.
This still seems yucky.
So, looking for alternatives, I have found that using quote space quote in the space of "Program Files" actually works with start.
In the EXECUTE class above change the string builder appends to:
append( "cmd /C start \"Some title\" " ).
append( java.lang.System.getProperty( "java.home" ).replaceAll(" ", "\" \"") ).
append( java.io.File.separator ).
append( "bin" ).
append( java.io.File.separator ).
append( "java" ).
append( " " ).
append( new java.io.File( "." ).getAbsolutePath() ).
append( java.io.File.separator ).
append( CLASS_TO_BE_EXECUTED ).
I am working to run/execute SSIS package from my application developed in MVC 5.
The package is developed using Visual studio 2017 data tools.
I have written the following code to run the package
try
{
Application app = new Application();
Package package = null;
package = app.LoadPackage(#"C:\Users\EMISDb\Desktop\NEOC\EOC data need\NEOC_ETL\NEOC_ETL\99_Dash_WHO.dtsx", null);
//Excute Package
Microsoft.SqlServer.Dts.Runtime.DTSExecResult results = package.Execute();
if (results == Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure)
{
foreach (Microsoft.SqlServer.Dts.Runtime.DtsError local_DtsError in package.Errors)
{
Console.WriteLine("Package Execution results: {0}", local_DtsError.Description.ToString());
Console.WriteLine();
}
}
}
catch (DtsException ex)
{
throw;
}
but when i run, it gives the following error message. I tried to find 64 bit but could not find to install. Any help please.
**Retrieving the COM class factory for component with CLSID {4F0FC44B-C99C-441D-B86A-D60D7E22143D} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).**
and
**An Integration Services class cannot be found. Make sure that Integration Services is correctly installed on the computer that is running the application. Also, make sure that the 64-bit version of Integration Services is installed if you are running a 64-bit application.** '
I'm trying to connect to a SFTP server via an SSIS package. The package executes WinSCP with the following connection string in a .txt file:
open sftp://username:fc$#6444#example.com:22
However the package keeps failing without being able to connect. Is it something to do with the special characters in the password?
I am able to connect to a different SFTP if I replace the string so I know it must be something to do with the syntax above. I've tried putting double quotes around the string as follows without any success:
open "sftp://username:fc$#6444#example.com:22"
I had to do this too, for one of my work projects recently. We used the WinSCP .NET assembly inside an SSIS Scripting Task, as this is what WinSCP also recommends as the way to achieve SFTP using WinSCP in SSIS.
See this guide - Using WinSCP .NET Assembly from SQL Server Integration Services (SSIS). It walks you through the install and setup and also contains working sample code (after you change the script to your needs of course!).
Sample code - after you reference the WinSCPnet.dll assembly - is below.
using System;
using Microsoft.SqlServer.Dts.Runtime;
using Microsoft.SqlServer.Dts.Tasks.ScriptTask;
using System.AddIn;
using WinSCP;
namespace ST_5a30686e70c04c5a8a93729fd90b8c79.csproj
{
[AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : VSTARTScriptObjectModelBase
{
public void Main()
{
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Sftp,
// To setup these variables, go to SSIS > Variables.
// To make them accessible from the script task, in the context menu of the task,
// choose Edit. On the Script task editor on Script page, select ReadOnlyVariables,
// and tick the below properties.
HostName = (string) Dts.Variables["User::HostName"].Value,
UserName = (string) Dts.Variables["User::UserName"].Value,
Password = (string) Dts.Variables["User::Password"].Value,
SshHostKeyFingerprint = (string) Dts.Variables["User::SshHostKeyFingerprint"].Value
};
try
{
using (Session session = new Session())
{
// As WinSCP .NET assembly has to be stored in GAC to be used with SSIS,
// you need to set path to WinSCP.exe explicitly, if using non-default location.
session.ExecutablePath = #"C:\winscp\winscp.exe";
// Connect
session.Open(sessionOptions);
// Upload files
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
TransferOperationResult transferResult;
transferResult = session.PutFiles(#"d:\toupload\*", "/home/user/", false, transferOptions);
// Throw on any error
transferResult.Check();
// Print results
bool fireAgain = false;
foreach (TransferEventArgs transfer in transferResult.Transfers)
{
Dts.Events.FireInformation(0, null,
string.Format("Upload of {0} succeeded", transfer.FileName),
null, 0, ref fireAgain);
}
}
Dts.TaskResult = (int)DTSExecResult.Success;
}
catch (Exception e)
{
Dts.Events.FireError(0, null,
string.Format("Error when using WinSCP to upload files: {0}", e),
null, 0);
Dts.TaskResult = (int)DTSExecResult.Failure;
}
}
}
}
Install WinSCP and then create a folder where you want a file from client or put the file.Then Open a Execute Process Task and then go to Expression tab and set the Executable and Arguments with below codes(Please change accordingly).
Write this code in notepad and save as winscp.txt at the path C:\path\to\winscp.txt.
Open sftp://Host_Name:Password#apacsftp01.mftservice.com/ -hostkey="ssh-rsa 2048 xxxxxxxxxxx...="
get -delete /home/client/Share/MediaData/Media_file.xlsx
exit
I am doing a development to integrate a non Java OMS system with QuickFIX/J to send buy/sell orders to multiple brokerage systems .
I have written the belog logic to send the messages
I have written this under main function which is in the same class created by implementing Application "public class Initiator implements Application"
InputStream inp = InitiatorSocket.class.getResourceAsStream("test.cfg");
SessionSettings sessionSetting = new SessionSettings(inp);
Application myApp = new Initiator();
FileStoreFactory factory = new FileStoreFactory(sessionSetting);
ScreenLogFactory sfactory = new ScreenLogFactory(sessionSetting);
DefaultMessageFactory defaultMsgFactory = new DefaultMessageFactory();
initiator = new SocketInitiator(myApp, factory, sessionSetting,sfactory,defaultMsgFactory);
initiator.start();
SessionID sessionId = initiator.getSessions().get(0);
I am using the below code to send messages after continuously listening a directory using while Loop.
while(true)
{
readFilefromSrcDirectory();
prepareFixMessage();
Session.sendToTarget(fixMessage, sessionId);
}
My above code is getting executed while debugging but when I run it normally, the Session.sendToTarget(fixMessage, sessionId); and other file read related logic which is next to initiator.start(); is not getting executed.
Kindly note that the same above code is getting executed if we add some console print statements such as System.out.print("Test");
Please help me.
Are your test.cfg settings between debug and run different? I would add console print statements everywhere and work out exactly where the runtime is failing.
I'm working in SSIS and Visual Studio 2008. When executed, I need to have the SSIS package perform the following tasks:
Check a folder for a file
If a file exists take the file and use it as the source for the flat file
Store the name of the file into a global variable that I can access in other parts of my package
The package will be run by some other script. Thus we need it to check for the file every time the package runs. We are trying to prevent the scenario where we have to monitor the folder and execute the package manually when the file appears.
Any suggestions?
The easiest way would be to set up a Foreach Loop container that has all the "work" of your package inside of it (optionally, you can it as a precursor step and use a conditional expression off of it). Assuming you have 2 variables called FileName (which is what you will have the value assigned to) and an InputFolder variable that contains the "where" we should be looking
ForEach Loop Editor
Collection tab:
Enumerator = Foreach File Enumerators
Expression: Directory = #[User:InputFolder]
FileSpec: "YD.*"
Retrieve file name
* Fully qualified
Variable Mappings tab:
Variable: User::FileName
Index: 0
You can also do this via a script task, if you'd like to see that, let me know.
EDIT
This script again assumes you have the variables InputFolder and FileName defined. Create a Script Task Component and check InputFolder as a read only variable, FileName as a read/write variable.
using System;
using System.Data;
using System.IO; // this needs to be added
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
// namespace will vary
namespace ST_bc177fa7cb7d4faca15531cb700b7f11.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
#region VSTA generated code
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
public void Main()
{
string inputFolder;
string fileName;
inputFolder = Dts.Variables["InputFolder"].Value.ToString();
// File, if exists will look like YD.CCYYMMDD.hhmmss.done
string fileMask = "YD.*.done";
// this array will catch all the files matching a given pattern
string[] foundFiles = null;
foundFiles = System.IO.Directory.GetFiles(inputFolder, fileMask);
// Since there should be only one file, we will grab the zeroeth
// element, should it exist
if (foundFiles.Length > 0)
{
fileName = foundFiles[0];
// write the value to our global SSIS variable
Dts.Variables["FileName"].Value = fileName;
}
Dts.TaskResult = (int)ScriptResults.Success;
}
}
}
Here is a possible option. You can achieve this using the Foreach Loop container. Please find the example that I have provided below. Hopefully, that gives an idea.
Step-by-step process:
On the SSIS package, create 3 variables are shown in screenshot #1. Scope CheckFile represents the package name. Variable Folder will represent the folder that you would like to check for the file. Filename represents the file name to check for. Variable FilePath will be the global variable that you will need. It will be filled in with the file path value if the file exists, otherwise it will be empty.
On the package's Control Flow tab, place a Foreach Loop container and a Script Task. Script Task is to showcase that the variable retains the value after the Foreach Loop container execution is complete. Refer screenshot #2.
Configure ForEach Loop container as shown in screenshots #3 and #4.
Replace the Main() method within the Script Task with the code given under the Script task code section. This is to demonstrate the value retained by the variable FilePath.
Screenshots #5 shows no files exist in the path c:\temp\ and screenshot #6 shows the corresponding package execution.
Screenshots #7 shows the file TestFile.txt exists in the path c:\temp\ and screenshot #8 shows the corresponding package execution.
If you would like to process the file when it exists, you can place a Data Flow Task within the Foreach Loop container to do that.
Hope that helps.
Script task code:
C# code that can be used only in SSIS 2008 and above..
public void Main()
{
Variables varCollection = null;
Dts.VariableDispenser.LockForRead("User::FilePath");
Dts.VariableDispenser.GetVariables(ref varCollection);
if (String.IsNullOrEmpty(varCollection["User::FilePath"].Value.ToString()))
{
MessageBox.Show("File doesn't exist.");
}
else
{
MessageBox.Show("File " + varCollection["User::FilePath"].Value.ToString() + " exists.");
}
Dts.TaskResult = (int)ScriptResults.Success;
}
Screenshot #1:
Screenshot #2:
Screenshot #3:
Screenshot #4:
Screenshot #5:
Screenshot #6:
Screenshot #7:
Screenshot #8: