I am using the EzAPI to create SSIS packages via .NET, but when I load an existing package as a template that has existing components (sequence containers and execute SQL tasks etc) the EzExec collection is empty, while the DTS Executables collections has many members. I need to reference some of these existing components as parents and precedents to tasks I want to add to the package via the EzAPI.
Am I missing something in the initalization of the package, or is this even possible?
Below is a edited sample of the code I am attempting with the removal of layout info, this is still not working, count of Executables is 7, count of EzExexs is 0.
Thanks,
Andrew
public static EzPackage loadPackageTemplate(string templateLocation)
{
EzPackage ezPackage = new EzPackage();
try
{
StreamReader s = new StreamReader(templateLocation);
string templateContents = s.ReadToEnd();
s.Close();
templateContents = removeLayoutInformation(templateContents);
ezPackage.LoadFromXML(templateContents);
}
catch (Exception)
{
throw;
}
//need to remove layout from template
return ezPackage;
}
public static string removeLayoutInformation(string strXML)
{
try
{
//Remove the layout information.
while (strXML.IndexOf("<DTS:PackageVariable>") > -1)
{
strXML = strXML.Remove(strXML.IndexOf("<DTS:PackageVariable>"), strXML.IndexOf("</DTS:PackageVariable>") - strXML.IndexOf("<DTS:PackageVariable>") + 22);
}
}
catch (Exception)
{
throw;
}
return strXML;
}
public static EzExecutable GetExecutable(EzPackage ezPac, string identifier)
{
EzExecutable toReturn = null;
foreach (EzExecutable ezEx in ezPac.EzExecs)
{
if (ezEx.EzName == identifier)
{
toReturn = ezEx;
break;
}
}
return toReturn;
}
EzPackage pac = SSISGen.loadPackageTemplate(#"C:\Temp\SSISPackageTemplates\LoadFact.dtsx");
The problem is the layout data throws off the API. There's a discussion on the Codeplex site covering this problem. The poser, Josh Robinson, also blogged about his experience.
Anyways, the crazy thing about SSIS, the layout stuff BIDS/SSDT presents is bolted on to the actual package markup. That interferes with the ezapi stuff so the fix is to strip it out as Josh demonstrates.
Code copied here for future preservation
//Save the package object to XML
string strXML = null;
strXML = TestPackage.SaveToXML();
//Count instances of existing SSIS layout code in package.
int LayoutCount = Regex.Matches(strXML, "<DTS:PackageVariable>").Count;
//Remove the layout information.
for (int i = 0; i < LayoutCount; i++)
{
strXML = strXML.Remove(strXML.IndexOf("<DTS:PackageVariable>"), strXML.IndexOf("</DTS:PackageVariable>") - strXML.IndexOf("<DTS:PackageVariable>") + 22);
}
Related
I'm trying send a set of query results in an email using SSIS. Below is the screen shot and steps for the flow.
Screenshot:
Steps:
1. Using SQL task to get the results of the query in "queryresult" object variable
2. Using the "queryresult" object variable in the Foreach loop and getting the column Values in the string variables "ProjectRefID" and "Accountnumber"
3.Using script task inside the foreachloop container to capture the data in the object variable "query result"
Below is the code inside the script task which I copied from the internet.
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
endregion
namespace ST_77e6b4ea18824b909adc5568475fcf5c
{
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public void Main()
{
Variables varCollection = null;
string header = string.Empty;
string message = string.Empty;
Dts.VariableDispenser.LockForWrite("User::ValidationEmailMessage");
Dts.VariableDispenser.LockForRead("User::Accountnumber");
Dts.VariableDispenser.LockForRead("User::ProjectRefID");
Dts.VariableDispenser.GetVariables(ref varCollection);
if (varCollection["User::ValidationEmailMessage"].Value == string.Empty)
{
header = "Below are the list of Invalid ProjecRefID and Accountnumbers that are not matching with our existing data:\n\n";
header += string.Format("{0}\t{1}\t\t\t{2}\n", "ProjectRefID", "Accountnumber");
varCollection["User::ValidationEmailMessage"].Value = header;
varCollection.Unlock();
}
//Format the query result with tab delimiters
message = string.Format("{0}\t{1}\t{2}",
varCollection["User::ProjectRefID"].Value,
varCollection["User::Accountnumber"].Value);
varCollection["User::ValidationEmailMessage"].Value = varCollection["User::ValidationEmailMessage"].Value + message;
varCollection.Unlock();
Dts.TaskResult = (int)ScriptResults.Success;
}
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
}
}
I tried to resolve this error but somehow I'm not able to figure this out. Let me know if anyone knows how to resolve it.
Unless you require the locking functionality for some reason, you should be able to write the Main method simply as this:
public void Main()
{
string header = string.Empty;
string message = string.Empty;
if (Dts.Variables["User::ValidationEmailMessage"].Value == string.Empty)
{
header = "Below are the list of Invalid ProjecRefID and Accountnumbers that are not matching with our existing data:\n\n";
header += string.Format("{0}\t{1}\t\t\t{2}\n", "ProjectRefID", "Accountnumber");
Dts.Variables["User::ValidationEmailMessage"].Value = header;
}
//Format the query result with tab delimiters
message =
string.Format("{0}\t{1}\t{2}",
Dts.Variables["User::ProjectRefID"].Value,
Dts.Variables["User::Accountnumber"].Value);
Dts.Variables["User::ValidationEmailMessage"].Value = Dts.Variables["User::ValidationEmailMessage"].Value + message;
Dts.TaskResult = (int)ScriptResults.Success;
}
Also, with your string.Format code, you are specifying three indexes, {0}, {1} and {2}, yet you are only providing 2 arguments, i.e. , "ProjectRefID", "Accountnumber");.
So I have an SSIS package that I am working on right now that requires something that I have yet to do. The package currently just has a For Each Loop container that stores the value of the file name it finds in the User::WatchFolder to a variable and then moves that file to another folder for a process to pick up. What I have been tasked with is augmenting this so that the process remains unchanged for .837 file that does not contain a certain set of character but redirecting the files that come through with the word 'RELAY' in them. From there I also need to open up this EDI file and replace the string '5010' with '5010R', save it and move to a separate folder.
I have moved data in a Data flow task based on certain criteria using a Conditional Split, but this is not data from a table or database, so I'm not sure if this can be accomplished in a Control Flow task. Also, I'm assuming that the string can be replaced via a Script Task, but I'm not sure (again) if this is something that would live in the Control flow or in some sort of Data Flow task.
This is what the package looks like thus far.
SSIS Package so far
EDIT: So far I have created a script task using C# to find and replace the values using ReadFile(FilePath) into a variable called FileContent and then doing a FileContent.Replace("someText","someOtherText")
and then writing the contents back to the file using StreamReader and StreamWriter. That part seems to work fine, but I'm not sure now how to move the file depending on whether it contains a certain value in the FileName.
public void Main()
{
String ErrInfo = "";
String FilePath = Dts.Variables["User::FileName"].Value.ToString();
try
{
String FileContent; //Variable to store File Contents
FileContent = ReadFile(FilePath, ErrInfo);
if (ErrInfo.Length > 0)
{
Dts.Log("Error while reading File " + FilePath, 0, null);
Dts.Log(ErrInfo, 0, null);
Dts.TaskResult = (int)ScriptResults.Failure;
return;
}
//FileContent Before Replace;
MessageBox.Show(FileContent);
//Find and Replace --> Modify WHERE clause
FileContent = FileContent.Replace(
"Relay5010 ",
"Relay5010R"
);
//FileContent After Replace;
MessageBox.Show(FileContent);
Dts.Variables["User::FileContent"].Value = FileContent;
//Write the contents back to File
WriteToFile(FilePath, FileContent, ErrInfo);
if (ErrInfo.Length > 0)
{
Dts.Log("Error while writing File " + FilePath, 0, null);
Dts.Log(ErrInfo, 0, null);
Dts.TaskResult = (int)ScriptResults.Failure;
return;
}
}
catch (Exception e)
{
Dts.Log(e.Message, 0, null);
Dts.TaskResult = (int)ScriptResults.Failure;
}
}
public String ReadFile(String FilePath, String ErrInfo)
{
String strContents;
StreamReader sReader;
try
{
sReader = File.OpenText(FilePath);
strContents = sReader.ReadToEnd();
sReader.Close();
return strContents;
}
catch (Exception e)
{
MessageBox.Show(ErrInfo);
ErrInfo = e.Message;
return "";
}
}
public void WriteToFile(String FilePath, String strContents, String ErrInfo)
{
StreamWriter sWriter;
try
{
sWriter = new StreamWriter(FilePath);
sWriter.Write(strContents);
sWriter.Close();
}
catch (Exception e)
{
MessageBox.Show(ErrInfo);
ErrInfo = e.Message;
}
}
#region ScriptResults declaration
/// <summary>
/// This enum provides a convenient shorthand within the scope of this class for setting the
/// result of the script.
///
/// This code was generated automatically.
/// </summary>
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
}
}
Truth is, I'd love to leave the existing files alone and let them run down their normal path and just divert the ones containing the "R" value and those can have the script task applied to them. Is there possibly a way other than another script task? I am just trying to see if SSIS lends me another tool so that this can be done in the most straight forward way possible
This is my first question posted as well so if there is something else I missed here, I won't be offended if it is pointed out!
I am having one VSIX project, which will made some changes in Project.json file of ASPNET5 project. am using the following to edit .json file.
ProjectJson jsonObj = JsonConvert.DeserializeObject<ProjectJson>(jsonContents);
jsonObj = JsonConvert.DeserializeObject<ProjectJson>(jsonContents);
var resultJson = JsonConvert.SerializeObject(jsonObj, Formatting.Indented);
JsonSerializer serializer = new JsonSerializer();
using (StreamWriter sw = new StreamWriter(projectObjects.ProjectJsonPath))
{
var writer = new JsonTextWriter(sw);
serializer.Serialize(writer, resultJson);
}
// File.WriteAllText(projectObjects.ProjectJsonPath, resultJson);
by using both stream writer and writealltext am getting the following message in ASPNET 5 project
The file has unsaved changes inside this editor and has been changed
externally. do you want to reload it?
how to edit .json file without getting the above message?
Its actually the opposite. Since the environment thinks that the file wants to reload with unsaved changes.
You should uncheck the detect file changes. And when you do, it won't detect the external changes and will not warn you, beware though, that if you try to save the file after it has been modified you will lose the external change.(not a problem in your case I guess) and in order to see the changes you will have to close, not save the file and reopen it.
Source : VS2008: Disable asking whether to reload files changed outside the IDE
This is the option you want to check programmatically. I don't know how exactly you can do that but you can find topics about settings at MSDN (Creating an option page and Creating a setting category). Using those topics you can have a sense of how options are created.
Basically what you need to do is to load VS settings file (VS.vssettings) and inject another Xml line. (Have a look at Examining the Settings File section on MSDN)
Update
To be extremely clear the VS settings file is located under
Documents\Your_VS_Version\Settings\CurrentSettings.vssettings
and you need to load the xml and change 'AutoloadExternalChanges' to value 'true'.
You need to tell the environment to ignore file changes. This can be achieved using the IVsFileChangeEx and IVsDocDataFileChangeControl interfaces.
Here is a utility class (derived from the original Visual Studio 2010 SDK Managed Package Framework sample that you can still find here: http://www.getcodesamples.com/src/8641B4F/98B3955E) that should help:
using (SuspendFileChanges suspend = new SuspendFileChanges(site, filePath))
{
// do something with files
suspend.Sync(); // if you optionally want to tell the IDE it has changed
}
The utility class:
public class SuspendFileChanges: IDisposable
{
private readonly IServiceProvider _serviceProvider;
private readonly List<string> _urls;
private readonly IVsDocDataFileChangeControl[] _controls;
public SuspendFileChanges(IServiceProvider serviceProvider, string url)
: this(serviceProvider, new string[] { url })
{
}
public SuspendFileChanges(IServiceProvider serviceProvider, params string[] urls)
{
if (serviceProvider == null)
throw new ArgumentNullException("serviceProvider");
if (urls == null)
throw new ArgumentNullException("urls");
_serviceProvider = serviceProvider;
_urls = new List<string>(urls);
_controls = new IVsDocDataFileChangeControl[_urls.Count];
// or use Package.GetGlobalService ...
IVsRunningDocumentTable rdt = (IVsRunningDocumentTable)serviceProvider.GetService(typeof(SVsRunningDocumentTable));
IVsFileChangeEx fileChange = (IVsFileChangeEx)serviceProvider.GetService(typeof(SVsFileChangeEx));
for(int i = 0; i < _urls.Count; i++)
{
string url = _urls[i];
if (url == null)
continue;
fileChange.IgnoreFile(0, url, 1);
IVsHierarchy hierarchy;
uint itemId;
uint docCookie;
IntPtr docData;
rdt.FindAndLockDocument((uint)_VSRDTFLAGS.RDT_NoLock, url, out hierarchy, out itemId, out docData, out docCookie);
if (docData != IntPtr.Zero)
{
_controls[i] = Marshal.GetObjectForIUnknown(docData) as IVsDocDataFileChangeControl;
if (_controls[i] != null)
{
_controls[i].IgnoreFileChanges(1);
}
Marshal.Release(docData);
}
}
}
public void Sync()
{
IVsFileChangeEx fileChange = (IVsFileChangeEx)_serviceProvider.GetService(typeof(SVsFileChangeEx));
if (fileChange == null)
throw new InvalidOperationException();
foreach (string url in _urls)
{
if (url == null)
continue;
fileChange.SyncFile(url);
}
}
public void Dispose()
{
IVsFileChangeEx fileChange = (IVsFileChangeEx)_serviceProvider.GetService(typeof(SVsFileChangeEx));
if (fileChange != null)
{
foreach (string url in _urls)
{
if (url == null)
continue;
fileChange.IgnoreFile(0, url, 0);
}
}
foreach (IVsDocDataFileChangeControl control in _controls)
{
if (control != null)
{
control.IgnoreFileChanges(0);
}
}
}
}
Basically I've written a C# script for a Script task in SSIS that looks in a User::Directory for 1 csv, if & only if there is one file, it stores that in the instance variable which then maps to the package variables of SSIS.
When I exicute, it gives me the red filled in box of the Script task. I think it's related to how I'm looking at the directory, but I'm not sure.
Please help!
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;
namespace ST_e8b4bbbddb4b4806b79f30644240db19.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
private String fileName = "";
private String RootDirictory;
private String FilePath;
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
public ScriptMain()
{
RootDirictory = Dts.Variables["RootDir"].Value.ToString();
FilePath = RootDirictory + "\\" + "SourceData" + "\\";
}
public void setFileName()
{
DirectoryInfo YDGetDir = new DirectoryInfo(FilePath);
FileInfo[] numberOfFiles = YDGetDir.GetFiles(".csv");
if (numberOfFiles.Length < 2)
{
fileName = numberOfFiles[0].ToString();
}
int fileNameLen = fileName.Length;
String temp = fileName.Substring(0, fileNameLen - 5);
fileName = temp;
}
public void mapStateToPackage()
{
if((fileName!=null)||(fileName!=""))
{
Dts.Variables["ExDFileName"].Value = fileName;
}
}
public void Main()
{
setFileName();
mapStateToPackage();
Dts.TaskResult = (int)ScriptResults.Success;
}
}
}
This could simply be done using Foreach loop container as explained in this Stack Overflow question, which was asked by you. :-)
Anyway, to answer your question with respect to Script Task code that you have provided. Below mentioned reasons could be cause of the issues:
You are looking for .csv. This won't return any results because you are looking for a file with no name but extension .csv. The criteria should be *.csv
If you are looking for exactly one file, then the condition if (numberOfFiles.Length < 2) should be changed to if (numberOfFiles.Length == 1)
The section of code after the if section which extracts the file name should be within the above mentioned if condition and not out side of it. This has to be done to prevent applying substring functionality on an empty string.
Modified code can be found under the Script Task Code section.
Sorry, I took the liberty to simplify the code a little. I am not suggesting this is the best way to do this functionality but this is merely an answer to the question.
Hope that helps.
Script Task Code:
C# code that can be used only in SSIS 2008 and above.
/*
Microsoft SQL Server Integration Services Script Task
Write scripts using Microsoft Visual C# 2008.
The ScriptMain is the entry point class of the script.
*/
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;
namespace ST_3effcc4e812041c7a0fea69251bedc25.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
Variables varCollection = null;
String fileName = string.Empty;
String fileNameNoExtension = string.Empty;
String rootDirectory = string.Empty;
String filePath = string.Empty;
#region VSTA generated code
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
public void Main()
{
Dts.VariableDispenser.LockForRead("User::RootDir");
Dts.VariableDispenser.LockForWrite("User::ExDFileName");
Dts.VariableDispenser.GetVariables(ref varCollection);
rootDirectory = varCollection["User::RootDir"].Value.ToString();
filePath = rootDirectory + #"\SourceData\";
DirectoryInfo YDGetDir = new DirectoryInfo(filePath);
FileInfo[] numberOfFiles = YDGetDir.GetFiles("*.csv");
if (numberOfFiles.Length == 1)
{
fileName = numberOfFiles[0].ToString();
fileNameNoExtension = fileName.Substring(0, fileName.LastIndexOf("."));
}
if (!String.IsNullOrEmpty(fileNameNoExtension))
{
varCollection["User::ExDFileName"].Value = fileNameNoExtension;
}
Dts.TaskResult = (int)ScriptResults.Success;
}
}
}
Per the Google Page Speed recommendations, I want to Specify image dimensions to "Optimize browser rendering."
Specifying a width and height for all
images allows for faster rendering by
eliminating the need for unnecessary
reflows and repaints.
I am investigating ways to traverse through the image files (PNG, JPEG) in my static content project and output a file with the path and filename of each image file as well as the height and width in pixels. I would then use that to help me construct the tags by using the src attribute data to lookup the values to use for the height and width attributes.
\images\logo.png,100,25
My first ideas was looking for an ANT task, since our static content build uses Ant for other purposes (like using YUI Compressor on JavaScript and CSS files). I am open to other ideas as well, including other methods to solve this problem. I would prefer to not have to manually do this work.
You could try this https://github.com/mattwildig/image-size-report-task, which I've made just for this question.
Here was what I implemented so far (needs testing and clean up). Basically, used Tutorial: Tasks using Properties, Filesets & Paths to get me started in an Ant task and How to get image height and width using java? to extract the image dimensions. I'm going to compare against matt's answer before I deploy.
The test build script from my project:
<project name="ImagesTask" basedir="." default="test">
<target name="init">
<taskdef name="images" classname="ImageInfoTask" classpath="..\dist\ImageTask.jar"/>
</target>
<target name="test" depends="init">
<images outputFile="data/images.xml">
<fileset dir="data" includes="images/**/*.jpg"/>
<fileset dir="data" includes="images/**/*.gif"/>
<fileset dir="data" includes="images/**/*.png"/>
</images>
</target>
</project>
The Java source (without imports):
public class ImageInfoTask extends Task {
private String outputFile;
private List fileSetList = new ArrayList();
private PrintStream outputFileStream;
public void setOutputFile(String outputFile) {
this.outputFile = outputFile.replace("/", File.separator);
}
public void addFileset(FileSet fileset) {
fileSetList.add(fileset);
}
protected void validate() {
if (outputFile == null) {
throw new BuildException("file not set");
}
if (fileSetList.size() < 1) {
throw new BuildException("fileset not set");
}
}
protected void openOutputFile() throws IOException {
FileOutputStream out = new FileOutputStream(this.outputFile);
// Connect print stream to the output stream
this.outputFileStream = new PrintStream(out, true, "UTF-8");
this.outputFileStream.println("<images>");
}
protected void writeImgToOutputFile(String filename, Dimension dim) {
String imgTag = " <img src=\"/" + filename.replace("\\", "/")
+ "\" height=\"" + dim.height + "\" width=\"" + dim.width
+ "\" />";
this.outputFileStream.println(imgTag);
}
protected void closeOutputFile() {
this.outputFileStream.println("</images>");
this.outputFileStream.close();
}
#Override
public void execute() {
validate();
try {
openOutputFile();
for (Iterator itFSets = fileSetList.iterator(); itFSets.hasNext();) {
FileSet fs = (FileSet) itFSets.next();
DirectoryScanner ds = fs.getDirectoryScanner(getProject());
String[] includedFiles = ds.getIncludedFiles();
for (int i = 0; i < includedFiles.length; i++) {
String filename = includedFiles[i];
Dimension dim = getImageDim(ds.getBasedir() + File.separator + filename);
if (dim != null) {
writeImgToOutputFile(filename, dim);
}
}
}
closeOutputFile();
} catch (IOException ex) {
log(ex.getMessage());
}
}
private Dimension getImageDim(final String path) {
Dimension result = null;
String suffix = this.getFileSuffix(path);
Iterator<ImageReader> iter = ImageIO.getImageReadersBySuffix(suffix);
if (iter.hasNext()) {
ImageReader reader = iter.next();
try {
ImageInputStream stream = new FileImageInputStream(new File(path));
reader.setInput(stream);
int width = reader.getWidth(reader.getMinIndex());
int height = reader.getHeight(reader.getMinIndex());
result = new Dimension(width, height);
} catch (IOException e) {
log(path + ": " + e.getMessage());
} finally {
reader.dispose();
}
}
return result;
}
private String getFileSuffix(final String path) {
String result = null;
if (path != null) {
result = "";
if (path.lastIndexOf('.') != -1) {
result = path.substring(path.lastIndexOf('.'));
if (result.startsWith(".")) {
result = result.substring(1);
}
}
}
return result;
}
}
I'm not aware of such ant task readily available but it should be relatively simple to write one. In PNG format image size is stored right at the beginning of the file in IHDR header. There are numerous samples of PNG parsers on Google - for example this. Wrap it up in ant task and you're done.