I am trying to zip the contents of a Folder in SSIS, there are files and folders in the source folder and I need to zip them all individually. I can get the files to zip fine my problem is the folders.
I have to use 7.zip to create the zipped packages.
Can anyone point me to a good tutorial. I haven't been able to implement any of the samples that I have found.
Thanks
This is how I have configured it.
Its easy to configure but the trick is in constructing the Arguments. Though you see the Arguments as static in the screenshot, its actually coming from a variable and that variable is set in the Arguments expression of Execute Process Task.
I presume you will have this Execute Process task in a For Each File Ennumerator with Traverse SubFolders checked.
Once you have this basic setup in place, all you need to do is work on building the arguments to do the zipping, how you want them. A good place to find all the command line arguments is here.
Finally, the only issue I ran into was not providing a working directory in the command line arguments for 7zip. The package used to run fine on my dev environment but used to fail when running on the server via a SQL job. This was because 7zip didn't have access to the 'Temp' folder on the SQL Server, which it uses by default as the 'working directory'. I got round this problem by specifying the 'working directory as follows at the end of the command line arguments, using the -ws switch:
For e.g:
a -t7z DestinationFile.7z SourceFile -wS:YourTempDirectoryToWhichTheSQLAgentHasRights
Related
I am trying to automate weekly generation of a database. As a first step in this process, I need to obtain a set of files from network location M:\. The process is as follows:
Delete any possibly remaining old source files from my local folder (REMOVE_OLD_FILES).
Obtain the names of the required files using regular expressions (GET_FILES).
Copy the files from the network location to my local folder for further processing (COPY/MOVE FILES)
Step 3 is where I run into trouble, I frequently receive the below error:
Error processing files. Exception : org.apache.commons.vfs.FileNotFoundException: Could not read from "file:///M:/FILESOURCE/FILENAME.zip" because it is a not a file.
However, when I manually locatae the 'erroneous' file on the network location and try to open or copy it, there are no problems. If I then re-run the Spoon job, no errors occur for this file (although the next file might lead to an error).
So far, I have verified that steps 1 and 2 run correctly: more specifically, there are no errors in the file names returned from step 2.
Obviously, I would prefer not having to manually open all the files first to ensure that Spoon can correctly copy them. Does anyone have an idea what might be causing this behaviour?
For completeness, below are the parameters selected in the COPY/MOVE FILES step.
I was facing same issue with different clients and finally i tried with some basic approach and it got resolved. It might help in your case as well.
Also, other users can follow this rule.
Just try this: Create all required folder with Spoon Job "Create a Folder" and inactive/delete those hops from your job or transformation once your folders are created.
This is because, user you are using to delete the file/s is not recognized as Windows User. Once your folder is in place you can remove "Create a Folder" steps from your Job.
The path to the file is wrong. If you are running spoon in a Windows environment you should use the Windows format for filepaths. Try changing from
"file:///M:/FILESOURCE/FILENAME.zip"
To
"M:\FILESOURCE\FILENAME.zip"
By the way, it will only work if M: is an actual drive in the machine. If you want to access a file in the network you should use the network path to the shared folder, this way:
"\\MachineName\M$\FILESOURCE\FILENAME.zip"
or
"\\MachineName\FILESOURCE\FILENAME.zip"
If you try to access a file in a network mounted drive it won't work.
I have a tcl script named main.tcl in a folder called App. One of the lines in the script uses a command from the twapi module (that line is actually in a proc and I'm trying to minimize the app to system tray when a user closes the app through the 'X' window button):
package require twapi
# ... code here
set hand [twapi::load_icon_from_file tclkit.ico]
# ... code here
The file tclkit.ico is in the same directory as the script (i.e. in the folder App).
When main.tcl is run through wish, the script works without any issues, but after wrapping it into an executable through command line,
> tclkit sdx.kit wrap App -runtime tclsh863.exe
the executable raises an error, notably that the icon file could not be found:
The system cannot find the file specified.
The system cannot find the file specified.
while executing
"LoadImage $hmod $path $type $opts(width) $opts(height) $flags"
(procedure "twapi::_load_image" line 18)
invoked from within
"twapi::load_icon_from_file tclkit.ico"
(procedure "min_to_tray" line 2)
invoked from within
"min_to_tray"
(command for "WM_DELETE_WINDOW" window manager protocol)
The current workaround right now is to have a copy of the tclkit.ico file in the same directory as the .exe but I want to avoid that as much as possible and only have the standalone .exe file. I tried using the full path with:
set hand [twapi::load_icon_from_file [file join [pwd] App.exe tclkit.ico]]
which normally works when I want to read a file (.txt, .png files, etc.) within the .exe, without success.
So basically, is there a way to enable the .exe to load the .ico file from within itself or another workaround that will not require some dependence on a file outside the .exe app?
The core issue is that the relevant Windows API actually takes a filename, and not something that it's more easy to wrap loading-from-archive around (such as a buffer). This means that you have to copy the file out of the archive somewhere and then pass that name to the system call. This is in fact what Tcl does internally for load when it's pulling the DLL from a source that isn't directly visible to the OS; it doesn't do it automatically for TWAPI though, as that library takes the philosophical position of being just a thin wrapper and letting the caller handle the consequences (which does mean you can easily do more tricks, provided you're inventive).
I suggest copying the file to a temporary file somewhere (i.e., the standard location for these things; Tcl 8.6 has file tempfile to help with this sort of trick) and then passing the full filename into the TWAPI call. I think everywhere in the Windows API that you could pass a simple filename in, you can also pass a full filename. (That's actually very convenient…)
I created a simple SSIS package to import a flat file (.txt) into a database table. Tested that and it works perfectly. Since I have several files to import, I added a foreach loop to go through all the files, added the variables as recommended in several examples found on the net but now my flat file connection manager returns an error of "A valid file name must be selected." and the package will not run. I have so far been unsuccessful in finding the solution for this issue and would appreciate any suggestions by the SSIS gurus of this forum. Many thanks in advance!
Here is what I have in the way of variables:
SourceFileFolder which is the path to the folder that contains the files
FileName a string containing one of the names of the files I am seeking to import
SourceFilePath which is an expression driven variable that incorporates the previous two variables concatenated together. I can click "Evaluate Expression" and copy and paste it into windows explorer and open the file
ArchivePath which is an expression driven variable that creates the path to archive the file to once it is processed.
As the message says this is related to your connection manager not gathering the connection string. This can be handled using the following:
First of all clear the expression on the SourceFilePath variable.
With your Foreach Loop Container, set it up as follows:
This will use your variable SourceFileFolder as the Folder, you could also just hardcode the folder name C:\ for instance. Also make sure your folder is qualified correctly, I.E. make sure it finishes with a slash C: won't work but C:\ will work.
Next you need to map the fully qualified name to your other variable SourceFilePath
This should now store the full name of the file the loop has found into the SourceFilePath variable. For Instance C:\File.txt, you can now use this as a connection string expression on your file connection manager.
Under the properties of the connection manager make sure the expression is set to ConnectionString and then use the SourceFileName variable.
ALSO MAKE SURE DELAY VALIDATION IS SET TO TRUE
This hopefully should mean you can loop through the files.
I developed a SSIS package that creates several .txt files. These files are zipped and then the .txt files need to be removed. Using a foreach file enumerator, I loop through all the .txt files for a specific folder. The folder is retrieved from a variable in configuration and looks something like: C:\Folder\
The foreach loop uses: *.txt to gather all .txt files, does not traverse subfolder and uses the full qualified name.
In the Variable Mappings the "FileName" variable gets filled with the 0 index.
Within the foreachloop I use a File system task.
This task removes the .txt files which are generated before, using the FileName variable that is filled in the loop.
On the development machine this runs like a charm. All greens, no problem at all. Now I copy the package and the configuration file to the test environment. A basic version without the file removing was running perfectly fine here. I replaced the package. Nothing big.
Now I run the SQl Server Agent Job and it starts running. I can see all the text files appearing, and disappearing after it created the zipfiles. However, when all files are removed the package results with errors. Namely the error shown above in the title.
I tried looking for the connectionmanager that might have been removed
Looked for connection managers named in the config that don't exist in the package.
No such thing found. Annoying part is that the package is fully functioning, but still results with the error.
EDIT: I noticed that if I run the package using the execute package utility with the dev. config it gives the same errors.
Hopefully someone is able to help me out.
Thanks in advance!
I managed to "fix" the issue. Remove the File System Component responsible for deleting the files. Then add it again and configure it again.
I think this happens if you accidentally change General parameters before changing the Operation parameter. It holds the metadata to irrelevant parameters and upon execution says: "Wait, you defined this parameter but I don't need it, but I'm checking for it anyway, and it's not there!"
It's a bug for sure
I'm running an FTP task inside of SSIS to receive a file and the task executes successfully yet no file is returned to the local folder that I specified. Where did the file go? How can I make the FTP task download a file to the location that I need it at?
Make sure you have the FTP connection Manager and File Manager set up, and other stuff correct as per the list here (MSDN)
operation
"OverwriteFileAtDestination"
IsASCIITransfer
I've had this problem when I first used an FTP task, but can't recall exactly what I had to do. Double check every setting and also make sure a plain old command line FTP works too to ensure it's only an SSIS issue
Is it possible you have an expression or variable set up that puts the file in another folder? This might explain why the file is not where you expect it.
Also, check to make certain that you're not pulling from a UNIX/Linux box, as this is a known issue.