Before we start: Yes, I have to use 2005. No, I can't upgrade, it's not my decision.
I have an SSIS package that is suppsed to perform the following tasks:
1. Zip a text file using 7-zip
2. FTP the file to a specified site
3. Move the zipped file to an archive folder
Steps 1 and 2 work fine, but when it comes time to move the folder it fails. The error message says, "Could not find a part of the path '\Shares2\clntrial\DataMgt\C1460\DataTransfer\Data\Sent\TXT\archive\ABC701_XXX_2015-09-30.zip\ABC701_XXX_2015-09-30.zip'." As you can see, it has the zip file name in the path twice.
That file destination path is set by a connection manager with the connection string populated by an expression that uses variables. When I edit the expression, it evaluates to the correct path with the file name appearing only once, that is, "\Shares2\clntrial\DataMgt\C1460\DataTransfer\Data\Sent\TXT\archive\ABC701_XXX_2015-09-30.zip".
Why is SSIS adding the file name a second time when the package runs?
Never mind, I'm an idiot. By including the zip file name "ABC701_XXX_2015-09-30.zip" in the destination I'm telling the file system task to consider that the target folder. Duh.
Related
I have set up part of my ssis package to check a folder and move the csv file to another folder using a file system task.
Is it possible to rename a file without knowing what the source name will be? as I have been told it will be a new name on a daily bases. can I use a wildcard in a file system task? would I be best renaming before moving what is best practise?
For an unknown file name you can wrap your file system task in a foreach loop
wrap your file system task in a foreach loop and set to file enumerate
set the folder to your source location
set search string to *.csv
set to full file path
map to a variable called fname
use fname as variable in file system task for source
Make sure you delay validation on connection manager
I am trying to automate weekly generation of a database. As a first step in this process, I need to obtain a set of files from network location M:\. The process is as follows:
Delete any possibly remaining old source files from my local folder (REMOVE_OLD_FILES).
Obtain the names of the required files using regular expressions (GET_FILES).
Copy the files from the network location to my local folder for further processing (COPY/MOVE FILES)
Step 3 is where I run into trouble, I frequently receive the below error:
Error processing files. Exception : org.apache.commons.vfs.FileNotFoundException: Could not read from "file:///M:/FILESOURCE/FILENAME.zip" because it is a not a file.
However, when I manually locatae the 'erroneous' file on the network location and try to open or copy it, there are no problems. If I then re-run the Spoon job, no errors occur for this file (although the next file might lead to an error).
So far, I have verified that steps 1 and 2 run correctly: more specifically, there are no errors in the file names returned from step 2.
Obviously, I would prefer not having to manually open all the files first to ensure that Spoon can correctly copy them. Does anyone have an idea what might be causing this behaviour?
For completeness, below are the parameters selected in the COPY/MOVE FILES step.
I was facing same issue with different clients and finally i tried with some basic approach and it got resolved. It might help in your case as well.
Also, other users can follow this rule.
Just try this: Create all required folder with Spoon Job "Create a Folder" and inactive/delete those hops from your job or transformation once your folders are created.
This is because, user you are using to delete the file/s is not recognized as Windows User. Once your folder is in place you can remove "Create a Folder" steps from your Job.
The path to the file is wrong. If you are running spoon in a Windows environment you should use the Windows format for filepaths. Try changing from
"file:///M:/FILESOURCE/FILENAME.zip"
To
"M:\FILESOURCE\FILENAME.zip"
By the way, it will only work if M: is an actual drive in the machine. If you want to access a file in the network you should use the network path to the shared folder, this way:
"\\MachineName\M$\FILESOURCE\FILENAME.zip"
or
"\\MachineName\FILESOURCE\FILENAME.zip"
If you try to access a file in a network mounted drive it won't work.
I am working in BIDS (SSIS 2008). I have a flat file connection to a pip-delimited file with a .pip extension. The file name can change significantly. However, it will always have the word 'stuff' in the file name. For example, a valid file name is 123_x_stuff_456.pip. How can I set up a dynamic connection for this dynamic file name?
So far, I have created a variable to hold the UNC path of the file (e.g., \drive1\folder1). My next step (I think) is to create an expression in the flat file connection manager to concatenate the file name to the UNC path variable. The problem I am running into is that I can't specify wildcards for the text before and after the token that I'm search for in the filename. For example, in SSMS, I would concatenate the UNC path variable to something like '%stuff%. For all tutorials that I've seen on this, they always follow a prescribed file name format, whereas, in my situation, the format will always be different (other than the token that it will always contain). How do I go about this?
I created a simple SSIS package to import a flat file (.txt) into a database table. Tested that and it works perfectly. Since I have several files to import, I added a foreach loop to go through all the files, added the variables as recommended in several examples found on the net but now my flat file connection manager returns an error of "A valid file name must be selected." and the package will not run. I have so far been unsuccessful in finding the solution for this issue and would appreciate any suggestions by the SSIS gurus of this forum. Many thanks in advance!
Here is what I have in the way of variables:
SourceFileFolder which is the path to the folder that contains the files
FileName a string containing one of the names of the files I am seeking to import
SourceFilePath which is an expression driven variable that incorporates the previous two variables concatenated together. I can click "Evaluate Expression" and copy and paste it into windows explorer and open the file
ArchivePath which is an expression driven variable that creates the path to archive the file to once it is processed.
As the message says this is related to your connection manager not gathering the connection string. This can be handled using the following:
First of all clear the expression on the SourceFilePath variable.
With your Foreach Loop Container, set it up as follows:
This will use your variable SourceFileFolder as the Folder, you could also just hardcode the folder name C:\ for instance. Also make sure your folder is qualified correctly, I.E. make sure it finishes with a slash C: won't work but C:\ will work.
Next you need to map the fully qualified name to your other variable SourceFilePath
This should now store the full name of the file the loop has found into the SourceFilePath variable. For Instance C:\File.txt, you can now use this as a connection string expression on your file connection manager.
Under the properties of the connection manager make sure the expression is set to ConnectionString and then use the SourceFileName variable.
ALSO MAKE SURE DELAY VALIDATION IS SET TO TRUE
This hopefully should mean you can loop through the files.
I am trying to zip the contents of a Folder in SSIS, there are files and folders in the source folder and I need to zip them all individually. I can get the files to zip fine my problem is the folders.
I have to use 7.zip to create the zipped packages.
Can anyone point me to a good tutorial. I haven't been able to implement any of the samples that I have found.
Thanks
This is how I have configured it.
Its easy to configure but the trick is in constructing the Arguments. Though you see the Arguments as static in the screenshot, its actually coming from a variable and that variable is set in the Arguments expression of Execute Process Task.
I presume you will have this Execute Process task in a For Each File Ennumerator with Traverse SubFolders checked.
Once you have this basic setup in place, all you need to do is work on building the arguments to do the zipping, how you want them. A good place to find all the command line arguments is here.
Finally, the only issue I ran into was not providing a working directory in the command line arguments for 7zip. The package used to run fine on my dev environment but used to fail when running on the server via a SQL job. This was because 7zip didn't have access to the 'Temp' folder on the SQL Server, which it uses by default as the 'working directory'. I got round this problem by specifying the 'working directory as follows at the end of the command line arguments, using the -ws switch:
For e.g:
a -t7z DestinationFile.7z SourceFile -wS:YourTempDirectoryToWhichTheSQLAgentHasRights