I have setup ssis logging to a text file. In the connection manager I have selected create file and given path as c:\logs\log.txt
Notice that log file is not generated if the log folder is absent. How to ensure that folder is created if not exists? I tried choosing create folder on connection manager but that is also not creating the log file in absence of the c:\log folder.
How to ensure folder is auto created and log is always generated?
You have a chicken and egg scenario here. Consider the following replication of your problem
I have the connection manager driven by a variable LogFileName which generates the date and time. That file lives in whatever folder is specified by LogPath and the first thing my package does is create the folder if it does not already exist. "This thing can run anywhere and all is good." I've said that plenty and have the scars to show for it.
The following shows the events you can choose to log (based on what is in my package).
I am only logging OnPostExecute events. So I'm good, right? Because the post execute event won't fire until after that File System Task has completed.
If that were the case, you wouldn't have posted a question.
The first event that a package generates is a PackageStart event. Look at that list of events - no ability to filter that out. It doesn't matter whether you want that event logged or not, the logging handlers hear the PackageStart event and record it. Always.
The specified Text file logger should be used to record the data and it's ready to record PackageStart to file... "oh that path doesn't exist."
It will exist once the very first task (File System Task, Create Folder) has completed but alas, it it too late. You either get the complete sequence of events or none.
In your Output window, you would see something like the following
SSIS package "C:\Users\bfellows\source\repos\PackageDeploymentModel\PackageDeploymentModel\ChickenAndEgg_Logging.dtsx" starting.
Error: 0xC001404B at ChickenAndEgg_Logging, Log provider "SSIS log provider for Text files": The SSIS logging provider has failed to open the log. Error code: 0x80070003.
The system cannot find the path specified.
Error: 0xC001404B at ChickenAndEgg_Logging, Log provider "SSIS log provider for Text
files": The SSIS logging provider has failed to open the log. Error code: 0x80070003.
The system cannot find the path specified.
SSIS package "C:\Users\bfellows\source\repos\PackageDeploymentModel\PackageDeploymentModel\ChickenAndEgg_Logging.dtsx" finished: Success.
The package will show your Control Flow objects as all having gone green/OK and the status message will say it "Package execution completed with success" but on the results tab, you'll have a red X showing the log provider couldn't open the log.
What do I do
Preconfigure your environments as part of the package deployment process. When we used the native logger as you're inquiring about, we had a document that laid out all that new developers/new servers needed to have done to ensure all of this stuff was laid out and configured as it needed to be.
Unless a client has a strong business case for using the classic logging methodology, I would encourage them to not use it and instead rely on the SSISDB's native logging. It's cleaner, easier to manage, no special setup required. To quote the fine folks in Cupertino - it just works
Related
I need to run a MS Access job as an automated task. I know Access isn't really built for this type of task, but I have MOST of it working except for one, critical part. In short, this is what it's supposed to do:
Generate a PDF report for a user
Generate an email for the user
Attach the PDF to the email
Send the email via SMTP
It works if a user is logged into a desktop session. The process needs to run as an automated process, without requiring a user to be logged in. Using Powershell and the built-in Task Scheduler (Windows 7 Ultimate, 64-bit), I'm able to get it running on schedule. But the Access code fails when it tries to save the PDF. Through experimentation, I learned that I need to save to the "My Documents" folder, and I have the process running as "me", but I keep getting the same error message:
8/18/2014 4:00:17 PM Report Error in <method name>
2302
-1
0
<project name> can't save the output data to the file you've selected.
MTS
So I suspect that if I select the correct location to save the PDF, it will work. Is there a special location that the system and/or Task Scheduler (TS) can save to? Is there a special way to share a folder that it will allow TS to write to it (without requiring a user to be logged in)?
I personally usually save all these types of files/reports into the %TEMP% folder, which seems appropriate for this application since it is only to store it until it is emailed.
I haven't had any permission issues saving into this folder yet.
If you're unfamiliar with %TEMP% you can search for Environ variables and there are usually different useful file paths to common folders used by the system e.g. %APPDATA%,%USERPROFILE%` etc
Thanks for all the input. After exhaustive testing--based on the response from ashareef above--I've demonstrated that it can't be done. I tried saving to the following environment variable locations:
TEMP
APPDATA
LOCALAPPDATA
PUBLIC
USERPROFILE
I also tried:
C:\Users
C:\Temp
C:\Users\<my user name>\Documents
And none of those worked if I set the task to run whether I was logged in or not:
One location does work, but only if I'm logged in and I set the task to run only when I'm logged in:
C:\Users\<my user name>\Documents
To sum up:
Saving a file from Access
Running as a Scheduled Task
Whether you're logged in or not
Is not possible! So here at work, we're going with Plan B. Thanks for your help!
I have created a master control package that calls several packages using the execute package task control. I have set all the package file locations to use a sql config table which contains the file location and then used an expression to include the package name. Everything works as expected however I want to include some handlers to notify me if the package location does not exist (just in case somebody changes the path in the config table). To test I set an incorrect file name in one of the expressions which turned the execute package task control red as expected however I can't figure out how to add the notification task. I have tried all the error handler events assocaited with it but no joy plus I added a mail task to the task in question for failure and this did not execute!
Any advice greatly appreciated.
Thanks.
That's why you have OnError event handler, just configure it properly. Are you sure, you're checking Event Handlers for the package and not for one of the blocks inside?
There's a plenty of system variables with OnError scope. Check grey x in Variable window to see them. You might want to use:
- ErrorDescription
- ErrorCode
- SourceName
but choose them according to the report format.
Now in event handler create a script which will put a message into a new variable, and finally send it with send mail task.
I have a custom SSIS logging provider to push logs from SSIS processes to a custom logging service. The logging configuration is in the package level.
Process:
1. In the OpenLog, I create an operation ID to the log, to be assigned to the existing logging service.
2. When the log method is executed, I log an event in the existing logging service as a child of the operation created in the OpenLog method.
Everything works fine, however a new SSIS package was created with child packages. Therefore, now I have multiple Operation ID's, and since the packages are executed in parallel, I have to use the proper Operation ID to each log.
Problem? The Log method doesn't provide the current package. The SourceID provides the component ID, but not the Package ID.
Is there a way (from the API) to retrieve the package logging an entry?
Thanks,
Rob
So far SSIS doesn't provide a direct way to access the package name or id inside the custom log provider.
What you can do is enable the package logging through the custom log provider and provide the package name as Configuration property to custom provider.
Then you can access ConfigString property within the custom log provider methods and create an event source using it.
In your case you can create a one custom provider for both parent and child and provide different names for configuration property.
I have the following set up on a server
Two MS-Access queries
Exported the data from the queries and saved the export steps
Created a macro to run these export steps then exit MS-Access
Created a batch file to open ms-access and run the macro
This works fine and dandy if I log into the server and double click the batch file, but if I set up a scheduled task to "run even if not logged in" then the ms-access macro doesn't seem to run....so it seems like access is never ran.
Does anyone have any ideas? Can you do this sort of process when "not logged in"? Is there something else I need to do? I am really stuck and need to have this automated.
Thanks!
My best guess is that you are trying to access a mapped network location that is not available.
Many networks have logon scripts that map network drives to local drive letters. These logon scripts are not run when you set up a scheduled task to "run even if not logged on."
So if you either:
A) are exporting your query results to a file on a mapped network drive (ie, N:\MyExport.txt) or,
B) have one or more tables in the source of your query that are linked to .mdb's on mapped network drives (ie, CurrentDb.Tables("MyTable").Connect = "N:\MyBackend.mdb")
then the process will fail.
If you are expecting to see an error message when you log on, you won't see one because the Access process is not visible when run through a scheduled task. However, if you go to Task Manager you should see an MSACCESS.EXE process listed.
If this is the problem there are two possible workarounds:
Use UNC paths (ie, \\fileserver\folder\MyExport.txt instead of N:\MyExport.txt)
Set up your scheduled task to run a batch file that:
maps the network drives using NET USE commands
opens your MS Access program with the appropriate command line arguments
I am new to SSIS packages and I am trying to get logging to work from within a custom script. We have it logging the native messages from the package already but I want to add my own custom log messages to it. I see on the Microsoft.SqlServer.Dts.Pipeline.ScriptComponent class there is a Log method but I am unsure what to use for the dataCode and dataBytes arguments so I used 0 and an empty array but this did not log anything.
So how do I get the logging to work from within my script?
Are there any configurations that I need to know about to enable it?
Thanks
Note: I am working with SqlServer 2008 SP2 (not R2)
You need to make sure that the task is enabled for logging. Select SSIS > Logging... from the BIDS menu. Select your data flow task. On the Providers and Logs tab, ensure that a log provider is selected. Select the Details tab and Check the ScriptComponentLogEntry event. Note that this event is not inherited from the Package settings; so you have to select the data flow task. Now your logging should be captured.
You may also be interested in the ComponentMetaData.FireInformation method to log an information event. Here's more information on FireInformation and related methods. You may find these easier to configure, since the associated events (OnInformation for FireInformation) are inherited from the package settings. In other words, if you set logging for the OnInformation event at the package level, all tasks will log the OnInformation event.