how to create dynamic excel sheets based on table data - ssis

Given one table in SQL Server which holds consolidated data from three source tables including one column called OFFICE which differentiates the records from each other.
The three source tables hold data from three offices.
I want to create an Excel file dynamically which will have 3 sheets in one workbook based on the three different different offices (ex. office1, office2, office3) resulting in each sheet containing the relevant data according to its office.
Please recommend an approach using dynamic Excel destination in SSIS as I don't want to use an approach which creates a template file and then copies that template to destination excel file.

While this can be accomplished using a scipt task and C#, a far easier solution is demonstrated at
http://www.rafael-salas.com/2006/12/import-header-line-tables-_116683388696570741.html
and the follow-up
http://www.rafael-salas.com/2008/03/ssis-and-dynamic-excel-destinations_01.html#!
But to summarize the relevant details, you need to use an 'Execute SQL Task' to dynamically create the sheet at runtime prior to using it as a destination.
Create a new variable to hold the Sheet name and set this variable to the Office you are working with as you iterate through them.
Also, create a variable to hold the Create table statement that will create each sheet.
For example,
"CREATE TABLE "+ #[User::SheetName] + "(HeaderID INTEGER, HeaderName NVARCHAR(50), LineID INTEGER, LineName NVARCHAR(50), LineDetails NVARCHAR(50))"
and Set the SQLSourceType Property of the Execute SQL task inside of the For Each container to Variable and choose the Variable you created to hold the create statement.
In the Excel Destination Component, Change the data access mode to ‘Table Name or View Name Variable’ and choose the sheet name variable you created from the variable dropdown list.

I have several SSIS packages that perform a similar function. A single Excel file consists of multiple worksheets with each worksheet populated by results from a separate SQL query. Here are the basic generic steps I applied. Before you begin, make certain you create a connection manager for both the database to be applied and the output Excel file.
1) Create a Script task in Control flow and populate it like the following. Here I am creating the Excel file along with the worksheets it will contain. (Worksheets should never include any spaces or special characters.) My code below is in C#.
using System;
using System.IO;
using System.Collections.Generic;
using System.Data;
using System.Text;
using Excel = Microsoft.Office.Interop.Excel;
using Microsoft.SqlServer.Dts.Runtime;
namespace ST_87e8d62a054b4e16b60297154afc19d8.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
#region VSTA generated code
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
public void Main()
{
Excel.Application xlApp;
Excel.Workbook xlWorkBook;
Excel.Worksheet xlWorkSheet;
object misValue = System.Reflection.Missing.Value;
xlApp = new Excel.ApplicationClass();
xlWorkBook = xlApp.Workbooks.Add(misValue);
//Create First worksheet
xlWorkSheet = (Excel.Worksheet)xlWorkBook.Worksheets.get_Item(1);
xlWorkSheet.Name = "Names";
//Define column headers for "RawData" WorkSheet
xlWorkSheet.Cells[1, 1] = "First Name";
xlWorkSheet.Cells[1, 2] = "Last Name";
xlWorkSheet.Cells[1, 3] = "Title";
// Create Second Worksheet
xlWorkSheet = (Excel.Worksheet)xlWorkBook.Worksheets.get_Item(2);
xlWorkSheet.Name = "Addresses";
//Define column headers for "CCDN" WorkSheet
xlWorkSheet.Cells[1, 1] = "Street";
xlWorkSheet.Cells[1, 2] = "City";
xlWorkSheet.Cells[1, 3] = "State";
xlWorkSheet.Cells[1, 4] = "Zip";
xlWorkSheet.Cells[1, 5] = "Country";
string Filename = "C:\\MyFile.xls";
if (File.Exists(Filename))
{
File.Delete(Filename);
}
xlWorkBook.SaveAs(Filename, Excel.XlFileFormat.xlWorkbookNormal, misValue, misValue, misValue, misValue, Excel.XlSaveAsAccessMode.xlExclusive, misValue, misValue, misValue, misValue, misValue);
xlWorkBook.Close(true, misValue, misValue);
xlApp.Quit();
releaseObject(xlWorkSheet);
releaseObject(xlWorkBook);
releaseObject(xlApp);
Dts.TaskResult = (int)ScriptResults.Success;
}
2) Create in your database two tables that will be populated temporarily. That is, one table will be populated for the results of the first worksheet and the second table will be populated for the results of the second worksheet. It is a good naming approach to preface the names of each table with "Working_" so that you know the purpose of each. I took the approach of using tables instead of views is because I like to sort (ORDER BY) my results, which cannot be done with a view.
3) Add to the SSIS package two Execute SQL Tasks under Control Flow. The first task will run an INSERT SQL statement that will populate the first table you just created and the second task will run another INSERT SQL statement that will populate the second table just created.
4) Add to the SSIS package two Data Flow tasks under Control Flow. The first will be for populating the first worksheet and the second for populating the second worksheet.
5) Select the first Data Flow task and add to it under Data Flow an OLE DB Source where you will define the OLE DB conenction manager (your database) and then the table or view. Select the first new table created. Make certain all of the columns of interest are selected and that you can perform a preview.
6) Add a Data Conversion flow task and then an Excel Destination flow task.
7) Repeat steps 5 and 6 for the second worksheet and table.
8) Finally under Control Flow add an Excel SQL Task that will remove the contents of the two Working tables. You do not want the old contents to be included the next time the package is run.
Now, if you want to play around with formatting of the Excel file after it is completed and impress your manager, you can also do that in code with a final Task Script (also using C#). The nice part about this approach is that you are not having to apply any special formatting functions in your SQL, Excel is doing all the work. You could actually include the formatting in Step 1 and as soon as you copy the data over in the following steps, it is automatically formatted. As with any report output, there is not point in making SQL perform formatting steps (adding additional work to the database server) when it is more efficient to let Excel or SSRS do what they do best.
public void Main()
{
Excel.Application xlApp;
Excel.Workbook xlWorkBook;
Excel.Worksheet xlWorkSheet;
object misValue = System.Reflection.Missing.Value;
Excel.Range xlRange;
xlApp = new Excel.ApplicationClass();
string Filename = "C:\\MyFile.xls";
xlWorkBook = xlApp.Workbooks.Open(FileName, 0, false, 5, "", "", true, Microsoft.Office.Interop.Excel.XlPlatform.xlWindows, "\t", false, false, 0, true, 1, 0);
//Format cells in Names worksheet
xlWorkSheet = (Excel.Worksheet)xlWorkBook.Worksheets.get_Item(1);
//Set the header range in bold font
xlRange = xlWorkSheet.get_Range("a1", "p1");
xlRange.Font.Bold = true;
xlRange.WrapText = true;
//Freeze first row listing headers
xlWorkSheet.Application.ActiveWindow.SplitRow = 1;
xlWorkSheet.Application.ActiveWindow.FreezePanes = true;
//Auto adjust the width of each column
xlWorkSheet.Columns.AutoFit();
xlRange = xlWorkSheet.get_Range("c1", "j6467");
xlRange.Cells.Locked = false;
xlRange.Interior.Color = 65535;
xlRange = xlWorkSheet.get_Range("o1", "p6467");
xlRange.Cells.Locked = false;
xlRange.Interior.Color = 65535;
//Do not alert when saving changes to Excel file.
xlWorkBook.Application.DisplayAlerts = false;
//Save Excel file modifications
xlWorkBook.Save();
//Close workbook and application
xlWorkBook.Close(true, misValue, misValue);
xlApp.Quit();
//Release from cache.
releaseObject(xlWorkSheet);
releaseObject(xlWorkBook);
releaseObject(xlApp);
//Set formatting of percent cells
xlRange = xlWorkSheet.get_Range("d3", "d7");
xlRange.NumberFormat = "###,###%";
//Define the top left cell and bottom right cell of the table in the Excel worksheet
xlRange = xlWorkSheet.get_Range("c1", "c7");
//Draw grid of thin line around each cell in table
xlRange.BorderAround(Excel.XlLineStyle.xlContinuous, Excel.XlBorderWeight.xlThin, Excel.XlColorIndex.xlColorIndexAutomatic, 1);
//Draw thick border around entire table
xlRange = xlWorkSheet.get_Range("a1", "d7");
xlRange.BorderAround(Excel.XlLineStyle.xlContinuous, Excel.XlBorderWeight.xlThick, Excel.XlColorIndex.xlColorIndexAutomatic, 1);
//Right justify columns B and C
xlRange = xlWorkSheet.get_Range("b3", "c7");
xlRange.HorizontalAlignment = Excel.XlHAlign.xlHAlignRight;
//Do not alert when saving changes to Excel file.
xlWorkBook.Application.DisplayAlerts = false;
//Save Excel file modifications
xlWorkBook.Save();
//Close workbook and application
xlWorkBook.Close(true, misValue, misValue);
xlApp.Quit();
//Release from cache.
releaseObject(xlWorkSheet);
releaseObject(xlWorkBook);
releaseObject(xlApp);
Dts.TaskResult = (int)ScriptResults.Success;
}
And that's about it. Notice that just for the purpose of this example, I'm hardcoding the filename. But in my actual code I am applying a User variable which is then populated by another SQL statement pulling the name from another database table. For best practices, it a good idea to keep your SSIS packages entirely table-driven. That way, any changes made to names and locations are made in a database table in a record specific to your SSIS package... avoiding any need to update your SSIS package and go through the dev to QA to production lifecycle again.
Hope this helps and please let me know if you have any questions.

Related

How to add condition in SSIS package to send mails only if attachments are available

I have created an SSIS package that includes one script task to add new attachment name and path in a variable after "|" and running it under foreach loop to include all attachment names and path in the variable value. then I am passing that variable as an attachment to send mail task. This package is running fine via batch file execution and sending one email containing multiple files as well.
Now, I want to schedule that batch file to run on every hour, and to accomplish that, I need to add logic in the package to send mail only if 2 attachments are available. It should not send any email if there are no or single attachment present. that way I want to remove manual job execution. Can you please help? I am new to SSIS development. Script task code as below:
if (Dts.Variables["User::FileNamesList"].Value.ToString() == "")
{
Dts.Variables["User::FileNamesList"].Value = Dts.Variables["User::InputPath"].Value.ToString() + Dts.Variables["User::Year"].Value.ToString() +"\\" + Dts.Variables["User::FileName"].Value.ToString();
}
else
{
Dts.Variables["User::FileNamesList"].Value = Dts.Variables["User::FileNamesList"].Value.ToString() + "|" + Dts.Variables["User::InputPath"].Value.ToString() + Dts.Variables["User::Year"].Value.ToString() + "\\" + Dts.Variables["User::FileName"].Value.ToString();
}
Anything involving mail is much better handled in script task.
Let me explain "better". You have a lot more control on To, Cc, Bcc, ReplyTo, etc. As well as being able to send a html formatted body.
This should run your entire application:
{
var fis = Directory.GetFiles(#"C:\folder"); //a 2nd param can be added for search pattern (ex. "*.xls")
//Check for number of files
if(fis.Length>1)
{
SendEmail("Where's it going", "Here are some files", "What you want in the body", fis);
//Archive
foreach (var f in fis)
File.Move(f, #"archive folder path" + new FileInfo(f).Name);
}
}
public static void SendEmail(string to, string subject, string body, string[] filePaths)
{
string from = "SendingMail#blah.com";
using (MailMessage mail = new MailMessage())
{
SmtpClient SmtpServer = new SmtpClient("YourSMTPClient");
mail.From = new MailAddress(from);
mail.To.Add(new MailAddress(to));
mail.Subject = subject;
mail.IsBodyHtml = true;
mail.Body = body;
foreach (var f in filePaths)
{
Attachment file;
file = new Attachment(f);
mail.Attachments.Add(file);
}
SmtpServer.Port = 9999; //Your SMTP port
SmtpServer.Credentials = new NetworkCredential(from, pword);
SmtpServer.EnableSsl = true;
SmtpServer.Send(mail);
}
}
This was resolved on same day. I was able add variable in my script task to count number of files (basically it will increase by 1 on each loop execution), then I added a simple condition into arrow connecting to mail send task to first check if the variable value is matching with expected values (created a variable in SSIS and assigned max expected file count), so it now it will send mail only if 2 files are available in folder. I have added a block only just to reset the variables and that also has second arrow attached from foreach loop , to run if the condition is not matching. The tool is now sending mail only for year which will have both files present or skip for that loop. I was able to schedule that as well.
Arrow condition true: before send mail task:
condition in arrow for script task to resent variable 2:
Counter:

Kendo Grid : detect new row has been added by 'create' tool bar button

I'm working on master/detail page - master records are in a Kendo drop down list and associated detail data is in a kendo grid.
The dd & grid are bound to remote data.
Updating existing grid rows is working fine.
When a new row is saved to the grid, I need to insert the id of the selected drop down item (master record id) and add it to the json data.
My problem is I don't know how to determine if the data being saved is a new record or an edit.
I'm getting this error: "Uncaught TypeError : Cannot read property 'isNew' of undefined"
Thanks for any guidance here.
$issuegrid
->addColumn($issueOwnerCol)
->addColumn($issueDescriptionCol)
->addColumn($issueDueDateCol)
->pageable(false) //this is the toolbar in the footer
->height(300)
->navigatable(true)
->editable(true)
->save('onSave')
->edit('onEdit')
**->saveChanges('onSaveChanges')**
->addToolbarItem($igridCreate)
->addToolbarItem($igridSave)
->addToolbarItem($igridCancel);
Here's the js function:
function **onSaveChanges**(e){
var grid = $("#issuesGrid").data("kendoGrid");
var URL ="/issues/updaterecord.json";
var ddl = $("#woDD").data("kendoDropDownList");
var v = ddl.value();
if (grid.dataSource.data.model.isNew()){
alert("New Record")
}
grid.dataSource.transport.options.update.url = URL;
grid.dataSource.sync();
}
Well, it's a workaround, but in my onSaveChanges function, I had to append the required id to the create url and then extract the it on the server. I would have preferred to just add a key/value pair to the json payload.
function onSaveChanges(e){
var grid = $("#issuesGrid").data("kendoGrid");
var ddl = $("#woDD").data("kendoDropDownList");
var woID = ddl.value();
var createURL ="/issues/addrecord.json?woID=" + woID;
grid.dataSource.transport.options.create.url = createURL;
//when updating an edited record, transport will use the default url defined in php code.
}

can ssis pick up the file with the latest date creation?

I am thinking of using ssis reading excel files in the folder.
The folder is updated daily by putting new file in without deleting any old files.
I am now using a 'for each' container to loop all the files and loading them into a consolidated table.
However, the boss only wants the latest file to be loaded into the table and he does not want a incremental table.
Can ssis check the file creation date using some functions in ssis and only load the latest one?
ou can use this script:
public void Main()
{
// TODO: Add your code here
var directory= new DirectoryInfo(Dts.Variables["User::VarFolderPath"].Value.ToString());
FileInfo[] files = directory.GetFiles();
DateTime lastModified = DateTime.MinValue;
foreach (FileInfo file in files)
{
if (file.LastWriteTime > lastModified)
{
lastModified = file.LastWriteTime;
Dts.Variables["User::VarFileName"].Value = file.ToString();
}
}
MessageBox.Show(Dts.Variables["User::VarFileName"].Value.ToString());
Dts.TaskResult = (int)ScriptResults.Success;
}
also you can cgeck this link below:
http://sqlage.blogspot.in/2013/12/ssis-how-to-get-most-recent-file-from.html

Allowing others to add Google tasks

I am looking for a way for employees to send me an email or add information to a spreadsheet that will then add tasks to my task list. Ideally, the script would capture the task list, task, due date, and any notes.
I have already successfully implemented five scripts (five task lists) that allow my employees to add tasks to specific tasklists, following this script shown below. This works OK but does not have the capacity to add due dates or notes:
Automated email to task list API
I recently came across references to scripts that monitors task lists, and then posts them to a spread sheet, including task, due dates, notes, etc. It strikes me that a spreadsheet might be a better way to do this though it does not have the convenience of email:
Task list to spreadsheet API
I wonder if the REVERSE can be done. I envision a spreadsheet that I could give my employees access to, with two worksheets (NEW and PROCESSED) with columns:
TASKLIST TASK DUE DATE NOTES
and the script would run through this every hour or two. Anything in NEW would be processed and added to my task list, then moved to the end of PROCESSED.
Does anyone know of something like that out there? Alternatively, perhaps there are ways to change the email script so that it moves anything in the body of the email into the NOTES section of the task. I am a raw newbie at this BTW. Thanks.
you should replace
var newTask = Tasks.newTask().setTitle(title);
by
var newTask = Tasks.newTask().setTitle(title).setDue(date).setNotes(notes);
I'm also stuck in the way
I can from a spreadsheet :
- Create a new tasklist
- Create a new task in a dedicated tasklist (with due date and notes)
I can from the Gtasks :
- Check if the task is completed and mark it as completed in the spreadsheet
- Check if the task still exists in the spreadsheet and remove it if necessary
I'm still looking for a way to make a task completed in GTasks when it's closed in spreadsheet
All the functionality exists for you to accomplish this, but I don't know if there is a pre-built script out there that does what you want. You may want to look into use a Google Form that saves data to the spreadsheet, and then create a trigger for form submit that scoops up the data and creates a new task using it.
Is this [part] of what you're looking for?
https://developers.google.com/apps-script/articles/google_apis_reading_list
It syncs a Spreadsheet based task list with a your regular Task List, and if you mark the task done in gmail, it records that back in the spreadsheet.
// Fetch the list of URLs to keep synchronized
var articleUrls = SpreadsheetApp.getActiveSheet().getRange("A2:A");
for (var rowNum = 0; rowNum < articleUrls.getNumRows(); rowNum++) {
// Limit our range to a single cell containing a URL
var oneUrlCell = articleUrls.offset(rowNum, 0, 1, 1);
if (oneUrlCell.getComment() === "") {
// This is a new URL that needs to be shortened/inserted
var urlText = oneUrlCell.getValue();
if (urlText !== "") {
// Shorten the URL
Logger.log("Adding task for url: " + urlText);
var toShorten = UrlShortener.newUrl().setLongUrl(urlText);
var shortened = UrlShortener.Url.insert(toShorten);
// Insert the shortened URL into our reading list
var taskToInsert = Tasks.newTask().setTitle(shortened.getId());
taskToInsert.setNotes(urlText);
var newTask = Tasks.Tasks.insert(taskToInsert, readingListId);
// Save the new ID as our comment.
oneUrlCell.setComment(newTask.getId());
}
} else {
// This URL has already been inserted, update the status
var existingTask = Tasks.Tasks.get(readingListId, oneUrlCell.getComment());
if (existingTask.getStatus() === "completed") {
var absRowNum = oneUrlCell.getRow();
var completedCell = sheet.getRange(absRowNum, 2);
completedCell.setValue("Yes");
}
}
Should be part of the solution, no?
I'm looking to make something a bit bigger myself.

Adding new output column to custom data flow component SSIS

I'm trying to add new output column using synchronous custom data flow component(below is the code). While testing I found that input columns I added are not getting displayed in Output columns only the new added column is getting displayed. I'm not sure where problem is. Please help!
Public Overloads Overrides Sub ProvideComponentProperties()
Dim input As IDTSInput100 = ComponentMetaData.InputCollection.New()
input.Name = "Input"
Dim Output As IDTSOutput100 = ComponentMetaData.OutputCollection.New()
Output.Name = "Output Rows"
Output.Description = "Output rows with unique row ID appended."
'Adds new column RowID to output columns list
Dim rowIDColumn As IDTSOutputColumn100 = Output.OutputColumnCollection.[New]
rowIDColumn.Name = "Row ID"
rowIDColumn.SetDataTypeProperties(DataType.DT_UI8, 0, 0, 0, 0)
Output.SynchronousInputID = input.ID
Thanks in advance
Sai
my bad! for Data flow components only new columns will be displayed in the advance editor, but when we take the output arrow and attach to destination(like excel or file or db) then it shows all the columns including the input columns and newly added columns in the mapping tab of destination editor. :)