dynamic SQL execution and saving the result in flat file in SSIS - ssis

Question

I want to create a SSIS package which writes a file with data generated by executing a SQL Statement. This generic package will be invoked by other packages passing in correct SQL as a variable.
Thus in the generic package :
I want to execute a dynamic SELECT query and fetch dynamic number of columns from a single database instance, the connection string does not per call and store the result into a flat file.
What would be an ideal way to accomplish this in SSIS.
What I tried :
The simplest solution that I could find was a writing a script task which would open a SQL connection , execute the SQL using SQLCommand, populate a datatable using the data fetched and write the contents directly to the file system using System.io.File and Release the connection.
I tried using OLE Database source with the SQLsupplied by a variable (with Validation set to false) and directing the rows into a Flat file connection. However due to the dynamic number and names of the columns I ran into errors.
Is there a more standard way of achieving this without using a script task?

Answer 1

How about this ... concatenate all field values into one field, and map AllFields to a field in a text file destination.
SELECT [f1]+',' + [f2] AS AllFields FROM [dbo].[A]
All of the "other"packages will know how to create the correct SQL. Their only contract with the "generic" package would be to eventually have only one field nameed "AllFields".

Answer 2

To answer your question directly, I do not think there is a "standard" way to do this. I believe the solution from Anoop would work well and while I have not tested the idea I wish I would have investigated it before writing my own solution. You should not need a script task in that solution...
In any case, I did write my own way to generate csv files from SQL tables that may run up against edge cases and need polishing but works rather well right now. I am looping through multiple tables before this task so the CurrentTable variable can be replaced with any variable you want.
Here is my code:
public void Main()
{
string datetime = DateTime.Now.ToString("yyyyMMddHHmmss");
try
{
string TableName = Dts.Variables["User::CurrentTable"].Value.ToString();
string FileDelimiter = ",";
string TextQualifier = "\"";
string FileExtension = ".csv";
//USE ADO.NET Connection from SSIS Package to get data from table
SqlConnection myADONETConnection = new SqlConnection();
myADONETConnection = (SqlConnection)(Dts.Connections["connection manager name"].AcquireConnection(Dts.Transaction) as SqlConnection);
//Read data from table or view to data table
string query = "Select * From [" + TableName + "]";
SqlCommand cmd = new SqlCommand(query, myADONETConnection);
//myADONETConnection.Open();
DataTable d_table = new DataTable();
d_table.Load(cmd.ExecuteReader());
//myADONETConnection.Close();
string FileFullPath = Dts.Variables["$Project::ExcelToCsvFolder"].Value.ToString() + "\\Output\\" + TableName + FileExtension;
StreamWriter sw = null;
sw = new StreamWriter(FileFullPath, false);
// Write the Header Row to File
int ColumnCount = d_table.Columns.Count;
for (int ic = 0; ic < ColumnCount; ic++)
{
sw.Write(TextQualifier + d_table.Columns[ic] + TextQualifier);
if (ic < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
// Write All Rows to the File
foreach (DataRow dr in d_table.Rows)
{
for (int ir = 0; ir < ColumnCount; ir++)
{
if (!Convert.IsDBNull(dr[ir]))
{
sw.Write(TextQualifier + dr[ir].ToString() + TextQualifier);
}
if (ir < ColumnCount - 1)
{
sw.Write(FileDelimiter);
}
}
sw.Write(sw.NewLine);
}
sw.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception exception)
{
// Create Log File for Errors
//using (StreamWriter sw = File.CreateText(Dts.Variables["User::LogFolder"].Value.ToString() + "\\" +
// "ErrorLog_" + datetime + ".log"))
//{
// sw.WriteLine(exception.ToString());
//}
Dts.TaskResult = (int)ScriptResults.Failure;
throw;
}
Dts.TaskResult = (int)ScriptResults.Success;