I want to accomplish a fairly simple task (I'd think).
I have one table with a shiftid (INT), shiftstart (datetime), shiftend (datetime).
I'd like to query that table, then run a query (on an entirely different database) that asks for production (which is calculated in an odd way - requiring three separate queries) using the start and end times, and store that in the original database with the shiftid and a production amount for the shift.
I've tried to do this using a Foreach Loop and a script task that builds a variable that would contain the query, but I'm continually hitting a brick wall there.
Dts.Variables("User::SQLshiftstart").Value = "SELECT value FROM[dbo].[AnalogHistory] WHERE TagName = 'Z_HISTFMZ_P2_0004' AND DateTime = '" & Dts.Variables("User::shiftstart").ToString
I keep getting an error - "Command text was not set for the command object". And googling that error doesn't push me any further down the path.
Help!
Well, I decided to go a different way instead of using a script object to build a variable. I actually created the variable in my SELECT:
SELECT (CONCAT
('SELECT CAST(value AS DECIMAL(10,4)) AS beg FROM [dbo].[AnalogHistory] WHERE TagName = ''Z_HISTDATA_P1_0007'' AND DateTime = '' ',
DateAdd(hh,-6,shiftstart),
' '' AND wwTimeZone = ''UTC'' '))
This way, I avoid having to build an intermediate script object and can directly query based on the variable name in my FOREACH loop.
Related
I want to insert values into database when the biml code is ran and the package has completed expansion is this possible using BIML or c#?
I have a table called BIML expansion created in my DB and I have test.biml which loads the package test.dtsx whenever the BIML expansion is completed a record should be inserted into my table that expansion has been completed.
Let me know if you have any questions or needs any additional info.
From comments
I tried your code
string connectionString = "Data Source=hq-dev-sqldw01;Initial Catalog=IM_Stage;Integrated Security=SSPI;Provider=SQLNCLI11.1";
string SrcTablequery=#"INSERT INTO BIML_audit (audit_id,Package,audit_Logtime) VALUES (#audit_id, #Package,#audit_Logtime)";
DataTable dt = ExternalDataAccess.GetDataTable(connectionString,SrcTablequery);
It has an error below must declare the scalar variable audit_id can you let me know the issue behind it?
In it's simplest form, you'd have content like this in your Biml script
// Define the connection string to our database
string connectionStringSource = #"Server=localhost\dev2012;Initial Catalog=AdventureWorksDW2012;Integrated Security=SSPI;Provider=SQLNCLI11.1";
// Define the query to be run after *ish* expansion
string SrcTableQuery = #"INSERT INTO dbo.MyTable (BuildDate) SELECT GETDATE()";
// Run our query, nothing populates the data table
DataTable dt = ExternalDataAccess.GetDataTable(connectionStringSource, SrcTableQuery);
Plenty of different ways to do this - you could have spun up your own OLE/ADO connection manager and used the class methods. You could have pulled the connection string from the Biml Connections collection (depending on the tier this is executed in), etc.
Caveats
Depending on the product (BimlStudio vs BimlExpress), there may be a background process compiling your BimlScript to ensure all the metadata is ready for intellisense to pick it up. You might need to stash that logic into a very high tiered Biml file to ensure it's only called when you're ready for it. e.g.
<## template tier="999" #>
<#
// Define the connection string to our database
string connectionStringSource = #"Server=localhost\dev2012;Initial Catalog=AdventureWorksDW2012;Integrated Security=SSPI;Provider=SQLNCLI11.1";
// Define the query to be run after *ish* expansion
string SrcTableQuery = #"INSERT INTO dbo.MyTable (BuildDate) SELECT GETDATE()";
// Run our query, nothing populates the data table
DataTable dt = ExternalDataAccess.GetDataTable(connectionStringSource, SrcTableQuery);
#>
Is that the problem you're trying to solve?
Addressing comment/questions
Given the query of
string SrcTablequery=#"INSERT INTO BIML_audit (audit_id,Package,audit_Logtime) VALUES (#audit_id, #Package,#audit_Logtime)";
it errors out due to #audit_id not being specified. Which makes sense - this query specifies it will provide three variables and none are provided.
Option 1 - the lazy way
The quickest resolution would be to redefine your query in a manner like this
string SrcTablequery=string.Format(#"INSERT INTO BIML_audit (audit_id,Package,audit_Logtime) VALUES ({0}, '{1}', '{2})'", 123, "MyPackageName", DateTime.Now);
I use the string library's Format method to inject the actual values into the placeholders. I assume that audit_id is a number and the other two are strings thus the tick marks surrounding 1 and 2 there. You'd need to define a value for your audit id but I stubbed in 123 as an example. If I were generating packages, I'd likely have a variable for my packageName so I'd reference that in my statement as well.
Option 2 - the better way
Replace the third line with .NET library usage much as you see in heikofritz on using parameters inserting data into access database.
1) Create a database Connection
2) Open connection
3) Create a command object and associate with the connection
4) Specify your statement (use ? as your ordinal marker instead of named parameters since this is oledb)
5) Create an Parameter list and associate with values
Many, many examples out there beyond the referenced but it was the first hit. Just ignore the Access connection string and use your original value.
I have a table tblItems with a list of inventory items. The table has many columns to describe these items, including columns for SupplierName, SupplierOrderNumber and PredictedArrivalDate.
If I order several new items from a supplier, I will record each item separately in the table with the same supplier name, order number and a predicted arrival date.
I would like to add a data macro, so that if I update the PredictedArrivalDate for one record, the value will be copied to the PredictedArrivalDate column of other records/items with the same SupplierName AND SupplierOrderNumber.
The closest I've got is:
SetLocalVar (MySupplierName, [SupplierName])
SetLocalVar (MySupplierOrderNumber , [SupplierOrderNumber ])
SetLocalVar (MyPredictedArrivalDate, [PredictedArrivalDate])
For Each Record in tblItems
Where Condition = [SupplierOrderNumber] Like [MySupplierOrderNumber] And [SupplierName] Like [MySupplierName] And [PredictedArrivalDate]<>[MyPredictedArrivalDate]
Alias OtherRecords
EditRecord
SetField ([OtherRecords].[PredictedArrivalDate], [MyPredictedArrivalDate])
End EditRecord
However, when I run this, only 5 records update, and the error log reports error -20341:
"A data macro resource limit was hit. This may be caused by a data
macro recursively calling itself. The Updated() function may be
used to detect which field in a record has been updated to help
prevent recursive calls."
How can I get this working?
I'm not one for using macro's to do anything, so I'd use VBA and recordsets/an action query to do the updating.
You can call a user-defined function inside a data macro by setting a local var equal to its result.
Access doesn't like data macros triggering themselves (which you are doing, you're using an on update macro and updating fields in the same table on a different record), because there is a risk of accidentally creating endless loops. Looks like you triggered a measure that's made to prevent this. I'd try to avoid that as much as possible.
Note: using user-defined functions inside data macros can cause problems when you're linking to the table from outside of Access (via ODBC for example).
This isn't a good solution (it's not a data macro), but it does work as a temporary fix.
I created an update query called "updatePredictedArrivalDate":
PARAMETERS
ItemID Long,
MyPredictedArrivalDate DateTime,
MySupplierName Text ( 255 ),
MySupplierOrderNumber Text ( 255 );
UPDATE tblItems
SET tblItems.PredictedArrivalDate = [MyPredictedArrivalDate]
WHERE (((tblItems.SupplierName) = [MySupplierName])
AND ((tblItems.SupplierOrderNumber) = [MySupplierOrderNumber])
AND ((tblItems.ID) <> [ItemID]));
On the PredictedArrivalDate form field .AfterUpdate event, I then added this macro:
IF [PredictedArrivalDate].[OldValue]<>[PredictedArrivalDate] Or [PredictedArrivalDate]<>""
OpenQuery (updatePredictedArrivalDate, Datasheet, Edit, [ID], [PredictedArrivalDate], [SupplierName], [SupplierOrderNumber])
I now have to remember to add this .AfterUpdate event to any other forms I create that amend that particular field.
If anyone has a better solution, please let me know.
I have a sub in Access 2010 that references a query. They're supposed to run through three columns and find the lowest one. When I run it, I get an error: "The expression you entered as a query parameter produced this error: 'Projects.ProjectID'
Sub:
Private Sub UpdatePriority_Click()
Overall_Priority = DMin("MinvonGeoPri", "qryOverallPriority", "Projects.ProjectId=1")
End Sub
Query:
SELECT
Min(Projects.GeoPavePri) AS MinvonGeoPri
, Min(Projects.StrPri) AS MinvonStrPri
, Min(Projects.SOPri) AS MinvonSOPri
, Projects.ProjectId
FROM
Projects
WHERE
Projects.ProjNo=Activity.ProjNo;
Google suggested that I add quotations around 1, so I changed it to "Projects.ProjectID=" & 1 & "", but it didn't help. I've double checked spelling, field names, and I'm running out of ideas. Any suggestions would be great.
Once you have created a qryOverallPriority from the select statement, referencing the Projects.[ProjectId] field is simply [ProjectId]. In short, you lose the ability to reference the parent table although it could be referenced as qryOverallPriority.[ProjectId] but that is not necessary..
Private Sub UpdatePriority_Click()
Overall_Priority = DMin("MinvonGeoPri", "qryOverallPriority", "ProjectId=1")
End Sub
Any field pulled by a saved query is referenced by how that save query sees it, not from the underlying SQL select statement that made up the saved query. Likewise, aliased fields would be referenced by their aliases; e.g. [ProjectId] and [ProjectId2] for both [ProjectId] fields in a join.
You are also using the aggregate MIN function without a GROUP BY clause. Use the Access front end to make sure that you are getting the results you want. Perhaps something like,
SELECT
Min(p.GeoPavePri) AS MinvonGeoPri
,Min(p.StrPri) AS MinvonStrPri
,Min(p.SOPri) AS MinvonSOPri
,p.ProjectId
FROM
Projects p
WHERE
p.ProjNo=Activity.ProjNo;
GROUP BY p.GeoPavePri, p.StrPri, p.SOPri, p.ProjectId
You should be able to use,
Overall_Priority = DLookup("MinvonGeoPri", "qryOverallPriority", "ProjectId=1")
That is untested. I have not built a full test environment and I have no idea where Activity.ProjNo comes from.
In my SSIS package I have a dataflow that looks something like this.
My requirement is to log the end time of each flatfile destination (Or the time when each of the flat files is created) , in a SQL server table. To be more clear, there will be one row per flatfile in the log table. Is there any simple way(preferably) to accomplish this? Thanks in advance.
Update: I ended up using a script task after the dataflow and read the creation time of each of the file created in the dataflow. I also used same script task to insert logs into the table, just to keep things in one place. For details refer the post masked as answer.
In order to get the accurate date and timestamp of each flat file created as the destination, you'll need to create three new global variables and set up a for-each loop container in the control flow following your current data flow task and then add to the for-each loop container a script task that will read from one flat file at a time the date/time information. That information will then be saved to one of the new global variables that can then be applied in a second SQL task (also in the for-each loop) to write the information to a database table.
The following link provides a good example of the steps you'll need to apply. There are a few extra steps not applicable that you can easily exclude.
http://microsoft-ssis.blogspot.com/2011/01/use-filedates-in-ssis.html
Hope this helps.
After looking more closely at the toolbox, I think the best way to do this is to move each source/destination pairing into its own dataflow and use the OnPostExecute event of each dataflow to write to the SQL table.
Wanted to provide more detail to #TabAlleman's approach.
For each control flow task with a name like Bene_hic, you will have a source file and a destination file.
On the 'Event Handlers' tab for that executable (use the drop-down list,) you can create the OnPostExecute event.
In that event, I have two SQL tasks. One generates the SQL to execute for this control flow task, the second executes the SQL.
These SQL tasks are dependent on two user variables scoped in the OnPostExecute event. The EvaluateAsExpression property for both is set to True. The first one, Variable1, is used as a template for the SQL to execute and has a value like:
"SELECT execSQL FROM db.Ssis_onPostExecute
where stgTable = '" + #[System::SourceName] + "'"
#[System::SourceName] is an SSIS system variable containing the name of the control flow task.
I have a table in my database named Ssis_onPostExecute with two fields, an execSQL field with values like:
DELETE FROM db.TableStats WHERE TABLENAME = 'Bene_hic';
INSERT INTO db.TableStats
SELECT CreatorName ,t.tname, CURRENT_TIMESTAMP ,rcnt FROM
(SELECT databasename, TABLENAME AS tname, CreatorName FROM dbc.TablesV) t
INNER JOIN
(SELECT 'Bene_hic' AS tname,
COUNT(*) AS rcnt FROM db.Bene_hic) u ON
t.tname = u.tname
WHERE t.databasename = 'db' AND t.tname = 'Bene_hic';
and a stgTable field with the name of the corresponding control flow task in the package (case-sensitive!) like Bene_hic
In the first SQL task (named SQL,) I have the SourceVariable set to a user variable (User::Variable1) and the ResultSet property set to 'single row.' The Result Set detail includes a Result Name = 0 and Variable name as the second user variable (User::Variable2.)
In the second SQL task (exec,) I have the SQLSourceType property set to Variable and the SourceVariable property set to User::Variable2.
Then the package is able to copy the data in the source object to the destination, and whether it fails or not, enter a row in a table with the timestamp and number of rows copied, along with the table name and anything else you want to track.
Also, when debugging, you have to run the whole package, not just one task in the event. The variables won't be set correctly otherwise.
HTH, it took me forever to figure all this stuff out, working from examples on several web sites. I'm using code to generate the SQL in the execSQL field for each of the 42 control flow tasks, meaning I created 84 user variables.
-Beth
The easy solution will be:
1) drag the OLE DB Command from the tool box after the Fatfile destination.
2) Update Script to update table with current date when Flat file destination is successful.
3) You can create a variable (scope is project) with value systemdatetime.
4) You might have to create another variable depending on your package construct if Success or fail
If I have the following query, is it possible to be able to run a function inside? Let's say I want to add WHERE zip_code = user_distance(zip_code)?
I want to take data from each row and run it through a function before actually selecting it.
#posts = Listing.find_by_sql(["SELECT * FROM listings WHERE industry = ? && ", current_user.industry])
If you are mainly looking to get this working and not worrying so much about performance (because going straight to the SQL is faster than going through ActiveRecord) then you could do:
listings = []
Listing.all.each do |listing|
listings << listing if user_distance(listing.zip_code)
end
So, it will go through each listing and add it to that array if the user_distance method returns true (or however it is set up).
Another thing you could do is set up a stored procedure ("stored proc") on your database that takes in a zip code and returns what it is you want (i.e, does the same thing as user_distance), and that user defined variable max_distance could be in a database table so it's accessible to your stored procedure. Then you could call that stored proc from the SQL and still be able to pass in the zip_code of each row.