I am getting inconsistent data in the List, the data in the list added in the Rule 1. Rule 2 modifies the object which holds the list, due to the update/modify the Rules get called repeatedly.
rule "Rule 1"
dialect "java"
no-loop true
when
$notification:NotificationVO();
$snrData : SensorDataVO(getOffsetChngesInterval()!=null,
getOffsetChngesInterval().size()>0);
$pestWeight:Integer() from $snrData.getOffsetChngesInterval();
$masterData: MasterDataVO($pestWeight==rodentWeight);
then
System.out.println("Rule 2");
modify($notification){getFilteredMasterData().add($masterData)};
end
rule "Rule 2"
dialect "java"
no-loop true
when
$notification: NotificationVO(getFilteredMasterData()!=null,
getFilteredMasterData().size()>3,
getReasoningData().size()==0);
then
System.out.println("Rule 3");
modify($notification) {
getReasoningData().put($notification.getFilteredMasterData().get(0)
.getRodentWeight(),
$notification.getFilteredMasterData().get(0))
}
end
Can you please tell me what is going wrong.
It's quite simple: whenever a fact is modified, all rules referencing that fact are evaluated again, and if the condition is true, they result in another activation.
To break this vicious circle, you have several options, but using no-loop on both rules isn't one of them.
First, you can add the condition that one of the rules establishes by its right hand side isn't given in the fact data. So, as Rule 1 adds $masterData to the list filteredMasterData: add the constraint that this element is absent from the list:
when
$notification:NotificationVO( $fmd: filteredMasterData)
$snrData : SensorDataVO(getOffsetChngesInterval()!=null,
getOffsetChngesInterval().size()>0)
$pestWeight:Integer() from $snrData.getOffsetChngesInterval()
$masterData: MasterDataVO($pestWeight==rodentWeight)
eval( ! $fmd.contains( $masterData ) )
then
Another possibility depends on the use of the Map reasoningData that is updated in Rule 2. If this is not used in any condition, you can simply modify the Map without calling update or modify, doing (what I call) a "dirty update".
Related
I have three validators each sequentially dependent on the previous. I might use all three or only the first two or only the first one. E.g.
Validator 1: Checks whether user exists
Validator 2: Checks whether requested order exists in the user's order history
Validator 3: Checks whether that order has a cancel option
Each validator in this list will throw an error if the previous condition is not met. I'm currently using this pattern:
function validator1() { ... }
function validator2() { validator1(); ... }
function validator3() { validator2(); ... }
But this feels like an anti-pattern to me because all of the prerequisite validation is done implicitly so when I run validtor2(), it is not clear that validator1() is also being run.
I also considered doing this:
function validator1() { ... }
function validator2() { ... }
function validator3() { ... }
function someTask() { validator1(); validator2(); validator3(); ... }
But this is unsafe because there is nothing stopping me from running validator3() and throwing an error if condition 2 hasn't been met.
Is there a way to do this that is both safe and explicit?
Edit
This is a great reference for the design pattern mentioned in the accepted answer: https://refactoring.guru/design-patterns/chain-of-responsibility
I suggest looking into the Chain of Responsibility and the Builder Pattern specific for validation.
See here:
https://medium.com/#aleksanderkolata/java-tips-01-handle-validation-in-an-elegant-manner-using-chain-of-responsibility-design-pattern-ad3dcc22479e
https://medium.com/henrydchan/input-validation-using-design-patterns-9d7b96f87702
You could also look into usage of the Decorator Pattern:
https://www.google.com/amp/s/robdodson.me/javascript-design-patterns-decorator/amp/
Is there a way to do this that is both safe and explicit?
But before considering one of the patterns which might of course provide more flexibility but also more complexity consider if your rules might really get extended a lot and will be reused and that many cases. Otherwise it might be a bit of over engineering to go with one of the suggested patterns right away.
Also, you can make a lot explizit by using meaningful function names that explicitly tell the reader of the code what specific use case is being performed. Such a function than acts more like an orchestrator of the required validation steps. So your approach (despite of the bad name someTask()) might already fit your needs.
But this is unsafe because there is nothing stopping me from running validator3() and throwing an error if condition 2 hasn't been met.
You could either have each validate exception throw an exception or introduce guard clauses having the major task return if one of the validations fail.
Another option would be to pass in or register a list of validations with the same interface and loop them through. If you need to check for all validations and collect the results (e.g. error code or messages) for each validation step or throw an exception on the first validation is up to your needs.
How is setted the property descriptor for the built-in method? For example for isPrototypeOf? I did not find such a place in the specification. Rather found, but there is no description of the attribute [[Value]]. It sounds like this:
"Every other data property is described in clauses 18 through 26 and
in Annex B.2 has the attributes {[[Writable]]: true, [[Enumerable] ]:
false, [[Configurable]]: true} unless otherwise specified"
Now we know what the other attributes will have, but we do not know what value will be set in [[Value]]. You can say that the value in [[Value]] will be set to the function object and you will be right. However, I want to know by specification where this is the place where for built-in properties is set [[Value]]
However, I want to know by specification where this is the place where for built-in properties is set [[Value]]
The specification doesn't have algorithmic steps for initializing the global objects, which makes sense since they are expected to exist before execution of any code actually begins.
Instead of simply states what structure should exist, so for instance in 19.2.3 Properties of the Function Prototype Object, it simply states
19.2.3.1 Function.prototype.apply ( thisArg, argArray )
in the header with the expectation that it is clear that it means the apply function is a property of the %FunctionPrototype% global object. Given that, there are only two types of properties,
Accessor properties
Data properties
Data properties are explicitly simple properties that contain a value. Function.prototype.apply doesn't have get or set prefix, or use an ## prefix), which means that this statement is specifying a data string-keyed data property. By definition, this means that it is clear that the [[Value]] is the function object itself, because there is nowhere else to put it, and [[Value]] is the clear correct choice.
The section you linked in 17 ECMAScript Standard Built-in Objects states
Every other data property described in clauses 18 through 26 and in Annex B.2 has the attributes { [[Writable]]: true, [[Enumerable]]: false, [[Configurable]]: true } unless otherwise specified.
is just filling in the rest of the data property fields, since it wouldn't make sense to repeat them for every single method.
I have three dropdown boxes on a Main_Form. I will add the chosen content into three fields on the form, Form_Applications.
These three lines are added :
Form_Applications.Classification = Form_Main_Form.Combo43.Value
Form_Applications.Countryname_Cluster = Form_Main_Form.Combo56.Value
Form_Applications.Application = Form_Main_Form.Combo64.Value
The first two work perfectly but the last one gives error code 438!
I can enter in the immediate window :
Form_Applications.Classification = "what ever"
Form_Applications.Countryname_Cluster = "what ever"
but not for the third line. Then, after enter, the Object doesn't support this property or method error appears.
I didn't expect this error as I do exactly the same as in the first two lines.
Can you please help or do you need more info ?
In VBA Application is a special word and should not be used to address fields.
FormName.Application will return an object that points to the application instance that is running that form as opposed to an object within that form.
From the Application object you can do all sorts of other things such as executing external programs and other application level stuff like saving files/
Rename your Application field to something else, perhaps ApplicationCombo and change your line of code to match the new name. After doing this the code should execute as you expect.
Form_Applications.Application is referring to the application itself. It is not a field, so therefore it is not assignable (at least with a string).
You really haven't provided enough code to draw any real conclusions though. But looking at what you have posted, you definitely need to rethink your approach.
It's to say definitely but you are not doing the same. It looks like you are reading a ComboBox value the same (I will assume Combo64 is the same as 43 and 56) but my guess is that what you are assigning that value to is the problem:
Form_Applications.Application =
Application is not assignable. Is there another field you meant to use there?
What is the simple way to find the column name from Lineageid in SSIS. Is there any system variable avilable?
I remember saying this can't be that hard, I can write some script in the error redirect to lookup the column name from the input collection.
string badColumn = this.ComponentMetaData.InputCollection[Row.ErrorColumn].Name;
What I learned was the failing column isn't in that collection. Well, it is but the ErrorColumn reported is not quite what I needed. I couldn't find that package but here's an example of why I couldn't get what I needed. Hopefully you will have better luck.
This is a simple data flow that will generate an error once it hits the derived column due to division by zero. The Derived column generates a new output column (LookAtMe) as the result of the division. The data viewer on the Error Output tells me the failing column is 73. Using the above script logic, if I attempted to access column 73 in the input collection, it's going to fail because that is not in the collection. LineageID 73 is LookAtMe and LookAtMe is not in my error branch, it's only in the non-error branch.
This is a copy of my XML and you can see, yes, the outputColumn id 73 is LookAtme.
<outputColumn id="73" name="LookAtMe" description="" lineageId="73" precision="0" scale="0" length="0" dataType="i4" codePage="0" sortKeyPosition="0" comparisonFlags="0" specialFlags="0" errorOrTruncationOperation="Computation" errorRowDisposition="RedirectRow" truncationRowDisposition="RedirectRow" externalMetadataColumnId="0" mappedColumnId="0"><properties>
I really wanted that data though and I'm clever so I can union all my results back together and then conditional split it back out to get that. The problem is, Union All is an asynchronous transformation. Async transformations result in the data being copied from one set of butters to another resulting in...new lineage ids being assigned so even with a union all bringing the two streams back together, you wouldn't be able to call up the data flow chain to find that original lineage id because it's in a different buffer.
Around this point, I conceded defeat and decided I could live without intelligent/helpful error reporting in my packages.
I know this is a long dead thread but I tripped across a manual solution to this problem and thought I would share for anyone who happens upon this same problem. Granted this doesn't provide a programmatic solution to the problem but for simple debugging it should do the trick. The solution uses a Derived Column as an example but this seems to work for any Data Flow component.
Answer provided by Todd McDermid and taken from AskSQLServerCentral:
"[...] Unfortunately, the lineage ID of your columns is pretty well hidden inside SSIS. It's the "key" that SSIS uses to identify columns. So, in order to figure out which column it was, you need to open the Advanced Editor of the Derived Column component or Data Conversion. Do that by right clicking and selecting "Advanced Editor". Go to the "Input and Output Properties" tab. Open the first node - "Derived Column Input" or "Data Conversion Input". Open the "Input Columns" tab. Click through the columns, noting the "LineageID" property of each. You may have to do the same with the "Derived Column Output" node, and "Output Columns" inside there. The column that matches your recorded lineage ID is the offending column."
For anyone using SQL Server versions before SS2016, here are a couple of reference links for a way to get the Column name:
http://www.andrewleesmith.co.uk/2017/02/24/finding-the-column-name-of-an-ssis-error-output-error-column-id/
which is based on:
http://toddmcdermid.blogspot.com/2016/04/finding-column-name-for-errorcolumn.html
I appreciate we aren't supposed to just post links, but this solution is quite convoluted, and I've tried to summarise by pulling info from both Todd and Andrew's blog posts and recreating them here. (thank you to both if you ever read this!)
From Todd's page:
Go to the "Inputs and Outputs" page, and select the "Output 0" node.
Change the "SynchronousInputID" property to "None". (This changes
the script from synchronous to asynchronous.)
On the same page, open the "Output 0" node and select the "Output
Columns" folder. Press the "Add Column" button. Change the "Name"
property of this new column to "LineageID".
Press the "Add Column" button again, and change the "DataType"
property to "Unicode string [DT_WSTR]", and change the "Name"
property to "ColumnName".
Go to the "Script" page, and press the "Edit Script" button. Copy
and paste this code into the ScriptMain class (you can delete all
other method stubs):
public override void CreateNewOutputRows() {
IDTSInput100 input = this.ComponentMetaData.InputCollection[0];
if (input != null)
{
IDTSVirtualInput100 vInput = input.GetVirtualInput();
if (vInput != null)
{
foreach (IDTSVirtualInputColumn100 vInputColumn in vInput.VirtualInputColumnCollection)
{
Output0Buffer.AddRow();
Output0Buffer.LineageID = vInputColumn.LineageID;
Output0Buffer.ColumnName = vInputColumn.Name;
}
}
} }
Feel free to attach a dummy output to that script, with a data viewer,
and see what you get. From here, it's "standard engineering" for you
ETL gurus. Simply merge join the error output of the failing
component with this metadata, and you'll be able to transform the
ErrorColumn number into a meaningful column name.
But for those of you that do want to understand what the above script
is doing:
It's getting the "first" (and only) input attached to the script
component.
It's getting the virtual input related to the input. The "input" is
what the script can actually "see" on the input - and since we
didn't mark any columns as being "ReadOnly" or "ReadWrite"... that
means the input has NO columns. However, the "virtual input" has
the complete list of every column that exists, whether or not we've
said we're "using" it.
We then loop over all of the "virtual columns" on this virtual
input, and for each one...
Get the LineageID and column name, and push them out as a new row on
our asynchronous script.
The image and text from Andrew's page helps explain it in a bit more detail:
This map is then merge-joined with the ErrorColumn lineage ID(s)
coming down the error path, so that the error information can be
appended with the column name(s) from the map. I included a second
script component that looks up the error description from the error
code, so the error table rows that we see above contain both column
names and error descriptions.
The remaining component that needs explaining is the conditional split
– this exists just to provide metadata to the script component that
creates the map. I created an expression (1 == 0) that always
evaluates to false for the “No Rows – Metadata Only” path, so no rows
ever travel down it.
Whilst this solution does require the insertion of some additional
plumbing within the data flow, we get extremely valuable information
logged when errors do occur. So especially when the data flow is
running unattended in Production – when we don’t have the tools &
techniques available at design time to figure out what’s going wrong –
the logging that results gives us much more precise information about
what went wrong and why, compared to simply giving us the failed data
and leaving us to figure out why it was rejected.
There is no simple way to find out column name by lineage id.
If you want to know this using BIDS You have to inspect all components inside dataflow using Advanced properties, Input and Output columns tab and see LineageID for each column and input/output path.
But You can:
inspect XML - this is very difficult
write .NET application and use FindColumnByLineageId
However, second solution includes a lot of coding and understanding of pipeline because You have to programmaticaly: open the package, iterate over tasks, iterate inside containers, iterate over transformations inside data flows to find particular component to use proposed method.
Here is a solution that:
Works at package runtime (not pre-populating)
Is automated through a Script Task and Component
Doesn't involve installing new assemblies or custom components
Is nicely BIML compatible
Check out the full solution here.
EDIT
Here is the short version.
Create 2 Object variables, execsObj and lineageIds
Create Script Task in Control flow, give it ReadWrite access to both variables
Add an assembly reference to your script task for Microsoft.SqlServer.DTSPipelineWrap.dll (may required SQL Client SDK be installed; required for the MainPipe object below)
Insert the following code into your Script Task
Dictionary<int, string> lineageIds = null;
public void Main()
{
// Grab the executables so we have to something to iterate over, and initialize our lineageIDs list
// Why the executables? Well, SSIS won't let us store a reference to the Package itself...
Dts.Variables["User::execsObj"].Value = ((Package)Dts.Variables["User::execsObj"].Parent).Executables;
Dts.Variables["User::lineageIds"].Value = new Dictionary<int, string>();
lineageIds = (Dictionary<int, string>)Dts.Variables["User::lineageIds"].Value;
Executables execs = (Executables)Dts.Variables["User::execsObj"].Value;
ReadExecutables(execs);
Dts.TaskResult = (int)ScriptResults.Success;
}
private void ReadExecutables(Executables executables)
{
foreach (Executable pkgExecutable in executables)
{
if (object.ReferenceEquals(pkgExecutable.GetType(), typeof(Microsoft.SqlServer.Dts.Runtime.TaskHost)))
{
TaskHost pkgExecTaskHost = (TaskHost)pkgExecutable;
if (pkgExecTaskHost.CreationName.StartsWith("SSIS.Pipeline"))
{
ProcessDataFlowTask(pkgExecTaskHost);
}
}
else if (object.ReferenceEquals(pkgExecutable.GetType(), typeof(Microsoft.SqlServer.Dts.Runtime.ForEachLoop)))
{
// Recurse into FELCs
ReadExecutables(((ForEachLoop)pkgExecutable).Executables);
}
}
}
private void ProcessDataFlowTask(TaskHost currentDataFlowTask)
{
MainPipe currentDataFlow = (MainPipe)currentDataFlowTask.InnerObject;
foreach (IDTSComponentMetaData100 currentComponent in currentDataFlow.ComponentMetaDataCollection)
{
// Get the inputs in the component.
foreach (IDTSInput100 currentInput in currentComponent.InputCollection)
foreach (IDTSInputColumn100 currentInputColumn in currentInput.InputColumnCollection)
lineageIds.Add(currentInputColumn.ID, currentInputColumn.Name);
// Get the outputs in the component.
foreach (IDTSOutput100 currentOutput in currentComponent.OutputCollection)
foreach (IDTSOutputColumn100 currentoutputColumn in currentOutput.OutputColumnCollection)
lineageIds.Add(currentoutputColumn.ID, currentoutputColumn.Name);
}
}
Create Script Component in Dataflow with ReadOnly access to lineageIds and the following code.
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
Dictionary<int, string> lineageIds = (Dictionary<int, string>)Variables.lineageIds;
int? colNum = Row.ErrorColumn;
if (colNum.HasValue && (lineageIds != null))
{
if (lineageIds.ContainsKey(colNum.Value))
Row.ErrorColumnName = lineageIds[colNum.Value];
else
Row.ErrorColumnName = "Row error";
}
Row.ErrorDescription = this.ComponentMetaData.GetErrorDescription(Row.ErrorCode);
}
I've noticed something about my coding that is slightly undefined. Say we have a two dimension array, a matrix or a table and we are looking through it to check if a property is true for every row or nested dimension.
Say I have a boolean flag that is to be used to check if a property is true or false. My options are to:
Initialize it to true and check each cell until proven false. This
gives it a wrong name until the code
is completely executed.
Start on false and check each row until proven true. Only if all rows are true will the data be correct. What is the cleanest way to do this, without a counter?
I've always done 1 without thinking but today it got me wondering. What about 2?
Depends on which one dumps you out of the loop first, IMHO.
For example, with an OR situation, I'd default to false, and as soon as you get a TRUE, return the result, otherwise return the default as the loop falls through.
For an AND situation, I'd do the opposite.
They actually both amount to the same thing and since you say "check if a property is true for every row or nested dimension", I believe the first method is easier to read and perhaps slightly faster.
You shouldn't try to read the value of the flag until the code is completely executed anyway, because the check isn't finished. If you're running asynchronous code, you should guard against accessing the value while it is unstable.
Both methods "give a wrong name" until the code is executed. 1 gives false positives and 2 gives false negatives. I'm not sure what you're trying to avoid by saying this - if you can get the "correct" value before fully running your code, you didn't have run your code in the first place.
How to implement each without a counter (if you don't have a foreach syntax in your language, use the appropriate enumerator->next loop syntax):
1:
bool flag = true;
foreach(item in array)
{
if(!check(item))
{
flag = false;
break;
}
}
2:
bool flag = false;
foreach(item in array)
{
if(!check(item))
{
break;
}
else if(item.IsLast())
{
flag = true;
}
}
Go with the first option. An algorithm always has preconditions, postconditions and invariants. If your invariant is "bool x is true iff all rows from 0-currentN have a positve property", then everything is fine.
Don't make your algorithm more complex just to make the full program-state valid per row-iteration. Refactor the method, extract it, and make it "atomic" with your languages mechanics (Java: synchronized).
Personally I just throw the whole loop into a somewhat reusable method/function called isPropertyAlwaysTrue(property, array[][]) and have it return a false directly if it finds that it finds a case where it's not true.
The inversion of logic, by the way, does not get you out of there any quicker. For instance, if you want the first non-true case, saying areAnyFalse or areAllTrue will have an inverted output, but will have to test the exact same cases.
This is also the case with areAnyTrue and areAllFalse--different words for the exact same algorithm (return as soon as you find a true).
You cannot compare areAnyFalse with areAnyTrue because they are testing for a completely different situation.
Make the property name something like isThisTrue. Then it's answered "yes" or "no" but it's always meaningful.
In Ruby and Scheme you can use a question mark in the name: isThisTrue?. In a lot of other languages, there is a convention of puttng "p" for "predicate" on the name -- null-p for a test returning true or false, in LISP.
I agree with Will Hartung.
If you are worried about (1) then just choose a better name for your boolean. IsNotSomething instead of IsSomething.