Data model for timeline event synchronisation - language-agnostic

I am looking for ideas on the data model for the following problem (and the proper CS terminology):
A (horizontal) "timeline" with several rows (A,B,C) contains "events" (1,2,3) width different durations (width) at different times (absolute x position or by delay "." after previous event):
A 1111....222222
B 33333
------------------
T 0123456789ABCDEF
(The rows are only interesting for graphical representation of overlapping/parallel "events", so they probably are not essential to the data model.)
Event duration may vary, affecting the whole timing:
A 11....222222
B 33333+3
------------------
T 0123456789ABCDEF
But let event 2 require events 1 and 3 to be finished, so the timing should look like this:
A 11.... 222222
B 33333+3
------------------
T 0123456789ABCDEF
(let's ignore that the original delay at T=7 is now missing.)
Originally I thought I'd have to have some "elastic" synchronization elements, one for each row:
A 11....####222222
B 33333+3#
------------------
T 0123456789ABCDEF
Thus the original problem of how to model and sync the sync elements in the two different "rows". But, as established above, this is only a matter of graphical/parallel representation.
Rather, the sync is a condition that could be "attached" to event 2, modifiying or determining its beginning.
If an event "has" a condition, it will not have an absolute or relative start time. Its start can only be determined at the ends of the "linked" events (1 and 3).
So, given (a list of) some events with variable duration and either an absolute start time or a delay relative to another event's end, how could the condition "events 1 and 3 ended" be modelled to determine the start of "event 2"?
(I will prototype this in JavaScript and eventually implement in C/C++, so any sample code provided should not use high-level data types or libraries.)

What you need is an object that I would call a TimeFrame. The object would have the attributes duration, link and type, where link can be a precise time or a link to another TimeFrame and type accounts for the kind of link. For instance, a given TimeFrame that starts at a known time would have that time as its link attribute and the type would be TIME. A TimeFrame that is linked to the end of another would have that other TimeFrame as its link attribute and START-END as its type and so on.
Using the combination between link and type you could also support other types of links such as START-START, END-START or END-END.
UPDATE
Also, in order to allow some time interval between say, the end of a TimeFrame and the start of the next, one can add the attribute lag, which represents any delay between events. So, for instance if tf1 and tf2 are TimeFrames such that tf2 must start 5 time units after the end of tf1 the attributes of tf2 would be link = tf1, type = START-END, duration = <something> and lag = 5. Note also that the lag could be negative, which would extend the expressiveness of the model to a broad range of relationships.

While #Leandro Caniglia nicely rephrased my question into an Object and Attributes, essentially, I see two options:
the whole list of "events" needs to be evaluated at "condition" (start/end) to check dependent "events".
adding a "link" to a "parent" also creates a link to the "child" (no need to evaluate all pending event's links).
Also:
The "link" property needs to be a List or Array to be able hold several references (e.g. 2:[1,3]).
Analogous to the link property start_me_on_condition a stop_me_on_condition association may be desirable (see Leandro's suggestion of type, it would need to be extended to support multiple links+type)
An independet delay "event" might be more practical than a lag property.

Related

How does the element section of wasm module in binary format looks?

I am reading this docs to study wasm binary format. I am finding it very tough to understand the composition of element section. Can someone please give me an example / explanation about it ? Maybe similar to one given here
The element segments section
The idea of this section is to fill the WebAssembly.Table objects with content. Initially there was only one table, and its only possible content were indexes/ids of functions. You could write:
(elem 0 (offset (i32.const 1)) 2)
It means: during the instantiation of the instance fill index 1 of table 0 with a value of 2, like tables[0][1] = 2;. Here 2 is the index of the function the table will store.
The type of element segment above is called active nowadays, and after the instantiation it will no longer be accessible by the application (they are "dropped"). From the specs:
An active element segment copies its elements into a table during instantiation, as specified by a table index and a constant expression defining an offset into that table
So far so good. But it was recognized that there is a need for a more powerful element segment section. Introduced were the passive and the declarative element segments.
The passive segment is not used during the instantiation and it is always available at runtime (until is not dropped, by the application itself, with elem.drop). There are instructions (from the Bulk memory and table instructions proposal, already integrated into the standard) that can be used to do operations with tables and element segments.
A declarative element segment is not available at runtime but merely serves to forward-declare references that are formed in code with instructions like ref.func.
Here is the test suite, where you can see many examples of element segments (in a text format).
The binary format
Assuming that you parser the code, you read one u32, and based on its value you expect the format from specification:
0 means an active segment, as the one above, for an implicit table index of 0, and a vector of func.refs.
1 means a passive segment, the elemkind (0x00 for func.ref at this time), followed by a vector of the respective items (func.refs).
2 is an active segment.
3 means a declarative segment.
4 is an active segment where the values in the vector are expressions, not just plain indexes (so you could have (i32.const 2) in the above example, instead of 2).
5 passive with expressions
6 active with table index and expressions
7 declarative with expressions
For this reason the specs says that from this u32 [0..7] you can use its three lower bits to detect what is the format that you have to parse. For example, the 3th bit signifies "is the vector made of expressions?".
Now, all that said, it seems that the reference types proposal is not (yet) fully integrated into the specification's binary format (but seems to be in the text one). When it is you will be able to have other then 0x00 (func.ref) for an elemkind.
It is visible that some of this formats overlap, but the specification evolves, and for backward compatibility reasons with the earliest versions the format is like this today.

How do I make action for certain instances of object?

I don't want to make action to the object in general
I have 8 instances for example (0,1,2,3,4,5,6,7)
I need to make the action on 2 , 5 and 7 only for example
how ??
This is something that really confused me about Construct 2 coming from a software engineering background.
Picking which instances of a Sprite to effect works sort of like filtering a database. You start with all the instances in a list, and then you filter them out using conditions. When possible, Construct 2 will automatically guess which instance you want. (Like if you just spawned Enemy at EnemySpawn, it will know your next reference Enemy is only the last one created).
This magically works most of the time, but you can select a set of objects with a few of these conditions:
Compare instance variable
Compare the current value of one of the object's instance variables.
Is boolean instance variable set
Test if one of the object's boolean instance variables is set to true. (Invert the condition to test if false.)
Pick by unique ID (UID)
Pick the instance matching a given unique ID (UID) number.
In my experience, adding an instance variable and setting it to keep track of a subset of Sprites or the state of the sprite is the cleanest way to limit an action to a specific group of sprites.
See also Object Expressions from the Scirra Manual:
You can add a 0-based object index to get expressions from different object instances. For example Sprite(0).X gets the first Sprite instance's X position, and Sprite(1).X gets the second instance's X position. For more information see index IDs (IIDs) in common features. You can also pass another expression for the index. Negative numbers start from the opposite end, so Sprite(-1).X gets the last Sprite's X position.
These are not "specific" object IDs (Construct 2 calls them Index IDs, or IIDs) but the refer to all the objects of that type in the order they were created.
So:
Enemy(0) is the first
Enemy(-1) is the most recent
Enemy(5) is the 6th Enemy created, ignoring destroyed Sprites (See below).
You have to keep in mind though, that if you destroy an object with IID 3, then all the objects after it will shift in the list and their IID will decrease by one. (Like a LinkedList data structure)
If you destroy instance D, the others shift
Instance |A|B|C|D|E|F| --> |A|B|C|E|F|
IID |0|1|2|3|4|5| --> |0|1|2|3|4|
Hopefully one of those methods can help you get to what you need!

Proton CEP Fiware: delete old received events

I've got this kind of problem with Proton CEP: i currently have a "Sequence" EPA; its input are 2 events. But these events have different granularity: let's say i have A and B events; i receive N "A" events, and M "B" events, where M << N.
So i'd like to have a rule like "if event of type A is not consumed within X seconds, remove it", otherwise i've got a long A events queue; i only need the rule to be evaluated for closest (temporally) events.
Practically, i've got a fake room temperature sensor that sends its temperature updates every 5seconds, and i've got another program that checks external weather and sends it every minute.
Any idea how to solve this situation?
Thank you very much!
I guess that in "consume" you mean arrival, so do you want to evaluate the time the A event took to get to the proton pcoressor? or the time between A events? Do you want to ensure that the A events are indeed continuous in a fix rate? "Removing" an event means to ignore it, since events are not kept anywhere, just processed. At the end, what is that you want to detect here? Like, what is the trend of room temperature compared to the outside temperature? then, emit output events accordingly?
Thanks.
all the relevant event instances are kept within the local state of a corresponding EPA.
For each EPA operand you have policies which dictates how the state is gathered and how the matching set for event derivation is built.
For example, instance selection policy which is defined per operand, and has the values of "Each", "First" and "Last" will tell you if all A instances are examined for match with B instance, or the first (in the order of arrival), or the last.
The consumption policy says what to do with the operand state once a seqence is detected - should the instances of say A which participated in sequence be removed from EPA's state ("consume" value of the policy) or should they remain.
Playing with combination of those policies should give you the behaviour you require

CEP's sequence detection‏

In developing for Fiware's Proton CEP, I came across an issue with Sequence event detection. I'll take advantage of DoSAttack example project, that comes with the software, to explain the issue.
I make two main changes to an original copy of DoSAttack:
-One is to make ExpectedCrash event have 3 more variables. This way I can log to DoSAttackTRConsumer file the 3 values that triggered it.
-Then I also change the Cardinality Policy of the Agent from Single to Unrestricted. This way the event can be triggered several times in a row, as TrafficReports come in (this may be a source to the issue).
I test this result and I find it works ok. I can see in the log that the values that trigger detection are the sequence of 3 values that arrived just before the event, after the first three events have arrived.
This, taking into account that the test beeing made on those 3 values still remains the original example test: (TR3.volume>1.50* TR2.volume AND TR2.volume>1.50 * TR1.volume).
The issue arrises if I make the test be just (TR3.volume>1.50* TR2.volume), for example, then CEP doesn't hold TR1 correctly. Now TR1 is the same as TR2, so cep loses "memory" of this value.
Going a step further, I make the test, just the condition (3>2) which is always true and should trigger a detection on any event that arrives. In this case, as events arrive, all TR1, TR2 and TR3 are the same and CEP has no memory of past values, even though the agent is of Type: Sequence.
The desired application would be for the CEP to recieve 22 readings as a sequence of input events and analyse only the 1st, 8th, 15th and 22nd values of this sequence, at each value that enters. But I find I can't make CEP remember the values correctly unless I'm testing all of them explicitly in the Condition view-box.
What would be the correct way to analyse the 1st, 8th, 15th and 22nd values that arrived, evaluating each time a new one arrives?
Here is the specificatin of DoSAttack, after altering it:
{"epn":{"events":[{"name":"TrafficReport","attributes":[{"name":"volume","type":"Integer","dimension":0}]},{"name":"ExpectedCrash","attributes":[{"name":"Cost","type":"Double","dimension":0},{"name":"TR1","type":"Integer","dimension":"0"},{"name":"TR2","type":"Integer","dimension":"0"},{"name":"TR3","type":"Integer","dimension":"0"}]}],"epas":[{"name":"IncreasingTraffic","epaType":"Sequence","context":"3MinAfterStartUp","inputEvents":[{"name":"TrafficReport","alias":"TR1","consumptionPolicy":"Consume","instanceSelectionPolicy":"First"},{"name":"TrafficReport","alias":"TR2","consumptionPolicy":"Consume","instanceSelectionPolicy":"First"},{"name":"TrafficReport","alias":"TR3","consumptionPolicy":"Consume","instanceSelectionPolicy":"First"}],"computedVariables":[],"assertion":"3>2","evaluationPolicy":"Immediate","cardinalityPolicy":"Unrestricted","internalSegmentation":[],"derivedEvents":[{"name":"ExpectedCrash","reportParticipants":false,"expressions":{"Cost":"10","TR1":"TR1.volume","TR2":"TR2.volume","TR3":"TR3.volume"}}],"derivedActions":[]}],"contexts":{"temporal":[{"name":"3MinAfterStartUp","type":"TemporalInterval","atStartup":true,"neverEnding":false,"initiators":[],"terminators":[{"terminatorType":"RelativeTime","terminationType":"Terminate","relativeTime":"180000"}]}],"segmentation":[],"composite":[]},"consumers":[{"name":"SysTemCrashConsumer","type":"File","properties":[{"name":"filename","value":"/opt/tomcat10/sample/DoSAttack_PredictedCrash.txt"},{"name":"formatter","value":"json"},{"name":"delimiter","value":";"},{"name":"tagDataSeparator","value":"="},{"name":"SendingDelay","value":"1000"}],"events":[{"name":"ExpectedCrash"}],"actions":[]},{"name":"DoSAttackTRConsumer","type":"File","properties":[{"name":"filename","value":"/opt/tomcat10/sample/DoSAttack_TrafficReport.txt"},{"name":"formatter","value":"json"},{"name":"delimiter","value":";"},{"name":"tagDataSeparator","value":"="},{"name":"SendingDelay","value":"1000"}],"events":[{"name":"TrafficReport"}],"actions":[]}],"producers":[{"name":"TrafficReportFileProducer","type":"File","properties":[{"name":"filename","value":"/opt/tomcat10/sample/DoSAttackScenarioJSON.txt"},{"name":"pollingInterval","value":"1000"},{"name":"sendingDelay","value":"1500"},{"name":"formatter","value":"json"},{"name":"delimiter","value":";"},{"name":"tagDataSeparator","value":"="}],"events":[]}],"actions":[],"name":"DoSAttack"}}
The producer file, DoSAttackScenarioJSON.txt, is still the original one, unaltered:
{"Name":"TrafficReport", "volume":"1000"}
{"Name":"TrafficReport", "volume":"1600"}
{"Name":"TrafficReport", "volume":"2500"}
If you do include more values than 3 you can see that the issue propagates.
If you need more information let me know.
Thank you
In the Sequence pattern, the engine looks for event instances that occurred in a particular order.
In Sequence (A, B, C), the engine looks for three event instances, the first one of type A, the second of type B and the third of type C, where:
(A's detection time) <= (B's detection time) AND (B's detection time) <= (C's detection time)
Usually in a Sequence pattern, either the event types are different, or there is other condition above the participants events (as in the DoSAttack example).
When you use the same event type in a sequence (e.g., Sequence(A, A, A)), then the same event instance can be used in all the three places, since it holds the detection order listed above.
In addition, if you use a "consumptionPolicy": "Consume" for a participant event, then after the event was used to detect the pattern, it will not be used for future detections of this pattern.
This is why when you have a Sequence(A, A, A) with no condition, and event instance A1 of type A arrives, it causes a pattern detection, and since it has Consume policy, it will not be kept for future detections. Later when event A2 of type A arrives, it causes another detection based on A2 alone.
Also, according to the Sequence built-in condition over the detection time, a sequence of events can be detected although other events arrived in between.
Please describe the pattern you would like to detect. Maybe you can use a Trend or Aggregate EPA instead.

Reconstructing state from time series data events

For a particular project, we acquire data for a number of events and collect variables about those events at the same time. After the data has been collected, we perform a user-customizable analysis on said data to determine whatever it is that the user is interested in.
The data is collected in a form similar to this:
Timestamp Event
0 x = 0
0 y = 1
3 Event A occurred
3 x = 1
4 Event A occurred
4 x = 2
9 Event B occurred
9 y = 2
9 x = 0
To understand the entire state at any time, the most straightforward approach is to walk over the entire set of data. For example, if I start at time 0, and "analyze" until timestamp 5, I know that at that point x = 2, y = 1, and Event A has occurred twice. That's a really simple example. The user might be (and often is) interested in the time between events, say from A to B, and they might specify the first occurrence of A, then B, or the last occurrence of A, then B (respectively, 9-3 = 6 or 9-4 = 5). Like I said, this is easy to analyze when you're walking over the entire set.
Now, we need to adapt the model to analyze an arbitrary window of time. If we look at 0-N, that's the easy case. But if I look at 1-5, for instance, I have no notion of y unless I begin at 0 and know that y was initially 1 and did not change in the window 1-5.
Our approach is to essentially create a dictionary of variables, and run callbacks on events. If one analysis was "What is x when Event A occurs and time is > 3" then we would run that callback on the first Event A, and it would immediately return because time is not greater than 3. It would run again at 4, and and it would report that x was 1 at t=4.
To adapt to the "time-windowing", I think I am going to (in the background) tack on additional conditions to the analysis. If their analysis is just "What is x when Event A occurs", and the current window is 1-5, then I will change it to "What is x when Event A occurs and time >= 1 and time <= 5". Then if the next window is 6-10, I can readjust the condition as necessary.
My main question is: what pattern does this fit? We are obviously not the first people to approach a problem like this, but I have not been able to find how others have approached it. I probably just don't know what exactly to search on Google. Is there any other approach besides keeping a dictionary of the entire global state for looking up a single state at a given time? Note also that the data could have several, maybe tens of thousands of records, so the fewer iterations over the data set, the better.
I think your best approach here would be to take periodic "snapshots" of the full state data, say every 1000 samples (for example), along with recording the deltas. When you're storing your data as offsets from some original value (aka deltas), you don't have any choice but to reconstruct the full data starting with the original values. Storing periodic snapshots will lessen the amount of reconstruction you have to do - the design tradeoff is between low storage requirements but long reconstruction time on the one hand, and higher storage requirements but shorter reconstruction time on the other.
MPEGs, for example, store each frame as the differences between the current frame and the previous frame. Ordinarily, this would force an MPEG to be viewed from the beginning, but the format also periodically stores full frames so that the decoder doesn't have to backtrack all the way to the beginning of the file.
You can search by time in Log(N), and you can have a feeling about how many updates ares acceptable... hence here's my solution:
Pick a number, N, of updates that are acceptable in order to return a result. 256 might be good, given the scales you've mentioned so far.
Every N records, commit an entry of all state to a dictionary, with a timestamp.
Now, you have a tradeoff, dictionary size against speed. N->\infty is regular searching. N<-1 is your current solution, N anywhere else will require less memory, but be slower.
Your implementation is now (for time X):
Log(n) search of subsampled global dictionary to timestamp before X, (timestamped as Y).
Log(n) search of eventlist to timestamp Y, and perform less than N updates.
Picking N as a power of two will even allow you to do some nice shift tricks to do a rounded-down integer divide nice and fast.