What is the meanings of the input and output parameters of Kalman Filter in WSO2 CEP? - kalman-filter

I would like to have some information about the input (named: measuredValue, measuredChangingRate, measurementNoiseSD and timestamp) and output (2 values) of Kalman Filter Extension in WSO2 CEP 4.2.0. I can't find any kind of documentation.
I suppose that measuredValue is the value to filter, timestamp is a numerical label about the instant of acquisition of measuredValue; and about measuredChangingRate and measurementNoiseSD?
Furthermore I noticed a problem: when I try to use this function, Siddhi says:
No extension exist for StreamFunctionExtension{namespace='kf'} in execution plan "ExecutionPlan"
Is it possible that there isn't any extension for kalmanFilter function?
My fragment query is that:
#Import('InputStream:1.0.0')
define stream InStream (energy double, timestamp long);
define stream ResKalmanFilterStream (energyEstimated double, x double);
from InStream#kf:kalmanFilter(energy)
select *
insert into ResKalmanFilterStream;

Regarding following error:
No extension exist for StreamFunctionExtension{namespace='kf'} in
execution plan "ExecutionPlan"
kf:kalmanFilter(energy) is a function which takes an attribute (energy in this case) as a parameter and outputs a new value. In other words, it produces a new attribute, therefore it has to be used in the select statement.
from InStream
select kf:kalmanFilter(energy) as filteredEnergy
insert into ResKalmanFilterStream;
Regarding parameters; I could find following information on the parameters in the Kalman Filter extension source code (refer the class-level comment).
measuredValue - measured value eg:40.695881
measuredChangingRate -
Changing rate. eg: Velocity of the point which describes from measured
value - 0.003d meters per second
measurementNoiseSD - standard
deviation of the noise. eg: 0.01
timestamp - the timestamp at the
measured time eg: 1445234861l

Related

Assert json parameter value in second response with respect to same first request

App is related to an f&b business, I have following scenarios (api calls):
Create Order - on creating an order of any food item (let it be X), inventory of which is updated at back-end (this value is under test, let it be P1)
Get Inventory - (this call will fetch me the updated value of inventory of ordered item (X) i.e. inventory value, P1)
Cancel Order - this will cancel the order, i created in my first call, and hence P1 should be rolled back.
Get Inventory - Again i will hit this call to get the value, so as to verify that the inventory value of P1 has been updated properly.
In API call, (2) I extract P1 value using jp#gc Json Extractor and same I did for same call (4).
Now as per my expectations, value obtained in both these extractors should be equal as order has been cancelled now.
To assert these values, I am using JSON Assertion , either I am making use of wrong assertion or lacking a big amount of information here.
May be there is something like I can save the value first in some variable, and then assert.
Image of my test suite:
You can add JSR223 Assertion with checking different variables e.g. a and b:
if (!vars.get("a").equals(vars.get("b"))) {
AssertionResult.setFailureMessage("message");
AssertionResult.setFailure(true);
}
The script can check various aspects of the SampleResult. If an error is detected, the script should use AssertionResult.setFailureMessage("message") and AssertionResult.setFailure(true)

In Gecode, accessing home/space variables values from merit function

In Gecode, I am using a merit function to select variables when branching.
In order to compute variable v's merit, I need to access some other variables values, but it looks like at the time the merit function is called, the space variables have not yet been asigned any values :
Exception: IntVar::val: Attempt to access value of unassigned variable.
Am I doing something wrong? Is there a way to access variables values in merit functions?
The problem is that while you are still searching a variable won't just have 1 value, its domain is still larger than 1. This means that there might still be different values that a variable can take. Until there is only one value left in its domain you are not allowed to use the val method.
There are different solutions for this problem depending on how you want to use the value domain:
The best way to test a variable against a single value is using the in method. This method returns true if the value is in the domain of the variable.
To check variables against each other you would generally use the min and max methods to compare their domains.
If the value is only relevant when it is assigned, then you would check that the cardinality (size of the domain) is 1, using the size method, before using the val method.
These are the most general cases, but there are countless ways to interact with variables. Be sure to check the IntVar documentation, where these and all other methods for the IntVar class are described.

Array Condition within EPA CEP PROTON

Using CEP Proton I want to check if, for the events received in a time interval, the id of the second and following events is contained in an attribute of type array coming in the first event. Let's say the first event is coming with the attribute called group that is an array of integers (10, 20, 30). The following events will come with an id, ex 20, 25 and so on. I want to check if the id of those events is included in the attribute group included in the first event.
For that I have created a condition in EPA
ArrayContains(ArrayGet(myevent.group,0),ArrayGet(myevent.id,ArraySize(myevent.id)-1))
However I'm getting an error when I click on verify
Error Error in EPA: expression error of undefined myepa
Am I doing anything wrong?
Cannot I access to the attribute array using ArrayGet? Or maybe ArrayContains cannot work with the result of ArrayGet?
JSON
{"epn":{"events":[{"name":"GupsyContextUpdate","createdDate":"Thu Sep 29 2016","attributes":[{"name":"entityId","type":"String","dimension":0},{"name":"entityType","type":"String","dimension":0,"description":"The cost of this event occurrence. Negative if this is an opportunity"},{"name":"SF","type":"Integer","dimension":0,"description":"Used in case the this event occur within an interval"},{"name":"occupied","type":"Integer","dimension":"0"},{"name":"framecounter","type":"Integer","dimension":"0"},{"name":"RSSI","type":"String","dimension":"0"},{"name":"hwversion","type":"String","dimension":"0"},{"name":"swversion","type":"String","dimension":"0"},{"name":"compassz","type":"String","dimension":"0"},{"name":"MACBS","type":"String","dimension":"0"},{"name":"SNR","type":"String","dimension":"0"},{"name":"devEUI","type":"String","dimension":"0"},{"name":"battery","type":"String","dimension":"0"},{"name":"devicetype","type":"String","dimension":"0"},{"name":"compassx","type":"String","dimension":"0"},{"name":"compassy","type":"String","dimension":"0"},{"name":"compasst","type":"String","dimension":"0"},{"name":"neighbours","type":"String","dimension":"1"},{"name":"timestamp","type":"String","dimension":"0"}]},{"name":"gupsyoutput","createdDate":"Thu Sep 29 2016","attributes":[{"name":"Certainty","type":"Double","defaultValue":"1","dimension":0,"description":"The certainty that this event happen (value between 0 to 1)"},{"name":"OccurrenceTime","type":"Date","dimension":0,"description":"No value means it equals the event detection time, other option is to use one of the defined distribution functions with parameters"},{"name":"ExpirationTime","type":"Date","dimension":0},{"name":"Cost","type":"Double","dimension":0,"description":"The cost of this event occurrence. Negative if this is an opportunity"},{"name":"Duration","type":"Double","defaultValue":"0","dimension":0,"description":"Used in case the this event occur within an interval"},{"name":"test","type":"String","dimension":"1"}]}],"epas":[{"name":"gupsyepa","createdDate":"Thu Sep 29 2016","epaType":"Aggregate","context":"gupsyct","inputEvents":[{"name":"GupsyContextUpdate","alias":"gupsyupdate","consumptionPolicy":"Consume","instanceSelectionPolicy":"First"}],"computedVariables":[{"name":"count","aggregationType":"Count","gupsyupdate":"ArrayContains(ArrayGet(gupsyupdate.neighbours,0),ArrayGet(gupsyupdate.id,ArraySize(gupsyupdate.id)-1))"}],"assertion":"count == true","evaluationPolicy":"Immediate","cardinalityPolicy":"Single","internalSegmentation":[],"derivedEvents":[{"name":"gupsyoutput","reportParticipants":false,"expressions":{"Duration":"0","test":"ArrayContains(ArrayGet(gupsyupdate.neighbours,0),ArrayGet(gupsyupdate.devEUI,ArraySize(gupsyupdate.devEUI)-1))"}}]}],"contexts":{"temporal":[{"name":"gupsyct","createdDate":"Thu Sep 29 2016","type":"TemporalInterval","atStartup":false,"neverEnding":false,"initiators":[{"initiatorType":"Event","initiatorPolicy":"Ignore","name":"GupsyContextUpdate"}],"terminators":[{"terminatorType":"RelativeTime","terminationType":"Terminate","relativeTime":"15000"}]}],"segmentation":[],"composite":[]},"consumers":[{"name":"gupsyconsumer","createdDate":"Thu Sep 29 2016","type":"Rest","properties":[{"name":"URL","value":"http://127.0.0.1/ProtonParser/TestEvent2Log.php"},{"name":"contentType","value":"application/xml"},{"name":"formatter","value":"xml"},{"name":"delimiter","value":";"},{"name":"tagDataSeparator","value":"="},{"name":"dateFormat","value":"dd/MM/yyyy-HH:mm:ss"}],"events":[{"name":"gupsyoutput"}]}],"producers":[],"name":"gupsy"}}
Please share the whole JSON definition of your application.
From the information you have provided it is not clear which type of EPA it is (to access a series of events you need a statefull EPA and you need to make sure you write the application in such a way as to be able to distinguish between the first and subsequent events).
Additionally it is not clear why you try to access the same event instance- myevent in both parts of the expression.
ArrayContains recieves (array, value) as operands.
You pass it (value, value), and thats assuming that "myevent.id" is also an array, which from your description I understood it is not?
Anyway, I see lots of potential problems with your expression, therefore I would like to see the whole JSON, or at least the part relevant to the "myepa" EPA.
I have reviewed the JSON application, there are multiple problems:
1)You are using an EPA of type aggregate, to calculate a COUNT. Count is for counting events. It is an integer variable. No boolean values can be assigned to it.
Therefore comparison expressions like "count == true"
Or assigment of boolean values like : "count : ArrayContains(ArrayGet(gupsyupdate.neighbours,0),ArrayGet(gupsyupdate.id,ArraySize(gupsyupdate.id)-1))"
Are not valid.
2)Additionally, assuming you use this expression somewhere, and not in count variable:
-the gupsyupdate event has no attribute "id", so you cannot use this in an expression.
-ArrayGet(gupsyupdate.neighbours,0) - will return a single object within an array... So you cannot use it in ArrayContains expression which is supposed to receive an array
3)The policies of the EPA doesn't make sense... It is an aggregation EPA, but you use it with evaluationPolicy of "Immediate" and cardinality policy "Single". That means this EPA will be evaluated only once, immediately as it receives the first gupsyupdate event. So it will not really act as aggregator....
Please revise your application according to this input.

google analytics core api results sampling level

I have a script pulling data from the Google Analytics core api. Since I am using the results of the data to successfully populate a sheet in GSheets I know that my data pull is a success.
I'm reading the documentation here.
In particular, this table:
However, I would like to Logger.log() the sampling level of the query:
// check sampling for each report
if(!results.containsSampledData) {
Logger.log('sampling: none');
} else {
Logger.log('sampling: ' + results.query.samplingLevel);
}
When I view the logs I get 'sampling: undefined'.
How do I get the sampling results from the results object?
Here is what generates the results object, though I don;t think it's relavant (but may be wrong):
// get GA data from core api
function getReportDataForProfile(profile, len_results, start_num) {
var startDate = getLastNdays(30); // set date range here
var endDate = getLastNdays(0);
var optArgs = {
'dimensions': 'ga:dimension5, ga:dimension4', // Comma separated list of dimensions.
'start-index': start_num,
'max-results': len_results,
'filters': 'ga:source==cj'
};
// Make a request to the API.
var results = Analytics.Data.Ga.get( // mcf for multi channel api, Ga for core
profile, // Table id (format ga:xxxxxx).
startDate, // Start-date (format yyyy-MM-dd).
endDate, // End-date (format yyyy-MM-dd).
'ga:goalCompletionsAll, ga:users, ga:sessions', // Comma seperated list of metrics.
optArgs);
return results;
}
I think you missed this sentence:
The following table summarizes all the query parameters accepted by
the Core reporting API.
Those are query parameters. In other words, values that YOU supply. So, you should already know what the sampling level is since you determine it.
Here's the doc on sampling level. If not supplied, it sets samplingLevel to DEFAULT.
EDIT: Here's the doc on the response. I see that it indeed includes a samplingLevel field, but if you scroll further down, samplingLevel isn't one of the fields described in the Response Fields table. I suspect it is either included in the response by accident or you cannot rely upon that field given the lack of documentation.
Ah. If I read further down I would have seen this paragraph:
Sampling
Google Analytics calculates certain combinations of dimensions and
metrics on the fly. To return the data in a reasonable time, Google
Analytics may only process a sample of the data.
You can specify the sampling level to use for a request by setting the
samplingLevel parameter.
If a Core Reporting API response contains sampled data, then the
containsSampledData response field will be true. In addition, 2
properties will provide information about the sampling level for the
query: sampleSize and sampleSpace. With these 2 values you can
calculate the percentage of sessions that were used for the query. For
example, if sampleSize is 201,000 and sampleSpace is 220,000 then the
report is based on (201,000 / 220,000) * 100 = 91.36% of sessions.
See Sampling for a general description of sampling and how it is used
in Google Analytics."
So to get sample size as a percentage (what I'm used to seeing) I do this: results.sampleSize/results.sampleSpace

how do i decode/encode the url parameters for the new google maps?

Im trying to figure out how to extract the lat/long of the start/end in a google maps directions link that looks like this:
https://www.google.com/maps/preview#!data=!1m4!1m3!1d189334!2d-96.03687!3d36.1250439!4m21!3m20!1m4!3m2!3d36.0748342!4d-95.8040972!6e2!1m5!1s1331-1399+E+14th+St%2C+Tulsa%2C+OK+74120!2s0x87b6ec9a1679f9e5%3A0x6e70df70feebbb5e!3m2!3d36.1424613!4d-95.9736986!3m8!1m3!1d189334!2d-96.03687!3d36.1250439!3m2!1i1366!2i705!4f13.1&fid=0
Im guessing the "!" is a separator between variables followed by XY where x is a number and y is a lower case letter, but can not quite figure out how to reliably extract the coordinates as the number/order of variables changes as well as their XY prefixes.
ideas?
thanks
Well, this is old, but hey. I've been working on this a bit myself, so here's what I've figured out:
The data is an encoded javascript array, so the trick when trying to generate your own data string is to ensure that your formatting keeps the structure of the array intact. To do this, let's look at what each step represents.
As you're correctly figured out, each exclamation point defines the start of a value definition. The first character, an int value, is an inner count, and (I believe) acts as an identifier, although I'm not 100% certain on this. It seems to be pretty flexible in terms of what you can have here, as long as it's an int. The second character, however, is much more important. It defines the data type of the value. I don't know if I've found all the data types yet, but the ones I have figured out are:
m: matrix
f: float
d: double
i: integer
b: boolean
e: enum (as integer)
s: string
u: unsigned int
x: hexdecimal value?
the remaining characters actually hold the value itself, so a string will just hold the string, a boolean will be '1' or '0', and so on. However, there's an important gotcha: the matrix data type.
The value of the matrix will be an integer. This is the length of the matrix, measured in the number of values. That is, for a matrix !1mx, the next x value definitions will belong to the matrix. This includes nested matrix definitions, so a matrix of form [[1,2]] would look like !1m3!1m2!1i1!2i2 (outer matrix has three children, inner matrix has 2). this also means that, in order to remove a value from the list, you must also check it for matrix ancestors and, if they exist, update their values to reflect the now missing member.
The x data type is another anomaly. I'm going to guess it's hexdecimal encoded for most purposes, but in my particular situation (making a call for attribution info), they appear to also use the x data type to store lat/long information, and this is NOT encoded in hex, but is an unsigned long with the value set as
value = coordinate<0 ? (430+coordinate)*1e7 : coordinate*1e7
An example (pulled directly from google maps) of the x data type being used in this way:
https://www.google.com/maps/vt?pb=!1m8!4m7!2u7!5m2!1x405712614!2x3250870890!6m2!1x485303036!2x3461808386!2m1!1e0!2m20!1e2!2spsm!4m2!1sgid!2sznfCVopRY49wPV6IT72Cvw!4m2!1ssp!2s1!8m11!13m9!2sa!15b1!18m5!2b1!3b0!4b1!5b0!6b0!19b1!19u12!3m1!5e1105!4e5!18m1!1b1
For the context of the question asked, it's important to note that there are no reliable identifiers in the structure. Google reads the values in a specific order, so always keep in mind when building your own encoded data that order matters; you'll need to do some research/testing to determine that order. As for reading, your best hope is to rebuild the matrix structure, then scan it for something that looks like lat/long values (i.e. a matrix containing exactly two children of type double (or x?))
Looks like the developer tools from current browsers (I am using Chrome for that) can give you a lot of info.
Try the following:
Go to Google Maps with Chrome (or adapt the instructions for other browser);
Open Developer Tools (Ctrl + Shift + I);
Go to Network tab. Clear the current displayed values;
Drag the map until some url with encoded data appears;
Click on that url, and then go to the Preview sub-tab;
Try this.
function URLtoLatLng(url) {
this.lat = url.replace(/^.+!3d(.+)!4d.+$/, '$1');
this.lng = url.replace(/^.+!4d(.+)!6e.+$/, '$1');
return this;
}
var url = new URLtoLatLng('https://www.google.com/maps/preview#!data=!1m4!1m3!1d189334!2d-96.03687!3d36.1250439!4m21!3m20!1m4!3m2!3d36.0748342!4d-95.8040972!6e2!1m5!1s1331-1399+E+14th+St%2C+Tulsa%2C+OK+74120!2s0x87b6ec9a1679f9e5%3A0x6e70df70feebbb5e!3m2!3d36.1424613!4d-95.9736986!3m8!1m3!1d189334!2d-96.03687!3d36.1250439!3m2!1i1366!2i705!4f13.1&fid=0');
console.log(url.lat + ' ' + url.lng);