Orion - pattern subscription to send multiple updates - fiware

I use Orion to send updates to Proton-CEP when a change of an object occurs.
However, I have many such objects updates occurring every few seconds. So I send these updates to Orion in batches, like this:
{
"contextElements": [
{
"type": "Queue",
"isPattern": "false",
"id": "Queue.routes_queue",
"attributes": [
{
"name": "volume",
"type": "integer",
"value": "3"
}
]
},
{
"type": "Queue",
"isPattern": "false",
"id": "Queue.optimizer_queue",
"attributes": [
{
"name": "volume",
"type": "integer",
"value": "4"
}
]
}
],
"updateAction": "APPEND"
}
I have a subscription which is supposed to send these event (when the volume has changed) to CEP for processing.
{
"entities": [
{
"type": "Queue",
"isPattern": "true",
"id": "Queue.*"
}
],
"attributes": [
"volume"
],
"reference": "http://<cep-host>:8080/ProtonOnWebServer/rest/events",
"duration": "P1M",
"notifyConditions": [
{
"type": "ONCHANGE",
"condValues": [
"volume"
]
}
],
"throttling": "PT1S"
}
CEP is configured to listen to these events, and write them to a file. Unfortunately, only the first element is written.
I have confirmed that this is not a CEP problem by using the provided accumulator-server script. I can see only a single update being exposed:
POST http://localhost:1028/accumulate
Content-Length: 741
User-Agent: orion/0.24.0 libcurl/7.19.7
Host: localhost:1028
Accept: application/xml, application/json
Content-Type: application/xml
<notifyContextRequest>
<subscriptionId>5645bb21abde60e23096acef</subscriptionId>
<originator>localhost</originator>
<contextResponseList>
<contextElementResponse>
<contextElement>
<entityId type="Queue" isPattern="false">
<id>Queue.routes_queue</id>
</entityId>
<contextAttributeList>
<contextAttribute>
<name>volume</name>
<type>integer</type>
<contextValue>4</contextValue>
</contextAttribute>
</contextAttributeList>
</contextElement>
<statusCode>
<code>200</code>
<reasonPhrase>OK</reasonPhrase>
</statusCode>
</contextElementResponse>
</contextResponseList>
</notifyContextRequest>
=======================================
::1 - - [13/Nov/2015 10:29:05] "POST /accumulate HTTP/1.1" 200 -
Can Orion send updates for all elements that have changed?

The solution is simply to remove the throttling. Throttling applies to the entire subscription, not just to a single object in the matched pattern.

Related

Fiware Context Provider hello world not working

I'm trying to register a Context Provider as source for several KPIs.
So far, it seems registering might be working, as GET http://{{orion}}/v2/registrations returns something similar to what I set in creation:
{
// each registering a new id is returned: "id": "60991a887032541f4539a71d",
"description": "City Inhabitants",
"dataProvided": {
"entities": [
{
"id": "city.inhabitants"
//,"type": "KeyPerformanceIndicator"
}
]
},
"provider": {
"http": {
"url": "http://myhost/v2/inhabitants"
}
//, "legacyForwarding": false //perhaps there's a bug, cause although unset or set to false, broker still returns true.
}
}
However GET http://{{orion}}/v2/entities/city.inhabitants results in:
{
"error": "BadRequest",
"description": "Service not found. Check your URL as probably it is wrong."
}
GET http://{{orion}}/v2/entities?type=KeyPerformanceIndicator returns [] and context-provider is not being invoked any way.
I'm coding a Node.js application from scratch, to better understand what's going on https://github.com/FIWARE/tutorials.Context-Providers, and using tutorial containers as actual Fiware Broker, so all stores/shelf/products are working but not my KPI.
Using the tutorial application where a microservice is listening on the http://context-provider:3000/random/weatherConditions POST endpoint.
The following registration:
curl -L -X POST 'http://localhost:1026/v2/registrations' \
-H 'Content-Type: application/json' \
--data-raw '{
"description": "Get Weather data for KPI 1",
"dataProvided": {
"entities": [
{
"id" : "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator"
}
],
"attrs": [
"temperature", "relativeHumidity"
]
},
"provider": {
"http": {
"url": "http://context-provider:3000/random/weatherConditions"
},
"legacyForwarding": false
},
"status": "active"
}'
Will forward and retrieve data for the following request:
curl -L -X GET 'http://localhost:1026/v2/entities/urn:ngsi-ld:KPI:010'
or
curl -L -X GET 'http://localhost:1026/v2/entities/?type=KeyPerformanceIndicator'
Returns:
{
"id": "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator",
"temperature": {
"type": "Number",
"value": 11,
"metadata": {}
},
"relativeHumidity": {
"type": "Number",
"value": 39,
"metadata": {}
}
}
The context broker is forwarding the request to the http://context-provider:3000/random/weatherConditions POST end-point with the following payload:
{
"entities": [
{
"id": "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator"
}
],
"attrs": [
"temperature",
"relativeHumidity"
]
}

adding nested OPC-UA Variable results in "String cannot be coerced to a nodeId"

Error: String cannot be coerced to a nodeId
Hi,
I was busy setting up a connection between the Orion Broker and an PLC with OPC-UA Server using the opcua iotagent agent.
I managed to setup all parts and I am able to receive (test) data, but I am unable to follow the tutorial with regards to adding an entity to the Orion-Broker using a json file:
curl http://localhost:4001/iot/devices -H "fiware-service: plcservice" -H "fiware-servicepath: /demo" -H "Content-Type: application/json" -d #add_device.json
The expected result would be an added entity to the OrionBroker with the supplied data, but this only results in a error message:
{"name":"Error","message":"String cannot be coerced to a nodeId : ns*4:s*MAIN.mainVar"}
suspected Error
Is it possible that the iotagent does not work nicely with nested Variables?
steps taken
doublechecked availability of OPC Data:
OPC data changes every second, can be seen in Broker log
reduced complexity of setup to only include Broker and IOT-agent
additional information:
add_device.json file:
{
"devices": [
{
"device_id": "plc1",
"entity_name": "PLC1",
"entity_type": "plc",
"attributes": [
{
"object_id": "ns*4:s*MAIN.mainVar",
"name": "main",
"type": "Number"
}
],
"lazy": [
],
"commands" : []
}
]
}
config of IOT-agent (from localhost:4081/config):
{
"config": {
"logLevel": "DEBUG",
"contextBroker": {
"host": "orion",
"port": 1026
},
"server": {
"port": 4001,
"baseRoot": "/"
},
"deviceRegistry": {
"type": "memory"
},
"mongodb": {
"host": "iotmongo",
"port": "27017",
"db": "iotagent",
"retries": 5,
"retryTime": 5
},
"types": {
"plc": {
"service": "plcservice",
"subservice": "/demo",
"active": [
{
"name": "main",
"type": "Int16"
},
{
"name": "test1",
"type": "Int16"
},
{
"name": "test2",
"type": "Int16"
}
],
"lazy": [],
"commands": []
}
},
"browseServerOptions": null,
"service": "plc",
"subservice": "/demo",
"providerUrl": "http://iotage:4001",
"pollingExpiration": "200000",
"pollingDaemonFrequency": "20000",
"deviceRegistrationDuration": "P1M",
"defaultType": null,
"contexts": [
{
"id": "plc_1",
"type": "plc",
"service": "plcservice",
"subservice": "/demo",
"polling": false,
"mappings": [
{
"ocb_id": "test1",
"opcua_id": "ns=4;s=test.TestVar.test1",
"object_id": null,
"inputArguments": []
},
{
"ocb_id": "test2",
"opcua_id": "ns=4;s=test.TestVar.test2",
"object_id": null,
"inputArguments": []
},
{
"ocb_id": "main",
"opcua_id": "ns=4;s=MAIN.mainVar",
"object_id": null,
"inputArguments": []
}
]
}
]
}
}
I'm one of the maintainers of the iotagent-opcua repo, we have identified and fixed the bug you were addressing, please update your agent to the latest version (1.4.0)
If you haven't ever heard about it, starting from 1.3.8 we have introduced a new configuration property called "relaxTemplateValidation" which let you use previously forbidden characters (e.g. = and ; ). I suggest you to have a look at it on the configuration examples provided.

How to make an Orion NGSIv2 subscription that notifies cygnus in NGSIv1?

EDIT 2
curl --include \
--header 'Content-Type: application/json' \
--request POST \
--data-binary '{
"description": "Day subscription",
"subject": {
"entities": [
{
"idPattern": "es-leon-.*",
"type": "Event"
}
],
"condition": {
"attrs": [
"Title",
"dFlag"
],
"expression": {
"q": "dFlag>0"
}
}
},
"notification": {
"http": {
"url" : "http://localhost:5050/notify"
},
"attrs": [
"Title",
"dFlag"
],
"attrsFormat":"legacy"
}
}' \
'http://localhost:1026/v2/subscriptions'
In Orion the subscription register ok (thanks for the tips in the comments) but I'm having the same issue as said in here even though I use "attrsFormat":"legacy"
I'm making something wrong? why orion is not using ngsiv1 to send the notification to cygnus?
The cygnus error trace.
Cygnus shows orion is responding this:
{
"subscriptionId": "574315e77775f31b8d3da719",
"data": [{
"id": "es-leon-0",
"type": "Event",
"Title": {
"type": "none",
"value": "pepe",
"metadata": {}
},
"dFlag": {
"type": "text",
"value": "1",
"metadata": {}
}
}]
}
When it has to respond the lines below to be in ngsiv1:
{
"subscriptionId": "5743178d7775f31b8d3da71a",
"originator": "localhost",
"contextResponses": [{
"contextElement": {
"type": "Event",
"isPattern": "false",
"id": "es-leon-0",
"attributes": [{
"name": "Title",
"type": "text",
"value": "pep"
}, {
"name": "dFlag",
"type": "text",
"value": "1"
}]
},
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
}]
}
Original Question
As said in other question :
EDIT: note also that you can use NGSIv2 to create/update entities at
Orion and have notifications in NGSIv1 if you:
Create the subscription using NGSIv1 operations
Create the subscription using NGSIv2 operations with attrsFormat equal to legacy.
Have a look to more detailed information here.
So I made up this subscription:
curl --include \
--header 'Content-Type: application/json' \
--request POST \
--data-binary '{
"description": "Day subscription",
"subject": {
"entities": [
{
"idPattern": "es-leon-.*",
"type": "Event"
}
],
"condition": {
"attributes": [
"Title",
"dFlag"
],
"expression": {
"q": "dFlag > 0"
}
}
},
"notification": {
"callback": "http://localhost:5050/notify",
"attributes": [
"Title",
"dFlag"
]
}
}' \
'http://localhost:1026/v1/subscriptions'
but orion don't let me register it throwing this error:
HTTP/1.1 400 Bad Request
Connection: Keep-Alive
Content-Length: 67
Content-Type: application/json
Fiware-Correlator: 2ecdfc74-1c2f-11e6-82d7-000d3a23bf27
Date: Tue, 17 May 2016 12:59:25 GMT
{"error":"BadRequest","description":"no condition attrs specified"}
Is this the way to use legacy attributes so it can work with cygnus with NGSIv2?
Thanks for the help.
EDIT 1:
Following the answer the subscription should be like this: Right?
curl --include \
--header 'Content-Type: application/json' \
--request POST \
--data-binary '{
"description": "Day subscription",
"subject": {
"entities": [
{
"idPattern": "es-leon-.*",
"type": "Event"
}
],
"condition": {
"attributes": [
"Title",
"dFlag"
],
"expression": {
"q": "dFlag > 0"
}
}
},
"notification": {
"http": {
"url" : "http://localhost:5050/notify"
},
"attributes": [
"Title",
"dFlag"
],
"attrsFormat":"legacy"
}
}' \
'http://localhost:1026/v2/subscriptions'
But I'm still getting an error:
HTTP/1.1 100 Continue
HTTP/1.1 400 Bad Request
Connection: Keep-Alive
Content-Length: 67
Content-Type: application/json
Fiware-Correlator: 60a0a1d2-1ddf-11e6-8bd6-000d3a23bf27
Date: Thu, 19 May 2016 16:33:11 GMT
{"error":"BadRequest","description":"no condition attrs specified"}
Use attrs instead of attributes (both in subject.conditions and in notification).
In addition, note that callback is no longer used. You have to use:
"notification": {
"http": {
"url": "http://localhost:5050/notify"
},
..
}
In addition, if you want notifications to use NGSIv1 format you should include attrsFormat field with value legacy in notification as described in the documentation.
Finally, note that the rigth operation URL is /v2/subscriptions, not /v1/subscriptions as shown in your example.
EDIT 1: considering the new payload in EDIT 1 section in the question, note that:
You should use attrs inside condition, not attributes (the error message is complaining about that)
You should use attrs inside notification, not attributes
You should use "q": "dFlag>0" (i.e. no whitespaces in the query string). Look to the example in the NGSIv2 specification:
Eg:
"expression": {
"q": "temperature>40"
}
EDIT 2: Orion versions previous to 1.2 use attrsFormat outside notification field. Thus, if you are using Orion 1.1, try to use "attrsFormat": "legacy" as first level field in the subscription payload JSON.

Orion JSON Bad Request

I'm currently trying to subscribe Orion and Cosmos. All data sent to Orion is being updated without any issue. But, when posting to http://xxx.xxx.xx.xx:1026/v1/subscribeContext I'm getting the following error:
{
"subscribeError": {
"errorCode": {
"code": "400",
"reasonPhrase": "Bad Request",
"details": "JSON Parse Error"
}
}
}
This is the json string I'm sending:
{
"entities": [
{
"type": "Location",
"isPattern": "false",
"id": "Device-1"
}
],
"reference": "http://52.31.144.170:5050/notify",
"duration": "PT10S",
"notifyConditions": [
{
"type": "ONCHANGE",
"condValues": [
"position"
]
}
],
"attributes": [
"position"
]
}
The entity updating OK in Orion is:
{
"type": "Location",
"isPattern": "false",
"id": "Device-1",
"attributes": [
{
"name": "position",
"type": "coords",
"value": "24,21",
"metadatas": [
{
"name": "location",
"type": "string",
"value": "WGS84"
}
]
},
{
"name": "id",
"type": "device",
"value": "1"
}
]
}
I've tried with many different examples from readthedocs, and other responses in StackOverflow unsuccessfully.
Which is the correct format? Should I call /subscribeContext before or after updating Orion with /contextEntities?
Orion Context Broker version is 0.26.1.
Thank you in advance.
Taking into account that the same payload works ok when send using curl (see this execution session) I tend to think that some issue in the client (maybe hiden by a programming framework?) is causing the problem.

Can't use parenthesis ( ) in an entity's attribute value

I've been experimenting with Orion the last few days and and after a while I made it work the way I intended to.
I am facing an unexpected annoyance though, where a string attribute for a given entity context will fail to register.
Example:
http POST [my_orion_server_url]:1026/v1/updateContext < create_push.json
Where create_push.json is:
{
"contextElements": [
{
"type": "Push",
"isPattern": "false",
"id": "Push1",
"attributes": [
{
"name": "message",
"type": "string",
"value": "this is a test)"
}
]
}
],
"updateAction": "APPEND"
}
The server replies with:
HTTP/1.1 200 OK
Content-Length: 129
Content-Type: application/json
Date: Thu, 19 Nov 2015 11:47:03 GMT
{
"errorCode": {
"code": "400",
"details": "Illegal value for JSON field",
"reasonPhrase": "Bad Request"
}
}
If, on the other hand, I remove the parenthesis, everything works as expected:
json file:
{
"contextElements": [
{
"type": "Push",
"isPattern": "false",
"id": "Push1",
"attributes": [
{
"name": "message",
"type": "string",
"value": "this is a test"
}
]
}
],
"updateAction": "APPEND"
}
Server response:
{
"contextResponses": [
{
"contextElement": {
"attributes": [
{
"name": "message",
"type": "string",
"value": ""
}
],
"id": "Push1",
"isPattern": "false",
"type": "Push"
},
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
}
]
}
Is this intended or a bug?
I can overcome the issue coding the parenthesis with something else and then decoding on reception, but being able to send a parenthesis straight with the entity would be way smoother and less annoying.
That is the expected behaviour. Please have a look to Orion documentation.