Orion Subscription to update entity attributes from CrateDB, on value changes - fiware

Is there a convenient way to subscribe to update an attribute, each time another attribute changes?
I have this entity on Orion:
{
"id": "Asset:001",
"type": "SomeType",
"Input": {
"type": "Text",
"value": "State",
"metadata": {}
},
"Output": {
"type": "Float",
"value": null,
"metadata": {}
}
}
I subscribe to notify value changes to Quantumleap:
curl -s -o /dev/null -X POST \
'http://orion:1026/v2/subscriptions/' \
-H 'Content-Type: application/json' \
-d '{
"description": "Orion notify Quantumleap on State changhes",
"subject": {
"entities": [
{
"idPattern": ".*",
"type": "Sometype"
}
],
"condition": {
"attrs": [
"state"
]
}
},
"notification": {
"http": {
"url": "http://quantumleap:8668/v2/notify"
},
"attrs": [
"State"
],
"metadata": []
}
}'
And I got my data on CrateDB where I made some calculation.
Then I would like to get the results of this computing, and put back as an attribute to the entity Asset:001.
Now I get this done by a script, but it would be nice to achieve the same result with a subscription, is this possible?
Thanks!!

The attributes in attrs within notification are the ones to be included in the notification. The attributes in attrs within condition are the ones which change trigger the notification.
You can use a different set of attributes in each array, so the use case you describe in covered.
For instance, changes in attribute A will notify the value of attribute B using something like this:
{
...
"subject": {
...
"condition": {
"attrs": [
"A"
]
}
},
"notification": {
...
"attrs": [
"B"
]
}
}

Related

Delete json block with jq command

I have json file with multiple domains which is formated as is showed below. How can I delete whole blocks with domains? For example if I will want to delete whole block in json for domain domain.tld?
I tryed this, but output is error:
jq '."http-01"."domain"[]."main"="domain.tld"' acme.json
jq: error (at acme.json:11483): Cannot iterate over null (null)
formating example file:
{
"http-01": {
"Account": {
"Email": "mail#placeholder.tld",
"Registration": {
"body": {
"status": "valid",
"contact": [
"mailto:mail#placeholder.tld"
]
},
"uri": "https://acme-v02.api.letsencrypt.org/acme/acct/110801506"
},
"PrivateKey": "main_priv_key_string",
"KeyType": "4096"
},
"Certificates": [
{
"domain": {
"main": "www.some_domain.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
},
{
"domain": {
"main": "some_domain.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
},
{
"domain": {
"main": "www.some_domain2.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
},
{
"domain": {
"main": "some_domain2.tld"
},
"certificate": "cert_string",
"key": "key_string",
"Store": "default"
}
]
}
}
To delete domain block "www.some_domain.tld" :
jq '."http-01".Certificates |= map(select(.domain.main != "www.some_domain.tld"))' input.json
Your question is quite broad. What is a "block"?
Let's assume you want to delete from within the object under http-01 each field that is of type array and has at index 0 an object satisfying .domain.main == "domain.tld". Then first navigate to where you want to delete from, and update it (|=) using del and select which performs the filtered deletion.
jq '
."http-01" |= del(
.[] | select(arrays[0] | objects.domain.main == "domain.tld")
)
' acme.json
{
"http-01": {
"Account": {
"Email": "email#domain.tld",
"Registration": {
"body": {
"status": "valid",
"contact": [
"mailto:email#domain.tld"
]
},
"uri": "https://acme-v02.api.letsencrypt.org/acme/acct/110801506"
},
"PrivateKey": "long_key_string",
"KeyType": "4096"
}
}
}
Demo
If your "block" is deeper, go deeper before updating. If it is higher, the whole document for instance, there's no need to update, just start with del.

Fiware Context Provider hello world not working

I'm trying to register a Context Provider as source for several KPIs.
So far, it seems registering might be working, as GET http://{{orion}}/v2/registrations returns something similar to what I set in creation:
{
// each registering a new id is returned: "id": "60991a887032541f4539a71d",
"description": "City Inhabitants",
"dataProvided": {
"entities": [
{
"id": "city.inhabitants"
//,"type": "KeyPerformanceIndicator"
}
]
},
"provider": {
"http": {
"url": "http://myhost/v2/inhabitants"
}
//, "legacyForwarding": false //perhaps there's a bug, cause although unset or set to false, broker still returns true.
}
}
However GET http://{{orion}}/v2/entities/city.inhabitants results in:
{
"error": "BadRequest",
"description": "Service not found. Check your URL as probably it is wrong."
}
GET http://{{orion}}/v2/entities?type=KeyPerformanceIndicator returns [] and context-provider is not being invoked any way.
I'm coding a Node.js application from scratch, to better understand what's going on https://github.com/FIWARE/tutorials.Context-Providers, and using tutorial containers as actual Fiware Broker, so all stores/shelf/products are working but not my KPI.
Using the tutorial application where a microservice is listening on the http://context-provider:3000/random/weatherConditions POST endpoint.
The following registration:
curl -L -X POST 'http://localhost:1026/v2/registrations' \
-H 'Content-Type: application/json' \
--data-raw '{
"description": "Get Weather data for KPI 1",
"dataProvided": {
"entities": [
{
"id" : "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator"
}
],
"attrs": [
"temperature", "relativeHumidity"
]
},
"provider": {
"http": {
"url": "http://context-provider:3000/random/weatherConditions"
},
"legacyForwarding": false
},
"status": "active"
}'
Will forward and retrieve data for the following request:
curl -L -X GET 'http://localhost:1026/v2/entities/urn:ngsi-ld:KPI:010'
or
curl -L -X GET 'http://localhost:1026/v2/entities/?type=KeyPerformanceIndicator'
Returns:
{
"id": "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator",
"temperature": {
"type": "Number",
"value": 11,
"metadata": {}
},
"relativeHumidity": {
"type": "Number",
"value": 39,
"metadata": {}
}
}
The context broker is forwarding the request to the http://context-provider:3000/random/weatherConditions POST end-point with the following payload:
{
"entities": [
{
"id": "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator"
}
],
"attrs": [
"temperature",
"relativeHumidity"
]
}

Which characters can be used in an attribute name? - FIWARE Orion

I'd like to know details about character set and character code that can be used in an attribute name. I tried to create an entity that has an attribute name "日本語". I was able to create it in case of NGSIv1 but not in case of NGSIv2.
In case of NGSIv1 API,
Request:
(curl localhost:1026/v1/updateContext -sS --header 'Content-Type: application/json' \
--header 'Accept: application/json' -d #- <<EOF) | jq .
{
"contextElements": [
{
"type": "Test",
"isPattern": "false",
"id": "Testv1",
"attributes": [
{
"name": "japanese",
"type": "Integer",
"value": "1"
},
{
"name": "日本語",
"type": "Integer",
"value": "2"
}
]
}
],
"updateAction": "APPEND"
}
EOF
Response:
{
"contextResponses": [
{
"contextElement": {
"type": "Test",
"isPattern": "false",
"id": "Testv1",
"attributes": [
{
"name": "japanese",
"type": "Integer",
"value": ""
},
{
"name": "日本語",
"type": "Integer",
"value": ""
}
]
},
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
}
]
}
In case of NGSIv2 API,
Request:
(curl localhost:1026/v2/entities -s -S -H 'Content-Type: application/json' \
-d #- <<EOF) | jq .
{
"id": "Test1",
"type": "Test",
"japanese": {
"value": 1,
"type": "Integer"
},
"日本語": {
"value": 2,
"type": "Integer"
}
}
EOF
Response:
{
"error": "BadRequest",
"description": "Invalid characters in attribute name"
}
The relevant sections in NGSIv2 specification regarding allowed characters are "Field syntax restrictions", "Attribute names restrictions" and "Metadata names restrictions".
In particular, related with the topic of this question:
Allowed characters are the ones in the plain ASCII set, except the following ones: control characters, whitespace, &, ?, / and #.
Note also
In addition to the above rules, given NGSIv2 server implementations could add additional syntactical restrictions in those or other fields, e.g., to avoid cross script injection attacks.
which, in the case of Orion it's specified as part of its documentation here.

How to make an Orion NGSIv2 subscription that notifies cygnus in NGSIv1?

EDIT 2
curl --include \
--header 'Content-Type: application/json' \
--request POST \
--data-binary '{
"description": "Day subscription",
"subject": {
"entities": [
{
"idPattern": "es-leon-.*",
"type": "Event"
}
],
"condition": {
"attrs": [
"Title",
"dFlag"
],
"expression": {
"q": "dFlag>0"
}
}
},
"notification": {
"http": {
"url" : "http://localhost:5050/notify"
},
"attrs": [
"Title",
"dFlag"
],
"attrsFormat":"legacy"
}
}' \
'http://localhost:1026/v2/subscriptions'
In Orion the subscription register ok (thanks for the tips in the comments) but I'm having the same issue as said in here even though I use "attrsFormat":"legacy"
I'm making something wrong? why orion is not using ngsiv1 to send the notification to cygnus?
The cygnus error trace.
Cygnus shows orion is responding this:
{
"subscriptionId": "574315e77775f31b8d3da719",
"data": [{
"id": "es-leon-0",
"type": "Event",
"Title": {
"type": "none",
"value": "pepe",
"metadata": {}
},
"dFlag": {
"type": "text",
"value": "1",
"metadata": {}
}
}]
}
When it has to respond the lines below to be in ngsiv1:
{
"subscriptionId": "5743178d7775f31b8d3da71a",
"originator": "localhost",
"contextResponses": [{
"contextElement": {
"type": "Event",
"isPattern": "false",
"id": "es-leon-0",
"attributes": [{
"name": "Title",
"type": "text",
"value": "pep"
}, {
"name": "dFlag",
"type": "text",
"value": "1"
}]
},
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
}]
}
Original Question
As said in other question :
EDIT: note also that you can use NGSIv2 to create/update entities at
Orion and have notifications in NGSIv1 if you:
Create the subscription using NGSIv1 operations
Create the subscription using NGSIv2 operations with attrsFormat equal to legacy.
Have a look to more detailed information here.
So I made up this subscription:
curl --include \
--header 'Content-Type: application/json' \
--request POST \
--data-binary '{
"description": "Day subscription",
"subject": {
"entities": [
{
"idPattern": "es-leon-.*",
"type": "Event"
}
],
"condition": {
"attributes": [
"Title",
"dFlag"
],
"expression": {
"q": "dFlag > 0"
}
}
},
"notification": {
"callback": "http://localhost:5050/notify",
"attributes": [
"Title",
"dFlag"
]
}
}' \
'http://localhost:1026/v1/subscriptions'
but orion don't let me register it throwing this error:
HTTP/1.1 400 Bad Request
Connection: Keep-Alive
Content-Length: 67
Content-Type: application/json
Fiware-Correlator: 2ecdfc74-1c2f-11e6-82d7-000d3a23bf27
Date: Tue, 17 May 2016 12:59:25 GMT
{"error":"BadRequest","description":"no condition attrs specified"}
Is this the way to use legacy attributes so it can work with cygnus with NGSIv2?
Thanks for the help.
EDIT 1:
Following the answer the subscription should be like this: Right?
curl --include \
--header 'Content-Type: application/json' \
--request POST \
--data-binary '{
"description": "Day subscription",
"subject": {
"entities": [
{
"idPattern": "es-leon-.*",
"type": "Event"
}
],
"condition": {
"attributes": [
"Title",
"dFlag"
],
"expression": {
"q": "dFlag > 0"
}
}
},
"notification": {
"http": {
"url" : "http://localhost:5050/notify"
},
"attributes": [
"Title",
"dFlag"
],
"attrsFormat":"legacy"
}
}' \
'http://localhost:1026/v2/subscriptions'
But I'm still getting an error:
HTTP/1.1 100 Continue
HTTP/1.1 400 Bad Request
Connection: Keep-Alive
Content-Length: 67
Content-Type: application/json
Fiware-Correlator: 60a0a1d2-1ddf-11e6-8bd6-000d3a23bf27
Date: Thu, 19 May 2016 16:33:11 GMT
{"error":"BadRequest","description":"no condition attrs specified"}
Use attrs instead of attributes (both in subject.conditions and in notification).
In addition, note that callback is no longer used. You have to use:
"notification": {
"http": {
"url": "http://localhost:5050/notify"
},
..
}
In addition, if you want notifications to use NGSIv1 format you should include attrsFormat field with value legacy in notification as described in the documentation.
Finally, note that the rigth operation URL is /v2/subscriptions, not /v1/subscriptions as shown in your example.
EDIT 1: considering the new payload in EDIT 1 section in the question, note that:
You should use attrs inside condition, not attributes (the error message is complaining about that)
You should use attrs inside notification, not attributes
You should use "q": "dFlag>0" (i.e. no whitespaces in the query string). Look to the example in the NGSIv2 specification:
Eg:
"expression": {
"q": "temperature>40"
}
EDIT 2: Orion versions previous to 1.2 use attrsFormat outside notification field. Thus, if you are using Orion 1.1, try to use "attrsFormat": "legacy" as first level field in the subscription payload JSON.

How can I index .JSON in elasticsearch

I am starting with elasticsearch now and i don't know anything about it.
I have folowing .JSON:
[
{
"label": "Admin Law",
"tags": [
"#admin"
],
"owner": "generalTopicTagText"
},
{
"label": "Judicial review",
"tags": [
"#JR"
],
"owner": "generalTopicTagText"
},
{
"label": "Admiralty/Shipping",
"tags": [
"#shipping"
],
"owner": "generalTopicTagText"
}
]
My mapping is this:
{
"topic_tax": {
"properties": {
"label": {
"type": "string",
"index": "not_analyzed"
},
"tags": {
"type": "string",
"index_name": "tag"
},
"owner": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
I need to put the first .Json into Elasticsearch, but it does not work.
All I know is that i am defining only 1 of this:
{
"label": "Judicial review",
"tags": [
"#JR"
],
"owner": "generalTopicTagText"
}
So when i try to put all of them with my elasticsearch.init, it will not work.
But I really don't know how to declare the mapping.Json to put the all .Json, it is like i need something like a for there.
You have to insert them json after json. But what you should do is use the bulk api of elasticsearch to insert multiple documents in one request. Check this api doc to see how it works
You can do something like this
curl -XPUT 'localhost:9000/es/post/1?version=2' -d '{
"text" : "your test message!"
}'
here is the documentation for index json with elasticsearch