IoTAgent (JSON) over MQTT doesn't receive measurements from mosquitto - json

Goal: to use the IoTAgent (JSON) provided by FIWARE with the MQTT transport protocol. In particular, I would like to provision a service group rather than individual devices, such that anonymous devices can send their measurements to the IoTAgent via the Mosquitto Broker.
Problem: Mosquitto Broker receives messages (sent by an MQTT publisher) but IoTAgent does not. Both are on the same network (I used docker compose), so ruled out this could be the reason of the problem.
The docker-compose.yaml file looks like this:
version: "3.5"
services:
mosquitto:
image: eclipse-mosquitto:1.6.14
hostname: mosquitto
container_name: mosquitto
expose:
- "1883"
- "9001"
ports:
- "1883:1883"
- "9001:9001"
networks:
- default
iot-agent:
image: fiware/iotagent-json:latest
hostname: iot-agent
container_name: fiware-iot-agent
depends_on:
- mongo-db
- mosquitto
networks:
- default
expose:
- "4041"
ports:
- "4041:4041"
environment:
- IOTA_CB_HOST=orion
- IOTA_CB_PORT=1026
- IOTA_CB_NGSI_VERSION=ld
- IOTA_JSON_LD_CONTEXT=http://context/ngsi-context.jsonld
- IOTA_NORTH_PORT=4041
- IOTA_MQTT_HOST=mosquitto
- IOTA_MQTT_PORT=1883
- IOTA_MQTT_QOS=1
- IOTA_MQTT_KEEPALIVE=60
- IOTA_DEFAULT_RESOURCE= # Default is blank. I'm using MQTT so I don't need a resource
- IOTA_REGISTRY_TYPE=mongodb
- IOTA_MONGO_HOST=mongo-db
- IOTA_MONGO_PORT=27017
- IOTA_MONGO_DB=iotagentjson
- IOTA_LOG_LEVEL=DEBUG
- IOTA_TIMESTAMP=true
- IOTA_AUTOCAST=true
- IOTA_FALLBACK_TENANT=openiot
The full docker-compose.yaml is available on GitLab repository.
The service group provisioning is shown below:
{
"services": [
{
"apikey": "4jggokgpepnvsb2uv4s40d59ov",
"entity_type": "TrafficFlowObserved",
"resource": "",
"expressionLanguage": "jexl",
"attributes": [
{"name": "id", "type": "Text", "expression": "'urn:ngsi-ld:TrafficFlowObserved:'+idelem"},
{"name": "dateObserved", "type": "Text", "expression": "fecha_hora_inicio|toisodate+'/'+fecha_hora_finalizacion|toisodate"},
{"object_id": "intensidad", "name": "intensity", "type": "Number"}
]
}
]
}
I expect the IoTAgent to receive measurements from anonymous devices as well as mosquitto. Instead, currently, the IoTAgent does not receive the measurements and gives the following error when I provision the service group: DEVICE_GROUP_NOT_FOUND.
The logs are shown on GitLab repository
I can't figure out the problem.

On the assumption you have provisioned a device as follows:
curl -iX POST \
'http://localhost:4041/iot/services' \
-H 'Content-Type: application/json' \
-H 'fiware-service: openiot' \
-H 'fiware-servicepath: /' \
-d '{
"services": [
{
"apikey": "4jggokgpepnvsb2uv4s40d59ov",
"cbroker": "http://orion:1026",
"entity_type": "Thing",
"resource": ""
}
]
}'
curl -iX POST \
'http://localhost:4041/iot/devices' \
-H 'Content-Type: application/json' \
-H 'fiware-service: openiot' \
-H 'fiware-servicepath: /' \
-d '{
"devices": [
{
"device_id": "motion001",
"entity_name": "urn:ngsi-ld:Motion:001",
"entity_type": "Motion",
"protocol": "PDI-IoTA-UltraLight",
"transport": "MQTT",
"timezone": "Europe/Berlin",
"attributes": [
{ "object_id": "c", "name": "count", "type": "Integer" }
]
}
]
}'
You will need to post your data to the /json/<api-key>/<device-id>/attrs topic. It is important to include both the correct protocol json and the correct <api-key> - the message body is something like {"c": 1}.
This can be used to test the IoT Agent South Port without an actual device - assuming it works, you can then check that the Device is posting to the correct topic.

Related

Multiple entities can't be saved at same time in Mongo historical database

I am trying to save multiple data entities in historical database, but only the first one from the array is saved in Mongo historical database.
Below are the steps I performed.
Docker-compose for Cygnus:
version: "3.8"
services:
cygnus:
hostname: cygnus
container_name: cygnus
image: fiware/cygnus-ngsi:2.16.0
networks:
- default_config
ports:
- "5080:5080"
- "5051:5051"
expose:
- "5051"
- "5080"
environment:
- CYGNUS_MONGO_SERVICE_PORT=5051
- CYGNUS_MONGO_HOSTS=mongo_db:27017
- CYGNUS_MONGO_USER=root
- CYGNUS_MONGO_PASS=root
- CYGNUS_MONGO_AUTH_SOURCE=admin
- CYGNUS_MONGO_DATA_MODEL=dm-by-service-path
- CYGNUS_MONGO_ATTR_PERSISTENCE=column
- CYGNUS_SERVICE_PORT=5051
- CYGNUS_API_PORT=5080
- CYGNUS_LOG_LEVEL=DEBUG
- CYGNUS_ORION_HOST=orion
- CYGNUS_ORION_PORT=1026
- CYGNUS_ORION_SSL=true
networks:
default_config:
external: true
Create a Cygnus subscription:
curl --location --request POST 'http://localhost1026/v2/subscriptions' \
--header 'Content-Type: application/json' \
--data-raw '{
"description": "Notify Cygnus Mongo-DB of all context changes",
"subject": {
"entities": [
{
"idPattern": ".*"
}
]
},
"notification": {
"http": {
"url": "http://10.0.0.5:5051/notify"
}
},
"throttling": 5
}'
Batch Create a New Data Entity :
curl --location --request POST 'http://localhost:1026/v2/op/update/' \
--header 'Content-Type: application/json' \
--data-raw '{
"actionType":"append_strict",
"entities":[
{
"id":"urn:ngsi-ld:Product:100", "type":"Product",
"name":{"type":"Text", "value":"Brandy"},
"size":{"type":"Text", "value": "M"},
"price":{"type":"Integer", "value": 1199}
},
{
"id":"urn:ngsi-ld:Product:101", "type":"Product",
"name":{"type":"Text", "value":"Port"},
"size":{"type":"Text", "value": "M"},
"price":{"type":"Integer", "value": 1099}
}
]
}'
Result in Mongo:
Group updates also do not work.
Did I miss something in the configuration or there is no possibility for that in Cygnus?
Thank you in advance.
A first impression. At subscription you are using: "throttling": 5
throttling: Minimal period of time in seconds which must elapse
between two consecutive notifications. It is optional.
That implies that just ONE notificación will be send in every 5 seconds time window. With your NGSIv2 batch update surely all notification will fall in the 5 sec window.
Try to remove that property.
https://fiware-orion.readthedocs.io/en/master/user/ngsiv2_implementation_notes/index.html#notification-throttling

Cannot register commands in IoT Agent Fiware

I'm using IoT Agent with Ultralight run directly (not as a dockerized component). I'm able to register new device, which is mapped in Orion Context Broker. However I have an issue with registering Actuators which require commands within configuration data. I use:
curl -iX POST \
'http://MYIP:4061/iot/devices' \
-H 'Content-Type: application/json' \
-H 'fiware-service: openiot' \
-H 'fiware-servicepath: /' \
-d '{
"devices": [
{
"device_id": "lamp003",
"entity_name": "urn:ngsi-ld:Lamp:003",
"entity_type": "Lamp",
"protocol": "PDI-IoTA-UltraLight",
"transport": "MQTT",
"commands": [
{"name": "on","type": "command"},
{"name": "off","type": "command"}
]}
]
}
And in response I'm hit with:
op=IoTAgentNGSI.DeviceProvisioning | from=n/a | srv=n/a | subsrv=n/a |
msg=Device provisioning failed due to the following error: |
comp=IoTAgent Request error connecting to the Context Broker:
{"code":"400","reasonPhrase":"Bad Request","details":"missing isDomain
value for registration attribute"}
I tried adding "isDomain": "false" to elements of command table, but no luck. It used to work before and furthermore similar solution is presented on official guide.
isDomain part of the deprecated NGSI-v1 syntax. It is not used in NGSI-v2 or NGSI-LD
I think you may be running the IoT Agent as NGSI-v1. You'll need to set contextBroker.ngsiVersion in your config.js or set the following Env variable in Docker or Docker-compose.
IOTA_CB_NGSI_VERSION=v2
config.js
{
host: '192.168.56.101',
port: '1026',
ngsiVersion: 'v2'
}

Fiware orion subscription sending empty body

My setup:
docker-compose:
version: "3.1"
services:
mongo-db:
image: mongo:3.6
hostname: mongo-db
container_name: db-mongo
expose:
- "27017"
ports:
- "27017:27017"
networks:
- default
command: --bind_ip_all --smallfiles
volumes:
- mongo-db:/data
orion:
image: fiware/orion:2.1.0
hostname: orion
container_name: fiware-orion
depends_on:
- mongo-db
networks:
- default
expose:
- "1026"
ports:
- "1026:1026"
command: -dbhost mongo-db -logLevel DEBUG
logging:
driver: none
healthcheck:
test: curl --fail -s http://localhost:1026/version || exit 1
iot-agent:
image: fiware/iotagent-ul:latest
hostname: iot-agent
container_name: fiware-iot-agent
depends_on:
- mongo-db
networks:
- default
expose:
- "4041"
- "7896"
ports:
- "4041:4041"
- "7896:7896"
logging:
driver: none
environment:
- "IOTA_CB_HOST=orion"
- "IOTA_CB_PORT=1026"
- "IOTA_NORTH_PORT=4041"
- "IOTA_REGISTRY_TYPE=mongodb"
- "IOTA_LOG_LEVEL=DEBUG"
- "IOTA_TIMESTAMP=true"
- "IOTA_MONGO_HOST=mongo-db"
- "IOTA_MONGO_PORT=27017"
- "IOTA_CB_NGSI_VERSION=v2"
- "IOTA_MONGO_DB=iotagentul"
- "IOTA_HTTP_PORT=7896"
- "IOTA_PROVIDER_URL=http://iot-agent:4041"
networks:
default:
ipam:
config:
- subnet: 172.18.1.0/24
volumes:
mongo-db: ~
context:
curl -iX POST \
'http://localhost:1026/v2/op/update' \
-H 'Content-Type: application/json' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-d '{
"actionType":"APPEND",
"entities":[
{
"id":"urn:ngsi-ld:Furniture:unit001", "type":"Furniture",
"name":{
"type":"Text", "value":"Bürostuhl"
},
"price":{
"type":"Integer", "value":404.40
}
},
{
"id":"urn:ngsi-ld:Furniture:unit002", "type":"Furniture",
"name":{
"type":"Text", "value":"Bürotisch"
},
"price":{
"type":"Integer", "value":203.40
}
},
{
"id":"urn:ngsi-ld:DeliverService:unit001", "type":"ServiceProvider",
"name":{
"type":"Text", "value":"trinkajo"
},
"address":{
"type":"PostalAddress",
"value":{
"telephoneNumber":"43q4q53",
"internetAdresse":"https://www.345345.de/"
}
}
},
{
"id":"urn:ngsi-ld:BottleCounter:001", "type":"BottleCounter",
"name":{
"type":"Text", "value":"Wasserflaschenzähler"
},
"numberOfBottles": {"type":"Integer", "value":645}
}
]
}'
curl -iX POST \
'http://localhost:1026/v2/op/update' \
-H 'Content-Type: application/json' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-d '{
"actionType":"APPEND",
"entities":[
{
"id":"urn:ngsi-ld:Furniture:unit001", "type":"Furniture",
"refOffice": {
"type": "Relationship",
"value": "urn:ngsi-ld:Office:001"
}
},
{
"id":"urn:ngsi-ld:Furniture:unit002", "type":"Furniture",
"refOffice": {
"type": "Relationship",
"value": "urn:ngsi-ld:Office:001"
}
},
{
"id":"urn:ngsi-ld:DeliverService:unit001", "type":"ServiceProvider",
"refOffice": {
"type": "Relationship",
"value": "urn:ngsi-ld:Office:001"
}
}
]
}'
curl -iX POST \
'http://localhost:1026/v2/op/update' \
-H 'Content-Type: application/json' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-d '{
"actionType":"APPEND",
"entities":[
{
"id":"urn:ngsi-ld:Office:001", "type" : "Office",
"name":{
"type":"Text",
"value":"didi-Hamburg"
},
"sizeInM2":{
"type":"Integer",
"value":60
},
"address":{
"type":"PostalAddress",
"value":{
"country":"Germany",
"locality":"sdfg",
"street":"sdfg",
"houseNumber" : "34",
"postalCode":"34533"
}
}
}
]
}'
curl -iX POST \
'http://localhost:4041/iot/services' \
-H 'Content-Type: application/json' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-d '{
"services": [
{
"apikey": "4jggokgpepnvsb2uv4s40d59ov",
"cbroker": "http://orion:1026",
"entity_type": "Thing",
"resource": "/iot/d"
}
]
}'
curl -G -X GET \
'http://localhost:1026/v2/entities/urn:ngsi-ld:Office:001' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-d 'type=Office' \
-d 'options=keyValues' | json_pp
curl -iX POST \
'http://localhost:4041/iot/devices' \
-H 'Content-Type: application/json' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-d '{
"devices": [
{
"device_id": "bottleCounter001",
"entity_name": "urn:ngsi-ld:BottleCounter:001",
"entity_type": "BottleCounter",
"timezone": "Europe/Berlin",
"attributes": [
{ "object_id": "c", "name": "numberOfBottles", "type": "Integer" }
],
"static_attributes": [
{ "name":"refOffice", "type": "Relationship", "value": "urn:ngsi-ld:Office:001"}
]
}
]
}'
The subscription:
curl -iX POST \
--url 'http://localhost:1026/v2/subscriptions' \
--header 'Content-Type: application/json' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
--data '{
"description": "Notify me of low stock in Office 001",
"subject": {
"entities": [{"idPattern": ".*", "type" : "BottleCounter"}],
"condition": {
"attrs": ["numberOfBottles"],
"expression": {"q": "numberOfBottles<10;refOffice==urn:ngsi-ld:Office:001"}
}
},
"notification": {
"http": {
"url": "http://delivery:8080/subscription/office"
}
}
}'
How I successfully update the value numberOfBottles:
curl -iX POST \
'http://localhost:7896/iot/d?k=4jggokgpepnvsb2uv4s40d59ov&i=bottleCounter001' \
-H 'fiware-service: didi' \
-H 'fiware-servicepath: /' \
-H 'Content-Type: text/plain' \
-d 'c|2'
My usecase is something like:
There is an office with water bottles for the employee. There is a
'sensor' which can count the water bottles. When an employee is taking
a water bottle the sensor will update the value in fiware. When the
number is below 3 fiware should notify the subscriber. The subscriber
can now order new water bottles.
I have several problems:
With the condition numberOfBottles<10 the subscription never get triggered. When I use something like numberOfBottles!=10 it is working
There is only an empty body sent to the url http://delivery:8080/subscription/office
A bonus question: if I can fix the first issues:
Is there a way to send the attribute address from urn:ngsi-ld:DeliverService:unit001 to the url http://delivery:8080/subscription/office when the subscription get triggered?
As far as I remember, IOTAs are configured by default to use NGSIv1 to send updates to CB. One of the limitations of the NGSIv1 API (deprecated :) is that numbers are parsed always as strings. Thus it makes sense the numberOfBottles<10 filter doesn't work.
In order to overcome this problem IOTA should be configured to use NGSIv2. Add the following to the envrironment of the iot-agent in your docker-compose.yml and redeploy:
- IOTA_CB_NGSI_VERSION=v2
With regards to bonus question, I think #JasonFox has already answered it in the question comments.
EDIT: I have realized you are using UL agent. Different from JSON agent (which allows you to specify JSON supported values, such as numbers, in the request sent to the agent) UL encoding is based on text. Thus, in order to progress numeric values to CB you need (in addition to the IOTA_CB_NGSI_VERSION setting described above):
Set the IOTA_AUTOCAST configuration to true:
- IOTA_AUTOCAST=true
Use "Number" as attribute type at provisioning time (instead of "Integer").
This is described in more detail in this section of the documentation.

Setting up LWM2M device communicates with IDAS

I am new to Fiware and need help.
I want to configure a road side device (sensor) using CoAP protocol to the IDAS IoT agent (Lightweight M2M agent), so this device can send some data to IDAS.
How can I accomplish this task?
We are a company that works with LwM2M protocol through FIWARE technologies, may be our IoT FIWARE Dockerized infrastructure could help you.
https://gitlab.hopu.eu/software/FIWARE/fiware-docker-infrastructure
Reading your comments before, I understand that you want to build an scenario in order to connect your sensors to IotAgent-LWM2M.
In order to use LW2M2 IotAgent:
Install LW2M2 agent: git clone https://github.com/telefonicaid/lightweightm2m-iotagent.git
Install yarm to install all dependencies using npm packages
Install Lightweight M2M client: git clone https://github.com/telefonicaid/lwm2m-node-lib.git
Technical Requirements:
Mosquito MQTT v3.1 Broker
Orion latest
MongoDB v.3.2
NodeJS v0.12
I suggest you to use docker to install dependencies before comment
version : "2"
services:
mongo:
image: mongo:3.2
command: --nojournal
ports:
- "27017:27017"
expose:
- "27017"
orion:
image: fiware/orion
links:
- mongo
ports:
- "1026:1026"
command: -dbhost mongo
expose:
- "1026"
mosquitto:
image: ansi/mosquitto
ports:
- "1883:1883"
expose:
- "1883"
Note:
Mosquitto play a role like a sensor
Getting started: StepByStep
Step 1 : Create a device
(curl localhost:4041/iot/devices -s -S --header 'Content-Type: application/json' \
--header 'Accept: application/json' --header 'fiware-service: Factory' --header 'fiware-servicepath: /robots' \
-d #- | python -mjson.tool) <<EOF
{
"devices": [
{
"device_id": "robot1",
"entity_type": "Robot",
"attributes": [
{
"name": "Battery",
"type": "number"
}
],
"lazy": [
{
"name": "Message",
"type": "string"
}
],
"commands": [
{
"name": "Position",
"type": "location"
}
],
"internal_attributes": {
"lwm2mResourceMapping": {
"Battery" : {
"objectType": 7392,
"objectInstance": 0,
"objectResource": 1
},
"Message" : {
"objectType": 7392,
"objectInstance": 0,
"objectResource": 2
},
"Position" : {
"objectType": 7392,
"objectInstance": 0,
"objectResource": 3
}
}
}
}]}
EOF
Step 2 : Create a service
curl -X POST -H "Fiware-Service: myHome" -H "Fiware-ServicePath: /environment" -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{
"services": [
{
"resource": "/",
"apikey": "",
"type": "Robot",
"cbroker":"localhost:1026"
}]
}' 'http://localhost:4041/iot/services'
Step 3: connect your sensor to the client
(bin/iotagent-lwm2m-client.js)
Object Creation:
LWM2M-Client> create /7392/0
Battery attribute:
LWM2M-Client> set /7392/0 1 89
Message Attribute:
LWM2M-Client> set /7392/0 2 "First robot here"
Position attribute:
LWM2M-Client> set /7392/0 3 "[0,0]
Step 4: connect with the server
LWM2M-Client> connect localhost 5684 robot1 /
Step 5: Update attributes
set /7392/0 1 67
Step 6: Query to Orion to see updated attributes
curl -X POST http://localhost:1026/v1/queryContext -s -S
--header 'Content-Type: application/json' \
--header 'Accept: application/json' --header 'fiware-service: Factory'
--header 'fiware-servicepath: /robots' \
-d '
{
   "entities": [
       {
           "type": "Robot",
           "isPattern": "false",
           "id": "Robot:robot1"
       }
   ]
}

Provision device with JSON/MQTT IOT AGENT FIWARE

curl -X POST -vv -H "Fiware-Service: myHome" -H "Fiware-ServicePath: /environment" -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{
"devices": [
{
"device_id": "0000000000000000",
"entity_name": "BedRoomSensor",
"entity_type": "multiSensor",
"attributes": [
{ "object_id": "t", "name": "Temperature", "type": "celsius" },
{ "object_id": "h", "name": "Humidity", "type": "degrees" }
]
}
]
} 'http://localhost:4041/iot/devices'
I execute the above curl commmand in order to provision my device.However it doesn't show anything and the command never ends.
What i missunderstood?
I have solved the problem following the first two steps of this guide:http://fiwaretourguide.readthedocs.io/en/latest/connection-to-the-internet-of-things/how-to-read-measures-captured-from-iot-devices/
In the step-by-step guide these steps weren't described.
In the second step i change the protocol fielde as "MQTT".
Now whenever i request the measures from my device i take the correct value.