Sending a specific entity to FIWARE QuantumLeap - json

In my application I have a very specific entity format, where one of the attributes' values is a json array of values.
{
"id": "Proximity3",
"type": "SensorAgent",
"measurementType": {
"type": "string",
"value": "boolean",
"metadata": {}
},
"modifiedTime": {
"type": "string",
"value": "2020-06-08T12:30:11.091506Z",
"metadata": {}
},
"readings": {
"type": "array",
"value": [
{
"type": "SensorReading",
"value": {
"reading": {
"type": "boolean",
"value": false
}
}
}
],
"metadata": {}
},
"sanID": {
"type": "string",
"value": "SAN_3",
"metadata": {}
},
"sensorID": {
"type": "string",
"value": "Proximity3",
"metadata": {}
},
"sensorManufacturer": {
"type": "string",
"value": "Unknown",
"metadata": {}
},
"sensorType": {
"type": "string",
"value": "ON_OFF_SENSOR",
"metadata": {}
}
}
]
As you can see, readings attribute has an array of values stored in it (in this example only one.
Whenever I try to subscribe QuantumLeap to Orion Context Broker, subscription is successful, but there is no data received by QuantumLeap (even querying http://localhost:8668/v2/attrs gives me 'No records were found for such query.'). Tried using attrFormat on subscription, but unsuccessful.
At the same time, this https://quantumleap.readthedocs.io/en/latest/user/ suggests that the QuantumLeap expects a specific entity.
My question is, is there anything possible to do in order to get this entity to QuantumLeap, or I would need to change the entity?

It is clear from the QuantumLeap Data Insertion documentation that QuantumLeap expects the notification it receives to contain attributes passed as Native JSON types (like Number)
"temperature": {
"value": 24.2,
"type": "Number",
"metadata": {}
}
If your context broker holds attributes in a different way, direct notification will always fail, but there is nothing stopping you from proxying the notification and reformatting it in the manner that QuantumLeap prefers.
Create a simple subscription listener to read the current subscription data,
reformat the bits you need and forward to Quantum Leap and return the received HTTP success/failure back to the context broker.
const request = require('request');
function receiver(req, res) {
let data = [];
// Manipulate the data
req.body.data.readings.value.forEach((reading) => {
data.push[reading.value]
});
// Copy the rest of data from the original subscription
// Then Forward to QuantumLeap - replace with real values
const options = {
url: '/subscription/to/quantum/leap',
method: 'POST',
json: data,
headers:
};
// Respond back to context broker with response from QL
request(options, (error, response, body) => {
return error ? res.send(error) : res.status(204).send();
});
}

Related

Operator "missing" not working properly in JsonLogic

I am using JsonLogic to validate my input payload with the rules defined using JsonLogic. I am able to test the rules using "Play with it" tool and my rules work perfectly fine against my input data.
But when I run the same rules through my .net Core application by passing payload from Postman the rules always return the else condition even when it should the error from if condition.
{
"if": [
{
"missing": [
"ProposedProjectDetails.IsFreezone",
"ProposedProjectDetails.InterestedToLeaseFrom",
"ProposedProjectDetails.IndustryType",
"ProposedProjectDetails.OtherType",
"ProposedProjectDetails.ProjectDescription",
"ProposedProjectDetails.OutputofFacility",
"ProposedProjectDetails.ProductionCapacity",
"ProposedProjectDetails.ProductionCapacityVolume",
"ProposedProjectDetails.MainRawMaterials",
"ProposedProjectDetails.RawMaterialQuantity",
"ProposedProjectDetails.RawMaterialEstimatedCost",
"ProposedProjectDetails.RawMaterialEstimatedTotInvestment",
"ProposedProjectDetails.AnnualSalesRevenue",
"ProposedProjectDetails.ConstructionStartDate",
"ProposedProjectDetails.Skilledjobs",
"ProposedProjectDetails.TotalAccomodationRequired",
"ProposedProjectDetails.TotalWorkerSalary",
"ProposedProjectDetails.EBITDA",
"ProposedProjectDetails.PortCargoImports",
]
},
"Missing mandatory inputs",
"all good"
]
}
Sample input payload is
{
"companyId": "string",
"serviceCode": "IPA",
"serviceType": "string",
"serviceName": "string",
"crmApplicationId": "string",
"crmReferenceNumber": "string",
"portalReferenceNumber": "string",
"data": {
"proposedProjectDetails": {
"outputofFacility": 2,
"productionCapacity": 0,
"productionCapacityVolume": 0,
"others": "string",
"shiftsPerDay": 0
}
}
}
My .Net code which is evaluating this is
public ValidationResponse Validate(JObject validation, JObject data)
{
var evaluator = new JsonLogicEvaluator(EvaluateOperators.Default);
var result = evaluator.Apply(validation, data);
return new ValidationResponse
{
IsValid = (string.Equals("All Fine", result.ToString(), StringComparison.OrdinalIgnoreCase)),
Message = result
};
}
When I run above code with JsonRules and above Payload, I always get all good in the response. But since I am missing required data in payload, it should get the error Missing mandatory inputs which I do get in JsonLogic "Play with it" tool.
Try keeping the names as json mapping fields camel case once like below
"if": [
{
"missing": [
"proposedProjectDetails.isFreezone",
"proposedProjectDetails.interestedToLeaseFrom",
]
},
"Missing mandatory inputs",
"all good"
]
}

How to add TimeInstant, CreationDate and ModifiedDate into CrateDB with Orion Context Broker?

I'm setting up a Firmware-Framework, where I unforutunately have to add historically Sensor Values. But I also need the creationDate and the modificationDate for other usecases.
Therefore I add the Attribute "Metadata" with the variable "TimeInstant". Then I create an Entity, create an Orion-Subscription for that Entity and update the Entity with my old Sensor-Valses.
The Json-File I send to the Orion-Context Broker to update the Attribute looks like this:
{
"metadata": {
"TimeInstant": {
"type": "DateTime",
"value": "2015-02-02T11:35:25.0000Z"
}
},
"type": "Number",
"value": 0.0132361 }
The Output in my Mongo-DB like this:
"_id": {
"id": "urn:ngsi-ld:SensorB-K1200____",
"type": "Sensor",
"servicePath": "/test/servicepath"
},
"attrNames": [
"Sensor_value"
],
"attrs": {
"Sensor_value": {
"value": 0.01632361,
"type": "Number",
"md": {
"TimeInstant": {
"type": "DateTime",
"value": 1422876989
}
},
"mdNames": [
"TimeInstant"
],
"creDate": 1568712813,
"modDate": 1568735930
}
},
"creDate": 1568712813,
"modDate": 1568735930,
"lastCorrelator": "0a129232-d964-11e9-8e5a-0242ac130009" }
But my Crate-DB only has the columns:
entity_id entity_type fiware_servicepath sensor_value time_index
My Subscription File looks like this:
{
"expires": "2019-12-24T18:00:00",
"notification": {
"http": {
"url": "http://quantumleap:8668/v2/notify"
},
"metadata": [
"dateCreated",
"dateModified",
"TimeInstant"
]
},
"subject": {
"entities": [
{
"id": "urn:ngsi-ld:SensorB-K1200____",
"type": "Sensor"
}
]
},
"throttling": 0 }
I've tried changing the "Metadata" Attributes in the Subscription-File, also tried restartig Crate-DB, ContextBroker e.g..
I excpect the CrateDb to show all three values: "dateCreated", "dateModified" and "TimeInstant".
Did you check what's the message notification actually sent by Orion to QuantumLeap?
As regards the payload I would try as follow:
{
"TimeInstant": {
"type": "DateTime",
"value": "2015-02-02T11:35:25.0000Z"
},
"type": "Number",
"value": 0.0132361
}
Internally we usually use as attribute name for this type of scenario dateObserved, but it would not make any difference w.r.t. TimeInstant.
I am not actually sure you can attach metadata to the root of NGSI message, I believe they are supposed to be attached only to attributes.
Anyhow, QuantumLeaps does not support NGSI metadata (i.e. metadata attached to NGSI attributes). Still it support time indexing based on them.
The way Quantum Leap handles TimeInstant metadata and other time metadata is via time_index. See documentation here: https://quantumleap.readthedocs.io/en/latest/user/#time-index

Postman: schema validation passes even with a wrong response

I have the following schema for bellow happy path response
var responseSchema =
{
"type": "object",
"properties": {
"value": {
"type": "object",
"properties":{
"items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"patientGuid": {"type": "string" },
"givenName": {"type": "string" },
"familyName": {"type": "string" } ,
"combinedName" : {"type": "string" }
},
"required": ["patientGuid","givenName","familyName"]
}
}
}
}
}
};
Happy path response:
{
"value": {
"items": [
{
"patientGuid": "e9530cd5-72e4-4ebf-add8-8df51739d15f",
"givenName": "Vajira",
"familyName": "Kalum",
"combinedName": "Vajira Kalum"
}
],
"href": "http://something.co",
"previous": null,
"next": null,
"limit": 10,
"offset": 0,
"total": 1
},
"businessRuleResults": [],
"valid": true
}
I check if condition to validate whether response schema is correct:
if(responseBody !== null & responseBody.length >0)
{
var responseObject = JSON.parse(responseBody);
if(tv4.validate(responseObject, responseSchema))
{
// do something
}
else
{
// log some msg
}
}
Schema validation condition (nested if) get passed even when I get bellow bad response.
{
"value": {
"items": [],
"href": "http://something.co",
"previous": null,
"next": null,
"limit": 10,
"offset": 0,
"total": 0
},
"businessRuleResults": [],
"valid": true
}
Why response schema validation not failed as a response not has required fields?
When I ran your code and removed a property from the response data it seemed to work ok:
I would suggest a couple of things though - The tv4 module is not fantastic, it's not actively being worked on (for a couple of years I think) and there's a bunch of complaints/issue raised on the Postman project about how poor it is and asking for it to be replaced in the native app.
Have you considered creating a test to check the schema instead? This is very basic and could be easily refactored and wrapped in some logic but it would check the different value types of your response.
pm.test("Response data format is correct", () => {
var jsonData = pm.response.json()
// Check the type are correct
pm.expect(jsonData).to.be.an('object')
pm.expect(jsonData.value).to.be.an('object')
pm.expect(jsonData.value.items).to.be.an('array')
pm.expect(jsonData.value.items[0]).to.be.an('object')
// Check the value types are correct
pm.expect(jsonData.value.items[0].combinedName).to.be.a('string')
pm.expect(jsonData.value.items[0].givenName).to.be.a('string')
pm.expect(jsonData.value.items[0].familyName).to.be.a('string')
pm.expect(jsonData.value.items[0].patientGuid).to.a('string')
});
Also the syntax that you're using is the older style now and you don't need to JSON.parse(responseBody) the response body anymore as pm.response.json() is basically doing the same thing.

AWS Lambda Handler extend S3event

I have the following pipeline:
A file is uploaded to S3, it triggers a Lambda (Let's call it L1) which runs and does some processing.
So at the moment, my entry point looks like this:
public Response handleRequest(S3Event event, Context context) {
....
}
Now, a S3Event JSON looks like this:
{
"Records": [
{
"awsRegion": "xxxxx",
"eventName": "ObjectCreated:Put",
"eventSource": "aws:s3",
"eventTime": "2017-09-12T09:27:59.471Z",
"eventVersion": "2.0",
"requestParameters": {
"sourceIPAddress": "xxxxxx"
},
"responseElements": {
"x-amz-id-2": "xxxxxx",
"x-amz-request-id": "xxxx"
},
"s3": {
"configurationId": "xxxxxx1",
"bucket": {
"name": "xxxxx",
"ownerIdentity": {
"principalId": "xxxxx"
},
"arn": "xxx"
},
"object": {
"key": "xxx",
"size": xxx,
"eTag": "xxxx",
"versionId": null,
"sequencer": "xxx",
"urlDecodedKey": "xxx"
},
"s3SchemaVersion": "1.0"
},
"userIdentity": {
"principalId": "xxxx"
}
}
],
}
If you pass this JSON in the "Test" section, it will succeed.
Now, to the point: I wish to add information to this JSON, something that would look like this:
{
"Records": [
{
"awsRegion": "xxxxx",
"eventName": "ObjectCreated:Put",
"eventSource": "aws:s3",
"eventTime": "2017-09-12T09:27:59.471Z",
"eventVersion": "2.0",
"requestParameters": {
"sourceIPAddress": "xxxxxx"
},
"responseElements": {
"x-amz-id-2": "xxxxxx",
"x-amz-request-id": "xxxx"
},
"s3": {
"configurationId": "xxxxxx1",
"bucket": {
"name": "xxxxx",
"ownerIdentity": {
"principalId": "xxxxx"
},
"arn": "xxx"
},
"object": {
"key": "xxx",
"size": xxx,
"eTag": "xxxx",
"versionId": null,
"sequencer": "xxx",
"urlDecodedKey": "xxx"
},
"s3SchemaVersion": "1.0"
},
"userIdentity": {
"principalId": "xxxx"
}
}
],
"MyErrorMessage":
{
"EnvelopeErrors": [
{
"EnvelopeErrorTrace": "stackTrace",
"EnvelopeErrorPositions": 1,
"EnvelopeErrorLength": 2
},
{
"EnvelopeErrorTrace": "SecondTrace",
"EnvelopeErrorPositions": 3,
"EnvelopeErrorLength": 4
}
],
}
}
Notice is the S3Event JSon but with a bit more data.
My question problem is the following: I want to have a custom input that also works when a pure S3Event is called.
public Response handleRequest(MyS3Event event, Context context) {
....
}
However, I have not been able to achieve this.
I have tried a custom POJO but it does not work when I upload to S3 a file.
I tried to extend the S3EventNotification class (from which S3Event extends), but again with no success.
Is it possible what I am trying to do?
What you can do is to have your Lambda (L1) call itself (asynchronously) by sending it the new, modified event similar to how recursive functions work.
Just be very careful though. You have to put a limit as to how deep you want to keep recursing. You don't want to end up with infinite calls. I am not sure if AWS guards against this.
In the AWS SDK Lambda has an invoke method:
Invokes a specific Lambda function. For an example, see Create the Lambda Function and Test It Manually.
If you are using the versioning feature, you can invoke the specific
function version by providing function version or alias name that is
pointing to the function version using the Qualifier parameter in the
request. If you don't provide the Qualifier parameter, the $LATEST
version of the Lambda function is invoked. Invocations occur at least
once in response to an event and functions must be idempotent to
handle this. For information about the versioning feature, see AWS Lambda Function Versioning and Aliases.
This operation requires permission for the lambda:InvokeFunction
action.
var params = {
FunctionName: 'STRING_VALUE', /* required */
ClientContext: 'STRING_VALUE',
InvocationType: Event | RequestResponse | DryRun,
LogType: None | Tail,
Payload: new Buffer('...') || 'STRING_VALUE',
Qualifier: 'STRING_VALUE'
};
lambda.invoke(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
One of the params you send a Payload which is the event the invoked function receives so you can send your MyErrorMessage in/as this payload to get the desired result.

How to pass nested json object through soapui in request api

I am new to REST and i am making test cases for server client application. where server is made in php using symphony framework and clients are available both in android and ios. Server implements Rest instead of Soap. Now i have to make a test case in soapui to add a group in which nested JSON parameter is going but i am failed to send parameter as nested. I can show you the request from the log on server side.
[2014-09-24 11:14:36] las_api.INFO:
{
"request": {
"url": "http:\/\/192.168.1.134\/las_web\/web\/app_dev.php\/api\/users\/2\/groups.json",
"attributes": {
"_controller": "Las\\ApiBundle\\Controller\\User\\GroupController::cpostAction",
"_format": "json",
"userId": "2",
"_route": "api_post_user_groups",
"_route_params": {
"_format": "json",
"userId": "2"
}
},
"post": {
"group": {
"name": "Seniors"
},
"token": "{81A9B0D9-CA0E-E70F-ADFF-116EE7A1A980}"
},
"get": [
]
},
"ip": "192.168.1.134"
}
Below is the parameter request parameter. where in group there are two parameters first is name of the group and second is the token generated by server on login time.
{
"group": {
"name": "Seniors"
},
"token": "{81A9B0D9-CA0E-E70F-ADFF-116EE7A1A980}"
}
I have successfully made test case for login. I can show you the request from server log while logging.
[2014-09-24 10:11:37] las_api.INFO:
{
"request": {
"url": "http:\/\/192.168.1.134\/las_web\/web\/app_dev.php\/api\/tokens",
"attributes": {
"_controller": "Las\\ApiBundle\\Controller\\TokenController::cpostAction",
"_format": "json",
"_route": "api_post_tokens",
"_route_params": {
"_format": "json"
}
},
"post": {
"password": "abc123",
"username": "abc#gmail.com"
},
"get": [
]
},
"ip": "192.168.1.134"
}