I am trying to adapt an event calendar vuejs module called dayspan calendar. Current entry object for an event as json is a bit strange and I want to balance the parsing of the payload before the post request and handling of data in DRF serializers. So I can get an optimum and performant client-server rest API communication. Json output is as below before any parsing:
{
"data": {
"title": "YOK",
"description": "",
"location": "",
"color": "#1976d2",
"forecolor": "#ffffff",
"calendar": "",
"busy": true,
"icon": ""
},
"schedule": {
"times": [
"17"
],
"dayOfMonth": [
11
],
"year": [
2021
],
"month": [
5
]
}
}
There are more schedule fields like "dayOFWeek", "duration" etc. that is to be expected for different entries.
What would be the best approach in terms of parsing json payload before posting to DRF and in deserializing stage before saving into database? I appreciate any ideas.
Related
I am looking for a way to extract data out of an azure environment. Problem i'm currently having is that when I use my API call I receive about 60 lines of json while I only need 4 of those lines. To reduce load, increase efficiency and remove the need for parsing withing the other environment where I need to data, I want to find a way to filter the data in the api call. Currently my call looks like this.
https://management.azure.com/subscriptions/{subscription}/resourceGroups/{resourcegroup}/providers/Microsoft.Web/sites/{application or resource}/providers/microsoft.insights/metrics?api-version=2021-05-01&metricnames=IoWriteBytesPerSecond,IoReadBytesPerSecond&timeSpan=PT1M
now the ouput looks something like this.
{
"cost": 0,
"timespan": "2022-10-11T10:18:00Z/2022-10-11T10:19:00Z",
"interval": "PT1M",
"value": [
{
"id": "/subscriptions//resourceGroups//providers/Microsoft.Web/sites//providers/Microsoft.Insights/metrics/IoWriteBytesPerSecond",
"type": "Microsoft.Insights/metrics",
"name": {
"value": "IoWriteBytesPerSecond",
"localizedValue": "IO Write Bytes Per Second"
},
"displayDescription": "The rate at which the app process is writing bytes to I/O operations. For WebApps and FunctionApps.",
"unit": "BytesPerSecond",
"timeseries": [
{
"metadatavalues": [],
"data": [
{
"timeStamp": "2022-10-11T10:18:00Z",
"total": 288.0
}
]
}
],
"errorCode": "Success"
},
{
"id": "/subscriptions//resourceGroups//providers/Microsoft.Web/sites//providers/Microsoft.Insights/metrics/IoReadBytesPerSecond",
"type": "Microsoft.Insights/metrics",
"name": {
"value": "IoReadBytesPerSecond",
"localizedValue": "IO Read Bytes Per Second"
},
"displayDescription": "The rate at which the app process is reading bytes from I/O operations. For WebApps and FunctionApps.",
"unit": "BytesPerSecond",
"timeseries": [
{
"metadatavalues": [],
"data": [
{
"timeStamp": "2022-10-11T10:18:00Z",
"total": 284.0
}
]
}
],
"errorCode": "Success"
}
],
"namespace": "Microsoft.Web/sites",
"resourceregion": "westeurope"
}
Out of all these lines I only need about 4 objects, Is it possible to use the $filter function within the URL api call? If yes, can someone redirect me to a forum, doc or example where this is used?
Thanks, regards
I'm having a hard time understanding how context providers work in the Orion Context Broker.
I followed the examples in the step-by-step guide written by Jason Fox. However, I still do not exactly get what happens in the background and how the context broker exactly creates the POST from the registration. Here is what I am trying to do:
I do have a WeatherStation that provides sensor data for a neighborhood.
{
"id": "urn:ngsi-ld:WeatherStation:001",
"type": "Device:WeatherStation",
"temperature": {
"type": "Number",
"value": 20.5,
"metadata": {}
},
"windspeed": {
"type": "Number",
"value": 60.0,
"metadata": {}
}
}
Now I like the WeatherStation to be a context provider for all buildings.
{
"id": "urn:ngsi-ld:building:001",
"type": "Building"
}
Here is the registration that I try to use.
{
"id": null,
"description": "Random Weather Conditions",
"provider": {
"http": {
"url": "http://localhost:1026/v2"
},
"supportedForwardingMode": "all"
},
"dataProvided": {
"entities": [
{
"id": "null",
"idPattern": ".*",
"type": "Building",
"typePattern": null
}
],
"attrs": [
"temperature",
"windspeed"
],
"expression": null
},
"status": "active",
"expires": null,
"forwardingInformation": null
}
The context broker accepts both entities and the registration without any error.
Since I have a multi-tenant setup I use one fiware_service for the complete neighborhood but every building would later have a seperate fiware_servicepath. Hence, the weatherstation has a different servicepath than the building. Although I also tried to put them both on the same path.
For now I used the same headers for all entities.
{
"fiware-service": "filip",
"fiware-servicepath": "/testing"
}
Here is the log of the context broker (version: 3.1.0):
INFO#2021-09-23T19:17:17.944Z logTracing.cpp[212]: Request forwarded (regId: 614cd2b511c25270060d873a): POST http://localhost:1026/v2/op/query, request payload (87 bytes): {"entities":[{"idPattern":".*","type":"Building"}],"attrs":["temperature","windspeed"]}, response payload (2 bytes): [], response code: 200
INFO#2021-09-23T19:17:17.944Z logTracing.cpp[130]: Request received: POST /v2/op/query?options=normalized%2Ccount&limit=1000, request payload (55 bytes): {"entities": [{"idPattern": ".*", "type": "Building"}]}, response code: 200
The log says that it receives the request and forwards it as expected. However, as I understand it this would simply point to the same building entity again. Hence, it is somehow a circular forwarding. I also cannot tell anything about the headers of the request.
I do not understand how the forwarded request from the building can actually query the weather station for information. When I query my building I still only receive the entity with no own properties:
{
"id": "urn:ngsi-ld:building:001",
"type": "Building"
}
I also tried to vary the url of the registration but with no success.
Is this scenario actually possible with the current implementation? It would be very useful
Is there any example for this including also the headers?
I know that I could simply use reference but that would put more work on the user.
Thanks for any help on this.
It is messy, but you could achieve this via a subscription. Hold the weather station as a separate entity in the context broker and poll or push updates into the entity. The subscription would fire whenever the data changes and make two NGSI requests:
Find all entities which have a Relationship servicedBy=WeatherStationX
Run an upsert on all entities to add a Property to each entity:
{
"temperature" : {
"type" : "Property",
"value" : 7,
"unitCode": "CEL",
"observedAt": "XXXXX",
"providedBy": "WeatherStation1"
}
}
Where observedAt comes either from the payload of the weather station or the notification timestamp.
Within the existing IoT Agents, provisioning the link attribute allows a device to propagate measures to a second entity (e.g. this Thermometer entity is measuring temperature for an associated Building entity)
{
"entity_type": "Device",
"resource": "/iot/d",
"protocol": "PDI-IoTA-UltraLight",
..etc
"attributes": [
{"object_id": "l", "name": "temperature", "type":"Float",
"metadata":{
"unitCode":{"type": "Text", "value" :"CEL"}
}
}
],
"static_attributes": [
{
"name": "controlledAsset",
"type": "Relationship",
"value": "urn:ngsi-ld:Building:001",
"link": {
"attributes": ["temperature"],
"name": "providedBy",
"type": "Building"
}
}
]
}
At the moment the logic just links direct one-to-one, but it would be possible to raise a PR to check for an Array and update multiple entities in an upsert - the relevant section of code is here
I have build a small microservice using SpringBoot2 and Spring 5 which has a REST service exposed (HTTP GET Method) and which internally consumes another REST GET service (third party API). Using Postman when I call my service (GET), then I get a JSON response but the problem is I get a complete whole object in response like below :-
[
{
"id": "1",
"name": "Open Catalogue",
"subcategories": [
{
"id": "106",
**"name": "Components",**
"subcategories": [
{
"id": "816",
"name": "Power Supplies",
"subcategories": [
{
"id": "814",
"name": "Rechargeable Batteries",
"subcategories": [],
"sample": {
"empty": true,
"lazy": false,
"async": false
}
},
{
"id": "829",
"name": "Battery Chargers",
"subcategories": [],
"samples": {
"empty": true,
"lazy": false,
"async": false
}
},
My service URL used in post man is like this :-
http://localhost:8080/test-search?searchKey=ball
So my requirement is whenever a user consume this service by a sub-category name then only that sub-category details along with its immediate child details should be returned and not the child of child.
Here searchKey in URL is nothing but a free text to search for a sub-category. For instance when I say :-
http://localhost:8080/test-search?searchKey=Components
then only below details should be returned like this :-
"id": "106",
"name": "Components",
"subcategories": [
{
"id": "816",
"name": "Power Supplies",
Response should not have sub-categories of Power Supplies i.e. "subcategories": [ "id": "814",
"name": "Rechargeable Batteries",
Is there any efficient way to do the filtering while preparing the JSON response or first fetch whole object and then start filtering?
Please advise, thank you
I'm aware of this question but I don't believe that there is no solution with standars component. I'm using Talend ESB Studio 5.4.
I have to parse a JSON string from a REST web service into multiple output, and add them to a database.
Database has two tables:
User (user_id, name, card, card_id, points)
Action (user_id, action_id, description, used_point)
My JSON Structure is something like that:
{
"users": [
{
"name": "foo",
"user_id": 1,
"card": {
"card_id": "AAA",
"points": 10
},
"actions": [
{
"action_id": 1,
"description": "buy",
"used_points": 2
},
{
"action_id": 3,
"description": "buy",
"used_points": 1
}
]
},
{
"name": "bar",
"user_id": 2,
"card": {
"card_id": "BBB",
"points": -1
},
"actions": [
{
"id": 2,
"description": "sell",
"used_point": 5
}
]
}
]
}
I have tried to add a JSON Schema Metadata but it is not clear to me how to "flat" the JSON. I have tried to look at tXMLMap, tExtractJSONFields.. but no luck till now.
I also had a look at tJavaRow but I don't understand how to make a Schema for that.
It's a pity because till now I'm loving Talend! Any advice?
You can save a json file in your disk, then create new json file in the metadata of Talend studio, the wizard retrieve the schema for you, after saving, you ca, copie schema in the generic schema of the metadata, and it's done, use that generic schema where you want, this is how to use generic schema in the tRestClient component:
I am new to Liftweb. I want to Store a Json File in Mysql database using Lift Mapper
My Json File Like Below:-
[
{
"name": "Root Category",
"Id": "1",
"dispName": "",
"childs": [
{
"name": "Sub Category",
"Id": "",
"dispName": "",
"childs": [
{
"name": "Spec1",
"Id": "",
"dispName": "",
"childs": []
}
]
}
]
},
{
"name": "Root Category",
"Id": "",
"dispName": "",
"childs": [
{
"name": "Sub Category",
"Id": "",
"dispName": "",
"childs": [
{
"name": "Spec1",
"Id": "",
"dispName": "",
"childs": []
}
]
}
]
}
]
Is it Possible to store a Json File in Lift Mapper .Please give me Suggestions. It will be great if some one provide any sample
Best Regards
GSY
At the moment there is no good support for storing JSON in MySQL. I mean it's not going to provide capabilities MongoDB provides for example. However there are some JSON processing functions provided by community if you want. Given all that you can store it in VARCHAR. TEXT or BLOB field type as simple text. Here is a Mapper example:
import net.liftweb.mapper._
import net.liftweb.common._
class SomeDbClass extends LongKeyedMapper[SomeDbClass] with IdPK {
def getSingleton = SomeDbClass
// set limit of chars - can be used in `validate()`
object quota_type extends MappedString(this, 1024)
}
object SomeDbClass extends SomeDbClass with LongKeyedMetaMapper[SomeDbClass]
For one of my projects I store JSON as a string in Postgres similarly because I just need to read and write it without having to parse it in DB and query by fields. Whenever I need efficient JSON storage with query and update support I use MongoDB with Record + ( Casbah or Rogue ).