Retrieving forecast data from OpenWeatherMap in FIWARE ORION - fiware

I am trying to get weather forecasts data from OpenWeatherMap and integrate them in Orion by performing a registeration request.
I was able to register and get the API key from OpenWeatherMap, however, the latter returns a JSON file with all the data inside, which is not supported by ORION.
I have followed the step by step tutorial https://fiware-tutorials.readthedocs.io/en/latest/context-providers/index.html#context-provider-ngsi-proxy where they have acquired the data from OpenWeatherMap using NGSI proxy, an API key is required to be indicated in the docker-compose file as an environment variable, however, the data acquired is the "current data" and not forecast and also specific to Berlin.
I have tried to access the files inside the container "fiware/tutorials.context-provider" and try to modify and match the parameters to my needs but I feel like I am taking a long blocked path.
I don't think that's even considered as good practice but I have run out of ideas :(
Can anyone suggest how I could bring the forecast data to Orion and register it as a context provider?
Thank you in advance.

I imagine you aim to implement a context provider, able to speak NGSI with Orion.
OpenWeatherMap surely doesn't implement NGSI ...
If you have the data from OpenWeatherMap, as a JSON string, perhaps you should parse the JSON and create your entities using some select key-values from the parsed OpenWeatherMap? Save the entity (entities) locally and then register those keys in Orion.
Alternatively (easier but I wouldn't recommend it), create local entities with the entire OpenWeatherMap data as the value of an attribute of the entity:
{
"id": "id-from-OpenWeatherMap",
"type": "OpenWeatherMap",
"weatherData": {
"value":
...
}
...
}
Then you register id/weatherData in Orion.

Related

Nifi RestLookupService in LookupRecord gives JsonParseException

I have a basic NIFI flow with one GenerateFlowFile processor and one LookupRecord processor with RestLookupService.
I need to do a rest lookup and enrich the incoming flow file with rest response.
I am getting errors that the lookup service is unable to lookup coordinates with the value I am extracting from the incoming flow file.
GenerateFlowFile is configured with simple JSON
LookupRecord is configured to extract the key from the JSON and populate it to the RestLookupService. Also, JsonReader and JsonSetWriter is configured to read the incoming flow file and to write the response back to the flow file
The RestLookupService itself exits with JsonParseException about unexpected character '<'
RestLookupService is configured with my API running in the cloud in which I am trying to use the extracted KEY from the incoming flow file.
The most interesting bug is that when I configure the URL to point for example to mocky.io everything works correctly so it means that the issue itself is tight with the API URL I am using (http://iotosk.ddns.net:3006/devices/${key}/getParsingData). I have tried also removing the $key, using the exact URL, using different URLs..
Of course the API is working OK over postman/curl anything else. I have checked the logs on the container that the API is running on and there is no requests in the logs what means that nifi is failing even before reaching the API. At least on application level...
I am absolutely out of options without any clue how to solve this. And with nifi also google is not helpful.
Does anybody see any issue in the configuration or can point me in some direction what can cause this issue?
After some additional digging. The issue was connected with authentication logic even
before the API receives it and that criples the request and and returned XML as Bryan Bende suggested in the comment.
But definitely better logging in nifi will help to solve this way faster...

how to get a Pre-Authenticated request for an object in OCI object storage using python SDK?

I want to create pre-authenticated request for an object inside a bucket in the OCI object storage using python SDK. I found out that I can use get_preauthenticated_request for the bucket to put objects inside the bucket but not to get the objects pre-authenticated. I can create a pre-authenticated request using the OCI console but I need to do it in a python script. can anybody help me in this issue?
You can use create_preauthenticated_request (see code) for both buckets and individual objects.
The difference is in the access type:
ANY_OBJECT_WRITE is for the whole bucket
OBJECT_READ, OBJECT_READ_WRITE and OBJECT_WRITE are for objects
So you should be able to create a Pre-Authenticated Request with something like
request_details = create_preauthenticated_request_details()
request_details.access_type("ObjectReadWrite")
par = create_preauthenticated_request("namespace", "bucket", request_details)
You can find more on the request details here and for the request itself here.
Let me know if this works for you, I don't have an account to test against at the moment.

Receive alert when a JSON Data Object is added

Can anyone think of a way to get alerted when a new JSON Object is added to a publically accessible JSON URL?
If it were you, how would you go about it?
Thanks
There are 2 principle models allowing to get notified on changed: push and pull model.
for the push model, a resource owning the object (in this case a site hosting the URL) supposed to provide a subscription mechanism.
if push model is unavailable you then default to pull model: this requires polling the resource on your end - you need to come up the scripted way of doing it on your local side.
Then the basic pseudo would be like:
poll JSON object (via REST)
compare pulled JSON with preserved one
if JSONs are the same: sleep for some time and repeat
else: save pulled JSON, send notification and repeat

Call rest service that returns JSON in mule esb?

How to call a rest service that returns JSON in a APIKit based message flow. I want to prepare the request for Rest service and want to extract the JSON message in a message flow.
Can any one help me to do this?
Thanks
The recommended way to invoke/consume REST services in mule is the use of HTTP Request
Do let us know if there is a specific you are facing
The request connector is especially handy when consuming a RESTful API
that is described in a RAML file. If you reference the API's RAML file
in the connector's configuration, it will proactively offer you the
set of available resources and operations contained in the RAML file,
as well as enforce the policies described in the file. It will also
expose the API metadata to Studio, which can then be used by other
elements such as DataWeave to autocomplete fields and make
configuration much easier
Use Rest URI path to invoke the Service and you can also get json response as output based on the type of service you invoke

Implementing JSON REST Service to elasticsearch using Spring/Camel

I am a complete Camel FNG migrating from a JMS service to a RESTful service; part of that is integrating with an elasticsearch server. I am using Fuse with the Spring DSL.
The main idea is to poll for data from a whole bunch of servers running a REST service, marshal them to XML (if necessary, see below), strip the top level key (foo:{...}) and push each of the keys to an elasticsearch database. a reply from the service would be of the form:
{"foo": {"bar":{"type":"tweet","info":"blahblah"},"baz":{"type":"tweet","info":"blahblah"}}}
My questions are as follows:
1-How do I get data from the webservice? I dug around looking for a concise example, to no avail. If I understand correctly, it involves the jetty component in camel. say I want to pull json from http://myrestservice:7700.
2- Does the json need to be marshaled to XML to remove the top-level key?
Thanks in advance!
You can use camel-cxf [2] producer to call those running REST services and pass the result which will be your json to a processor and then use JSonPath [2] to traverse over your json and customize it the way you want it.
[1] http://camel.apache.org/cxfrs.html (see "Consuming a REST Request - Default Binding Style" section)
[2] http://camel.apache.org/jsonpath.html