I currently got the struggle that I don't know how to get the data from coinbase into Amazon Kinesis (Firehose) to analyse it their.
Through the coinbase API I set up a websocket that gives me a json file of the data. How do I pass this json into Kinesis? I know you have to use the putRecords() method but where and how do I implement this?
Am Thankful for every help, my thesis depends on it...
Related
I want to create pre-authenticated request for an object inside a bucket in the OCI object storage using python SDK. I found out that I can use get_preauthenticated_request for the bucket to put objects inside the bucket but not to get the objects pre-authenticated. I can create a pre-authenticated request using the OCI console but I need to do it in a python script. can anybody help me in this issue?
You can use create_preauthenticated_request (see code) for both buckets and individual objects.
The difference is in the access type:
ANY_OBJECT_WRITE is for the whole bucket
OBJECT_READ, OBJECT_READ_WRITE and OBJECT_WRITE are for objects
So you should be able to create a Pre-Authenticated Request with something like
request_details = create_preauthenticated_request_details()
request_details.access_type("ObjectReadWrite")
par = create_preauthenticated_request("namespace", "bucket", request_details)
You can find more on the request details here and for the request itself here.
Let me know if this works for you, I don't have an account to test against at the moment.
I am trying to get weather forecasts data from OpenWeatherMap and integrate them in Orion by performing a registeration request.
I was able to register and get the API key from OpenWeatherMap, however, the latter returns a JSON file with all the data inside, which is not supported by ORION.
I have followed the step by step tutorial https://fiware-tutorials.readthedocs.io/en/latest/context-providers/index.html#context-provider-ngsi-proxy where they have acquired the data from OpenWeatherMap using NGSI proxy, an API key is required to be indicated in the docker-compose file as an environment variable, however, the data acquired is the "current data" and not forecast and also specific to Berlin.
I have tried to access the files inside the container "fiware/tutorials.context-provider" and try to modify and match the parameters to my needs but I feel like I am taking a long blocked path.
I don't think that's even considered as good practice but I have run out of ideas :(
Can anyone suggest how I could bring the forecast data to Orion and register it as a context provider?
Thank you in advance.
I imagine you aim to implement a context provider, able to speak NGSI with Orion.
OpenWeatherMap surely doesn't implement NGSI ...
If you have the data from OpenWeatherMap, as a JSON string, perhaps you should parse the JSON and create your entities using some select key-values from the parsed OpenWeatherMap? Save the entity (entities) locally and then register those keys in Orion.
Alternatively (easier but I wouldn't recommend it), create local entities with the entire OpenWeatherMap data as the value of an attribute of the entity:
{
"id": "id-from-OpenWeatherMap",
"type": "OpenWeatherMap",
"weatherData": {
"value":
...
}
...
}
Then you register id/weatherData in Orion.
I've started using Apache Nifi and I'm still learning it and experimenting with it. I really want to use Nifi to get JSON documents from API's and put them in my Elasticsearch database. So far using the built-in getTwitter and putElasticsearch controllers this works.
However now I want to do this with other APIs than Twitter, and I'm kinda stuck here. First off I really don't even know which controller to use? I would think getHttp or invokeHttp even with 'GET' as http verb then but it doesn't seem to work. If I use the getHttp I have to give an SSL service with keystore and truststore .. like why would I have to do that?
Apache Nifi is still quite new so hard to find decent guides / information about these kinds of things. I have read and searched the documentation but haven't gotten the wiser.
An example JSON to pick up from an API is:
https://api.ssllabs.com/api/v2/getEndpointData?host=www.bnpparibasfortis.be&s=193.58.4.82
Thanks in advance for anyone that can offer some help / insight.
What processor you use to get the JSON data is entirely dependent on the API you want to hit. The GetHttp or InvokeHttp processors should work to grab the data from a URL. If you'll notice, the SSL service is an optional property for both GetHttp and InvokeHttp so you only need to you use it when you want to communicate via HTTPS. Also, from the UI you can right click on a processor and then click "usage" to bring up the documentation for that processor.
At this link[1] you can find a NiFi template that uses GetHttp to get JSON data from randomuser.me and does various processing on it. It's primarily a template to show-case the different Avro processors but the method of grabbing the JSON should be relevant.
[1] https://github.com/hortonworks-gallery/nifi-templates/blob/master/templates/Convert_To_Avro_From_CSV_and_JSON.xml
Can anyone tell me how to send JSON request in Perl and how can I access JSON request object in catalyst?
To start the process I'd look at this thread: Perl JSON::RPC::Client using LWP::Agent
This gives two methods json rpc, I've used this in the past and a tailor-made method using LWP, I haven't tried anything like that.
Tell us how you do, best regards Hugh Barnard
I'm learning how to use a third party API called Wunderground and I don't know how to request, receive, and use their results which is in a Json format.
If you see their website a developer can sign up for free and receives an API KEY. You can then request weather data in the following URL format.. http://api.wunderground.com/api/KEY/FEATURE/[FEATURE…]/[SETTING…]/q/QUERY.FORMAT
So I have tried it in my web browser by typing some parameters and I received a very long Json file with the correct information (I checked). Problem is I don't have the slightest idea of how to create a variable which can make this request, and even if I was able to do that I don't know where should I receive the file and how to get only the results I want (in this case current weather).
You have to use Titanium.Network.HTTPClient to make request.
For code examples related to Json parsing you can use:
Appcelerator: Using JSON to Build a Twitter Client
HTTPClient()