I would like to obtain the latest data by specifying Lambda's IoT Analytics dataset.
If you use getDatasetContent of IoTAnalytics of aws sdk, only the link for downloading the file will be returned.
Data itself can not be acquired.
I would like to know how to obtain information on the IoT Analytics data set from Lambda.
Hi and welcome to Stack Overflow!
If I understand your question correctly, you are asking how to get the data from an IoT Analytics Dataset using a Lambda function?
You are correct that get_dataset_content only returns the URI, but it is simple to then fetch the actual content, for example in Python it would look like this;
# Code Fragment to retrieve content from IoT Analytics Dataset
iota = boto3.client('iotanalytics')
response = iota.get_dataset_content(datasetName='my_data_set',versionId='$LATEST')
contentState = response['status']['state']
if (contentState == 'SUCCEEDED') :
url = response['entries'][0]['dataURI']
stream = urllib.request.urlopen(url)
reader = csv.DictReader(codecs.iterdecode(stream, 'utf-8'))
for record in reader:
# Process the record as desired, you can refer to columns in the CSV
# by using record['column_name'] using the DictReader iterator
Note that this code is specifically looking at the most recent results using the $LATEST version - you can also look for the $LATEST_SUCCEEDED version.
There's more documentation here for Boto - the AWS Python SDK, but you can use the same approach in all other sdk supported languages.
Hope that helps,
Roger
Related
It is possible to change value of the parameters in Power Bi Desktop from dashboard (main-page) level ?
I'm using parameters in two web requests as json body (DateFrom DateTo).
I know that there is "Transform data" option and I was trying :
https://learn.microsoft.com/en-gb/power-bi/connect-data/desktop-dynamic-m-query-parameters
but I cannot bind table to parameter - there is no "Bind to parameter" field.
Bind to parameter
My Api Request
You can only use query parameters in Power Query (M language). Here is an example that works for me. Assuming the output is JSON, you would do it like this (you do it in the advanced editor):
let
src = Web.Contents(baseUrl, [RelativePath = urlPath, Query=[#"api_token" = apiKey, limit=Text.From(limit), start=Text.From(offset)]])
json = Json.Document(src)
in
json
In the above examples, parameters are:
baseUrl for root API path
urlPath for API endpoint
apiKey for API authorization token
limi and offset for getting a subset of data
That's just an example, but the api_token parameter is crucial, if you want the query to refresh in the Power BI Service (assuming you're using token-based authentication).
EDIT:
So, I think I finally understood where your problem is. I don't think it can be done with REST API, as the requirement for the Bind parameter function to work is for the query to be in Direct Query mode as opposed to Import. And I don't think you can set it for this.
I want to create pre-authenticated request for an object inside a bucket in the OCI object storage using python SDK. I found out that I can use get_preauthenticated_request for the bucket to put objects inside the bucket but not to get the objects pre-authenticated. I can create a pre-authenticated request using the OCI console but I need to do it in a python script. can anybody help me in this issue?
You can use create_preauthenticated_request (see code) for both buckets and individual objects.
The difference is in the access type:
ANY_OBJECT_WRITE is for the whole bucket
OBJECT_READ, OBJECT_READ_WRITE and OBJECT_WRITE are for objects
So you should be able to create a Pre-Authenticated Request with something like
request_details = create_preauthenticated_request_details()
request_details.access_type("ObjectReadWrite")
par = create_preauthenticated_request("namespace", "bucket", request_details)
You can find more on the request details here and for the request itself here.
Let me know if this works for you, I don't have an account to test against at the moment.
I'm working on a simple ruby script with cli that will allow me to browse certain statistics inside the terminal.
I'm using API from the following website: https://worldcup.sfg.io/matches
require 'httparty'
url = "https://worldcup.sfg.io/matches"
response = HTTParty.get(url)
I have to goals in mind. First is to somehow save the JSON response (I'm not using a database) so I can avoid unnecessary requests. Second is to check if the new data is available, and if it is, to override the previously saved response.
What's the best way to go about this?
... with cli ...
So caching in memory is likely not available to you. In this case you can save the response to a file on disk.
Second is to check if the new data is available, and if it is, to override the previously saved response.
The thing is, how can you check if new data is available without doing a request for the data? Not possible (given the information you provided). So you can simply keep fetching data every 5 minutes or so and updating your local file.
I'm trying to use the New York City Bus API in order to create an app. The last API I worked with, was while learning Python. I have no idea where to begin.
This is the link to the documentation
http://bustime.mta.info/wiki/Developers/SIRIIntro
If you guys could point me towards how I can get the data from the API. I already got the Key.
Thank You so Much
You can use requests to fetch data from api.
import requests
url = "http://api.prod.obanyc.com/api/siri/vehicle-monitoring.json?key={}".format(key)
# fetch data
data = requests.get(url).json()
# do something with data.
Is there a way to find it programatically? I need this as part of an automated run; So this would be very helpful if there is an existing remote API call which can give this.
You don't need to parse the HTML - most of the Hudson pages can be turned into API calls by adding URL suffix, e.g. make GET calls to:
http://hudson:8080/computer/api/json
Switch the JSON for either XML or Python if you prefer these over JSON.
If you use just the API suffix, you'll get a short generic help page on the API.
Groovy script to get all computers:
def jenkins = Jenkins.instance
def computers = jenkins.computers
computers.each{
println "${it.displayName} ${it.hostName}"
}
Look at http://hudson:8080/computer/