How can I get boto to generate signed URLs for S3 that trigger downloads? - boto

I've attempted this:
connection = S3Connection(
aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY)
url = connection.generate_url(
60,
'GET',
settings.AWS_STORAGE_BUCKET_NAME,
self.get_object().image,
headers={'Content-Type': 'application/octet-stream'})
but when I try to visit the signed URL I get this message:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
If I remove the headers attribute on the other hand, I can access the image just fine. What am I doing wrong?

The issue is that you can't set arbitrary headers using generate_url, and while the response_headers option exists AWS also has limited options as to what you can set. This will do what's expected in this case.
url = connection.generate_url(
60,
'GET',
settings.AWS_STORAGE_BUCKET_NAME,
self.get_object().image,
response_headers={
'response-content-type': 'application/octet-stream'
})

Related

How to pull data from Toggl API with Power Query?

First timer when it comes to connecting to API. I'm trying to pull data from Toggl using my API token but I can't get credentials working. I tried to replicate the method by Chris Webb (https://blog.crossjoin.co.uk/2014/03/26/working-with-web-services-in-power-query/) but I can't get it working. Here's my M code:
let
Source = Web.Contents(
"https://toggl.com/reports/api/v2/details?workspace_id=xxxxx&client=xxxxxx6&billable=yes&user_agent=xxxxxxx",
[
Query=[ #"filter"="", #"orderBy"=""],
ApiKeyName="api-token"
])
in
Source
After that I'm inputting my API Token into Web API method in Access Web content windows but I get an error that credentials could not be authenticated. Here's Toggl API specification:
https://github.com/toggl/toggl_api_docs/blob/master/reports.md
Web.Contents function receives two parameters: url + options
Inside options, you define the headers and the api_key, and other queryable properties, such as:
let
baseUrl = "https://toggl.com/",
// the token part can vary depending on the requisites of the API
accessToken = "Bearer" & "insert api token here"
options = [
Headers = [Authorization = accessToken, #"Content-Type" =
"application/Json"], RelativePath ="reports/api/v2/details", Query =
[workspace_id=xxxxx, client=xxxxxx6 , billable=yes, user_agent=xxxxxxx]
]
Source = Web.Contents(baseUrl, options)
// since Web.Contents() doesn't parse the binaries it fetches, you must use another
// function to see if the data was retreived, based on the datatype of the data
parsedData = Json.Document(Source)
in
parsedData
The baseUrl is the smallest url that works and never changes;
The RelativePath is the next part of the url before the first "?".
The Query record is where you define all the attributes to query as a record.
This is usually the format, but check the documentation of the API you're querying to see if it is similar.

NativeScript Throwing Error Response with status: 200 for URL: null

I am using Angular4 with TypeScript version 2.2.2
My web app is running fine when I call JSON with Filters but my NativeScript app fails when I call the Filter Values as an Object but works fine when I call filter values as a string.
Error Response with status: 200 for URL: null
THIS WORKS
https://domainname.com/api/v1/searchevents?token=057001a78b8a7e5f38aaf8a682c05c414de4eb20&filter=text&search=upcoming
If the filter value and search value is STRING it works whereas if they are objects as below, it does not work
THIS DOES NOT WORK
https://api.domainname.com/api/v1/searchevents?token=057001a78b8a7e5f38aaf8a682c05c414de4eb20&filter={"limit":"12","skip":"0"}&search={"search":"","latitude":"","longitude":"","categories":"","address":"","type":"upcoming"}
The Code I used is below
getData(serverUrl, type, skip_limit) {
console.log(serverUrl);
let headers = this.createRequestHeader();
let token_value = localStorage.getItem('access_token')
let url;
var filter;
filter = '{"limit":"10","skip":"0"}'
url = this.apiUrl + serverUrl + '?token=' + token_value + '&filter=' + filter
return this.http.get(url, { headers: headers })
.map(res => res.json());
}
The URL as formed above for the API is fine and works fine. Yet the error comes Error Response with status: 200 for URL: null
CAN ANYONE HELP ME SOLVE THIS?
Looks like the issue is the "filter" values are of different type and from what you mentioned as what worked, your service is expecting a string and not an object/array. So it fails to send the proper response when it gets one. With an object in the URL, you may have to rewrite the service to read it as an object (parse the two attributes and get them individually)
To make it simple, you can make these two as two different variables in the URL. like below,
https://api.domainName.in/api/v1/oauth/token?limit=10&skip=0
Be more precise in whats happening in your question,
1) Log the exact URL and post it in the question. No one can guess what goes in "text" in your first URL.
2) Your URL which you mentioned as worked have "token" as part of path, but in the code, its a variable which will have a dynamic value from "token_value".
3) Post your service code. Especially the signature and input parsing part.
Got the solution:
All you have to do is encode the Filter and Search Parameters if it is an Object or Array using Typescript encodeURI()
var filter = '{"limit":"12","skip":"0"}'
var search = '{"search":"","latitude":"","longitude":"","categories":"","address":"","type":"upcoming"}'
var encoded_filter = encodeURI(filter);
var encoded_search = encodeURI(search);
url = this.apiUrl+serverUrl+'?token='+token_value+'&filter='+encoded_filter+'&search='+encoded_search

Unexpected Error on UrlFetchApp

I'm trying to call my ServiceNow JSON web service. I'm getting an unexpected error when I execute URLFetchApp. I'm guessing I'm passing in the authorization headers in the wrong way but both the GAS and ServiceNow documentation is beyond terrible. I've seen some of the other SO questions similar to this but none have worked. Any help would be appreciated.
function getOpenTickets(){
var headers = {
"Authorization":"Basic RgRJ5U6EsxHt00229KX5Hj0WV1z18q08==",
"Content-Type":"application/json",
"Username":"myusername",
"Password":"mypassword"
}
var url = "https://mysninstance.service-now.com/u_equipment_repair.do?JSONv2=&sysparm_view=vendor&displayvalue=true&sysparm_action=getRecords&sysparm_query=state=500^assignment_group.name=MyGroup^ORDERBYDESCnumber";
var url = encodeURIComponent(url);
var options = {
"method":"get",
"headers":headers
}
var result = UrlFetchApp.fetch(url,options);
Logger.log(result.getContentText());
}
OK so I found the solution. There were actually two problems.
The first was with the way I was passing the authorization headers. I was passing the basic authentication as an already encoded base64 string, on top of which I was still passing the username and password which was redundant. For whatever reason Google Apps Script (GAS) doesn't like this. Once I changed the headers and the options as shown below it was fine.
The second problem was the the URI encoding. The query string did need to be encoded because of the caret "^" symbols, but for whatever reason GAS's encodeURIComponent was not encoding it properly. As soon as I manually replaced the caret symbols with their URL encoded equivalents , which is "%5E", everything worked fine and I was able to retrieve my ServiceNow data via Google Apps Script.
function getOpenTickets3(){
var headers =
{
Authorization : "Basic " + Utilities.base64Encode('myusername:mypassword'),
"Content-Type":"application/json"
}
var options =
{
"method" : "get",
"headers": headers
};
var url = "https://mysninstance.service-now.com/u_equipment_repair.do?JSONv2=&sysparm_view=vendor&displayvalue=true&sysparm_action=getRecords&sysparm_query=state=500%5Eassignment_group.name=Somevendor%5EORDERBYDESCnumber";
var result = UrlFetchApp.fetch(url,options);
Logger.log(result.getContentText());
}
You are URI encoding your entire URL in this line:
var url = encodeURIComponent(url);
In your URL, the base path needs to be unescaped when passed to fetch(...):
https://mysninstance.service-now.com/u_equipment_repair.do
Each parameter following the ? is a URI component, like:
sysparm_view=vendor
In this case, the parameter name is sysparm_view and the value is vendor, you would need to URI encode the value (vendor) if it contained special characters like one of /?&.
In the static URL you provide, there's actually nothing that needs to be encoded, so removing that call to encodeURIComponent(url) should work.
If you are dealing with dynamic values for your URL parameters, then you'd want to URI encode each parameter value separately, before concatenating onto the main string.

Playframework handling post request

In my routes:
POST /forms/FormValidator1/validateForm controllers.FormValidator1.validateForm(jsonForm:String)
There is a controller method defined for that route:
def validateForm(jsonForm:String) = Action { ...
Then I try to send POST request by chrome POSTMAN plugin (see pic above).
I use:
url: http://localhost:9000/forms/FormValidator1/validateForm
headers: Content Type: application/json
json data: {name: "me", surname: "my"}
So, sending this POST request I can not reach controller's method by mentioned route / url. Why?
UPDATE:
Interestly enough: after I got it working on my laptop (see my answer below) then push it on gitHub and pull it to another machine it starts working differently. Now it complains than Bad Request is [Invalid XML] nevertheless I use "application/json" header and did not change any line of code after commit. I wonder maybe it is a bug.
It seems I got it.
Here:
https://groups.google.com/forum/#!topic/play-framework/XH3ulCys_co
And here:
https://groups.google.com/forum/#!msg/play-framework/M97vBcvvL58/216pTqm22HcJ
There is wrong and correct way explained:
Doesn't work: curl -d "name=sam" http://localhost:9000/test
Works: curl -d "" http://localhost:9000/test?name=sam
This is the way how POST params are passing..in play. (second link is explanation WHY):
Sometimes you have to make compromises. In Play 1 you could bind your
action parameters from any parameter extracted from the URL path,
query string or even the request body. It was highly productive but
you had no way to control the way the form was uploaded. I mean, if a
user uploads a big file you needed to load the entire request in
memory to be able to handle it.
In Play 2 you can control the request body submission. You can reject
it early if something is wrong with the user, you can process big
files or streams without filling your memory with more than one HTTP
chunk… You gain a high control of what happens and it can help you to
scale you service. But, the other side of the coin is that when a
request is routed, Play 2 only uses the request header to make its
decision: the request body is not available yet, hence the inability
to directly bind an action parameter from a parameter extracted from
the request body.
UPDATE:
Interestly enough: after I got it working on my laptop then push it on gitHub and pull it to another machine it starts working differently. Now it complains than Bad Request is [Invalid XML] nevertheless I use "application/json" header and did not change any line of code after commit.
UPDATE 2
So I fixed it like this:
On angular side (we even can comment dataType and headers):
var data = $scope.fields
$http({
url: '/forms/FormValidator1/validateForm',
method: "POST",
//dataType: "json",
data: data,
//headers: {'Content-Type': 'application/json'}
}).success(function (data, status, headers, config) {
console.log("good")
}).error(function (data, status, headers, config) {
console.log("something wrong")
});
On playFramework side: (use BodyParser)
def validateForm = Action { request =>
val body: AnyContent = request.body
val jsonBody: Option[JsValue] = body.asJson
// Expecting text body
jsonBody.map { jsValue =>
val name = (jsValue \ "name")
val surname = (jsValue \ "surname")
....
}
Routes (don't define parameters at all !):
POST /forms/FormValidator1/validateForm controllers.FormValidator1.validateForm

Consuming broadbandmap.gov json service errors

I'm trying to consume the json services from broadbandmap.gov so that I can display broadband providers and their speeds in an area. Here is a sample url:
http://www.broadbandmap.gov/internet-service-providers/70508/lat=30.1471824/long=-92.033638/%3Ejson
I'm using jquery to consume the service, however it's giving me an invalid label error in firebug:
var url = "http://www.broadbandmap.gov/internet-service-providers/70508/lat=30.1471824/long=-92.033638/%3Ejson";
//var url = "http://www.broadbandmap.gov/broadbandmap/broadband/fall2010/wireline?latitude=" + lat + "&longitude=" + long + "&format=json";
$.ajax({
url: url,
dataType: 'json',
type: 'POST',
contentType: "application/json; charset=utf-8",
success: function (result) {
console.debug("in success");
console.debug(result);
//success, execute callback function.
},
error: function (result) {
console.debug("in error");
console.debug(result);
}
});
The strange thing is that under the Invalid Label error in Firebug it actually has the correct response:
{"status":"OK","responseTime":7,"messa...//www.cscic.state.ny.us/broadband/"}}}
I have tried setting the dataType to json, jsonp, and other types as well to no avail. I have also tried GET instead of POST but that didn't work either. Does anyone know what I'm missing?
That error is occurring because the service is returning JSON and not JSONP. Your browser is not going to let you process straight JSON from a cross-domain source.
In order to make the service return JSONP you have to use a specially formatted URL. If you go to the search results page without the "/>json" modifier (link) you'll see a link on the page that reads "API Call". If you hover over this link it will give you the correct URL to use for the wireless/wired API call. Use one of those URL's in your ajax call with a JSONP return type & callback and you should be all set.
I created an updated fiddle at http://jsfiddle.net/qsY7h/1/.
This is a cross-domain request so you should use JSONP datatype - the API supports this return type. The URL you provided in your example didn't return anything for me so I checked out Broadbandmap's Developer Docs and found an alternate call. Please find an example at http://jsfiddle.net/szCAF/.
The most important point to note is "callback=?" in the URL. jQuery uses this to tell the API what function name to wrap around the output (this is all done transparently by jQuery).