I am trying to implement a logic as below to route the requests across different backends depending on if some part of the URL is "odd" or "even" number. The logic is described as below:
Parse the URL and find the Id.
If Id even number (id%2 == 0) then route the request to "backendX". Else route to "backendY".
Sample URL: http://api.example.com/devices/demo-devices/{id}
Is it possible to implement such a logic as explained above using the ACL's? Or any alternate approach?
You can use lua to extend HAProxy 5 Ways to Extend HAProxy with Lua.
Related
I'm working in an app that makes many Ajax calls and results in a huge Network log in the Chrome inspector. I know that there are ways to query by things like mime-type, but I'm looking for a more fully-featured query capability.
For example, I'd like to be able to only see the pub/sub polling calls with a query like:
request_url:match(/pubnub.com/)
Or just see the GETs with:
request_method:GET
Is there a tool that makes queries like this possible?
You have a several options available. There are various pre-defined filter modes, as mentioned in the Network Analysis Reference (as you referenced in your question).
You can use the method filter with method:GET or method:POST in the input to only show requests of a particular method type. If you place a - beforehand, the filter will negate, e.g. -method:GET, will show all requests other than ones that are GETs.
There's also a filter type called domain, which is useful for only showing requests that match a particular domain. The options are limited though:
domain:stackoverflow.com would show all requests for the StackOverflow domain.
domain:*.google.co.uk would show all requests that are sub-domains
of Google UK.
Filtering request path (Method 1)
There's a better approach to filtering particular request paths. You can simply put pubnub.com in the filter input and it will match exactly what you put. You can also negate it with - beforehand, so entering -pubnub.com will show all requests that don't contain that in the path.
Filtering request path (Method 2)
You can also use Regex in the filter input, but not for the special filter modes (e.g. method, domain, etc.). So, in your case you could also use /pubnub.com/ as you filter. You can do more complex regular expressions, for instance, using /^((?!pubnub.com).)*$/ would do the equivalent of -pubnub.com via negative lookahead.
The reason I highlight Method 2 is because I fixed the feature a while ago in DevTools, as a result of another similar question that ended up being a bug in Method 1. Both bugs are completely fixed now though. See this for history of the problem if you're interested.
I have an API that produces JSON like this:
)]}',
{
//JSON DATA
}
The //JSON DATA is valid JSON, but the )]}', up top is not.
When I try to GET this data via a Logic App, I get:
BadRequest. Http request failed: the content was not a valid JSON.
So, a few related questions:
1) Can I tell the logic app to return the invalid JSON anyway?
2) How can debug the issue better? I happen to know that the response is invalid, but what if I didn't? Can I see the raw data somewhere?
3) This is all done via the Azure web portal. Are there better tools? Visual Studio?
I should also mention that if I call a route on the same API that returns XML instead of JSON, then the Logic App works fine. So it definitely doesn't like the JSON response in particular.
Thanks!
First of all, please do not post three questions as a single question.
Question 1). The best thing you can do is make the API return a valid JSON object. This is good for million reasons. Here're a few:
it's pretty much a standard (either valid JSON or XML -- yeah, old school way);
therefore, no users of this API (including you) will need to struggle and guess what's going on and why;
your Logic App's step will just work without adding extra complexity;
you will make this world and your karma better.
If API-side changes are not within your reach, I don't think you can do much. If you're lucky and the HTTP action is successful (Status Code 2xx), you can try to use a Query Action with a function that truncates the first characters. It will look something like this (I don't know the exact syntax): #Substring(body('myHttpGet'), 4, length(body('myHttpGet')) - 4) where myHttpGet is the id of the Http Get action.
However, once again, if possible, I strongly recommend fixing up the API which is the root cause of the problem, instead of dealing with garbage response after that.
UPDATE Another thing you can do is wrap the dirty API. For example, you could create a trivial Azure Function that invokes the API you don't directly control, and sanitizes the response for you consumption requirements. This Azure Function function should be easy to call from the Logic App. It costs almost nothing (unless we're talking millions of requests/month). The only drawback here is the increasing latency, which may be not an issue at all -- test it and see whether it adds less than 100ms or so... Oh, and don't forget to file a ticket with the API owner, they make our world a bad place!
Question 2) In Azure Logic App web UI you can Look into the execution details and the error will definitely be there.
Question 3) You're asking for a tool recommendation which is by definition a highly subjective thing and is off-topic on StackOverflow.
TL/DR: The other app is not producing valid JSON.
Meaning, this is not a problem for you to solve. The other app has to return valid JSON if the owner claims it should.
If they cannot or will not produce valid JSON, then the first thing you need to do is inform your management that you will have to spend a lot of extra time accommodating their non-standard format.
A REST API can have arguments in several places:
In the request body - As part of a json body, or other MIME type
In the query string - e.g. /api/resource?p1=v1&p2=v2
As part of the URL-path - e.g. /api/resource/v1/v2
What are the best practices and considerations of choosing between 1 and 2 above?
2 vs 3 is covered here.
What are the best practices and considerations of choosing between 1
and 2 above?
Usually the content body is used for the data that is to be uploaded/downloaded to/from the server and the query parameters are used to specify the exact data requested. For example when you upload a file you specify the name, mime type, etc. in the body but when you fetch list of files you can use the query parameters to filter the list by some property of the files. In general, the query parameters are property of the query not the data.
Of course this is not a strict rule - you can implement it in whatever way you find more appropriate/working for you.
You might also want to check the wikipedia article about query string, especially the first two paragraphs.
I'll assume you are talking about POST/PUT requests. Semantically the request body should contain the data you are posting or patching.
The query string, as part of the URL (a URI), it's there to identify which resource you are posting or patching.
You asked for a best practices, following semantics are mine. Of course using your rules of thumb should work, specially if the web framework you use abstract this into parameters.
You most know:
Some web servers have limits on the length of the URI.
You can send parameters inside the request body with CURL.
Where you send the data shouldn't have effect on debugging.
The following are my rules of thumb...
When to use the body:
When the arguments don't have a flat key:value structure
If the values are not human readable, such as serialized binary data
When you have a very large number of arguments
When to use the query string:
When the arguments are such that you want to see them while debugging
When you want to be able to call them manually while developing the code e.g. with curl
When arguments are common across many web services
When you're already sending a different content-type such as application/octet-stream
Notice you can mix and match - put the the common ones, the ones that should be debugable in the query string, and throw all the rest in the json.
The reasoning I've always used is that because POST, PUT, and PATCH presumably have payloads containing information that customers might consider proprietary, the best practice is to put all payloads for those methods in the request body, and not in the URL parms, because it's very likely that somewhere, somehow, URL text is being logged by your web server and you don't want customer data getting splattered as plain text into your log filesystem.
That potential exposure via the URL isn't an issue for GET or DELETE or any of the other REST operations.
I'm attempting to move an existing (and working) client-side jQuery validation schema to JSONSchema to allow myself to validate arbitrary JSON on both the client and server.
My application is essentially a bunch of gigantic forms with lots of complex logic determining which questions should be asked based on the user's response to other questions. The forms each have over 200 fields.
Right now I'm only doing client-side validation and that works well about 99% of the time. Browser issues have cropped up on a few occasions, but nothing catastrophic. That being said, I want to do server-side validation (!).
After reading the JSONSchema draft and browsing around some of the v3 implementations, it seems like I might lose some of the more complex rules that my application has come to depend upon. I want to be sure that I'm not missing something before moving too far in any direction.
Some examples:
"If x == 10, then y is required, otherwise it's optional"
10 could be a literal value, an enum, etc., but I need to be able to reference another field in the same structure and guarantee it's value not only exists, but is equivalent to a specific type / value.
I think this is addressed in this thread on the JSONSchema list.
"If x = today's date, and y = tomorrow's date, then x > y"
This logic will be used to ensure that the "from" date comes before the "to" date.
From what I can see there's nothing like this and the only way I can see doing it is passing in a freshly eval-ed chunk of JSON as the schema.
The closest thing I've found to meet the above needs is CERNY.
If I'm barking up the wrong tree, please let me know. I'm also looked into running backbone.js on both the client and server.
tl;dr;
I want to maintain one set of validation rules for large and complex forms and apply these validation rules to arbitrary JSON documents on both the client and server side.
there is many tricks but not all of them are possible. For example
if x == 10 then y is required can be achieved with something like (draft 3):
"type":[
{"properties":{"x":{"enum":[10]}, "y":{"required":true}}},
{"properties":{"x":{"disallow":[{"enum":[10]}]}}}
]
Let's say, it's possible but very tricky… a schema is basically supposed to validate the structure, not it's content (even if there is few properties for this)
Another possible way I personally do like is to "extend" the current validation graph with an external url based schema. The idea is to send parameters of the current document on an url which one will return a relevant schema according to those parameters.
Example:
{
"extends":{"$ref":"http://checkCustomValidity/{x}/{y}/"};
}
Where at "runtime" the schema sent back could be a {"disallow":"any"} if not allowed or {} if ok
This is useful as the url can be both used for the client and server side (your client will not be completely standalone but in some cases, you just cannot)
A real life usage for this is in cases where it is obliged to use a remote service. For example if I do have to check if my nickname is already used on the server during the registration. I code a server side web service answering to the request path: http://www.server.com/isNicknameUsed/{nickname}
In regards to this Haacked blog, I'm hesitant to implement the proposed anti-JSON GET hijacking solutions since
The recommended solutions to mitigating JSON hijacking involve non-REST-full JSON POSTs to GET data
The alternate solution (object wrapping) causes problems with 3rd party controls I don't have source-code access to.
I can't find a community-vetted implementation that implements the Alternative Solution (listed below) on how to compose the security token, or securely deliver it within the webpage. I also won't claim to be enough of an expert to roll my own implementation.
Referrer headers can't be relied upon
Background
This blog describes a CSRF issue regarding JSON Hijacking and recommends using JSON POSTs to GET data. Since using a HTTP POST to GET data isn't very REST-full, I'd looking for a more RESTfull solution that enables REST actions per session, or per page.
Another mitigation technique is to wrap JSON data in an object as described here. I'm afraid this may just delay the issue, until another technique is found.
Alternative Implementation
To me, it seems natural to extend the use ASP.NET MVC's AntiForgeryToken with jQuery HTTP GETs for my JSON.
For example if I GET some sensitive data, according to the Haacked link above, the following code is vulnerable:
$.getJSON('[url]', { [parameters] }, function(json) {
// callback function code
});
I agree that it isn't RESTfull to GET data using the recommended POST workaround. My thought is to send a validation token in the URL. That way the CSRF-style attacker won't know the complete URL. Cached, or not cached, they won't be able to get the data.
Below are two examples of how a JSON GET query could be done. I'm not sure what implementation is most effective, but may guess that the first one is safer from errant proxies caching this data, thus making it vulnerable to an attacker.
http://localhost:54607/Home/AdminBalances/ENCODEDTOKEN-TOKEN-HERE
or
http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE
... which might as well be MVC3's AntiForgeryToken, or a variant (see swt) thereof. This token would be set as an inline value on whatever URL format is chosen above.
Sample questions that prevent me from rolling my own solution
What URL format (above) would you use to validate the JSON GET (slash, questionmark, etc) Will a proxy respond to http://localhost:54607/Home/AdminBalances with http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE data?
How would you deliver that encoded token to the webpage? Inline, or as a page variable?
How would you compose the token? Built in AntiforgeryToken, or by some other means?
The AntiForgeryToken uses a cookie. Would a backing cookie be used/needed in this case? HTTP Only? What about SSL in conjunction with HTTP Only?
How would you set your cache headers? Anything special for the Google Web Accelerator (for example)
What are the implications of just making the JSON request SSL?
Should the returned JSON array still be wrapped in an object just for safety's sake?
How will this solution interop with Microsoft's proposed templating and databinding features
The questions above are the reasons I'm not forging ahead and doing this myself. Not to mention there likely more questions I haven't thought of, and yet are a risk.
The Asp.net MVC AntiForgeryToken won't work through HTTP GET, because it relies on cookies which rely on HTTP POST (it uses the "Double Submit Cookies" technique described in the OWASP XSRF Prevention Cheat Sheet). You can also additionally protect the cookies sent to the client by setting the as httponly, so they cannot be spoofed via a script.
In this document you can find various techniques that can be used to prevent XSRF. It seems the you described would fall into the Approach 1. But we have a problem on how to retrieve the session on the server when using Ajax HTTP GET request since the cookies are not sent with the request. So you would also have to add a session identifier to you action's URL (aka. cookieless sessions, which are easier to hijack). So in order to perform an attack the attacker would only need to know the correct URL to perform the GET request.
Perhaps a good solution would be to store the session data using some key from the users SSL certificate (for example the certs thumb-print). This way only the owner of the SSL certificate could access his session. This way you don't need to use cookies and you don't need to send session identifiers via query string parameters.
Anyway, you will need to roll out your own XSRF protection if you don't want to use HTTP POST in Asp.net MVC.
I came to this problem and the solution was not so trivial however there is a fantastic blog to get you started this can be used with get and post ajax.
http://johan.driessen.se/posts/Updated-Anti-XSRF-Validation-for-ASP.NET-MVC-4-RC
If you place the following in the global name space all your post/gets can take advantage having an anti forgery token and you don't have to modify your ajax calls. Create an input element in a common page.
<form id="__AjaxAntiForgeryForm" action="#" method="post">#Html.AntiForgeryToken()</form>
The following javascript will read the anti forgery tokken and add it to the request header.
// Wire up the global jQuery ajaxSend event handler.
$(document).ajaxSend(namespace.ajax.globalSendHandler);
// <summary>
// Global handler for all ajax send events.
// </summary>
namespace.ajax.globalSendHandler = function (event, xhr, ajaxOptions) {
// Add the anti forgery token
xhr.setRequestHeader('__RequestVerificationToken', $("#__AjaxAntiForgeryForm input[name=__RequestVerificationToken]").val());
};
I think it is legitimate to use AntiforgeryToken (AFT) within an ajax http GET request provided that it is embedded in a form that already provides the AFT and associated cookie. The ajax handler can then do the validate on the server just how it would in a normal form post.