I'm trying to develop scenarios for load testing an application with Artillery. The application in question uses the GET, POST, PUT, and DELETE html methods, but all of the available documentation I've been able to find only discusses the get: and post: keywords (under scenarios: - flow:) which I assume would use the GET and POST methods. At least two sources I've found, including one at artillery.io, imply that PUT, PATCH, and DELETE are supported, but then provide no information on how to specify the http method on an http request. Can someone advise how to specify PUT and DELETE requests in an Artillery scenario?
(By the way, there are at least 10 other html methods besides the 5 mentioned here.)
PUT and DELETE will follow similar patterns as GET / POST
config:
target: "https://example.com"
phases:
- duration: 10
arrivalRate: 1
scenarios:
- flow:
- delete:
url: "/users/12345"
- put:
url: "/users/12345"
json:
firstName: "hello world"
lastName: "Huh!"
Related
I am looking at defining a REST API using HATEOAS. In particular, I find very interesting the concept of indicating for a given resource the actions that are available right now.
Some of the HATEOAS specifications include too much overhead for my needs, so I was looking at the HAL specification, as I find it very concise and practical:
{
_links: {
self: { href: "/orders/523" },
warehouse: { href: "/warehouse/56" },
invoice: { href: "/invoices/873" }
},
currency: "USD",
status: "shipped",
total: 10.20
}
However, links in HAL only contain a list of related resources, but not the available actions on them. As per the example above, am I allowed to cancel the order now, or not anymore? Some HAL examples solve this by using a specific URL for cancellations, and add a corresponding link in the response only if cancellation is possible:
"cancel": { "href": "/orders/523/cancel" }
But that is not very RESTful. Cancellations are not a resource. Cancellations are a DELETE of a resource, i.e.:
DELETE /orders/523
Is there a nice way to represent this with HAL, or should I use a different HATEOAS specification?
I am considering returning a "cancel" link with the same URL as self, but in this case the client would have to know that to cancel they have to use the DELETE verb, which is not really being described in the HATEOAS response.
self: { "href": "/orders/523" },
cancel: { "href": "/orders/523" }
Would this be the recommended approach as per HATEOAS / HAL? I understand HAL does not have any "method" parameter, and adding it myself would be against the HAL specification.
Some HAL examples solve this by using a specific URL for cancellations, and add a corresponding link in the response only if cancellation is possible
Yes. Just like web sites: if you want to alert the client to the possibility of reaching some other application state, you provide the client with a link, including the identifier for the resource involved.
But that is not very RESTful.
It may not be "RESTful", but it is certainly conforms to the REST architectural style.
Cancellations are a DELETE of a resource, i.e.: DELETE /orders/523
You are confusing the actions on the domain model with the actions on the integration model. What a REST API does is guide the client through a protocol to achieve some end; it is not a mapping of domain semantics onto HTTP.
Jim Webber phrased it this way:
The web is not your domain; it's a document management system. All of the HTTP verbs apply to the document management domain. URIs do NOT map onto domain objects -- that violates encapsulation. Work (ex: issuing commands to the domain model) is a side effect of managing resources.
One of the REST constraints is the uniform interface; in the case of HTTP, it means that all resources understand methods in a uniform way; DELETE means the semantics described in RFC 7231, section 4.3.5.
In other words, if I send the request
OPTIONS /x/y/z/foobar ...
and the response includes DELETE in the Allow header, then I know what it means. The side effects in your domain? I don't know anything about the side effects.
In the definition of DELETE, note the following
Relatively few resources allow the DELETE method -- its primary use is for remote authoring environments, where the user has some direction regarding its effect.
Anyway, you aren't really asking about DELETE, but about HAL
Is there a nice way to represent this with HAL, or should I use a different HATEOAS specification?
From what I can tell, the official way to do it is to document it with the link relation. In other words, instead of using "cancel" as the link relation, you use something like
https://www.rfc-editor.org/rfc/rfc5023#section-5.4.2
And then your consumers, if they want to discover what a link is for, can follow the relation to learn what is going on.
HAL Discuss: Why No Methods? has a lot of good information.
I like Mike Kelly's summary:
The idea is the available methods can be conveyed via the link
relation documentation and don't need to be in the json messages.
According to this article from LosTechies, in a CQRS perspective, Its accepted to use URLs such as: /orders/<id>/<command> and call these with PUT requests. So its OK to use a "cancel": { "href": "/orders/523/cancel" }.
However, if you absolutely want to use DELETE and you only use command links to modify your ressources (i.e. /orders/<id>/<command>) like proposed the article, why can't you just add a link such as "cancel": { "href": "/orders/523" } and deduct the HTTP verb?
I mean according to REST there is only 5 main verbs (GET, POST, PUT, PATCH and DELETE). We can't use POST on a URL such as /<ressource>/<id>, GET is already define as the "self" relation, we mentioned above that modifications (PUT) will be handled by command links (i.e. /<ressource>/<id>/<command>) and because we use command links there is no need to use PATCH. After that, the only option left is: DELETE.
It's not perfect, but it works and it doesn't break any convention.
I am trying to stick to the Restful design pattern for both JSON and HTML. My issue is the design for creating a new resource (amongst others, but this is the gist of the issue). IE:
JSON – POST to /resource creates a new resource.
JSON – GET to /resource returns a list of resources.
JSON – GET to /resource/{id} returns a resource.
HTML – POST to /resource creates a new resource.
HTML – GET to /resource returns a list of resources.
HTML – GET to /resource/{id} returns a resource.
All good so far – but I need a HTML form to actually create the data to send to the HTML POST. Obviously POST and GET already do things. I could use one of the below to return the HTML form:
HTML – GET to /resource?CREATE
HTML - GET to /resource?action=CREATE
HTML – GET to /resources/CREATE
But they seem like a kludge and not that intuitive.
Any thoughts or ideas?
EDIT - See my answer to my question below. At present this is (I consider) the best option.
I would indeed use something like /resources/create. If you want to allow for non-numeric identifiers, then this will not work. In that case you can identify a resource with a prefix, such as /resources/resource-{id} and then you can still use /resources/create.
I found this blog post really helpful to make URI scheme decisions: http://blog.2partsmagic.com/restful-uri-design/
In fact, you should leverage content negotiation (CONNEG) when you want to handle several formats within RESTful services.
I mean:
Set the Content-Type header to specify the type of sent data
Set the Accept header to specify the type of data you want to receive
The server resources should leverage these hints to make the appropriate data conversion.
In the case of JSON, the content type would be obviously application/json. For HTML form, you should leverage the content type application/x-www-form-urlencoded (or multipart/form-data if you want to upload files as well). See the specification for more details.
Otherwise, you shouldn't use action in URL since it's not really RESTful. The HTTP verb should determine the action to do on the resource. I mean, to create a resource, the POST method should be used. The GET method aims to retrieve the state of a resource.
For more details, you could have a look at this blog post:
Designing a Web API (i.e. RESTful service).
I have an answer. I'll use standard RESTful POST from a HTML page, but when I have no form parameters sent and my accept header is text/html, I'll send a HTML form to the requestor. Keeps RESTful URI design and allows a clean HTML form + process (2 step).
HTML - POST - /resources (with no form attributes) generates a HTML form
HTML - POST - /resources (with form attributes) adds a resource
JSON - POST - /resources (with form attributes) adds a resource
OK, it's not "strictly" RESTful as I'm POSTing but not creating a new resource so in theory I should use a GET for that, but it's the best of a mismatched design.
If anyone can provide a better solution, I'm still all ears :-)
I'd rather add and endpoint called /templates/ that returns a template/form/whatever you need for given action. It also seems that the server should be unaware of such form existence. It can accept or reject a request and it's client job to submit it in an appropriate format.
I guess that you mix processing the view with preparing RESTful endpoints. The backend site should be completely unaware of the fact that some sort of view/form is required. It's client job to prepare such form.
A simple REST API:
GET: items/{id} - Returns a description of the item with the given id
PUT: items/{id} - Updates or Creates the item with the given id
DELETE: items/{id} - Deletes the item with the given id
Now the API-extension in question:
GET: items?filter - Returns all item ids matching the filter
PUT: items - Updates or creates a set of items as described by the JSON payload
[[DELETE: items - deletes a list of items described by JSON payload]] <- Not Correct
I am now being interested in the DELETE and PUT operation recycling functionality that can be easily accessed by PUT/DELETE items/{id}.
Question: Is it common to provide an API like this?
Alternative: In the age of Single Connection Multiple Requests issuing multiple requests is cheap and would work more atomic since a change either succeeds or fails but in the age of NOSQL database a change in the list might already have happend even if the request processing dies with internal server or whatever due to whatever reason.
[UPDATE]
After considering White House Web Standards and Wikipedia: REST Examples the following Example API is now purposed:
A simple REST API:
GET: items/{id} - Returns a description of the item with the given id
PUT: items/{id} - Updates or Creates the item with the given id
DELETE: items/{id} - Deletes the item with the given id
Top-resource API:
GET: items?filter - Returns all item ids matching the filter
POST: items - Updates or creates a set of items as described by the JSON payload
PUT and DELETE on /items is not supported and forbidden.
Using POST seems to do the trick as being the one to create new items in an enclosing resource while not replacing but appending.
HTTP Semantics POST Reads:
Extending a database through an append operation
Where the PUT methods would require to replace the complete collection in order to return an equivalent representation as quoted by HTTP Semantics PUT:
A successful PUT of a given representation would suggest that a subsequent GET on that same target resource will result in an equivalent representation being returned in a 200 (OK) response.
[UPDATE2]
An alternative that seems even more consistent for the update aspect of multiple objects seems to be the PATCH method. The difference between PUT and PATCH is described in the Draft RFC 5789 as being:
The difference between the PUT and PATCH requests is reflected in the way the server processes the enclosed entity to modify the resource identified by the Request-URI. In a PUT request, the enclosed entity is considered to be a modified version of the resource stored on the origin server, and the client is requesting that the stored version be replaced. With PATCH, however, the enclosed entity contains a set of instructions describing how a resource currently residing on the origin server should be modified to produce a new version. The PATCH method affects the resource identified by the Request-URI, and it also MAY have side effects on other resources; i.e., new resources may be created, or existing ones modified, by the application of a PATCH.
So compared to POST, PATCH may be also a better idea since PATCH allows an UPDATE where as POST only allows appending something meaning adding without the chance of modification.
So POST seems to be wrong here and we need to change our proposed API to:
A simple REST API:
GET: items/{id} - Returns a description of the item with the given id
PUT: items/{id} - Updates or Creates the item with the given id
DELETE: items/{id} - Deletes the item with the given id
Top-resource API:
GET: items?filter - Returns all item ids matching the filter
POST: items - Creates one or more items as described by the JSON payload
PATCH: items - Creates or Updates one or more items as described by the JSON payload
You could PATCH the collection, e.g.
PATCH /items
[ { id: 1, name: 'foo' }, { id: 2, name: 'bar' } ]
Technically PATCH would identify the record in the URL (ie PATCH /items/1 and not in the request body, but this seems like a good pragmatic solution.
To support deleting, creating, and updating in a single call, that's not really supported by standard REST conventions. One possibility is a special "batch" service that lets you assemble calls together:
POST /batch
[
{ method: 'POST', path: '/items', body: { title: 'foo' } },
{ method: 'DELETE', path: '/items/bar' }
]
which returns a response with status codes for each embedded requests:
[ 200, 403 ]
Not really standard, but I've done it and it works.
Updating Multiple Resources With One Request - Is it standard or to be avoided?
Well, sometimes you simply need to perform atomic batch operations or other resource-related operations that just do not fit the typical scheme of a simple REST API, but if you need it, you cannot avoid it.
Is it standard?
There is no universally accepted REST API standard so this question is hard to answer. But by looking at some commonly quoted api design guidelines, such as jsonapi.org, restfulapi.net, microsoft api design guide or IBM's REST API Conventions, which all do not mention batch operations, you can infer that such operations are not commonly understood as being a standard feature of REST APIs.
That said, an exception is the google api design guide which mentions the creation of "custom" methods that can be associated via a resource by using a colon, e.g. https://service.name/v1/some/resource/name:customVerb, it also explicitly mentions batch operations as use case:
A custom method can be associated with a resource, a collection, or a service. It may take an arbitrary request and return an arbitrary response, and also supports streaming request and response. [...] Custom methods should use HTTP POST verb since it has the most flexible semantics [...] For performance critical methods, it may be useful to provide custom batch methods to reduce per-request overhead.
So in the example case you provided you do the following according to google's api guide:
POST /api/items:batchUpdate
Also, some public APIs decided to offer a central /batch endpoint, e.g. google's gmail API.
Moreover, as mentioned at restfulapi.net, there is also the concept of a resource "store", in which you store and retrieve whole lists of items at once via PUT – however, this concept does not count for server-managed resource collections:
A store is a client-managed resource repository. A store resource lets an API client put resources in, get them back out, and decide when to delete them. A store never generates new URIs. Instead, each stored resource has a URI that was chosen by a client when it was initially put into the store.
Having answered your original questions, here is another approach to your problem that has not been mentioned yet. Please note that this approach is a bit unconventional and does not look as pretty as the typical REST API endpoint naming scheme. I am personally not following this approach but I still felt it should be given a thought :)
The idea is that you could make a distinction between CRUD operations on a resource and other resource-related operations (e.g. batch operations) via your endpoint path naming scheme.
For example consider a RESTful API that allows you to perform CRUD operations on a "company"-resource and you also want to perform some "company"-related operations that do not fit in the resource-oriented CRUD-scheme typically associated with restful api's – such as the batch operations you mentioned.
Now instead of exposing your resources directly below /api/companies (e.g. /api/companies/22) you could distinguish between:
/api/companies/items – i.e. a collection of company resources
/api/companies/ops – i.e. operations related to company resources
For items the usual RESTful api http-methods and resource-url naming-schemes apply (e.g. as discussed here or here)
POST /api/companies/items
GET /api/companies/items
GET /api/companies/items/{id}
DELETE /api/companies/items/{id}
PUT /api/companies/items/{id}
Now for company-related operations you could use the /api/companies/ops/ route-prefix and call operations via POST.
POST /api/companies/ops/batch-update
POST /api/companies/ops/batch-delete
POST /api/companies/ops/garbage-collect-old-companies
POST /api/companies/ops/increase-some-timestamps-just-for-fun
POST /api/companies/ops/perform-some-other-action-on-companies-collection
Since POST requests do not have to result in the creation of a resource, POST is the right method to use here:
The action performed by the POST method might not result in a resource that can be identified by a URI.
https://www.rfc-editor.org/rfc/rfc2616#section-9.5
As far as I understand the REST concept it covers an update of multiple resources with one request. Actually the trick here is to assume a container around those multiple resources and take it as one single resource. E.g. you could just assume that list of IDs identifies a resource that contains several other resources.
In those examples in Wikipedia they also talk about resources in Plural.
I have a collection resource called Columns. A GET with Accept: application/json can't directly return a collection, so my representation needs to nest it in a property:-
{ "propertyName": [
{ "Id": "Column1", "Description": "Description 1" },
{ "Id": "Column2", "Description": "Description 2" }
]
}
Questions:
what is the best name to use for the identifier propertyName above? should it be:
d (i.e. is d an established convention or is it specific to some particular frameworks (MS WCF and MS ASP.NET AJAX ?)
results (i.e. is results an established convention or is it specific to some particular specifications (MS OData)?)
Columns (i.e. the top level property should have a clear name and it helps to disambiguate my usage of generic application/json as the Media Type)
NB I feel pretty comfortable that there should be something wrapping it, and as pointed out by #tuespetre, XML or any other representation would force you to wrap it to some degree anyway
when PUTting the content back, should the same wrapping in said property be retained [given that it's not actually necessary for security reasons and perhaps conventional JSON usage idioms might be to drop such nesting for PUT and POST given that they're not necessary to guard against scripting attacks] ?
my gut tells me it should be symmetric as for every other representation but there may be prior art for dropping the d/*results** [assuming that's the answer to part 1]*
... Or should a PUT-back (or POST) drop the need for a wrapping property and just go with:-
[
{ "Id": "Column1", "Description": "Description 1" },
{ "Id": "Column2", "Description": "Description 2" }
]
Where would any root-level metadata go if one wished to add that?
How/would a person crafting a POST Just Know that it needs to be symmetric?
EDIT: I'm specifically interested in an answer that with a reasoned rationale that specifically takes into account the impacts on client usage with JSON. For example, HAL takes care to define a binding that makes sense for both target representations.
EDIT 2: Not accepted yet, why? The answers so far don't have citations or anything that makes them stand out over me doing a search and picking something out of the top 20 hits that seem reasonable. Am I just too picky? I guess I am (or more likely I just can't ask questions properly :D). Its a bit mad that a week and 3 days even with an )admittedly measly) bonus on still only gets 123 views (from which 3 answers ain't bad)
Updated Answer
Addressing your questions (as opposed than going off on a bit of a tangent in my original answer :D), here's my opinions:
1) My main opinion on this is that I dislike d. As a client consuming the API I would find it confusing. What does it even stand for anyway? data?
The other options look good. Columns is nice because it mirrors back to the user what they requested.
If you are doing pagination, then another option might be something like page or slice as it makes it clear to the client, that they are not receiving the entire contents of the collection.
{
"offset": 0,
"limit": 100,
"page" : [
...
]
}
2) TBH, I don't think it makes that much difference which way you go for this, however if it was me, I probably wouldn't bother sending back the envelope, as I don't think there is any need (see below) and why make the request structure any more complicated than it needs to be?
I think POSTing back the envelope would be odd. POST should let you add items into the collection, so why would the client need to post the envelope to do this?
PUTing the envelope back could make sense from a RESTful standpoint as it could be seen as updating metadata associated with the collection as a whole. I think it is worth thinking about the sort of meta data you will be exposing in the envelope. All the stuff I think would fit well in this envelope (like pagination, aggregations, search facets and similar meta data) is all read only, so it doesn't make sense for the client to send this back to the server. If you find yourself with a lot of data in the envelope that the client is able to mutate - then it could be a sign to break that data out into a separate resource with the list as a sub collection. Rubbish example:
/animals
{
"farmName": "farm",
"paging": {},
"animals": [
...
]
}
Could be broken up into:
/farm/1
{
"id": 1,
"farmName": "farm"
}
and
/farm/1/animals
{
"paging": {},
"animals": [
...
]
}
Note: Even with this split, you could still return both combined as a single response using something like Facebook's or LinkedIn's field expansion syntax. E.g. http://example.com/api/farm/1?field=animals.offset(0).limit(10)
In response, to your question about how the client should know what the JSON payload they are POSTing and PUTing should look like - this should be reflected in your API documentation. I'm not sure if there is a better tool for this, but Swagger provides a spec that allows you to document what your request bodies should look like using JSON Schema - check out this page for how to define your schemas and this page for how to reference them as a parameter of type body. Unfortunately, Swagger doesn't visualise the request bodies in it's fancy web UI yet, but it's is open source, so you could always add something to do this.
Original Answer
Check out William's comment in the discussion thread on that page - he suggests a way to avoid the exploit altogether which means you can safely use a JSON array at the root of your response and then you need not worry about either of you questions.
The exploit you link to relies on your API using a Cookie to authenticate a user's session - just use a query string parameter instead and you remove the exploit. It's probably worth doing this anyway since using Cookies for authentication on an API isn't very RESTful - some of your clients may not be web browsers and may not want to deal with cookies.
Why Does this fix work?
The exploit is a form of CSRF attack which relies on the attacker being able to add a script tag on his/her own page to a sensitive resource on your API.
<script src="http://mysite.com/api/columns"></script>
The victims web browser will send all Cookies stored under mysite.com to your server and to your servers this will look like a legitimate request - you will check the session_id cookie (or whatever your server-side framework calls the cookie) and see the user is authenticated. The request will look like this:
GET http://mysite.com/api/columns
Cookie: session_id=123456789;
If you change your API you ignore Cookies and use a session_id query string parameter instead, the attacker will have no way of tricking the victims web browser into sending the session_id to your API.
A valid request will now look like this:
GET http://mysite.com/api/columns?session_id=123456789
If using a JavaScript client to make the above request, you could get the session_id from a cookie. An attacker using JavaScript from another domain will not be able to do this, as you cannot get cookies for other domains (see here).
Now we have fixed the issue and are ignoring session_id cookies, the script tag on the attackers website will still send a similar request with a GET line like this:
GET http://mysite.com/api/columns
But your server will respond with a 403 Forbidden since the GET is missing the required session_id query string parameter.
What if I'm not authenticating users for this API?
If you are not authenticating users, then your data cannot be sensitive and anyone can call the URI. CSRF should be a non-issue since with no authentication, even if you prevent CSRF attacks, an attacker could just call your API server side to get your data and use it in anyway he/she wants.
I would go for 'd' because it clearly separates the 'envelope' of your resource from its content. This would also make it easier for consumers to parse your responses, as opposed to 'guessing' the name of the wrapping property of a given resource before being able to access what it holds.
I think you're talking about two different things:
POST request should be sent in application/x-www-form-urlencoded. Your response should basically mirror a GET if you choose to include a representation of the newly created resource in your reply. (not mandatory in HTTP).
PUTs should definitely be symmetric to GETs. The purpose of a PUT request is to replace an existing resource representation with another. It just makes sense to have both requests share the same conventions, doesn't it?
Go with 'Columns' because it is semantically meaningful. It helps to think of how JSON and XML could mirror each other.
If you would PUT the collection back, you might as well use the same media type (syntax, format, what you will call it.)
I know that there are two methods of submitting a form: 'GET' and 'POST'. We can also use request method for accessing the content of the submitted.
I want to know whether there is any other method of submitting the form. As far as my knowledge there are only two methods. But some one asked me this question in a interview that there are 5 method of submitting the form.
If any one has any idea about this please tell me.
The question was probably about HTTP request methods. There 9 request methods:
HTTP defines nine methods (sometimes referred to as "verbs")
indicating the desired action to be performed on the identified
resource. What this resource represents, whether pre-existing data or
data that is generated dynamically, depends on the implementation of
the server. Often, the resource corresponds to a file or the output of
an executable residing on the server.
HEAD: Asks for the response identical to the one that would
correspond to a GET request, but without the response body. This is
useful for retrieving meta-information written in response headers,
without having to transport the entire content.
GET: Requests a representation of the specified resource. Requests
using GET (and a few other HTTP methods) "SHOULD NOT have the
significance of taking an action other than retrieval". The W3C has
published guidance principles on this distinction, saying, "Web
application design should be informed by the above principles, but
also by the relevant limitations." See safe methods below.
POST: Submits data to be processed (e.g., from an HTML form) to
the identified resource. The data is included in the body of the
request. This may result in the creation of a new resource or the
updates of existing resources or both.
PUT: Uploads a representation of the specified resource.
DELETE: Deletes the specified resource.
TRACE: Echoes back the received request, so that a client can see
what (if any) changes or additions have been made by intermediate
servers.
OPTIONS: Returns the HTTP methods that the server supports for
specified URL. This can be used to check the functionality of a web
server by requesting '*' instead of a specific resource.
CONNECT: Converts the request connection to a transparent TCP/IP
tunnel, usually to facilitate SSL-encrypted communication (HTTPS)
through an unencrypted HTTP proxy.
PATCH: Is used to apply partial modifications to a resource.
HTTP servers are required to implement at least the GET and HEAD
methods
The HTML form element's method only accepts two parameters, GET and POST. Evidenced by this entry on the W3 Standards site:
method (GET|POST) GET -- HTTP method used to submit the form--
They may have been asking you about ways to submit the data. In which case there are many more, like AJAX, Flash, P2P types, etc.
However if they specifically said FORM, as in the HTML FORM element -- then no. POST and GET.
Addendum: Here is a StackOverflow question asked on a similar topic. In that the answerer highlights other methods which can be submitted via AJAX. Again, though, note that these are down through AJAX and not strictly through the FORM element.