Short: Is it standard-compliant, RESTful and otherwise good idea to enable PATCH requests to update a collection of resources, not just a single one, but still individually?
Long:
I'm considering exposing a method for enabling batch, atomic updates to my collection of resources. Example:
PATCH /url/myresources
[
{
"op": "add",
"path": "/1", // ID if the individual resource
"value":
{
... full resource representation ...
}
},
{
"op": "remove",
"path": "/2"
},
{
"op": "replace",
"path": "/3/name",
"value": "New name"
}
]
The context is a public API of a commercial solution. The benefits of allowing such PATCHes is the atomicity as well as batch-friendliness without spamming requests, handling failures individually etc.
I've consulted https://www.rfc-editor.org/rfc/rfc6902 and https://www.rfc-editor.org/rfc/rfc5789 but couldn't find a definitive answer if this is compliant. The RFCs mostly refer to "a resource", but a collection of resources could also be treated as such.
Is this a good idea? Are there better alternatives?
I like this idea. A collection is a resource, too. So acting on it is perfectly good REST.
The semantic of your PATCH request would be that every subresource not listed in the request body is to be left as it is. Every subresource that is listed is to be changed as described. Yes, that sounds good to me.
As long as every segment of the request can be executed in a single request, I see no problems. Both your "all in one" request and single requests like this would be fine.
PATCH /url/myresources/1
{
"op": "add",
"value":
{
... full resource representation ...
}
}
Related
I want to monitor the progress of an extraction job of a file stored in a bucket.
I've gone through the Webhooks API documentation, and if I understood everything correctly, the events only work for files stored in a folder, so it can't be used with a bucket. Please correct me if I'm wrong.
So, besides polling the GET :urn/manifest endpoint in the Model Derivate API, and assuming the Webhooks API can't be used with a bucket, is there any other way to monitor the progress of an extraction job?
Thanks.
Webhook now supports the Model Derivative event extraction.finished. This type is exactly used in the scenario of translating model of the bucket.
This is a blog on the API. I copied it as a reference on SO:
https://forge.autodesk.com/blog/introducing-webhook-model-derivative-api
Webhook now supports the Model Derivative event extraction.finished, so your app can be notified when translation job finishes.
To use it, (1) create a hook specifying a scope.workflow:
{
"callbackUrl": "http://bf067e05.ngrok.io/callback",
"scope": {
"workflow": "my-workflow-id"
}
}
And (2) when posting a translation job, specify the misc.workflow:
{
"input": {
"urn":
"dXJuOmFkc2sub2JqZWN0czpv...."
},
"output": {
"formats": [{
"type": "obj"
}]
},
"misc": {
"workflow": "my-workflow-id"
}
}
With my test, the workflow can be guid of your WebHook.The post body from Forge will tell you which file is translated and its urn, location etc.
Hope it helps.
I'm building API for SinglePageApplication, which handle by Angular in frontend. One thing is not clear to me.
Supose the web applcation has delati journal paige wich display journal,some articles which belongs to this journal and some cool authors which can be not connected to this journal.
Should I build my api urls based on each need page content, for example:
from url /api/journal/<journal_id>
send json:
{
"journal": {
"id": 10,
"name": "new_journal",
"articles": [
{
"name": "cool_article",
"id": 42
},
{
"name": "another_cool_article",
"id": 43
}
]
},
"authors": [
{
"name": "some_name",
"id": 42
},
{
"name": "another_name",
"id": 43
}
]
}
Or I should build my api based on concrete objects and related objects of them.
With urls like this:
/api/journals/<journal_id>
/api/authors/
And frontend side build this page with two GET requests for fetching content.
Sory if my question too broad, I just want to find best bractice for building API to SinglePageApplications.
Does it have any difference of building API enpoints for external web-apps and what I should do if page need to display more objects, which not belong together? Which of the options above is better?
There isn't really a universal right answer for this. It largely depends on the use case for that data you're fetching. I would say to err on the side of splitting this into multiple requests as it grants you flexibility and efficiency in terms of partial updates to the page. That approach also makes exposing an API to the public much easier in terms of being able to just expose what you already have.
If you're dealing with a potentially large (an intentionally relative term) number of concurrent requests though, you may build some composites of related data to mitigate that.
Of course, you could also do a combination of the two as well (first load makes 1 large request, subsequent updates are segmented).
I am designing a web service to regularly receive updates to lists. At this point, a list can still be modeled as a single entity (/lists/myList) or an actual collection with many resources (/lists/myList/entries/<ID>). The lists are large (millions of entries) and the updates are small (often less than 10 changes).
The client will get web service URLs and lists to distribute, e.g.:
http://hostA/service/lists: list1, list2
http://hostB/service/lists: list2, list3
http://hostC/service/lists: list1, list3
It will then push lists and updates as configured. It is likely but undetermined if there is some database behind the web service URLs.
I have been researching and it seems a HTTP PATCH using the JSON patch format is the best approach.
Context and examples:
Each list has an identifying name, a priority and millions of entries. Each entry has an ID (determined by the client) and several optional attributes. Example to create a list "requiredItems" with priority 1 and two list entries:
PUT /lists/requiredItems
Content-Type: application/json
{
"priority": 1,
"entries": {
"1": {
"color": "red",
"validUntil": "2016-06-29T08:45:00Z"
},
"2": {
"country": "US"
}
}
}
For updates, the client would first need to know what the list looks like now on the server. For this I would add a property "revision" to the list entity.
Then, I would query this attribute:
GET /lists/requiredItems?property=revision
Then the client would see what needs to change between the revision on the server and the latest revision known by the client and compose a JSON patch. Example:
PATCH /list/requiredItems
Content-Type: application/json-patch+json
[
{ "op": "test", "path": "revision", "value": 3 },
{ "op": "add", "path": "entries/3", "value": { "color": "blue" } },
{ "op": "remove", "path": "entries/1" },
{ "op": "remove", "path": "entries/2/country" },
{ "op": "add", "path": "entries/2/color", "value": "green" },
{ "op": "replace", "path": "revision", "value": 10 }
]
Questions:
This approach has the drawback of slightly less client support due to the not-often-used HTTP verb PATCH. Is there a more compatible approach without sacrificing HTTP compatibility (idempotency et cetera)?
Modelling the individual list entries as separate resources and using PUT and DELETE (perhaps with ETag and/or If-Match) seems an option (PUT /lists/requiredItems/entries/3, DELETE /lists/requiredItems/entries/1 PUT /lists/requiredItems/revision), but how would I make sure all those operations are applied when the network drops in the middle of an update chain? Is a HTTP PATCH allowed to work on multiple resources?
Is there a better way to 'version' the lists, perhaps implicitly also improving how they are updated? Note that the client determines the revision number.
Is it correct to query the revision number with GET /lists/requiredItems?property=revision? Should it be a separate resource like /lists/requiredItems/revision? If it should be a separate resource, how would I update it atomically (i.e. the list and revision are both updated or both not updated)?
Would it work in JSON patch to first test the revision value to be 3 and then update it to 10 in the same patch?
This approach has the drawback of slightly less client support due to the not-often-used HTTP verb PATCH.
As far as I can tell, PATCH is really only appropriate if your server is acting like a dumb document store, where the action is literally "please update your copy of the document according to the following description".
So if your resource really just is a JSON document that describes a list with millions of entries, then JSON-Patch is a great answer.
But if you are expecting that the patch will, as a side effect, update an entity in your domain, then I'm suspicious.
Is a HTTP PATCH allowed to work on multiple resources?
RFC 5789
The PATCH method affects the resource identified by the Request-URI, and it also MAY have side effects on other resources
I'm not keen on querying the revision number; it doesn't seem to have any clear advantage over using an ETag/If-Match approach. Some obvious disadvantages - the caches between you and the client don't know that the list and the version number are related; a cache will happily tell a client that version 12 of the list is version 7, or vice versa.
Answering my own question. My first bullet point may be opinion-based and, as has been pointed out, I've asked many questions in one post. Nevertheless, here's a summary of what was answered by others (VoiceOfUnreason) and my own additional research:
ETags are HTTP's resource 'hashes'. They can be combined with If-Match headers to have a versioning system. However, ETag-headers are normally not used to declare the ETag of a resource that is being created (PUT) or updated (POST/PATCH). The server storing the resource usually determines the ETag. I've not found anything explicitly forbidding this, but many implementations may assume that the server determines the ETag and get confused when it is provided with PUT or PATCH.
A separate revision resource is a valid alternative to ETags for versioning. This resource must be updated at the same time as the resource it is the revision of.
It is not semantically enforceable on a HTTP level to have commit/rollback transactions, unless by modelling the transaction itself as a ReST resource, which would make things much more complicated.
However, some properties of PATCH allow it to be used for this:
A HTTP PATCH must be atomic and can operate on multiple resources. RFC 5789:
The server MUST apply the entire set of changes atomically and never provide (e.g., in response to a GET during this operation) a partially modified representation. If the entire patch document cannot be successfully applied, then the server MUST NOT apply any of the changes.
The PATCH method affects the resource identified by the Request-URI, and it also MAY have side effects on other resources; i.e., new resources may be created, or existing ones modified, by the application of a PATCH. PATCH is neither safe nor idempotent
JSON PATCH can consist of multiple operations on multiple resources and all must be applied or none must be applied, making it an implicit transaction. RFC 6902: Operations are applied sequentially in the order they appear in the array.
Thus, the revision can be modeled as a separate resource and still be updated at the same time. Querying the current revision is a simple GET. Committing a transaction is a single PATCH request containing first a test of the revision, then the operations on the resource(s) and finally the operation to update the revision resource.
The server can still choose to publish the revision as ETag of the main resource.
The problem is even if I put condValues to PT10S, when I send request to contextBroker it requests back the reference url rigth away, not after 10 sec, and then it continues to send requests at 10 sec.
My question: is there a way to avoid the first initial request?
Here is a body of the request that I send to server where contextBroker is installed.
{
"entities": [{
"type": "Cycle",
"isPattern": "false",
"id": "someid"
}],
"attributes": [
...
],
"reference": "someurl"
"duration": "P1M",
"notifyConditions": [{
"type": "ONTIMEINTERVAL",
"condValues": [
"PT10S"
]
}]
}
At the present moment (Orion 1.1) initial notification cannot be avoided. However, being able to configure that behaviour would be an interesting feature to develop in the future and, consecuently, a github issue was created time ago about it.
In addition, note that ONTIMEINTERVAL subscriptions are no longer supported so you should avoid to use them:
ONTIMEINTERVAL subscriptions have several problems (introduce state in CB, thus making horizontal scaling configuration much harder, and makes it difficult to introduce pagination/filtering). Actually, they aren't really needed, as any use case based on ONTIMEINTERVAL notification can be converted to an equivalent use case in which the receptor runs queryContext at the same frequency (and taking advantage of the features of queryContext, such as pagination or filtering).
EDIT: the posibility of avoiding initial notification has been finally implemented at Orion. Details are at this section of the documentation. It is now in the master branch (so if you use fiware/orion:latest docker you will get it) and will be include in next Orion version (2.2.0).
Is there a way to display a bunch of JSON objects, WITH relations, as a domain model?
Models could be something like this:
{
"name": "Order",
"status": {
"type": "number",
"null": false,
"default": 1
}
},
"relations": {
"customer": {
"type": "belongsTo",
"model": "Customer",
"foreignKey": "customerId"
}
}
}
Basically, as this image shows, Domain model -to-> code... but the other way around.?
Thanks
PS: If there are no tools that do this out of the box, I assume that there might be frameworks that I could use to create this.. any recommendations?
As I understand your question you are asking if a utility exists that can turn your JSON code into a domain model. If I understand correctly the answer is yes. There is one project on github that is doing something similar: json-discoverer
From the project page you will see the tool was inspired by some research which was published in ICWE (International Conference on Web Engineering) 2013 and 2014. Below is a link to the main article, as it is quite lengthy and detailed I will not attempt to summarize it here.
Discovering Implicit Schemas in JSON Data
Unfortunately, as you mention you can't then edit the domain afterward. But I have still found it to be a fairly useful tool.
I am unaware of any other existing utilities. The only other alternatives are fairly easy to find with a simple search, but only allow for conversion to trees and/or tables.