So let's say I have a resources called articles. These have a numeric id and you can access them under something such as:
GET /articles/1 for a specific article.
And let's say that returns something like:
{
"data": {
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON:API paints my bikeshed!",
"body": "A bunch of text here"
}
}
}
Now my question is how to handle a request to GET /articles. I.e. how to deal with the request to the collection.
You see, accessing the body of the article is slow and painful. The last thing I want this REST API to do is actually try to get all that information. Yet as far as I can tell the JSON API schema seems to assume that you can always return full resources.
Is there any "allowed" way to return just the IDs (or partial attributes, like "title") under JSON API while actively not providing the ability to get the full resource?
Something like:
GET /articles returning:
{
"data": [
{
"type": "article_snubs",
"id": 1,
"attributes": {
"title": "JSON:API paints my bikeshed!"
}
}, {
"type": "article_snubs",
"id": 2,
"attributes": {
"title": "Some second thing here"
}
}
]
}
Maybe with links to the full articles?
Basically, is this at all possible while following JSON API or a REST standard? Because there is absolutely no way that GET /articles is ever going to be returning full resources due to the associate cost of getting the data, which I do not think is a rare situation to be in.
As far as I understand the JSON API specification there is no requirement that an API must return all fields (attributes and relationships) of a resource by default. The only MUST statement regarding fields inclusion that I'm aware of is related to Sparse Fieldsets (fields query param):
Sparse Fieldsets
[...]
If a client requests a restricted set of fields for a given resource type, an endpoint MUST NOT include additional fields in resource objects of that type in its response.
https://jsonapi.org/format/#fetching-sparse-fieldsets
Even so this is not forbidden by spec I would not recommend that approach. Returning only a subset of fields makes consuming your API much harder as you have to consult the documentation in order to get a list of all supported fields. It's much more within the meaning of the spec to let the client decide which information (and related resources) should be included.
The "attributes" object of a JSON-API doc does not need to be a complete representation:
attributes: an attributes object representing some of the resource’s data.
You can provide a "self" link to get the full representation, or perhaps even a "body" link to get just the body:
links: a links object containing links related to the resource.
E.g.
{
"data": [
{
"type": "articles_snubs",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!"
},
"links": {
"self": "/articles/1",
"body": "/articles/1/body"
}
},
{
"type": "article_snubs",
"id": "2",
"attributes": {
"title": "Some second thing here"
},
"links": {
"self": "/articles/2",
"body": "/articles/2/body"
}
}
]
}
Related
Wondering if I can create a "dynamic mapping" within an elasticsearch index. The problem I am trying to solve is the following: I have a schema that has an attribute that contains an object that can differ greatly between records. I would like to mirror this data within elasticsearch if possible but believe that automatic mapping may get in the way.
Imagine a scenario where I have a schema like the following:
{
name: string
origin: string
payload: object // can be of any type / schema
}
Is it possible to create a mapping that supports this? I do not need to query the records by this payload attribute, but it would be great if I can.
Note that I have checked the documentation but am confused on if what elastic calls dynamic mapping is what I am looking for.
It's certainly possible to specify which queryable fields you expect the payload to contain and what those fields' mappings should be.
Let's say each doc will include the fields payload.livemode and payload.created_at. If these are the only two fields you'll want to perform queries on, and you'd like to disable dynamic, index-time mappings autogenerated by Elasticsearch for the rest of the fields, you can use dynamic templates like so:
PUT my-payload-index
{
"mappings": {
"dynamic_templates": [
{
"variable_payload": {
"path_match": "payload",
"mapping": {
"type": "object",
"dynamic": false,
"properties": {
"created_at": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
"livemode": {
"type": "boolean"
}
}
}
}
}
],
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"origin": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
Then, as you ingest your docs:
POST my-payload-index/_doc
{
"name": "abc",
"origin": "web.dev",
"payload": {
"created_at": "2021-04-05 08:00:00",
"livemode": false,
"abc":"def"
}
}
POST my-payload-index/_doc
{
"name": "abc",
"origin": "web.dev",
"payload": {
"created_at": "2021-04-05 08:00:00",
"livemode": true,
"modified_at": "2021-04-05 09:00:00"
}
}
and verify with
GET my-payload-index/_mapping
no new mappings will be generated for the fields payload.abc nor payload.modified_at.
Not only that — the new fields will also be ignored, as per the documentation:
These fields will not be indexed or searchable, but will still appear in the _source field of returned hits.
Side note: if fields are neither stored nor searchable, they're effectively the opposite of enabled.
The Big Picture
Working with variable contents of a single, top-level object is quite standard. Take for instance the stripe event object — each event has an id, an api_version and a few other shared params. Then there's the data object that's analogous to your payload field.
Now, all is fine, until you need to aggregate on the contents of your payload. See, since the content is variable, so are the data paths / accessors. But wildcards in aggregation paths don't work in Elasticsearch. Scripts do but are onerous to maintain.
Back to stripe. They partially solved it through what they call polymorphic, typed hashes — as discussed in their blog on API design:
A pretty neat approach that's worth emulating.
P.S. I discuss dynamic templates in more detail in the chapter "Mapping Automation" of my ES Handbook.
Before implementation I'm considering the structure of JSON response that produces REST API I'm working on. I went through many Q/A here on SO, read many articles, recommendations and pseudo standards.
Requirements
Inform client about some useful meta information - HTTP status code etc.
Paging and filtering information - offset, limit and filtering queries (API client is aware of all parameters that influenced the result).
Information about data collection - total records count in collection and number of returned items. API client is then able create pagination.
Links to previous and next pages (just considering, not sure if this is useable for API clients but many REST APIs links section use - e.g. PayPal)
Response
This is my first draft of structure of returning search results data:
{
"meta": {
"status_code": 200,
"success": true,
"server_time": "2017-06-29T15:24:40+0200"
},
"request": {
"offset": 5,
"limit": 5,
"query": [
"foo",
"bar"
]
},
"response": {
"count": 5,
"total_count": 754,
"data": [
{
"id": "88b60cc6-70bc-4b1a-8f26-c919355d47d3",
"name": "Name of entity 1"
},
{
"id": "2f4ccda5-11bc-4ef7-b663-30c506f5118c",
"name": "Name of entity 2"
},
{
"id": "1333f2fe-a958-474e-9a82-8b343fda3aff",
"name": "Name of entity 3"
},
{
"id": "f5187143-f3b8-412b-a416-1e3a5830baee",
"name": "Name of entity 4"
},
{
"id": "2dd17750-bbdf-460a-abec-1f74e1170726",
"name": "Name of entity 5"
}
]
},
"links": {
"previous": {
"href": "http:\/\/api.example.com\/envelopes?offset=0&limit=5",
"rel": "previous",
"method": "GET"
},
"self": {
"href": "http:\/\/api.example.com\/envelopes?offset=5&limit=5",
"rel": "self",
"method": "GET"
},
"next": {
"href": "http:\/\/api.example.com\/envelopes?offset=10&limit=5",
"rel": "next",
"method": "GET"
}
}
}
I would like to avoid an "opinion question" to discuss the most appropriate JSON structure. I saw many opinions about envelopes in response, some services / standards it recommends, some not.
Questions:
Is it good idea to return the result in this structure?
Do you see some problems with this structure? What to do better?
Do you see some missing values that are needed for API client? Some unnecessary values?
Is needed return URL to self?
Opinion questions are hard, but I'll try.
First of all, your question should not be addressed to community, but to client itself. Nothing clears assumptions about missing/necessary values better than such feedback.
The structure itself is good enough, at least as a draft. When designing responses you need to remember that you are basically locking yourself up, since clients don't like fundamental changes in API. Only lot of incremental "please add just one more field here". You are doing good job in thinking far enough, about meta fields, pagination and separating actual response, but don't think you can predict everything. You won't. Maybe look for something like HAL or JSON Collection. At least as an inspiration.
In the end design of API is evolutionary and mostly client driven process. So talk to your client.
If I have some arbitrary JSON how can I do deep sets and gets on the nested properties using a slice of map keys and/or slice indexes?
For example, in the following excerpt from the JSON API example:
{
"data": [{
"type": "posts",
"id": "1",
"title": "JSON API paints my bikeshed!",
"links": {
"self": "http://example.com/posts/1",
"author": {
"self": "http://example.com/posts/1/links/author",
"related": "http://example.com/posts/1/author",
"linkage": { "type": "people", "id": "9" }
}
}
}]
}
I'd like to get the string "9" located at data.0.links.author.linkage.id using something like:
[]interface{}{"data",0,"links","author","linkage","id"}
I know the ideal way to do this is to create nested structs that map to the JSON object which I do for production code, but sometimes I need to do some quick testing which would be nice to do in Go as well.
You have stretchr/objx that provide a similar approach.
Example use:
document, _ := objx.FromJSON(json)
document.Get("path.to.field[0].you.want").Str()
However, unless you really don't know at all the structure of your JSON input ahead of time, this isn't the preferred way to go in golang…
I`m very new to JS/APIs/JSON.
I`ve got an API I built using deployd, a great tool that allows to to quickly set up an API.
My API called "app" has two resources zips and people. Each entry/object in zips has only one property zipcode. Each people has the properties name and phonenumber.
I want to figure out a way to have people be associated with various zipcodes. Either one or many if necessary. This would, as I understand, involve adding a property to people such as assoczipcodes and using a relationship model to indicate with zipcodes are related.
I've done some research here about how a relationship is structured but I simply don't understand the syntax and format.
Q: What datatype is correct for associzipcodes?
Q: What do I enter into each assoczipcodes to indicate which zips are related?
Cristik's answer appears to be outdated. According to the current documentation, you'd want to nest the relationship like this:
{
"type": "people",
"id": "1",
"attributes": {
"name": "John Doe"
},
"relationships": {
"zipcodes": {
"links": {
"self": "http://myserver.com/people/1/zipcodes"
}
}
}
}
Based on the contents from the Resource Relationships section, the simplest approach would be to return an URL for the relationship, and have that URL return all information needed:
{
"type": "people",
"id": "1",
"name": "John Doe",
"links": {
"zipcodes": "http://myserver.com/people/1/zipcodes"
}
}
I have a REST service that returns a list of objects. Each object contains objectcode and objectname.
This is my first time building a REST service, so I'm not sure how to format the response.
Should it be:
{
"objects": {
"count": 2,
"object": [
{
"objectcode": "1",
"objectname": "foo"
},
{
"objectcode": "2",
"objectname": "bar"
},
...more objects
]
}
}
OR
[
{
"objectcode": "1",
"objectname": "foo"
},
{
"objectcode": "2",
"objectname": "bar"
},
...more objects
]
I realize this might be a little subjective, but which would be easier to consume? I would also need to support XML formatted response later.
They are the same to consume, as a library handles both just fine. The first one has an advantage over the second though: You will be able to expand the response to include other information additional to the objects (for example, categories) without breaking existing code.
Something like
{
"objects": {
"count": 2,
"object": [
{
"objectcode": "1",
"objectname": "foo"
},
{
"objectcode": "2",
"objectname": "bar"
},
...more objects
]
}
"categories": {
"count": 2,
"category" : [
{ "name": "some category"}
]
}
}
Additionally, the json shouldn't be formatted in any way, so remove whitespace, linebreaks etc. Also, the count isn't really necessary, as it will be saved while parsing the objects themselves.
I often see the first one. Sometimes it's easier to manipulate data to have meta-data. For exemple google API use first one : http://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&sensor=true
It's not only the question of personal preference; it's also the question fo your requirements. For example, if I was in the same situation and I did need object count on client side then I'd go with first approach otherwise I will choose the second one.
Also please note that "classic" REST server mostly will work a bit different way. If some REST function is to return a list of objects then it should return only a list of URLs to those objects. The URLs should be pointing to details endpoints - so by querying each endpoint you may get details on specific single object.
As a client I would prefer the second format. If the first format only includes the number of "objects", this is redundant information.