Related
According to the HAL standard (see here and here) the links to other resources should be placed in a specific embedded section.
So for instance this is not valid HAL, is my understanding correct?
{
"movies": [
{
"id": "123",
"title": "Movie title 1",
"_links": {
"subtitles": {
"href": "/movies/123/subtitles"
}
}
},{
"id": "456",
"title": "Movie title 2",
"_links": {
"subtitles": {
"href": "/movies/456/subtitles"
}
}
}
],
"_links": {
"self": {
"href": "/movies"
}
}
}
The reason for which the above JSON is not valid HAL is that the links should be placed in an embedded section ("_embedded") that links to the ID in the main body. So the right approach would be:
{
"movies": [
{
"id": "123",
"title": "Movie title 1",
},{
"id": "456",
"title": "Movie title 2",
}
],
"_embedded": {
"movies": [
{
"id": "123",
"_links": {
"href": "movies/123/subtitles"
}
},
{
"id": "456",
"_links": {
"href": "movies/456/subtitles"
}
}
]
}
"_links": {
"self": {
"href": "/movies"
}
}
}
Is all the above correct?
Thanks
Gonna use this as a case study on re desiging with hal. #darrel millers answer is good, but not great and has some things i think should be cleared up. this is gonna be LONG.
The big question is what is the context...IE what is this resource you are returning. All you've got is something that relates to movies somehow. Like suppose this is a search result...having a movie relationship is the wrong approach..as the movie relates to the search result "top level resource" as an item. so it should be something more like
{
title : "Results for search XXXXX"
_links : {
self : { href: "https://host.com/search/with/params/XXXXX"},
item : [
{ href : "https://host.com/url/to/movie/result/one", title : "A great Movie"},
{ href : "https://host.com/url/to/movie/result/two", title : "A Terrible Movie"},
]
}
}
but this structure would be expensive for a client to construct a UI around as it would have to make 3 calls..following the N+1 rule (1 for the result set..then N for each result) so thus was born _embedded which is just hal implementation of the hypertext pre fetch pattern (in http2 the server could actually send each result as it's own document and the client's cache would be pre-filled with those results and you wouldn't necessarily need _embedded). That structure looks more like this:
{
title : "Results for search XXXXX"
_links : {
self : { href: "https://host.com/search/with/params/XXXXX"},
item : [
{ href : "https://host.com/url/to/movie/result/one", title : "A great Movie"},
{ href : "https://host.com/url/to/movie/result/two", title : "A Terrible Movie"},
]
},
_embedded : {
item : [
{
_links : {
profile : {href : "https://host.com/result-movie"},
canonical : {href : "https://host.com/url/to/movie/result/one"}
},
title : "a great movie",
rating : "PG",
},
{
_links : {
profile : {href : "https://host.com/result-movie"},
canonical : {href : "https://host.com/url/to/movie/result/two"}
},
title : "a terrbile movie"
rating : "G",
}
]
}
}
That's pretty great 1 http request to get 3 resources. 1 request not N+1. Thank you HAL!
So why item? well does the search result ONLY EVER contain movies...that's very unlikely..and even if it does today...do you want it to only ever contain movies tomorrow...that's pretty narrow and this structure is a contract that you have to maintain basically forever. But your UI really wants to show the result as a movie. That what the profile link i've added is for...the client uses the profile link to know what the resource it's currently processing is..and what fields it can use to build up a UI. A decent client when processing a collection displays what profiles it can..and just ignores what ones it can't (logging a warning maybe). It up to the client dev to upgrade their app to support new profiles...don't believe me? think about how a web browser parses tags it doesn't understand in html...put a <thing-not-invented-yet></think-not-invented-yet> in your html doc and see how a good client works.
another thing you should notice is i do NOT use self links but canonical..i've changed my stance on this over the years. as of late i default to canonical and i ONLY use self when I'm maintaining versions of the target resource and it's important to have the embedded object link to the specific version that was embedded. this is very rare in my experience. I tell clients to follow self it it's present, ortherwise follow canonical when navigating to the embedded item. this gives the server complete control on where it wants to take the client. The top level resource, in this case a result should still have a self...and in this case it makes sense as soem random search probably does not have a canonical link...unless it's a VERY common keyword search...then it probably should as other users could use the same url.
Let take a moment to talk about why item as the rel...cause this is really important. Since it's a search result..why not have the rel be result. There's a really easy answer..result is not a member of the IANA link registry https://www.iana.org/assignments/link-relations/link-relations.xhtml and therefore result is completely invalid...now you could "namespace" your extension rel with my:result or our:result (the "namespace" is up to you, these are just example) but why bother with that if a perfectly good one already exists in the IANA registry..and it does item.
Let's talk about items vs item (or x:movies vs x:movie) . Well items isn't in IANA either..so it'd have to be x:items but instead of doing that let's think about why. If our result doc was represented in HTML it'd be looking like this (ignore my missing body head etc not well-formedness for brevity):
<html>
<title>Results for search XXXXX</title>
<a rel="item" href="https://host.com/url/to/movie/result/one" >A Great Movie</a>
<a rel="item" href="https://host.com/url/to/movie/result/two" >A Great Movie</a>
</html>
This is the SAME resource as the first example (without embedding sub resources). Just represented as text/html and not application/hal+json. If i've lost you here (this is where most people get REALLY confused, the best i can offer is to watch my talk on this at https://www.youtube.com/watch?v=u_pZBBELeEQ ) Hear it's clear the appropriate relationship of each target resource is a SINGLE ITEM and not a set of ITEMS. each link targets one item (or one, singular movie).
There's a trap with HAL to treat it like JSON and that leads to statements like the one the comments that movies is machine readable or better. Let me explain how this comes about by continuing with this HTML representation in a use case.
When a client parses this document looking for item links it must parse EVERY a tag and filter down to only those where rel="item" attribute is present. That's a "full table scan"..and how do we get away from those? we create an index. JSON has the concept of an index built into it's structure. It's a key with an array value. index : [ {entry 1}, {entry 2} ]. The author of HAL knew the most common way to retrieve links (in _links or the prefetched ones in _embedded) would be by relationship..so he structured his spec such that rel is indexed. so when you see:
_links : {
self : { href: "https://host.com/search/with/params/XXXXX"},
item : [
{ href : "https://host.com/url/to/movie/result/one", title : "A great Movie"},
{ href : "https://host.com/url/to/movie/result/two", title : "A Terrible Movie"},
]
},
know that it is REALLY
_links : {
self : { rel: "self", href: "https://host.com/search/with/params/XXXXX"},
item : [
{ rel:"item", href : "https://host.com/url/to/movie/result/one", title : "A great Movie"},
{ rel:"item", href : "https://host.com/url/to/movie/result/two", title : "A Terrible Movie"},
]
},
because the rel is an attribute of the LINK OBJECT and NOT THE RESOURCE. but bytes over http are expensive (gzip would get rid of this) and devs don't like redundancies (a whole other topic) so when we have hal we OMIT the rel attribute since the HAL structure already makes the rel apparent. though it's not really apparent when your parser encounters just this:
{ href : "https://host.com/url/to/movie/result/one", title : "A great Movie"}
what's the rel? you have to pass that in from the parent node..that's always been ugly...anyways all this is to show that redundancy is eliminated in HAL generally. once this redundancy is eliminated it's tempting to change that index key to the plural form items but know that would mean you are saying your link (once redundancies are PUT BACK) would be {rel: "items", href : "https://host.com/url/to/movie/result/one", title : "A great Movie"} and that is clearly wrong..that link is not to many items...just one.
So removing redundancy in this case probably wasn't the best..but it's evil with benefits and HAL follows that pattern for _links and _embedded and that's what we're gonna do with our search result..given that ALL the item links have no been pre-fetched and are present as _embedded it's unimportant to keep them in _links. and as such it should look like this:
{
title : "Results for search XXXXX"
_links : {
self : { href: "https://host.com/search/with/params/XXXXX"}
},
_embedded : {
item : [
{
_links : {
profile : {href : "https://host.com/result-movie"},
canonical : {href : "https://host.com/url/to/movie/result/one"}
},
title : "a great movie",
rating : "PG",
},
{
_links : {
profile : {href : "https://host.com/result-movie"},
canonical : {href : "https://host.com/url/to/movie/result/two"}
},
title : "a terrbile movie"
rating : "G",
}
]
}
}
Now we have a pretty good search result that includes 2 movies (and can include more things in the future without breaking the contract). Note: if you ever went live with JUST _links and no _embedded...you can NOT remove the _links as some client out there is depending on them being present..so it's best to think of this stuff early...thought a well behaving client should always check _embedded before _links when using the HAL representation of a resource...so it's really up to you to know if all your clients are well behaving.
Ok so let's move to a case where x:movie is the correct relationship..that probably would be good if the top level resource is an actor. so something like:
{
Name : "Paul Bettany"
_links : {
canonical : { href: "https://host.com/paul-bettany"},
"x:movie" : [
{ href : "https://host.com/url/to/movie/result/one", title : "A great Movie"},
{ href : "https://host.com/url/to/movie/result/two", title : "A Terrible Movie"},
],
"x:spouse" : { href: "", title: "Jennifer Connely"}
},
_embedded : {
"x:movie" : [
{
_links : {
profile : {href : "https://host.com/result-movie"},
canonical : {href : "https://host.com/url/to/movie/result/one"}
},
title : "a great movie",
rating : "PG",
},
{
_links : {
profile : {href : "https://host.com/result-movie"},
canonical : {href : "https://host.com/url/to/movie/result/two"}
},
title : "a terrbile movie"
rating : "G",
}
]
}
}
Notes: i used canoncial instead of self at the top level because an actor is long-lived resource..that actor will always exist..and an actor is not versioned. For completeness i left both x:movie in _links and _embedded, however in practice i would NOT have the ones in _item. I also kept them in _links to show the reasons to have x:movie is so that you can differentiate it from x:spouse (that semantic differentiation did NOT make sense in the search result case we started with). Finally it's useful to note that i embedded x:movie but NOT x:spouse this is just to illustrate that it is not an either / or thing. you can pre-fetch/embed the link you need for your use case. In fact i often embed things based on the identity of the client..ie i know iOS can display something that android can not.
Those notes aside, the reason i went here is that i wanted to make it clear that you do NOT and SHOULD NOT have that movies: data field that you have...just rely on the movie data in _embedded. You said soemthign like matching up the values in the movies to teh ones in _links or _embedded...you should NOT be doing that..that doesn't make any sense. a movie is a resource...use the linked resource of a movie not some data field. You need to decide early on what is a resource and what is a piece of data. my best tip is if a thing has link relationships..then it's a resource. In my talk i go into MUCH MORE DETAIL on this with broader terms (hypermedia controls) that i don't want to get into here yet.
A final note..in hypermedia applications you KNOW you are doing something wrong if you are exposing internal id fields..as you have done here. That should be a huge red flag that something is wrong. The use case for the id's you described was to match up the data field movies with the _embedded x:movie. As stated...you should NOT be doing that..and the presence of an id field should key you in to that bad practice.
I was asked to answer here..so i hope this helps.
The "_Links" property must go in the root of a resource object. That resource object might be at the root, or it could be in the _embedded object.
I suspect some of the confusing is coming from having an "_embedded" key point to an array. This is only done when you want to represent multiple instances of a related resource.
In the example the key is movies which infers you are embedding a resource object which represents multiple movies. However, the array indicates there are multiple embedded resource objects. Each resource object is one movie.
By changing the name of the key to "movie" you get this:
{
"movies": [
{
"id": "123",
"title": "Movie title 1",
},{
"id": "456",
"title": "Movie title 2",
}
],
"_embedded": {
"movie": [
{
"id": "123",
"_links": {
"href": "movies/123/subtitles"
}
},
{
"id": "456",
"_links": {
"href": "movies/456/subtitles"
}
}
]
}
"_links": {
"self": "https://example.org/movielist"
}
}
So, now you have a representation of a "movie-list" resource object and you have embedded a bunch of "movie" resource objects for each item in the "movie-list". Each resource object pointed to by "movie" has a "_links" property to related information. I'm assuming that "subtitles" link was supposed to be a self link.
As observable in the specs you posted, you can have links and/or embedded resources:
A resource's links should sit as a property of that resource:
{
"movies": [
{
"id": "123",
"title": "Movie title 1",
"_links": {
"subtitles": {
"href": "/movies/123/subtitles"
}
}
}, {
"id": "456",
"title": "Movie title 2",
"_links": {
"subtitles": {
"href": "/movies/456/subtitles"
}
}
}
]
}
The alternative would be to directly embed the movie's subtitles resource:
{
"movies" : [
{
"id" : "123",
"title" : "Movie title 1",
"_embedded" : {
"subtitles" : [{
"name" : "movie 1 subtitles"
}
]
}
}, {
"id" : "456",
"title" : "Movie title 2",
"_embedded" : {
"subtitles" : [{
"name" : "movie 2 subtitles"
}
]
}
}
]
}
I am working on WebAPI on ExpressJS. I am fetching data from MySQL database by using node module mysql. I have a mater and a detail table let say
Countries(ID, Name, Code)
States (ID, Name, CountryId)
Now I want to query the data in JSON as following:
{
"country":{
"ID" : 1,
"Name" : "United States of America"
"Code" : "USA"
"Cities" :
[
{
"1",
"Alabama",
"AL"
},
{
"2",
"Alaska",
"AK"
},
{
"3",
"California",
"CA"
}
]
}
}
How can I query data in the above format?
Thanks and Regards.
How can I query data in the above format?
You can't because this is not a valid JSON. No matter what you do, you will never be able to represent your data in that format.
To represent data coming from a relational database as a JavaScript nested objects and arrays, it will be easiest to use an ORM like Sequelize, Bookshelf, ORM2 or Waterline. See:
http://docs.sequelizejs.com/
http://bookshelfjs.org/
https://github.com/dresende/node-orm2
http://sailsjs.com/documentation/reference/waterline-orm
ORM is an Object-Relational Mapper and lets you map the relational model to objects, which is exactly what you're trying to do here.
I have the following hierarchical JSON.
How should I model it so that it can be represented as Tree.
The products array is at the same level as bookFamilies.
Please can anyone provide some help.
I followed this fiddle.sencha
https://fiddle.sencha.com/#fiddle/9vi
It does not work with 4.2. And I don't know how to restructure to bring the products.
{
"Families" : [
{
"FamilyName" : "FICAsia",
"bookFamilies" : [
{
"name" : "Ndf\/Nds Nja",
"book" : ["3782"]
}
],
"products" : ["CorpZC", "CorpInfl", "CorpFix", "CorpFlt"]
},
{
"FamilyName" : "FICUStandard",
"bookFamilies" : [{
"name" : "Local Markets Nja",
"book" : ["3787", "3785"]
}, {
"name" : "Ndf\/Nds Pwja",
"book" : ["3782"]
}
],
"products" : ["Drawdown", "Xorward", "FXY", "wap", "Top"]
}
]
}
I was thinking to restructure the JSON but not sure how to do with the products so that it can be represented as tree.
By default, for each tree node:
"text" is what will be displayed in the tree. This can be changed to a different field by editing the displayField property of the tree panel (as done in the fiddle you linked). However, it doesn't look like this feature is in 4.2.
"leaf" says whether the item can be expanded (default is true). You want to make sure this is set on the children, otherwise you will get strange behavior when trying to open them.
"expanded" says whether the item/parent is expanded (default is false). If you set this to true on a leaf node, nothing will happen.
"children" opens up the recursive hierarchy for the rest of the relationships.
Some tricky things about the fiddle you linked:
The transform function takes the incoming request and parses into a javascript object and is the "data" argument. This is handy when you need to make some transformations to the data. In this case, you see they take the "items" key and turn it into "children."
The "rootProperty" is a private variable in ExtJS 4.2, but a public one in ExtJS >5
Make sure to look at the docs to see the other features in ExtJS 4.2. Be careful about ExtJS 5,6 documentation because it looks like quite a few things were added/removed.
http://docs.sencha.com/extjs/4.2.2
Here's a partial example json of what you listed earlier. Replace the "data" property with this, remove the "rootProperty" and "transform" properties, remove the "displayField" property, and set the version to ExtJS 4.2.1.
{
"children":[
{
"text":"FICAsia",
"children":[
{
"text":"Book Families",
"children":[
{
"text":"Ndf\/Nds Nja",
"children":[
{
"text":"3782",
"leaf":"true"
}
]
}
]
},
{
"text":"Products",
"children":[
{
"text":"CorpZC",
"leaf":"true"
},
{
"text":"CorpInfl",
"leaf":"true"
},
{
"text":"CorpFix",
"leaf":"true"
},
{
"text":"CorpFlt",
"leaf":"true"
}
]
}
]
}
]
}
I am planning to move some of my RDBMS tables to a NoSQL environment. I have different modules and associated tables as shown below.
1. General
News table
Trainings table
Knowledge table
2. Application_x
Clicks table
Crashes table
Note : This is a sample scenario
In this case, this is how i defined Nosql (Couchbase server) structure. Please correct me if this is wrong.
The purpose of migration is mainly intended for searching.
{
"General":[
"news" : [{
"id" : "123",
"title" : "test title"
},
{
"id" : "345",
"title" : "test title 2"
}],
"trainings" :[{
"id" : "1",
"name" : "training 1"
},
{
"id" : "2",
"name" : "training 2"
}],
"knowledge" :[{
"id" : "1",
"categ" : "programming"
},
{
"id" : "2",
"categ" : "management"
}]
],
"Application_x": [
"clicks" : [{
"userid" : "1",
"area" : "1850",
},
{
"userid" : "2",
"area" : "258",
}],
"crashes" :[{
"userid" : "1",
"severity" : "1",
},
{
"userid" : "2",
"severity" : "8",
}]
]
}
Can someone correct me if my approach is not correct ?
Thanks in advance,
Tismon Varghese.
By reading your question, I am left scratching my head as to what your approach is. In a NoSQL database (such as couchbase) - you don't have the idea of tables and columns. Each object is serialized to JSON and stored in plain text. Yes, this creates duplication, but the drawbacks of duplication are greatly outweighed by the benefits of scalability.
In this example, using Couchbase, you would probably want to create one Couchbase bucket per application. That way, should you need to migrate to a different Couchbase cluster at a later date, a minimal amount of configuration is required. Each row in your tables gets created as a separate object in Couchbase. There is no need to separate the object types within the bucket.
This blog entry contains some detailed instructions on how to migrate from mySQL to Couchbase.
On a side note, I might recommend using Couchbase combined with Elasticsearch.
Is there a way to update a complex MongoDB document from C# using JSON? For example, suppose I have the following document:
{
"name": "John Smith",
"age": 35,
"readingList":
[{
"title": "Title1",
"ISBN": 6246246426724,
"author":
{
"name": "James Johnson",
"age": 40
}
},
{
"title": "Title2",
"ISBN": 3513531513551,
"author":
{
"name": "Sam Hill",
"age": 20
}
}]
}
Now I want to update the age of the second book's author (Sam Hill) from 20 to 21. Suppose I have the following JSON representation:
{
"readingList":
[
{
"title": "Title2",
"author":
{
"age": 21
}
}]
}
Basically the second JSON string is like the first one, minus all the fields and array elements that don't change, except for one field in any array being looked at that uniquely identifies that index. In this case, the "age" field is included since it is being updated with the given value. The "title" field is given to locate the right array element while searching for the field to update. There may also be even more subdocuments and arrays to go through, and the format is not static (it may change at a later time). This is just a simplified example.
Is it possible to pass in something like this to some function and update the correct field that way? Is there something at least similar to this, so I can just pass in some JSON to do the update?
The reason I am looking to do it this way, rather than through simpler means, is because I want to keep track of a history of changes to documents, and if I want to backtrack to an earlier version, I want an easy way to do so that can handle this level of complexity.
UPDATE:
I have some clarifications to make. In this particular scenario I have no way to predict what kinds of changes would need to be made. A change could be made to any field at any time, and that field could be anywhere in the document, possibly at the top level, or within multiple nested subdocuments/arrays. The data we're dealing with is for a separate party that may use it and modify it at will, so we have no control over what they choose to do with it. In addition, there is no fixed schema. The other party could add new fields, including new subdocuments or arrays, or delete them.
The reason I'm asking this question is because I would like to store a history of changes to documents in such a way that I could revert to an older snapshot of the document by applying the changes in reverse. In this case, changing the age from to 20 to 21 would revert the document to an older state (assuming that someone messed with the age beforehand and made it 20, and I wanted to fix it back to 21). Since somebody could make any change they wanted to the system, including to the underlying structure of the data itself, I can't just come up with my own schema, or hardcode a solution that changes specific fields using this specific schema.
In this example, the change in age from 20 to 21 would be from a record in the history whose structure I couldn't predict beforehand. So I am looking for an efficient solution to apply an unpredictable update to a document given a simplified JSON representation of the change to be made.
I am also open to alternatives that don't involve JSON if they are fairly efficient. I brought up JSON because I figured that, given MongoDB's usage of JSON to structure documents, it would make the most sense, and perhaps be superior to something like string manipulation. Another alternative I considered would involve storing the change using some kind of custom dot notation, like this: readingList[ISBN:3513531513551].author.age=21"
This would require me to create a custom function to interpret the string and turn it into something useful though, so it doesn't sound like the best solution.
Hi friend I used below JSON document
{
"_id" : ObjectId("56a99c121f25cc3a3c709151"),
"name" : "John Smith",
"age" : 35,
"readingList" : [
{
"title" : "Title1",
"ISBN" : NumberLong(6246246426724),
"author" : {
"name" : "James Johnson",
"age" : 40
}
},
{
"title" : "Title2",
"ISBN" : NumberLong(3513531513551),
"author" : {
"name" : "Sam Hill",
"age" : "25"
}
}
]
}
I just used condition as author name is Sam Hill and execute below query in C# and its work.
IMongoQuery query = Query.And(Query.EQ("name", "John Smith"), Query.EQ("readingList.author.name", "Sam Hill"));
var result =collection.Update(query,
MongoDB.Driver.Builders.Update.Set("readingList.$.author.age", "21"));
you can query your main document let's assume your main collection is named "books" this is the structure:
{
"id":"123",
"name": "John Smith",
"age": 35,
"readingList":
[{
"title": "Title1",
"ISBN": 6246246426724,
"author":
{
"name": "James Johnson",
"age": 40
}
},
{
"title": "Title2",
"ISBN": 3513531513551,
"author":
{
"name": "Sam Hill",
"age": 20
}
}]
}
// you need a query that returns the main document by id for example, when you have the main document you can query at the one you want to modify in the list and assing it to a varibale let's say readItem, then do the modifications you need and after that you can update only the fields you need using set and onle the element in the array using "$" something like:
readItem.title = "some new title";
readItem.age++;
var update = MongoDB.Driver.Builders.Update.Set("readingList.$", BsonDocumentWrapper.Create(readItem));
Update<Book>(query, update);
Actually I would not advise you to choose this kind of data model because in my experience it will get pretty messy. Still, you might have some very specific requirements which might force you to have this and only this data model.
I would create two collections: persons and readinglists.
persons would look like:
{
"id":"123",
"name": "John Smith",
"age": 35
}
and readinglists would look like (note that it has a compound natural id):
{
"_id": { "personid":"123", "title": "Title1"},
"ISBN": 6246246426724,
"author":
{
"name": "James Johnson",
"age": 40
}
}
Then you can easily update the readinglist:
var query = Query.EQ("_id", new BsonDocument(new BsonElement[]{ new BsonElement("personid":"123"), BsonElement("title":"Title1")}));
readingListCollection.Update(query, Update.Set("author.age": 22));
In your data mode you need to know the array index of the second document. It is better to model readingList attribute as a map. In following example I used isbn as a map key:
{
"id":"123",
"name":"John Smith",
"age":35,
"readingList":{
"6246246426724":{
"title":"Title1",
"ISBN":6246246426724,
"author":{
"name":"James Johnson",
"age":40
}
},
"3513531513551":{
"title":"Title2",
"ISBN":3513531513551,
"author":{
"name":"Sam Hill",
"age":20
}
}
}
}
In this data model you can access second book directly. For instance by dot notation:
db.authors.update(
{ item: "123" },
{ $set: { "readingList.3513531513551.author.age": 22 } }
)
Unfortunately I do know C# notation for that but should be straight forward.