JSON Best Practices: GUIDS for contained data - json

I have a customer that uses JSON to represent objects. For example:
var person = {
fname: "John",
lname: "Doe",
nicknames: ["jake", "kevin"]
}
For the purposes of editing these entries, we would like to include GUIDS for each object. A method of embedding the GUIDs has not yet been established. I am currently thinking of doing something like this:
var person = {
_guids: {
fname: "XXX",
lname: "XXX",
_nicknames: "XXX", /* This GUID corresponds to the nickname container */
nicknames: ["jake-guid", "kevin-guid"],
}
fname: "John",
lname: "Doe",
nicknames: ["jake", "kevin"]
}
If each item were to be its own object, it would become exceedingly messy, and would prohibit a clean exchange in cases where the GUIDs are not necessary. However, this approach also leads to the question of how to deal with something like this:
var person = {
_guids: {
fname: "XXX",
lname: "XXX",
sacks_of_marbles: ["container-guid",
"first-sacks-guid", ["XXX", "XXX"],
"second-sacks-guid", ["XXX", "XXX"]]
}
fname: "John",
lname: "Doe",
sacks_of_marbles: [["red", "blue-with-swirls"], ["green", "pink"]]
}
Any recommendations for how to maintain cleanliness, lack of verbosity, as well as the ability to include GUIDs?

In my opinion, the cleanest "identification strategy" is, firstly to include only relevant identifiers, and secondly to include those identifiers within the scope of the object.
In your case, I'd be genuinely shocked to learn that fname and lname require identifiers of their own. They're properties or attributes of an single person object or record. The person will likely have an ID, and the relevant fname and lname values should be easily identified by the person context:
var person = {
id: <guid>,
fname: "Robert",
lname: "Marley",
nicknames: ["Bob Marley"]
};
Or, if the nicknames attribute is actually a list of references, maybe this:
var person = {
id: <guid>,
fname: "Robert",
lname: "Marley",
nicknames: [{id:<guid>,nickname:"Bob Marley"}]
};
If, for some peculiar reason, each name is its own semi-independent record with attributes and identifiers, treat it as a fully qualified object:
var person = {
id: <guid>,
fname: {id:<guid>, name:"Robert"},
lname: {id:<guid>, name:"Marley"},
nicknames: [ {id:<guid>,nickname:"Bob Marley"} ]
};
And while the 3rd example here isn't terribly confusing, it's probably far more verbose than is necessary. I'd highly recommend shooting for the first example and letting context do it's job.

You're not going to get cleanliness or lack of verbosity if you're including GUIDs for everything! That said, one approach would be to turn your property values into hashes like
var person = {
fname: {
value: "John",
guid: "..."
},
lname: {
value: "Doe",
guid: "..."
}
}
This has the advantage that you can nest the GUIDs next to the corresponding values, arbitrarily deeply. However, this will also add some overhead to each access to the object's values (since person.fname will have to become person.fname.value, and so on).

After reviewing the proposals here as well as the use-case, the following solution was adopted, and seems to be working reasonably well.
var person_with_guids = {
obj:{
type: "person",
fname: "fname=guid",
lname: "lname=guid",
nicknames: ["larry-guid", "curly=guid", "moe-guid"],
father: "nick-nolte-guid",
},
refs:{
fname-guid:{
display: "Johnny",
edit: "johnny"
},
lname-guid:{
display: "Walker",
edit: "walker"
},
larry-guid:{
type: "person",
...
}
}
}
var person_without_guids = {
fname: "Johnny",
lname: "Walker",
nicknames: ["Larry", "Curly", "Moe"],
father: {fname: "Nick", lname: "Nolte"}
}
Without getting too far into the specifics of the choice, the listed scheme has the advantage of permitting the guid version to be translated by machine into the sans-guid version. There will be times when a human-readable lightweight JSON will be required, and other times when even the subtle difference between the display-text and the edit-text is required.
This same sort of thing could be accomplished by using schemes proposed above making everything into an object and embedding the guid within it. Doing so, however, causes the guids to be included in an already nested system making it even more difficult to interpret visually. To maintain a more flattened representation, a lookup table methodology was selected.
My thanks to all who responded.

Related

MongoDB, change array of objects to an array of strings containing their ObjectId

I have an database full of objects that contain information, as well as other arrays of objects. I would like to change the inner arrays to only be arrays with each index as an ObjectId type with their respective ObjectId
I am using the mongoose populate function to retrieve this information later in the program. So only the ObjectId is needed for the reference.
job {
_id: 1,
name: "name",
parts: [
{ _id: ObjectId("5c790ce7d3dc8d00ccc2688c"), name: "name"},
{ _id: ObjectId("5c790ce7d3dc8d00ccc2688b"), name: "name"},
{ _id: ObjectId("5c790ce7d3dc8d00ccc2688a"), name: "name"},
]
}
Desired Result
job {
_id: 1,
name: "name",
parts: [
ObjectId("5c790ce7d3dc8d00ccc2688c"),
ObjectId("5c790ce7d3dc8d00ccc2688b"),
ObjectId("5c790ce7d3dc8d00ccc2688a")
]
}
I tried a few mongoDB queries from the command line but none of them are quite giving the result I need. This one has no errors, but it doesn't seem to change anything.
db.jobs.update(
{},
{
$set: {"parts.$[element]": "element._id"}
},
{
multi: true,
arrayFilters: [{ "element": {} }]
}
)
I'm not sure if this is possible or not using only the mongo shell.
Mongo v4.2 introduced pipelined updates this allows the use of aggregation operators within the update and more importantly updating a document using it's own values. which is what you want to achieve.
db.jobs.updateMany(
{},
[
{
'$set': {
'parts': {
'$map': {
'input': '$parts',
'as': 'part',
'in': '$$part._id'
}
}
}
}
]);
Unfortunately this is not possible for earlier Mongo versions. Specifically $arrayFilters allows you to filter an array but again prior to the new update syntax accessing own values within the update is not possible.
You'll have to iterate over all documents and do the update one by one in code.
As #Tom Slabbaert mentioned in the other answer you will have to use updates with aggregation pipelines available from v4.2 if you want to update the documents in bulk in one operation.
As an alternative to using $map, if you want only a single value at the top level and the value is accessible using the dot notation. You can simply use $set with the dot notation.
db.jobs.updateMany({}, [{
$set: { parts: "$parts._id" }
}])

Correct implementation of MySQL tables in MongoDB?

I'm new to mongoDB, and I wonder how I can translate tables relations in MongoDB.
For example in my MySQL base, I have two tables : Manager and Employee.
Manager {
name: String,
ID: Int Primary Key
}
Employee {
name: String,
ID: Int Primary Key,
Manager: int Foreign key (Manager.ID)
}
I would like to know what is the correct way to implement this with MongoDB.
MongoDB is a non-relational document database.
'Document' is a record in a collection and each document can have any number of fields, including arrays, maps, etc. The strength and weakness of this are that two documents in the same collection may hold completely different fields. Normal forms of relational databases do not apply. Also, each document has a unique id.
How to link documents/collections together is completely up to the app logic.
Few options:
Keep everything together in once collection.
//
{
name: "Jon Appleseed",
manager: "Jonh Bosch"
},
{
name:"Another employee",
manager: "Jonh Bosch"
}
Use two collections as suggested by #iprakashv
Use a mixed approach:
// Employees
{
_id: "_234234",
name: "Jon Appleseed",
manager:{
name:"John Bosh",
id: _12321
}
},
{
_id: "_233234",
name: "Another employee",
manager:{
name:"John Bosh",
id: "_12321"
}
}
// Managers
{
_id: _123213,
name:"John Bosh",
subordinates:
[
{name:"Jon Appleseed", id: "_234234" },
{name:"Another employee", id: "_12321" },
]
}
As you see, you gain much more flexibility at the expense of normalization.
As mongodb is a NoSQL database for schema-less and relation-less collections (though you can put validations over a collection), join-less schema and nested objects make more sense here.
Now as per our requirement, each Manager object is linked to its employee so, should ideally have an array containing all its employee objects, but, it is not practical when we need all the employees irrespective of the Manager.
So, we may want to have just the array of employee id's instead of an array of employee objects and employee collection (table) to be maintained separately to fetch employee objects.
The samples documents for the above schema would look like:
//employee collection document
{
"ID": "E12334",
"name": "John Doe"
}
//manager collection document
{
"ID": "M453",
"name": "Jane Doe",
"employeeIDs": ["E12334", "E12343"]
}

JSON-like serialization/config notation, but it's not JSON (nor YAML)

UPDATE: I might be thinking of nginx configs. More later.
Anyone know what this format is called?
I remember seeing a data notation in the last three months or so that resembles JSON, and somewhat resembles YAML, but it's without quotes and commas, and isn't indent-sensitive, and is perhaps one atom or pair per line. Something like that. If I recall correctly, it looks sort of like this:
{
my_first_name: Eneri
my_last_name: Ikkuddan
my_homeworld: Efate
}
Universal Configuration Language: https://github.com/vstakhov/libucl
You can take a look at the Eclog format, and maybe you will like it.
This is an example:
# Person.ecl
firstName: John
lastName: Smith
isAlive: true
age: 27
address:
{
streetAddress: "21 2nd Street"
city: "New York"
state: NY
postalCode: "10021-3100"
}
phoneNumbers:
[
{ type: home, number: "212 555-1234" }
{ type: office, number: "646 555-4567" }
{ type: mobile, number: "123 456-7890" }
]
children: []
spouse: null
In addition to unquoted strings, it also provides raw strings and heredoc strings.

Embedded json model with one-to-one relationship

I searched for a clear explanation on how this works but was not able to find one yet. I'm looking for a clear and detailed explanation with the assumption that I'm a total newbie with all these frameworks.
So here's my problem, I'm writing an app with frontend using Ember.js (with Ember-cli) and backend under Play-framework in Java. I'm trying to get my frontend to digest some json coming out from my API. Here's the json :
{
"buildings": [
{
"id": 0,
"createdDateTime": "2015-03-27T06:39:19.913Z",
"address": {
"id": 1,
"city": "City",
"civicNumber": 1287,
"country": "Canada",
"postalZipCode": "G1Q1Q9",
"provinceOrState": "QC",
"streetName": "A Street Name"
},
"cost": 1000000,
"earnings": 2300,
"mainPicturePath": "http://dummyimage.com/200x200/999/fff.png&text=Building",
"type": "DUPLEX",
"yearOfBuilt": 1998
},
{
"id": 1,
"createdDateTime": "2015-03-27T06:39:19.935Z",
"address": {
"id": 2,
"city": "City",
"civicNumber": 1289,
"country": "Canada",
"postalZipCode": "G1Q1Q9",
"provinceOrState": "QC",
"streetName": "A Street Name"
},
"cost": 980000,
"earnings": 670,
"mainPicturePath": "http://dummyimage.com/200x200/999/fff.png&text=Building",
"type": "TRIPLEX",
"yearOfBuilt": 1980
}]
}
And here's my emberjs code :
//models/building.js
export default DS.Model.extend({
type: DS.attr('string'),
address: DS.belongsTo('address', {embedded: 'always'}),
cost: DS.attr('number'),
yearOfBuilt: DS.attr('number'),
earnings: DS.attr('number'),
createdDateTime: DS.attr('date'),
mainPicturePath: DS.attr('string')
});
//models/address.js
export default DS.Model.extend({
civicNumber: DS.attr('number'),
streetName: DS.attr('string'),
city: DS.attr('string'),
provinceOrState: DS.attr('string'),
country: DS.attr('string'),
postalZipCode: DS.attr('string'),
building: DS.belongsTo('building', {embedded: 'always'})
});
//serializers/building.js
export default DS.RESTSerializer.extend(DS.EmbeddedRecordsMixin, {
attrs: {
address: {embedded: 'always'}
}
});
This doesn't throw any errors and works fine. The problem is, I don't fully understand the code for the embedded object address in building (I do understand how the DS models, the export from ember-cli and the belongsTo work). Could someone explains me in details how does it works (the RESTSerializer with EmbeddedRecordsMixin, the {embedded: 'always'} attribute and the other available options)? Any clarifications will be more than appreciated.
Lets also bring this to the next level, say I do not want to have an id for each address since they should never be used more than once (cannot have 2 buildings at the same address). How would I achieve that? Maybe I will store the address in the same record of my building object in the db and don't want an extra table for addresses. Based on these solutions, what would be the best approach (feel free to propose better solution if you have)?
Please be advised that I have already read the following links :
Ember-data: deserialize embedded model
http://mozmonkey.com/2013/12/loading-json-with-embedded-records-into-ember-data-1-0-0-beta/ (which doesn't say anything about EmbeddedRecordsMixin)
How to access nested object in json with Ember data (talking about the App.Adapter.Map which I don't understand either)
https://github.com/lytics/ember-data.model-fragments (Very interesting project that may quit be exactly what I'm looking for)
Thank you!
The answers to most of your questions can be found by closely reading the EmbeddedRecordsMixin Docs
Specifically:
using { embedded: 'always' } as an option to DS.attr is not a valid
way to setup embedded records.
(meaning as long as you've defined your belongsTo relationship you're good)
and
The attrs option for a resource { embedded: 'always' } is shorthand
for:
1 {
2 serialize: 'records',
3 deserialize: 'records'
4 }
But the real answer is, it's all in the code! Go read here and a little bit further down here, it's written pretty well and you should be able to trace what's going on.
Basically what happens is once you tell the serializer that there are embedded records (via {key: {embedded: 'always' } }), it will find that key in your JSON and deserialize it into an Ember object (as long as it finds a defined Ember object with that key, in your case 'address').
As far as your next level questions, I'd respond with my own question: Do your buildings have tenants? If so, they will probably have addresses, and you'll likely want to access it through tenant.address, rather than tenant.building.address, so go ahead and make address it's own table. It will likely save you some headaches in the near future.

Inserted Nested Item in Generated JSON in Groovy/Grails

I am using Grails 2.1 to render JSON as part of a RestFul API I created. The Domain class, based on a SqlServer table, looks like this:
String firstName
String lastName
String officialAddress1
String officalAddress2
String preferredAddress1
String preferredAddress2
(etc.). . .
Which returns JSON similar to this:
{
"firstName": "Joe",
"lastName": "Hill",
"officialAddress1": "1100 Wob Hill",
"officialAddress2": "Apt. # 3",
"preferredAddress1": "1100 Wobbly Lane.",
"preferredAddress2": "Apartment 3."
}
It is working fine but the client wants me to nest the results in this fashion:
{
"firstName": "Joe",
"lastName": "Hill",
preferredAddress {
"preferredAddress1": "1100 Wobbly Lane.",
"preferredAddress1": "Apartment 3."
},
officialAddress {
"officialAddress1": "1100 Wob Hill",
"officialAddress2": "Apt. # 3"
}
}
My question is since the domain class, and the database, are not structure in a way to return this type of nested result how can I easily change this in my returned JSON? Do I have to abandon my way of just regurgitating the JSON based on the database/domain object and do a custom converter of some kind?
i'm new to this stackoverflow thing and i hope i will not mess it but i think i know what you need. in your bootstrap.groovy file you find "def init = { servletContext -> " line
put in there something like this:
JSON.registerObjectMarshaller(YourDomainName) {
def returnArray = [:]
returnArray['firstName'] = it.firstName
returnArray['lastName'] = it.lastName
returnArray['preferredAddress'] = [it.preferredAddress1 ,it.preferredAddress2]
returnArray['officialAddress'] = [it.officialAddress1 ,it.officialAddress2]
return returnArray
}
now when you use the render with JSON as you did grails will look in bootstrap and
render the domain as you asked.
hope this helps
The posted answer was correct. I just wanted to add the slight change I made to get the exact results I needed:
Thanks! That did it. I originally that it would not work exactly how I needed it but I was wrong. I changed the syntax slightly to get the results I needed.
returnArray['preferredAddress'] = [address1: it.preferredAddress1?.trim(),
address2: it.preferredAddress2?.trim(),
address3: it.preferredAddress3?.trim(),
city: it.preferredCity,
state: it.preferredState,
postCode: it.preferredPostCode,
country: it.preferredCountry
]