Recursive as_array for One-to-many relationships with Kohana 3 ORM - json

To begin with I'd like to say I know how to create ugly solutions for my problem. I am searching good solutions and best practices :)
How do I create deep hierarchial arrays (to json_encode later) from Kohana 3 ORM objects including related objects where the relation type is one-to-many?
The problem is that the ORM->as_array() method does work recursively for the "has one" and "belongs to" relationships but will stop and force you to use ->find_all() manually when you encounter a "has many" relationship.
Say I am creating a JSON API REST server using Kohana 3 and the built in ORM.
When someone looks at this URL: www.example.com/api/user?id=5
They will be served a JSON object for the user where id=5.
These are the orm-models and the relations:
User belongs to a Country
User has many Messages.
Message belongs to a Category
I would like this to work:
echo json_encode(
ORM::factory('user', 5)
->with('country')
->with('messages')
->with('messages:category')
->find()
->as_array()
);
and give me output like this:
{
name: "John"
age: 54,
country_id: 5,
country: {
name: 'Sweden',
code: 'SE'
},
messages: {
{
content: 'Lorem ipsum dolor...',
category_id: 1,
category: {...}
},
{
content: 'Sit amet elit...',
category_id: 2,
category: {...}
},
{
content: 'Consectetur ipsum dolor...',
category_id: 3,
category: {...}
}
}
}
BUT THAT WONT WORK.
This is all you would get:
{
name: "John"
age: 54,
country_id: 5,
country: {
name: 'Sweden',
code: 'SE'
}
}
Has someone forked or extended the Kohana 3 ORM to support this kind functionality?
Does someone know of any good Kohana 3 api module that takes care of this issue for you somehow?

As far as I know, there is no way to do this using only the ORM. This is the case for two reasons:
If a message has a relation for category, that usually means that category has a corresponding relation for the messages in the category. If you want to get the message "in full" - i.e. including its category - let's say that you also want to get the category "in full" - i.e. including its messages. This is obviously a very bad idea, as you could very easily design yourself into an infinite loop. In other words, if there was a magic "recurse all relations" capability, how would it know when to stop recursing?
All that the ORM is doing behind the scenes when you call find() is building a SQL query that returns a row of data from the database. However, what you are trying to do is too complicated for a single query that returns a single row. (There is a way to retrieve multiple rows as one row with comma-separated values in each field using MySQL's GROUP_CONCAT function, but I guarantee you that it is not worth the trouble.)
For both of these reasons, the ORM's with() method only works on belongs_to (in your case, the user's country and the message's category).
One way to do this query is to break it up into three steps, like this:
// Step 1: Get the user
$user = ORM::factory('user', 5)
->with('country')
->find();
// Step 2: Get the messages
$user->messages
->with('category')
->find_all();
// Step 3: Make user and the messages into arrays.
// User is easy, but messages are a little harder
// because they need to turn
// from "an object containing an array of objects"
// into "an array of arrays."
$user_arr = $user->as_array();
$fixer = function($obj)
{
return $obj->as_array();
};
$user_arr['messages'] = array_map($fixer, $user->messages->as_array());
// Now you can output it
echo json_encode($user_arr);

Related

Sorting users in Rails based on values in array attribute of user model

I have a User model which has an array inside of it. This array is used to store points the user has scored in various activities. It basically looks like this:
<ActiveRecord::Relation [#<User id: 1, fullname: "Kaja Sunniva Edvardsen", points: [0, 4170, 3860, 2504, 2971, 3859, 4346]>, #<User id: 2, fullname: "Alexander Lie Sr.", points: [0, 3273, 3681, 2297, 2748, 4202, 3477]>]>
I want to sort all Users by the different values in the points array to be able to create ranking list for each of the different activities, points[0], points[1], etc...
Sorting by points[1] should return Kaja first, 4170>3273, sorting by points[6] should put Alexander first, 4202>3859
How do I do this?
As far as I know, MySQL does not have an integrated array type.
Assuming you have a model like this:
class User < ActiveRecord::Base
# ...
serialize :points, Array
# ...
end
You cannot sort with order queries, but you can try another solution (less efficient), handling the resources as an array:
User.all.sort { |user1, user2| user2.points[1] <=> user1.points[1] }
Which will return an array instead of an ActiveRecord query. Also, bear in mind that this code will not handle nil values (i.e. What if an user only have 2 elements in points?).

Using JSON-based Database for unordered data

I am working on a simple app for Android. I am having some trouble using the Firebase database since it uses JSON objects and I am used to relational databases.
My data will consists of two users that share a value. In relational databases this would be represented in a table like this:
**uname1** **uname2** shared_value
In which the usernames are the keys. If I wanted the all the values user Bob shares with other users, I could do a simple union statement that would return the rows where:
uname1 == Bob or unname == Bob
However, in JSON databases, there seems to be a tree-like hierarchy in the data, which is complicated since I would not be able to search for users at the top level. I am looking for help in how to do this or how to structure my database for best efficiency if my most common search will be one similar to the one above.
In case this is not enough information, I will elaborate: My database would be structured like this:
{
'username': 'Bob'
{
'username2': 'Alice'
{
'shared_value' = 2
}
}
'username': 'Cece'
{
'username2': 'Bob'
{
'shared_value' = 4
}
}
As you can see from the example, Bob is included in two relationships, but looking into Bobs node doesn't show that information. (The relationship is commutative, so who is "first" cannot be predicted).
The most intuitive way to fix this would be duplicate all data. For example, when we add Bob->Alice->2, also add Alice->Bob->2. In my experience with relational databases, duplication could be a big problem, which is why I haven't done this already. Also, duplication seems like an inefficient fix.
Is there a reason why you don't invert this? How about a collection like:
{ "_id": 2, "usernames":[ "Bob", "Alice"]}
{ "_id": 4, "usernames":[ "Bob", "Cece"]}
If you need all the values for "Bob", then index on "usernames".
EDIT:
If you need the two usernames to be a unique key, then do something like this:
{ "_id": {"uname1":"Bob", "uname2":"Alice"}, "value": 2 }
But this would still permit the creation of:
{ "_id": {"uname1":"Alice", "uname2":"Bob"}, "value": 78 }
(This issue is also present in your as-is relational model, btw. How do you handle it there?)
In general, I think implementing an array by creating multiple columns with names like "attr1", "attr2", "attr3", etc. and then having to search them all for a possible value is an artifact of relational table modeling, which does not support array values. If you are converting to a document-oriented storage, these really should be an embedded list of values, and you should use the document paradigm and model them as such, instead of just reimplementing your table rows as documents.
You can still have old structure:
[
{ username: 'Bob', username2: 'Alice', value: 2 },
{ username: 'Cece', username2: 'Bob', value: 4 },
]
You may want to create indexes on 'username' and 'username2' for performance. And then just do the same union.
To create a tree-like structure, the best way is to create an "ancestors" array that stores all the ancestors of a particular entry. That way you can query for either ancestors or descendants and all documents that are related to a particular value in the tree. Using your example, you would be able to search for all descendants of Bob's, or any of his ancestors (and related documents).
The answer above suggest:
{ "_id": {"uname1":"Bob", "uname2":"Alice"}, "value": 2 }
That is correct. But you don't get to see the relationship between Bob and Cece with this design. My suggestion, which is from Mongo, is to store ancestor keys in an ancestor array.
{ "_id": {"uname1":"Bob", "uname2":"Alice"}, "value": 2 , "ancestors": [{uname: "Cece"}]}
With this design you still get duplicates, which is something that you do not want. I would design it like this:
{"username": "Bob", "ancestors": [{"username": "Cece", "shared_value": 4}]}
{"username": "Alice", "ancestors": [{"username": "Bob", "shared_value": 2}, {"username": "Cece"}]}

CSV Parser through angularJS

I am building a CSV file parser through node and Angular . so basically a user upload a csv file , on my server side which is node the csv file is traversed and parsed using node-csv
. This works fine and it returns me an array of object based on csv file given as input , Now on angular end I need to display two table one is csv file data itself and another is cross tabulation analysis. I am facing problem while rendering data, so for a table like
I am getting parse responce as
For cross tabulation we need data in a tabular form as
I have a object array which I need to manipulate in best possible way so as to make easily render on html page . I am not getting a way how to do calculation on data I get so as to store cross tabulation result .Any idea on how should I approach .
data json is :
[{"Sample #":"1","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"2","Gender":"Male","Handedness;":"Left-handed;"},{"Sample #":"3","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"4","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":"5","Gender":"Male","Handedness;":"Left-handed;"},{"Sample #":"6","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":"7","Gender":"Female","Handedness;":"Right-handed;"},{"Sample #":"8","Gender":"Female","Handedness;":"Left-handed;"},{"Sample #":"9","Gender":"Male","Handedness;":"Right-handed;"},{"Sample #":";"}
There are many ways you can do this and since you have not been very specific on the usage, I will go with the simplest one.
Assuming you have an object structure such as this:
[
{gender: 'female', handdness: 'lefthanded', id: 1},
{gender: 'male', handdness: 'lefthanded', id: 2},
{gender: 'female', handdness: 'righthanded', id: 3},
{gender: 'female', handdness: 'lefthanded', id: 4},
{gender: 'female', handdness: 'righthanded', id: 5}
]
and in your controller you have exposed this with something like:
$scope.members = [the above array of objects];
and you want to display the total of female members of this object, you could filter this in your html
{{(members | filter:{gender:'female'}).length}}
Now, if you are going to make this a table it will obviously make some ugly and unreadable html so especially if you are going to repeat using this, it would be a good case for making a directive and repeat it anywhere, with the prerequisite of providing a scope object named tabData (or whatever you wish) in your parent scope
.directive('tabbed', function () {
return {
restrict: 'E',
template: '<table><tr><td>{{(tabData | filter:{gender:"female"}).length}}</td></tr><td>{{(tabData | filter:{handedness:"lefthanded"}).length}}</td></table>'
}
});
You would use this in your html like so:
<tabbed></tabbed>
And there are ofcourse many ways to improve this as you wish.
This is more of a general data structure/JS question than Angular related.
Functional helpers from Lo-dash come in very handy here:
_(data) // Create a chainable object from the data to execute functions with
.groupBy('Gender') // Group the data by its `Gender` attribute
// map these groups, using `mapValues` so the named `Gender` keys persist
.mapValues(function(gender) {
// Create named count objects for all handednesses
var counts = _.countBy(gender, 'Handedness');
// Calculate the total of all handednesses by summing
// all the values of this named object
counts.Total = _(counts)
.values()
.reduce(function(sum, num) { return sum + num });
// Return this named count object -- this is what each gender will map to
return counts;
}).value(); // get the value of the chain
No need to worry about for-loops or anything of the sort, and this code also works without any changes for more than two genders (even for more than two handednesses - think of the aliens and the ambidextrous). If you aren't sure exactly what's happening, it should be easy enough to pick apart the single steps and their result values of this code example.
Calculating the total row for all genders will work in a similar manner.

How to enter multiple table data in mongoDB using json

I am trying to learn mongodb. Suppose there are two tables and they are related. For example like this -
1st table has
First name- Fred, last name- Zhang, age- 20, id- s1234
2nd table has
id- s1234, course- COSC2406, semester- 1
id- s1234, course- COSC1127, semester- 1
id- s1234, course- COSC2110, semester- 1
how to insert data in the mongo db? I wrote it like this, not sure is it correct or not -
db.users.insert({
given_name: 'Fred',
family_name: 'Zhang',
Age: 20,
student_number: 's1234',
Course: ['COSC2406', 'COSC1127', 'COSC2110'],
Semester: 1
});
Thank you in advance
This would be a assuming that what you want to model has the "student_number" and the "Semester" as what is basically a unique identifier for the entries. But there would be a way to do this without accumulating the array contents in code.
You can make use of the upsert functionality in the .update() method, with the help of of few other operators in the statement.
I am going to assume you are going this inside a loop of sorts, so everything on the right side values is actually a variable:
db.users.update(
{
"student_number": student_number,
"Semester": semester
},
{
"$setOnInsert": {
"given_name": given_name,
"family_name": family_name,
"Age": age
},
"$addToSet": { "courses": course }
},
{ "upsert": true }
)
What this does in an "upsert" operation is first looks for a document that may exist in your collection that matches the query criteria given. In this case a "student_number" with the current "Semester" value.
When that match is found, the document is merely "updated". So what is being done here is using the $addToSet operator in order to "update" only unique values into the "courses" array element. This would seem to make sense to have unique courses but if that is not your case then of course you can simply use the $push operator instead. So that is the operation you want to happen every time, whether the document was "matched" or not.
In the case where no "matching" document is found, a new document will then be inserted into the collection. This is where the $setOnInsert operator comes in.
So the point of that section is that it will only be called when a new document is created as there is no need to update those fields with the same information every time. In addition to this, the fields you specified in the query criteria have explicit values, so the behavior of the "upsert" is to automatically create those fields with those values in the newly created document.
After a new document is created, then the next "upsert" statement that uses the same criteria will of course only "update" the now existing document, and as such only your new course information would be added.
Overall working like this allows you to "pre-join" the two tables from your source with an appropriate query. Then you are just looping the results without needing to write code for trying to group the correct entries together and simply letting MongoDB do the accumulation work for you.
Of course you can always just write the code to do this yourself and it would result in fewer "trips" to the database in order to insert your already accumulated records if that would suit your needs.
As a final note, though it does require some additional complexity, you can get better performance out of the operation as shown by using the newly introduced "batch updates" functionality.For this your MongoDB server version will need to be 2.6 or higher. But that is one way of still reducing the logic while maintaining fewer actual "over the wire" writes to the database.
You can either have two separate collections - one with student details and other with courses and link them with "id".
Else you can have a single document with courses as inner document in form of array as below:
{
"FirstName": "Fred",
"LastName": "Zhang",
"age": 20,
"id": "s1234",
"Courses": [
{
"courseId": "COSC2406",
"semester": 1
},
{
"courseId": "COSC1127",
"semester": 1
},
{
"courseId": "COSC2110",
"semester": 1
},
{
"courseId": "COSC2110",
"semester": 2
}
]
}

Iterating through couchbase keys without a view

In couchbase, I was wondering if there was a way - WITHOUT using a view - to iterate through database keys. The admin interface appears to do this, but maybe its doing something special. What I'd like to is make a call like this to retrieve an array of keys:
$result = $cb->get("KEY_ALBERT", "KEY_FRED");
having the result be an array [KEY_ALEX, KEY_BOB, KEY_DOGBERT]
Again, I don't want to use a view unless there's no alternative. Doesn't look like its possible, but since the "view documents" in the admin appears to do this, I thought i'd double-check. I'm using the php interface if that matters.
Based on your comments, the only way is to create a simple view that emit only the id as par of the key:
function(doc, meta) {
emit( meta.id );
}
With this view you will be able to create query with the various options you need :
- pagination, range, ...
Note: you talk about the Administration Console, the console use an "internal view" that is similar to what I have written above (but not optimized)
I don't know about how couchbase admin works, but there are two options. First option is to store your docs as linked list, one doc have property (key) that points to another doc.
docs = [
{
id: "doc_C",
data: "somedata",
prev: "doc_B",
next: "doc_D"
},
{
id: "doc_D",
data: "somedata",
prev: "doc_C",
next: "doc_E"
}
]
The second approach is to use sequential id. You should have one doc that contain sequence and increment it on each add. It would be something like this:
docs = [
{
id: "doc_1",
data: "somedata"
},
{
id: "doc_2",
data: "somedata"
}
...
]
In this way you can do "range requests". To do this you form array of keys on server side:
[doc_1, doc_2 .... doc_N]and execute multiget query. Here is also a link to another example
The couchbase PHP sdk does support multiget requests. For a list of keys it will return an array of documents.
getMulti(array $ids, array $cas, int $flags) : array
http://www.couchbase.com/autodocs/couchbase-php-client-1.1.5/classes/Couchbase.html#method_getMulti