I have two mysql tables: Owners & Pets
Owner case class:
Owner(id: Int, name: String, age: Int)
Pet case class:
Pet(id: Int, ownerId: Int, type: String, name: String)
I want to create out of those tables list of OwnerAndPets:
case class OwnerAndPets(ownerId: Int,
name: String,
age: String,
pets: List[Pet])
(its for migrations purposes, I want to move those tables to be a collection of mongodb, which the collection documents would be OwnerAndPets objects)
I have two issues:
when I use join with quill on Owner & Pet I get list of tuples [(Owner, Pet)]
and if I have few pets for an owner I will get:
[(Owner(1, "john", 30), Pet(3,1,"dog","max")),
(Owner(1, "john", 30), Pet(4,1,"cat","snow"))]
I need it as (Owner(1, "john", 30), [Pet(3,1,"dog","max"), Pet(4,1,"cat","snow")])
how can I make it like this?
when I use join with quill on Owner & Pet I will not get owners that dont have pets and its fine cause this is what it supposed to be, but in my script in this case I would want to create object like:
OwnerAndPets(Owner(2, "mark", 30), List[])
Would appreciate any help, thanks
this is my join query:
query[Owner].join(query[Pet]).on((o, p) => o.id == p.o_id)
Your question highlights one of the major differences between FRM (Functional Relational Mapping) systems like Quill and Slick as opposed to ORMs like Hibernate. The purpose of FRM systems is not to build a particular domain-specific object hierarchy e.g. OwnersAndPets, but rather, to be able translate a single database query into some set of objects that can reasonably be pulled out of that single query's result set - this is typically a tuple. This means it is up to you to join the tuples (Owner_N, Pet_1-N) object into a single OwnersAndPets object in memory. Typically this can be done via groupBy and map operators:
run(query[Owner].join(query[Pet]).on((o, p) => o.id == p.o_id))
.groupBy(_._1)
.map({case (owner,ownerPetList) =>
OwnerAndPets(
owner.id,owner.name,owner.age+"", // Not sure why you made 'age' a String in OwnerAndPets
ownerPetList.map(_._2))
})
That said, there are some database vendors (e.g. Postgres) that internally implement array types so in some cases you can do the join on the database-level but this is not the case for MySQL and many others.
Related
I am trying to create a database for gym workout plans. This would have each member's workout plan, which comprises multiple exercises.
My question is: given that one member might have 5 exercises, another member 3, another 6, and so on, how can I store this in the database?
My thought process has been around hardcoding a set number of exercises in the table, and give the possibility of them being null (for exemple 10 exercises, but if you only use 5 exercises, everything else can be stored with nothing). But this doesn't feel very correct.
I am using MySQL for this.
Like #Sami pointed out, you are going to work on whats called a relational database.
Below is a simple database structure that will help you get started.
Members table
Table Name: tbl_members
Table Structure:
|member_id|member_name|member_contact|
Exercise or schedule table
Table Name: tbl_member_schedule
Table Structure:
|schedule_id|member_id|assigned_date|Schedule
In the tbl_member_schedule you can store the entire schedule as a JSON structure or you can further normalize. Since this is an example I am storing the entire schedule in JSON format inside the Schedule field.
Now when you want to retrieve the schedule of a specific member (member id: 1) you can query the database as follows:
select
tbl_members.member_id, tbl_members.member_name,
tbl_member_schedule.Schedule
from
tbl_members join
tbl_member_schedule on tbl_members.member_id = tbl_member_schedule.member_id
where
tbl_members.member_id = 1;
So you JSON structure for your schedule could be
{
day1: [
{exerciseName: 'dumbbell curls', reps: '8 x 4'},
{exerciseName: 'leg extension', reps: '8 x 3'}
],
day2: [
{exerciseName: 'dumbbell curls', reps: '8 x 4'},
{exerciseName: 'leg extension', reps: '8 x 3'}
]
}
It is possible to store json in postgres using the json data type. Check this tutorial for an introduction: http://www.postgresqltutorial.com/postgresql-json/
Consider I am storing the following json in such a field:
{
"address": {
"street1": "123 seasame st"
}
}
I want a to store separately a reference to the street field. For example, I might have another object which is using data from this json structure and wants to store a reference to where it got the data. Maybe something like this:
class Product():
__tablename__ = 'Address'
street_1 = Column(String)
data_source = ?
Now I could make data_source a string and just store namespaces like address.street, but if I did this postgres has no idea what that means. Working with that in queries would mean parsing the string and other inefficient stuff. Does postgres support referring to fields stored inside json data structures?
This question is related to JSON foreign keys in PostgreSQL , but in this case I don't necessarily want a fk relationship. I just want to create a reference, which is not necessarily enforced in the way a fk is.
update:
To be more clear, I want to reference the location of something in the json structure on another attribute and store that reference in a column. In the below code, Address.data_source is a reference to the location of the street data (for example address.street1 in this case)
class Address():
__tablename__ = 'Address'
street_1 = Column(String)
sample_id = Column(Integer, ForeignKey('DataSample.uid'))
data_source = ?
class DataSample():
__tablename__ = 'DataSample'
uid = Column(Integer, primary_key=True)
data = Column(JSONB)
body = {
"address": {
"street1": "123 seasame st"
}
}
datasample = DataSample(data=body)
address = Address(street_1=datasample.data['address']['street_1'],
sample_id=datasample.uid,
data_source=?)
As clarified, the question is seeking a way to flexibly specify a path within a JSON object of a particular record. Keys are being handled in normal columns. Constraints on JSONB fields are not available, and there is no specific support for specifying paths within JSON objects.
I worked with the following in SQL Fiddle using PostgreSQL 9.6:
CREATE TABLE datasample (
id integer PRIMARY KEY,
data jsonb
);
CREATE TABLE address (
id integer PRIMARY KEY,
street_1 text,
sample_id integer REFERENCES datasample (id),
data_source text
);
INSERT INTO datasample(id, data)
VALUES (1, '{"address":{"street_1": "123 seasame st"}}');
INSERT INTO address(id,street_1, sample_id, data_source)
VALUES (1,'123 seasame st',1,'datasample.data->''address''->>''street''');
A typical lookup of the street address (needed to retrieve street_1) would resemble:
SELECT datasample.data->'address'->>'street_1'
FROM datasample
WHERE id=1;
There is no special postgres type for identifying columns. Strings are the closest available and you will need to retrieve the string (or array of strings, or object containing strings, if one of those simplifies parsing) and use it to build the query. In tbe first code block, I stored it as the (escaped) fragment of query - 'datasample.data->''address''->>''street'''. Though longer, it would require only retrieval and unescaping to use in a new custom query. I did not find a way to use the string as a fragment within the same SQL statement, though it might be possible to combine it with other bits of text to form a full statement that could be run through EXECUTE.
I have a User model which has an array inside of it. This array is used to store points the user has scored in various activities. It basically looks like this:
<ActiveRecord::Relation [#<User id: 1, fullname: "Kaja Sunniva Edvardsen", points: [0, 4170, 3860, 2504, 2971, 3859, 4346]>, #<User id: 2, fullname: "Alexander Lie Sr.", points: [0, 3273, 3681, 2297, 2748, 4202, 3477]>]>
I want to sort all Users by the different values in the points array to be able to create ranking list for each of the different activities, points[0], points[1], etc...
Sorting by points[1] should return Kaja first, 4170>3273, sorting by points[6] should put Alexander first, 4202>3859
How do I do this?
As far as I know, MySQL does not have an integrated array type.
Assuming you have a model like this:
class User < ActiveRecord::Base
# ...
serialize :points, Array
# ...
end
You cannot sort with order queries, but you can try another solution (less efficient), handling the resources as an array:
User.all.sort { |user1, user2| user2.points[1] <=> user1.points[1] }
Which will return an array instead of an ActiveRecord query. Also, bear in mind that this code will not handle nil values (i.e. What if an user only have 2 elements in points?).
I'm using the sqlalchemy to store a binary tree data in the db:
class Distributor(Base):
__tablename__ = "distributors"
id = Column(Integer, primary_key=True)
upline_id = Column(Integer, ForeignKey('distributors.id'))
left_id = Column(Integer, ForeignKey('distributors.id'))
right_id = Column(Integer, ForeignKey('distributors.id'))
how can I generate json "tree" format data like the above listed:
{'id':1,children:[{'id':2, children:[{'id':3, 'id':4}]}]}
I'm guessing you're asking to store the data in a JSON format? Or are you trying to construct JSON from the standard relational data?
If the former, why don't you just create entries like:
{id: XX, parentId: XX, left: XX, right: XX, value: "foo"}
For each of the nodes, and then reconstruct the tree manually from the entries? Just start form the head (parentId == null) and then assemble the branches.
You could also add an additional identifier for the tree itself, in case you have multiple trees in the database. Then you would just query where the treeId was XXX, and then construct the tree from the entries.
I hesitate to provide this answer, because I'm not sure I really understand your the problem you're trying to solve (A binary tree, JSON, sqlalchemy, none of these are problems).
What you can do with this kind of structure is to iterate over each row, adding edges as you go along. You'll start with what is basically a cache of objects; which will eventually become the tree you need.
import collections
idmap = collections.defaultdict(dict)
for distributor in session.query(Distributor):
dist_dict = idmap[distributor.id]
dist_dict['id'] = distributor.id
dist_dict.setdefault('children', [])
if distributor.left_id:
dist_dict.['children'].append(idmap[distributor.left_id])
if distributor.right_id:
dist_dict.['children'].append(idmap[distributor.right_id])
So we've got a big collection of linked up dicts that can represent the tree. We don't know which one is the root, though;
root_dist = session.query(Distributor).filter(Distributor.upline_id == None).one()
json_data = json.dumps(idmap[root_dist.id])
To begin with I'd like to say I know how to create ugly solutions for my problem. I am searching good solutions and best practices :)
How do I create deep hierarchial arrays (to json_encode later) from Kohana 3 ORM objects including related objects where the relation type is one-to-many?
The problem is that the ORM->as_array() method does work recursively for the "has one" and "belongs to" relationships but will stop and force you to use ->find_all() manually when you encounter a "has many" relationship.
Say I am creating a JSON API REST server using Kohana 3 and the built in ORM.
When someone looks at this URL: www.example.com/api/user?id=5
They will be served a JSON object for the user where id=5.
These are the orm-models and the relations:
User belongs to a Country
User has many Messages.
Message belongs to a Category
I would like this to work:
echo json_encode(
ORM::factory('user', 5)
->with('country')
->with('messages')
->with('messages:category')
->find()
->as_array()
);
and give me output like this:
{
name: "John"
age: 54,
country_id: 5,
country: {
name: 'Sweden',
code: 'SE'
},
messages: {
{
content: 'Lorem ipsum dolor...',
category_id: 1,
category: {...}
},
{
content: 'Sit amet elit...',
category_id: 2,
category: {...}
},
{
content: 'Consectetur ipsum dolor...',
category_id: 3,
category: {...}
}
}
}
BUT THAT WONT WORK.
This is all you would get:
{
name: "John"
age: 54,
country_id: 5,
country: {
name: 'Sweden',
code: 'SE'
}
}
Has someone forked or extended the Kohana 3 ORM to support this kind functionality?
Does someone know of any good Kohana 3 api module that takes care of this issue for you somehow?
As far as I know, there is no way to do this using only the ORM. This is the case for two reasons:
If a message has a relation for category, that usually means that category has a corresponding relation for the messages in the category. If you want to get the message "in full" - i.e. including its category - let's say that you also want to get the category "in full" - i.e. including its messages. This is obviously a very bad idea, as you could very easily design yourself into an infinite loop. In other words, if there was a magic "recurse all relations" capability, how would it know when to stop recursing?
All that the ORM is doing behind the scenes when you call find() is building a SQL query that returns a row of data from the database. However, what you are trying to do is too complicated for a single query that returns a single row. (There is a way to retrieve multiple rows as one row with comma-separated values in each field using MySQL's GROUP_CONCAT function, but I guarantee you that it is not worth the trouble.)
For both of these reasons, the ORM's with() method only works on belongs_to (in your case, the user's country and the message's category).
One way to do this query is to break it up into three steps, like this:
// Step 1: Get the user
$user = ORM::factory('user', 5)
->with('country')
->find();
// Step 2: Get the messages
$user->messages
->with('category')
->find_all();
// Step 3: Make user and the messages into arrays.
// User is easy, but messages are a little harder
// because they need to turn
// from "an object containing an array of objects"
// into "an array of arrays."
$user_arr = $user->as_array();
$fixer = function($obj)
{
return $obj->as_array();
};
$user_arr['messages'] = array_map($fixer, $user->messages->as_array());
// Now you can output it
echo json_encode($user_arr);