sql & django: Making aggregation on database level - mysql

Hello I am having the following situation in my models:
class Package(models.Model):
name
class SetOfItems(models.Model)
name
class Item(models.Model):
package(FK)
setofitems(FK)
And the problem I am trying to solve is to simplify queries from SetOfItems. E.g. in case I would need to get a package name from SetOfItems, I would need to write something like
SetOfItems.objects.values('item__package__name'), which actually does not work, well in admin site (e.g. if I am creating an extra field there).
So what I am looking for is to store the information about Packages in SetOfItem model without making a direct relation (as I think that if I will put a FK there the data integrity will suffer, e.g. it will be possible to have on package inside Item and another one in SetOfItems)..
So maybe there is some possibility to have SQL view, or something like this... to have a table column automatically pre-filled from other table? Or something else?

Item is the through table in a many-to-many relationship between Package and SetOfItems. So you should declare it as such:
class SetOfItems(models.Model):
packages = models.ManyToManyField(Package, through='Item')
This doesn't change your actual table structure at all, but it allows you to do my_package.setofitems.all().

Related

Database design for keeping track of experiment data

I am designing a database to record experiment results. Basically, an experiment has several input parameters and an output response. Therefore, the data table will look like following:
run_id parameter_1 parameter_2 ... parameter_n response
1 ... ... ... ...
2 ... ... ... ...
.
.
.
However, the structure of this table is not determinant since different experiments have different number of columns. Then the question is: when a user instantiate an experiment, is it a good idea to create data table dynamically on the fly? Otherwise, what is the elegant solution for that? Thanks.
When I find myself trying to dynamically create tables during runtime, it usually means I need another table to resolve a relationship between entities. In short, I would recommend treating your input parameters as a separate entity and store them in a separate table.
It sounds like your entities are:
experiment
runs of an experiment, which consist of a response and one or more:
input parameters
The relationships between entities is:
One experiment to zero or more runs
One run to one or more input parameter values (one to many)
This last relationship will require an additional table to resolve. You can have a separate table that stores your input parameters, and associate the input parameters with a run_id. This table could look like:
run_parameter_id ... run_id_fk ... parameter_keyword ... parameter_value
Where run_id_fk is a foreign key to the appropriate row in the Runs table (described in your question). The parameter_keyword is just used to keep track of the name of the parameter (parameter_1_exp1, parameter_2_exp1, etc).
Your queries to read/write from the database now become a bit more complicated (needing a join), but no longer reliant on creating tables on the fly.
Let me know if this is unclear and I can provide a potential database diagram.

which solution is good practice hibernate save data in database?

I have two entities which are Student and Class entities.
Student and Class are many to one relationship. So student contains class attribute.
Now i want to save or create a student associate with existing class(means i know primary key ID already).
Solution 1:
Student student = new Student();
Class class = session.load(classId);
student.setClass(class);
session.save(student);
Solution 2:
Student student = new Student();
Class class = new Class();
class.setClassId(classId);
student.setClass(class);
session.save(student);
My question here is in solution 1 it will issue two SQL, one is to get Class another is to insert student. But in solution 2 only need to have one SQL. If I have more class attribute,
i will load and issue more select sql before insert. It seems not that efficient. Is there any side-effect in solution 2?
which way to do save/insert is better? By the way, i do not set up cascade.
Thank you
Yee Chen
Solution 1 won't issue an SQL query to load Class. Unlike get(), load() returns a proxy object with the specified identifier and doesn't perform a database query immediately. Thus, load() method is a natural choice for this scenario (when you actually need to load an object, use get()).
Possible side effect of solution 2 depends on cascading configuration of relationship and so on. Even if it works fine in your current case, it makes your code more fragile, since seemingly unrelated changes in the code may break it.
So, I recommend you to use Solution 1 and don't worry about performance.
Is there any side-effect in solution 2?
First of all, you haven't associated your student with any class there.
Secondly, where do you get that class id from in the general case? At some earlier point in time, you had to either fetch an existing class instance from the DB, or create a new instance and persist it, so that you get its id. Of course, reusing an entity you already have is fine, but juggling with ids like you do above is IMHO not.
Thirdly, it is not a good idea to prematurely optimize your app. Get it to work properly first, then measure performance, and optimize only if and where needed.

Using Multiple Foreign Keys to the same table in LINQ

I have a table Users and a table Items
In the Items table, I have fields such as
ModifiedBy
CreatedBy
AssignedTo
which all have a userId integer. The database is set up to have these as foreign keys back to the Users table.
When using LINQToSQL, the relationships which are automatically built from the dbml end up giving me names like User, User1 and User2
e.g. myItem.User1.Name or myItem.User2.Name
Obviously this isn't very readable and I'd like it be along the lines of
myItem.CreatedByUser.Name or myItem.ModifiedByUser.Name etc
I could change the names of the relationships but that means I have to redo that every time I change the db schema and refresh the dbml.
Is there any way round this?
The simple answer: No.
Someone has suggested the idea of creating partial Association classes where the property names get defined, but that won't work either: Renaming LINQ 2 SQL Entity Properties Through Partial Classes.
Your choice is to either spend a little time learning more about LINQ-to-SQL "behind-the-scenes" so that you can manually make the necessary modifications or to just change the property names through the Properties window. Personally, I just delete/redrag/rename, because not setting a property correctly is a pain to debug because the exceptions that get thrown give you little to no clue as to what caused it. I even went so far as to create a unit test library that takes each MetaTable object in the model and verifies the field count, the ServerDataType contents of each field, the association count, the names of each association, and the names of each end of the association. Every few changes, I run the unit tests to make sure that the model is intact.
Firstly, no... the names are created based on the second table in the relationship.
But what you should know is that you don't have to "refresh" (meaning, delete the table in the DBML then re-drag-and-drop it).
For the project I'm working on, we have over 200 tables... about 50 of which we have manually tweaked after dragging them from the database. We never delete and re-drag tables as there have been so many changes post-auto-generation.
I just add a small partial class to extend the object with suitably named properties, example below:
namespace Database.TableModels {
partial class WTSR_Induction {
public EmailTemplate ConfirmationEmailTemplate {
get { return EmailTemplate1; }
}
public EmailTemplate InviteEmailTemplate {
get { return EmailTemplate; }
}
}
}
In this example, the WTSR_Inductions table has two links to the EmailTemplates table, hence the EmailTemplate and EmailTemplate1 properties.
A bit late but you can do this by selecting the relationship on the linq model and go to properties and update the parent property name.
You could use linq to sql without the dbml it may be extra work upfront but from the perspective of a change to a table column name it may be easier than changes to the dbml as you have described.
I suggest creating extension methods mapping the names you want to the names you get from the autogenerated code. That way after each auto-generation you don't have to change the autogenerated code, but only your own extension methods. That, plus the unit tests to do sanity checks as suggested elsewhere on this page should work fine.
I have just faced this problem myself and I'm off to try to implement my own suggestion.
EDIT: This seems relevant:
SQLMetal Multiple Foreign Keys Pointing to One Table Issue

Linq2sql naming convention

I have just started using Linq2sql, it generates all of the Classes after my tables which is awesome. my problem is that i have a lot of objects that have the same name as my tables.
this is forcing my to fully namespace everything which i don’t really like as i think it makes my code look messy.
Has anyone found an elegant way to get around this?
ps: i know about namespace aliases and im not really a fan.
If this really is an issue for you, you can change the name of your generated table items in the Linq designer. Select the table and change the value in the Name field in the properties (L2S keeps track of the source table as a separate item, so the two don't have to be named the same). Note that regenerating your DBML file will wipe these changes out.
Along the lines of Rails convention (which I use in my .net) frequently I will name the database table the plural name and the class the single name eg. Table = Users vs. Class = User.
You could try turning off pluarisation of the names it generates.
Tools -> Options -> Database Tools -> O/R Designer

Limiting results of System.Data.Linq.Table<T>

I am trying to inherit from my generated datacontext in LinqToSQL - something like this
public class myContext : dbDataContext {
public System.Data.Linq.Table<User>() Users {
return (from x in base.Users() where x.DeletedOn.HasValue == false select x);
}
}
But my Linq statement returns IQueryable which cannot cast to Table - does anyone know a way to limit the contents of a Linq.Table - I am trying to be certain that anywhere my Users table is accessed, it doesn't return those marked deleted. Perhaps I am going about this all wrong - any suggestions would be greatly appreciated.
Hal
Another approach would to be use views..
CREATE VIEW ActiveUsers as SELECT * FROM Users WHERE IsDeleted = 0
As far as linq to sql is concerned, that is just the same as a table. For any table that you needed the DeletedOn filtering, just create a view that uses the filter and use that in place of the table in your data context.
You could use discriminator column inheritance on the table, ie. a DeletedUsers table and ActiveUsers table where the discriminator column says which goes to which. Then in your code, just reference the Users.OfType ActiveUsers, which will never include anything deleted.
As a side note, how the heck do you do this with markdown?
Users.OfType<ActiveUsers>
I can get it in code, but not inline
Encapsulate your DataContext so that developers don't use Table in their queries. I have an 'All' property on my repositories that does a similar filtering to what you need. So then queries are like:
from item in All
where ...
select item
and all might be:
public IQueryable<T> All
{
get { return MyDataContext.GetTable<T>.Where(entity => !entity.DeletedOn.HasValue); }
}
You can use a stored procedure that returns all the mapped columns in the table for all the records that are not marked deleted, then map the LINQ to SQL class to the stored procedure's results. I think you just drag-drop the stored proc in Server Explorer on to the class in the LINQ to SQL designer.
What I did in this circumstance is I created a repository class that passes back IQueryable but basically is just
from t in _db.Table
select t;
this is usually referenced by tableRepository.GetAllXXX(); but you could have a tableRepository.GetAllNonDeletedXXX(); that puts in that preliminary where clause to take out the deleted rows. This would allow you to get back the deleted ones, the undeleted ones and all rows using different methods.
Perhaps my comment to Keven sheffield's response may shed some light on what I am trying to accomplish:
I have a similar repository for most
of my data access, but I am trying to
be able to traverse my relationships
and maintain the DeletedOn logic,
without actually calling any
additional methods. The objects are
interrogated (spelling fixed) by a StringTemplate
processor which can't call methods
(just props/fields).
I will ultimately need this DeletedOn filtering for all of the tables in my application. The inherited class solution from Scott Nichols should work (although I will need to derive a class and relationships for around 30 tables - ouch), although I need to figure out how to check for a null value in my Derived Class Discriminator Value property.
I may just end up extended all my classes specifically for the StringTemplate processing, explicitly adding properties for the relationships I need, I would just love to be able to throw StringTemplate a [user] and have it walk through everything.
There are a couple of views we use in associations and they still appear just like any other relationship. We did need to add the associations manually. The only thing I can think to suggest is to take a look at the properties and decorated attributes generated for those classes and associations.
Add a couple tables that have the same relationship and compare those to the view that isn't showing up.
Also, sometimes the refresh on the server explorer connection doesn't seem to work correctly and the entities aren't created correctly initially, unless we remove them from the designer, close the project, then reopen the project and add them again from the server explorer. This is assuming you are using Visual Studio 2008 with the linq to sql .dbml designer.
I found the problem that I had with the relationships/associations not showing in the views. It seems that you have to go through each class in the dbml and set a primary key for views as it is unable to extract that information from the schema. I am in the process of setting the primary keys now and am planning to go the view route to isolate only non-deleted items.
Thanks and I will update more later.