How to get Sum of Difference of two fields in Django ORM?
I have a modal where user activities are mapped. I add Time when user login, and add time when user logs out.
I need the difference of these two fields and then all the instances together to get Total time spent by user on the site.
User.objects.filter(**user_kwargs).annotate(
visit_count=Count('visit_history'),
time_on_site=Avg('visit_history__time_on_site'),
).filter(visit_count__gt=0).order_by('-time_on_site')
How to find total time spent by user?
I think you cant do that with django orm.
Not without using at least raw query anyway.
Let me repeat things so you can verify, if i understood you correctly.
You want to select the sum of difference betweem model.field_a and model.field_b.
Django orm only allows you to select model fields, not their differences or sums. If you want to use django orm here, then create additonal model and use raw query and aggregate function. Raw query would be the one that creates ghost field, by having something like
SELECT ..., row.one_value - row.second_value AS row.somecolumn, ... FROM foo ...
But since you already need to get down and dirty with pure sql, then i suggest you skip the orm completely and use transactions and connections and just pure sql to get what you need.
Related
How to generate MySQL Querys with LUIS and fetch data from the DB hosted in Azure?
Should generate a natural language query to an MySQL Query.
e.g.
How much beer was drunken on the oktoberfest 2018?
--> GET amountOfBeer FROM Oktoberfest WHERE Year ==2018;
Does anyone has an idea how to get this to work?
Already generated small Intents in LUIS e.g. GetAmountOfBeer
Dont know how to generate the MySQL Statements and how to get the data from the DB.
Thanks.
You should be able to achieve this, or something similar, using intents and entities. How successful this can be depends on how many and how diverse your queries need to be. First lets start with the phrase you mentioned: "How much beer was drunken on the oktoberfest 2018". You can easily (as you've done) add this as an utterance for an intent, GetAmountOfBeer. Though I'm a fan of intent names that you can read as "I want to GetAmountOfBeer", here you may want to name the intent amountOfBeer so you can use it in your query directly.
Next you need to set up you entities. For year (or datetime rather) that should be easy, as I believe there are some predefined entities for this. I think you need to use a datetime recognizer to parse out the right attribute (like year), but I haven't tried to do this before. Next, Oktoberfest seems to be a specific holiday or event in your DB, so you could create a list entity of all the events you have.
What you are left with is something like (pseudocode) GET topIntent FROM eventEntity WHERE Year ==datetime.Year, or something like that.
If your query set is more complex, you might have to have multiple GET statements, but you could put those in a switch statement by topIntent so that, no matter what the intent is, you can parse out the correct values. You also might want to build this into a dialog where you can check if the entities exist, and if not, you can prompt the user for the missing data.
So I am trying to make a social network on Django. Like any other social network users get the option to like a post, and each of these likes are stored in a model that is different from the model used for posts that show up in the news feed. Now I have tried two choices to get the like status on the go.
1.Least database hits:
Make one sql query and get the like entry for every post id if they exist.Now I use a custom django template tag to see if the like entry for the current post exist in the Queryset by searching an array that contains like statuses of all posts.
This way I use the database to get all values and search for a particular value from the list using python.
2.Separate Database Query for each query:
Here i use the same custom template tag but rather that searching through a Queryset I use the mysql database for most of the heavy lifting.
I use model.objects.get() for each entry.
Which is a more efficient algorithm. Also I was planning on getting another database server, can this change the choice if network latency is only around 0.1 ms.
Is there anyway that I can get these like statuses on the go as boolean values along with all the posts in a single db query.
An example query for the first method can be like
Let post_list be the post QuerySet
models.likes.objects.filter(user=current_user,post__in = post_list)
This is not a direct answer to your question, but I hope it is useful nonetheless.
and each of these likes are stored in a model that is different from the model used for news feed
I think you have a design issue here. It is better if you create a model that describes a post, and then add a field users_that_liked_it as a many-to-many relationship to your user model. Then, you can do something like post.users_that_liked_it and get a query set of all users that liked your page.
In my eyes you should also avoid putting logic in templates as much as possible. They are simply not made for it. Logic belongs into the model class, or, if it is dependent on the page visited, in the view. (As a rule of thumb).
Lastly, if performance is your main worry, you probably shouldn't be using Django anyway. It is just not that fast. What Django gives you is the ability to write clean, concise code. This is much more important for a new project than performance. Ask yourself: How many (personal) projects fail because their performance is bad? And how many fail because the creator gets caught in messy code?
Here is my advice: Favor clarity over performance. Especially in a young project.
Trying to do a simple Model.all.page(1)
But whenever .page is called, it creates a SQL COUNT call. (My actual code is more complex than above, but simplified for ease of reading.) Is there a way to prevent .page from calling a SQL count? I'm dealing with millions of products and this calls is making the page refresh take an extra 2 seconds to load. I already have my own custom count which is instant so I don't need this .page count.
Edit: Def not using .all. Bad example sorry.
Heres a really simple example that is basically my code in a nutshell:
Product.limit(1).page(1)
With my real code SQL produces: (1495.3ms) SELECT COUNT(*) FROM 'products' LEFT OUTER JOIN...
I have joins on the products table that I don't need to be counted, hence the fact I have my own count methods I want to use and don't need .page to produce it's own count.
When you call Model.all.page(1) you are getting back an array instead of an ActiveRecord relation.
Try just calling Model.page(1) and you should get what you want... If what you want is:
Model.page(1)
# results in SELECT "models".* FROM "models" LIMIT 30 OFFSET 0
Edit:
So the issue ended up being in the will_paginate gem as it was calling count on the query to know the total number of entries so it can get an accurate number of pages. However will_paginate does provide an option to the paginate method which allows you to pass in a custom total_entries count which is useful if you have a massive table and don't care to get the precise number of pages for every record that matches the query.
You can pass in the option like so:
Model.paginate(:page => params[:page], :per_page => 30, :total_entries => 100)
You are concerned about doing a COUNT query, yet you are selecting ALL records from your database by doing Model.all? Are you joking right now?
Also you need to provide code in order to get help. We cant read your mind, we cant make up what code you might have. Especially when you say "my actual code is more complex than above". Don't try to simplify issues or hide code that you THINK is irrelevant.
What does your code look like? What does your log look like, specifically query time and total page time (rendering and ActiveRecord split out). You need to give more information.
Apologies if this is redundant, and it probably is, I gave it a look but couldn't find a question here that fell in with what I wanted to know.
Basically we have a table with about ~50000 rows, and it's expected to grow much bigger than that. We need to be able to allow admin users to add in custom data to an item based on its category, and users can just pick which fields defined by the administrators they want to add info to.
Initially I had gone with an item_categories_fields table which pairs up entries from item_fields to item_categories, so admins can add custom fields and reuse them across categories for consistency. item_fields has a relationship to item_field_values which links values with fields, which is how we handled things in .NET. The project is using CAKEPHP though, and we're just learning as we go, so it can get a bit annoying at times.
I'm however thinking of maybe just adding an item_custom_fields table that is essentially the item_id and a text field that stores XMLish formatted data. This is just for the values of the custom fields.
No problems if I want to fetch the item by its id as the required data is stored in the items table, but what if I wanted to do a search based on a custom field? Would a
SELECT * FROM item_custom_fields
WHERE custom_data LIKE '%<material>Plastic</material>%'
(user input related issues aside) be practical if I wanted to fetch items made of plastic in this case? Like how slow would that be?
Thanks.
Edit: I was afraid of that as realistically this thing will be around 400k rows for that one table at launch, thanks guys.
Any LIKE query that starts with % will not use any indexes you have on the column, so the query will scan the whole table to find the result.
The response time for that depends highly on your machine and the size of the table, but it definitely won't be efficient in any shape or form.
Your previous/existing solution (if well indexed) should be quite a bit faster.
I'm having an issue with a certain requirement to one of my Homework Assignments. I am required to take a list of students and print out all of the students with credit hours of 12 or more. The Credit hours are stored in a separate table, and referenced through a third table
basically, a students table, a classes table with hours, and an enrolled table matching student id to Course id
I used a SUM aggregate grouped by First name from the tables and that all works great, but I don't quite understand how to filter out the people with less than 12 hours, since the SQL doesn't know how many hours each person is taking until it's done with the query.
my string looks like this
'SELECT Students.Fname, SUM(Classes.Crhrs) AS Credits
FROM Students, Classes, Enrolled
WHERE Students.ID = Enrolled.StudentID AND Classes.ID = Enrolled.CourseID
GROUP BY Students.Fname;'
It works fine and shows the grid in the Delphi Project, but I don't know where to go from here to filter the results, since each query run deletes the previous.
Since it's a homework exercise, I'm going to give a very short answer: look up the documentation for HAVING.
Beside getting the desired result directly from SQL as Martijn suggested, Delphi datasets have ways to filter data on the "client side" also. Check the Filter property and the OnFilter record.
Anyway, remember it is usually better to apply the best "filter" on the database side using the proper SQL, and then use client side "filters" only to allow for different views on an already obtained data set, without re-querying the same data, thus saving some database resources and bandwidth (as long as the data on the server didn't change meanwhile...)