Tricky MySQL Query for messaging system in Rails - Please Help - mysql

I'm writing a facebook style messaging system for a Rails App and I'm having trouble selecting the Messages for the inbox (with will_paginate).
The messages are organized in threads, in the inbox the most recent message of a thread will appear with a link to it's thread. The thread is organized via a parent_id 1-n relationship with itself.
So far I'm using something like this:
class Message < ActiveRecord::Base
belongs_to :sender, :class_name => 'User', :foreign_key => "sender_id"
belongs_to :recipient, :class_name => 'User', :foreign_key => "recipient_id"
has_many :children, :class_name => "Message", :foreign_key => "parent_id"
belongs_to :thread, :class_name => "Message", :foreign_key => "parent_id"
end
class MessagesController < ApplicationController
def inbox
#messages = current_user.received_messages.paginate :page => params[:page], :per_page => 10, :order => "created_at DESC"
end
end
That gives me all the messages, but for one thread the thread itself and the most recent message will appear (and not only the most recent message). I can also not use the GROUP BY clause, because for the thread itself (the parent so to say) the parent_id = nil of course.
Anyone got an idea on how to solve this in an elegant way? I already thought about adding the parent_id to the parent itself and then group by parent_id, but I'm not sure if that works.
Thanks

My solution would be to get a list of threads (which I'm assuming could be obtained by messages with no parent id). Then on the Message model, add a method that will find the latest message in the thread and return it. You can then use that method to obtain the latest method in each thread and put in a link to the head of the thread easily.
(Pseudo-)code:
class Message < ActiveRecord::Base
belongs_to :sender, :class_name => 'User', :foreign_key => "sender_id"
belongs_to :recipient, :class_name => 'User', :foreign_key => "recipient_id"
has_many :children, :class_name => "Message", :foreign_key => "parent_id"
belongs_to :thread, :class_name => "Message", :foreign_key => "parent_id"
def get_last_message_in_thread()
last_message = self
children.each do |c|
message = c.get_last_message_in_thread()
last_message = message if message.created_at > last_message.created_at
end
return last_message
end
end
class MessagesController < ApplicationController
def inbox
#messages = current_user.received_messages.find_by_parent_id(Null).paginate :page => params[:page], :per_page => 10, :order => "created_at DESC"
end
end
You could probably do a lot better than having a recursive function to find the last message in the thread, but it's the simplest solution I can think of to demonstrate the idea. I'm also not sure I have the correct syntax for finding unset parent id's in the inbox function, which is why I marked the code as pseudo code :)

giving the parent itself as parent makes it very easy to create queries that operate on the whole thread, because you can group (or anything similar) by parent_id.
if you handle the parents differently, all your queries have to take care of this too

The only efficient way would be to have a Thread model and use GROUP BY as you mentioned - Anything else would require iteration over the messages.
read update in comments

I figured the only good solution is using a second model to store the most recent messages for every thread (because of performance issues when using GROUP BY with a subselect, see my comments). It won't take a lot of space in the DB because we're only storing id's and no text or even blobs.
The RecentMessages Model would look something like this:
create_table :recent_messages do |t|
t.integer :sender_id
t.integer :recipient_id
t.integer :message_id
t.integer :message_thread_id
t.timestamps
end
class RecentMessage < ActiveRecord::Base
belongs_to :message
belongs_to :message_thread, :class_name => 'Message'
belongs_to :sender, :class_name => 'User', :foreign_key => "sender_id"
belongs_to :recipient, :class_name => 'User', :foreign_key => "recipient_id"
end
The main idea is: All the messages are stored in one model (Messages). Whenever a new message is added to a thread (or a thread is created), two things happen (e.g. with a after_save callback):
Store the new message in the RecentMessages model (that means sender_id, recipient_id, message_id, message_thread_id (= parent_id || id))
Get the most recent message (from this thread in messages), where sender_id == recipient_id and vice versa (note: This only works if the message model should only support messages between 2 users) and store it in the RecentMessages model as well (if found and if it's not already there)
Of course there should only be max. 2 recent_messages stored in the DB for every message_thread at any given time.
If one wants to show i.e. the inbox, the following has to happen:
#messages = current_user.recent_received_messages.paginate :page => params[:page], :per_page => 10, :order => "created_at DESC", :include => :message
That's the best I figured out so far. I still think it's ugly but it's fast and it works. If anyone comes up with a better solution, I'll be gratefull!

I don't know how to accomplish this in Rails, but this is how I did it directly in MySQL:
select * from messages where message_id in (
select max(message_id) from messages where to_uid = 51 group by thread_id
) order by timestamp desc
I used a subquery to grab the most recent message in a thread and then the main query to grab all of the fields for the messages found in the subquery.

Related

Cant list conversations for current_user that is recipient of the conversation

I have a problem listing one of my models.. I am following the tutorial http://josephndungu.com/tutorials/gmail-like-chat-application-in-ruby-on-rails
In this tutorial you make a conversation that has a recipient_id and a sender_id these are both foreign keys to the conversation table. the current_user is set on either recipient or sender when the conversation is made.
-I can only list the current_user.conversations when the current_user is a sender.
-When i try with a current_user that is a recipient I get no result.
-Figured out that in my user model I have a has_many conversations with a foreign_key :sender_id. When I change this to recipient_id, then only the current_user that is a recipient can list conversations.
-With this I assume that I need to have two foreign keys referencing the same user? What can I do to get conversations to list for both current_users?
User.rb "model"
has_many :conversations, class_name: "Conversation", :foreign_key => :sender_id
listing conversation for current_user:
<% current_user.conversations.each do |conversation| %>
<p>hi</p>
<% end %>
Conversation.rb "model"
class Conversation < ActiveRecord::Base
belongs_to :sender, :foreign_key => :sender_id, class_name: 'User'
belongs_to :recipient, :foreign_key => :recipient_id, class_name: 'User'
has_many :messages, dependent: :destroy
validates_uniqueness_of :sender_id, :scope => :recipient_id
scope :involving, -> (user) do
where("conversations.sender_id =? OR conversations.recipient_id =?",user.id,user.id)
end
scope :between, -> (sender_id,recipient_id) do
where("(conversations.sender_id = ? AND conversations.recipient_id =?) OR (conversations.sender_id = ? AND conversations.recipient_id =?)", sender_id,recipient_id, recipient_id, sender_id)
end
end
The problem is that in your User.rb model you are saying that The conversations of this user are those whose sender_id is this user. That is the reason why when you use current_user.conversations you only get those records where sender_id is the current user.
I would do the following in your User.rb model:
has_many :conversations_started, class_name: "Conversation", :foreign_key => :sender_id
has_many :conversations_continued, class_name: "Conversation", :foreign_key => :recipient_id
This way you would be able to invoke:
current_user.conversations_started
or
current_user.conversations_continued
However, if you want to be able to list them all, you will need to define an additional method in your User.rb model:
def conversations
conversations_started + conversations_continued
end
If a user can start a conversation with itself, I would add .uniq at the end of the conversations method

Efficently querying multi language categories and category items

So I have a bit of a server response time issue - which I think is caused due to obsolete queries. One major query chain that I have takes up to 370ms, which is obviously causing an issue.
Here are the requirements:
5 Different languages
There are several Product Categories (i.e. Cat 1, Cat 2, Cat 3, etc.)
Categories displayed depend on language. For example whilst category 1 is displayed in all languages, category 2 is only displayed in Germany and France but not in the UK
Each category contains x number of items (has_many belongs_to relationship). Again some items are displayed in certain languages others are not. For example even category 2 is displayed in France and Germany, only in Germany you can buy Item 1 and hence Item 1 should not be displayed in France but Germany.
The categories and items do have boolean fields named after the locale. This way I can set via flag whether or not to display the category and item in a specific language.
My solution:
Building the solution is quiet easy. In controller I read out all the categories for the current locale:
application_controller.rb (since it is used on every single page)
#product_categories = ProductCategory.where("lang_" + I18n.locale.to_s + " = ?", true)
And in the view (the navigation) I do the following:
layouts/navs/productnav.html.haml
- #product_categories.each do |category|
...
- category.products.includes(:product_teasers).where("lang_" + I18n.locale.to_s + " = ? AND active = ?", true, true).in_groups_of(3).each do |group|
...
The issue with this solution is that each time I fire a lot of queries towards the database. Using "includes" does not solve it as I can not specify what items to pull. Furthermore I require the in_groups_of(3) in my loop to display the items correctly on the page.
I was also looking into memchached solutions to have the queries cached all together - i.e. Dalli however, this would require me to change a lot of code as I am guessing I would require to query all categories for each language and cache them. In addition to it I have to query each item for each langugage depending on language and store that somehow in an array ?!
My question:
How to approach this ? There must be a simpler and more efficient solution. How to efficiently query respectively cache this?
Thank you!
UPDATE:
As requested here are my two Models:
1.) ProductCategory
class ProductCategory < ActiveRecord::Base
translates :name, :description, :slug, :meta_keywords, :meta_description, :meta_title, :header, :teaser, :fallbacks_for_empty_translations => true
extend FriendlyId
friendly_id :name, :use => [:globalize, :slugged]
globalize_accessors :locales => [:at, :de, :ch_de, :ch_fr, :fr, :int_en, :int_fr], :attributes => [:slug]
has_paper_trail
has_many :products, :dependent => :destroy
validates :name, presence: true, uniqueness: { case_sensitive: false }
default_scope { includes(:translations) }
private
def slug_candidates
[
[:name]
]
end
end
2.) Product
And every product Category can have 0..n Products, and each Product must belongs to one category.
class Product < ActiveRecord::Base
translates :slug, :name, :meta_keywords, :meta_description, :meta_title, :teaser, :power_range, :product_page_teaser, :product_category_slider_teaser, :fallbacks_for_empty_translations => true
extend FriendlyId
friendly_id :name, :use => :globalize
before_save :change_file_name
searchable do
text :name, :teaser, :product_page_teaser, :product_category_slider_teaser
integer :product_category_id
end
belongs_to :product_category
has_many :product_teasers, :dependent => :destroy
has_many :product_videos, :dependent => :destroy
has_many :product_banners, :dependent => :destroy
has_many :product_documents, :dependent => :destroy
has_many :product_tabs, :dependent => :destroy
has_many :active_relationships, class_name: "Relationship",
foreign_key: "follower_id",
dependent: :destroy
has_many :passive_relationships, class_name: "Relationship",
foreign_key: "followed_id",
dependent: :destroy
has_many :following, through: :active_relationships, source: :followed
has_many :followers, through: :passive_relationships, source: :follower
has_many :references
has_and_belongs_to_many :contacts
accepts_nested_attributes_for :product_teasers, :reject_if => :all_blank, :allow_destroy => true
accepts_nested_attributes_for :product_tabs, :reject_if => :all_blank, :allow_destroy => true
accepts_nested_attributes_for :product_videos, :reject_if => :all_blank, :allow_destroy => true
accepts_nested_attributes_for :product_banners, :reject_if => :all_blank, :allow_destroy => true
accepts_nested_attributes_for :product_documents, :reject_if => :all_blank, :allow_destroy => true
has_paper_trail
validates :name, presence: true, uniqueness: { case_sensitive: false }
default_scope {includes(:translations)}
.... a lot more going on here ...
end
Please note: That category contains language flags (booleans), i.e lang_at, lang_de, lang_fr, etc. and if set then this category is displayed in that particualar language. SAME applies to products, as certain products are not displayed in all langauges altough the category might be.
Examples:
#product_categories = ProductCategory.where("product_categories.lang_" + I18n.locale.to_s + " = ?", true)
#products = Product.where("product_categories.lang_" + I18n.locale.to_s + " = ?", true)
I skipped any includes on purpose above - it is just to demonstrate the language logic.
UPDATE
The system have spent a lot of times to loop data in nested loop. Avoid to fetch data in nested loop. You have to use join or includes to catch your data more effective. For example:
Controller
#products = Product.includes(:product_category).where("product_categories.lang_" + I18n.locale.to_s + " = ? AND product_categories.active = ?", true, true).group_by(&:category)
View
- #products.each do |category, products|
<%= category.name %>
- products.each do |product|
<%= product.title %>
It needs to fix with your necessary code. I just help the main query. For example: active = ? is for products field or product_categories field. I hope It can help you.

Converting rails query from Mysql to PostgreSQL

Can anyone tell me how to convert this query to PostgreSQL
routes_controller.rb
#routes = Route.joins([:departure_location, :destination_location]).where("mdm_locations.name like ? or destination_locations_mdm_routes.name like ?" , "%#{k}%", "%#{k}%")
routes.rb (model)
module Mdm
class Route < ActiveRecord::Base
belongs_to :uom
belongs_to :distance_uom, :class_name => "Uom", :foreign_key => "distance_uom_id"
belongs_to :location
belongs_to :departure_location, :class_name => "Location", :foreign_key => "departure"
belongs_to :destination_location, :class_name => "Location", :foreign_key => "destination"
has_many :voyages, :dependent => :restrict
attr_accessible :description, :distance, :distance_uom_id, :departure, :std_consm, :destination, :uom_id
validates_presence_of :departure, :destination
end
end
Error :
PG::Error: ERROR: operator does not exist: integer = character varying
LINE 1: ...NNER JOIN "mdm_locations" ON "mdm_locations"."id" = "mdm_rou...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
: SELECT COUNT(*) FROM "mdm_routes" INNER JOIN "mdm_locations" ON "mdm_locations"."id" = "mdm_routes"."departure" INNER JOIN "mdm_locations" "destination_locations_mdm_routes" ON "destination_locations_mdm_routes"."id" = "mdm_routes"."destination" WHERE (LOWER(mdm_locations.name) like '%futong%' or LOWER(destination_locations_mdm_routes.name) like '%futong%')
Your error message says:
operator does not exist: integer = character varying
and points you at this part of the SQL:
INNER JOIN "mdm_locations" ON "mdm_locations"."id" = "mdm_routes"."departure"
-- ------------------------------------------------^
Combining those tells us that mdnm_locations.id is an integer (as expected) but mdm_routes.departure is a varchar. You can't compare integers and strings in SQL without explicitly casting one of them to make the types compatible.
You need to fix your schema, mdm_routes.departure should be an integer column, not a string.
MySQL tries to be friendly by attempting to guess your intent and lets you get away with a lot of sloppy practices. PostgreSQL tries to be friendly by forcing you to say exactly what you mean to avoid confusions, incorrect guesses, and hidden bugs.

MySQL / Ruby on Rails - How to "SUM" in a :has_many case

I have the following tables:
User :has_many Purchases
Item :has_many Purchases
Item has a column "amount" (can be + or -) and I need to find all Users that have a positive SUM of "Item.amounts" (over all Purchases each one has made).
How does this query look like? (I'm not sure how to handle "SUM" correctly, in this case.)
I started out with the following, but obviously, it's wrong... (it wouldn't "include" Purchases that have an Item with a negative Item.amount...)
#users = User.find(:all,
:include => {:purchases => :item},
:select => "SUM(item.amount)",
:order => "...",
:conditions => "...",
:group => "users.id",
:having => "SUM(item.amount) > 0"
)
Thanks for your help with this!
Tom
Try this:
User.all(:joins => items, :group => "users.id",
:having => "SUM(items.amount) > 0")
It sounds like this is a good case for some model methods.
I didn't test this but I think you want to do something similar to the following:
class User < ActiveRecord::Base
has_many :purchases
has_many :items, :through => :purchases
def items_total
#get all the items iterate over them to get the amount,
#compact to get rid of nils
#and reduce with a sum function to total and return
items.all.each{|item| item.amount}.compact.reduce(:+)
end
then
User.items_total

Multiple column foreign keys / associations in ActiveRecord/Rails

I have badges (sorta like StackOverflow).
Some of them can be attached to badgeable things (e.g. a badge for >X comments on a post is attached to the post). Almost all come in multiple levels (e.g. >20, >100, >200), and you can only have one level per badgeable x badge type (= badgeset_id).
To make it easier to enforce the one-level-per-badge constraint, I want badgings to specify their badge by a two-column foreign key - badgeset_id and level - rather than by primary key (badge_id), though badges does have a standard primary key too.
In code:
class Badge < ActiveRecord::Base
has_many :badgings, :dependent => :destroy
# integer: badgeset_id, level
validates_uniqueness_of :badgeset_id, :scope => :level
end
class Badging < ActiveRecord::Base
belongs_to :user
# integer: badgset_id, level instead of badge_id
#belongs_to :badge # <-- how to specify?
belongs_to :badgeable, :polymorphic => true
validates_uniqueness_of :badgeset_id, :scope => [:user_id, :badgeable_id]
validates_presence_of :badgeset_id, :level, :user_id
# instead of this:
def badge
Badge.first(:conditions => {:badgeset_id => self.badgeset_id, :level => self.level})
end
end
class User < ActiveRecord::Base
has_many :badgings, :dependent => :destroy do
def grant badgeset, level, badgeable = nil
b = Badging.first(:conditions => {:user_id => proxy_owner.id, :badgeset_id => badgeset,
:badgeable_id => badgeable.try(:id), :badgeable_type => badgeable.try(:class)}) ||
Badging.new(:user => proxy_owner, :badgeset_id => badgeset, :badgeable => badgeable)
b.level = level
b.save
end
end
has_many :badges, :through => :badgings
# ....
end
How I can specify a belongs_to association that does that (and doesn't try to use a badge_id), so that I can use the has_many :through?
ETA: This partially works (i.e. #badging.badge works), but feels dirty:
belongs_to :badge, :foreign_key => :badgeset_id, :primary_key => :badgeset_id, :conditions => 'badges.level = #{level}'
Note that the conditions is in single quotes, not double, which makes it interpreted at runtime rather than loadtime.
However, when trying to use this with the :through association, I get the error undefined local variable or method 'level' for #<User:0x3ab35a8>. And nothing obvious (e.g. 'badges.level = #{badgings.level}') seems to work...
ETA 2: Taking EmFi's code and cleaning it up a bit works. It requires adding badge_set_id to Badge, which is redundant, but oh well.
The code:
class Badge < ActiveRecord::Base
has_many :badgings
belongs_to :badge_set
has_friendly_id :name
validates_uniqueness_of :badge_set_id, :scope => :level
default_scope :order => 'badge_set_id, level DESC'
named_scope :with_level, lambda {|level| { :conditions => {:level => level}, :limit => 1 } }
def self.by_ids badge_set_id, level
first :conditions => {:badge_set_id => badge_set_id, :level => level}
end
def next_level
Badge.first :conditions => {:badge_set_id => badge_set_id, :level => level + 1}
end
end
class Badging < ActiveRecord::Base
belongs_to :user
belongs_to :badge
belongs_to :badge_set
belongs_to :badgeable, :polymorphic => true
validates_uniqueness_of :badge_set_id, :scope => [:user_id, :badgeable_id]
validates_presence_of :badge_set_id, :badge_id, :user_id
named_scope :with_badge_set, lambda {|badge_set|
{:conditions => {:badge_set_id => badge_set} }
}
def level_up level = nil
self.badge = level ? badge_set.badges.with_level(level).first : badge.next_level
end
def level_up! level = nil
level_up level
save
end
end
class User < ActiveRecord::Base
has_many :badgings, :dependent => :destroy do
def grant! badgeset_id, level, badgeable = nil
b = self.with_badge_set(badgeset_id).first ||
Badging.new(
:badge_set_id => badgeset_id,
:badge => Badge.by_ids(badgeset_id, level),
:badgeable => badgeable,
:user => proxy_owner
)
b.level_up(level) unless b.new_record?
b.save
end
def ungrant! badgeset_id, badgeable = nil
Badging.destroy_all({:user_id => proxy_owner.id, :badge_set_id => badgeset_id,
:badgeable_id => badgeable.try(:id), :badgeable_type => badgeable.try(:class)})
end
end
has_many :badges, :through => :badgings
end
While this works - and it's probably a better solution - I don't consider this an actual answer to the question of how to do a) multi-key foreign keys, or b) dynamic-condition associations that work with :through associations. So if anyone has a solution for that, please speak up.
Seems like it might workout best if you separate Badge into two models. Here's how I'd break it down to achieve the functionality you want. I threw in some named scopes to keep the code that actually does things clean.
class BadgeSet
has_many :badges
end
class Badge
belongs_to :badge_set
validates_uniqueness_of :badge_set_id, :scope => :level
named_scope :with_level, labmda {|level
{ :conditions => {:level => level} }
}
named_scope :next_levels, labmda {|level
{ :conditions => ["level > ?", level], :order => :level }
}
def next_level
Badge.next_levels(level).first
end
end
class Badging < ActiveRecord::Base
belongs_to :user
belongs_to :badge
belongs_to :badge_set
belongs_to :badgeable, :polymorphic => true
validates_uniqueness_of :badge_set_id, :scope => [:user_id, :badgeable_id]
validates_presence_of :badge_set_id, :badge_id, :user_id
named_scope :with_badge_set, lambda {|badge_set|
{:conditions => {:badge_set_id => badge_set} }
}
def level_up(level = nil)
self.badge = level ? badge_set.badges.with_level(level).first
: badge.next_level
save
end
end
class User < ActiveRecord::Base
has_many :badgings, :dependent => :destroy do
def grant badgeset, level, badgeable = nil
b = badgings.with_badgeset(badgeset).first() ||
badgings.build(
:badge_set => :badgeset,
:badge => badgeset.badges.level(level),
:badgeable => badgeable
)
b.level_up(level) unless b.new_record?
b.save
end
end
has_many :badges, :through => :badgings
# ....
end