How to remove duplicates in MySQL using Rails? - mysql

Question

I have to following table:
Relations
[id,user_id,status]
1,2,sent_reply
1,2,sent_mention
1,3,sent_mention
1,4,sent_reply
1,4,sent_mention
I am looking for a way to remove duplicates, so that only the following rows will remain:
1,2,sent_reply
1,3,sent_mention
1,4,sent_reply
(Preferably using Rails)

Answer 1

I know this is way late, but I found a good way to do it using Rails 3. There are probably better ways, though, and I don't know how this will perform with 100,000+ rows of data, but this should get you on the right track.
# Get a hash of all id/user_id pairs and how many records of each pair
counts = ModelName.group([:id, :user_id]).count
# => {[1, 2]=>2, [1, 3]=>1, [1, 4]=>2}
# Keep only those pairs that have more than one record
dupes = counts.select{|attrs, count| count > 1}
# => {[1, 2]=>2, [1, 4]=>2}
# Map objects by the attributes we have
object_groups = dupes.map do |attrs, count|
ModelName.where(:id => attrs[0], :user_id => attrs[1])
end
# Take each group and #destroy the records you want.
# Or call #delete instead to save time if you don't need ActiveRecord callbacks
# Here I'm just keeping the first one I find.
object_groups.each do |group|
group.each_with_index do |object, index|
object.destroy unless index == 0
end
end

Answer 2

It is better to do it through SQL. But if you prefer to use Rails:
(Relation.all - Relation.all.uniq_by{|r| [r.user_id, r.status]}).each{ |d| d.destroy }
or
ids = Relation.all.uniq_by{|r| [r.user_id, r.status]}.map(&:id)
Relation.where("id IS NOT IN (?)", ids).destroy_all # or delete_all, which is faster
But I don't like this solution :D