I am new to Ruby on Rails. Now I am working on performance issues of a Rails application. I am using New Relic rpm to find out the bottlenecks of the code. While doing this I find something that I cannot figure out. The problem is that here in my Rails application I have used two models A, B and C where model B has two properties: primary key of A and primary key of C like following:
class B
include DataMapper::Resource
belongs_to :A, :key=>true
belongs_to :C, :key=>true
end
Model of A is as follows:
class A
include DataMapper::Resource
property :prop1
...
has n, :bs
has n, :cs, :through => :bs
end
While issuing the following statement a.find(:c.id=>10) then internally it is executing the following SQL query:
select a.prop1, a.prop2,... from a INNER JOIN b on a.id = b.a_id INNER JOIN c on b.c_id = c.id where (c.id=10) GROUP BY a.prop1, a.prop2,....[here in group by all the properties that has been mentioned in select appears, I don't know why]
And this statement is taking too much time during web transaction. Interesting thing is that, when I am executing the same auto generated query in mysql prompt of my terminal it's taking very less amount of time. I think it's because of mentioning so many fields in group by clause. I cannot understand how the query is being formed. If anyone kindly help me to figure this out and optimize this, I will be really grateful. Thank you.
I assume you have you model associations properly configured, something like this:
class A < ActiveRecord
has_many :B
has_many :C, through: :B
end
class B < ActiveRecord
belongs_to :A
belongs_to :C
end
class C < ActiveRecord
has_many :B
has_many :A, through: :B
end
then you could simply call:
a.c.find(10) #mind the plural forms though
You will get better performance this way.
Related
I am building an rails 5 app that connects to 2 different databases (dbA & dbB). My databases are on the same database host.
I want to make a wishlist. Pretty easy when using the same DB, but I am stuck with an "interesting" error.
This is what the databases look like:
the models are as follow:
user.rb
class User < ApplicationRecord
has_one :wishlist, dependent: :destroy
end
wishlist.rb
class Wishlist < ApplicationRecord
belongs_to :user
# has_and_belongs_to_many :wines
# The above did not work
# so I had to revert to has_many through
has_many :wines_wishlists
has_many :wines, through: :wines_wishlists
end
wines_wishlist.rb
class WinesWishlist < ApplicationRecord
belongs_to :wine
belongs_to :wishlist
def self.table_name_prefix
"dbA_#{Rails.env}."
end
# I added the above to prevent ActiveRecord from
# looking for the table in the wrong database
end
wine.rb (legacy model)
class Wine < ApplicationRecord
self.abstract_class = true
establish_connection LEGACY_DB
# LEGACY_DB is the legacy database connection info from a yaml file
# located in config.
def self.table_name_prefix
"dbB_#{Rails.env}."
end
end
This is quite straigth forward IMHO. Now the interresting error:
When I try the following :
user = User.last
user.wishlist.wines
It works on my local machine in development. It doesn't work on my staging server! When I try in the rails console, I get this:
ActiveRecord::StatementInvalid: Mysql2::Error: Table 'dbA_staging.wines_wishlists' doesn't exist: SELECT `dbB_staging`.`wines`.* FROM `dbB_staging`.`wines` INNER JOIN `dbA_staging`.`wines_wishlists` ON `dbB_staging`.`wines`.`id` = `dbA_staging`.`wines_wishlists`.`wine_id` WHERE `dbA_staging`.`wines_wishlists`.`wishlist_id` = 1
This is the expected SQL.
user.wishlist.wines.to_sql
=> "SELECT `dbB_staging`.`wines`.* FROM `dbB_staging`.`wines` INNER JOIN `dbA_staging`.`wines_wishlists` ON `dbB_staging`.`wines`.`id` = `dbA_staging`.`wines_wishlists`.`wine_id` WHERE `dbA_staging`.`wines_wishlists`.`wishlist_id` = 1"
Even better, when I try the same SQL in rails db on my staging machine, it works!! It doesn't work in rails even though the SQL is correct, but it works in mysql command line.
I based my code on the following article and made some research, but I can't seem to figure out how to go around this problem.
I am using (same for development and staging):
Rails 5.1.1
ruby 2.4.0p0
mysql 5.6.34 (staging)
mysql 5.7.17 (development)
Any help would be greatly appreciated!
Taking a look at the article you linked to, it seems to be using a gem st-elsewhere, i.e.
has_many_elsewhere :wines, :through => :wines_wishlist
Also, as stated in the article, you can't make JOIN queries across database connections. The gem circumvents this using some less efficient queries, the details of which I did not look up.
Plugins Model:
class Plugin < ActiveRecord::Base
belongs_to :report
has_many :vulns
end
Vulns Model:
class Vuln < ActiveRecord::Base
belongs_to :plugins
end
I'm doing the following in rails:
#using * for now to select everything
#data = Plugin.select("*").joins(:vulns).where('plugins.id'=> plugin.plugin_id)
Which does the following query in the terminal:
SELECT * FROM `plugins` INNER JOIN `vulns` ON `vulns`.`plugin_id` = `plugins`.`id` WHERE `plugins`.`id` = 186
It's the right query but it doesn't select the content from the vulns table. I know it's the right query because I tried it in phpmyadmin and it returned the data on the vulns table too. When I do it in rails (using <%= debug(#data) %>) it only shows content from the plugins table.
How do I make it select everything from the vulns table too? (Each plugin has multiple vulns)
Apparently you can't have a column with the name "type".
If anybody has similar issues, rename the column called "type". I generated a new migration:
rails g migration RenameColumnOnVulnsTable
And then added the following:
def change
rename_column :vulns, :type, :vulnerability_type
end
The query works fine now.
I'm using Rails 3.2 with ActiveRecord and MySQL and I have models with one to many association:
class Author
has_many :books
end
class Book
belongs_to :author
attr_accessible :review
end
I want to find authors that have all the books without review. I tried:
Author.includes(:books).where('book.review IS NIL')
but is obviously didn't work, because it finds authors that have at least one book without review. What query should I use?
SQL is quite simple:
SELECT authors.name, count(books.review is not null)
FROM authors LEFT JOIN books ON (authors.id=books.author_id)
GROUP BY authors.name
HAVING count(books.review) == 0
Translating it to the AR query language may take me some time...
OK, so it seems to look like this:
Author.count('books.review', joins: :books, select: 'name',
group:'name', having: 'count_books_review=0')
As for me SQL looks much less weird then this ;-)
Basing on the WRz answer I prepared my own query:
Author.joins(:books).group('authors.id').having("count(books.reviews)=0")
It's better suited for me, because it returns an AR Relation (and WRz's query returns a Hash).
Try this
Author.joins(:books).where('books.review is null')
edit: This will fetch all the authors with at least one book with no review. I just realized your question is a bit different.
It would be something like this.
Authors.joins(:books).select('authors.*, count(books.id) as
total_books, count('books.review is null')
as books_without_review.group('authors.id').having(total_books ==
books_without_review)
P.S: This is not the exact syntax and it is untested
Try the following code.
class Author
has_many :books
end
class Book
belongs_to :author
attr_accessible :review
end
authors = Author.all.collect do |author|
if author.books.where(:review => nil).size == author.books.size
author
end
end
authors.compact!
After this code, authors will be an array containing all the authors having all the books unreviewed. Also note that I changed the author association in Book model to belongs_to instead of has_one. It is always a good practice to have has_many relation on one side and belongs_to association on the other side.
I'm trying to do a multi-table join that has a NOT IN component. Tables are
Post -> Term Relationship -> Term
Post
has_many :term_relationships
has_many :terms, :through => :term_relationships
TermRelationship
belongs_to :post
belongs_to :term
Term
has_many :term_relationships
has_many :posts, :through => :term_relationships
The goal is to get all posts except for those in "featured" let's say. My current query would looks like:
WpPost.includes(:terms).where("terms.term NOT IN (?)", ["featured"])
This works great if the only term that it has attached is "featured". If the post belongs to "featured" and "awesome" it will still show because of "awesome".
Anyway to exclude a row entirely? Will it require a subquery? And if it does, how would I go about doing that in rails?
Thanks all!
Justin
You misuse the includes. It's for eager loading, not for joining!
But you're right about the approach. It can be used in your case. But Rails won't issue nested request for NOT IN (?) even if it would be logical. You'll get 2 queries instead (you'll get NOT IN (id1, id2....,) instead of NOT IN (SELECT ....)).
So I would recommend you to use the squeel gem:
regular AR code (can also be prettified with squeel):
featured_posts = WpPost.joins(:terms).where(terms:{term: ['featured']}).uniq
and then use the sqeel's power:
WpPost.where{id.not_in featured_posts}
(in and not_in are also aliased as >> and << but I didn't want to scary anybody)
Note the using blocks and absence of symbols.
Some measurements based on Chinook Database under SQLite:
> Track.all
Track Load (35.0ms) SELECT "Track".* FROM "Track"
Relation with joins and like:
oldie = Track.joins{playlists}.where{playlists.name.like_any %w[%classic% %90%]}
Here's NOT IN:
> Track.where{trackId.not_in oldie}.all
Track Load (37.5ms) SELECT "Track".* FROM "Track" WHERE "Track"."trackId"
NOT IN (SELECT "Track"."TrackId" FROM "Track" INNER JOIN "PlaylistTrack" ON
"PlaylistTrack"."TrackId" = "Track"."TrackId" INNER JOIN "Playlist" ON
"Playlist"."PlaylistId" = "PlaylistTrack"."PlaylistId"
WHERE (("Playlist"."name" LIKE '%classic%' OR "Playlist"."name" LIKE '%90%')))
FYI:
Track.where{trackId.not_in oldie}.count # => 1971
Track.count # => 3503
# join table:
PlaylistTrack.count # => 8715
Conclusion: I don't see the overhead caused by NOT IN. 35.0 vs 37.5 isn't noticeable difference. Few times 35.0 became 37.5 and vice verse.
One option is to do an OUTER JOIN and put the featured argument there. Then you just select all posts where no term was joined. I don't know any way of doing it in a plain "Rails way" but with some extra SQL you could do it like this:
Post.joins("LEFT OUTER JOIN term_relationships ON posts.id = term_relationships.post_id
LEFT OUTER JOIN terms ON term_relationships.term_id = terms.id AND terms.term = ?", "featured").
where("terms.id IS NULL")
This query executes just fine:
p = PlayersToTeam.select("id").joins(:player).limit(10).order("players.FirstName")
This query causes my whole system to come to a screeching halt:
p = PlayersToTeam.select("id").includes(:player).limit(10).order("players.FirstName")
Here are the models:
class PlayersToTeam < ActiveRecord::Base
belongs_to :player
belongs_to :team
accepts_nested_attributes_for :player
end
class Player < ActiveRecord::Base
has_many :players_to_teams
has_many :teams, through: :players_to_teams
end
As far as I can tell, the includes does a LEFT JOIN and joins does an INNER JOIN. The query spit out (for joins) from Rails is:
SELECT players_to_teams.id FROM `players_to_teams` INNER JOIN `players` ON `players`.`id` = `players_to_teams`.`player_id` ORDER BY players.FirstName LIMIT 10
Which executes just fine on the command line.
SELECT players_to_teams.id FROM `players_to_teams` LEFT JOIN `players` ON `players`.`id` = `players_to_teams`.`player_id` ORDER BY players.FirstName LIMIT 10
also executes just fine, it just takes twice as long.
Is there an efficient way I can sort the players_to_teams records via players? I have an index on FirstName for players.
EDIT
Turns out the query required heavy optimization to run even half decently. Splitting the query was the best solution short of restructuring the Data or customizing the query
You also might consider to split it into 2(3) queries. First - to get ids by sorting with joins:
players_to_teams = PlayersToTeam.select("id").joins(:player).limit(10).order("players.FirstName")
Second (which is inside contains 2 queries) - to get PlayersToTeams with players pre-loaded.
players_to_teams = PlayersToTeam.include(:player).where(:id => players_to_teams.map(&:id))
So after that you will have fully initialized players_to_teams with players loaded and initialized.
One thing to note is that include will add a second db access to do the preloading. You should check what that one looks like (it should contain a big IN statement on the player_ids from players_to_teams).
As for how to avoid using include, if you just need the name from players, you can do it like this:
PlayersToTeam.select("players_to_teams.id, players.FirstName AS player_name").joins(:player).limit(10).order("players.FirstName")