In my views I do a lot of this:
<% cache("sports_menu_" +session[:lang], {:expires_in => 60.minutes}) do %>
...... # a load of stuff
<% end %>
However I've discovered a lot of time is spent querying the DB for data that doesn't change that often. Is there a way to cache this data in a similar manner?
For instance:
Model.find(:all, :select => "only a few fields", :conditions => "nasty conditions", :include => "some joins", :order => "date_time desc")
This takes about 7 seconds, the main table keeps about 20M records. A lot of users hit this particular action and the query only runs once/hit. But it would make sense caching that for a number of minutes so that for everyone else it will load from the cache. I'm using memcache by the way.
I can't cache the entire action because there are some parameters that change on occasion and some locale-specific code within the view.
I have considered moving that to the view level but don't feel too comfortable about that, it would kind of defeat the point of using Rails.
TIA!
It looks to me like ActiveRecord::Cache::Store is what you want. You can use it like this.
Related
I'm trying to set up an ability to get some numbers from my Sphinx indexes, but not sure how to get the info I want.
I have a mysql db with articles, sphinx index set up for that db and full text search, all working. What I want is to get some numbers:
How many times search text (keyword, or key phrase) appears over all articles for all time (more likely limited to "articles from time interval from X and to Y")
Same as previous but for how many times 2 keywords or keyphrases (so "x AND y") appear in same articles
I was doing something similar to first manually using bat file I made
indexer ind_core -c c:\%SOME_PATH%\development.sphinx.conf --buildstops stats.txt 10000 --buildfreqs
Which generated me a txt with all repeating keywords and how often they appear at early development stages, which helped to form a list of keywords I'm interested in. Now I'm trying to do the same but just for a finite list of predetermined keywords and integrated into my rails project to be able to build charts in future.
I tried running some queries like
#testing = Article.search 'Keyword1 AND Keyword2', :ranker => :wordcount
but I'm not sure how it works and how to process the result, as well as if that's what I'm looking for.
Another approach I tried was manual mysql queries such as
SELECT id,title,WEIGHT() AS w FROM ind_core WHERE MATCH('#title keyword1 | keyword2') OPTION ranker=expr('sum(hit_count)');
but I'm not sure how to process results from here either (as well as how to actually implement it into my existing rails project), and it's limited to 20 lines per query (which I think I can change somewhere in settings?). But at least looking at mysql results what I'm interested in is hit_count over all articles (or all articles from set timeframe).
Any ideas on how to do this?
UPDATE:
Current way I found was to add
#testing = Article.search params[:search], :without => {:is_active => false}, :ranker => :bm25
to controller with some conditions (so it doesn't bug out from nil search). :is_active is my soft delete flag, don't want to search deleted entries, so don't mind it. And in view I simply displayed
<%= #testing.total_entries %>
Which if I understand it correct shows me number of matches sphinx found (so pretty much what I was looking for).
So, to figure out the number of hits per document, you're pretty much on the right track, it's just a matter of getting it into Ruby/Thinking Sphinx.
To get the raw Sphinx results (if you don't need the ActiveRecord objects):
search = Article.search "foo",
:ranker => "expr('SUM(hit_count)')",
:select => "*, weight()",
:middleware => ThinkingSphinx::Middlewares::RAW_ONLY
… this will return an array of hashes, and you can use the weight() string key for the hit count, and the sphinx_internal_id string key for the model's primary key (id is Sphinx's own primary key, which isn't so useful).
Or, if you want to use the ActiveRecord objects, Thinking Sphinx has the ability to wrap each search result in a helper object which passes appropriate methods through to the underlying model instances, but lets weight respond with the values from Sphinx:
search = Article.search "foo",
:ranker => "expr('SUM(hit_count)')",
:select => "*, weight()"; ""
search.context[:panes] << ThinkingSphinx::Panes::WeightPane
search.each do |article|
puts article.weight
end
Keep in mind that panes must be added before the search is evaluated, so if you're testing this in a Rails console, you'll want to avoid letting the console inspect the search variable (which I usually do by adding ; "" at the end of the initial search call.
In both of these cases, as you've noted, the search results are paginated - you can use the :page option to determine which page of results you want, and :per_page to determine the number of records returned in each request. There is a standard limit of 1000 results overall, but that can be altered using the max_matches setting.
Now, if you want the number of times the keywords appear across all Sphinx records, then the best way to do that while also taking advantage of Thinking Sphinx's search options, is to get the raw results of an aggregate SUM - similar to the first option above.
search = Article.search "foo",
:ranker => "expr('SUM(hit_count)')",
:select => "SUM(weight()) AS count",
:middleware => ThinkingSphinx::Middlewares::RAW_ONLY
search.first["count"]
I have a Model called Person and Person has multiple posts. When I want to query post count for each person it takes a long time to process since it needs to iterate over each person and query each posts to get the aggregation.
class Person < ActiveRecord::Base
has_many :posts
end
Output (JSON):
Person1
PostsType1Count: 15
PostsType2Count: 45
Person2
PostsType3Count: 33
.
.
.
I want to calculate all the post count for each Person in a optimum way. What would be the best solution?
Here's one way to do this, if you have a small and pre-defined set of Types
class Person < ActiveRecord::Base
has_many :type_1_posts, :class_name => 'Post', :conditions => 'post_type = 1'
has_many :type_2_posts, :class_name => 'Post', :conditions => 'post_type = 2'
has_many :type_3_posts, :class_name => 'Post', :conditions => 'post_type = 3'
end
Then you can write code that looks like this to get all the data:
#all_people = Person.includes(:type_1_posts, :type_2_posts, :type_3_posts).all
The eager loading of the posts allows the count of each type of post to be available, as well as all the posts of each type.
If you need extra performance for this code, because you perform this query a lot, then you can look into using the Rails counter cache mechanism to keep track of the counts of each type on the Person object.
The beauty of Rails here is that your main display code doesn't need to change during this process of making the code faster for reading (adding a counter cache makes adding/deleting posts slower, so you may not want it in all cases).
Write initial code
Use eager loading to make it faster
Use counter cache to make it even faster
Try this May it will work for you
#In Controller
#persons = Person.all
#In View
#persons.each do |person|
person.posts.count # It will gives all posts count
person.posts.collect{|x| true if x.type==1 }.compact.count #If you want to get the post counts based on type
end
Suppose if you want to get any mehods just check in console or debug is person.methods.sort it will give all methods.
try in rails console person.posts.methods also it will give types also then check counts based on type. because i dont know which fields in posts model. so check it.
I've got a Rails 3.1 app running a mysql server for storing data.
90% of the data in the app fits really well in a relational database.
The other 10% is a pretty large hash that I need to pull out, change, and put back fairly quickly. It is a fairly large query in mysql to bring all these data pieces together, across multiple tables, but once I have it once, I figured I would save it as a hash, and the user can interact with the hash and make changes. Those changes never get persisted back to mysql, as mysql doesn't need them.
so, I decided to add redis to my rails application and the redis-objects gem was recommended by a friend.
I have created my active_hash model and controller as so
class ActiveHash < ActiveRecord::Base
include Redis::Objects
end
class ActiveHashesController < ApplicationController
def show
#this is a big query with a bunch of merges, but simplified here as it isn't important
active = Game.find(params[:id])
active_hash_in_redis = ActiveHash.new()
if active_hash_in_redis.save
render :json => active_hash
else
render :text => "didn't save"
end
end
end
when I navigate to active_hashes/id, I get an error that there is no MySQL table active_hashes, which is right, because that is supposed to be my redis db, as defined in the model.
can anybody explain to me how to use both dbs in my app, and/or point me to a tutorial on doing this? I haven't been able to find anything. Is using Redis-Objects the wrong way to go with this?? Any other recommendations?
It turns out this was a bit of confusion on my part, but hopefully this helps somebody else.
I didn't end up using the redis-objects, gem, I installed redis-rb with gem redis.
Then I set-up the config file as
require 'redis'
$redis = Redis.new()
My model is actually blank at the moment, in my controller, I've used
class ActiveHashesController < ApplicationController
def show
#this is a big query with a bunch of merges, but simplified here as it isn't important
active = Game.find(params[:id])
$redis.set params[:id], active.to_json
get_game = $redis.get params[:id]
render :json => get_game
end
end
end
I have a long complicated home page where a company is shown, for each project, information about recent events. The idea is that they have a kind of data-heavy information center from which they can monitor all activity.
I've had trouble getting this page to perform well - two days ago local load times were 4.5s(!) and they are currently at ~2.5s(!). The most alarming part about this horrible performance is that these are the load times with only 3 projects and practically no events. Performance on the live app is slightly better, but not nearly enough.
Purpose: Improve load time on home page
Here are the current queries.
# controller
#projects = #company.projects.order("project_title ASC").includes({:events => :owner}).search(params[:search], params[:page])
# view
#projects.each do |project|
#events = project.events.where(:active => true).includes(:owner).order("priority DESC")
end
Removing the .where(:active => true).includes(:owner).order("priority DESC") is shaving off 1.1 seconds on an app with only 3 projects and 4 events in total.
How should these queries be written optimally? Should indexing play a role in this case?
I've been playing around with database indexes for the looped query in the view but I haven't gotten one to cut down the time yet.
Your .includes(:events => :owners) is not doing what you think, as when you call .where on events later you have to retrieve from the data base again.
Also, if your search method is using the events and owners table you may want to used .joins() instead of .includes().
I would make sure you have indexes on every foreign key (xxx_id) and on events active.
I would also give this a shot (not sure if it works, may need some tweaking):
class Project < AR::Base
has_many :events
has_many :active_events,
:class_name => 'Event',
:conditions => {:active => true},
:order => "events.priority DESC"
:include => :owner
end
#in controller:
#projects = #company.projects.order("project_title ASC").includes(:active_events).search(...)
#in view: (abstract this to a render collection method if possible)
#project.each do |project|
#events = project.active_events
end
I am working on a rails project and am having some issues with the following join:
#page = Page.find(params[:id], :joins => "LEFT JOIN page_translations ON page_translations.page_id = pages.id")
For some reason its only pulling back everything from the Pages table.
Here is my model for Page
class Page < ActiveRecord::Base
has_many :users_pages
has_many :users, :through => :users_pages
has_many :page_translations
has_many :categories
accepts_nested_attributes_for :page_translations
accepts_nested_attributes_for :categories
end
Here is my model for PageTranslation
class PageTranslation < ActiveRecord::Base
belongs_to :pages
end
Thanks in advance for all of the help!
Edit (#thenduks)
The log runs two separate queries:
Page Load (0.5ms) SELECT `pages`.* FROM `pages` WHERE (`pages`.`id` = 1) LIMIT 1
PageTranslation Load (0.5ms) SELECT `page_translations`.* FROM `page_translations` WHERE (`page_translations`.page_id = 1)
Here is what my controller looks like:
#page = Page.find(params[:id], :include => :page_translations)
I was stumped about this same thing and wasted a few hours trying to figure it out. It turns out that using the joins method of the query interface doesn't initialize the models related to the tables being joined. You can see this by watching the SQL statements in the server console, or by even redirecting ActiveRecord logging to STDOUT in your Rails console.
I was very disappointed by this. It just doesn't seem like how the joins method should work -- it certainly wasn't what I was expecting. I was expecting it to eager load, since it was in the eager load section of the Edge Guides.
Anyway, I couldn't waste any more time trying to figure it out, so what I did instead is use the fancy query interface to simply build my query, used to_sql to get the SQL for my query, and then passed the SQL to select_all, which returns an array of hashes, where each element in the array (each hash) represents a row.
Example:
query = Post.joins("LEFT JOIN categories ON post.category_id = categories.id")
query.select("posts.*, category.category_name")
con = ActiveRecord::Base.connection
results = con.select_all(query.to_sql)
Results:
[{"id": 1, "title": "joins, why have you forsaken me", "category_name": "frustration"},{"id": 2, "title": "pizza", "category_name": "food"}]
To be honest, I would still like to know for certain if it is possible to do it the way we think it should work, or the way it ought to work. Otherwise, I see no reason for having a joins method, other than to help us build the query. So if any Rails gurus out there know how to use joins to populate models related to those tables, PLEASE LET ME (US) KNOW!
Anyway, I hope this helps you move along for now.
UPDATE: So I think I just figured it out. I stumbled across this blog post. As it turns out, when using the joins method of the query interface, Rails will actually add the columns you selected from the joined tables as attribute methods of the model being joined against.
Using the same example above, I can actually access the category_name of each post by simply calling post.category_name. #$%! Unbelievably simple, but no documentation whatsoever on this!
Here it is once again:
query = Post.joins("LEFT JOIN categories ON post.category_id = categories.id")
query.select("posts.*, category.category_name")
posts = query.all
# access a post's category name
puts posts[0].category_name
# this is what I thought I would be able to do
# without Rails making another query to the database
puts posts[0].category.category_name
I hope this helps! :)
How about:
Page.find( params[:id], :include => :page_translations )
Edit:
Ok, so some time recently the behavior of ActiveRecord when it comes to joins/includes seems to have changed. The guides still refer to being able to do this though two associations, as in has_many :orders, :include => :line_items and similar... but as far as including records from a has_many... After consulting with a co-worker we came across some info on the subject. Seems that the single monolithic queries were just getting too complex and ugly and it was causing problems for some of the fancier niceties that ActiveRecord gives you and duplicate rows, that kind of thing.
TL;DR: It doesn't work like that anymore. 2 queries is expected.