I've got a Rails 3.1 app running a mysql server for storing data.
90% of the data in the app fits really well in a relational database.
The other 10% is a pretty large hash that I need to pull out, change, and put back fairly quickly. It is a fairly large query in mysql to bring all these data pieces together, across multiple tables, but once I have it once, I figured I would save it as a hash, and the user can interact with the hash and make changes. Those changes never get persisted back to mysql, as mysql doesn't need them.
so, I decided to add redis to my rails application and the redis-objects gem was recommended by a friend.
I have created my active_hash model and controller as so
class ActiveHash < ActiveRecord::Base
include Redis::Objects
end
class ActiveHashesController < ApplicationController
def show
#this is a big query with a bunch of merges, but simplified here as it isn't important
active = Game.find(params[:id])
active_hash_in_redis = ActiveHash.new()
if active_hash_in_redis.save
render :json => active_hash
else
render :text => "didn't save"
end
end
end
when I navigate to active_hashes/id, I get an error that there is no MySQL table active_hashes, which is right, because that is supposed to be my redis db, as defined in the model.
can anybody explain to me how to use both dbs in my app, and/or point me to a tutorial on doing this? I haven't been able to find anything. Is using Redis-Objects the wrong way to go with this?? Any other recommendations?
It turns out this was a bit of confusion on my part, but hopefully this helps somebody else.
I didn't end up using the redis-objects, gem, I installed redis-rb with gem redis.
Then I set-up the config file as
require 'redis'
$redis = Redis.new()
My model is actually blank at the moment, in my controller, I've used
class ActiveHashesController < ApplicationController
def show
#this is a big query with a bunch of merges, but simplified here as it isn't important
active = Game.find(params[:id])
$redis.set params[:id], active.to_json
get_game = $redis.get params[:id]
render :json => get_game
end
end
end
Related
I have to update a lot of data to mysql (~100Mio records!). Some records already exists, some have to be created. I also have to create some nested resources for each record.
I know the activerecord-import gem but as far as i know it can't handle nested records (or only with ugly workarounds). The issue is that I dont know the ID's for all nested records before they are created - and creating them in single queries takes time.
So lets say there is a model called Post and can have many Comments. My current code looks like this:
Post.transaction do
import_posts.each do |import_post|
post = Post.find_or_initialize_by(somevalue: import_post['somevalue']
post.text = import_post['text']
import_post['comments'].each do |import_comment|
comment = post.comments.find_or_initialize_by(someothervalue: import_comment['someothervalue'])
comment.text = import_comment['text']
end
post.save(validate: false) #Dont need validation - saves some time
end
end
This is just an example and it works but its far away from 'damn fast'. Are there any ideas how to speed up the data upload? Am I totally wrong?
Im working with Rails5 and ruby 2.4.
Thanks in advance!
I've got a really, really odd problem manifesting on a big Rails e-commerce app and thought I'd see if anyone has good insight. I have an"Order" model with many associations. If I create a new instance, and then set one particular column value and "save!" the "save!" is succeeding without errors, but the change isn't actually persisted to the DB. I'll run through the scenario below:
#order = Order.create!(<some attributes>)
=> true
#order.shipping_method_id
=> 1
#order.shipping_method_id = 203
=> 203
#order.save!
=> true
#order.shipping_method_id
=> 1
To try and debug this I actually prepended a before_save filter and I can see that when this first filter is called after setting the value, it is correct ("203") BUT the very next before_save after the 6-or-so built-in "autosave_foo_bar_quux" filters (for nested associations) it is back to "1".
Oddly, if I just reload the order (#order.reload), change the column value and save! the update does succeed.
In both cases, doing #order.changed shows that ActiveModel recognizes the column value change for shipping_method_id. In the first, though, the SQL logging shows that the order row is not updated.
I feel like I'm going insane. Any ideas? Also, let me know if there's anything else I can post here for more context.
I'm relative new to Rails, so I don't know if my way to solve the problem is the correct, but there's some problem with it
I have installed in my PC MySQL, but Heroku uses PostgreSQL, so I'm designing a solution to work with both DBMS in certain problem.
I have the next code:
begin
#products_with_valid_offers = Product.joins(:variants).active.discount_valid.discount_date_valid_mySQL
rescue ActiveRecord::StatementInvalid
#products_with_valid_offers = Product.joins(:variants).active.discount_valid.discount_date_valid_postgreSQL
end
And the scopes are:
scope :active, includes([:assets, :store]).where(:active => true, :deleted_at => nil, :stores => { :deleted_at => nil, :active => true }).order("products.created_at DESC")
scope :discount_date_valid_mySQL, where('DATE_FORMAT(NOW(),"%Y-%m-%d 23:59:59") + '" BETWEEN discount_from AND discount_to')
scope :discount_date_valid_postgreSQL, where('now()::date BETWEEN discount_from AND discount_to')
As you see, I need 2 different forms to manage the date format, one with each DBMS.
The issue is that the flow never enters in the exception. If #products_with_valid_offers is a SQL Error in MySQL, never enters to the rescue block to execute the PostgreSQL line, and it returns an error.
Some help, please?. :D
Your quotes are messed up in discount_date_valid_mySQL, you want to say this:
scope :discount_date_valid_mySQL, where("DATE_FORMAT(NOW(),'%Y-%m-%d 23:59:59') BETWEEN discount_from AND discount_to")
That at least is valid Ruby code.
That said, your current approach is bad in various ways:
You're ensuring that you always have an exception in production.
You're developing on top of one database but deploying on top of another, this alone will cause you all sorts of problems.
You should be using just date_format(now(), '%Y-%m-%d') in your MySQL version.
There are probably other issues but there's no need to spend more time picking apart your code: there is a better way. Both MySQL and PostgreSQL (and even SQLite) support current_date so you can just use one thing for both:
scope :discount_date_valid, where('current_date between discount_from and discount_to')
Assuming of course that you want everything to assume UTC. If you want things to use some other time zone then:
Store your discount_from and discount_to as date columns, dates don't have timezones in SQL. Presumably you're doing this already but I just want to make sure.
Adjust your scope to get the current date from your client code (which is presumably configured to use the right timezone):
def self.discount_date_valid
where(':current_date between discount_from and discount_to', :current_date => Date.today)
end
You have to use a class method for the scope to ensure that Date.today is evaluated at the right time (i.e. when the scope is used instead of when the class is loaded).
And you really should develop on top of PostgreSQL if you're deploying on top of PostgreSQL. You should even make sure you're developing and deploying on the same version of PostgreSQL. Developing on top of one stack and deploying on another causes all sorts of pointless frustration and confusion.
I have two models User and Company associated by has_and_belongs_to_many.
I can fetch all users belonging to a certain company using
Company.find(id).users
The problem I've is finding all users that DO NOT belong to a certain company. Well, I could do that using
User.all - Company.find(id).users
But, I feel there's certainly a better way to achieve this. Is there an activerecord solution to this ?
Currently, I have 8 users (id from 1 to 8). When I try,
User.joins(:companies).where('companies.id = ?', 13).map(&:id)
I get result [7, 8] which is as expected. When I place != in the above query, the results are not what I want, i.e. array of 1 to 6.
Need help.
Thanks.
The easiest way might be with a class method... something like:
class User
has_and_belongs_to_many :companies
class << self
def exclude_company(company)
User.scoped.reject! {|u| u.companies.include? company}
end
end
end
Then in your code:
#company = Company.find(company_id)
#users = User.exclude_company(#company)
YMMV; I've done similar things in the past, but the above code is untested.
Also, if this is a common query pattern, you would probably be better served by the extensions to ARel provided in MetaWhere and its companion MetaSearch. N.B. These are replaced by new gems by the same author in Rails 3.1
Hope this helps.
I am working on a rails project and am having some issues with the following join:
#page = Page.find(params[:id], :joins => "LEFT JOIN page_translations ON page_translations.page_id = pages.id")
For some reason its only pulling back everything from the Pages table.
Here is my model for Page
class Page < ActiveRecord::Base
has_many :users_pages
has_many :users, :through => :users_pages
has_many :page_translations
has_many :categories
accepts_nested_attributes_for :page_translations
accepts_nested_attributes_for :categories
end
Here is my model for PageTranslation
class PageTranslation < ActiveRecord::Base
belongs_to :pages
end
Thanks in advance for all of the help!
Edit (#thenduks)
The log runs two separate queries:
Page Load (0.5ms) SELECT `pages`.* FROM `pages` WHERE (`pages`.`id` = 1) LIMIT 1
PageTranslation Load (0.5ms) SELECT `page_translations`.* FROM `page_translations` WHERE (`page_translations`.page_id = 1)
Here is what my controller looks like:
#page = Page.find(params[:id], :include => :page_translations)
I was stumped about this same thing and wasted a few hours trying to figure it out. It turns out that using the joins method of the query interface doesn't initialize the models related to the tables being joined. You can see this by watching the SQL statements in the server console, or by even redirecting ActiveRecord logging to STDOUT in your Rails console.
I was very disappointed by this. It just doesn't seem like how the joins method should work -- it certainly wasn't what I was expecting. I was expecting it to eager load, since it was in the eager load section of the Edge Guides.
Anyway, I couldn't waste any more time trying to figure it out, so what I did instead is use the fancy query interface to simply build my query, used to_sql to get the SQL for my query, and then passed the SQL to select_all, which returns an array of hashes, where each element in the array (each hash) represents a row.
Example:
query = Post.joins("LEFT JOIN categories ON post.category_id = categories.id")
query.select("posts.*, category.category_name")
con = ActiveRecord::Base.connection
results = con.select_all(query.to_sql)
Results:
[{"id": 1, "title": "joins, why have you forsaken me", "category_name": "frustration"},{"id": 2, "title": "pizza", "category_name": "food"}]
To be honest, I would still like to know for certain if it is possible to do it the way we think it should work, or the way it ought to work. Otherwise, I see no reason for having a joins method, other than to help us build the query. So if any Rails gurus out there know how to use joins to populate models related to those tables, PLEASE LET ME (US) KNOW!
Anyway, I hope this helps you move along for now.
UPDATE: So I think I just figured it out. I stumbled across this blog post. As it turns out, when using the joins method of the query interface, Rails will actually add the columns you selected from the joined tables as attribute methods of the model being joined against.
Using the same example above, I can actually access the category_name of each post by simply calling post.category_name. #$%! Unbelievably simple, but no documentation whatsoever on this!
Here it is once again:
query = Post.joins("LEFT JOIN categories ON post.category_id = categories.id")
query.select("posts.*, category.category_name")
posts = query.all
# access a post's category name
puts posts[0].category_name
# this is what I thought I would be able to do
# without Rails making another query to the database
puts posts[0].category.category_name
I hope this helps! :)
How about:
Page.find( params[:id], :include => :page_translations )
Edit:
Ok, so some time recently the behavior of ActiveRecord when it comes to joins/includes seems to have changed. The guides still refer to being able to do this though two associations, as in has_many :orders, :include => :line_items and similar... but as far as including records from a has_many... After consulting with a co-worker we came across some info on the subject. Seems that the single monolithic queries were just getting too complex and ugly and it was causing problems for some of the fancier niceties that ActiveRecord gives you and duplicate rows, that kind of thing.
TL;DR: It doesn't work like that anymore. 2 queries is expected.