I have additional calculation columns (based on joins) I want to include in my CSV.
If I open and calculate it individually for every record
csv do
column :complicated_calculation { |r| r.calculate_things }
end
it's going to take a long time to generate with thousands of records.
I need to customize the SELECT query for when my CSV is generated and then use the columns in that query. How do I do that?
Customizing resource retrieval, in documentation, shows you how without rewriting the whole csv builder: modifying scoped_collection.
So if you have your query nicely waiting in your model:
class Person < ActiveRecord::Base
def self.with_calculation
select("people.*, (mumbo + jumbo * fumbo) "\
"AS complicated_calculation") # notice we name the attribute here
.joins("LEFT JOIN games ON person_id = people.id")
.group("people.id")
end
end
with your calculation you can do:
controller do
def scoped_collection
super.with_calculation
end
end
and then your CSV will have the attribute for free:
csv do
column :complicated_calculation
end
Related
Hello I had given query
refund1 = Spree::Order.joins(:refunds).group('currency').sum(:total)
=> {"USD"=>#<BigDecimal:7f896ea15ed8,'0.17641E4',18(18)>, "SGD"=>#<BigDecimal:7f896ea15d98,'0.11184E3',18(18)>, "EUR"=>#<BigDecimal:7f896ea15ca8,'0.1876E3',18(18)>}
2.2.1 :212 >
refund1 = Spree::Order.joins(:refunds).group('currency').count
=> {"USD"=>2, "SGD"=>1, "EUR"=>2}
refund1.each do |k,v| refund1[k]=[v,refund2[k]] end
=> {"USD"=>[2, #<BigDecimal:7f896f1d83a0,'0.17641E4',18(18)>], "SGD"=>[1, #<BigDecimal:7f896f1d3fa8,'0.11184E3',18(18)>], "EUR"=>[2, #<BigDecimal:7f896f1d3aa8,'0.1876E3',18(18)>]}
refund1 = Spree::Order.joins(:refunds).group('currency').sum(refund.amount)
this is not working i need to sum refund amount not an order total
I need to fetch date also i.e on 02-08-2017 two orders refunded of 100USD
Please guide me how to fetch that.
Rails/ActiveRecord are good for relatively easy groupings, and you can group on multiple attributes instead of just the currency, but applying a function to one of the grouped values and returning multiple aggregations (sum and count) requires some effort.
It will also not be very performant unless you either start specifying SQL fragments in your select clause select("date_trunc(...), currency, sum(...), count(...)") or start using Arel (which to me always looks more complex than SQL with very few redeeming benefits).
I (because I am quite a SQL-ey person) would be tempted here to place a database view in the system that defines the aggregations that you want at the grouping level you want, and reference that in Rails through a model.
Create View spree_refund_day_currency_agg as select ....;
... and ...
class Spree::RefundDayCurrencyAgg < ActiveRecord::Base
def self.table_name
spree_refund_day_currency_agg
end
def read_only?
true
end
belongs_to ....
end
You can then access your aggregated data in the Rails environment as if it were a magically maintained set of data (similar to a materialised view, without the materialisation) in a totally flexible manner (as intended with an RDBMS) using logic defined in Rails.
For example, with scopes defined in the model
def self.bad_day_in_canada
where(currency: CANADA_CURR)
end
Not to everyone's taste though, I'm sure.
I have two Sidekiq jobs. The first loads a feed of articles in JSON and splits it into multiple jobs. It also creates a log and stores a start_time.
class LoadFeed
include Sidekiq::Worker
def perform url
log = Log.create! start_time: Time.now, url: url
articles = load_feed(url) # this one loads the feed
articles.each do |article|
ProcessArticle.perform_async(article, log.id)
end
end
end
The second job processes an article and updates the end_time field of the former created log to find out, how long the whole process (loading the feed, splitting it into jobs, processing the articles) took.
class ProcessArticle
include Sidekiq::Worker
def perform data, log_id
process(data)
Log.find(log_id).update_attribute(:end_time, Time.now)
end
end
But now I have some problems / questions:
Log.find(log_id).update_attribute(:end_time, Time.now) isn't atomic, and because of the async behaviour of the jobs, this could lead to incorrect end_time values. Is there a way to do an atomic update of a datetime field in MySQL with the current time?
The feed can get pretty long (~ 800k articles) and updating a value 800k times when you would just need the last one seems like a lot of unnecessary work. Any ideas how to find out which one was the last job, and only update the end_time field in this job?
For 1) you could do an update with one less query and let MySQL find the time:
Log.where(id: log_id).update_all('end_time = now()')
For 2) one way to solve this would be to update your end time only if all articles have been processed. For example by having a boolean that you could query. This does not reduce the number of queries but would certainly have better performance.
if feed.articles.needs_processing.none?
Log.where(id: log_id).update_all('end_time = now()')
end
This is the problem Sidekiq Pro's Batch feature solves. You create a set of jobs, it calls your code when they are all complete.
class LoadFeed
include Sidekiq::Worker
def on_success(status, options)
Log.find(options['log_id']).update_attribute(:end_time, Time.now)
end
def perform url
log = Log.create! start_time: Time.now, url: url
articles = load_feed(url) # this one loads the feed
batch = Sidekiq::Batch.new
batch.on(:success, self.class, 'log_id' => log.id)
batch.jobs do
articles.each do |article|
ProcessArticle.perform_async(article, log.id)
end
end
end
end
Here's an example of a 'component' that I have.
Is this example, the rails cache is hit once every loop. Is there a way to avoid this?
It is only used in my mobile views. Which means I fetching the data in the controller will fetch unnecessary data for the desktop views.
Is it possible to avoid doing show_ids = current_user.get_show_ids in the view, outside the loop?
View:
#shows.each do |show|
if logged_in?
if current_user.is_following(show) ### Hitting the cache (!)
#remove button
else
#add button
end
else
#something else
end
end
User model:
def is_following(show)
get_show_ids.include?(show.id)
end
def get_show_ids
Rails.cache.fetch([self, "all_shows_ids", "v3"]) do
Following.where("user_id = ?", id).pluck(:show_id)
end
end
I've tried doing #ids ||= get_show_ids.include?(show.id) with the same result.
I assume that you have the models Show and User connected by a Following model for the n:m relation.
When you retrieve the shows, you can left join the Followings with a user_id of current-user. The result set will have the fields for Following set if there is such a following, and not if the user is not following.
Note you need to select the fields from Following model explicitly, otherwise the join will only return fields from the Show model
User.select(:name).group(:name).having("count(*) > 1")
this query works fine to select records having duplicate user name. But problem am facing is when there is space in name.
For example.
recoed1 = "Username"
record2 = "Username "
This are the two records but they are having same name, but above query consider them as different records because space is there in the second record. So while selecting I did not get this record.
Any solution using normal mysql query or rails will do.
OR
How I can strip or trim all the column data first from table using rails/mysql query. Then I can apply above query.
What i would do here is make sure your data is tidy in the first place.
You could put in a pre-validation method to call strip on usernames. You could do it like so
#in lib/trimmer.rb
module Trimmer
# Make a class method available to define space-trimming behavior.
def self.included base
base.extend(ClassMethods)
end
module ClassMethods
# Register a before-validation handler for the given fields to
# trim leading and trailing spaces.
def trimmed_fields *field_list
before_validation do |model|
field_list.each do |field|
model.send("#{field}=", model.send("#{field}").strip) if model.send("#{field}").respond_to?('strip')
end
end
end
end
end
Make sure this module is required, wherever you require things in lib in your config.
Now, you can say, in any models, like so (in this example i'm doing some other fields besides username)
class User < ActiveRecord::Base
include Trimmer
trimmed_fields :username, :email, :first_name, :last_name
...
So, that will fix you going forward. The remaining step is to tidy up your existing data. I would do this in a migration. (again, i'm doing some other fields as an example)
tables_and_cols = {"users" => %w(username email first_name last_name), "foos" => %w(bar baz)}
tables_and_cols.each do |table, cols|
cols.each do |col|
ActiveRecord::Base.connection.execute("update #{tablename} set #{col} = trim(#{col})")
end
end
Now, after doing this trim, you may have some duplicate usernames. You will need to decide how you are going to deal with that, since the records involved are no longer valid. If you haven't publically deployed this yet, ie if you don't have any active real users, then it doesn't matter: you can change the usernames to something else. But if you do have real people using it you will probably need to change the username for some of them and inform them. This is unavoidable if you want to maintain a situation where people can't have spaces at the start or end of their username.
You can use mysql's string functions:
User.select("lower(trim(name))").group("lower(trim(name))").having("count(*) > 1")
The employees list page on my rails app has a requirement of showing links for each of the starting characters in employee names, to facilitate quickly searching for users by a letter.
The link list should include only those letters which have names starting from them. Eg: If the table contains 3 names - Joe User, Jill User, and Example User - only two links E and J should be displayed.
Wondering what's the most efficient way to do this.
As a first attempt, I added a class method in the Employee class as follows:
def self.list_of_starting_characters
array = []
('A'..'Z').to_a.each do |char|
array << char unless self.where("name like '#{char}%'").count.zero?
end
array
end
This gets called on every render of the view; so changed it to use a class variable as follows:
##starting_characters_list = []
def self.list_of_starting_characters
return ##starting_characters_list unless ##starting_characters_list.empty?
array = []
('A'..'Z').to_a.each do |char|
array << char unless self.where("name like '#{char}%'").count.zero?
end
##starting_characters_list = array
end
Now it is called only once per session. Are there better ways to accomplish this?
One other option I am considering is to store the list of starting characters in a separate table and update it when the employee data is modified, but worried it might be too much of a hassle, to keep the separate table in sync with the main table.
The following should work for you:
first_letters = self.pluck(:name).group_by{ |name| name[0].upcase }.keys
It grabs all User's names, group them by the first letter of the name and get the only the keys of the hash generated by the group_by.