Performance difference in Concern method ran with Hook vs on Model? - mysql

Basically I notice a big performance difference in dynamically overriding a getter for ActiveRecord::Base models within an after_initialize hook and simply within the model itself.
Say I have the following Concern:
module Greeter
extend ActiveSupport::Concern
included do
after_initialize { override_get_greeting }
end
def override_get_greeting
self.class::COLS.each do |attr|
self.class.class_eval do
define_method attr do
"Hello #{self[attr]}!"
end
end
end
end
end
I then have the following model, consisting of a table with names.
CREATE TABLE 'names' ( 'name' varchar(10) );
INSERT INTO names (name) VALUES ("John")
class Name < ActiveRecord::Base
COLS = %w("name")
include Greeter
end
john = Name.where(name: 'John').first
john.name # Hello John!
This works fine. However, if I try to do this a more Rails way it is significantly slower.
Essentially, I want to simply pass a parameter into Greeter method that contains COLS and then overrides the getters. It'll look something like:
# Greeter
module Greeter
extend ActiveSupport::Concern
def override_get_greeting(cols)
cols.each do |attr|
self.class.class_eval do
define_method attr do
"Hello #{self[attr]}!"
end
end
end
end
end
# Name
class Name < ActiveRecord::Base
include Greeter
override_get_greeting [:name]
end
Now Name.where(name: 'John').first.name # Hello John! is about 2 seconds slower on the first call.
I can't put my finger in it. I have an assumption that the the application is just slower to start with the first example, but not really sure.
I prefer the second example but the performance difference is a big no no.
Has anyone came across something like this?

Unless the real application code is radically different to what you've shown above, there's no way this should be causing a 2 second performance hit!
However, it's still a needlessly verbose and inefficient way to write the code: You're redefining methods on on the class instance, every time you initialize the class.
Instead of using after_initialize, you can just define the methods once. For example, you could put this in the Greeter module:
included do |klass|
klass::COLS.each do |attr|
define_method attr do
"Hello #{self[attr]}!"
end
end
end
Also worth noting is that instead of self[attr], you may instead wish to use super(). The behaviour will be the same (assuming no other overrides are present), except that an error will be raised if the column does not exist.

Related

Proper way to dynamically define LUA functions

I have been playing with Lua for the past week and I ended up writing this peace of code. I find it really useful that you can dynamically create new functions that "inherit" other functions, so programmers must have a name for it. My questions are:
What is this called?
Is there a better way to do this? (cycle over a data structure and create add-on functions that "improve" existing functions)
D = {
name = {
value = nil,
offset = 0,
update = function (self)
self.value = "Berlin"
end,
},
}
--Dynamic function definition
for i in pairs(D) do
D[i].upAndWrite = function(self)
self:update()
print("upAndWrite was here")
end
end
print(D.name.value)
D.name:upAndWrite()
print(D.name.value)
Result:
nil
upAndWrite was here
Berlin
I don't think that what you're doing have special name for it, it's just on-the-fly function creation.
There are few notes regarding your code:
Proper for loop
for i in pairs(D) do
…
end
In programming, variable i is generally used for counter loops, like in
for i=1,100 do
…
end
Here, pairs returns an iterator function and the idiomatic way to use it is
for k,v in pairs(D) do
…
end
Here k is a key (like i in your code) and v is a value (use it instead of indexing table like D[k] or D[i] in your code when you need to access the corresponding value).
There's no need to create functions on the fly!
Another important thing is that you create new function on each loop iteration. While this feature is very powerful, you're not using it at all as you don't store anything using upvalues and only access data through arguments.
A better way to do it would be creating function once and assigning it to every field:
-- Define D here
do
local function upAndWrite(self)
self:update()
print("upAndWrite was here")
end
for k,v in pairs(D) do
v.upAndWrite = upAndWrite -- Use our function
end
end
-- Perform tests here
What does on-the-fly function creation allow?
As mentioned above, you can utilize this very powerful mechanism of closures in certain situations. Here's a simple example:
local t = {}
for i = 1, 100 do
-- Create function to print our value
t[i] = function() print(i) end
-- Create function to increment value by one
t[-i] = function() i=i+1 end
end
t[1]() -- Prints 1
t[20]() -- Prints 20
t[-20]() -- Increment upvalue by one
t[20]() -- Now it is 21!
This example demonstrates one possible usage of upvalues and the fact that many functions can share them. This can be useful in a variety of situations together with the fact that upvalues can't be changed by side code (without use of debug library) and can be trusted in general.
I hope my answer covers what you wanted to know.
Note: Also, this language is called Lua, not LUA.
As a whole it doesn't have a name, no. There's lots of concepts that play into this:
First Class Functions aka. functions that can be assigned to variables and passed around just like numbers or strings.
Anonymous Functions aka. functions that are created without giving it a name explicitly. All functions in Lua are technically anonymous, but often they are assigned into a variable right after creation.
Metaprogramming aka. writing programs that write programs. A loop that creates functions (or methods) on an arbitrary number of objects is very simple, but I'd count it as metaprogramming.
Lua Tables; this may seem obvious, but consider that not all languages have a feature like this. Javascript has objects which are similar, but Ruby for example has no comparable feature.
If you're gonna use pairs, you might as well make use of both variables.
for key, object in pairs(D) do
function object:upAndWrite(self)
self:update()
print("upAndWrite was here")
end
end
Though that would create many closures, which means more work for the garbage collector, more memory usage and slower execution speed.
for key, object in pairs(D) do
print(object.upAndWrite) -- All the functions are different
end
It's a good first stage, but after refactoring it a bit you could get this:
do
local method = function(self) -- Create only one closure
self:update()
print("upAndWrite was here")
end
for key, object in pairs(D) do
object.upAndWrite = method -- Use single closure many times
end
end
Now there's only one closure that's shared among all the tables.
for key, object in pairs(D) do
print(object.upAndWrite) -- All the functions are the same
end

ActiveModelSerializers Polymorphic Json

Been wrestling with trying to get polymorphic serializers working and testing data via rspec. Just upgraded to 0.10+
I found this post, which makes a lot of sense, and does give me a entry into generating the serializations, however, when doing it for polymorphs, I never get the type and id properly named (expecting to see asset_id and asset_type nested)
{:id=>1,
:label=>"Today I feel amazing!",
:position=>0,
:status=>"active",
:media_container_id=>1,
:asset=>
{:id=>4
Test ActiveModel::Serializer classes with Rspec
class MediaSerializer < ApplicationSerializer
attributes :id,
:label,
has_one :asset, polymorphic: true
end
I noticed that the tests dont even seem to properly add the polymorphic identifiers either (ie asset_id, asset_type -- or in the test case imageable_id, imageable_type)
https://github.com/rails-api/active_model_serializers/commit/045fa9bc072a04f5a94d23f3d955e49bdaba74a1#diff-c3565d7d6d40da1b2bf75e13eb8e6afbR36
If I go straight up MediaSerialzer.new(media) I can poke at the .associations, but I cant seem to get them to render as if I was generating a full payload
From the docs
https://github.com/rails-api/active_model_serializers
serializer_options = {}
serializer = SomeSerializer.new(resource, serializer_options)
serializer.attributes
serializer.associations
Im pretty sure Im missing something/doing something wrong - any guidance would be great.
Thanks
It isn't easy to get the effect you are looking for, but it is possible.
You can access the hash generated by the serializer by overriding the associations method.
class MediaSerializer < ApplicationSerializer
attributes :id,
:label,
has_one :asset, polymorphic: true
def associations details
data = super
data[:asset] = relabel_asset(data[:asset])
data
end
def relabel_asset asset
labelled_asset = {}
asset.keys.each do |k|
labelled_asset["asset_#{k}"] = asset[k];
end
labelled_asset
end
end
I learnt alot about ActiveModelSerializer to get the hang of this! I referred to Ryan Bates' podcast on the topic:
http://railscasts.com/episodes/409-active-model-serializers
In there he describes how you can override the attributes method and call super to get access to the hash generated by the serializer. I guessed I could do the same trick for the associations method mentioned in your post. From there it takes a little bit of Ruby to replace all the keys, but, if I have understood correctly what you require, it is technically possible.
Hope that helps!

ActiveRecord undefined method `has_key?' for nil:NilClass error

I have a fairly simple forms-over-data Rails app that calls a remote MySql 5.5 db. Using Rails 3.2.21, Ruby 1.9.3.
One of the pages in the app is throwing the following error:
NoMethodError in GvpController#input
undefined method `has_key?' for nil:NilClass
app/controllers/gvp_controller.rb:9:in `input'
Here is the offending code from the controller:
class GvpController < ApplicationController
def input
# irrelevant stuff
#list = Vendor.gvp_vendor_names.map { |x| x.vendor_name }
# more irrelevant stuff
end
# other irrelevant methods
end
I'm assuming the call to gvp_vendor_names is returning nil.
Here is the Vendor model class:
class Vendor < ActiveRecord::Base
establish_connection :vendor_sql
self.table_name = 'reporting_dw.vp_vendor_mapping'
scope :gvp_vendor_names, -> {
select('reporting_dw.vp_vendor_mapping.vendor_name')}
end
I have searched other posts with this error message and so far haven't found one that seems relevant. I am not overriding the initialize method (one possible cause) and I think the syntax is correct (another).
As an additional wrinkle, I am using vagrant for development, so I thought perhaps I'm not successfully communicating with the database from the vagrant box - maybe an ssh or permissions issue. To test it, I opened an ssh session on the vagrant box, successfully connected with the db via command line ran a select statement and lo and behold, get the full list of results I was expecting. I also tried it with mysql workbench via ssh and had no problems. So, it seems I can communicate remotely with the db, execute queries against it, have the proper permissions etc.
Does anyone have any suggestions as to what the problem might be?
I assume that you haven't any value on your DB tables. That's why the issue arise in controller action block during you call gvp_vendor_names mapped value vendor_name
You should handle this type of case by checking the object value rather than accessing firstclass
GvpController < ApplicationController
def input
# irrelevant stuff
#list = Vendor.gvp_vendor_names.map { |x| x.vendor_name if x.present?}
# more irrelevant stuff
end
# other irrelevant methods
end
In this way you need to compact the nil value. So use this finally if you want to handle the scenario from controller:
class GvpController < ApplicationController
def input
# irrelevant stuff
#list = Vendor.gvp_vendor_names.map { |x| x.vendor_name if x.present?}.compact
# more irrelevant stuff
end
# other irrelevant methods
end
The real problem may just be that I'm a Rails/ActiveRecord n00b. After a little more experimentation, I found the following changes corrected the error.
In the model I added attr_accessible and then used engineersmnky's suggestion of using a method rather than scope, as follows:
class Vendor < ActiveRecord::Base
establish_connection :vendor_sql
attr_accessible :vendor_name
self.table_name = 'reporting_dw.vp_vendor_mapping'
def self.gvp_vendor_names
pluck(:vendor_name).sort
end
end
Then in the controller:
class GvpController < ApplicationController
def input
#irrelevant stuff
#list = Vendor.gvp_vendor_names
#irrelevant stuff
end
end
That fixed it. Thank you everyone for the suggestions!

ActiveRecord, Intentionally truncate string to db column width

In Rails 4, ActiveRecord and it's MySQL adapter are set up so if you try to save an attribute in an AR model to a MySQL db, where the attribute string length is too wide for the MySQL column limits -- you'll get an exception raised.
Great! This is much better default than Rails3, where it silently truncated the string.
However, occasionally I have an attribute that I explicitly want to be simply truncated to the maximum size allowed by the db, with no exception. I'm having trouble figuring out the best/supported way to do this with AR.
It should ideally happen as soon as the attribute is set, but I'd take it happening on save. (This isn't exactly a 'validation', as I never want to raise, just truncate, but maybe the validation system is the best supported way to do this?)
Ideally, it would automatically figure out the db column width through AR's db introspection, so if the db column width changed (in a later migration), the truncation limit would change accordingly. But if that's not possible, I'll take a hard-coded truncation limit.
Ideally it would be generic AR code that would work with any db, but if there's no good way to do that I'd take code that only worked for MySQL
You could truncate your data before inserting in db with a before_save or a before_validation
See Active Record Callbacks — Ruby on Rails Guides and ActiveRecord::Callbacks
You can retrieve informations on your table with MODEL.columns and MODEL.columns_hash.
See ActiveRecord::ModelSchema::ClassMethods
For example (not tested):
class User < ActiveRecord::Base
before_save :truncate_col
......
def truncate_col
col_size = User.columns_hash['your_column'].limit
self.your_column = self.your_column.truncate(col_size)
end
end
I'm pretty sure you can accomplish this with a combination of ActiveRecord callbacks and ConnectionsAdapters. ActiveRecord contains several callbacks you can override to perform specific logic at different points during the save flow. Since the exception is being thrown at save, I would recommend adding your logic to the before_save method. Using the column ConnectionAdapter you should be able to determine the limit of the column you wish to insert, though the logic will most likely be different for strings vs ints, etc. Off the top of my head you'll probably want to implement something like:
class User < ActiveRecord::Base
def before_save
limit = User.columns_hash['attribute'].limit
self.attribute = self.attribute[0..limit-1] if self.attribute.length > limit
end
end
The above example is for a string, but this solution should work for all connection adapters assuming they support the limit attribute. Hopefully that helps.
I'd like to address a few points:
If the data type of your_column is text, in Rails 4 User.columns_hash['your_column'].limit will return nil. It returns a number in case of int or varchar.
The text data type in MySQL has a storage limit of 64k. Meaning truncating upon char length is not enough if the content has non ascii chars like ç which needs more than 1 byte to be stored.
I've bumped into this problem very recently, here is a hotfix for it:
before_save :truncate_your_column_to_fit_into_max_storage_size
def truncate_your_column_to_fit_into_max_storage_size
return if your_column.blank?
max_field_size_in_bytes = 65_535
self.your_column = your_column[0, max_field_size_in_bytes]
while your_column.bytesize > max_field_size_in_bytes
self.your_column = your_column[0..-2]
end
end
Here's my own self-answer, which truncates on attribute set (way before save). Curious if anyone has any feedback. It seems to work!
# An ActiveRecord extension that will let you automatically truncate
# certain attributes to the maximum length allowed by the DB.
#
# require 'truncate_to_db_limit'
# class Something < ActiveRecord::Base
# extend TruncateToDbLimit
# truncate_to_db_limit :short_attr, :short_attr2
# #...
#
# Truncation is done whenever the attribute is set, NOT waiting
# until db save.
#
# For a varchar(4), if you do:
# model.short_attr = "123456789"
# model.short_attr # => '1234'
#
#
# We define an override to the `attribute_name=` method, which ActiveRecord, I think,
# promises to call just about all the time when setting the attribute. We call super
# after truncating the value.
module TruncateToDbLimit
def truncate_to_db_limit(*attribute_names)
attribute_names.each do |attribute_name|
ar_attr = columns_hash[attribute_name.to_s]
unless ar_attr
raise ArgumentError.new("truncate_to_db_limit #{attribute_name}: No such attribute")
end
limit = ar_attr.limit
unless limit && limit.to_i != 0
raise ArgumentError.new("truncate_to_db_limit #{attribute_name}: Limit not known")
end
define_method "#{attribute_name}=" do |val|
normalized = val.slice(0, limit)
super(normalized)
end
end
end
end

Overwriting all default accessors in a rails model

class Song < ActiveRecord::Base
# Uses an integer of seconds to hold the length of the song
def length=(minutes)
write_attribute(:length, minutes.to_i * 60)
end
def length
read_attribute(:length) / 60
end
end
This is an easy example by rails api doc.
Is it possible overwrite all attributes for a model without overwrite each one?
Do you look for something like that? Don't know why you would want to do it, but here you go :)
class Song < ActiveRecord::Base
self.columns_hash.keys.each do |name|
define_method :"#{name}=" do
# set
end
define_method :"#{name}" do
# get
end
# OR
class_eval(<<-METHOD, __FILE__, __LINE__ + 1)
def #{name}=
# set
end
def #{name}
# get
end
METHOD
end
end
I'm not sure of a use case where this would be a good idea. However, all rails models dynamically have their properties assigned to them (assuming it isn't already in the class). The answer is partially in your question.
You can override the read_attribute() and write_attribute() methods. That would apply your transformations to every attribute whether they were written to by the accessor or populated in bulk in the controller. Just be careful to not mutate important attributes like the 'id' attribute.
Ruby has a shortcut that is used in rails code a fair bit that can help you. It's the %w keyword. %w will create an array of words based on the symbols inside the parentheses. Because it's an array you can do useful things like this:
#excludes = %w(id, name)
def read_attribute name
value = super
if(not #excludes.member? name)
value = value.to_i * 60
end
value
end
def write_attribute name, value
if(not #excludes.member? name)
value = value.to_i / 60
end
super
end
That should get you started. There are more advanced constructs like using lambdas, etc. Keep in mind you should write some thorough unit tests to make sure you don't have any unintended consequences. You may have to include more attribute names in the list of excludes.
edit: (read|write)_attributes -> (read|write)_attribute