So I'm trying to write a method in the model that will allow me to return posts who have a specific field value that is greater than 0.
So I have posts that have fields that are essentially tags. Basically I post has four fields, hiphop, electro, house and pop. Each field has a value between 0 and 10.
I'm trying to make it so if someone clicks on a button the the view that says "Hip Hop" it will return all posts that have a hiphop field value that is greater than 0.
I know this is wrong but I'm thinking something like this
def self.tagSearch(query)
where("#{query} > 0")
end
and in my controller I would have something like this
def index
if params[:search]
#songs = Song.search(params[:search]).order("created_at DESC")
elsif params[:tag]
#songs = Songs.tagSearch(params[:tag]).order("created_at DESC")
else
#songs = Song.all
end
end
And I'm not sure about the view but maybe a button that passes the tag value parameter. The thing is I just want it to be a button, I don't need them to input anything.
I hope this isn't too confusing.
Thank you!
Matt
To expand on RaVen post:
1) Use ruby naming conventions tagSearch should be tag_search; methods are snake case (lower case with underscores).
2) where("#{query} > 0") is exposing you to SQL injection attacks - recommended to install the brakeman gem which can expose security issues like this:
http://brakemanscanner.org/docs/warning_types/sql_injection/
http://brakemanscanner.org/docs/
3) You can simplify your code by chaining scopes, scopes that return nil will not effect the query
class Song
scope :search, -> (query) do
where("name LIKE ?", "#{query}%") if query.present?
end
scope :tag_search, -> (tag) do
where(tag > 0) if tag.present?
end
scope :ordered, -> do
order(created_at: :desc)
end
end
class SongsController
def index
#songs = Song.search(params[:search])
.tag_search(params[:tag])
.ordered
end
end
4) Making queries based on a user specified column and avoiding sql injection:
This is one way to do it, there are probably other better ones available, like using the models arel_table, anyhow this one is pretty straight forward
scope :tag_search, -> (tag) do
where("#{self.white_list(tag)} > 0") if tag.present?
end
def self.white_list(column_name)
# if the specified column_name matches a model attribute then return that attribute
# otherwise return nil which will cause a sql error
# but it won't let arbitrary sql execution
self.attribute_names.detect { |attribute| attribute == column_name }
end
Rails support "scopes" which return an ActiveRecord::Relation which means you can chain them together.
class Song
scope :tag_search, -> (something) { where(something > 0) }
scope :ordered, -> { order(created_at: :desc) }
end
class SongsController
def index
if params[:search]
#songs = Song.search(params[:search]).ordered
elsif params[:tag]
#songs = Songs.tag_search(params[:tag]).ordered
else
#songs = Song.all
end
end
end
I would overthink the design of this.
Plus your tagSearch function is really dangerous. SQL INJECTION!
Related
My challenge is that in the database, JSON code was untidily stored.
{'isr_comment':'Test Comment',
'isr_depression_1': '1',
'isr_depression_2': '1'
'isr_depression_3': '1'
'isr_tested': 'true'
}
You see, all values are defined as string but some should be integers. It would be the best to have clean data already in the database but I cannot control how the data is entered. However my model looks like this.
class SessionPart < ApplicationRecord
...
serialize :answers, JSON
...
end
As expected after deserialization is done I get strings as well.
#data=
{"isr_Comment"=>"Test Comment",
"isr_depression_1"=>"1",
"isr_depression_2"=>"1",
"isr_depression_3"=>"1",
"isr_tested" => "true"}
But I need to do some calculation with this data so I need all possible values with a meaningful type.
#data=
{"isr_Comment"=>"Test Comment",
"isr_depression_1"=>1,
"isr_depression_2"=>1,
"isr_depression_3"=>1,
"isr_tested" => true}
Is there any way to cast such data automatically?
You can pass your custom serializer to serialize function. That custom serializer would use JSON as source serializer and update the values as per your requirements.
class SessionPart < ApplicationRecord
...
serialize :answers, CustomSerializer #CustomSerializer must write 2 class level function named dump & load for serializing and de-serializing respectively
...
end
class CustomSerializer
def self.load(value)
normalize_hash(JSON.load(value))
end
def self.dump(value)
JSON.dump(value)
end
private
def self.normalize_hash hash
return hash unless hash.is_a? Hash
hash.transform_values {|v| normalize(v)}
end
#change this function as per your requirement, Currently it's handling boolean,integer,float & null rule set
def self.normalize(value)
case (value)
when 'true'
true
when 'false'
false
when 'null','nil'
nil
when /\A-?\d+\z/
value.to_i
when /\A-?\d+\.\d+\z/
value.to_f
else
value.is_a?(Hash) ? normalize_hash(value) : value
end
end
end
The suggested CustomSerializer seems to do its job very well, thanks. I did some small adjustments to be able to nest hashes.
class CustomSerializer
def self.load(value)
normalize_hash(JSON.load(value))
end
def self.dump(value)
JSON.dump(value)
end
private
def self.normalize_hash hash
return hash unless hash.is_a? Hash
hash.transform_values {|v| normalize(v) }
end
#change this function as per your requirement, Currently it's handling boolean,integer,float & null rule set
def self.normalize(value)
case (value)
when 'true'
true
when 'false'
false
when 'null','nil'
nil
when /\A-?\d+\z/
value.to_i
when /\A-?\d+\.\d+\z/
value.to_f
else
value.is_a?(Hash) ? normalize_hash(value) : value
end
end
end
Im trying to replicate the searching list style of crunchbase using ruby on rails.
I have an array of filters that looks something like this:
[
{
"id":"0",
"className":"Company",
"field":"name",
"operator":"starts with",
"val":"a"
},
{
"id":"1",
"className":"Company",
"field":"hq_city",
"operator":"equals",
"val":"Karachi"
},
{
"id":"2",
"className":"Category",
"field":"name",
"operator":"does not include",
"val":"ECommerce"
}
]
I send this json string to my ruby controller where I have implemented this logic:
filters = params[:q]
table_names = {}
filters.each do |filter|
filter = filters[filter]
className = filter["className"]
fieldName = filter["field"]
operator = filter["operator"]
val = filter["val"]
if table_names[className].blank?
table_names[className] = []
end
table_names[className].push({
fieldName: fieldName,
operator: operator,
val: val
})
end
table_names.each do |k, v|
i = 0
where_string = ''
val_hash = {}
v.each do |field|
if i > 0
where_string += ' AND '
end
where_string += "#{field[:fieldName]} = :#{field[:fieldName]}"
val_hash[field[:fieldName].to_sym] = field[:val]
i += 1
end
className = k.constantize
puts className.where(where_string, val_hash)
end
What I do is, I loop over the json array and create a hash with keys as table names and values are the array with the name of the column, the operator and the value to apply that operator on. So I would have something like this after the table_names hash is created:
{
'Company':[
{
fieldName:'name',
operator:'starts with',
val:'a'
},
{
fieldName:'hq_city',
operator:'equals',
val:'karachi'
}
],
'Category':[
{
fieldName:'name',
operator:'does not include',
val:'ECommerce'
}
]
}
Now I loop over the table_names hash and create a where query using the Model.where("column_name = :column_name", {column_name: 'abcd'}) syntax.
So I would be generating two queries:
SELECT "companies".* FROM "companies" WHERE (name = 'a' AND hq_city = 'b')
SELECT "categories".* FROM "categories" WHERE (name = 'c')
I have two problems now:
1. Operators:
I have many operators that can be applied on a column like 'starts with', 'ends with', 'equals', 'does not equals', 'includes', 'does not includes', 'greater than', 'less than'. I am guessing the best way would be to do a switch case on the operator and use the appropriate symbol while building the where string. So for example, if the operator is 'starts with', i'd do something like where_string += "#{field[:fieldName]} like %:#{field[:fieldName]}" and likewise for others.
So is this approach correct and is this type of wildcard syntax allowed in this kind of .where?
2. More than 1 table
As you saw, my approach builds 2 queries for more than 2 tables. I do not need 2 queries, I need the category name to be in the same query where the category belongs to the company.
Now what I want to do is I need to create a query like this:
Company.joins(:categories).where("name = :name and hq_city = :hq_city and categories.name = :categories[name]", {name: 'a', hq_city: 'Karachi', categories: {name: 'ECommerce'}})
But this is not it. The search can become very very complex. For example:
A Company has many FundingRound. FundingRound can have many Investment and Investment can have many IndividualInvestor. So I can select create a filter like:
{
"id":"0",
"className":"IndividualInvestor",
"field":"first_name",
"operator":"starts with",
"val":"za"
}
My approach would create a query like this:
SELECT "individual_investors".* FROM "individual_investors" WHERE (first_name like %za%)
This query is wrong. I want to query the individual investors of the investments of the funding round of the company. Which is a lot of joining tables.
The approach that I have used is applicable to a single model and cannot solve the problem that I stated above.
How would I solve this problem?
You can create a SQL query based on your hash. The most generic approach is raw SQL, which can be executed by ActiveRecord.
Here is some concept code that should give you the right idea:
query_select = "select * from "
query_where = ""
tables = [] # for selecting from all tables
hash.each do |table, values|
table_name = table.constantize.table_name
tables << table_name
values.each do |q|
query_where += " AND " unless query_string.empty?
query_where += "'#{ActiveRecord::Base.connection.quote(table_name)}'."
query_where += "'#{ActiveRecord::Base.connection.quote(q[fieldName)}'"
if q[:operator] == "starts with" # this should be done with an appropriate method
query_where += " LIKE '#{ActiveRecord::Base.connection.quote(q[val)}%'"
end
end
end
query_tables = tables.join(", ")
raw_query = query_select + query_tables + " where " + query_where
result = ActiveRecord::Base.connection.execute(raw_query)
result.to_h # not required, but raw results are probably easier to handle as a hash
What this does:
query_select specifies what information you want in the result
query_where builds all the search conditions and escapes input to prevent SQL injections
query_tables is a list of all the tables you need to search
table_name = table.constantize.table_name will give you the SQL table_name as used by the model
raw_query is the actual combined sql query from the parts above
ActiveRecord::Base.connection.execute(raw_query) executes the sql on the database
Make sure to put any user submitted input in quotes and escape it properly to prevent SQL injections.
For your example the created query will look like this:
select * from companies, categories where 'companies'.'name' LIKE 'a%' AND 'companies'.'hq_city' = 'karachi' AND 'categories'.'name' NOT LIKE '%ECommerce%'
This approach might need additional logic for joining tables that are related.
In your case, if company and category have an association, you have to add something like this to the query_where
"AND 'company'.'category_id' = 'categories'.'id'"
Easy approach: You can create a Hash for all pairs of models/tables that can be queried and store the appropriate join condition there. This Hash shouldn't be too complex even for a medium-sized project.
Hard approach: This can be done automatically, if you have has_many, has_one and belongs_to properly defined in your models. You can get the associations of a model using reflect_on_all_associations. Implement a Breath-First-Search or Depth-First Search algorithm and start with any model and search for matching associations to other models from your json input. Start new BFS/DFS runs until there are no unvisited models from the json input left. From the found information, you can derive all join conditions and then add them as expressions in the where clause of the raw sql approach as explained above. Even more complex, but also doable would be reading the database schema and using a similar approach as defined here by looking for foreign keys.
Using associations: If all of them are associated with has_many / has_one, you can handle the joins with ActiveRecord by using the joins method with inject on the "most significant" model like this:
base_model = "Company".constantize
assocations = [:categories] # and so on
result = assocations.inject(base_model) { |model, assoc| model.joins(assoc) }.where(query_where)
What this does:
it passes the base_model as starting input to Enumerable.inject, which will repeatedly call input.send(:joins, :assoc) (for my example this would do Company.send(:joins, :categories) which is equivalent to `Company.categories
on the combined join, it executes the where conditions (constructed as described above)
Disclaimer The exact syntax you need might vary based on the SQL implementation you use.
Full blown SQL string is a security issue, because it exposes your application to a SQL injection attack. If you can get your way around this, it is completely ok to make those query concatenations, as long as you make them compatible with your DB(yes, this solution is DB specific).
Other than that you can make some field that marks some querys as joined, as I have mentioned in the comment, you would have some variable to mark the desired table to be the output of the query, something like:
[
{
"id":"1",
"className":"Category",
"field":"name",
"operator":"does not include",
"val":"ECommerce",
"queryModel":"Company"
}
]
Which, when processing the query, you would use to output the result of this query as the queryModel instead of the className, in those cases the className would be used only to join the table conditions.
I would suggest altering your JSON data. Right now you only send name of the model, without the context, it would be easier if your model would have context.
In your example data would have to look like
data = [
{
id: '0',
className: 'Company',
relation: 'Company',
field: 'name',
operator: 'starts with',
val: 'a'
},
{
id: '1',
className: 'Category',
relation: 'Company.categories',
field: 'name',
operator: 'equals',
val: '12'
},
{
id: '3',
className: 'IndividualInvestor',
relation: 'Company.founding_rounds.investments.individual_investors',
field: 'name',
operator: 'equals',
val: '12'
}
]
And you send this data to QueryBuilder
query = QueryBuilder.new(data)
results = query.find_records
Note: find_records returns array of hashes per model on which you execute query.
For example it would return [{Company: [....]]
class QueryBuilder
def initialize(data)
#data = prepare_data(data)
end
def find_records
queries = #data.group_by {|e| e[:model]}
queries.map do |k, v|
q = v.map do |f|
{
field: "#{f[:table_name]}.#{f[:field]} #{read_operator(f[:operator])} ?",
value: value_based_on_operator(f[:val], f[:operator])
}
end
db_query = q.map {|e| e[:field]}.join(" AND ")
values = q.map {|e| e[:value]}
{"#{k}": k.constantize.joins(join_hash(v)).where(db_query, *values)}
end
end
private
def join_hash(array_of_relations)
hash = {}
array_of_relations.each do |f|
hash.merge!(array_to_hash(f[:joins]))
end
hash.map do |k, v|
if v.nil?
k
else
{"#{k}": v}
end
end
end
def read_operator(operator)
case operator
when 'equals'
'='
when 'starts with'
'LIKE'
end
end
def value_based_on_operator(value, operator)
case operator
when 'equals'
value
when 'starts with'
"%#{value}"
end
end
def prepare_data(data)
data.each do |record|
record.tap do |f|
f[:model] = f[:relation].split('.')[0]
f[:joins] = f[:relation].split('.').drop(1)
f[:table_name] = f[:className].constantize.table_name
end
end
end
def array_to_hash(array)
if array.length < 1
{}
elsif array.length == 1
{"#{array[0]}": nil}
elsif array.length == 2
{"#{array[0]}": array[1]}
else
{"#{array[0]}": array_to_hash(array.drop(1))}
end
end
end
I feel you are over complicating things by having one single controller for everything. I would create a controller for every model or entity that you would want to show and then implement the filters like you said.
Implementing a dynamic where and order by is not very hard but if, as you said, you need to have also the logic to implement some joins you are not only over complicating the solution (because you will have to keep this controller updated every time you add a new model, entity or change the basic logic) but you are also enabling people start playing with your data.
I am not very familiar with Rails so sadly I cannot give you any specific cde other than saying that your approach seems OK to me. I would explode it into multiple controllers.
I have a function that is inserting a record into my DB (MySQL). It has many columns, many of which have default values in the DB. Passing in values for these variables is therefore optional.
def assign_X_to_Y( options = {} )
. . .
#bar.var1 = options[:foo]
. . .
end
I would like to do the following:
-If a variable exists (ex: options[:foo]), add it to the record I'm making.
#bar.var1 = options[:foo]
-If it doesn't, I don't want to add it--I want to use the DB default.
I know I can simply do an if:
if options[:foo]
#bar.var1 = options[:foo]
end
But I have a lot of these variables and so I think there must be a nicer way that having loads of if-statements. Something like the "if doesn't exist set to null" expression:
#bar.var1 = options[:foo] || nil
Is there anything like what I am saying? I can't use the above expression because I don't want to set it to null (which I think it would do), I want to use the default value…
Thanks in advance!
If #bar is an model you can simply pass a hash:
Bar.create(hash) # creates a Bar with the defaults from your schema
#bar.assign_attributes(hash)
#bar.update(hash) # same as object but commits the changes to the db
If bar is a Plain Old Ruby class you can give it the same functionality by:
class Bar
attr_accessor :foo
attr_accessor :baz
attr_accessor :woggle
def initialize(hash)
assign_values(hash)
end
def assign_attributes(hash)
assign_values(hash)
end
private
def assign_values(hash)
hash.each do |k,v|
send "#{k}=", v
end
end
end
Then I can simply create an object with:
Bar.new(foo: 1, baz: 3)
Note that this will respect object encapsulation - if I try to do:
Bar.new(haxxored: true)
It will raise a NoMethodError. Just like #bar.haxxored = true.
If I'm understanding your question correctly, the best way to handle this would be to use the public_send method in Ruby:
def set_new_property(obj, prop_name, prop_value)
obj.class.__send__(attr_accessor: "#{prop_name}")
obj.public_send("#{prop_name}=", prop_value)
end
Bear in mind that you'll have to set explicit attribute accessors for each potential property being assigned.
I'm struggling with ruby's mysql gem and prepares statements.
I want to end up with the same as I would do with each_hash over the result, but it's nor supported in prepares statements.
So I came with this horrible mess.
stmt = #db.prepare("SELECT mat_id, name, qty FROM materials WHERE mat_id = ? ")
#those 3 lines hurt my eyes
res = stmt.execute(params[:id])
mat_id, name, qty = res.bind_result(Integer, String, Integer).fetch
#material = [mat_id: mat_id, name: name, qty: qty]
There has to be a better way to fetch the results and get an array of hashes.
A better mysql gem could be a valid answer. An ORM is NOT.
Seeing the comments, I'll still post the Sequel link as an answer:
http://sequel.rubyforge.org/
You don't need to use the model part of Sequel at all. In fact, the docs has an entire section dedicated to SQL junkies:
http://sequel.rubyforge.org/rdoc/files/doc/sql_rdoc.html
example query:
DB.fetch("SELECT * FROM albums WHERE name LIKE ?", 'A%') do |row|
puts row[:name]
end
Oneliner!
Hash[stmt.result_metadata.fetch_fields.map(&:name).zip( stmt.fetch )]
Or more robust
row = stmt.fetch
Hash[stmt.result_metadata.fetch_fields.map(&:name).zip( row )] if row
According to http://tmtm.org/en/mysql/ruby/, results have an "each_hash" method, but statements don't. What a pain in the ass...
#A proxy for the statement class
class Stmt
def each_hash
fields = #target.result_metadata.fetch_fields.map do |f| f.name.to_sym end
#target.execute.each do |x|
hash = {}
fields.zip(x).each do |pair|
hash[pair[0]] = pair[1]
end
yield hash
end
end
def initialize(target)
#target = target
end
def method_missing(name, *args, &block)
#target.send(name, *args, &block)
end
end
Now, you can do this:
Stmt.new(#db.prepare(...).execute(...)).each_hash do |x|
puts x
end
and you can loop through each row as a hash.
I still haven't tested this for multiple executions
Is there an easy way (i.e. a configuration) to force ActiveRecord to save empty strings as NULL in the DB (if the column allows)?
The reason for this is that if you have a NULLable string column in the DB without a default value, new records that do not set this value will contain NULL, whereas new records that set this value to the empty string will not be NULL, leading to inconsistencies in the database that I'd like to avoid.
Right now I'm doing stuff like this in my models:
before_save :set_nil
def set_nil
[:foo, :bar].each do |att|
self[att] = nil if self[att].blank?
end
end
which works but isn't very efficient or DRY. I could factor this out into a method and mix it into ActiveRecord, but before I go down that route, I'd like to know if there's a way to do this already.
Yes, the only option at the moment is to use a callback.
before_save :normalize_blank_values
def normalize_blank_values
attributes.each do |column, value|
self[column].present? || self[column] = nil
end
end
You can convert the code into a mixin to easily include it in several models.
module NormalizeBlankValues
extend ActiveSupport::Concern
included do
before_save :normalize_blank_values
end
def normalize_blank_values
attributes.each do |column, value|
self[column].present? || self[column] = nil
end
end
end
class User
include NormalizeBlankValues
end
Or you can define it in ActiveRecord::Base to have it in all your models.
Finally, you can also include it in ActiveRecord::Base but enable it when required.
module NormalizeBlankValues
extend ActiveSupport::Concern
def normalize_blank_values
attributes.each do |column, value|
self[column].present? || self[column] = nil
end
end
module ClassMethods
def normalize_blank_values
before_save :normalize_blank_values
end
end
end
ActiveRecord::Base.send(:include, NormalizeBlankValues)
class User
end
class Post
normalize_blank_values
# ...
end
Try if this gem works:
https://github.com/rubiety/nilify_blanks
Provides a framework for saving incoming blank values as nil in the database in instances where you'd rather use DB NULL than simply a blank string...
In Rails when saving a model from a form and values are not provided by the user, an empty string is recorded to the database instead of a NULL as many would prefer (mixing blanks and NULLs can become confusing). This plugin allows you to specify a list of attributes (or exceptions from all the attributes) that will be converted to nil if they are blank before a model is saved.
Only attributes responding to blank? with a value of true will be converted to nil. Therefore, this does not work with integer fields with the value of 0, for example...
Another option is to provide custom setters, instead of handling this in a hook. E.g.:
def foo=(val)
super(val == "" ? nil : val)
end
My suggestion:
# app/models/contact_message.rb
class ContactMessage < ActiveRecord::Base
include CommonValidations
include Shared::Normalizer
end
# app/models/concerns/shared/normalizer.rb
module Shared::Normalizer
extend ActiveSupport::Concern
included do
before_save :nilify_blanks
end
def nilify_blanks
attributes.each do |column, value|
# ugly but work
# self[column] = nil if !self[column].present? && self[column] != false
# best way
#
self[column] = nil if self[column].kind_of? String and self[column].empty?
end
end
end
Sorry for necroposting, but I didn't find exact thing here in answers, if you need solution to specify fields which should be nilified:
module EnforceNil
extend ActiveSupport::Concern
module ClassMethods
def enforce_nil(*args)
self.class_eval do
define_method(:enforce_nil) do
args.each do |argument|
field=self.send(argument)
self.send("#{argument}=", nil) if field.blank?
end
end
before_save :enforce_nil
end
end
end
end
ActiveRecord::Base.send(:include, EnforceNil)
This way:
class User
enforce_nil :phone #,:is_hobbit, etc
end
Enforcing certain field is handy when let's say you have field1 and field2. Field1 has unique index in SQL, but can be blank, so you need enforcement(NULL considered unique, "" - not by SQL), but for field2 you don't actually care and you have already dozens of callbacks or methods, which work when field2 is "", but will dig your app under the layer of errors if field2 is nil. Situation I faced with.
May be useful for someone.
Strip Attributes Gem
There's a handy gem that does this automatically when saving a record, whether that's in a user form or in the console or in a rake task, etc.
It's called strip_attributes and is extremely easy to use, with sane defaults right out of the box.
It does two main things by default that should almost always be done:
Strip leading and trailing white space:
" My Value " #=> "My Value"
Turn empty Strings into NULL:
"" #=> NULL
" " #=> NULL
Install
You can add it to your gem file with:
gem strip_attributes
Usage
Add it to any (or all) models that you want to strip leading/trailing whitespace from and turn empty strings into NULL:
class DrunkPokerPlayer < ActiveRecord::Base
strip_attributes
end
Advanced Usage
There are additional options that you can pass on a per-Model basis to handle exceptions, like if you want to retain leading/trailing white space or not, etc.
You can view all of the options on the GitHub repository here:
https://github.com/rmm5t/strip_attributes#examples
I use the attribute normalizer gem to normalize attributes before they into the db.