I'm trying to accomplish a multiple word searching in a quotes database using Ruby, ActiveRecord, and MySQL. The way I did is shown bellow, and it is working, but I would like to know if there a better way to do.
# receives a string, splits it in a array of words, create the 'conditions'
# query, and send it to ActiveRecord
def search
query = params[:query].strip.split if params[:query]
like = "quote LIKE "
conditions = ""
query.each do |word|
conditions += (like + "'%#{word}%'")
conditions += " AND " unless query.last == word
end
#quotes = Quote.all(:conditions => conditions)
end
I would like to know if there is better way to compose this 'conditions' string. I also tried it using string interpolation, e.g., using the * operator, but ended up needing more string processing. Thanks in advance
First, I strongly encourage you to move Model's logic into Models. Instead of creating the search logic into the Controller, create a #search method into your Quote mode.
class Quote
def self.search(query)
...
end
end
and your controller becomes
# receives a string, splits it in a array of words, create the 'conditions'
# query, and send it to ActiveRecord
def search
#quotes = Quote.search(params[:query])
end
Now, back to the original problem. Your existing search logic does a very bad mistake: it directly interpolates value opening your code to SQL injection. Assuming you use Rails 3 you can take advantage of the new #where syntax.
class Quote
def self.search(query)
words = query.to_s.strip.split
words.inject(scoped) do |combined_scope, word|
combined_scope.where("quote LIKE ?", "%#{word}%")
end
end
end
It's a little bit of advanced topic. I you want to understand what the combined_scope + inject does, I recommend you to read the article The Skinny on Scopes.
MySQL fulltext search not working, so best way to do this:
class Quote
def self.search_by_quote(query)
words = query.to_s.strip.split
words.map! { |word| "quote LIKE '%#{word}%'" }
sql = words.join(" AND ")
self.where(sql)
end
end
The better way to do it would be to implement full text searching. You can do this in MySQL but I would highly recommend Solr. There are many resources online for implementing Solr within rails but I would recommend Sunspot as an entrance point.
Create a FULLTEXT index in MySQL. With that, you can leave string processing to MySQL.
Example : http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
Related
I have a database entry that looks like the following:
name = servername\vs1
We have a search that is looking for this term.
scope :search, ->(term) {
if term
where('name LIKE ?', "%#{term}%")
else
all
end
}
However, it isn't finding it. When someone searches for severname, of course, it shows up. But when they include the backslash it isn't found.
After doing some research, I found that rails is currently adding a single backslash to the query term prior to search (servername\\vs1) but mysql needs the following format: (servername\\\\vs1).
So, I was hoping there was an easy rails way to add additional backslashes. Looking for any good solution.
Thanks
Easiest to use Arel, like this:
scope :search, ->(term){
t = arel_table
term ? where( t[:name].matches("%#{term}%") )
: all
}
Example:
Simple.search('\a').to_sql
"SELECT \"simples\".* FROM \"posts\" WHERE (\"simples\".\"title\" LIKE '%\\a%')"
I can imagine it can be very easy just for lazy people like me to populate any fake data to db just with one rake (terminal) command.
I know about Faker, Populator and others but all of them, as far as i can see, need to write some (primitive but) code to make data more human friendly (defining type of random data directly and manually: emalis, names, prices and so on).
It makes sense in most cases but now in my case it would be enough for me to fill mysql varchar fields with any strings, text fields with any long text, int - with numbers and so on
any suggestions?
If speed is your aim, you should do two things:
Use an in-memory database for your tests until you get to acceptance testing. In other words, consider something like SQLite for your integration tests (Some might say unit tests) rather than MySQL.
Use Factory Girl to generate your fake data. Apparently, the data created by tools like that makes more sense than you prefer, but it is weird to me that you care about that. Regardless, it is a lot faster to use existing tools than to write code that generates gibberish just because you don't want data that look "too good."
Some example code that shows how to do that:
SKIP_COLUMNS = %w(id created_at updated_at)
RECORDS_COUNT = 10
# random data to fill
int = rand(1..100)
varchar = 'lorem'
text = 'big lorem'
# get models
#models = ActiveRecord::Base.connection.tables.collect {|t| t.underscore.singularize.camelize }
#models.select {|m| m.constantize rescue #models.delete(m) }
# fill in data
#models.map(&:constantize).each do |model|
model.columns_hash.each do |column|
next if SKIP_COLUMNS.include?(column.first)
# column_name = column.first
# column_type = column.last.type
RECORDS_COUNT.times do
record = model.new
case column.last.type
when :integer
record.send("#{column.first}=", int)
when :string
record.send("#{column.first}=", varchar)
when :text
record.send("#{column.first}=", text)
end
record.save!
end
end
end
You can put that to rake task.
i found a code that written in Rogue that is
Rogue is a type-safe internal Scala DSL for constructing and executing find and modify commands against MongoDB in the Lift web framework
i found code that is written in scala that is query to fetch data
and i want to convert that code into mysql query
def getBlogScore(word: String, blog: String): Long = Keyword.where(_.word eqs word).and(_.blog eqs blog)
.fetch.map(_.score._1)
.reduceLeftOption(_ + _).getOrElse(0)
give me some idea!
Since .reduceLeft( _ + _ ) would add all the scores together, this is probably what you're looking for:
SELECT SUM(score)
FROM keyword
WHERE blog = 'blog'
AND word = 'word'
I'm using Rails with ActiveAdmin gem. And I want to select some information from mysql database.
sql = ActiveRecord::Base.connection();
s="SELECT word FROM dics WHERE word LIKE 'tung%'";
ten = sql.execute(s);
But when I printed out "ten" to screen, it showed that:
#<Mysql2::Result:0x4936260>
How can I get the information of records?
I suggest that you don't use ActiveRecord::Base.connection directly. Sticking with ARel syntax should work for most cases, and your example doesn't seem like an edge case.
As stated in the comments above, try the following:
dics = Dic.select(:word).where(["word LIKE ?", "tung%"]).all
In order to pluck some special field of object, not objects themselves, use pluck instead of all:
# instead of .pluck(:word) use real field identifier
dics = Dic.where(["word LIKE ?", "tung%"]).pluck(:word)
Assume Rails 3 with MySQL DB with Case Insensitive collation
What's the story:
Rails allows you to validate an attribute of a Model with the "uniqueness" validator. BUT the default comparison is CASE SENSITIVE according to Rails documentation.
Which means that on validation it executes SQL like the following:
SELECT 1 FROM `Users` WHERE (`Users`.`email` = BINARY 'FOO#email.com') LIMIT 1
This works completely wrong for me who has a DB with CI Collation. It will consider the 'FOO#email.com' valid, even if there is another user with 'foo#email.com' already in Users table. In other words, this means, that if the user of the application tries to create a new User with email 'FOO#email.com' this would have been completely VALID (by default) for Rails and INSERT will be sent to db. If you do not happen to have unique index on e-mail then you are boomed - row will be inserted without problem. If you happen to have a unique index, then exception will be thrown.
Ok. Rails says: since your DB has case insensitive collation, carry out a case insensitive uniqueness validation.
How is this done? It tells that you can override the default uniqueness comparison sensitivity by setting ":case_sensitive => false" on the particular attribute validator. On validation it creates the following SQL:
SELECT 1 FROM `Users` WHERE (LOWER(`Users`.`email`) = LOWER('FOO#email.com') LIMIT 1
which is a DISASTER on a database table Users that you have designed to have a unique index on the email field, because it DOES NOT USE the index, does full table scan.
I now see that the LOWER functions in SQL are inserted by the UniquenessValidator of ActiveRecord (file uniqueness.rb, module ActiveRecord, module Validations class UniquenessValidator). Here is the piece of code that does this:
if value.nil? || (options[:case_sensitive] || !column.text?)
sql = "#{sql_attribute} #{operator}"
else
sql = "LOWER(#{sql_attribute}) = LOWER(?)"
end
So Question goes to Rails/ActiveRecord and not to MySQL Adapter.
QUESTION: Is there a way to tell Rails to pass the requirement about uniqueness validation case sensitivity to MySQL adapter and not be 'clever' about it to alter the query? OR
QUESTION REPHRASED FOR CLARIFICATION: Is there another way to implement uniqueness validation on an attribute (PLEASE, CAREFUL...I AM NOT TALKING ABOUT e-mail ONLY, e-mail was given as an example) with case sensitivity OFF and with generation of a query that will use a simple unique index on the corresponding column?
These two questions are equivalent. I hope that now, I make myself more clear in order to get more accurate answers.
Validate uniqueness without regard to case
If you want to stick to storing email in upper or lower case then you can use the following to enforce uniqueness regardless of case:
validates_uniqueness_of :email, case_sensitive: false
(Also see this question:
Rails "validates_uniqueness_of" Case Sensitivity)
Remove the issue of case altogether
Rather than doing a case insensitive match, why not downcase the email before validating (and therefore also):
before_validation {self.email = email.downcase}
Since case is irrelevant to email this will simplify everything that you do as well and will head off any future comparisons or database searches you might be doing
I have searched around and the only answer, according to my knowledge today, that can be acceptable is to create a validation method that does the correct query and checks. In other words, stop using :uniqueness => true and do something like the following:
class User
validate :email_uniqueness
protected
def email_uniqueness
entries = User.where('email = ?', email)
if entries.count >= 2 || entries.count == 1 && (new_record? || entries.first.id != self.id )
errors[:email] << _('already taken')
end
end
end
This will definitely use my index on email and works both on create and update (or at least it does up to the point that I have tested that ok).
After asking on the RubyOnRails Core Google group
I have taken the following answer from RubyOnRails Core Google Group: Rails is fixing this problem on 3.2. Read this:
https://github.com/rails/rails/commit/c90e5ce779dbf9bd0ee53b68aee9fde2997be123
Workaround
If you want a case-insensitive comparison do:
SELECT 1 FROM Users WHERE (Users.email LIKE 'FOO#email.com') LIMIT 1;
LIKE without wildcards always works like a case-insensitive =.
= can be either case sensitive or case-insensitive depending on various factors (casting, charset...)
starting with http://guides.rubyonrails.org/active_record_querying.html#finding-by-sql
then adding their input
#Johan,
#PanayotisMatsinopoulos
and this
http://guides.rubyonrails.org/active_record_validations_callbacks.html#custom-methods
and http://www.w3schools.com/sql/sql_like.asp
then we have this:
class User < ActiveRecord::Base
validate :email_uniqueness
protected
def email_uniqueness
like_emails = User.where("email LIKE ?", email))
if (like_emails.count >= 2 || like_emails.count == 1
&& (new_record? || like_emails.first.id != self.id ))
errors[:email] << _('already taken')
end
end
end
validates :email, uniqueness: {case_sensitive: false}
Works like a charm in Rails 4.1.0.rc2
;)
After fighting with MySQL binary modifier, i found a way that removes that modifier from all queries comparing fields (not limited to uniqueness validation, but includes it).
First: Why MySQL adds that binary modifier? That's because by default MySQL compares fields in a case-insensitive way.
Second: Should I care? I always had design my systems to suppose that String comparison are made in a case-insensitive way, so that is a desired feature to me. Be warned if you don't
This is where is added the binary modifier:
https://github.com/rails/rails/blob/ee291b9b41a959e557b7732100d1ec3f27aae4f8/activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb#L545
def case_sensitive_modifier(node)
Arel::Nodes::Bin.new(node)
end
So i override this. I create an initializer (at config/initializers) named "mysql-case-sensitive-override.rb" with this code:
# mysql-case-sensitive-override.rb
class ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter < ActiveRecord::ConnectionAdapters::AbstractAdapter
def case_sensitive_modifier(node)
node
end
end
And that's it. No more binary modifier on my queries :D
Please notice that this does not explain why the "{case_sensitive: false}" option of the validator doesn't works, and does not solves it. It changes the default-and-unoverrideable-case-sensitive behavior for a default-and-unoverrideable-case-insensitive new behavior. I must insist, this also changes for any comparison that actually uses binary modifier for case-sensitive behavior (i hope).