example model Users :
create_table "users"
t.string "name"
t.boolean "admin"
t.boolean "internal"
end
needs to be queried with "admin" or "internal" in the where clause of the SQL query. At the time of writing the query, it is not known which boolean will be used, so a variable, say usertype, is used to store the string "admin" or "internal". It basically means I would like to use a referenced attribute in my where clause.
Doing the following query :
User.select("id").where("#{usertype} = 't'")
works fine in my development sqlite database (since sqlite stores true as 't' and false as 'f'), but I guess that using the same query on another database (Mysql or Postrgres e.g.) might not work.
Alternatively I tried
User.select("id").where("#{usertype} = ?", true)
it also works and does not use the sqlite-specific 't'. However, is this a good and database independent solution ?
You should always use true or false (or variables or code which will evaluate 'truthy' or 'falsy`) and let rails handle the translation into however your DBMS stores booleans.
You do this in your second example
User.select("id").where("#{usertype} = ?", true)
So, the answer is "yes this is a good, database-independent solution". But, you can make it a bit neater: if all your conditions are of the "=" variety then you can use a hash, like
{:foo => "bar"}
{"foo" => "bar"}
which are both equivalent to
["foo = ?", "bar"]
This means in your case you can dispense with the string evalution and just pass your variable in.
User.select("id").where({usertype => true})
or, if you want to skip the optional { & }
User.select("id").where(usertype => true)
Edit: reply to comment about using a dynamically defined method in your ruby code, to get a value from an object.
You can use the send method to call the appropriate accessor on the object. For example,
#foo.bar == 123
is equivalent to
#foo.send("bar") == 123
So, in your example you would say
a.send(usertype)
As a sidenote, if you wanted to set the value of usertype (which i don't think would make sense in your case, but just an an illustration), you would need to call the "setter" version of the method, which would be either admin= or internal=. So, in this case you would need to use string evaluation:
#user.send("#{usertype}=", "foo")
which is like saying
#user.admin = "foo"
or
#user.internal = "foo"
Related
Im trying to replicate the searching list style of crunchbase using ruby on rails.
I have an array of filters that looks something like this:
[
{
"id":"0",
"className":"Company",
"field":"name",
"operator":"starts with",
"val":"a"
},
{
"id":"1",
"className":"Company",
"field":"hq_city",
"operator":"equals",
"val":"Karachi"
},
{
"id":"2",
"className":"Category",
"field":"name",
"operator":"does not include",
"val":"ECommerce"
}
]
I send this json string to my ruby controller where I have implemented this logic:
filters = params[:q]
table_names = {}
filters.each do |filter|
filter = filters[filter]
className = filter["className"]
fieldName = filter["field"]
operator = filter["operator"]
val = filter["val"]
if table_names[className].blank?
table_names[className] = []
end
table_names[className].push({
fieldName: fieldName,
operator: operator,
val: val
})
end
table_names.each do |k, v|
i = 0
where_string = ''
val_hash = {}
v.each do |field|
if i > 0
where_string += ' AND '
end
where_string += "#{field[:fieldName]} = :#{field[:fieldName]}"
val_hash[field[:fieldName].to_sym] = field[:val]
i += 1
end
className = k.constantize
puts className.where(where_string, val_hash)
end
What I do is, I loop over the json array and create a hash with keys as table names and values are the array with the name of the column, the operator and the value to apply that operator on. So I would have something like this after the table_names hash is created:
{
'Company':[
{
fieldName:'name',
operator:'starts with',
val:'a'
},
{
fieldName:'hq_city',
operator:'equals',
val:'karachi'
}
],
'Category':[
{
fieldName:'name',
operator:'does not include',
val:'ECommerce'
}
]
}
Now I loop over the table_names hash and create a where query using the Model.where("column_name = :column_name", {column_name: 'abcd'}) syntax.
So I would be generating two queries:
SELECT "companies".* FROM "companies" WHERE (name = 'a' AND hq_city = 'b')
SELECT "categories".* FROM "categories" WHERE (name = 'c')
I have two problems now:
1. Operators:
I have many operators that can be applied on a column like 'starts with', 'ends with', 'equals', 'does not equals', 'includes', 'does not includes', 'greater than', 'less than'. I am guessing the best way would be to do a switch case on the operator and use the appropriate symbol while building the where string. So for example, if the operator is 'starts with', i'd do something like where_string += "#{field[:fieldName]} like %:#{field[:fieldName]}" and likewise for others.
So is this approach correct and is this type of wildcard syntax allowed in this kind of .where?
2. More than 1 table
As you saw, my approach builds 2 queries for more than 2 tables. I do not need 2 queries, I need the category name to be in the same query where the category belongs to the company.
Now what I want to do is I need to create a query like this:
Company.joins(:categories).where("name = :name and hq_city = :hq_city and categories.name = :categories[name]", {name: 'a', hq_city: 'Karachi', categories: {name: 'ECommerce'}})
But this is not it. The search can become very very complex. For example:
A Company has many FundingRound. FundingRound can have many Investment and Investment can have many IndividualInvestor. So I can select create a filter like:
{
"id":"0",
"className":"IndividualInvestor",
"field":"first_name",
"operator":"starts with",
"val":"za"
}
My approach would create a query like this:
SELECT "individual_investors".* FROM "individual_investors" WHERE (first_name like %za%)
This query is wrong. I want to query the individual investors of the investments of the funding round of the company. Which is a lot of joining tables.
The approach that I have used is applicable to a single model and cannot solve the problem that I stated above.
How would I solve this problem?
You can create a SQL query based on your hash. The most generic approach is raw SQL, which can be executed by ActiveRecord.
Here is some concept code that should give you the right idea:
query_select = "select * from "
query_where = ""
tables = [] # for selecting from all tables
hash.each do |table, values|
table_name = table.constantize.table_name
tables << table_name
values.each do |q|
query_where += " AND " unless query_string.empty?
query_where += "'#{ActiveRecord::Base.connection.quote(table_name)}'."
query_where += "'#{ActiveRecord::Base.connection.quote(q[fieldName)}'"
if q[:operator] == "starts with" # this should be done with an appropriate method
query_where += " LIKE '#{ActiveRecord::Base.connection.quote(q[val)}%'"
end
end
end
query_tables = tables.join(", ")
raw_query = query_select + query_tables + " where " + query_where
result = ActiveRecord::Base.connection.execute(raw_query)
result.to_h # not required, but raw results are probably easier to handle as a hash
What this does:
query_select specifies what information you want in the result
query_where builds all the search conditions and escapes input to prevent SQL injections
query_tables is a list of all the tables you need to search
table_name = table.constantize.table_name will give you the SQL table_name as used by the model
raw_query is the actual combined sql query from the parts above
ActiveRecord::Base.connection.execute(raw_query) executes the sql on the database
Make sure to put any user submitted input in quotes and escape it properly to prevent SQL injections.
For your example the created query will look like this:
select * from companies, categories where 'companies'.'name' LIKE 'a%' AND 'companies'.'hq_city' = 'karachi' AND 'categories'.'name' NOT LIKE '%ECommerce%'
This approach might need additional logic for joining tables that are related.
In your case, if company and category have an association, you have to add something like this to the query_where
"AND 'company'.'category_id' = 'categories'.'id'"
Easy approach: You can create a Hash for all pairs of models/tables that can be queried and store the appropriate join condition there. This Hash shouldn't be too complex even for a medium-sized project.
Hard approach: This can be done automatically, if you have has_many, has_one and belongs_to properly defined in your models. You can get the associations of a model using reflect_on_all_associations. Implement a Breath-First-Search or Depth-First Search algorithm and start with any model and search for matching associations to other models from your json input. Start new BFS/DFS runs until there are no unvisited models from the json input left. From the found information, you can derive all join conditions and then add them as expressions in the where clause of the raw sql approach as explained above. Even more complex, but also doable would be reading the database schema and using a similar approach as defined here by looking for foreign keys.
Using associations: If all of them are associated with has_many / has_one, you can handle the joins with ActiveRecord by using the joins method with inject on the "most significant" model like this:
base_model = "Company".constantize
assocations = [:categories] # and so on
result = assocations.inject(base_model) { |model, assoc| model.joins(assoc) }.where(query_where)
What this does:
it passes the base_model as starting input to Enumerable.inject, which will repeatedly call input.send(:joins, :assoc) (for my example this would do Company.send(:joins, :categories) which is equivalent to `Company.categories
on the combined join, it executes the where conditions (constructed as described above)
Disclaimer The exact syntax you need might vary based on the SQL implementation you use.
Full blown SQL string is a security issue, because it exposes your application to a SQL injection attack. If you can get your way around this, it is completely ok to make those query concatenations, as long as you make them compatible with your DB(yes, this solution is DB specific).
Other than that you can make some field that marks some querys as joined, as I have mentioned in the comment, you would have some variable to mark the desired table to be the output of the query, something like:
[
{
"id":"1",
"className":"Category",
"field":"name",
"operator":"does not include",
"val":"ECommerce",
"queryModel":"Company"
}
]
Which, when processing the query, you would use to output the result of this query as the queryModel instead of the className, in those cases the className would be used only to join the table conditions.
I would suggest altering your JSON data. Right now you only send name of the model, without the context, it would be easier if your model would have context.
In your example data would have to look like
data = [
{
id: '0',
className: 'Company',
relation: 'Company',
field: 'name',
operator: 'starts with',
val: 'a'
},
{
id: '1',
className: 'Category',
relation: 'Company.categories',
field: 'name',
operator: 'equals',
val: '12'
},
{
id: '3',
className: 'IndividualInvestor',
relation: 'Company.founding_rounds.investments.individual_investors',
field: 'name',
operator: 'equals',
val: '12'
}
]
And you send this data to QueryBuilder
query = QueryBuilder.new(data)
results = query.find_records
Note: find_records returns array of hashes per model on which you execute query.
For example it would return [{Company: [....]]
class QueryBuilder
def initialize(data)
#data = prepare_data(data)
end
def find_records
queries = #data.group_by {|e| e[:model]}
queries.map do |k, v|
q = v.map do |f|
{
field: "#{f[:table_name]}.#{f[:field]} #{read_operator(f[:operator])} ?",
value: value_based_on_operator(f[:val], f[:operator])
}
end
db_query = q.map {|e| e[:field]}.join(" AND ")
values = q.map {|e| e[:value]}
{"#{k}": k.constantize.joins(join_hash(v)).where(db_query, *values)}
end
end
private
def join_hash(array_of_relations)
hash = {}
array_of_relations.each do |f|
hash.merge!(array_to_hash(f[:joins]))
end
hash.map do |k, v|
if v.nil?
k
else
{"#{k}": v}
end
end
end
def read_operator(operator)
case operator
when 'equals'
'='
when 'starts with'
'LIKE'
end
end
def value_based_on_operator(value, operator)
case operator
when 'equals'
value
when 'starts with'
"%#{value}"
end
end
def prepare_data(data)
data.each do |record|
record.tap do |f|
f[:model] = f[:relation].split('.')[0]
f[:joins] = f[:relation].split('.').drop(1)
f[:table_name] = f[:className].constantize.table_name
end
end
end
def array_to_hash(array)
if array.length < 1
{}
elsif array.length == 1
{"#{array[0]}": nil}
elsif array.length == 2
{"#{array[0]}": array[1]}
else
{"#{array[0]}": array_to_hash(array.drop(1))}
end
end
end
I feel you are over complicating things by having one single controller for everything. I would create a controller for every model or entity that you would want to show and then implement the filters like you said.
Implementing a dynamic where and order by is not very hard but if, as you said, you need to have also the logic to implement some joins you are not only over complicating the solution (because you will have to keep this controller updated every time you add a new model, entity or change the basic logic) but you are also enabling people start playing with your data.
I am not very familiar with Rails so sadly I cannot give you any specific cde other than saying that your approach seems OK to me. I would explode it into multiple controllers.
I want to save a true/false in my MySQL database. I'm saving 1/0 in an INT column to do this. When I select it, I get the 1 or 0, but I want it to return true/false to my PHP code, without having to rewrite the database.
Can I use another column type? Should I save it differently?
Update: My question is about not wanting to rewrite the returned value. I'm getting a lot of results from my database. Many of those are true/false, but some are 0s because the price is 0, so I don't want to universally rewrite all 1s and 0s. I also don't want to manually rewrite 10 columns.
To follow up my comment, here's a more detailed response which also covers the PHP side, although this probably belongs on StackOverflow.
I've always just used tinyint, although you can use bool/boolean which are synonyms for tinyint(1)
However as of MySQL 5.0.3 you can use the bit type:
As of MySQL 5.0.3, the BIT data type is used to store bit-field values. A type of BIT(M) enables storage of M-bit values. M can range from 1 to 64.
Next, assuming you have an active column, perhaps to store if a user is active, you could use PHP's automatic type conversion to handle this quite simply.
// Obviously you'd replace this with your database call
$results = [['active' => 1], ['active' => 0]];
foreach($results as $row) {
if ($row['active'] == true) {
echo "true\n";
}
else {
echo "false\n";
}
}
You don't strictly need to do anything.
PHP does not, and can not, use strongly typed variables. So, if you receive an (int) 1 from your query results, you can simply use this 1 as a boolean without rewriting or changing anything.
$intOne = (int) 1; //explicitly treat the variable as an integer
var_dump((bool) $intOne); //treat the variable as a boolean
When used in any boolean context, like if ($variable)... then any of these types will be considered to be false by PHP:
the boolean FALSE itself
the integer 0 (zero)
the float 0.0 (zero)
the empty string, and the string "0"
an array with zero elements
an object with zero member variables (PHP 4 only)
the special type NULL (including unset variables)
SimpleXML objects created from empty tags
... And, most importantly;
Every other value is considered TRUE (including any resource).
Source: PHP Manual > Booleans (english)
So while you can change the storage type of your column in mysql, this won't really change the way PHP handles the variable retrieved from your results at all.
Historically, I've always used a column of type TINYINT(1) to store boolean values in mysql, and as Tom Green points out, recent mysql versions provide a new BIT type, which might be appropriate. To the best of my knowledge, mysql does not currently have an actual boolean data type.
You could just as easily use a column of type VARCHAR(1), though, because PHP can and will use any value as a boolean, thanks to the glorious, majestic, and sometimes maddening, PHP Type Juggling.
If you're trying to use the values you're retrieving for boolean logic, just use the values you receive from mysql like booleans and it will work:
if ($valueFromResults) {
//The value was something like true
} else {
//The value was something like false
}
If you're trying to actually echo out the words "true" and "false", then you're probably best served by explicitly echoing the words out yourself, like this;
if ($valueFromResults) {
echo "true";
} else {
echo "false";
}
or, in my preferred shorthand;
echo ($valueFromResults) ? "true" : "false" ;
Update You mentioned in a comment that you want to pass the values through json_encode() and use them in javascript.
JavaScript treats any real value, like int 1, as true and any empty value, like int 0, or an empty string, as false. So if your json_encode() output gets used in actual JavaScript, the int values will still work as boolean values. So the integer values from your database should still work as they are.
Just check that your integer results are encoded as integers by PHP and not as strings - they should be encoded correctly by default - because "0" == true in javascript, but 0 == false.
For a boolean value (true/false), you should use the mySql type bit or tinyint(1).
$boolean = $mysql_data ? true : false;
I have a mysql query that returns this type of data:
{"id"=>1, "serviceCode"=>"1D00", "price"=>9.19}
{"id"=>2, "serviceCode"=>"1D01", "price"=>9.65}
I need to return the id field based on a match of the serviceCode.
i.e. I need a method like this
def findID(serviceCode)
find the row that has the service code and return the ID
end
I was thinking of having a serviceCodes.each do |row| method and loop through and essentially go
if row == serviceCode
return row['id']
end
is there a faster / easier way?
You can use the method Enumerable#find:
service_codes = [
{"id"=>1, "serviceCode"=>"1D00", "price"=>9.19},
{"id"=>2, "serviceCode"=>"1D01", "price"=>9.65}
]
service_codes.find { |row| row['serviceCode'] == '1D00' }
# => {"id"=>1, "serviceCode"=>"1D00", "price"=>9.19}
If you use Rails Active Record as ORM and your Model named Product (only for example),
you can use something like this:
def findID(serviceCode)
Product.select(:id).where(serviceCode: serviceCode).first
end
If you have plain SQL Query in plain ruby class (not recommended), you should change this query to get only the id, as Luiggi mentioned. But aware of SQL Injections if your serviceCode coming from external Requests.
I'm working with a database schema that has a relationship that isn't always true, and I'm not sure how to describe it with sqlalchemy's ORM.
All the primary keys in this database are stored as a blob type, and are 16 byte binary strings.
I have a table called attribute, and this table has a column called data_type. There are a number of built in data_types, that are not defined explicitly in the database. So, maybe a data_type of 00 means it is a string, and 01 means it is a float, etc (those are hex values). The highest value for the built in data types is 12 (18 in decimal).
However, for some rows in attribute, the value of the attribute stored in the row must exist in a pre-defined list of values. In this case, data_type referrs to lookup.lookup_id. The actual data type for the attribute can then be retrieved from lookup.data_type.
I'd like to be able to call just Attribue.data_type and get back 'string' or 'number'. Obviously I'd need to define the {0x00: 'string', 0x01: 'number'} mapping somewhere, but how can I tell sqlalchemy that I want lookup.data_type if the value of attribute.data_type is greater than 18?
There are a couple of ways to do this.
The simplest, by far, is to just put your predefined data types into the table lookup. You say that you "need to define the... mapping somewhere", and a table is as good a place as any.
Assuming that you can't do that, the next simplest thing is to create a python property on class Attribute. The only problem will be that you can't query against it. You'll want to reassign the column data_type so that it maps to _data_type:
data_type_dict = {0x00: 'string',
0x01: 'number,
...}
class Attribute(Base):
__tablename__ = 'attribute'
_data_type = Column('data_type')
...
#property
def data_type(self):
dt = data_type_dict.get(self._data_type, None)
if dt is None:
s = Session.object_session(self)
lookup = s.query(Lookup).filter_by(id=self._data_type).one()
dt = lookup.data_type
return dt
If you want this to be queryable, that is, if you want to be able to do session.query(Attribute).filter_by(data_type='string'), you need to map data_type to something the database can handle, i.e., an SQL statement. You could do this in raw SQL as a CASE expression:
from sqlalchemy.sql.expression import select, case
class Attribute(Base):
...
data_type = column_property(select([attribute, lookup])\
.where(attribute.data_type==lookup.lookup_id)\
.where(case([(attribute.data_type==0x00, 'string'),
(attribute.data_type==0x01, 'number'),
...],
else_=lookup.data_type))
I'm not 100% certain that last part will work; you may need to explicitly join the tables attribute and lookup to specify that it's an outer join, though I think SQLAlchemy does that by default. The downside of this approach is that you are always going to try to join with the table lookup, though to query using SQL, you sort of have to do that.
The final option is to use a polymorphism, and map the two cases (data_type greater/less than 18) to two different subclasses:
class Attribute(Base):
__tablename__ = 'attribute'
_data_type = Column('data_type')
_lookup = column_property(attribute.data_type > 18)
__mapper_args__ = {'polymorphic_on': _lookup}
class FixedAttribute(Attribute):
__mapper_args__ = {'polymorphic_identity': 0}
data_type = column_property(select([attribute.data_type])\
.where(case([(attribute.data_type==0x00, 'string'),
(attribute.data_type==0x01, 'number'),
...])))
class LookupAttribute(Attribute):
__mapper_args__ = {'polymorphic_identity': 1}
data_type = column_property(select([lookup.data_type],
whereclause=attribute.data_type==lookup.lookup_id))
You might have to replace the 'polymorphic_on': _lookup with an explicit attribute.data_type > 18, depending on when that ColumnProperty gets bound.
As you can see, these are all really messy. Do #1 if it's at all possible.
I have this:
var sortName = Request.Params["sortName"];
var query = Request.Params["query"];
Func<UsuarioEndereco, bool> whereClause = (uen => uen.GetPropValue<string>(sortName).Contains(query));
The "uen.GetPropValue<string>(sortName)" will be filled dynamically with the sortName the user typed in the page.
For example, if an user looks for a person named "Joe", the snippet will be:
(uen => uen.namePerson.Contains(Joe))
But, I'm having problems with LINQ Case-sensitive searches. If I type "Joe", I will something. On the other hand, If I type "joe", it bring nothing.
How can I make this "Contains(sortName)" works with Case-Insensitive?? I've tried some things with String.Comparer but it reports errors on build solution.
Thanks!!
I believe the following will generate proper SQL:
uen=>(uen.GetPropValue<string>(sortName)).ToLower().Contains(query.ToLower()))
If this is really LINQ-to-SQL, try using the SqlMethods.Like method instead of String.Contains.
However, I think the problem is that this is NOT LINQ-to-SQL, because you are using delegates instead of Expression trees. So this is being brought client side, then executed locally ("LINQ to Objects"). Hence, String.Contains is doing what it does locally.
In that way, James's answer is correct, since he's calling ToLower() on both the value and the query. (Although, beware of culture issues -- perhaps specify which culture you want.)
You could also use the String.IndexOf Method (String, Int32, StringComparison) (http://msdn.microsoft.com/en-us/library/ms224424.aspx). This method allows you to specify if the matching should be done case-sensitively or not, and if it should use a Invariant culture or not.
So in your example:
Func<UsuarioEndereco, bool> whereClause = (uen => uen.GetPropValue<string>(sortName).IndexOf(query, 0, StringComparison.OrdinalIgnoreCase));
I'm not commenting on if this is a better solution than the one provided by James Curran. It could or could not be, performance wise.
This is the entire code:
var sortOrder = Request.Params["sortorder"];
var sortName = Request.Params["sortname"];
var query = Request.Params["query"];
IEnumerable<UsuarioEndereco> pagedEndereco;
Func<UsuarioEndereco, bool> whereClause = (uen => uen.GetPropValue<string>(sortName).Contains(query));
pagedEndereco = sortOrder.Equals("asc", StringComparison.CurrentCultureIgnoreCase) ?
_agendaServico.SelecionaUsuarioEnderecos(u.codUsuario).Where(whereClause).OrderByDescending(uen => uen.GetPropValue<IComparable>(sortName)) :
_agendaServico.SelecionaUsuarioEnderecos(u.codUsuario).Where(whereClause).OrderBy(uen => uen.GetPropValue<IComparable>(sortName));
The Extension Method GetPropValue is:
public static T GetPropValue<T>(this object component, string propertyName)
{
return (T)TypeDescriptor.GetProperties(component)[propertyName].GetValue(component);
}