Model:
class TechRequest < ActiveRecord::Base
...
define_index do
...
indexes :hot_request
indexes :status_id, :as => :current_status_id
...
has :hot_request , :as => :hot_request
set_property :delta => true
end
DB:
hot_request - tinyint(1)
When I execute the controller code-
#query_string = '(#hot_request 1)(#current_status_id 1 | 2 | 3)'
#tech_requests = TechRequest.search #query_string, :match_mode => :extended
the following error is thrown up:
ThinkingSphinx::SphinxError: index tech_request_core,tech_request_delta: query error: no field 'tech_hot_request' found in schema
from D:/Current/TechAssistTest/vendor/plugins/thinking-sphinx/lib/thinking_sphinx/search.rb:392:in 'populate'
from D:/Current/TechAssistTest/vendor/plugins/thinking-sphinx/lib/thinking_sphinx/search.rb:508:in 'call'
from D:/Current/TechAssistTest/vendor/plugins/thinking-sphinx/lib/thinking_sphinx/search.rb:508:in 'retry_on_stale_index'
from D:/Current/TechAssistTest/vendor/plugins/thinking-sphinx/lib/thinking_sphinx/search.rb:379:in 'populate'
from D:/Current/TechAssistTest/vendor/plugins/thinking-sphinx/lib/thinking_sphinx/search.rb:167:in 'method_missing'
from D:/ruby/lib/ruby/1.8/irb.rb:302:in 'output_value'
from D:/ruby/lib/ruby/1.8/irb.rb:151:in 'eval_input'
from D:/ruby/lib/ruby/1.8/irb.rb:263:in 'signal_status'
from D:/ruby/lib/ruby/1.8/irb.rb:147:in 'eval_input'
from D:/ruby/lib/ruby/1.8/irb.rb:146:in 'eval_input'
from D:/ruby/lib/ruby/1.8/irb.rb:70:in 'start'
from D:/ruby/lib/ruby/1.8/irb.rb:69:in 'catch'
from D:/ruby/lib/ruby/1.8/irb.rb:69:in 'start'
from D:/ruby/bin/irb:13
The search works fine when I use hot_request as an attribute. The
search also works fine when I use #query_string = '(#current_status_id 1 | 2 | 3)'.
I've just run into similar looking problems - there are two possible reasons why this errors that I can see. First is that according to http://sphinxsearch.com/forum/view.html?id=2103 you can use an sql column as a field or an attribute but not both (without cloning it). The other, which had me baffled for a while, is that you may need to specify the type - so if hot_request is actually an integer, you probably need to have something like
indexes hot_request :as => hr, :type => :integer
or you get that cryptic error message
Hope this helps someone ...
Related
I have a Ruby on Rails app with a model Project, which has an attribute wcag_version of type float.
I have a record with wcag_version = 2 and one with wcag_version = 2.1:
What I noticed is that Rails finds the record with wcag_version = 2 using Project.where(wcag_version: 2.0):
[4] pry(#<WcagElementsController>)> Project.where(wcag_version: 2.0)
=> [
[0] #<Project:0x007f9fa900c6e8> { :id => 1 ... }
But it doesn't find the record with wcag_version = 2.1 using Project.where(wcag_version: 2.1):
[5] pry(#<WcagElementsController>)> Project.where(wcag_version: 2.1)
=> []
It seems to be an SQL issue, not a Rails issue though. The generated SQL is:
[6] pry(#<WcagElementsController>)> Project.where(wcag_version: 2.1).to_sql
=> "SELECT `projects`.* FROM `projects` WHERE `projects`.`wcag_version` = 2.1 ORDER BY `projects`.`name` ASC"
Which doesn't return anything either if I execute it manually directly on the database:
What's going on here? I'm using MySQL by the way.
Dealing with Rails, it seems that I can work around the problem like this:
Project.all.each do |project|
if project.wcag_version >= reference_criterion.wcag_version
# Do stuff
end
end
While this is less performant, it saves me a lot of headache.
For sake of completeness, here's the original code:
Project.where("wcag_version >= #{reference_criterion.wcag_version}").each do |project|
# Do stuff
end
Why does rails not populate an auto-incrementing column in the item returned from a create? Is there a better way to do this?
In rails, when you do a = Foo.create then a.id is populated
But if you have a field that was created via
def up
execute "ALTER TABLE my_table ADD COLUMN my_auto_incrementing_column INTEGER AUTO_INCREMENT not null UNIQUE KEY;"
end
Then that field does not appear when you use create. You have to use a reload also.
a = Foo.create
a.id # not nil
a.my_auto_incrementing_column # nil
a.reload
a.my_auto_incrementing_column # is now populated
Version information:
$ ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin14.5.0]
$ bundle exec rails -v
Rails 3.2.12
$ mysql --version
mysql Ver 14.14 Distrib 5.6.26, for osx10.10 (x86_64) using EditLine wrapper
Some background:
This code is being applied to a large existing in-production rails codebase that requires that all id fields be UUIDs. The auto_increment column is not a primary key, because it was added after we had discovered that a new external integration partner could not handle using our existing long unique identifiers (UUIDs).
We are working hard to update our version of ruby but we don't want to wait for that as a solution to this problem. Also, after reading changelogs in activerecord, I still don't have proof that any future version of ruby/rails will contain a bugfix for this problem.
The code which I want to improve:
class Foo < ActiveRecord::Base
has_one :object_containing_auto_incrementing_column
def my_method
if self.object_containing_auto_incrementing_column.nil?
self.object_containing_auto_incrementing_column = ObjectContainingAutoIncrementingColumn.create(owner: self)
self.object_containing_auto_incrementing_column.reload
end
self.object_containing_auto_incrementing_column.my_auto_incrementing_column
end
end
After looking at the source code it does not appear that ActiveRecord tries to populate auto-incrementing columns. It only assigns the value that is returned by the INSERT statement to the #id attribute and nothing else.
# ActiveRecord:: Persistence::ClassMethods
def create
# ...
self.id ||= new_id if self.class.primary_key
# ...
end
If you want to populate my_auto_incrementing_column without hitting the DB twice, I think there is no way around patching ActiveRecord itself.
Have a look at how the insert method is implemented:
# Returns the last auto-generated ID from the affected table.
#
# +id_value+ will be returned unless the value is nil, in
# which case the database will attempt to calculate the last inserted
# id and return that value.
#
# If the next id was calculated in advance (as in Oracle), it should be
# passed in as +id_value+.
def insert(arel, name = nil, pk = nil, id_value = nil, sequence_name = nil, binds = [])
sql, binds = sql_for_insert(to_sql(arel, binds), pk, id_value, sequence_name, binds)
value = exec_insert(sql, name, binds)
id_value || last_inserted_id(value)
end
There might not be any trivial way to change the current API to populate your field in question.
Apparently it is still not possible to get in create() a field (not id) generated in autoincrement at database level by SEQUENCE.
I worked around the problem
in my case: PostgreSQL AUTOINCREMENT by SEQUENCE
with an after_create callback
Migration
class CreateFoo < ActiveRecord::Migration[7.0]
def change
create_table :foo do |t|
t.integer :autoinc_field
end
execute "CREATE SEQUENCE table_name_seq OWNED BY table_name.autoinc_field INCREMENT BY 1 START WITH 100000"
execute "ALTER TABLE table_name ALTER COLUMN autoinc_field SET DEFAULT nextval('table_name_seq');"
end
end
Model
class Foo < ApplicationRecord
after_create :reload
end
Result
>> Foo.create!.autoinc_field
=> 100000
Could REALLY use some help here. Struggling with displaying a dashboard with large data.
When working with # 2k records average # 2 sec.
The query in MySql Console take less than 3.5 seconds to return 150k rows. Same query in Ruby takes over 4 + minutes from time query is performed until all objects and ready.
Goal: Optimize data even further before adding cache server. Working with Ruby 1.9.2, Rails 3.0 and Mysql (Mysql2 gem)
Questions:
Does working with Hashes hurt performance?
Should I first put everything in one primary hash then manipulate the data I need afterwards?
Is there anything else I can do to help with performance?
Rows in DB:
GasStations and US Census has # 150,000 records
Person has # 100,000 records
Cars has # 200,000 records
FillUps has # 2.3 Million
Required for dashboard (query based on time periods of last 24 hours, last week, etc). All data returned in JSON format for JS.
Gas Stations, with FillUps and US Census data (zip code, Name, City, Population)
Top 20 cities with the most fill ups
Top 10 cars with Fill Ups
Cars grouped by how many times they filled up their tank
Code (sample of 6 months. Returns # 100k + records):
# for simplicity, removed the select clause I had, but removing data I don't need like updated_at, gas_station.created_at, etc. instead of returning all the columns for each table.
#primary_data = FillUp.includes([:car, :gas_staton, :gas_station => {:uscensus}]).where('fill_ups.created_at >= ?', 6.months.ago) # This would take # 4 + minutes
# then tried
#primary_data = FillUp.find_by_sql('some long sql query...') # took longer than before.
# Note for others, sql query did some pre processing for me which added attributes to the return. Query in DB Console took < 4 seconds. Because of these extra attributes, query took longer as if Ruby was checking each row for mapping attributes
# then tried
MY_MAP = Hash[ActiveRecord::Base.connection.select_all('SELECT thingone, thingtwo from table').map{|one| [one['thingone'], one['thingtwo']]}] as seen http://stackoverflow.com/questions/4456834/ruby-on-rails-storing-and-accessing-large-data-sets
# that took 23 seconds and gained mapping of additional data that was processing later, so much faster
# currently using below which takes # 10 seconds
# All though this is faster, query still only takes 3.5 seconds, but parsing it to the hashes does add overhead.
cars = {}
gasstations = {}
cities = {}
filled = {}
client = Mysql2::Client.new(:host => "localhost", :username => "root")
client.query("SELECT sum(fill_ups_grouped_by_car_id) as filled, fillups.car_id, cars.make as make, gasstations.name as name, ....", :stream => true, :as => :json).each do |row|
# this returns fill ups gouged by car ,fill_ups.car_id, car make, gas station name, gas station zip, gas station city, city population
if cities[row['city']]
cities[row['city']]['fill_ups'] = (cities[row['city']]['fill_ups'] + row['filled'])
else
cities[row['city']] = {'fill_ups' => row['filled'], 'population' => row['population']}
end
if gasstations[row['name']]
gasstations[row['name']]['fill_ups'] = (gasstations[row['name']]['fill_ups'] + row['filled'])
else
gasstations[row['name']] = {'city' => row['city'],'zip' => row['city'], 'fill_ups' => row['filled']}
end
if cars[row['make']]
cars[row['make']] = (cars[row['make']] + row['filled'])
else
cars[row['make']] = row['filled']
end
if row['filled']
filled[row['filled']] = (filled[row['filled']] + 1)
else
filled[row['filled']] = 1
end
end
Have the following models:
def Person
has_many :cars
end
def Car
belongs_to :person
belongs_to :uscensus, :foreign_key => :zipcode, :primary_key => :zipcode
has_many :fill_ups
has_many :gas_stations, :through => :fill_ups
end
def GasStation
belongs_to :uscensus, :foreign_key => :zipcode, :primary_key => :zipcode
has_many :fill_ups
has_many :cars, :through => :fill_ups
end
def FillUp
# log of every time a person fills up there gas
belongs_to :car
belongs_to :gas_station
end
def Uscensus
# Basic data about area based on Zip code
end
I don't use RoR, but returning 100k rows for a dashboard is never going to be very fast. I strongly suggest building or maintaining summary tables and run GROUP BYs in the database to summarize your dataset before presentation.
I have an object that is apparently randomly getting set to an id of 1 and the source of where this is happening is unknown in the codebase. Could be on an update attributes for a user for which school is associated.
How can I raise an error or otherwise log when this happens so I can track it down and resolve it?
Below is a first attempt but doesn't seem to take into account update_attributes
belongs_to :foo_school, :foreign_key => 'school_id'
def foo_school=(foo_school)
begin
raise "Found" if foo_school == FooSchool.find(1)
rescue Exception => e
# Track it down through the stack trace
Rails.logger.error e
ensure
write_attribute(:foo_school, foo_school)
end
end
An observer can do this for you. What you do with this observer is up to you.
$ rails g observer Oddity
Add this in config/application.rb (search for "observer" in that file, there's an example by default).
config.active_record.observers = :oddity_observer
class OddityObserver < ActiveRecord::Observer
observe :foo
def after_update(record)
raise "Boom" if record.id == 1
end
end
I'm creating an application in RoR and I'm implementing unit testing in all my models.
When I run every test on his own (by running ruby test/unit/some_test.rb) all tests are successful.
But when I run all tests together (by running rake test:units) some tables from both databases (development e test) are deleted.
I'm using raw SQL (mysql) do create tables because I need composite primary keys and physical constraints so I figured it would be the best. Maybe this be the cause?
All my tests are in this form:
require File.dirname(FILE) + '/../test_helper'
require File.dirname(FILE) + '/../../app/models/order'
class OrderTestCase < Test::Unit::TestCase
def setup
#order = Order.new(
:user_id => 1,
:total => 10.23,
:date => Date.today,
:status => 'processing',
:date_concluded => Date.today,
:user_address_user_id => 3,
:user_address_address_id => 5,
:creation_date => Date.today,
:update_date => Date.today
)
end
################ Happy Path
def test_happy_path
assert #order.valid?, #order.errors.full_messages
end
...
The errors I get when running the tests are something like this:
3) Error:
test_empty_is_primary(AddressTestCase):
ActiveRecord::StatementInvalid: Mysql::Error: Table 'shopshop_enterprise_test.addresses' doesn't exist: SHOW FIELDS FROM addresses
/test/unit/address_test.rb:9:in new'
/test/unit/address_test.rb:9:insetup'
Any guesses?
Thanks!
PS: When using postgres as the database engine, everything works fine with rake test:units! (of course, with the correct changes so the sql statements can work with postgres)