I want to return about 90k items in a JSON document but I'm getting this error when I make the call:
Timeout::Error in ApisController#api_b
time's up!
Rails.root: /root/api_b
I am simply running "rails s" with the default rails server.
What's the way to make this work and return the document?
Thanks
#bs.each do |a|
puts "dentro do bs.each"
#final << { :Email => a['headers']['to'], :At => a['date'], :subject => a['headers']['subject'], :Type => a['headers']['status'], :Message_id => a['headers']['message_id'] }
end
Being #bs the BSON object from MongoDB. The timeout is in "#final << ..."
If you are experiencing timeouts from rails and it is possible to cache the data (e.g. the data changes infrequently), I would generate the response in the background using resque or delayed_job and than have Rails dump that to the client. Or if the data cannot be cached, use a lightweight Rack handler like Sinatra and Metal to generate the responses.
Edited to reflect sample data
I was able to run the following code in a Rails 3.0.9 instance against a high performance Mongo 1.8.4 instance. I was using Mongo 1.3.1, bson_ext 1.3.1, webrick 1.3.1 and Ruby 1.9.2p180 x64. It did not time out but it took some time to load. My sample Mongo DB has 100k records and contains no indexes.
before_filter :profile_start
after_filter :profile_end
def index
db = #conn['sample-dbs']
collection = db['email-test']
#final = []
#bs = collection.find({})
#bs.each do |a|
puts "dentro do bs.each"
#final << { :Email => a['headers']['to'], :At => a['date'], :subject => a['headers']['subject'], :Type => a['headers']['status'], :Message_id => a['headers']['message_id'] }
end
render :json => #final
end
private
def profile_start
RubyProf.start
end
def profile_end
RubyProf::FlatPrinter.new(RubyProf.stop).print
end
A more efficient way to dump out the records would be
#bs = collection.find({}, {:fields => ["headers", "date"]})
#final = #bs.map{|a| {:Email => a['headers']['to'], :At => a['date'], :subject => a['headers']['subject'], :Type => a['headers']['status'], :Message_id => a['headers']['message_id'] }}
render :json => #final
My data generator
100000.times do |i|
p i
#coll.insert({:date =>Time.now(),:headers => {"to"=>"me#foo.com", "subject"=>"meeeeeeeeee", "status" => "ffffffffffffffffff", "message_id" => "1234634673"}})
end
Related
I have a field call query_data defined as text in my MySQL database.
In my model I defined this field as serialize :query_data, JSON.
The JSON format I would like to save and retrieve look like that:
{:items => [
{:id => 1},
{:id => 2},
{:id => 3}
]}
I have a collection (in that case, called items) that contain an array of objects.
I was wondering, what's the best way to add or delete an Item.
Ex: remove {:id => 2} from my items list and add `{:id => 4} to it
Ruby on Rails has some nice methods to move seamlessly between JSON and Ruby.
thing = {:items => [
{:id => 1},
{:id => 2},
{:id => 3}
]}
thing.to_json # "{\"items\":[{\"id\":1},{\"id\":2},{\"id\":3}]}"
thing.to_json is essentially what's happening in the serializer. If you want them back to Ruby, you can just do:
#items = #thing.query_data
JSON.parse(#items) # "items"=>[{"id"=>1}, {"id"=>2}, {"id"=>3}]}
Now that we can easily move between the two, lets just use Ruby syntax to deal with adding and deleting keys.
thing = {:items => [
{:id => 1},
{:id => 2},
{:id => 3}
]}
thing[:items] = thing[:items].append({:id => 4}) # adding a new item
thing[:items] = thing[:items].select { |item| item[:id] != 2 } # removing an item
First: The second argument to serialize should be the class of object you're storing in the field. You should have serialize :query_data, Hash instead.
Besides that, there aren't really any established best practices for working with serialized data. It really just depends too much on the structure of your data. You might as well ask, "what's the best way to add or delete an item from a hash?"
But since this is a hash you should make sure to keep dirty attributes in mind. If you were to do something like:
items = my_model.query_data[:items]
items.reject! {|item| item[:id] == 2}
items += {id: 4}
then the model wouldn't know that query_data changed and should be updated on save.
my_model.changed?
# => false
my_model.save
# Won't actually save changes to db.
To avoid this, you can:
A) Make sure you only ever set my_model.query_data directly
B) Explicitly call my_model.query_data_will_change! after changing that field so that it will be properly updated on save.
Base on #veridian-dynamics (thanks for your help!) Here what I did.
Model:
class MyModel < ApplicationRecord
serialize :item_data, JSON
end
Controller:
class ItemController < ApplicationController
before_action :authenticate_user!
def add_item
begin
mymodel = MyModel.find_or_create_by(id: param[:model_id])
if mymodel .item_data.blank?
item = {:items => []}
else
item = mymodel.item_data.deep_symbolize_keys
end
bookmark_exist = item[:items].any? {|i| i[:id] == params[:id]}
if !bookmark_exist
item[:items] = item[:items ].append({id: params[:id]}) # adding a new item
end
mymodel.item_data = item
mymodel.save
return render :json => item, :status=> 200
rescue Exception => e
return render :json =>{:errors=>e.message}, :status=> 400
puts "ERROR: #{e.message}"
end
end
def delete_item
begin
mymodel = MyModel.find_by(id: params[:model_id])
if mymodel.present? && mymodel.item_data.present?
item = mymodel.item_data.deep_symbolize_keys
item[:items] = (item[:items].select { |itm| itm[:id] != params[:id] }) # remove an item
mymodel.item_data = item
mymodel.save
return render :json => item, :status=> 200
end
rescue Exception => e
return render :json =>{:errors=>e.message}, :status=> 400
puts "ERROR: #{e.message}"
end
end
end
I'm not using sqlite3 gem
I'm using mysql2 gem
I'm retrieving data from MySQL database given that it meets the condition of a certain event type and severity. However, it returns only one row instead of an array of results. It really puzzles me. Shouldnt .map return an array?
result = connect.query("SELECT * FROM data WHERE event_type = 'ALARM_OPENED' AND severity = '2'")
equipments = result.map do |record|
[
record['sourcetime'].strftime('%H:%M:%S'),
record['equipment_id'],
record['description']
]
end
p equipments
I had misread your question...I think what you are looking for is in here.
UPDATE
You can use each instead, like this:
#!/usr/bin/env ruby
require 'mysql2'
connect= Mysql2::Client.new(:host => '', :username => '', :password => '', :database => '')
equipments = []
result = connect.query("SELECT * FROM data WHERE event_type = 'ALARM_OPENED' AND severity = '2'", :symbolize_keys => true).each do |row|
equipments << [
row[:sourcetime].strftime('%H:%M:%S'),
row[:equipment_id],
row[:description]
]
end
puts "#equipments {equipments}"
EDITED:
I forgot to add .each at the end of the query. So it was returning the initialized empty array instead.
You must need to change your sql statement :
result = connect.query("SELECT * FROM data WHERE event_type = 'ALARM_OPENED' AND severity = '2'", :as => :array)
I've been struggling with this for almost two weeks now and I hope someone may help me out on how to convert my sql query from mysql db into 2D array for sending into the google charts widgets in dashing please?
My actual .rb script (Job) does this >
require 'mysql2'
SCHEDULER.every '1m', :first_in => 0 do |job|
points = []
# Mysql connection
db = Mysql2::Client.new(:host => "192.168.xx.xx", :username => "xxxx", :password => "xxxxxx", :port => 3306, :database => "xxxxxx" )
# Mysql query
sql = "SELECT * FROM Table"
###Field 1 is Integer
###Field 2 is Decimal(65,30)
rs = db.query(sql, :as => :array)
subpoints = []
subpoints.push('Week')
subpoints.push('MTV')
points.push(subpoints)
rs.each do |row|
subpoints = []
subpoints.push(row['Field 1'])
subpoints.push(row['Field 2'])
points.push(subpoints)
end
###Update the List widget
send_event('mychart01', points: points)
Mysql2.close
end
I get the error:
./mysql_chart_01.rb:23:in []': no implicit conversion of String into Integer (TypeError) from ./mysql_chart_01.rb:23:inblock in '
from ./mysql_chart_01.rb:21:in each' from ./mysql_chart_01.rb:21:in'
Actually I'm not even sure to be able to make it work this way. Does anyone has a better suggestion or a Ruby script for Google charts?
Best regards.
I have created a script that I want to use to populate a new table in another database, as I'm moving this table out of one DB(table_1) and into another DB(db_2). I've already created the 'new_geo_fence' table in the new DB (db_2) and want to have the script run below to migrate data over. Script below:
class NewGeoFence < ActiveRecord::Base
attr_accessible :name, :address, :latitude, :longitude, :radius, :customer_id
belongs_to :customer, :foreign_key => 'customer_id'
end
require 'rubygems'
GeoFence.all.each do |g|
gf = NewGeoFence.new(
:id => g.id,
:name => g.name,
:address => g.address,
:latitude => g.latitude,
:longitude => g.longitude,
:radius => g.radius,
:created_at => g.created_at,
:updated_at => g.updated_at,
:customer_id => g.customer_id
)
gf.save
end
However, when I run it, I get this error:
/activerecord-4.0.13/lib/active_record/attribute_assignment.rb:47:in `rescue in _assign_attribute': unknown attribute: customer_id (ActiveRecord::UnknownAttributeError)
What have I missed to get this script running?
Thanks!
You're calling each on a class when you should be calling it on an array of objects, so
GeoFence.all.each do |g|
Rails 4 requires parameters to be whitelisted when doing mass assignment. To do so, use strong parameters
GeoFence.all.each do |g|
params = ActionController::Parameters.new({
geofence: {
id: g.id,
name: g.name,
address: g.address,
latitude: g.latitude,
longitude: g.longitude,
radius: g.radius,
created_at: g.created_at,
updated_at: g.updated_at,
customer_id: g.customer_id
}
})
gf = NewGeoFence.new(params.require(:geofence).permit!)
gf.save
end
I have in my application a few controllers that i want to use as a api. In this api i need to use versioning.
in my routes.rb i`m using this:
require 'api_constraints'
(...)
scope '/:target/:version', :module => :api, :constraints => { :version => /[0-z\.]+/ } , :defaults => { :format => 'json' } do
scope :module => :v1, :constraints => ApiConstraints.new(:version => 1, :default => true) do
match '/list' => 'sample#list'
end
end
my api_constraints.rb:
class ApiConstraints
def initialize(options)
#version = options[:version]
#default = options[:default]
end
def matches?(req)
#default || req.headers['Accept'].include?("application/waytaxi.api.v#{#version}")
end
def self.version
#version
end
end
in my SampleController.rb:
module Api
module V1
class SampleController < ApiBaseController
def list
render json: Model.find_by_id(params[:id])
end
end
end
end
the ApiBaseController:
module Api
class ApiBaseController < ApplicationController
before_filter :authenticate
skip_before_filter :verify_authenticity_token
private
def authenticate
# if params[:target] == "ios"
# render :json => {status: 404}
# return false
# end
end
end
end
the problem is:
whenever i try to call Model i get this error:
uninitialized constant Api::V1::SampleController::Model
If i use: ::Model i get this error:
uninitialized constant Model
And yes, i do have this models on my database. If i use Model.all outside the SampleController i get the objects.
P.S.: I'm using rails 3.2.8
Found my problem.
My Model was in plural and on my controller i was calling it in singular