I need to ensure that records on different database servers are present on the backup server.
I'm trying to execute this as a Sinatra project using RSpec:
describe 'BACKUP' do
puts 'BACKUP config'
puts DB_CONFIG[:BACKUP].inspect
p "Key.count #{Key.count}"
DB_CONFIG[:databases].each do |server,config|
p "****************************************************************"
p "Server #{server} "
p " Config #{config.inspect}"
p "Service.count #{Service.new(config).count}"
end
end
Where:
class Key < BillingTables
end
class BillingTables < ActiveRecord::Base
self.abstract_class = true
establish_connection DB_CONFIG[:BACKUP]
end
is connected to one database.
On the other hand I'm connecting simultaneously to another database with this class:
class Service < CoreTables
end
class CoreTables < ActiveRecord::Base
self.abstract_class = true
def initialize(params = {})
establish_connection params
end
end
Output of this script is as follows:
=== Comparison Spec ===
backup config
{"adapter"=>"mysql2", "encoding"=>"utf8", "reconnect"=>true, "database"=>"backup", "pool"=>1, "username"=>"backup", "password"=>"password", "host"=>"xxx.xxx"}
"Key.count 3902"
"****************************************************************"
"Server a1 "
" Config {\"adapter\"=>\"mysql2\", \"encoding\"=>\"utf8\", \"reconnect\"=>true, \"database\"=>\"s1\", \"pool\"=>1, \"username\"=>\"s1\", \"password\"=>\"password\", \"host\"=>\"yyy.yyy\"}"
/Users/password123/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:570:in `retrieve_connection': No connection pool for Service (ActiveRecord::ConnectionNotEstablished)
In the end, I dropped my idea of multipe active_records, and now I'm just performing mysql connections
begin
client = Mysql2::Client.new DB_CONFIG[:databases][server]
query = 'SELECT * FROM test'
result = client.query(client.escape(query),symbolize_keys: true)
rescue Exception => e
p "query error #{e.message}"
next
end
Related
I am using gem roo to import CSV data. It works smoothly, until the point where there is an association, and am hoping that roo can translate the string into the corresponding integer value in the association. In my case, I have a Staff model which belongs to State.
class State < ApplicationRecord
has_many :staffs
end
class Staff < ApplicationRecord
belongs_to :state
end
This means that I have state_id column in the staffs table. In my CSV, however, the end user has the names of the states, which correspond to the ones in the states tables. When I try to import the CSV, I get the error:
ActiveRecord::AssociationTypeMismatch in StaffsImportsController#create
State(#134576500) expected, got "Texas" which is an instance of String(#20512180)
The highlighted source is:
staff.attributes = row.to_hash
Is it possible for gem roo to translate 'Texas' in the csv file to, say, id 2, instead of the end user doing a lot of translation work before uploading the data?
Here is staffs_imports.rb
class StaffsImport
include ActiveModel::Model
require 'roo'
attr_accessor :file
def initialize(attributes={})
attributes.each { |name, value| send("#{name}=", value) }
end
def persisted?
false
end
def open_spreadsheet
case File.extname(file.original_filename)
when ".csv" then Csv.new(file.path, nil, :ignore)
when ".xls" then Roo::Excel.new(file.path, nil, :ignore)
when ".xlsx" then Roo::Excelx.new(file.path)
else raise "Unknown file type: #{file.original_filename}"
end
end
def load_imported_staffs
spreadsheet = open_spreadsheet
header = spreadsheet.row(1)
(2..spreadsheet.last_row).map do |i|
row = Hash[[header, spreadsheet.row(i)].transpose]
staff = Staff.find_by_national_id(row["national_id"]) || Staff.new
staff.attributes = row.to_hash
staff
end
end
def imported_staffs
#imported_staffs ||= load_imported_staffs
end
def save
if imported_staffs.map(&:valid?).all?
imported_staffs.each(&:save!)
true
else
imported_staffs.each_with_index do |staff, index|
staff.errors.full_messages.each do |msg|
errors.add :base, "Row #{index + 6}: #{msg}"
end
end
false
end
end
end
And finally the staff_imports_controller.rb:
class StaffsImportsController < ApplicationController
def new
#staffs_import = StaffsImport.new
end
def create
#staffs_import = StaffsImport.new(params[:staffs_import])
if #staffs_import.save
flash[:success] = "You have successfully uploaded your staff!"
redirect_to staffs_path
else
render :new
end
end
end
Any help/clues will be highly appreciated.
I managed to get a solution to this, thanks to a wonderfully detailed question and great answer provided here Importing CSV data into Rails app, using something other then the association "id"
I am fairly new to Django. I want to know how to fetch rows from mysql and get it in views.py and send it to html where it will be displayed.
My views.py:
def fetchDate1(request):
query = request.session.get('query')
date1 = request.session.get('date1');
db = pymysql.connect(host="localhost", # your host
user="root", # username
passwd="=", # password
db="Golden") # name of the database
# Create a Cursor object to execute queries.
cur = db.cursor()
# Select data from table using SQL query.
stmt = "SELECT * FROM golden_response WHERE query LIKE '%s' AND DATE(updated_at) = '%s' " % (
query.replace("'", r"\'"), date1)
cur.execute(stmt)
if cur.rowcount is None:
return None
else:
rows = cur.fetchall()
row_headers = [x[0] for x in cur.description] # this will extract row headers
json_data = []
for result in rows:
json_data.append(dict(zip(row_headers, result)))
return json.dumps(json_data)
I don't know where I am going wrong. Have also saved required configuration in settings.py.
However when i try to run my program :
ProgrammingError: (1146, "Table 'Golden.django_session' doesn't exist")
Please help!!
I dare to hypothesize that you have not made the first migrations.
python manage.py makemigrations
python manage.py migrate
Moreover, you should check the database connection parameters in settings.py like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'db_name',
'USER': 'db_user',
'PASSWORD': 'db_user_password',
'HOST': 'db_server',
'PORT': '3306',
}
}
I'm a newbie to this, and can't figure out how to solve this error that I get when attempting to query from the database. The error reads:
rm: cannot unlink 'C:/Users/myName/Documents/GitHub/active_record_lite/lib../cats.db': Permission denied
Error: unable to open database "'C:\Users\myName\Documents\GitHub\active_record_lite'": unable to open database file
By going to "Properties" and viewing the "Security" tab, everything is checked under "Allow" for security, so I assume that the project and all subfiles have full permission (is this correct?).
I have a "cats.db" and "cats.sql" file in the project folder.
This is where I attempt to query from the database:
require_relative 'db_connection'
require 'active_support/inflector'
class SQLObject
def self.columns
cols = DBConnection.execute2(<<-SQL, #table_name)
SELECT
*
FROM
?
SQL
And here is a db_connection.rb file:
require 'sqlite3'
# https://tomafro.net/2010/01/tip-relative-paths-with-file-expand-path
ROOT_FOLDER = File.join(File.dirname(__FILE__), '..')
CATS_SQL_FILE = File.join(ROOT_FOLDER, 'cats.sql')
CATS_DB_FILE = File.join(ROOT_FOLDER, 'cats.db')
class DBConnection
def self.open(db_file_name)
#db = SQLite3::Database.new(db_file_name)
#db.results_as_hash = true
#db.type_translation = true
#db
end
def self.reset
commands = [
"rm '#{CATS_DB_FILE}'",
"cat '#{CATS_SQL_FILE}' | sqlite3 '#{CATS_DB_FILE}'"
]
commands.each { |command| `#{command}` }
DBConnection.open(CATS_DB_FILE)
end
def self.instance
reset if #db.nil?
#db
end
def self.execute(*args)
puts args[0]
instance.execute(*args)
end
def self.execute2(*args)
puts args[0]
instance.execute2(*args)
end
def self.last_insert_row_id
instance.last_insert_row_id
end
private
def initialize(db_file_name)
end
end
Any advice/suggestions welcomed! Thanks!
I am trying to implement connection pooling(many clients are served by few db connections) with mysql-proxy.
I took a look at
ro-pooling.lua and it seems that some actions must be done in connect_server() hook.
If I want to create a new connection:
Assign target backend index to proxy.connection.backend_ndx.
Return nothing
If I want to use already existing idle connection:
Assign target backend index to proxy.connection.backend_ndx.
Return proxy.PROXY_IGNORE_RESULT
Now, what bothers me that returning proxy.PROXY_IGNORE_RESULT from connect_server() hook seems to have no impact on connection reuse - every time a client connects, a new connection is created and eventually I run into following error: "User 'username' has exceeded the 'max_user_connections' resource (current value: 4)"
So, the question is: What is the meaning of return proxy.PROXY_IGNORE_RESULT in connect_server() hook?
Also, any reference about how mysql-proxy creates and reuses connections would be very helpful - I did not manage to find any...
Any help would be greatly appreciated :)
EDIT:
this is source of script I'm current using:
local default_backend = 1
local min_idle_connections = 1
local max_idle_connections = 4
local connections_limit = 1
local user_name = "user"
if not proxy.global.count then -- never mind this, not using it...
proxy.global.count = 0
end
function connect_server()
local backend = proxy.global.backends[1]
local pool = backend.pool
local cur_idle = pool.users[""].cur_idle_connections
print ("CONNECT SERVER:")
print("current backend:" ..proxy.connection.backend_ndx)
if cur_idle >= min_idle_connections then
print("using pooled connection")
proxy.connection.backend_ndx=0
print("current backend:" ..proxy.connection.backend_ndx)
return proxy.PROXY_SEND_RESULT
end
proxy.global.count = proxy.global.count + 1
print("Creating new connection")
proxy.connection.backend_ndx = default_backend
print("current backend:" ..proxy.connection.backend_ndx)
end
function read_handshake()
print("READ_HANDSHAKE")
print("current backend:" ..proxy.connection.backend_ndx)
end
function read_auth()
local username = proxy.connection.client.username
print("READ_AUTH: " ..username)
print("current backend:" ..proxy.connection.backend_ndx)
end
function disconnect_client()
print ("DISCONNECT CLIENT. ")
print("current backend:" ..proxy.connection.backend_ndx)
end
function read_auth_result(auth)
print("READ_AUTH_RESULT")
if auth.packet:byte() == proxy.MYSQLD_PACKET_OK then
--auth was fine, disconnect from the server--
proxy.connection.backend_ndx = 0
print("disconnected backend after auth")
print("current backend:" ..proxy.connection.backend_ndx)
end
end
function read_query(packet)
print("READ_QUERY:")
print("current backend:" ..proxy.connection.backend_ndx)
if packet:byte() == proxy.COM_QUIT then
print("received signal QUIT")
proxy.response = {
type = proxy.MYSQLD_PACKET_OK,
}
return proxy.PROXY_SEND_RESULT
end
if proxy.connection.backend_ndx == 0 then
print("assigning backend to process query...")
proxy.connection.backend_ndx = default_backend
print("current backend:" ..proxy.connection.backend_ndx)
end
local username = proxy.connection.client.username
local cur_idle = proxy.global.backends[default_backend].pool.users[username].cur_idle_connections
print ("current idle user" ..username.." connections: " ..cur_idle)
if string.byte(packet) == proxy.COM_QUERY then
print("Query: " .. string.sub(packet, 2))
end
proxy.queries:append(1, packet)
return proxy.PROXY_SEND_QUERY
end
function read_query_result( inj )
print("READ_QUERY_RESULT:")
print("current backend:" ..proxy.connection.backend_ndx)
local res = assert(inj.resultset)
local flags = res.flags
if inj.id ~= 1 then
return proxy.PROXY_IGNORE_RESULT
end
is_in_transaction = flags.in_trans
if not is_in_transaction then
-- release the backend
print("releasing backend")
proxy.connection.backend_ndx = 0
end
print("current backend:" ..proxy.connection.backend_ndx)
end
I have created a rail application
i am connecting to mysql database
i already have 2 tables users and history
how can i fetch data from these tables
i can save data to database
with
def register
if(params[:register])
#registermsg=Users.where(:username=>params[:register][:username])
if #registermsg.length>0
flash[:success] = "Username exists:#{params[:username]}:"
else
#reg = Users.create(user_params)
#reg.save
flash[:success] = "success:#{params[:register][:username]}:"
end
end
end
def user_params
params.require(:register).permit(:username, :password, :password2)
end
but when i am fetching data from users
def login
if session[:user]
redirect_to '/dash'
end
if(params[:login])
#login1=Users.where("username=? AND password=?",params[:login][:username],params[:login][:password])
if #login1.length==1
session[:user] = #login1
hist = History.create(userid: **#login1.id**, ip: request.remote_ip )
// here #login1.id is giving me error
redirect_to '/dash'
else
flash[:success] = "Wrong"
end
end
end
#login1=Users.where("username=? AND password=?",params[:login][:username],params[:login][:password])
#login1.to_yaml is printing full data
how can i get id,username etc from #login1
presently #login1.id is getting error
I researched in many websites including stack overflow till now i didnt got any solution please help
Here #login1 is an ActiveRelation object. You will have to call first to get the actual user object
if #login1.count==1
session[:user] = #login1.first
hist = History.create(userid: #login1.first.id, ip: request.remote_ip )
redirect_to '/dash'