Related
We're nearly done with our upgrade from Rails 4.2.11.1 to Rails 5.2.3. All tests pass locally and no problems are encountered when running the app locally. On staging (and our failed attempt at updating production) we encounter a very strange error where the number of connections to our database grows and grows until the app becomes unresponsive.
The problem is seen sporadically on staging (hard to reproduce), but very consistent in our production environment. I think the main difference is the amount of traffic, since our staging servers are only used for our testing whereas the production environment was receiving traffic from several thousand users. I'm attempting to do some load testing on staging to recreate the issue so that we have a chance of pinpointing the problem, but for now we're at a loss as to why this is happening.
It's like the code is not reusing already established connections to the database each time it needs one, and instead creates a new one, uses it once (presumably), and then forgets about it. Has anyone else experienced a problem like this? We would really appreciate any help or insight anyone can provide.
Our specific software versions are listed below:
Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1057-aws x86_64)
nginx/1.17.6
Passenger Phusion 6.0.4
Ruby 2.6.5
MySQL 5.5.51-38.1
Gemfile
source 'http://rubygems.org'
ruby '2.6.5'
gem 'nokogiri'
gem 'rails', '5.2.3'
gem 'rake'
gem 'mysql2'
gem 'mini_magick'
gem 'liquid', '~> 3.0.1'
gem 'httparty'
gem 'mime-types', '>=1.16', require: 'mime/types'
gem 'oauth'
gem 'oauth2', '~> 1.4'
gem 'roxml'
gem 'sax-machine'
gem 'googlecharts', '1.6.7', require: 'gchart'
gem 'rubyzip', '~> 1.2.1', require: 'zip'
gem 'ruby-openid'
gem 'RedCloth', '~> 4.2.9'
gem 'scoped_search', '~> 4.1.3'
gem 'wicked_pdf', '~> 1.1.0'
gem 'wkhtmltopdf-binary-edge', '~> 0.12.4.0'
gem 'devise'
gem 'devise-encryptable'
gem 'paperclip'
gem 'aws-sdk'
gem 'whois', '~> 2.0.4'
gem 'validates_timeliness'
gem 'will_paginate'
gem 'escape_utils'
gem 'acts_as_list'
gem 'acts_as_paranoid'
gem 'prawn', '~> 2.2.2'
gem 'prawn-table', '~> 0.2.2'
gem 'net-dns', '~> 0.6.1'
gem 'in_place_editing', '~> 1.1.2'
gem 'aasm', '~> 3.0.26'
gem 'dynamic_form', '~> 1.1.4'
gem 'delayed_job_active_record', '~> 4.1', '>= 4.1.2'
gem 'daemons'
gem 'carmen-rails', github: 'jim/carmen-rails'
gem 'dalli'
gem 'Ascii85', '1.0.1'
gem 'rinku', require: 'rails_rinku'
gem 'jquery-rails'
gem 'sass-rails'
gem 'compass-rails', '3.0.2'
gem 'compass-rgbapng', '0.2.1', require: 'rgbapng'
gem 'fancy-buttons', '~> 1.1.1'
gem 'haml-rails'
gem 'haml'
gem 'slim-rails'
gem 'coffee-rails'
gem 'uglifier', '~> 2.3.1'
gem 'holidays', '~> 1.0.5'
gem 'rack-utf8_sanitizer'
gem 'autoprefixer-rails'
gem 'loofah'
gem 'yajl-ruby', require: 'yajl'
gem 'gretel'
gem 'crack'
gem 'lhm'
gem 'twilio-ruby'
gem 'open_uri_redirections'
gem 'godaddy-api', '~> 0.1.0'
gem 'html_truncator', '~> 0.4.1'
gem 'acme-client'
gem 'net-ssh'
gem 'mustache', "~> 1.0"
gem 'net-sftp', '~> 2.1', '>= 2.1.2'
gem 'activeresource'
gem 'geokit', '~> 1.13.1'
gem 'fomo', '~> 0.1.0'
gem 'nameable', '~> 1.1'
gem 'xmlrpc'
gem 'activemodel-serializers-xml'
gem 'airbrake', '~> 9.4', '>= 9.4.5'
1) check max_connections variable and see if it's consistent with number of connections in your productions environement
mysql> SHOW VARIABLES LIKE 'max_connections';
2) reduce the wait for timeout connections at database.yml
3) check active record connection pool settings
https://devcenter.heroku.com/articles/concurrency-and-database-connections#connection-pool
So I'm running rspec tests with jruby-1.7.16.1 after completing a bundle install, and they are all failing to the same error:
Failure/Error: Unable to find arjdbc/jdbc/RubyJdbcConnection.java to read failed line
ActiveRecord::JDBCError:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
# arjdbc/jdbc/RubyJdbcConnection.java:453:in `init_connection'
Occasionally I would get an error like this when I'd forget to start mysql server, but I ensured it was running.
The GemFile is as such
source 'http://rubygems.org'
gem 'rails', '3.2.8'
gem 'rubyzip', '0.9.9'
gem 'roo','1.12.1'
gem 'jdbc-mysql', '5.1.33', platform: :jruby
gem 'activerecord-jdbcmysql-adapter', '1.3.12', platform: :jruby
gem 'jquery-rails', '2.1.2'
gem 'haml', '3.1.7'
gem 'puma', '2.9.2'
gem 'devise', '2.1.2'
gem 'devise-async', '0.5.0'
gem 'cancan', '1.6.8'
gem 'simple_form', '2.0.4'
gem 'cocoon', '1.1.1'
gem 'inherited_resources', '1.3.1'
gem 'will_paginate', '3.0.3'
gem 'bootstrap-will_paginate', '0.0.9'
gem 'mechanize', '2.5.1'
gem 'delayed_job', '3.0.4'
gem 'paperclip', '3.4.0'
gem 'spreadsheet', '0.6.4.1'
gem 'geocoder', '1.1.6'
gem 'whenever', '0.8.2'
gem 'american_date', '1.0.0'
gem 'money','5.1.1'
gem 'rets', '0.11.0'
gem 'haversine','0.3.0'
gem 'celluloid','0.15.2'
gem 'sinatra', '1.4.6'
source "https://9789474f:e71b0546#gems.contribsys.com/" do
gem 'sidekiq-pro','1.9.2'
end
gem 'redis','3.2.1'
gem 'remote_lock','1.1.0'
gem 'sidekiq-limit_fetch','2.3.0'
group :assets do
gem 'stylus', '0.7.1'
gem 'coffee-rails', '3.2.2'
gem 'uglifier', '1.3.0'
end
group :development do
gem 'net-ssh', '2.9.4'
gem 'capistrano', '3.4.1'
gem 'capistrano-rails', '1.3.0'
gem 'capistrano-rvm', '0.1.2'
end
group :development, :test do
gem 'dotenv-rails', '2.2.1'
gem 'rspec-rails', '~> 3.5'
gem 'fantaskspec'
end
group :test do
gem 'factory_girl_rails', '~> 4.0'
gem 'fantaskspec'
gem 'vcr'
gem 'fakeweb'
end
gem 'rest-client', '1.8.0'
gem 'poltergeist', '1.8.1'
gem 'nokogiri', '1.5.11'
gem 'zip-codes', '0.2.1'
gem 'elasticsearch-model', '0.1.9'
gem 'elasticsearch-rails', '0.1.9'
gem "fog", '1.38.0'
gem 'newrelic_rpm', '3.18.1.330'
gem 'sidekiq-scheduler', '2.1.7'
gem 'aws-sdk', '2.10.28'
gem 'StreetAddress', '1.0.6'
If there is more information required I'd be happy to provide it, let me know if anyone has run into this.
The odd thing is, these spec tests used to work fine prior to a merge I did with our repository. The other dev doesn't seem to be running into this same issue, so I can only assume the Gemfiles are all correct and that my environment is the issue.
Make sure the database is up and running.
Does the user that's trying to connect to the database has permission to connect? Try logging in with that user in the client software.
Host 'xxx.xx.xxx.xxx' is not allowed to connect to this MySQL server
Try creating a new user with admin privileges and try connecting.
Advice: Try using a software that keeps your development environments in sync, and test it well. We use Vagrant to reproduce development environments.
When I try to run the following code in Rails using Mysql2 as database manager:
rake db:migrate
I obtain the following error:
rake aborted!
"Mysql2::Error: All parts of a PRIMARY KEY must be NOT NULL:"
Why do I get this error, if the primary key in a table by default is NOT "null"?
Migration code, however :
class CreateUsers < ActiveRecord::Migration
def change
create_table :users do |t|
t.string "first_name"
t.timestamps
end
end
end
I had a same problem before, and I solved according to here
https://github.com/rails/rails/pull/13247#issuecomment-32425844
With Rails 2.3.5, MySQL version 5.7.9 and mysql gem you need to have
this bit as an initializer in
config/initializers/abstract_mysql_adapter.rb:
class ActiveRecord::ConnectionAdapters::MysqlAdapter
NATIVE_DATABASE_TYPES[:primary_key] = "int(11) auto_increment PRIMARY KEY"
end
For mysql2, it should be config/initializers/abstract_mysql2_adapter.rb:
class ActiveRecord::ConnectionAdapters::Mysql2Adapter
NATIVE_DATABASE_TYPES[:primary_key] = "int(11) auto_increment PRIMARY KEY"
end
From the mysql 5.7 breaking change page:
Columns in a PRIMARY KEY must be NOT NULL, but if declared explicitly
as NULL produced no error. Now an error occurs. For example, a
statement such as CREATE TABLE t (i INT NULL PRIMARY KEY) is rejected.
The same occurs for similar ALTER TABLE statements. (Bug #13995622,
Bug #66987, Bug #15967545, Bug #16545198)
Changes in MySQL 5.7.3 (2013-12-03, Milestone 13)
This issue was fixed 3 months ago. I don't know what Rails version it's going to be part of
monkey patch is here
EDIT:
It's been more than a year that this fix was merged into master. So, it must be part of the last version of Rails.
EDIT:
Indeed, the commit that fixes this can be found at: https://github.com/yahonda/rails/commit/b6655885ef13cf8d1705dc9b5232846f0207febd , and shows that the fix is included in v4.2.0.beta1, v4.1.6.rc1, v4.1.5, v4.1.4, v4.1.3, v4.1.2, v4.1.2.rc3, v4.1.2.rc2, v4.1.2.rc1, v4.1.1, v4.1.0, v4.1.0.rc2, v4.1.0.rc1, v4.1.0.beta2, v4.1.0.beta1. If upgrading to at least v4.1.0 is an option for you, then it should fix the problem.
I had this issue running a rails 4.0.x app using mysql 5.7.x. I was able to fix it by upgrading to rails 4.2.x and upgrading my gems.
(I'm sure these gemfiles need work, but hopefully they are still helpful)
Old Gemfile
source 'https://rubygems.org'
ruby '2.0.0'
# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '4.0.13'
gem 'mysql2'
# Use unicorn as the app server
# gem 'unicorn'
gem 'sidekiq'
gem 'sinatra'
gem 'whenever'
gem 'thin'
gem 'faye'
gem 'faye-websocket'
#pdf generation
gem 'wicked_pdf'
gem 'wkhtmltopdf-binary'
# Assets
gem 'backbone-on-rails', '~> 0.9.9'
gem 'coffee-rails', '~> 4.0.1'
gem 'coffee-script-source', '=1.8.0'
gem 'ejs' # uses templates for backbone
gem 'jquery-rails'
gem 'jquery-ui-rails'
gem 'neat'
gem 'sass-rails', '~> 4.0.3'
gem 'therubyracer'
gem 'turbolinks'
gem 'uglifier', '>= 1.3.0'
gem 'active_model_serializers', '0.8.3'
gem 'acts_as_list'
gem 'bcrypt-ruby', '~> 3.1.2'
gem 'bootstrap-sass', '~> 3.2.0.0'
gem 'bower-rails'
gem 'carrierwave'
gem 'enumerize'
gem 'embedly' # Url to Attachment Processing
gem 'fancybox2-rails', '~> 0.2.4'
gem 'flamegraph'
gem 'fog' # s3 storage
gem 'globalize' # I18n for tagging, etc.
gem 'globalize-accessors'
gem 'i18n-tasks', '~> 0.8.3'
gem 'hirb'
gem 'kaminari'
gem 'nokogiri'
gem 'newrelic_rpm'
gem 'select2-rails' # Token Fields
gem 'twilio-ruby'
gem 'rails_autolink' # Convert urls to links
gem 'redis', '3.2.1'
gem 'rest-client'
gem 'rmagick'
gem 'rubyzip', :require => 'zip' # Zip files together
gem 'unf'
gem 'mailgun-ruby', '~>1.0.3', require: 'mailgun'
# Deployments
gem 'mina'
group :doc do
gem 'sdoc'
end
group :test do
gem 'rspec-rails', "= 2.14.2"
gem 'shoulda-matchers', :require => false
gem 'simplecov', '~> 0.9.2'
gem 'database_cleaner'
gem 'codeclimate-test-reporter', require: nil
end
group :development, :test do
gem 'quiet_assets'
gem 'factory_girl_rails', "~> 4.0"
gem 'guard', '>=2.1.0'
gem 'guard-rspec', '= 4.2.9'
gem 'spring', '1.3.6'
gem 'spring-commands-rspec'
gem 'foreman'
end
New Gemfile
source 'https://rubygems.org'
ruby '2.3.0'
# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '4.2.5'
gem 'mysql2'
gem 'stackprof'
# Use unicorn as the app server
# gem 'unicorn'
gem 'sidekiq'
gem 'sinatra'
gem 'whenever'
gem 'thin'
gem 'faye'
gem 'faye-websocket'
#pdf generation
gem 'wicked_pdf'
gem 'wkhtmltopdf-binary'
# Assets
gem 'backbone-on-rails', '~> 0.9.9'
gem 'coffee-rails', '~> 4.0.1'
gem 'coffee-script-source', '=1.8.0'
gem 'ejs' # uses templates for backbone
gem 'jquery-rails'
gem 'jquery-ui-rails'
gem 'neat'
gem 'sass-rails', '~> 4.0.3'
gem 'therubyracer'
gem 'turbolinks'
gem 'uglifier', '>= 1.3.0'
gem 'active_model_serializers', '0.8.3'
gem 'acts_as_list'
gem 'bcrypt', '~> 3.1.2'
gem 'bootstrap-sass', '~> 3.2.0.0'
gem 'bower-rails'
gem 'carrierwave'
gem 'enumerize'
gem 'embedly' # Url to Attachment Processing
gem 'fancybox2-rails', '~> 0.2.4'
gem 'flamegraph'
gem 'fog' # s3 storage
gem 'globalize' # I18n for tagging, etc.
gem 'globalize-accessors'
gem 'i18n-tasks', '~> 0.8.3'
gem 'hirb'
gem 'kaminari'
gem 'nokogiri'
gem 'newrelic_rpm'
gem 'select2-rails' # Token Fields
gem 'twilio-ruby'
gem 'rails_autolink' # Convert urls to links
gem 'redis', '3.2.1'
gem 'rest-client'
gem 'rmagick'
gem 'rubyzip', :require => 'zip' # Zip files together
gem 'unf'
gem 'mailgun-ruby', '~>1.0.3', require: 'mailgun'
gem 'rack-cors', :require => 'rack/cors'
# Deployments
gem 'mina'
group :doc do
gem 'sdoc'
end
group :test do
gem 'rspec-rails', "= 2.14.2"
gem 'shoulda-matchers', :require => false
gem 'simplecov', '~> 0.9.2'
gem 'database_cleaner'
gem 'codeclimate-test-reporter', require: nil
end
group :development, :test do
gem 'quiet_assets'
gem 'factory_girl_rails', "~> 4.0"
gem 'guard', '>=2.1.0'
gem 'guard-rspec', '= 4.2.9'
gem 'spring', '1.6.1'
gem 'spring-commands-rspec'
gem 'foreman'
end
I have a Rails app called "enrollment_app" that initializes and populates all the tables in the database with a MySQL seed file. I built the app, added some migrations and pushed my app to Heroku. However, since Heroku uses Postgres, I need a way to make my MySQL database compatible with Heroku so I am using the ClearDB addon.
When I try to open the app though, I get the message:
Application Error
An error occurred in the application and your page could not be served. Please try again in a few moments.
If you are the application owner, check your logs for details.
So, I checked the logs and see this error:
PG::UndefinedTable: ERROR: relation "enrollments" does not exist
I've been following along with this tutorial but apparently I don't know how to make ClearDB look like my local MySQL DB since I'm getting that error above. How can I do the equivalent of rake db:seed the MySQL seed file and rake db:migrate to the production ClearDB database?
Updated - Gemfile:
source 'https://rubygems.org'
gem 'rails', '4.2.1'
gem 'mysql2'
gem 'sass-rails', '~> 5.0'
gem 'uglifier', '>= 1.3.0'
gem 'coffee-rails', '~> 4.1.0'
gem 'jquery-rails'
gem 'turbolinks'
gem 'jbuilder', '~> 2.0'
gem 'sdoc', '~> 0.4.0', group: :doc
gem 'bootstrap-sass', '~> 3.3.5'
gem 'bootswatch-rails'
gem 'ransack'
gem 'jquery-turbolinks'
gem 'kaminari'
gem 'bootstrap-kaminari-views'
gem 'jquery-ui-rails'
gem 'espinita'
gem 'mysqltopostgres', git: "https://github.com/maxlapshin/mysql2postgres.git"
group :development, :test do
gem 'byebug'
gem 'web-console', '~> 2.0'
gem 'spring'
gem 'rspec-rails'
gem 'launchy'
gem 'pry'
gem 'pry-nav'
gem 'shoulda-matchers'
gem 'factory_girl_rails'
gem 'capybara'
gem 'newrelic_rpm'
gem 'poltergeist'
gem 'database_cleaner'
end
group :production do
gem 'rails_12factor'
end
This isn't an issue with ClearDB per se, it looks like you haven't fully divorced from the PostgreSQL defaults. I'd check:
is pg defined in your Gemfile? it shouldn't be. Instead ensure mysql2 is present.
Have you installed the Heroku ClearDB add-on for this app, and made it the default database your app will use? Have you uninstalled the Heroku Postgres database? See here for complete instructions.
Once your Heroku app can connect to the ClearDB database properly, you should be able to set up the database itself with no issues:
heroku run rake db:create
heroku run rake db:migrate
heroku run rake db:seed
I have a old rails project that is using rails 3.0.9 and I am trying to run it using rails s. It appears that when I do this is gives me the following error:
You will see that it says that the current adapter that I have installed is not supported by foriegner. I have updated my gemfile to look like the following:
gem 'rake', '0.9.2.2'
gem 'mysql2'
gem 'rails', '3.0.9'
gem "acl9"
gem 'foreigner'
gem 'devise'
gem 'will_paginate', '3.0.pre2'
gem 'jquery-rails'
gem 'lazy_high_charts'
gem 'client_side_validations'
gem 'carrierwave'
gem 'tiny_mce'
#gem 'devise_security_extension'
#gem 'rails_email_validator'
#gem 'easy_captcha'
group :development do
gem 'rspec-rails', '2.6.1'
gem 'jspec'
gem 'faker', '0.3.1'
gem 'rspec'
gem 'annotate'
end
group :test do
gem 'rspec-rails', '2.6.1'
gem 'spork', '0.9.0.rc8'
gem 'webrat'
gem 'autotest-growl'
gem 'autotest-rails'
gem 'autotest', '4.4.6'
gem 'autotest-rails-pure', '4.1.2'
gem 'factory_girl_rails'
gem 'ZenTest'
gem 'test-unit'
end
What I don't seem to understand is why it would say in the command prompt that the gem is missing and I need to add it to my gemfile when it is there. What is the best solution to tackling this.
It isnt working cause you are trying to install the latest version of mysql2 which doesnt work with rails 3.0.X. You can get around this by specifying the version this way
gem 'mysql2', '< 0.3' or gem "mysql2", "~> 0.2.7"
But to get around the foreigner dependency which is probably the main issue here:
You can upgrade your rails version to 3.1.x or higher and upgrade the mysql2 in the gem file. I see that you have a recent version of ruby.
After you update the gems in the gemfile, make sure you run bundle install in the command line...before trying to start the server. Also, ensure that you setup your database in the database.yml file in the config folder.
edit your config/database.yml file from:
adapter: mysql
to
adapter: mysql2