I'm trying to figure out why my reserved instance is not being used. I was deploying with region: us-east-1 and it was not utilizing my available reserved instance that is in us-east-1c. Amazon billing tells me to pay for premium support, even though I have presented the evidence that the aws-sdk-core rejects the us-east-1c subregion:
Seahorse::Client::NetworkingError: unable to connect to `elasticbeanstalk.us-east-1c.amazonaws.com`; SocketError: getaddrinfo: nodename nor servname provided, or not known
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/net_http/connection_pool.rb:292:in `start_session'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/net_http/connection_pool.rb:104:in `session_for'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/net_http/handler.rb:108:in `session'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/net_http/handler.rb:60:in `transmit'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/net_http/handler.rb:34:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/plugins/content_length.rb:12:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/xml/error_handler.rb:8:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/request_signer.rb:85:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:87:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:118:in `retry_request'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:101:in `retry_if_possible'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:89:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:118:in `retry_request'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:101:in `retry_if_possible'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:89:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:118:in `retry_request'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:101:in `retry_if_possible'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/retry_errors.rb:89:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/query/handler.rb:27:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/plugins/endpoint.rb:34:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/param_validator.rb:21:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/param_converter.rb:21:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/aws-sdk-core/plugins/response_paging.rb:26:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/plugins/response_target.rb:21:in `call'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/request.rb:70:in `send_request'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/aws-sdk-core-2.1.26/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer/aws_driver/beanstalk.rb:20:in `application_exists?'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer/throttling_handling.rb:13:in `block in method_missing'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer/utils.rb:13:in `backoff'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer/throttling_handling.rb:12:in `method_missing'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer/application.rb:86:in `create_application_if_not_exists'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer/application.rb:15:in `create_version'
/Users/kross/.rvm/gems/ruby-2.2.2#acme/gems/eb_deployer-0.6.3/lib/eb_deployer.rb:218:in `deploy'
Question: is anyone else having a problem making sure AWS is utilizing the reserved instance? Is there a configuration trick I'm missing?
This forum post has details on zones for instances: Placing beanstalk in specific availability zone. Availability Zones are note the same as Regions, therein lies my misunderstanding.
For example, this keeps it to one specific zone:
[
{ "Namespace":"aws:autoscaling:asg",
"OptionName":"Custom Availability Zones",
"Value":"us-east-1c" },
{ "Namespace":"aws:autoscaling:asg",
"OptionName":"Availability Zones",
"Value":"Any 1" }
]
For elastic-beanstalk gem, I alter the options in my eb.yml:
region: us-east-1
options:
aws:autoscaling:asg:
'Custom Availability Zones': us-east-1c
'Availability Zones': Any 1
For multiple AZs, this could be something like:
region: us-east-1
options:
aws:autoscaling:asg:
'Custom Availability Zones': 'us-east-1c, us-east-1d'
'Availability Zones': Any 2
Related
The error that I am getting is
Mysql2::Error::ConnectionError in BlogController#home
Access denied for user 'abhips'#'localhost' (using password: NO)
The stack is Rails-6 + MySQL-8 deployed with Nginx + puma-4.
I am getting the connection error when I deploy the code in production mode. It is working perfectly on my local machine.
Weird thing is that I am not getting that error when I stop puma and run the code with RAILS_ENV=production rails s on the server.
And I tested the connection in rails console in production mode, which is also working fine. So the config/database.yml has no problem.
No errors were showing up in the logs so I had to enable config.consider_all_requests_local = true in production.rb file. After that, I got the error details.
So my question is
Why am I getting this error? How can I fix this?
Why is the error log file not capturing the error?
I will list the configuration files below
config/puma/production.rb
# Change to match your CPU core count
workers 1
# Min and Max threads per worker
threads 1, 6
app_dir = File.expand_path("../../..", __FILE__)
shared_dir = "#{app_dir}/shared"
# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
environment rails_env
# Set up socket location
bind "unix://#{shared_dir}/sockets/puma.sock"
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{shared_dir}/pids/puma.pid"
state_path "#{shared_dir}/pids/puma.state"
activate_control_app
/etc/nginx/sites-enabled/myapp
upstream my_app {
server unix:///var/www/myapp.me/shared/sockets/puma.sock fail_timeout=100;
}
server {
if ($host = xxx.xx.xxx.x){
return 301 https://myapp.me$request_uri;
}
}
server {
listen 443 ssl http2; # managed by Certbot
listen [::]:443 ssl http2; # add http2 to enable http2
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
if ($host = www.myapp.me) {
return 301 https://myapp.me$request_uri;
} # managed by Certbot
server_name myapp.me www.myapp.me;
root /var/www/myapp.me;
access_log /var/www/myapp.me/shared/log/nginx.stdout.log;
error_log /var/www/myapp.me/shared/log/nginx.stderr.log;
# try_files $uri #my_app;
location / {
try_files $uri #my_app;
}
location #my_app {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://ny_app;
}
# error_page 500 502 503 504 /500.html;
# location = /500.html {
# root /var/www/myapp.me/public;
# }
ssl_certificate /etc/letsencrypt/live/myapp.me/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/myapp.me/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
listen 80;
listen [::]:80;
if ($host = www.myapp.me) {
return 301 https://myapp.me$request_uri;
} # managed by Certbot
if ($host = myapp.me) {
return 301 https://myapp.me$request_uri;
} # managed by Certbot
server_name myapp.me www.myapp.me;
return 404; # managed by Certbot
}
/etc/systemd/system/puma.service
[Unit]
Description=Puma HTTP Server
After=network.target
# Uncomment for socket activation (see below)
# Requires=puma.socket
[Service]
# Foreground process (do not use --daemon in ExecStart or config.rb)
Type=simple
# Preferably configure a non-privileged user
User=abhips
Group=abhips
# The path to the your application code root directory.
# Also replace the "<YOUR_APP_PATH>" place holders below with this path.
# Example /home/username/myapp
WorkingDirectory=/var/www/myapp.me
# Helpful for debugging socket activation, etc.
Environment=RAILS_ENV=production
# SystemD will not run puma even if it is in your path. You must specify
# an absolute URL to puma. For example /usr/local/bin/puma
# Alternatively, create a binstub with `bundle binstubs puma --path ./sbin` in the WorkingDirectory
ExecStart=/bin/bash -lc "/home/abhips/.rbenv/shims/puma -C /var/www/myapp.me/config/puma/production.rb"
ExecStop=/bin/bash -lc "/home/abhips/.rbenv/shims/puma -S /var/www/myapp.me/shared/pids/puma.state"
# Path to PID file so that systemd knows which is the master process
PIDFile=/var/www/myapp.me/shared/pids/puma.pid
Restart=always
# decides when to restart after a crash
RestartSec=10
[Install]
WantedBy=multi-user.target
Full trace
Rails.root: /var/www/myapp.me
Application Trace | Framework Trace | Full Trace
mysql2 (0.5.3) lib/mysql2/client.rb:90:in `connect'
mysql2 (0.5.3) lib/mysql2/client.rb:90:in `initialize'
activerecord (6.0.3.1) lib/active_record/connection_adapters/mysql2_adapter.rb:24:in `new'
activerecord (6.0.3.1) lib/active_record/connection_adapters/mysql2_adapter.rb:24:in `mysql2_connection'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:887:in `new_connection'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:931:in `checkout_new_connection'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:910:in `try_to_checkout_new_connection'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:871:in `acquire_connection'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:593:in `checkout'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:437:in `connection'
activerecord (6.0.3.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:1119:in `retrieve_connection'
activerecord (6.0.3.1) lib/active_record/connection_handling.rb:221:in `retrieve_connection'
activerecord (6.0.3.1) lib/active_record/connection_handling.rb:189:in `connection'
activerecord (6.0.3.1) lib/active_record/model_schema.rb:488:in `load_schema!'
activerecord (6.0.3.1) lib/active_record/attributes.rb:247:in `load_schema!'
activerecord (6.0.3.1) lib/active_record/attribute_decorators.rb:50:in `load_schema!'
activerecord (6.0.3.1) lib/active_record/model_schema.rb:478:in `block in load_schema'
activerecord (6.0.3.1) lib/active_record/model_schema.rb:475:in `synchronize'
activerecord (6.0.3.1) lib/active_record/model_schema.rb:475:in `load_schema'
activerecord (6.0.3.1) lib/active_record/model_schema.rb:362:in `attribute_types'
activerecord (6.0.3.1) lib/active_record/model_schema.rb:386:in `type_for_attribute'
activerecord (6.0.3.1) lib/active_record/table_metadata.rb:33:in `type'
activerecord (6.0.3.1) lib/active_record/relation/predicate_builder.rb:51:in `build'
activerecord (6.0.3.1) lib/active_record/relation/predicate_builder.rb:109:in `block in expand_from_hash'
activerecord (6.0.3.1) lib/active_record/relation/predicate_builder.rb:68:in `each'
activerecord (6.0.3.1) lib/active_record/relation/predicate_builder.rb:68:in `flat_map'
activerecord (6.0.3.1) lib/active_record/relation/predicate_builder.rb:68:in `expand_from_hash'
activerecord (6.0.3.1) lib/active_record/relation/predicate_builder.rb:21:in `build_from_hash'
activerecord (6.0.3.1) lib/active_record/relation/where_clause_factory.rb:19:in `build'
activerecord (6.0.3.1) lib/active_record/relation/query_methods.rb:656:in `where!'
activerecord (6.0.3.1) lib/active_record/relation/query_methods.rb:649:in `where'
activerecord (6.0.3.1) lib/active_record/querying.rb:21:in `where' paranoia (2.4.2) lib/paranoia.rb:247:in `paranoia_scope'
activerecord (6.0.3.1) lib/active_record/relation/delegation.rb:66:in `block in paranoia_scope'
activerecord (6.0.3.1) lib/active_record/relation.rb:407:in `block in scoping'
activerecord (6.0.3.1) lib/active_record/relation.rb:784:in `_scoping'
activerecord (6.0.3.1) lib/active_record/relation.rb:407:in `scoping'
activerecord (6.0.3.1) lib/active_record/relation/delegation.rb:66:in `paranoia_scope'
paranoia (2.4.2) lib/paranoia.rb:252:in `block in acts_as_paranoid'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:120:in `instance_exec'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:120:in `block (2 levels) in build_default_scope'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:118:in `each'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:118:in `inject'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:118:in `block in build_default_scope'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:142:in `evaluate_default_scope'
activerecord (6.0.3.1) lib/active_record/scoping/default.rb:117:in `build_default_scope'
activerecord (6.0.3.1) lib/active_record/scoping/named.rb:58:in `default_scoped'
activerecord (6.0.3.1) lib/active_record/scoping/named.rb:44:in `all'
activerecord (6.0.3.1) lib/active_record/querying.rb:21:in `order' app/controllers/blog_controller.rb:9:in `home'
actionpack (6.0.3.1) lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
actionpack (6.0.3.1) lib/abstract_controller/base.rb:195:in `process_action'
actionpack (6.0.3.1) lib/action_controller/metal/rendering.rb:30:in `process_action'
actionpack (6.0.3.1) lib/abstract_controller/callbacks.rb:42:in `block in process_action'
activesupport (6.0.3.1) lib/active_support/callbacks.rb:135:in `run_callbacks'
actionpack (6.0.3.1) lib/abstract_controller/callbacks.rb:41:in `process_action'
actionpack (6.0.3.1) lib/action_controller/metal/rescue.rb:22:in `process_action'
actionpack (6.0.3.1) lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
activesupport (6.0.3.1) lib/active_support/notifications.rb:180:in `block in instrument'
activesupport (6.0.3.1) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
activesupport (6.0.3.1) lib/active_support/notifications.rb:180:in `instrument'
actionpack (6.0.3.1) lib/action_controller/metal/instrumentation.rb:32:in `process_action'
actionpack (6.0.3.1) lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
activerecord (6.0.3.1) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
actionpack (6.0.3.1) lib/abstract_controller/base.rb:136:in `process'
actionview (6.0.3.1) lib/action_view/rendering.rb:39:in `process'
actionpack (6.0.3.1) lib/action_controller/metal.rb:190:in `dispatch'
actionpack (6.0.3.1) lib/action_controller/metal.rb:254:in `dispatch'
actionpack (6.0.3.1) lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
actionpack (6.0.3.1) lib/action_dispatch/routing/route_set.rb:33:in `serve'
actionpack (6.0.3.1) lib/action_dispatch/journey/router.rb:49:in `block in serve'
actionpack (6.0.3.1) lib/action_dispatch/journey/router.rb:32:in `each'
actionpack (6.0.3.1) lib/action_dispatch/journey/router.rb:32:in `serve'
actionpack (6.0.3.1) lib/action_dispatch/routing/route_set.rb:834:in `call'
warden (1.2.8) lib/warden/manager.rb:36:in `block in call'
warden (1.2.8) lib/warden/manager.rb:34:in `catch'
warden (1.2.8) lib/warden/manager.rb:34:in `call'
rack (2.2.3) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.3) lib/rack/etag.rb:27:in `call'
rack (2.2.3) lib/rack/conditional_get.rb:27:in `call'
rack (2.2.3) lib/rack/head.rb:12:in `call'
actionpack (6.0.3.1) lib/action_dispatch/http/content_security_policy.rb:18:in `call'
rack (2.2.3) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.3) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/cookies.rb:648:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
activesupport (6.0.3.1) lib/active_support/callbacks.rb:101:in `run_callbacks'
actionpack (6.0.3.1) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/actionable_exceptions.rb:17:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
railties (6.0.3.1) lib/rails/rack/logger.rb:37:in `call_app'
railties (6.0.3.1) lib/rails/rack/logger.rb:26:in `block in call'
activesupport (6.0.3.1) lib/active_support/tagged_logging.rb:80:in `block in tagged'
activesupport (6.0.3.1) lib/active_support/tagged_logging.rb:28:in `tagged'
activesupport (6.0.3.1) lib/active_support/tagged_logging.rb:80:in `tagged'
railties (6.0.3.1) lib/rails/rack/logger.rb:26:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/request_id.rb:27:in `call'
rack (2.2.3) lib/rack/method_override.rb:24:in `call'
rack (2.2.3) lib/rack/runtime.rb:22:in `call'
activesupport (6.0.3.1) lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/executor.rb:14:in `call'
rack (2.2.3) lib/rack/sendfile.rb:110:in `call'
actionpack (6.0.3.1) lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
railties (6.0.3.1) lib/rails/engine.rb:527:in `call'
puma (4.3.5) lib/puma/configuration.rb:228:in `call'
puma (4.3.5) lib/puma/server.rb:713:in `handle_request'
puma (4.3.5) lib/puma/server.rb:472:in `process_client'
puma (4.3.5) lib/puma/server.rb:328:in `block in run'
puma (4.3.5) lib/puma/thread_pool.rb:134:in `block in spawn_thread'
Please let me know if anything else to be posted here
I´m getting this severe error in my Rails 4 app and because of that my connection to my database is lost.
F, [2015-12-23T18:06:22.875935 #13919] FATAL -- :
ActiveRecord::ConnectionTimeoutError (could not obtain a database connection within 5.000 seconds (waited 5.009 seconds)):
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:190:in `block in wait_poll'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `loop'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `wait_poll'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:136:in `block in poll'
/home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:146:in `synchronize'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:134:in `poll'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:423:in `acquire_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:356:in `block in checkout'
/home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:355:in `checkout'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:265:in `block in connection'
/home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:264:in `connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:546:in `retrieve_connection'
activerecord (4.0.0) lib/active_record/connection_handling.rb:79:in `retrieve_connection'
activerecord (4.0.0) lib/active_record/connection_handling.rb:53:in `connection'
activerecord (4.0.0) lib/active_record/query_cache.rb:51:in `restore_query_cache_settings'
activerecord (4.0.0) lib/active_record/query_cache.rb:43:in `rescue in call'
activerecord (4.0.0) lib/active_record/query_cache.rb:32:in `call'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.0.0) lib/active_support/callbacks.rb:373:in `_run__285615481658568074__call__callbacks'
activesupport (4.0.0) lib/active_support/callbacks.rb:80:in `run_callbacks'
actionpack (4.0.0) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
railties (4.0.0) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.0.0) lib/rails/rack/logger.rb:21:in `block in call'
activesupport (4.0.0) lib/active_support/tagged_logging.rb:67:in `block in tagged'
activesupport (4.0.0) lib/active_support/tagged_logging.rb:25:in `tagged'
activesupport (4.0.0) lib/active_support/tagged_logging.rb:67:in `tagged'
railties (4.0.0) lib/rails/rack/logger.rb:21:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/request_id.rb:21:in `call'
rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
rack (1.5.2) lib/rack/runtime.rb:17:in `call'
activesupport (4.0.0) lib/active_support/cache/strategy/local_cache.rb:83:in `call'
railties (4.0.0) lib/rails/engine.rb:511:in `call'
railties (4.0.0) lib/rails/application.rb:97:in `call'
puma (2.10.1) lib/puma/configuration.rb:74:in `call'
puma (2.10.1) lib/puma/server.rb:490:in `handle_request'
puma (2.10.1) lib/puma/server.rb:361:in `process_client'
puma (2.10.1) lib/puma/server.rb:254:in `block in run'
puma (2.10.1) lib/puma/thread_pool.rb:96:in `call'
puma (2.10.1) lib/puma/thread_pool.rb:96:in `block in spawn_thread'
I´m trying to debug it but it´s quite complex. I don´t know where the issue is coming from.
I read this post before ActiveRecord::ConnectionTimeoutError: could not obtain a database connection within 5.000 seconds (waited 5.000 seconds) and there is an answer from #railsana that fits one possible scenario of mine.
I have some kind of batch processes that makes lots of queries called from a model (outside controller) by an scheduled task. I tried to follow their suggestion and added this code to this functionality, however the issue is still there.
ActiveRecord::Base.connection_pool.with_connection do
# your code
end
Just in case it´s a relevant information, my rails app is running in a Puma server and my database is MySQL.
Any suggestions, what can it be, or how to debug it? The bug is really fatal because my application cannot handle any query.
UPDATE:
Answering to the questions:
1. In production I have configured a pool with 5 connections in config/database.yml.
2. My Puma server startup log:
Puma starting in single mode...
* Version 2.10.1 (ruby 2.1.2-p95), codename: Robots on Comets
* Min threads: 0, max threads: 16
* Environment: production
* Daemonizing..
.
Only one worker is started.
UPDATE 2:
Checking puma.stderr.log I found this kind of error logs:
{ 70238458077400 rufus-scheduler intercepted an error:
70238458077400 job:
70238458077400 Rufus::Scheduler::EveryJob "90s" {}
70238458077400 error:
70238458077400 70238458077400
70238458077400 Net::ReadTimeout
70238458077400 Net::ReadTimeout
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http.rb:1378:in `request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rest-client-1.6.7/lib/restclient/net_http_ext.rb:51:in `request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi/adapter/net_http.rb:65:in `perform'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi/adapter/net_http.rb:42:in `block in request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi/adapter/net_http.rb:78:in `call'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi/adapter/net_http.rb:78:in `block in do_request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/2.1.0/net/http.rb:853:in `start'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi/adapter/net_http.rb:76:in `do_request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi/adapter/net_http.rb:33:in `request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi.rb:161:in `request'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/httpi-2.4.1/lib/httpi.rb:133:in `post'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/savon-2.11.1/lib/savon/operation.rb:94:in `block in call_with_logging'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/savon-2.11.1/lib/savon/request_logger.rb:12:in `call'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/savon-2.11.1/lib/savon/request_logger.rb:12:in `log'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/savon-2.11.1/lib/savon/operation.rb:94:in `call_with_logging'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/savon-2.11.1/lib/savon/operation.rb:54:in `call'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/savon-2.11.1/lib/savon/client.rb:36:in `call'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/lib/mmk_service.rb:560:in `block (2 levels) in getAvailabilities'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/lib/mmk_service.rb:544:in `each'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/lib/mmk_service.rb:544:in `block in getAvailabilities'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.0.0/lib/active_record/relation/delegation.rb:13:in `each'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.0.0/lib/active_record/relation/delegation.rb:13:in `each'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/lib/mmk_service.rb:530:in `getAvailabilities'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/lib/mmk_service.rb:65:in `updateAvailabilities'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/app/models/secretary.rb:182:in `block in getBoatsAvailability'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.0.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:294:in `with_connection'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/app/models/secretary.rb:181:in `getBoatsAvailability'
70238458077400 /home/ubuntu/env/production/www/yanpyapi/config/initializers/task_scheduler.rb:29:in `block in <top (required)>'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:224:in `call'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:224:in `do_trigger'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:269:in `block (3 levels) in start_work_thread'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:272:in `call'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:272:in `block (2 levels) in start_work_thread'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:258:in `loop'
70238458077400 /home/ubuntu/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/rufus-scheduler-3.0.3/lib/rufus/scheduler/jobs.rb:258:in `block in start_work_thread'
} 70238458077400 .
This is the portion of code that is sending throwing this exception (I think not catched exception):
# This method is in a model invoked by an scheduled task rufus-scheduler that is executed every 90 seconds.
def self.getBoatsAvailability
if Rails.env.production?
require 'service'
# This line was added to try to fix this problem (I read this approach in another post link above)
ActiveRecord::Base.connection_pool.with_connection do
Service.getAvailabilities(false)
end
end
end
def self.getAvailabilities(isMonthly)
#logger.debug "Importing availabilities..."
# Savon is a service to communicate with SOAP web services
client = Savon.client(wsdl: "url",
log_level: :debug,
log: true,
pretty_print_xml: true)
mmkCompanies = Mmk::Company.select("id").where(loading: true).order("id")
createdAt = Time.now
#logger.debug "getAvailabilities Before each company."
for mmkCompany in mmkCompanies do
lastModifiedReq = Mmk::Availability.find_by_sql(["SELECT max(a.created_at) as max_date
FROM mmk_availabilities a, mmk_resources r
WHERE a.resource_id = r.id
AND r.company_id = ?", mmkCompany.id]).first.max_date
(0..1).each do |i|
year = Time.now.year + i
message = {'in0' => credentials,
'in1' => credentials,
'in2' => credentials,
'in3' => mmkCompany.id,
'in4' => year,
'in5' => true,
'in6' => lastModifiedReq}
#logger.debug "getAvailabilities Before client call."
response = client.call(:get_availability_info, message: message)
#logger.debug "getAvailabilities After client call."
availabilitiesXML = response.to_hash[:get_availability_info_response][:out]
availabilitiesParsed = Nokogiri::XML(availabilitiesXML)
availabilities = availabilitiesParsed.xpath("//reservation")
#logger.debug "getAvailabilities Before transaction."
Mmk::Availability.transaction do
#logger.debug "getAvailabilities Transaction opened."
for availability in availabilities do
id = availability["id"]
# More parse parameters unrelevant code
mmkAvailability = Mmk::Availability.find_or_initialize_by(id: id)
mmkAvailability.update(resource_id: resourceId, status: status, blocks_availability: blocksAvailability,
date_from: dateFrom, date_to: dateTo, base_from: baseFrom, base_to: baseTo,
option_expiry_date: optionExpiryDate, last_modified: lastModified, created_at: createdAt)
mmkAvailability.save
end
end
#logger.debug "getAvailabilities After transaction (closed)?."
end
end
#logger.debug "Availability imported successfully."
end
This line in exception:
`70238458077400 /home/ubuntu/env/production/www/yanpyapi/lib/mmk_service.rb:560:in `block (2 levels) in getAvailabilities'`
correspondes to:
response = client.call(:get_availability_info, message: message)
My analysis (althouth not everything is matching for me), is:
- The method getAvailabilities is sending a webservice SOAP request to a third party system.
- Sometimes for some reason, this connection is lost or the third party service doesn´t response and Net::ReadTimeout exception is thrown.
- This exception is not caught. And a database connection (and this is the part I don´t understand) keeps opened.
- When this issue happens 5 times, the number of pool connections is 0 and I got the main issue.
To debug this problem I would start by adding an exception handler (rescue clause) to the self.getAvailabilities method.
def self.getAvailabilities(isMonthly)
... # existing code goes here
rescue Net::ReadTimeout => err
# I would use pry to put a breakpoint here and see what is keeping the DB connection open.
binding.pry
end
Use the pry gem to insert a breakpoint in the handler. Once the Savon call times out it should hit that breakpoint. Then you can use the REPL to determine who is holding the connection, and probably close it. See here: How to find current connection pool size on heroku.
If the exception is not caught I would change the rescue line to catch StandardError instead of Net::ReadTimeout and try again.
every time I run rhc snapshot save wp1 got this
Pulling down a snapshot to wp1.tar.gz...
Saving snapshot for secondary mysql-5.5 gear
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift- origin-node/model/v2_cart_model.rb:1313:in `block in do_control_with_directory': CLIENT_ERROR: Failed to execute: 'control pre-snapshot' for /var/lib/openshift/5322d44ce0b8cd3bbd0001cb/mysql (OpenShift::Runtime::Utils::ShellExecutionException)
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:1114:in `process_cartridges'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:1276:in `do_control_with_directory'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:1136:in `do_control'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:68:in `block in snapshot'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:80:in `block in each_cartridge'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:1120:in `block in process_cartridges'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:1118:in `each'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:1118:in `process_cartridges'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/v2_cart_model.rb:78:in `each_cartridge'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:67:in `snapshot'
from /usr/bin/gear:569:in `block (2 levels) in '
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:155:in `run'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:385:in `run_active_command'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:62:in `run!'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/delegates.rb:11:in `run!'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander
/import.rb:10:in `block in '
/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/utils/shell_exec.rb:133:in `block (2 levels) in oo_spawn': Shell command '/usr/bin/ssh -q -o 'BatchMode=yes' -o 'StrictHostKeyChecking=no' -i
$OPENSHIFT_APP_SSH_KEY 5322d44ce0b8cd3bbd0001cb#5322d44ce0b8cd3bbd0001cb-wpis.rhcloud.com 'snapshot' > mysql-5.5.tar.gz' returned an error. rc=1 (OpenShift::Runtime::Utils::ShellExecutionException)
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/utils/shell_exec.rb:96:in `pipe'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/utils/shell_exec.rb:96:in `block in oo_spawn'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/utils/shell_exec.rb:95:in `pipe'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/utils/shell_exec.rb:95:in `oo_spawn'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-container-selinux-0.7.1/lib/openshift/runtime/containerization/selinux_container.rb:339:in `run_in_container_context'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container.rb:764:in `run_in_container_context'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:104:in `block in handle_scalable_snapshot'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:100:in `each'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:100:in `handle_scalable_snapshot'
from /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.21.6/lib/openshift-origin-node/model/application_container_ext/snapshots.rb:58:in `snapshot'
from /usr/bin/gear:569:in `block (2 levels) in '
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:180:in `call'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/command.rb:155:in `run'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:385:in `run_active_command'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/runner.rb:62:in `run!'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/delegates.rb:11:in `run!'
from /opt/rh/ruby193/root/usr/share/gems/gems/commander-4.0.3/lib/commander/import.rb:10:in `block in '
Error in trying to save snapshot. You can try to save manually by running:
ssh ce2cf000931#wp1-wpis.rhcloud.com 'snapshot' > wp1.tar.gz
I try to restart my application,but not work.
ruby verison 2.0.0p247 (2013-06-27) [x86_64-linux]
rhc 1.21.3
am I doing something wrong?
``````
Please file a bug report with the above details here: https://bugzilla.redhat.com/enter_bug.cgi?product=OpenShift%20Online
I got a situation here.
I want to keep site up when MySQL server is down (my site should run when DB server 'mysql' is down)
I am using MySql2 gem with activerecord 4.0.0.
As far i know i need to write out a middleware that will check whether the DB server is up, if DB server is up then we are fine, but if DB server is down then we just need to bypass DB transaction request and render page/call(need to catch MYSQL error Lost Connection exception).
I know there are multiple gems like:
https://github.com/bdurand/seamless_database_pool 'seamless_database_pool'
https://github.com/schoefmax/multi_db 'multi_db'
https://github.com/taskrabbit/makara 'makara'
All these gems supports master/slave DB concept/architecture, where as i don't want to use master-slave db architecture.
So i had simply created a file in config/initializer/database_timeout.rb and it looks like below.
module Mysql2
class Client
def initialize(opts = {})
opts = Mysql2::Util.key_hash_as_symbols( opts )
#read_timeout = nil
#query_options = ##default_query_options.dup
#query_options.merge! opts
initialize_ext
[:reconnect, :connect_timeout, :local_infile, :read_timeout, :write_timeout, :default_file, :default_group, :secure_auth].each do |key|
next unless opts.key?(key)
case key
when :reconnect, :local_infile, :secure_auth
send(:"#{key}=", !!opts[key])
when :connect_timeout, :read_timeout, :write_timeout
send(:"#{key}=", opts[key].to_i)
else
send(:"#{key}=", opts[key])
end
end
# force the encoding to utf8
self.charset_name = opts[:encoding] || 'utf8'
ssl_options = opts.values_at(:sslkey, :sslcert, :sslca, :sslcapath, :sslcipher)
ssl_set(*ssl_options) if ssl_options.any?
if [:user,:pass,:hostname,:dbname,:db,:sock].any?{|k| #query_options.has_key?(k) }
warn "============= WARNING FROM mysql2 ============="
warn "The options :user, :pass, :hostname, :dbname, :db, and :sock will be deprecated at some point in the future."
warn "Instead, please use :username, :password, :host, :port, :database, :socket, :flags for the options."
warn "============= END WARNING FROM mysql2 ========="
end
user = opts[:username] || opts[:user]
pass = opts[:password] || opts[:pass]
host = opts[:host] || opts[:hostname]
port = opts[:port]
database = opts[:database] || opts[:dbname] || opts[:db]
socket = opts[:socket] || opts[:sock]
flags = opts[:flags] ? opts[:flags] | #query_options[:connect_flags] : #query_options[:connect_flags]
# Correct the data types before passing these values down to the C level
user = user.to_s unless user.nil?
pass = pass.to_s unless pass.nil?
host = host.to_s unless host.nil?
port = port.to_i unless port.nil?
database = database.to_s unless database.nil?
socket = socket.to_s unless socket.nil?
begin
connect user, pass, host, port, database, socket, flags
rescue StandardError => err
p 'sssssss'
end
end
end
end
Now I am getting following error in 'err' variable, which means i can successfully get the error exception. Notice this message come up while hitting to my website.
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
After above block server is throwing me following error
[2014-01-10T18:11:16.080922 #16264] INFO -- : Started GET "/" for 127.0.0.1 at 2014-01-10 18:11:16 +0500
"sssssss"
"sssssss"
F, [2014-01-10T18:13:53.570439 #16264] FATAL -- :
Mysql2::Error (closed MySQL connection):
activerecord (4.0.0) lib/active_record/connection_adapters/mysql2_adapter.rb:77:in `escape'
activerecord (4.0.0) lib/active_record/connection_adapters/mysql2_adapter.rb:77:in `quote_string'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/quoting.rb:15:in `quote'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:246:in `quote'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:772:in `block in configure_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:768:in `each'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:768:in `map'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:768:in `configure_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/mysql2_adapter.rb:265:in `configure_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/mysql2_adapter.rb:38:in `initialize'
activerecord (4.0.0) lib/active_record/connection_adapters/mysql2_adapter.rb:20:in `new'
activerecord (4.0.0) lib/active_record/connection_adapters/mysql2_adapter.rb:20:in `mysql2_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:440:in `new_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:450:in `checkout_new_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:421:in `acquire_connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:356:in `block in checkout'
/home/sannankhalid/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:355:in `checkout'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:265:in `block in connection'
/home/sannankhalid/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:264:in `connection'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:546:in `retrieve_connection'
activerecord (4.0.0) lib/active_record/connection_handling.rb:79:in `retrieve_connection'
activerecord (4.0.0) lib/active_record/connection_handling.rb:53:in `connection'
activerecord (4.0.0) lib/active_record/query_cache.rb:51:in `restore_query_cache_settings'
activerecord (4.0.0) lib/active_record/query_cache.rb:43:in `rescue in call'
activerecord (4.0.0) lib/active_record/query_cache.rb:32:in `call'
activerecord (4.0.0) lib/active_record/connection_adapters/abstract/connection_pool.rb:626:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.0.0) lib/active_support/callbacks.rb:373:in `_run__4494376046790926309__call__callbacks'
activesupport (4.0.0) lib/active_support/callbacks.rb:80:in `run_callbacks'
actionpack (4.0.0) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/reloader.rb:64:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
airbrake (3.1.14) lib/airbrake/rails/middleware.rb:13:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
railties (4.0.0) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.0.0) lib/rails/rack/logger.rb:21:in `block in call'
activesupport (4.0.0) lib/active_support/tagged_logging.rb:67:in `block in tagged'
activesupport (4.0.0) lib/active_support/tagged_logging.rb:25:in `tagged'
activesupport (4.0.0) lib/active_support/tagged_logging.rb:67:in `tagged'
railties (4.0.0) lib/rails/rack/logger.rb:21:in `call'
/home/sannankhalid/.rvm/gems/ruby-1.9.3-p448#astrology-site/bundler/gems/astro_logger-d53871b98caa/lib/astro_logger/middlewares/finish_logging.rb:17:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/request_id.rb:21:in `call'
rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
rack (1.5.2) lib/rack/runtime.rb:17:in `call'
activesupport (4.0.0) lib/active_support/cache/strategy/local_cache.rb:83:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
actionpack (4.0.0) lib/action_dispatch/middleware/static.rb:64:in `call'
airbrake (3.1.14) lib/airbrake/user_informer.rb:16:in `_call'
airbrake (3.1.14) lib/airbrake/user_informer.rb:12:in `call'
railties (4.0.0) lib/rails/engine.rb:511:in `call'
railties (4.0.0) lib/rails/application.rb:97:in `call'
rack (1.5.2) lib/rack/content_length.rb:14:in `call'
thin (1.6.1) lib/thin/connection.rb:82:in `block in pre_process'
thin (1.6.1) lib/thin/connection.rb:80:in `catch'
thin (1.6.1) lib/thin/connection.rb:80:in `pre_process'
thin (1.6.1) lib/thin/connection.rb:55:in `process'
thin (1.6.1) lib/thin/connection.rb:41:in `receive_data'
eventmachine (1.0.3) lib/eventmachine.rb:187:in `run_machine'
eventmachine (1.0.3) lib/eventmachine.rb:187:in `run'
thin (1.6.1) lib/thin/backends/base.rb:73:in `start'
thin (1.6.1) lib/thin/server.rb:162:in `start'
rack (1.5.2) lib/rack/handler/thin.rb:16:in `run'
rack (1.5.2) lib/rack/server.rb:264:in `start'
railties (4.0.0) lib/rails/commands/server.rb:84:in `start'
railties (4.0.0) lib/rails/commands.rb:78:in `block in <top (required)>'
railties (4.0.0) lib/rails/commands.rb:73:in `tap'
railties (4.0.0) lib/rails/commands.rb:73:in `<top (required)>'
bin/rails:4:in `require'
bin/rails:4:in `<top (required)>'
ruby-debug-ide (0.4.22) lib/ruby-debug-ide.rb:86:in `debug_load'
ruby-debug-ide (0.4.22) lib/ruby-debug-ide.rb:86:in `debug_program'
ruby-debug-ide (0.4.22) bin/rdebug-ide:110:in `<top (required)>'
-e:1:in `load'
-e:1:in `<main>'
Is there any work around? or how should i approach to overcome my needs!
Help really appreciated. Thanks.
I want to develop a "health check" page for showing if database/redis/queue are working correctly. However it seems rails will fail the request with 500 error if mysql is down.
Mysql2::Error (Can't connect to MySQL server on '127.0.0.1' (61)):
mysql2 (0.3.11) lib/mysql2/client.rb:44:in `connect'
mysql2 (0.3.11) lib/mysql2/client.rb:44:in `initialize'
activerecord (3.2.8) lib/active_record/connection_adapters/mysql2_adapter.rb:16:in `new'
activerecord (3.2.8) lib/active_record/connection_adapters/mysql2_adapter.rb:16:in `mysql2_connection'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:309:in `new_connection'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:319:in `checkout_new_connection'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:241:in `block (2 levels) in checkout'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:236:in `loop'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:236:in `block in checkout'
/Users/moshebergman/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:233:in `checkout'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:96:in `block in connection'
/Users/moshebergman/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:95:in `connection'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:404:in `retrieve_connection'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_specification.rb:170:in `retrieve_connection'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_specification.rb:144:in `connection'
activerecord (3.2.8) lib/active_record/query_cache.rb:67:in `rescue in call'
activerecord (3.2.8) lib/active_record/query_cache.rb:61:in `call'
activerecord (3.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:473:in `call'
actionpack (3.2.8) lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
activesupport (3.2.8) lib/active_support/callbacks.rb:405:in `_run__4605244939803304128__call__1430909676671874052__callbacks'
activesupport (3.2.8) lib/active_support/callbacks.rb:405:in `__run_callback'
activesupport (3.2.8) lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
activesupport (3.2.8) lib/active_support/callbacks.rb:81:in `run_callbacks'
actionpack (3.2.8) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
Is there a way to make rails not fail completely in this case?
I found an answer by monkey patching the QueryCache for certain url's:
module ActiveRecord
# = Active Record Query Cache
class QueryCache
def call(env)
unless env['REQUEST_PATH'].starts_with?('/health_check')
begin
old = ActiveRecord::Base.connection.query_cache_enabled
ActiveRecord::Base.connection.enable_query_cache!
status, headers, body = #app.call(env)
[status, headers, BodyProxy.new(old, body, ActiveRecord::Base.connection_id)]
rescue Exception => e
ActiveRecord::Base.connection.clear_query_cache
unless old
ActiveRecord::Base.connection.disable_query_cache!
end
raise e
end
else
status, headers, body = #app.call(env)
end
end
end
end
If anyone has any better answer, let me know.