How to make galera node run independently - mysql

I have 2 nodes of MySQL Galera cluster. I want to make it run independently when it's disconnected from each other. But it seems that it's becoming an initialized state even after I enabled below option:
SET GLOBAL wsrep_provider_options='pc.ignore_sb=TRUE';
It's showing below when the node is disconnected from the network:
| wsrep_local_state_comment | Initialized |
| wsrep_cert_index_size | 14 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.086022 |
| wsrep_open_transactions | 0 |
| wsrep_open_connections | 0 |
| wsrep_incoming_addresses | 10.201.127.76:3306 |
| wsrep_cluster_weight | 0 |
| wsrep_desync_count | 0 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list |
Any idea how to make it run independently even if the network is down?
Thanks!

Related

Etcd: how to check that each node can see each other

I have 3 etcd nodes on VMs (not k8s).
There was such problem that nodes are alive but can't see each other, error "connection timeout" during health check. But every single node has "alive" status and zabbix with "etcd by http" template doesn't generate any alerts.
Is there any way to check nodes visibility and to monitor it using zabbix?
Depending upon the version you run, here's an example to do this with 3.5.2
Command
ETCDCTL_API=3 ./bin/etcdctl endpoint status --cluster -w table --endpoints="member1.etcd:2384,member2.etcd:2384,member3.etcd:2384"
Output:
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+--------------------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| http://member1.etcd:2384 | 17ef476d9d7fec5f | 3.5.2 | 1.5 MB | false | false | 7 | 20033 | 20033 | |
| http://member2.etcd:2384 | 31e0ca30ec3c9d94 | 3.5.2 | 1.5 MB | false | false | 7 | 20033 | 20033 | |
| http://member3.etcd:2384 | 721948abbb0522bd | 3.5.2 | 1.5 MB | false | false | 7 | 20033 | 20033 | |
+--------------------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

Mysql table does not accept any update request

I am using Server version: 5.6.27-log MySQL Community Server (GPL) and I have a problem with a table.
I tried to update some fields with a GUI software, but when I came back to the command line, the lines I tried to update where not updated.
I tried to see if the table was locked using SHOW OPEN TABLES as stated in various other questions. But my table does not appear to be locked:
+--------------------+-------------------------------------------------+--------+-------------+
| Database | Table | In_use | Name_locked |
+--------------------+-------------------------------------------------+--------+-------------+
| arcdev | SCHEDULED_COMMAND | 0 | 0 |
And as soon as I try to make an update like:
UPDATE SCHEDULED_COMMAND SET field = 1;
The server just keeps loading and nothing happen. I tried on other tables and everything worked just fine.
I also tried some DELETE requests and even a DROP TABLE and nothing work so far...
What am I missing?
Thank you for your precious help!
EDIT: Here is the result of the SHOW PROCESSLIST command while a request is hanging:
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+
| 282588 | rdsadmin | localhost:17966 | mysql | Sleep | 2 | | NULL |
| 534575 | arc | XXXXXX:49376 | arcdev | Sleep | 17 | | NULL |
| 534579 | arc | XXXXXX:49443 | arcdev | Query | 0 | init | SHOW PROCESSLIST |
| 534659 | arc | XXXXXX:49836 | arcdev | Query | 14 | updating | DELETE FROM SCHEDULED_COMMAND |
+--------+----------+---------------------------------------+-----------+---------+------+----------+-------------------------------+

MySQL Connections causing server went away, nothing in processlist

I have a large amount of connections but when I issue a show full processlist I am not showing anything close to the connections I see. Are these connections orphans of some sort? I tried the flush hosts command and the connections persist, even with a reboot of the server and also restarting the mysql server.
I believe these connections are causing issues with making new connections to the database. User's are getting a "server went away" error. How do I clear these?
See commands below:
mysql> show status like '%onn%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| Aborted_connects | 5 |
| Connections | 11743 |
| Max_used_connections | 24 |
| Ssl_client_connects | 0 |
| Ssl_connect_renegotiates | 0 |
| Ssl_finished_connects | 0 |
| Threads_connected | 6 |
+--------------------------+-------+
7 rows in set (0.00 sec)
mysql> show full processlist;
+-------+---------+----------------------+--------------------+---------+-------+-------+-----------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-------+---------+----------------------+--------------------+---------+-------+-------+-----------------------+
| 4494 | rode | localhost:43411 | NULL | Sleep | 11159 | | NULL |
| 4506 | rode | localhost:43423 | information_schema | Sleep | 11159 | | NULL |
| 4554 | rode | localhost:43511 | performance_schema | Sleep | 11112 | | NULL |
| 11500 | ass | serv:1243 | Home-Tech | Sleep | 0 | | NULL |
| 11743 | root | localhost | NULL | Query | 0 | NULL | show full processlist |
| 11744 | ass | out:6070 | Home-Tech | Sleep | 4 | | NULL |
| 11745 | ass | out:6074 | HTGlobal | Sleep | 8 | | NULL
The MySQL server has gone away (error 2006) has two main causes
Server timed out and closed the connection. To fix, check that “wait_timeout” mysql variable in your my.cnf configuration file is large enough.
Server dropped an incorrect or too large packet. If mysqld gets a packet that is too large or incorrect, it assumes that something has gone wrong with the client and closes the connection. To fix, you can increase the maximal packet size limit “max_allowed_packet” in my.cnf file, eg. set max_allowed_packet = 128M, then sudo /etc/init.d/mysql restart.
there are two main ways to fix this. if the above change doesn't there may be an issue with your linux or windows mysql database server; you either need to increase ram on your server or watch it's process.
is this on a windows or linux box?

Slow MySQL Updates/Inserts/Deletes

I seem to be having slow inserts, updates and deletes on all tables on a specific database with MySQL. Not a lot of data in those tables (from 2k to 20k). Small number of columns (5-10), indexes (two of them), and no duplicate index issue. I'm running MySQL 5.0.45 with MyISAM.
I run the following query and it takes about 5-7 seconds:
UPDATE accounts SET updated_at = '2010-10-09 11:22:53' WHERE id = 8;
Selects seem to come back right away.
Explain gives me the following:
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | accounts | index | NULL | PRIMARY | 4 | NULL | 1841 | Using index |
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
The profiler doesn't show any significant data for anything other than a seemingly high number of context switches:
+----------------------+----------+-------------------+---------------------+
| Status | Duration | Context_voluntary | Context_involuntary |
+----------------------+----------+-------------------+---------------------+
| (initialization) | 0.000057 | 0 | 0 |
| checking permissions | 0.000008 | 0 | 0 |
| Opening tables | 0.000013 | 0 | 0 |
| System lock | 0.000005 | 0 | 0 |
| Table lock | 0.000005 | 0 | 0 |
| init | 0.000061 | 0 | 0 |
| Updating | 0.000101 | 0 | 0 |
| end | 7.957233 | 7951 | 2 |
| query end | 0.000008 | 0 | 0 |
| freeing items | 0.000011 | 0 | 0 |
| closing tables | 0.000007 | 1 | 0 |
| logging slow query | 0.000002 | 0 | 0 |
+----------------------+----------+-------------------+---------------------+
This might also help:
+----------------------+----------+-----------------------+---------------+-------------+
| Status | Duration | Source_function | Source_file | Source_line |
+----------------------+----------+-----------------------+---------------+-------------+
| (initialization) | 0.000057 | check_access | sql_parse.cc | 5306 |
| checking permissions | 0.000008 | open_tables | sql_base.cc | 2629 |
| Opening tables | 0.000013 | mysql_lock_tables | lock.cc | 153 |
| System lock | 0.000005 | mysql_lock_tables | lock.cc | 162 |
| Table lock | 0.000005 | mysql_update | sql_update.cc | 167 |
| init | 0.000061 | mysql_update | sql_update.cc | 429 |
| Updating | 0.000101 | mysql_update | sql_update.cc | 560 |
| end | 7.957233 | mysql_execute_command | sql_parse.cc | 5122 |
| query end | 0.000008 | mysql_parse | sql_parse.cc | 6116 |
| freeing items | 0.000011 | dispatch_command | sql_parse.cc | 2146 |
| closing tables | 0.000007 | log_slow_statement | sql_parse.cc | 2204 |
| logging slow query | 0.000002 | dispatch_command | sql_parse.cc | 2169 |
+----------------------+----------+-----------------------+---------------+-------------+
Additional info:
It's running on a CentOS-5 VPS with 4 gigs of ram guaranteed. No index on the updated_at column and not triggers anywhere.
[New things that I tried]
Created a new table (using like)
running innodb and inserted all
records from one of the affected
tables. (same problem)
Backed up the database and restored it to a
different database within the same
server instance. (same problem)
Restored that same backup to my
local machine and I didn't have a
problem.
Tried another database
within the same mysql server
instance that has the problem
database and the other database (a
Wordpress DB) ran
updates/inserts/deletes just fine.
Restarted mysqld and restarted the entire server last night (same problem)
Updated MySQL to version 5.0.77 (same problem)
Deleted all indexes from one of the affected tables (same problem)
Any ideas what to look at next or what might be the issue? Seems to be more of a recent problem though I can't say when it started to show up exactly.
If you have variable length rows, you might need to run OPTIMIZE TABLE occasionally.
Finally found the answer. That database was somehow missing the MYD and MYI files and still running. Not sure how that's possible considering that the MYD file holds the data for MyISAM tables but that was causing the slow inserts/updates/deletes.
I ran an ALTER TABLE to set the engine to MyISAM (which it already was) and it recreated those files. Updates/inserts/deletes running fast again!

mysql profiler "Sending data"

Is there an explanation of these statuses anywhere?
http://dev.mysql.com/tech-resources/articles/using-new-query-profiler.html
my specific question is in regards to this query:
select count(*)
from 135_5m.record_updates u, 135_5m.records r
where r.record_id = u.record_id and
(u.date_updated > null or null is null) and
u.date_updated <= '2011-01-03';
which returns a single number - 4053904. So why would the majority of the time be spent in "Sending data"? Is it just poorly named? Surely "Sending data" must be doing more than just sending data?
+--------------------------------+-----------+-------+
| Status | Duration | Swaps |
+--------------------------------+-----------+-------+
| starting | 0.000224 | 0 |
| checking query cache for query | 0.000188 | 0 |
| checking permissions | 0.000012 | 0 |
| checking permissions | 0.000017 | 0 |
| Opening tables | 0.000036 | 0 |
| System lock | 0.000015 | 0 |
| Table lock | 0.000067 | 0 |
| init | 0.000105 | 0 |
| optimizing | 0.000052 | 0 |
| statistics | 0.000254 | 0 |
| preparing | 0.000061 | 0 |
| executing | 0.000017 | 0 |
| Sending data | 32.079549 | 0 |
| end | 0.000036 | 0 |
| query end | 0.000012 | 0 |
| freeing items | 0.000089 | 0 |
| storing result in query cache | 0.000022 | 0 |
| logging slow query | 0.000008 | 0 |
| logging slow query | 0.000008 | 0 |
| cleaning up | 0.000011 | 0 |
+--------------------------------+-----------+-------+
http://dev.mysql.com/doc/refman/5.0/en/general-thread-states.html
Executing means the thread has started execution, Sending data apparently covers both the processing of the rows and sending the count back to the client.
before sending data to client, mysql need to read data, the phase of read data may take the majrity time of "sending data"