Software versions:
xtrabackup 8.0.12
percona-xtradb-cluster-server 8.0.18-9
I am running xtrabackup with this options:
--defaults-file=/etc/mysql/my.cnf --backup --user=backup
--password=**** --parallel=4 --no-timestamp --target-dir=/my-backup-dir
Some of server options:
binlog_format | ROW
gtid_mode | ON_PERMISSIVE
enforce_gtid_consistency | ON
File xtrabackup_binlog_info has only binlog file name and position:
mysql-bin.000159 251
No GTID, so I can not create GTID-based replication restoring a slave from this backup.
What should I do to make xtrabackup include this information?
UPDATE:
Check if GTIDs are enabled:
show global variables like '%gtid%';
| Variable_name | Value |
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| gtid_executed | c0e3de06-a2a6-11ea-913c-c7b046cf5782:1-3399594,
de211648-2642-ee18-628d-dc48283b005c:1-3697598:3877279-10141440 |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON_PERMISSIVE |
| gtid_owned | |
| gtid_purged | c0e3de06-a2a6-11ea-913c-c7b046cf5782:1-2661056,
de211648-2642-ee18-628d-dc48283b005c:1-3697598:3877279-10141440 |
| session_track_gtids | OFF |
show master status;
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
| mysql-bin.000166 | 15285372 | | | c0e3de06-a2a6-11ea-913c-c7b046cf5782:1-3358798,
de211648-2642-ee18-628d-dc48283b005c:1-3697598:3877279-10141440 |
(pavel.selivanov#localhost) [kassa_prod]> show binlog events in 'mysql-bin.000166' limit 7;
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info
| mysql-bin.000166 | 4 | Format_desc | 4315 | 124 | Server ver: 8.0.18-9, Binlog ver: 4 |
| mysql-bin.000166 | 124 | Previous_gtids | 4315 | 251 | c0e3de06-a2a6-11ea-913c-c7b046cf5782:1-3347159,
de211648-2642-ee18-628d-dc48283b005c:3697598:3877279-10141440 |
| mysql-bin.000166 | 251 | Gtid | 4315 | 330 | SET ##SESSION.GTID_NEXT= 'c0e3de06-a2a6-11ea-913c-c7b046cf5782:3347160' |
| mysql-bin.000166 | 330 | Query | 4315 | 411 | BEGIN |
| mysql-bin.000166 | 411 | Table_map | 4315 | 499 | table_id: 150 (db.table) |
| mysql-bin.000166 | 499 | Update_rows | 4315 | 3475 | table_id: 150 flags: STMT_END_F |
| mysql-bin.000166 | 3475 | Xid | 4315 | 3506 | COMMIT /* xid=9611331 */
From the manual:
ON_PERMISSIVE: New transactions are GTID transactions. Replicated transactions can be either anonymous or GTID transactions.
Check with show global variables like 'gt%'; or in your binary logs, if you actually have GTID transactions.
You actually don't have anything special to do, to have xtrabackup include GTIDs in the xtrabackup_binlog_info file: How to create a new (or repair a broken) GTID based slave
Related
How to modify the system variable audit_log_format to change to JSON format. The variable is read-only and cannot be changed when the server is running.
mysql> SHOW variables LIKE 'audit%';
+-----------------------------+---------------+
| Variable_name | Value |
+-----------------------------+---------------+
| audit_log_buffer_size | 1048576 |
| audit_log_exclude_accounts | |
| audit_log_exclude_commands | |
| audit_log_exclude_databases | |
| audit_log_file | audit.log |
| audit_log_flush | OFF |
| audit_log_format | OLD |
| audit_log_handler | FILE |
| audit_log_include_accounts | |
| audit_log_include_commands | |
| audit_log_include_databases | |
| audit_log_policy | ALL |
| audit_log_rotate_on_size | 0 |
| audit_log_rotations | 0 |
| audit_log_strategy | ASYNCHRONOUS |
| audit_log_syslog_facility | LOG_USER |
| audit_log_syslog_ident | percona-audit |
| audit_log_syslog_priority | LOG_INFO |
+-----------------------------+---------------+
18 rows in set (0.00 sec)
I edit my my.cnf in this way, but it doesnt work
bash-4.4$ cat ./etc/my.cnf
#
# The Percona Server 5.7 configuration file.
#
#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
# Please make any edits and changes to the appropriate sectional files
# included below.
#
!includedir /etc/my.cnf.d/
!includedir /etc/percona-server.conf.d/
[mysqld]
## Audit Logging ##
audit_log_format=JSON
The audit plugin is need to be loaded at first.
[mysqld]
plugin-load="audit_log=audit_log.so"
audit_log_policy=ALL
audit_log_format=JSON
audit_log_rotate_on_size=1024M
audit_log_rotations=10
I am facing high CPU utilization issue, is too many concurrent create temporary table statement cause high CPU utilization?
Is there any query through that we can capture queries which causing high CPU utilization?
Variable we set:-
tmp_table_size = 1G
max_heap_table_size = 1G
innodb_buffer_pool_size = 145 G
innodb_buffer_pool_instance = 8
innodb_page_cleaner = 8
Status Variables:-
mysql> show global status like '%tmp%';
+-------------------------+-----------+
| Variable_name | Value |
+-------------------------+-----------+
| Created_tmp_disk_tables | 60844516 |
| Created_tmp_files | 135751 |
| Created_tmp_tables | 107643364 |
+-------------------------+-----------+
mysql> show global status like '%innodb_buffer%';
+---------------------------------------+--------------------------------------------------+
| Variable_name | Value |
+---------------------------------------+--------------------------------------------------+
| Innodb_buffer_pool_dump_status | Dumping of buffer pool not started |
| Innodb_buffer_pool_load_status | Buffer pool(s) load completed at 170917 19:11:45 |
| Innodb_buffer_pool_resize_status | |
| Innodb_buffer_pool_pages_data | 8935464 |
| Innodb_buffer_pool_bytes_data | 146398642176 |
| Innodb_buffer_pool_pages_dirty | 18824 |
| Innodb_buffer_pool_bytes_dirty | 308412416 |
| Innodb_buffer_pool_pages_flushed | 122454921 |
| Innodb_buffer_pool_pages_free | 188279 |
| Innodb_buffer_pool_pages_misc | 377817 |
| Innodb_buffer_pool_pages_total | 9501560 |
| Innodb_buffer_pool_read_ahead_rnd | 0 |
| Innodb_buffer_pool_read_ahead | 585245 |
| Innodb_buffer_pool_read_ahead_evicted | 14383 |
| Innodb_buffer_pool_read_requests | 304878851665 |
| Innodb_buffer_pool_reads | 10537188 |
| Innodb_buffer_pool_wait_free | 0 |
| Innodb_buffer_pool_write_requests | 14749510186 |
+---------------------------------------+--------------------------------------------------+
Step 1 -
show processlist
Find is any process is locking table if yes than change it to myisam.
Step 2 -
Check Ram and your db size
Step 3 -
Explain complex queries and check if file sort or maximum number od rows are getting scan remove it either by making table flat , not more than 4 sub queries
Step 4 -
Use of joins efficiently
My issue is even i disable the root user from audit logging but still logging for these user. Anyone please help. Here is i did step by step.
[Setp -1] Check the audit log variable.
mysql> SHOW VARIABLES LIKE 'audit_log%';
+-----------------------------+--------------+
| Variable_name | Value |
+-----------------------------+--------------+
| audit_log_buffer_size | 1048576 |
| audit_log_connection_policy | ALL |
| audit_log_current_session | ON |
| audit_log_exclude_accounts | |
| audit_log_file | audit.log |
| audit_log_flush | OFF |
| audit_log_format | OLD |
| audit_log_include_accounts | |
| audit_log_policy | ALL |
| audit_log_rotate_on_size | 0 |
| audit_log_statement_policy | ALL |
| audit_log_strategy | ASYNCHRONOUS |
+-----------------------------+--------------+
12 rows in set (0.00 sec)
[Setp-2]
The following statement is disable audit logging for root account.
-- audit_log_include_accounts to NULL
SET GLOBAL audit_log_include_accounts = NULL;
SET GLOBAL audit_log_exclude_accounts = root#%;
Note: I used the root#% instead root#localhost because of this database server can access from another ip address.
[Setp-3] I call the select statement SELECT * FROM SSVR_AUDIT_LOG from remote PC.
[Step-4] I checked the audit log in DB server.
<AUDIT_RECORD TIMESTAMP="2016-04-22T03:49:11 UTC" RECORD_ID="593_2016-04-22T01:28:17" NAME="Query" CONNECTION_ID="6" STATUS="0" STATUS_CODE="0" USER="root[root] # [162.16.22.48]" OS_LOGIN="" HOST="" IP="162.16.22.48" COMMAND_CLASS="show_create_table" SQLTEXT="SHOW CREATE TABLE `SSVR_AUDIT_LOG`"/>
<AUDIT_RECORD TIMESTAMP="2016-04-22T03:49:12 UTC" RECORD_ID="594_2016-04-22T01:28:17" NAME="Query" CONNECTION_ID="7" STATUS="0" STATUS_CODE="0" USER="root[root] # [162.16.22.48]" OS_LOGIN="" HOST="" IP="162.16.22.48" COMMAND_CLASS="select" SQLTEXT="SELECT * FROM `SSVR_AUDIT_LOG` LIMIT 0, 1000"/>
<AUDIT_RECORD TIMESTAMP="2016-04-22T03:49:12 UTC" RECORD_ID="595_2016-04-22T01:28:17" NAME="Query" CONNECTION_ID="7" STATUS="0" STATUS_CODE="0" USER="root[root] # [162.16.22.48]" OS_LOGIN="" HOST="" IP="162.16.22.48" COMMAND_CLASS="show_fields" SQLTEXT="SHOW COLUMNS FROM `tldssvr`.`SSVR_AUDIT_LOG`"/>
<AUDIT_RECORD TIMESTAMP="2016-04-22T03:49:13 UTC" RECORD_ID="596_2016-04-22T01:28:17" NAME="Quit" CONNECTION_ID="7" STATUS="0" STATUS_CODE="0" USER="root" OS_LOGIN="" HOST="" IP="162.16.22.48" COMMAND_CLASS="connect"/>
Here is my reference link enter link description here
I got the answer for my question. Here is correct answer. When you facing like that issue, you can follow below the steps.
Audit Log Filtering by Account
List all ‘audit log’ configuration items
> mysql -u root -p
> SHOW VARIABLES LIKE ‘audit_log%’;
+-----------------------------+--------------+
| Variable_name | Value |
+-----------------------------+--------------+
| audit_log_buffer_size | 1048576 |
| audit_log_connection_policy | ALL |
| audit_log_current_session | OFF |
| audit_log_exclude_accounts | |
| audit_log_file | audit.log |
| audit_log_flush | OFF |
| audit_log_format | OLD |
| audit_log_include_accounts | |
| audit_log_policy | ALL |
| audit_log_rotate_on_size | 0 |
| audit_log_statement_policy | ALL |
| audit_log_strategy | ASYNCHRONOUS |
+-----------------------------+--------------+
To add the remote application server host name and ip address in database server.
> cat /etc/hosts
> 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
162.16.22.48 App_PC
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
To disable audit logging only for the application database user (root) local host and remote host accounts.
> mysql –u root –p
>SET GLOBAL audit_log_include_accounts = NULL;
>SET GLOBAL audit_log_exclude_accounts = 'root#localhost,root#App_PC';
List all ‘audit log’ configuration items and check the audit_log_exclude_account value.
> SHOW VARIABLES LIKE 'audit_log%';
> +-----------------------------+----------------------------+
| Variable_name | Value |
+-----------------------------+----------------------------+
| audit_log_buffer_size | 1048576 |
| audit_log_connection_policy | ALL |
| audit_log_current_session | OFF |
| audit_log_exclude_accounts | root#localhost,root#App_PC |
| audit_log_file | audit.log |
| audit_log_flush | OFF |
| audit_log_format | OLD |
| audit_log_include_accounts | |
| audit_log_policy | ALL |
| audit_log_rotate_on_size | 0 |
| audit_log_statement_policy | ALL |
| audit_log_strategy | ASYNCHRONOUS |
+-----------------------------+----------------------------+
In my MySQL 5.6.27 default installation the 'events_statements_summary_by_digest' table is missing in the 'performance_schema' database.
In fact, I think more tables are missing. Did I something wrong during the installation? How can I fix this?
use performance_schema;
show tables;
+----------------------------------------------+
| Tables_in_performance_schema |
+----------------------------------------------+
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
17 rows in set (0.00 sec)
The changelog for 5.6.9 says
"If you upgrade to this release of MySQL from an earlier version, you must run mysql_upgrade (and restart the server) to incorporate this change into the performance_schema database. (Bug #14075527)"
events_statements_summary_by_digest was added in 5.6.5.
I seem to be having slow inserts, updates and deletes on all tables on a specific database with MySQL. Not a lot of data in those tables (from 2k to 20k). Small number of columns (5-10), indexes (two of them), and no duplicate index issue. I'm running MySQL 5.0.45 with MyISAM.
I run the following query and it takes about 5-7 seconds:
UPDATE accounts SET updated_at = '2010-10-09 11:22:53' WHERE id = 8;
Selects seem to come back right away.
Explain gives me the following:
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | accounts | index | NULL | PRIMARY | 4 | NULL | 1841 | Using index |
+----+-------------+----------+-------+---------------+---------+---------+------+------+-------------+
The profiler doesn't show any significant data for anything other than a seemingly high number of context switches:
+----------------------+----------+-------------------+---------------------+
| Status | Duration | Context_voluntary | Context_involuntary |
+----------------------+----------+-------------------+---------------------+
| (initialization) | 0.000057 | 0 | 0 |
| checking permissions | 0.000008 | 0 | 0 |
| Opening tables | 0.000013 | 0 | 0 |
| System lock | 0.000005 | 0 | 0 |
| Table lock | 0.000005 | 0 | 0 |
| init | 0.000061 | 0 | 0 |
| Updating | 0.000101 | 0 | 0 |
| end | 7.957233 | 7951 | 2 |
| query end | 0.000008 | 0 | 0 |
| freeing items | 0.000011 | 0 | 0 |
| closing tables | 0.000007 | 1 | 0 |
| logging slow query | 0.000002 | 0 | 0 |
+----------------------+----------+-------------------+---------------------+
This might also help:
+----------------------+----------+-----------------------+---------------+-------------+
| Status | Duration | Source_function | Source_file | Source_line |
+----------------------+----------+-----------------------+---------------+-------------+
| (initialization) | 0.000057 | check_access | sql_parse.cc | 5306 |
| checking permissions | 0.000008 | open_tables | sql_base.cc | 2629 |
| Opening tables | 0.000013 | mysql_lock_tables | lock.cc | 153 |
| System lock | 0.000005 | mysql_lock_tables | lock.cc | 162 |
| Table lock | 0.000005 | mysql_update | sql_update.cc | 167 |
| init | 0.000061 | mysql_update | sql_update.cc | 429 |
| Updating | 0.000101 | mysql_update | sql_update.cc | 560 |
| end | 7.957233 | mysql_execute_command | sql_parse.cc | 5122 |
| query end | 0.000008 | mysql_parse | sql_parse.cc | 6116 |
| freeing items | 0.000011 | dispatch_command | sql_parse.cc | 2146 |
| closing tables | 0.000007 | log_slow_statement | sql_parse.cc | 2204 |
| logging slow query | 0.000002 | dispatch_command | sql_parse.cc | 2169 |
+----------------------+----------+-----------------------+---------------+-------------+
Additional info:
It's running on a CentOS-5 VPS with 4 gigs of ram guaranteed. No index on the updated_at column and not triggers anywhere.
[New things that I tried]
Created a new table (using like)
running innodb and inserted all
records from one of the affected
tables. (same problem)
Backed up the database and restored it to a
different database within the same
server instance. (same problem)
Restored that same backup to my
local machine and I didn't have a
problem.
Tried another database
within the same mysql server
instance that has the problem
database and the other database (a
Wordpress DB) ran
updates/inserts/deletes just fine.
Restarted mysqld and restarted the entire server last night (same problem)
Updated MySQL to version 5.0.77 (same problem)
Deleted all indexes from one of the affected tables (same problem)
Any ideas what to look at next or what might be the issue? Seems to be more of a recent problem though I can't say when it started to show up exactly.
If you have variable length rows, you might need to run OPTIMIZE TABLE occasionally.
Finally found the answer. That database was somehow missing the MYD and MYI files and still running. Not sure how that's possible considering that the MYD file holds the data for MyISAM tables but that was causing the slow inserts/updates/deletes.
I ran an ALTER TABLE to set the engine to MyISAM (which it already was) and it recreated those files. Updates/inserts/deletes running fast again!