Do you know if there is any manner to extract maximum number into a string in MySql ?
My table looks like this:
+-----------------------------------------------------------+
| Freq |
+-----------------------------------------------------------+
| 40.56 |
| 99.51 |
| 99.87 |
| T=0.00, TCCA=0.00, TTCC=0.03, TTCG=0.00, TTCT=55.0 |
| C=97.70, T=0.05 |
| 44.61 |
| 57.45 |
| 38.18 |
| 38.43 |
| 39.36 |
| 38.65 |
| 39.30 |
| C=57.53, T=0.17 |
| TAAAAAAAAAAA=0.51, TAAAAAAAAAA=68.03, TAAAAAAAAAAAG=31.42 |
+-----------------------------------------------------------+
Desired output:
55.0
97.70
MySQL 5.7.17 MGR deploy in single-primary mode, 3 node all one one machine, the same configuration.
And then we test insert on the primary node and observe that the slave nodes have high delay with the write node, even primary node finish the insert test, slave node data is incresing!
Why MySQL Group Replication slave nodes have high delay with write node?
here is the my.cnf:
[mysqld]
datadir=/dba/mysql/data/s1
basedir=/dba/mysql/mysql-5.7/
port=24801
socket=/dba/mysql/data/s1/s1.sock
server_id=1
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_checksum=NONE
log_slave_updates=ON
log_bin=binlog
binlog_format=ROW
transaction_write_set_extraction=XXHASH64
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
loose-group_replication_start_on_boot=off
loose-group_replication_local_address= "127.0.0.1:24901"
loose-group_replication_group_seeds= "127.0.0.1:24901,127.0.0.1:24902,127.0.0.1:24903"
loose-group_replication_bootstrap_group= off
loose-group_replication_single_primary_mode=true
loose-group_replication_enforce_update_everywhere_checks=false
slave_parallel_type=LOGICAL_CLOCK
slave_preserve_commit_order=1
slave_parallel_workers=4
and the MGR config :
mysql> show variables like '%group_replication%';
+----------------------------------------------------+-------------------------------------------------+
| Variable_name | Value |
+----------------------------------------------------+-------------------------------------------------+
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_allow_local_lower_version_join | OFF |
| group_replication_auto_increment_increment | 7 |
| group_replication_bootstrap_group | OFF |
| group_replication_components_stop_timeout | 31536000 |
| group_replication_compression_threshold | 1000000 |
| group_replication_enforce_update_everywhere_checks | OFF |
| group_replication_flow_control_applier_threshold | 25000 |
| group_replication_flow_control_certifier_threshold | 25000 |
| group_replication_flow_control_mode | QUOTA |
| group_replication_force_members | |
| group_replication_group_name | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa |
| group_replication_group_seeds | 127.0.0.1:24901,127.0.0.1:24902,127.0.0.1:24903 |
| group_replication_gtid_assignment_block_size | 1000000 |
| group_replication_ip_whitelist | AUTOMATIC |
| group_replication_local_address | 127.0.0.1:24901 |
| group_replication_poll_spin_loops | 0 |
| group_replication_recovery_complete_at | TRANSACTIONS_APPLIED |
| group_replication_recovery_reconnect_interval | 60 |
| group_replication_recovery_retry_count | 10 |
| group_replication_recovery_ssl_ca | |
| group_replication_recovery_ssl_capath | |
| group_replication_recovery_ssl_cert | |
| group_replication_recovery_ssl_cipher | |
| group_replication_recovery_ssl_crl | |
| group_replication_recovery_ssl_crlpath | |
| group_replication_recovery_ssl_key | |
| group_replication_recovery_ssl_verify_server_cert | OFF |
| group_replication_recovery_use_ssl | OFF |
| group_replication_single_primary_mode | ON |
| group_replication_ssl_mode | DISABLED |
| group_replication_start_on_boot | OFF |
+----------------------------------------------------+-------------------------------------------------+
32 rows in set (0.01 sec)
I have apache Hive - 0.12.0 and hadoop - 1.2.1 version on my machine.
and used mysql for hive metastore.
Now when i fire show tables for metastore test, it only shows 20 rows/tables.
// My Environment
mysql> use metastore_db
Database changed
mysql> show tables;
+---------------------------+
| Tables_in_metastore_db |
+---------------------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| DATABASE_PARAMS |
| DBS |
| PARTITION_KEYS |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TBLS |
| VERSION |
+---------------------------+
20 rows in set (0.06 sec)
Why am not able to see all the tables like below one. Do i need to use any other hive version for this ? Please suggest.
// Need to see all these tables
mysql> show tables;
+ --------------------------- +
| Tables_in_hive_demo |
+ --------------------------- +
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| GLOBAL_PRIVS |
| IDXS |
| INDEX_PARAMS |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| ROLES |
| ROLE_MAP |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TYPES |
| TYPE_FIELDS |
| VERSION |
+ --------------------------- +
39 rowsinset (0.00 sec)
Try to upgrade the hive schema by the available sql script (mysql in your case).
mysql> use metastore_db
Database changed
mysql> SOURCE $HIVE_HOME/scripts/metastore/upgrade/mysql/hive-schema-0.12.0.mysql.sql;
mysql> CREATE USER 'dbuser'#'%' IDENTIFIED BY 'dbpassword';
mysql> GRANT all on *.* to 'dbuser'#localhost identified by 'dbpassword';
mysql> flush privileges;
Now requery your tables. It should show all the tables now.
mysql> show tables;
I just dont understand one thing. When I type:
SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE';
I get:
+----------------------------------------------+
| table_name |
+----------------------------------------------+
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
41 rows in set (0.23 sec)
but selecting seems not to work at all:
mysql> select * from db;
ERROR 1109 (42S02): Unknown table 'db' in information_schema
mysql>
How is that possible? I mean, SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE'; showed that there IS a table named 'db' ...
Secondly, when I type: SELECT table_name FROM INFORMATION_SCHEMA.TABLES; it gives me :
+----------------------------------------------+
| table_name |
+----------------------------------------------+
| CHARACTER_SETS |
| COLLATIONS |
| COLLATION_CHARACTER_SET_APPLICABILITY |
| COLUMNS |
| COLUMN_PRIVILEGES |
| ENGINES |
| EVENTS |
| FILES |
| GLOBAL_STATUS |
| GLOBAL_VARIABLES |
| KEY_COLUMN_USAGE |
| PARAMETERS |
| PARTITIONS |
| PLUGINS |
| PROCESSLIST |
| PROFILING |
| REFERENTIAL_CONSTRAINTS |
| ROUTINES |
| SCHEMATA |
| SCHEMA_PRIVILEGES |
| SESSION_STATUS |
| SESSION_VARIABLES |
| STATISTICS |
| TABLES |
| TABLESPACES |
| TABLE_CONSTRAINTS |
| TABLE_PRIVILEGES |
| TRIGGERS |
| USER_PRIVILEGES |
| VIEWS |
| INNODB_BUFFER_PAGE |
| INNODB_TRX |
| INNODB_BUFFER_POOL_STATS |
| INNODB_LOCK_WAITS |
| INNODB_CMPMEM |
| INNODB_CMP |
| INNODB_LOCKS |
| INNODB_CMPMEM_RESET |
| INNODB_CMP_RESET |
| INNODB_BUFFER_PAGE_LRU |
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
81 rows in set (0.00 sec)
and Im able to do: mysql> select * from events; and Im getting some results. Why is that? Why can I do select only on upper-case-names? And further, how can I select from information_schema tables given only in uppercase? Cheers
If it matters: Im logged as root to my db.
You select only table_name, but what about table_schema:
MariaDB [(none)]> SELECT table_name, table_schema FROM INFORMATION_SCHEMA.TABLES WHERE
TABLE_TYPE = 'BASE TABLE';
+----------------------------------------------+--------------------+
| table_name | table_schema |
+----------------------------------------------+--------------------+
| columns_priv | mysql |
| db | mysql |
| event | mysql |
| func | mysql |
| general_log | mysql |
| help_category | mysql |
| help_keyword | mysql |
| help_relation | mysql |
| help_topic | mysql |
| host | mysql |
| ndb_binlog_index | mysql |
| plugin | mysql |
| proc | mysql |
| procs_priv | mysql |
| proxies_priv | mysql |
| servers | mysql |
| slow_log | mysql |
| tables_priv | mysql |
| time_zone | mysql |
| time_zone_leap_second | mysql |
| time_zone_name | mysql |
| time_zone_transition | mysql |
| time_zone_transition_type | mysql |
| user | mysql |
| cond_instances | performance_schema |
| events_waits_current | performance_schema |
| events_waits_history | performance_schema |
| events_waits_history_long | performance_schema |
| events_waits_summary_by_instance | performance_schema |
| events_waits_summary_by_thread_by_event_name | performance_schema |
| events_waits_summary_global_by_event_name | performance_schema |
| file_instances | performance_schema |
| file_summary_by_event_name | performance_schema |
| file_summary_by_instance | performance_schema |
| mutex_instances | performance_schema |
| performance_timers | performance_schema |
| rwlock_instances | performance_schema |
| setup_consumers | performance_schema |
| setup_instruments | performance_schema |
| setup_timers | performance_schema |
| threads | performance_schema |
+----------------------------------------------+--------------------+
And you get:
MariaDB [(none)]> select * from mysql.db;
Empty set (0.00 sec)
'information_schema' has ONLY SERVICE INFORMATION. It has info about table 'db' is exist, but it table IS NOT IN 'information_schema' DATABASE -- somewhere, but not in 'information_schema'.
Info about database which has needed table saved in TABLE_SCHEMA field
I want to order by Time,but seems no way to do that ?
mysql> show processlist;
+--------+-------------+--------------------+------+---------+--------+----------------------------------+------------------------------------------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+--------+-------------+--------------------+------+---------+--------+----------------------------------+------------------------------------------------------------------------------------------------------+
| 1 | system user | | NULL | Connect | 226953 | Waiting for master to send event | NULL |
| 2 | system user | | v3 | Connect | 35042 | Locked | update postings a
left join cities b on b.id=a.job_city_id
left join states h on h.id=b.stat |
| 313888 | irnadmin | 172.19.0.239:40136 | v3 | Sleep | 0 | | NULL |
| 314075 | irnadmin | 172.19.0.239:41113 | v3 | Sleep | 0 | | NULL |
| 314118 | irnadmin | 172.19.0.239:41282 | v3 | Query | 34978 | freeing items | SELECT id, screen_name, type, active, bound, LastLogin, robotno, protocol FROM accounts WHERE email_ |
| 314686 | irnadmin | 172.19.0.239:43251 | v3 | Sleep | 0 | | NULL |
| 314732 | irnadmin | 172.19.0.239:43436 | v3 | Query | 34978 | freeing items | SELECT id, screen_name, type, active, bound, LastLogin, robotno, protocol FROM accounts WHERE email_ |
| 314984 | irnadmin | 172.19.0.239:44366 | v3 | Sleep | 2 | | NULL |
| 315051 | irnadmin | 172.19.0.239:44713 | v3 | Query | 0 | NULL | NULL |
| 315198 | irnadmin | 172.19.0.239:51569 | v3 | Sleep | 2 | | NULL |
| 315280 | irnadmin | 172.19.0.239:51849 | v3 | Query | 34978 | freeing items | SELECT id, email_address, type, closed, robotno FROM accounts WHERE screen_name = 'ShantanuS' |
| 315320 | irnadmin | 172.19.0.239:52045 | v3 | Query | 34978 | freeing items | SELECT id, screen_name, type, active, bound, LastLogin, robotno, protocol FROM accounts WHERE email_ |
| 315384 | irnadmin | 172.19.0.239:52463 | v3 | Sleep | 1 | | NULL |
| 452248 | irnadmin | 172.19.0.28:54899 | v3 | Query | 34978 | freeing items | SELECT id, email_address, type, closed, robotno FROM accounts WHERE screen_name = 'LIZW0218' |
| 452291 | irnadmin | 172.19.0.28:55045 | v3 | Sleep | 1 | | NULL |
| 452316 | irnadmin | 172.19.0.28:55144 | v3 | Sleep | 0 | | NULL |
| 452353 | irnadmin | 172.19.0.28:55278 | v3 | Sleep | 0 | | NULL |
| 452382 | irnadmin | 172.19.0.28:55371 | v3 | Query | 34978 | freeing items | SELECT o.account_id FROM online o JOIN accounts a ON a.id=o.account_id WHERE o.server_id IS NULL AND |
| 452413 | irnadmin | 172.19.0.28:55479 | v3 | Sleep | 1 | | NULL |
| 452541 | irnadmin | 172.19.0.28:55946 | v3 | Query | 34978 | freeing items | SELECT o.account_id FROM online o JOIN accounts a ON a.id=o.account_id WHERE o.server_id IS NULL AND |
| 452626 | irnadmin | 172.19.0.28:56215 | v3 | Sleep | 2 | | NULL |
| 452711 | irnadmin | 172.19.0.28:39916 | v3 | Sleep | 0 | | NULL |
| 452781 | irnadmin | 172.19.0.28:40161 | v3 | Sleep | 1 | | NULL |
| 452904 | irnadmin | 172.19.0.28:40955 | v3 | Query | 34978 | freeing items | select a.id, aa.screen_name, i.requester from interview_requests i left join accounts aa on aa.id=i. |
| 453014 | irnadmin | 172.19.0.28:41291 | v3 | Query | 34978 | freeing items | SELECT o.account_id FROM online o JOIN accounts a ON a.id=o.account_id WHERE o.server_id IS NULL AND |
| 453057 | irnadmin | 172.19.0.28:41377 | v3 | Query | 34978 | freeing items | select a.id, aa.screen_name, i.requester from interview_requests i left join accounts aa on aa.id=i. |
| 453084 | irnadmin | 172.19.0.28:41441 | v3 | Sleep | 0 | | NULL |
| 453112 | irnadmin | 172.19.0.28:41536 | v3 | Sleep | 0 | | NULL |
| 453156 | irnadmin | 172.19.0.28:41653 | v3 | Query | 34978 | freeing items | SELECT protocol FROM accounts WHERE email_address= '***#gtalk.jabber.jobirn.c |
| 453214 | irnadmin | 172.19.0.28:41800 | v3 | Sleep | 5 | | NULL |
| 453243 | irnadmin | 172.19.0.28:41991 | v3 | Sleep | 0 | | NULL |
| 453313 | irnadmin | 172.19.0.28:42255 | v3 | Query | 34978 | freeing items | SELECT o.account_id FROM online o JOIN accounts a ON a.id=o.account_id WHERE o.server_id IS NULL AND |
| 453396 | irnadmin | 172.19.0.28:53718 | v3 | Sleep | 2 | | NULL |
| 453476 | irnadmin | 172.19.0.28:54019 | v3 | Sleep | 0 | | NULL |
| 453561 | irnadmin | 172.19.0.28:54352 | v3 | Sleep | 3 | | NULL |
| 453594 | irnadmin | 172.19.0.28:54456 | v3 | Sleep | 0 | | NULL |
| 453727 | irnadmin | 172.19.0.28:55166 | v3 | Query | 34978 | freeing items | SELECT id, screen_name, type, active, bound, LastLogin, robotno, protocol FROM accounts WHERE email_ |
| 453786 | irnadmin | 172.19.0.28:55320 | v3 | Sleep | 4 | | NULL |
| 610140 | irnadmin | 172.19.0.28:33848 | v3 | Query | 34978 | freeing items | select a.id, aa.screen_name, i.requester from interview_requests i left join accounts aa on aa.id=i. |
| 685119 | irnadmin | 172.19.0.27:37251 | v3 | Query | 34980 | Sending data | select postings.id id,category, job_desc_title,
IF(c1.name is not null,c1.name,IF(c2.name is not n |
| 685226 | irnadmin | 172.19.0.139:57274 | v3 | Query | 34735 | Locked | SELECT job_desc_title,job_desc,job_state_name,job_city_name,company_categories.name,postings.categor |
| 685229 | irnadmin | 172.19.0.139:57278 | v3 | Query | 34735 | Locked | SELECT job_desc_title,job_desc,job_state_name,job_city_name,company_categories.name,postings.categor |
| 685232 | irnadmin | 172.19.0.139:57283 | v3 | Query | 34734 | Locked | select job_desc_title,job_desc from postings where id=287650 |
| 685233 | irnadmin | 172.19.0.139:57286 | v3 | Query | 34734 | Locked | SELECT accounts.screen_name,postings.url url, accounts.type owner_type, postings.id ID, postings.job |
| 685235 | irnadmin | 172.19.0.28:37502 | v3 | Query | 34734 | Locked | SELECT accounts.screen_name,postings.url url, accounts.type owner_type, postings.id ID, postings.job |
| 686496 | irnadmin | 172.19.0.239:33306 | v3 | Query | 32589 | Locked | SELECT accounts.screen_name,postings.url url, accounts.type owner_type, postings.id ID, postings.job |
| 686503 | irnadmin | 172.19.0.28:54051 | v3 | Query | 32588 | Locked | SELECT job_desc_title, job_desc, IF(postings.category IS NOT NULL, postings.category, job_categories |
| 709550 | root | localhost | v3 | Query | 0 | NULL | show processlist |
| 710084 | irnadmin | 172.19.0.27:53285 | NULL | Query | 0 | removing tmp table | show status where Variable_name='Threads_running' |
+--------+-------------+--------------------+------+---------+--------+----------------------------------+------------------------------------------------------------------------------------------------------+
49 rows in set (0.00 sec)
Newer versions of SQL support the process list in information_schema:
SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST
You can ORDER BY in any way you like.
The INFORMATION_SCHEMA.PROCESSLIST table was added in MySQL 5.1.7. You can find out which version you're using with:
SELECT VERSION()
The command
show full processlist
can be replaced by:
SELECT * FROM information_schema.processlist
but if you go with the latter version you can add WHERE clause to it:
SELECT * FROM information_schema.processlist WHERE `INFO` LIKE 'SELECT %';
For more information visit this
Another useful tool for this from the command line interface, is the pager command.
eg
pager grep -v Sleep | more; show full processlist;
Then you can page through the results.
You can also look for certain users, IPs or queries with grep or sed in this way.
The pager command is persistent per session.
You can just capture the output and pass it through a filter, something like:
mysql show processlist
| grep -v '^\+\-\-'
| grep -v '^| Id'
| sort -n -k12
The two greps strip out the header and trailer lines (others may be needed if there are other lines not containing useful information) and the sort is done based on the numeric field number 12 (I think that's right).
This one works for your immediate output:
mysql show processlist
| grep -v '^\+\-\-'
| grep -v '^| Id'
| grep -v '^[0-9][0-9]* rows in set '
| grep -v '^ '
| sort -n -k12
If you use old version of MySQL you can always use \P combined with some nice piece of awk code.
Interesting example here
http://www.dbasquare.com/2012/03/28/how-to-work-with-a-long-process-list-in-mysql/
Isn't it exactly what you need?
...We don't have a newer version of MySQL yet, so I was able to do this (works only on UNIX):
host=maindb
echo "show full processlist\G" | mysql -h$host | grep -B 6 -A 1 Locked
The above will query for all locked sessions, and return the information and SQL that is involved.
...So- assuming you wanted to query for sessions that were sleeping:
host=maindb
echo "show full processlist\G" | mysql -h$host | grep -B 6 -A 1 Sleep
Or, assuming you needed to provide additional connection parameters for MySQL:
host=maindb
user=me
password=mycoolpassword
echo "show full processlist\G" | mysql -h$host -u$user -p$password | grep -B 6 -A 1 Locked
With a couple of tweaks, I'm sure a shell script could be easily created to query the processlist the way you want it.
Use the "|" as separator
mysqladmin processlist| sort -t\| -n -k7
From sort manual ( man sort )
-k, --key=KEYDEF
sort via a key; KEYDEF gives location and type
-t, --field-separator=SEP
use SEP instead of non-blank to blank transition
-n, --numeric-sort
compare according to string numerical value
-r, --reverse
reverse the result of comparisons