Unable to view all the hive metastore tables in mysql setup - mysql

I have apache Hive - 0.12.0 and hadoop - 1.2.1 version on my machine.
and used mysql for hive metastore.
Now when i fire show tables for metastore test, it only shows 20 rows/tables.
// My Environment
mysql> use metastore_db
Database changed
mysql> show tables;
+---------------------------+
| Tables_in_metastore_db |
+---------------------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| DATABASE_PARAMS |
| DBS |
| PARTITION_KEYS |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TBLS |
| VERSION |
+---------------------------+
20 rows in set (0.06 sec)
Why am not able to see all the tables like below one. Do i need to use any other hive version for this ? Please suggest.
// Need to see all these tables
mysql> show tables;
+ --------------------------- +
| Tables_in_hive_demo |
+ --------------------------- +
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| GLOBAL_PRIVS |
| IDXS |
| INDEX_PARAMS |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| ROLES |
| ROLE_MAP |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TYPES |
| TYPE_FIELDS |
| VERSION |
+ --------------------------- +
39 rowsinset (0.00 sec)

Try to upgrade the hive schema by the available sql script (mysql in your case).
mysql> use metastore_db
Database changed
mysql> SOURCE $HIVE_HOME/scripts/metastore/upgrade/mysql/hive-schema-0.12.0.mysql.sql;
mysql> CREATE USER 'dbuser'#'%' IDENTIFIED BY 'dbpassword';
mysql> GRANT all on *.* to 'dbuser'#localhost identified by 'dbpassword';
mysql> flush privileges;
Now requery your tables. It should show all the tables now.
mysql> show tables;

Related

Not able restore mysql database

When i try to delete mysql database, before restoring, getting an error,
ERROR 1146 (42S02): Table 'mysql.proc' doesn't exist
But the mysql.proc exists in the database before dropping database.
Any idea why MySql is behaving unexpectedly here?
NOTE: This is not upgraded database, this is newly installed MariaDb-10.0.26 database.
Database changed
MariaDB [mysql]> show tables;
+---------------------------+
| Tables_in_mysql |
+---------------------------+
| column_stats |
| columns_priv |
| db |
| event |
| func |
| general_log |
| gtid_slave_pos |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| index_stats |
| innodb_index_stats |
| innodb_table_stats |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| roles_mapping |
| servers |
| slow_log |
| table_stats |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
+---------------------------+
30 rows in set (0.00 sec)
MariaDB [mysql]> drop database mysql;
ERROR 1146 (42S02): Table 'mysql.proc' doesn't exist

Why MySQL Group Replication slave nodes have high delay with write node?

MySQL 5.7.17 MGR deploy in single-primary mode, 3 node all one one machine, the same configuration.
And then we test insert on the primary node and observe that the slave nodes have high delay with the write node, even primary node finish the insert test, slave node data is incresing!
Why MySQL Group Replication slave nodes have high delay with write node?
here is the my.cnf:
[mysqld]
datadir=/dba/mysql/data/s1
basedir=/dba/mysql/mysql-5.7/
port=24801
socket=/dba/mysql/data/s1/s1.sock
server_id=1
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_checksum=NONE
log_slave_updates=ON
log_bin=binlog
binlog_format=ROW
transaction_write_set_extraction=XXHASH64
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
loose-group_replication_start_on_boot=off
loose-group_replication_local_address= "127.0.0.1:24901"
loose-group_replication_group_seeds= "127.0.0.1:24901,127.0.0.1:24902,127.0.0.1:24903"
loose-group_replication_bootstrap_group= off
loose-group_replication_single_primary_mode=true
loose-group_replication_enforce_update_everywhere_checks=false
slave_parallel_type=LOGICAL_CLOCK
slave_preserve_commit_order=1
slave_parallel_workers=4
and the MGR config :
mysql> show variables like '%group_replication%';
+----------------------------------------------------+-------------------------------------------------+
| Variable_name | Value |
+----------------------------------------------------+-------------------------------------------------+
| group_replication_allow_local_disjoint_gtids_join | OFF |
| group_replication_allow_local_lower_version_join | OFF |
| group_replication_auto_increment_increment | 7 |
| group_replication_bootstrap_group | OFF |
| group_replication_components_stop_timeout | 31536000 |
| group_replication_compression_threshold | 1000000 |
| group_replication_enforce_update_everywhere_checks | OFF |
| group_replication_flow_control_applier_threshold | 25000 |
| group_replication_flow_control_certifier_threshold | 25000 |
| group_replication_flow_control_mode | QUOTA |
| group_replication_force_members | |
| group_replication_group_name | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa |
| group_replication_group_seeds | 127.0.0.1:24901,127.0.0.1:24902,127.0.0.1:24903 |
| group_replication_gtid_assignment_block_size | 1000000 |
| group_replication_ip_whitelist | AUTOMATIC |
| group_replication_local_address | 127.0.0.1:24901 |
| group_replication_poll_spin_loops | 0 |
| group_replication_recovery_complete_at | TRANSACTIONS_APPLIED |
| group_replication_recovery_reconnect_interval | 60 |
| group_replication_recovery_retry_count | 10 |
| group_replication_recovery_ssl_ca | |
| group_replication_recovery_ssl_capath | |
| group_replication_recovery_ssl_cert | |
| group_replication_recovery_ssl_cipher | |
| group_replication_recovery_ssl_crl | |
| group_replication_recovery_ssl_crlpath | |
| group_replication_recovery_ssl_key | |
| group_replication_recovery_ssl_verify_server_cert | OFF |
| group_replication_recovery_use_ssl | OFF |
| group_replication_single_primary_mode | ON |
| group_replication_ssl_mode | DISABLED |
| group_replication_start_on_boot | OFF |
+----------------------------------------------------+-------------------------------------------------+
32 rows in set (0.01 sec)

Mysql insert command to have verbose

I need some assistance with find a way for the following command to verbose the commands that are running:
cat /tmp/queuemetrics_queue_log.sql |mysql queuemetrics
Regards
To see which statements are being processed you can run the command with the -v flag:
$ mysql -v -e 'select database(); show variables like "a%" '
--------------
select database()
--------------
+------------+
| database() |
+------------+
| foobarz |
+------------+
--------------
show variables like "a%"
--------------
+-----------------------------+-------------------------------+
| Variable_name | Value |
+-----------------------------+-------------------------------+
| activate_all_roles_on_login | OFF |
| admin_address | |
| admin_port | 33062 |
| admin_ssl_ca | |
| admin_ssl_capath | |
| admin_ssl_cert | |
| admin_ssl_cipher | |
| admin_ssl_crl | |
| admin_ssl_crlpath | |
| admin_ssl_key | |
| admin_tls_ciphersuites | |
| admin_tls_version | TLSv1,TLSv1.1,TLSv1.2,TLSv1.3 |
| auto_generate_certs | ON |
| auto_increment_increment | 1 |
| auto_increment_offset | 1 |
| autocommit | ON |
| automatic_sp_privileges | ON |
| avoid_temporal_upgrade | OFF |
+-----------------------------+-------------------------------+

Get all tables from information_schema, MySQL

I just dont understand one thing. When I type:
SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE';
I get:
+----------------------------------------------+
| table_name |
+----------------------------------------------+
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
41 rows in set (0.23 sec)
but selecting seems not to work at all:
mysql> select * from db;
ERROR 1109 (42S02): Unknown table 'db' in information_schema
mysql>
How is that possible? I mean, SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE'; showed that there IS a table named 'db' ...
Secondly, when I type: SELECT table_name FROM INFORMATION_SCHEMA.TABLES; it gives me :
+----------------------------------------------+
| table_name |
+----------------------------------------------+
| CHARACTER_SETS |
| COLLATIONS |
| COLLATION_CHARACTER_SET_APPLICABILITY |
| COLUMNS |
| COLUMN_PRIVILEGES |
| ENGINES |
| EVENTS |
| FILES |
| GLOBAL_STATUS |
| GLOBAL_VARIABLES |
| KEY_COLUMN_USAGE |
| PARAMETERS |
| PARTITIONS |
| PLUGINS |
| PROCESSLIST |
| PROFILING |
| REFERENTIAL_CONSTRAINTS |
| ROUTINES |
| SCHEMATA |
| SCHEMA_PRIVILEGES |
| SESSION_STATUS |
| SESSION_VARIABLES |
| STATISTICS |
| TABLES |
| TABLESPACES |
| TABLE_CONSTRAINTS |
| TABLE_PRIVILEGES |
| TRIGGERS |
| USER_PRIVILEGES |
| VIEWS |
| INNODB_BUFFER_PAGE |
| INNODB_TRX |
| INNODB_BUFFER_POOL_STATS |
| INNODB_LOCK_WAITS |
| INNODB_CMPMEM |
| INNODB_CMP |
| INNODB_LOCKS |
| INNODB_CMPMEM_RESET |
| INNODB_CMP_RESET |
| INNODB_BUFFER_PAGE_LRU |
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
| cond_instances |
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_global_by_event_name |
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
| mutex_instances |
| performance_timers |
| rwlock_instances |
| setup_consumers |
| setup_instruments |
| setup_timers |
| threads |
+----------------------------------------------+
81 rows in set (0.00 sec)
and Im able to do: mysql> select * from events; and Im getting some results. Why is that? Why can I do select only on upper-case-names? And further, how can I select from information_schema tables given only in uppercase? Cheers
If it matters: Im logged as root to my db.
You select only table_name, but what about table_schema:
MariaDB [(none)]> SELECT table_name, table_schema FROM INFORMATION_SCHEMA.TABLES WHERE
TABLE_TYPE = 'BASE TABLE';
+----------------------------------------------+--------------------+
| table_name | table_schema |
+----------------------------------------------+--------------------+
| columns_priv | mysql |
| db | mysql |
| event | mysql |
| func | mysql |
| general_log | mysql |
| help_category | mysql |
| help_keyword | mysql |
| help_relation | mysql |
| help_topic | mysql |
| host | mysql |
| ndb_binlog_index | mysql |
| plugin | mysql |
| proc | mysql |
| procs_priv | mysql |
| proxies_priv | mysql |
| servers | mysql |
| slow_log | mysql |
| tables_priv | mysql |
| time_zone | mysql |
| time_zone_leap_second | mysql |
| time_zone_name | mysql |
| time_zone_transition | mysql |
| time_zone_transition_type | mysql |
| user | mysql |
| cond_instances | performance_schema |
| events_waits_current | performance_schema |
| events_waits_history | performance_schema |
| events_waits_history_long | performance_schema |
| events_waits_summary_by_instance | performance_schema |
| events_waits_summary_by_thread_by_event_name | performance_schema |
| events_waits_summary_global_by_event_name | performance_schema |
| file_instances | performance_schema |
| file_summary_by_event_name | performance_schema |
| file_summary_by_instance | performance_schema |
| mutex_instances | performance_schema |
| performance_timers | performance_schema |
| rwlock_instances | performance_schema |
| setup_consumers | performance_schema |
| setup_instruments | performance_schema |
| setup_timers | performance_schema |
| threads | performance_schema |
+----------------------------------------------+--------------------+
And you get:
MariaDB [(none)]> select * from mysql.db;
Empty set (0.00 sec)
'information_schema' has ONLY SERVICE INFORMATION. It has info about table 'db' is exist, but it table IS NOT IN 'information_schema' DATABASE -- somewhere, but not in 'information_schema'.
Info about database which has needed table saved in TABLE_SCHEMA field

MySQL producing garbage output for simple queries?

I have a MySQL table that is very simple, containing four fields:
id (autonumbered)
controllerType
onInstructions
offInstructions
I loaded the table with data from a CSV file. The file looks like this:
10,GK 212-02,,
11,GK 212-02 ex,,
12,GK 212-04,,
13,GK 212-06,,
14,GK 212-06 x 2,,
And the query I run looks like this:
load data local infile '/home/garfonzo/controllers.csv' into table systems_controller_type fields terminated by ',' lines terminated by '\n' (id, controllerType, onInstructions, offInstructions);
It loads the data into the table with no warnings:
Query OK, 42 rows affected (0.00 sec)
Records: 42 Deleted: 0 Skipped: 0 Warnings: 0
Then, when I run a select * from systems_controller_type I get this output:
+----+-------------------------------+----------------+-----------------+
| id | controllerType | onInstructions | offInstructions |
+----+-------------------------------+----------------+-----------------+
| | |
| Plus | |
|ar | |
|ar exterior | |
|ird | |
| | |
| | |
| | |
| | |
| | |
| ex | |
| | |
| | |
| x 2 | |
| | |
|06 | |
| | |
| 06 | |
| | |
| | |
| | |
|el | |
|troller | |
| | |
|E-12 | |
|EC-9 | |
| | |
| | |
| | |
|nder (x2) | |
|- 9 station | |
| | |
|rollers | |
|Modular | |
| | |
| | |
|om Command (I think) | |
|lar | |
|mmand | |
| 04 04 04 | |
| | |
| | |
+----+-------------------------------+----------------+-----------------+
If I do a select for one record, I get this:
select * from systems_controller_type where id=3;
+----+----------------+----------------+-----------------+
| id | controllerType | onInstructions | offInstructions |
+----+----------------+----------------+-----------------+
|ar | |
+----+----------------+----------------+-----------------+
1 row in set (0.00 sec)
Any idea as to what is going on?? Did I do something wrong at the import stage?
try to use the following query... (without updating auto increment column - hope you would have set auto increment value while creating the table) ..
load data local infile '/home/garfonzo/controllers.csv' into table systems_controller_type fields terminated by ',' lines terminated by '\n' (#dummy, controllerType, onInstructions, offInstructions);