df1=
PRISM|7662 PRISM|7847 PRISM|7855
# Feature / Sample
Methanosphaera_stadtmanae 0.006728 0.000000 0.000000
Rothia_mucilaginosa 0.000000 0.000000 0.000000
Propionibacterium_freudenreichii 0.000000 0.000000 0.000000
Bifidobacterium_adolescentis 0.029346 0.000064 0.000006
Bifidobacterium_animalis 0.000000 0.023177 0.000000
... ... ... ...
Collinsella_stercoris 0.000000 0.000000 0.000000
Bacteroides_clarus 0.000000 0.000000 0.000000
Bacteroides_plebeius 0.000000 0.000000 0.000000
Lactobacillus_delbrueckii 0.000000 0.000000 0.000000
Lactobacillus_sakei 0.000000 0.000000 0.000000
df2=
Validation|LLDeep_0001 \ Validation|LLDeep_0003
# Feature / Sample
Methanosphaera_stadtmanae 0.000000 0.000000
Rothia_mucilaginosa 0.000000 0.000000
Propionibacterium_freudenreichii 0.000000 0.000000
Bifidobacterium_adolescentis 0.028835 0.000000
Bifidobacterium_animalis 0.000000 0.000000
Collinsella_stercoris 0.000000 0.001327
Bacteroides_clarus 0.000000 0.000000
Bacteroides_plebeius 0.000000 0.000142
Lactobacillus_delbrueckii 0.000000 0.000000
Lactobacillus_sakei 0.000000 0.112162
How to find correlation between them?
Related
We are running mariadb 10.3.25:
$ mysql --version
mysql Ver 15.1 Distrib 10.3.25-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
It seems that there is some sort of corruption in some of our databases’ tables.
Exhibit A:
MariaDB [etherpad]> select * from store;
ERROR 2013 (HY000): Lost connection to MySQL server during query
In the meantime, this happens in the log:
Jan 16 19:51:52 hostname mysqld[31236]: 2021-01-16 19:51:52 0x7f0c884b8700 InnoDB: Assertion failure in file /build/mariadb-10.3-RRxkin/mariadb-10.3-10.3.25/storage/innobase/row/row0sel.cc line 2972
Jan 16 19:51:52 hostname mysqld[31236]: InnoDB: Failing assertion: prebuilt->trx->isolation_level == TRX_ISO_READ_UNCOMMITTED
Jan 16 19:51:52 hostname mysqld[31236]: InnoDB: We intentionally generate a memory trap.
Jan 16 19:51:52 hostname mysqld[31236]: InnoDB: [...]
Jan 16 19:51:52 hostname mysqld[31236]: 210116 19:51:52 [ERROR] mysqld got signal 6 ;
Jan 16 19:51:52 hostname mysqld[31236]: This could be because you hit a bug. It is also possible that this binary
Jan 16 19:51:52 hostname mysqld[31236]: or one of the libraries it was linked against is corrupt, improperly built,
Jan 16 19:51:52 hostname mysqld[31236]: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 16 19:51:52 hostname mysqld[31236]: [...]
Jan 16 19:51:52 hostname mysqld[31236]: We will try our best to scrape up some info that will hopefully help
Jan 16 19:51:52 hostname mysqld[31236]: diagnose the problem, but since we have already crashed,
Jan 16 19:51:52 hostname mysqld[31236]: something is definitely wrong and this may fail.
Jan 16 19:51:52 hostname mysqld[31236]: Server version: 10.3.25-MariaDB-0+deb10u1-log
Jan 16 19:51:52 hostname mysqld[31236]: key_buffer_size=16777216
Jan 16 19:51:52 hostname mysqld[31236]: read_buffer_size=131072
Jan 16 19:51:52 hostname mysqld[31236]: key_buffer_size=16777216 [55/647]
Jan 16 19:51:52 hostname mysqld[31236]: read_buffer_size=131072
Jan 16 19:51:52 hostname mysqld[31236]: max_used_connections=16
Jan 16 19:51:52 hostname mysqld[31236]: max_threads=153
Jan 16 19:51:52 hostname mysqld[31236]: thread_count=22
Jan 16 19:51:52 hostname mysqld[31236]: It is possible that mysqld could use up to
Jan 16 19:51:52 hostname mysqld[31236]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 352736 K bytes of memory
Jan 16 19:51:52 hostname mysqld[31236]: Hope that's ok; if not, decrease some variables in the equation.
Jan 16 19:51:52 hostname mysqld[31236]: Thread pointer: 0x7f0c500093b8
Jan 16 19:51:52 hostname mysqld[31236]: Attempting backtrace. You can use the following information to find out
Jan 16 19:51:52 hostname mysqld[31236]: where mysqld died. If you see no messages after this, something went
Jan 16 19:51:52 hostname mysqld[31236]: terribly wrong...
Jan 16 19:51:52 hostname mysqld[31236]: stack_bottom = 0x7f0c884b7dd8 thread_stack 0x30000
Jan 16 19:51:52 hostname mysqld[31236]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x563337b2b05e]
Jan 16 19:51:52 hostname mysqld[31236]: /usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x56333765e09d]
Jan 16 19:51:53 hostname mysqld[31236]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f0c91ef1730]
Jan 16 19:51:53 hostname mysqld[31236]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x10b)[0x7f0c914ae7bb]
Jan 16 19:51:53 hostname mysqld[31236]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x121)[0x7f0c91499535]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(+0x4e3433)[0x5633373a2433]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(+0x4d5d6c)[0x563337394d6c]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(+0x9d8814)[0x563337897814]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(+0x9dcdcf)[0x56333789bdcf]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(+0x918681)[0x5633377d7681]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x127)[0x563337662db7]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_Z13rr_sequentialP11READ_RECORD+0x1c)[0x56333776a43c]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x1e3)[0x5633374bdf03]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0xaaa)[0x5633374e01ba]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_ZN4JOIN4execEv+0x33)[0x5633374e03d3]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xef)[0x5633374deaaf]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x14d)[0x5633374df38d]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(+0x5c1d8c)[0x563337480d8c]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5857)[0x56333748d087]
Jan 16 19:51:53 hostname mysqld[31236]: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x1c9)[0x56333748f879]
Jan 16 19:51:54 hostname mysqld[31236]: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x111d)[0x56333749172d]
Jan 16 19:51:54 hostname mysqld[31236]: /usr/sbin/mysqld(_Z10do_commandP3THD+0x122)[0x563337492e82]
Jan 16 19:51:54 hostname mysqld[31236]: /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x23a)[0x5633375641ba]
Jan 16 19:51:54 hostname mysqld[31236]: /usr/sbin/mysqld(handle_one_connection+0x3d)[0x56333756433d]
Jan 16 19:51:55 hostname mysqld[31236]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3)[0x7f0c91ee6fa3]
Jan 16 19:51:55 hostname mysqld[31236]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f0c915704cf]
Jan 16 19:51:55 hostname mysqld[31236]: Trying to get some variables.
Jan 16 19:51:55 hostname mysqld[31236]: Some pointers may be invalid and cause the dump to abort.
Jan 16 19:51:55 hostname mysqld[31236]: Query (0x7f0c50012e20): select * from store
Jan 16 19:51:55 hostname mysqld[31236]: Connection ID (thread ID): 733
Jan 16 19:51:55 hostname mysqld[31236]: Status: NOT_KILLED
Jan 16 19:51:55 hostname mysqld[31236]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,de
rived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_joi
n_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_d
erived=on,split_materialized=on
Jan 16 19:51:55 hostname mysqld[31236]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
Jan 16 19:51:55 hostname mysqld[31236]: information that should help you find out what is causing the crash.
Jan 16 19:51:55 hostname mysqld[31236]: Writing a core file...
Jan 16 19:51:55 hostname mysqld[31236]: Working directory at /var/lib/mysql
Jan 16 19:51:55 hostname mysqld[31236]: Resource Limits:
Jan 16 19:51:55 hostname mysqld[31236]: Limit Soft Limit Hard Limit Units
Jan 16 19:51:55 hostname mysqld[31236]: Max cpu time unlimited unlimited seconds
Jan 16 19:51:55 hostname mysqld[31236]: Max file size unlimited unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max data size unlimited unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max file size unlimited unlimited bytes [0/647]
Jan 16 19:51:55 hostname mysqld[31236]: Max data size unlimited unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max stack size 8388608 unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max core file size 0 unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max resident set unlimited unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max processes 15390 15390 processes
Jan 16 19:51:55 hostname mysqld[31236]: Max open files 65536 65536 files
Jan 16 19:51:55 hostname mysqld[31236]: Max locked memory 65536 65536 bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max address space unlimited unlimited bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max file locks unlimited unlimited locks
Jan 16 19:51:55 hostname mysqld[31236]: Max pending signals 15390 15390 signals
Jan 16 19:51:55 hostname mysqld[31236]: Max msgqueue size 819200 819200 bytes
Jan 16 19:51:55 hostname mysqld[31236]: Max nice priority 0 0
Jan 16 19:51:55 hostname mysqld[31236]: Max realtime priority 0 0
Jan 16 19:51:55 hostname mysqld[31236]: Max realtime timeout unlimited unlimited us
Jan 16 19:51:55 hostname mysqld[31236]: Core pattern: core
Jan 16 19:52:02 hostname mysqld[6672]: [... innodb crash recovery ...]
A very similar thing happens with some other tables as well.
What I tried:
I wanted to dump all data, purge the entire mariadb installation and restore. Unsurprisingly, mysqldump runs into the same corruption (?) and the database crashes during the dump.
I tried following a guide that advises to create a MyISAM table and fill that with data from the innodb table, but this fails for the same reason.
What can be done about this? Naturally, we need the data in these tables. It appears that once the query hits a certain record/block (I am oblivious to inner workings of mysql) it crashes the server. So how do we salvage the data?
UPDATE 2021-01-18 as requested, here are the variables and status queries:
MariaDB [(none)]> show global variables like '%thread%';
+-----------------------------------------+---------------------------+
| Variable_name | Value |
+-----------------------------------------+---------------------------+
| aria_repair_threads | 1 |
| binlog_optimize_thread_scheduling | ON |
| debug_no_thread_alarm | OFF |
| innodb_encryption_threads | 0 |
| innodb_purge_threads | 4 |
| innodb_read_io_threads | 4 |
| innodb_thread_concurrency | 0 |
| innodb_thread_sleep_delay | 10000 |
| innodb_write_io_threads | 4 |
| max_delayed_threads | 20 |
| max_insert_delayed_threads | 20 |
| myisam_repair_threads | 1 |
| performance_schema_max_thread_classes | 50 |
| performance_schema_max_thread_instances | -1 |
| slave_domain_parallel_threads | 0 |
| slave_parallel_threads | 0 |
| thread_cache_size | 8 |
| thread_concurrency | 10 |
| thread_handling | one-thread-per-connection |
| thread_pool_idle_timeout | 60 |
| thread_pool_max_threads | 65536 |
| thread_pool_oversubscribe | 3 |
| thread_pool_prio_kickup_timer | 1000 |
| thread_pool_priority | auto |
| thread_pool_size | 1 |
| thread_pool_stall_limit | 500 |
| thread_stack | 196608 |
| wsrep_slave_threads | 1 |
+-----------------------------------------+---------------------------+
28 rows in set (0.001 sec)
MariaDB [(none)]> show global status like '%thread%';
+------------------------------------------+-------+
| Variable_name | Value |
+------------------------------------------+-------+
| Delayed_insert_threads | 0 |
| Performance_schema_thread_classes_lost | 0 |
| Performance_schema_thread_instances_lost | 0 |
| Slow_launch_threads | 0 |
| Threadpool_idle_threads | 0 |
| Threadpool_threads | 0 |
| Threads_cached | 7 |
| Threads_connected | 12 |
| Threads_created | 98 |
| Threads_running | 6 |
| wsrep_applier_thread_count | 0 |
| wsrep_rollbacker_thread_count | 0 |
| wsrep_thread_count | 0 |
+------------------------------------------+-------+
13 rows in set (0.001 sec)
MariaDB [(none)]> show global variables like '%timeout%';
+---------------------------------------+----------+
| Variable_name | Value |
+---------------------------------------+----------+
| connect_timeout | 10 |
| deadlock_timeout_long | 50000000 |
| deadlock_timeout_short | 10000 |
| delayed_insert_timeout | 300 |
| idle_readonly_transaction_timeout | 0 |
| idle_transaction_timeout | 0 |
| idle_write_transaction_timeout | 0 |
| innodb_flush_log_at_timeout | 1 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 86400 |
| net_read_timeout | 600 |
| net_write_timeout | 600 |
| rpl_semi_sync_master_timeout | 10000 |
| rpl_semi_sync_slave_kill_conn_timeout | 5 |
| slave_net_timeout | 60 |
| thread_pool_idle_timeout | 60 |
| wait_timeout | 28800 |
+---------------------------------------+----------+
19 rows in set (0.001 sec)
MariaDB [(none)]> show global status like '%timeout%';
+-------------------------------------+-------+
| Variable_name | Value |
+-------------------------------------+-------+
| Binlog_group_commit_trigger_timeout | 0 |
| Master_gtid_wait_timeouts | 0 |
| Ssl_default_timeout | 0 |
| Ssl_session_cache_timeouts | 0 |
+-------------------------------------+-------+
4 rows in set (0.001 sec)
MariaDB [(none)]> show global status like '%aborted%';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| Aborted_clients | 3 |
| Aborted_connects | 0 |
+------------------+-------+
2 rows in set (0.001 sec)
The server has 5 GB of RAM.
About the store table:
MariaDB [etherpad]> show create table store;
+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| store | CREATE TABLE `store` (
`key` varchar(100) COLLATE utf8_bin NOT NULL DEFAULT '',
`value` longtext COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`key`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin |
+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.000 sec)
MariaDB [etherpad]> select count(*) from store;
+----------+
| count(*) |
+----------+
| 779443 |
+----------+
1 row in set (1 min 19.378 sec)
Here is the iostat info:
$ iostat -xm 5 3
Linux 4.14.0-0.bpo.3-amd64 (hostname) 01/18/2021 _x86_64_ (1 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
8.63 2.39 16.53 22.68 0.23 49.54
Device r/s w/s rMB/s wMB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
xvdap2 7.67 37.91 0.07 0.67 0.13 37.91 1.67 50.00 16.35 2.54 0.05 9.40 18.01 4.35 19.82
xvdap1 0.51 1.25 0.00 0.01 0.02 0.07 3.58 5.64 7.52 27.01 0.03 4.15 4.24 1.63 0.29
avg-cpu: %user %nice %system %iowait %steal %idle
18.51 2.21 15.49 55.33 0.40 8.05
Device r/s w/s rMB/s wMB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
xvdap2 4.00 157.80 0.02 1.53 0.00 71.00 0.00 31.03 5.80 55.33 7.93 4.00 9.92 4.37 70.72
xvdap1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
avg-cpu: %user %nice %system %iowait %steal %idle
8.96 2.44 15.68 15.27 0.41 57.23
Device r/s w/s rMB/s wMB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
xvdap2 0.00 22.20 0.00 0.40 0.00 35.40 0.00 61.46 0.00 22.81 0.30 0.00 18.27 4.11 9.12
xvdap1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
UPDATE 2021-01-24: I tried to pinpoint the problem by quasy-bisecting the table with limit clauses and found that out of the ~800,000 records, every query that selects records after 663,187 crashes the DB. The few records preceding the 663,187. one contain seemingly mangled data, see below.
MariaDB [etherpad]> select * from store limit 663184, 1\G;
*************************** 1. row ***************************
key:
value:
f[Y
f[팩
Does this not hint at data corruption? What could I do about the problem? Get rid of these records?
From the information available at this time, consider in your my.cnf [mysqld] section
innodb_buffer_pool_size=2G # to use 40% of available RAM
REMOVE thread_cache_size to allow default sizing (or set it to 256)
REMOVE thread_stack to allow default calc of slightly larger thread_stack per ref man
When we know number of cores/cpus we may be able to provide additional suggestions.
When more than 1 cpu's are available, additional suggestions could be provided.
Your query: select count(*) from store;
may complete in less than a minute if your tried
SELECT COUNT(key) FROM store;
to only cause reading of the index, rather than every row.
Have a GREAT 2021.
I write this query to retrieve data from 3 different tables.
Here [all columns i selected are from the first and the second table], data is retrieved as i actually need :
SELECT DISTINCT SD.salary_component,SS.posting_date,SS.payroll_entry,
SD.parentfield,SD.amount,SS.employee,SS.name
FROM`tabSalary Detail` SD
INNER JOIN `tabSalary Slip` SS
ON SS.name=SD.parent
INNER JOIN `tabGL Entry` GL
ON GL.against_voucher=SS.payroll_entry
WHERE SS.employee='EMP0110' AND
SS.posting_date='2020-03-11' AND
SD.parentfield != 'indebtedness' AND
SS.payroll_entry='HR-PRUN-2019-01027';
the desired and the query result is
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04
=======
But , when i selected two column in the third table something goes wrong and data is duplicated :
SELECT DISTINCT SD.salary_component,SS.posting_date,SS.payroll_entry,
SD.parentfield,SD.amount,SS.employee,SS.name
GL.credit,GL.debit
FROM`tabSalary Detail` SD
INNER JOIN `tabSalary Slip` SS
ON SS.name=SD.parent
INNER JOIN `tabGL Entry` GL
ON GL.against_voucher=SS.payroll_entry
WHERE SS.employee='EMP0110' AND
SS.posting_date='2020-03-11' AND
SD.parentfield != 'indebtedness' AND
SS.payroll_entry='HR-PRUN-2019-01027';
the second query result is 54 duplicated rows
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 300.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 300.000000 0.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 500.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 500.000000 0.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 3600.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 3600.000000 0.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 445.890000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 445.890000 0.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 336.990000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 336.990000 0.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 95.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 95.000000 0.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 1150.000000
AAAA 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 1150.000000 0.000000
===================================================================================================================================
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 300.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 300.000000 0.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 500.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 500.000000 0.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 3600.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 3600.000000 0.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 445.890000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 445.890000 0.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 336.990000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 336.990000 0.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 95.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 95.000000 0.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 1150.000000
BBBB 2020-03-11 HR-PRUN-2019-01027 earnings 300.000000 EMP0110 Sal Slip-EMP0110-1911-04 1150.000000 0.000000
===================================================================================================================================
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 300.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 300.000000 0.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 500.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 500.000000 0.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 3600.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 3600.000000 0.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 445.890000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 445.890000 0.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 336.990000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 336.990000 0.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 95.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 95.000000 0.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 1150.000000
CCCC 2020-03-11 HR-PRUN-2019-01027 earnings 500.000000 EMP0110 Sal Slip-EMP0110-1911-04 1150.000000 0.000000
===================================================================================================================================
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 300.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 300.000000 0.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 500.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 500.000000 0.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 3600.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 3600.000000 0.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 445.890000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 445.890000 0.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 336.990000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 336.990000 0.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 95.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 95.000000 0.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 0.000000 1150.000000
DDDD 2020-03-11 HR-PRUN-2019-01027 earnings 3600.000000 EMP0110 Sal Slip-EMP0110-1911-04 1150.000000 0.000000
what is the wrong and what topics should i search to get the idea ?
finally , what is the best solution ?
I strongly suspect that you want GROUP BY:
SELECT SD.salary_component, SS.posting_date, SS.payroll_entry,
SD.parentfield, SD.amount, SS.employee, SS.name,
SUM(GL.credit), SUM(GL.debit)
FROM `tabSalary Detail` SD INNER JOIN
`tabSalary Slip` SS
ON SS.name = SD.parent INNER JOIN
`tabGL Entry` GL
ON GL.against_voucher=SS.payroll_entry
WHERE SS.employee = 'EMP0110' AND
SS.posting_date = '2020-03-11' AND
SD.parentfield <> 'indebtedness' AND
SS.payroll_entry = 'HR-PRUN-2019-01027'
GROUP BY SD.salary_component, SS.posting_date, SS.payroll_entry,
SD.parentfield, SD.amount, SS.employee, SS.name;
SELECT DISTINCT SD.salary_component,SS.posting_date,SS.payroll_entry,
SD.parentfield,SD.amount,SS.employee,SS.name
SUM(GL.credit) AS credit,SUM(GL.debit) AS debit
FROM`tabSalary Detail` SD
INNER JOIN `tabSalary Slip` SS
ON SS.name=SD.parent
INNER JOIN `tabGL Entry` GL
ON GL.against_voucher=SS.payroll_entry
WHERE SS.employee='EMP0110' AND
SS.posting_date='2020-03-11' AND
SD.parentfield != 'indebtedness' AND
SS.payroll_entry='HR-PRUN-2019-01027'
GROUP BY SD.salary_component, SS.posting_date, SS.payroll_entry,
SD.parentfield, SD.amount, SS.employee, SS.name;
I agree that even a couple of rows of sample data showing the duplicates would be very helpful.
Typically what is happening is that the JOIN to the third table
(ON GL.against_voucher=SS.payroll_entry)
combined with the lack of having any "GL." criteria in the WHERE clause is not specific enough to eliminate the creation of duplicates.
Thus having DISTINCT and/or GROUP BY will still produce duplicates in some fields even though the row is distinct.
If you provide a GOOD example of some duplicates then the syntax that will work will be easier to answer.
I am trying to name the nodes/faces in my obj file. So that after translation when I open the file in forge viewer I should be able to see the same names in model browser. I tried using 'o' parameter for object naming in my .obj file but still, I am seeing names like Node 1, Node 2 .... and so on.
This is my .obj file :
mtllib Classification.mat
v 0.000000 2.000000 2.000000
v 0.000000 0.000000 2.000000
v 2.000000 0.000000 2.000000
v 2.000000 2.000000 2.000000
v 0.000000 2.000000 0.000000
v 0.000000 0.000000 0.000000
v 2.000000 0.000000 0.000000
v 2.000000 2.000000 0.000000
# 8 vertices
usemtl Cone
o Cone
f 1 2 3 4
usemtl Cylinder
o Cylinder
f 8 7 6 5
usemtl plane
o plane
f 4 3 7 8
usemtl Sphere
o Sphere
f 5 1 4 8
usemtl Torus
o Torus
f 5 6 2 1
usemtl BSpline
o BSpline
f 2 6 7 3
# 6 elements
This is my Material file (Classification.mat):
newmtl Cone
Ka 1.00 1.00 1.00
Kd 0.3333333333333333 1.0 1.0
Killum 1
newmtl Cylinder
Ka 1.00 1.00 1.00
Kd 1.0 1.0 0.4980392156862745
Killum 1
newmtl plane
Ka 1.00 1.00 1.00
Kd 0.3333333333333333 0.6666666666666666 1.0
Killum 1
newmtl Sphere
Ka 1.00 1.00 1.00
Kd 1.0, 0.6666666666666666, 0.0
Killum 1
newmtl Torus
Ka 1.00 1.00 1.00
Kd 1.0, 0.3333333333333333, 1.0
Killum 1
newmtl BSpline
Ka 1.00 1.00 1.00
Kd 1.0, 0.0, 0.0
Killum 1
I am using a .zip for translation containing these two files. It would really great if anyone could help me out here ....
Thank You,
Unfortunately, it does look like the translator ignores object and group names at the moment when creating the SVF for the Viewer from the OBJ file 🙁
I logged this behaviour as "OBJ >> SVF does not take into account object or group name", but not sure yet when this might be improved.
Sorry for the bad news.
My question is "Can the tex1Dfetch function be used with pitched linear memory"?
Section B.8.1.1.tex1dfetch() says "fetches from the region of linear memory....".
I am optimizing a kernel which is bound by global memory reads. It uses a grid-stride loop to load a float4:
float4 x = XYZW[i]; // float4 const * const XYZW
float4 x = tex1Dfetch<float4<( XYZW, i ); // cudaTextureObject_t XYZW
float4 x = tex1D<float4<( XYZW, i ); // cudaTextureObject_t XYZW
The first example, using pointer argument, works fine. The tex1Dfetch form returns all zeros. The tex1D call returns aliased/psychadelic garbage.
In all cases, cudaMallocPitched is used. The texture loads profiled faster, with tex1Dfetch the fastest. So I'm keen to get it working correctly.
My question is "Can the tex1Dfetch function be used with pitched linear memory"?
In C/C++, a pointer is simply a bare number. Once you allocate a region, the returned pointer itself isn't conceptually any different whether it was returned by cudaMalloc or cudaMallocPitch; it is the address of a location in device memory.
tex1Dfetch can be made to work with either type of allocation (cudaMalloc or cudaMallocPitch). The actual values you get will depend on how you handle the pitch regions in the cudaMallocPitch case, and tex1Dfetch won't do anything "automatic" there for you. So it seems not very sensible (to me) to use tex1Dfetch with a pitched allocation, but it can be made to "work", ie. fetch the correct value.
Here is a simple demonstrator of this claim, based on slight modification to the code here:
$ cat t434.cu
#include <stdio.h>
#define N 32
#define M 128
// texture object is a kernel argument
__global__ void kernel(cudaTextureObject_t tex) {
int i = blockIdx.x *blockDim.x + threadIdx.x;
float x = tex1Dfetch<float>(tex, i);
if (i < 256) printf("%d %f\n", i, x);
}
void call_kernel(cudaTextureObject_t tex) {
dim3 block(128,1,1);
dim3 grid((N*M)/block.x,1,1);
kernel <<<grid, block>>>(tex);
}
int main() {
// declare and allocate memory
float *buffer, *h_buffer;
size_t pitch;
cudaMallocPitch(&buffer, &pitch, N*sizeof(float),M);
printf("pitch = %lu\n", pitch);
cudaMemset(buffer, 0, M*pitch);
h_buffer=(float *)malloc(N*M*sizeof(float));
for (int i = 0; i < M; i++)
for (int j = 0; j < N; j++) h_buffer[i*N+j] = i+1;
cudaMemcpy2D(buffer, pitch, h_buffer, N*sizeof(float), N*sizeof(float), M, cudaMemcpyHostToDevice);
// create texture object
cudaResourceDesc resDesc;
memset(&resDesc, 0, sizeof(resDesc));
resDesc.resType = cudaResourceTypeLinear;
resDesc.res.linear.devPtr = buffer;
resDesc.res.linear.desc.f = cudaChannelFormatKindFloat;
resDesc.res.linear.desc.x = 32; // bits per channel
resDesc.res.linear.sizeInBytes = M*pitch*sizeof(float);
cudaTextureDesc texDesc;
memset(&texDesc, 0, sizeof(texDesc));
texDesc.readMode = cudaReadModeElementType;
// create texture object: we only have to do this once!
cudaTextureObject_t tex=0;
cudaCreateTextureObject(&tex, &resDesc, &texDesc, NULL);
call_kernel(tex); // pass texture as argument
// destroy texture object
cudaDestroyTextureObject(tex);
cudaFree(buffer);
}
$ nvcc -arch=sm_61 -o t434 t434.cu
$ cuda-memcheck ./t434
========= CUDA-MEMCHECK
pitch = 512
0 1.000000
1 1.000000
2 1.000000
3 1.000000
4 1.000000
5 1.000000
6 1.000000
7 1.000000
8 1.000000
9 1.000000
10 1.000000
11 1.000000
12 1.000000
13 1.000000
14 1.000000
15 1.000000
16 1.000000
17 1.000000
18 1.000000
19 1.000000
20 1.000000
21 1.000000
22 1.000000
23 1.000000
24 1.000000
25 1.000000
26 1.000000
27 1.000000
28 1.000000
29 1.000000
30 1.000000
31 1.000000
32 0.000000
33 0.000000
34 0.000000
35 0.000000
36 0.000000
37 0.000000
38 0.000000
39 0.000000
40 0.000000
41 0.000000
42 0.000000
43 0.000000
44 0.000000
45 0.000000
46 0.000000
47 0.000000
48 0.000000
49 0.000000
50 0.000000
51 0.000000
52 0.000000
53 0.000000
54 0.000000
55 0.000000
56 0.000000
57 0.000000
58 0.000000
59 0.000000
60 0.000000
61 0.000000
62 0.000000
63 0.000000
96 0.000000
97 0.000000
98 0.000000
99 0.000000
100 0.000000
101 0.000000
102 0.000000
103 0.000000
104 0.000000
105 0.000000
106 0.000000
107 0.000000
108 0.000000
109 0.000000
110 0.000000
111 0.000000
112 0.000000
113 0.000000
114 0.000000
115 0.000000
116 0.000000
117 0.000000
118 0.000000
119 0.000000
120 0.000000
121 0.000000
122 0.000000
123 0.000000
124 0.000000
125 0.000000
126 0.000000
127 0.000000
64 0.000000
65 0.000000
66 0.000000
67 0.000000
68 0.000000
69 0.000000
70 0.000000
71 0.000000
72 0.000000
73 0.000000
74 0.000000
75 0.000000
76 0.000000
77 0.000000
78 0.000000
79 0.000000
80 0.000000
81 0.000000
82 0.000000
83 0.000000
84 0.000000
85 0.000000
86 0.000000
87 0.000000
88 0.000000
89 0.000000
90 0.000000
91 0.000000
92 0.000000
93 0.000000
94 0.000000
95 0.000000
128 2.000000
129 2.000000
130 2.000000
131 2.000000
132 2.000000
133 2.000000
134 2.000000
135 2.000000
136 2.000000
137 2.000000
138 2.000000
139 2.000000
140 2.000000
141 2.000000
142 2.000000
143 2.000000
144 2.000000
145 2.000000
146 2.000000
147 2.000000
148 2.000000
149 2.000000
150 2.000000
151 2.000000
152 2.000000
153 2.000000
154 2.000000
155 2.000000
156 2.000000
157 2.000000
158 2.000000
159 2.000000
192 0.000000
193 0.000000
194 0.000000
195 0.000000
196 0.000000
197 0.000000
198 0.000000
199 0.000000
200 0.000000
201 0.000000
202 0.000000
203 0.000000
204 0.000000
205 0.000000
206 0.000000
207 0.000000
208 0.000000
209 0.000000
210 0.000000
211 0.000000
212 0.000000
213 0.000000
214 0.000000
215 0.000000
216 0.000000
217 0.000000
218 0.000000
219 0.000000
220 0.000000
221 0.000000
222 0.000000
223 0.000000
160 0.000000
161 0.000000
162 0.000000
163 0.000000
164 0.000000
165 0.000000
166 0.000000
167 0.000000
168 0.000000
169 0.000000
170 0.000000
171 0.000000
172 0.000000
173 0.000000
174 0.000000
175 0.000000
176 0.000000
177 0.000000
178 0.000000
179 0.000000
180 0.000000
181 0.000000
182 0.000000
183 0.000000
184 0.000000
185 0.000000
186 0.000000
187 0.000000
188 0.000000
189 0.000000
190 0.000000
191 0.000000
224 0.000000
225 0.000000
226 0.000000
227 0.000000
228 0.000000
229 0.000000
230 0.000000
231 0.000000
232 0.000000
233 0.000000
234 0.000000
235 0.000000
236 0.000000
237 0.000000
238 0.000000
239 0.000000
240 0.000000
241 0.000000
242 0.000000
243 0.000000
244 0.000000
245 0.000000
246 0.000000
247 0.000000
248 0.000000
249 0.000000
250 0.000000
251 0.000000
252 0.000000
253 0.000000
254 0.000000
255 0.000000
========= ERROR SUMMARY: 0 errors
We see that the allocation pitch is 512 bytes. This corresponds to a total pitched width of 128 float quantities per row. However the 2D allocation only specifies 32 elements per row (N). We initialize a 2D region with 32 elements per row, where each value is the row index plus 1. The remaining "pitch" regions are initialized to zero. We observe in the output that the validly defined row 0 elements properly print out as 1, the validly defined row 1 elements properly print out as 2, and all other elements print out as zero, because we are fetching from the "pitch" region.
I'm using InnoDB in MySQL (0.5GB database) on Debian 6.0.
This morning (after 120 days of perfect uptime), suddenly the server hung (in fact the server hung a second time in the last two days) and I had to reboot it through hoster CP.
In kern.log, I've got this:
Apr 16 09:43:35 lhost kernel: [76680.208104] INFO: task mysqld:9755 blocked for more than 120 seconds.
Apr 16 09:43:35 lhost kernel: [76680.233913] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 16 09:43:35 lhost kernel: [76680.234249] mysqld D ffff88007dc67170 0 9755 1241 0x00000000
Apr 16 09:43:35 lhost kernel: [76680.234472] ffff88007dc67170 0000000000000086 0000000000000000 ffffffffa021c4c2
Apr 16 09:43:35 lhost kernel: [76680.234815] ffff880009ff1ad8 ffff880009ff1ad8 000000000000f9e0 ffff880009ff1fd8
Apr 16 09:43:35 lhost kernel: [76680.235181] 0000000000015780 0000000000015780 ffff88007d177170 ffff88007d177468
Apr 16 09:43:35 lhost kernel: [76680.235549] Call Trace:
Apr 16 09:43:35 lhost kernel: [76680.235732] [<ffffffffa021c4c2>] ? ext4_mark_iloc_dirty+0x46e/0x4f7 [ext4]
Apr 16 09:43:35 lhost kernel: [76680.235963] [<ffffffffa021cd61>] ? ext4_mark_inode_dirty+0x18f/0x1c4 [ext4]
Apr 16 09:43:35 lhost kernel: [76680.236212] [<ffffffff812fc2ab>] ? __mutex_lock_common+0x122/0x192
Apr 16 09:43:35 lhost kernel: [76680.236422] [<ffffffff812fc3d3>] ? mutex_lock+0x1a/0x31
Apr 16 09:43:35 lhost kernel: [76680.236626] [<ffffffff81113c8c>] ? __blockdev_direct_IO+0x1d3/0xa60
Apr 16 09:43:35 lhost kernel: [76680.236852] [<ffffffff810b51c2>] ? generic_file_buffered_write+0x1f5/0x278
Apr 16 09:43:35 lhost kernel: [76680.237097] [<ffffffffa021da26>] ? ext4_ind_direct_IO+0x10e/0x1be [ext4]
Apr 16 09:43:35 lhost kernel: [76680.237320] [<ffffffffa02205c9>] ? ext4_get_block+0x0/0xe2 [ext4]
Apr 16 09:43:35 lhost kernel: [76680.237528] [<ffffffff810b57bf>] ? generic_file_aio_read+0xf6/0x536
Apr 16 09:43:35 lhost kernel: [76680.237794] [<ffffffff81071dbf>] ? wake_futex+0x31/0x4e
Apr 16 09:43:35 lhost kernel: [76680.237984] [<ffffffff810ef061>] ? do_sync_read+0xce/0x113
Apr 16 09:43:35 lhost kernel: [76680.238278] [<ffffffff8100f79c>] ? __switch_to+0x285/0x297
Apr 16 09:43:35 lhost kernel: [76680.238471] [<ffffffff81065046>] ? autoremove_wake_function+0x0/0x2e
Apr 16 09:43:35 lhost kernel: [76680.238687] [<ffffffff81048282>] ? finish_task_switch+0x3a/0xaf
Apr 16 09:43:35 lhost kernel: [76680.238897] [<ffffffff810efa84>] ? vfs_read+0xa6/0xff
Apr 16 09:43:35 lhost kernel: [76680.239090] [<ffffffff810efb34>] ? sys_pread64+0x57/0x77
Apr 16 09:43:35 lhost kernel: [76680.239290] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
And just before complete blackout:
Apr 16 09:45:32 lhost kernel: [76800.253284] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
Apr 16 09:56:29 lhost kernel: [77453.643191] Clocksource tsc unstable (delta = -788398275 ns)
Apr 16 10:01:01 lhost kernel: imklog 4.6.4, log source = /proc/kmsg started.
So I decided to turn the slow queries log on, and this is what was logged:
# Time: 130416 13:38:41
# User#Host: lhost[lhost] # localhost []
# Query_time: 2.059521 Lock_time: 0.000026 Rows_sent: 0 Rows_examined: 1
use lhost;
SET timestamp=1366105121;
UPDATE t_fuser SET DATE_UPDATE=now() WHERE ID = 992277;
# Time: 130416 13:56:42
# User#Host: root[root] # localhost []
# Query_time: 3.663760 Lock_time: 0.000037 Rows_sent: 230 Rows_examined: 230
SET timestamp=1366106202;
SHOW TABLE STATUS FROM `lhost`;
# User#Host: root[root] # localhost []
# Query_time: 3.693291 Lock_time: 0.000031 Rows_sent: 230 Rows_examined: 230
SET timestamp=1366106202;
SHOW TABLE STATUS FROM `lhost`;
# Time: 130416 14:00:19
# User#Host: lhost[lhost] # localhost []
# Query_time: 7.139369 Lock_time: 0.000210 Rows_sent: 122 Rows_examined: 3027170
This looks really weird to me. I've tried to execute similar queries (there's more complicated ones on this log, but that's not the point) directly through the console and got results much faster, so it's not indexes.
What can I do to address this? Is it hardware?
# free -m
total used free shared buffers cached
Mem: 2012 1355 657 0 40 627
-/+ buffers/cache: 687 1325
Swap: 2047 129 1918
Thank in advance for your advice.
Additional data:
PROFILE of slow query:
mysql> show profile;
+--------------------------------+----------+
| Status | Duration |
+--------------------------------+----------+
| starting | 0.000020 |
| checking query cache for query | 0.000111 |
| checking permissions | 0.000003 |
| checking permissions | 0.000001 |
| checking permissions | 0.000002 |
| checking permissions | 0.000001 |
| checking permissions | 0.000003 |
| Opening tables | 0.000032 |
| System lock | 0.000004 |
| Table lock | 0.000006 |
| init | 0.000074 |
| optimizing | 0.000020 |
| statistics | 0.000171 |
| preparing | 0.000027 |
| Creating tmp table | 0.000160 |
| executing | 0.000003 |
| Copying to tmp table | 4.572503 |
| Creating sort index | 0.009622 |
| Copying to group table | 0.029744 |
| Removing duplicates | 0.010745 |
| Sorting result | 0.000086 |
| Sending data | 0.004350 |
| end | 0.000008 |
| removing tmp table | 0.001207 |
| end | 0.000005 |
| removing tmp table | 0.000040 |
| end | 0.000004 |
| query end | 0.000003 |
| freeing items | 0.000018 |
| removing tmp table | 0.000005 |
| freeing items | 0.002768 |
| logging slow query | 0.000004 |
| logging slow query | 0.000029 |
| cleaning up | 0.000006 |
+--------------------------------+----------+
34 rows in set (0.00 sec)
First try to execute fsck and then check IO performance: dd if=/dev/zero of=/var/lib/mysql/file1 bs=512 count=10000