Very Slow MySQL Performance on NAS - mysql

Just got myself a Asustor NAS to handle my videos, pictures, music and etc.
Having both a Desktop and a Notebook in the house, I figured it would be a good idea to setup my database on the NAS (which already comes with MariaDB preinstalled).
The Setup: RAID 1, max read speads of about 110MB/s from the disk, connected via 1.3 Mbps WiFi with gigabit connection. Getting about 60MB/s using BlackMagic Benchmark.
The query:
SELECT items.title, items.discount, items.qtd, items.price, ((price * qtd) - discount) AS total, DATE_FORMAT(orders.created_at, '%m-%y')
FROM items
INNER JOIN orders ON orders.order_id = items.order_id
ORDER BY created_at;
The table orders has about 1.8k rows, the table items has about 4.7k rows. The query affects 5k rows and takes between 4.8 to 7.0 seconds to run, which seems absurd for such a simple query. I used to run the same query in my localhost (ok, it is a NVMe SSD, which I get is a lot faster), on milliseconds. order_id is a VARCHAR with about 10 characters in it.
It took about 7 (9 last time) minutes to insert all the data in all tables:
`orders` - 1.7k rows, 11 columns
`items` - 4.8k rows, 12 columns
`customers` - 1.7k rows, 9 columns
My question:
Is it really that bad of a performance, or am I getting the wrong performance benchmark after having used NVMe SSD's?
If it is bad indeed, what can I do to improve it (still hosting my DB on my NAS)?
What could I expect performance-wise on an online hosted database?
Thanks a lot.
**Tables:**
`CREATE TABLE `orders` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`order_id` varchar(15) DEFAULT NULL COMMENT 'VA',
`created_at` datetime DEFAULT NULL,
`gateway` varchar(25) DEFAULT NULL,
`total` decimal(15,0) DEFAULT NULL,
`subtotal` decimal(15,0) DEFAULT NULL,
`status` varchar(20) DEFAULT NULL,
`discounts` decimal(15,0) DEFAULT NULL,
`total_price` decimal(15,0) DEFAULT NULL,
`order_number` varchar(15) DEFAULT NULL,
`processing` varchar(15) DEFAULT NULL,
`customer_id` varchar(15) DEFAULT NULL,
`number` varchar(15) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `number` (`number`),
UNIQUE KEY `order_id` (`order_id`),
KEY `customer_id` (`customer_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1712 DEFAULT CHARSET=utf8;`

Is it really that bad of a performance, or am I getting the wrong performance benchmark after having used NVMe SSD's?
Yes this is kind of bad performance. Correct indexing for the query will go part way in solving the performance problem of the query. Getting to get the NAS to use innodb_buffer_pool and free memory as disk cache is going to be difficult with only 512M on board.
If it is bad indeed, what can I do to improve it (still hosting my DB on my NAS)?
Correct indexing of tables with help the join and order. Design changes to use integer primary keys for joins. As a first step, if order_id really isn't utf8 and is just latin1 changing for that column that would make the key smaller, could change this to the primary key too.
As this is an entire two tables data search in the query, its only going to eliminate IO latency if it can all stay in RAM.
What could I expect performance-wise on an online hosted database?
Hosted database will offer more RAM, and probably a faster CPU.

My NAS mysql version is Mariadb 10.3.29 , blow change work for me , you can try it!
/volume1/#appstore/MariaDB10/usr/local/mariadb10/etc/mysql/my.cnf
Change
#innodb_flush_log_at_trx_commit = 1
TO
innodb_flush_log_at_trx_commit = 0
Restart MySQL
----OR----
Login MySQL
SET GLOBAL innodb_flush_log_at_trx_commit=0;
You can change it on runtime and check the different!

Related

How can I make this basic MySQL SELECT query faster on 1 billion rows?

I have a large 100GB dataset of website content that's around 1 billion rows that I've indexed into a single MySQL 8 table.
CREATE TABLE `inner_pages_content` (
`id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`page_id` INT(10) UNSIGNED NULL DEFAULT NULL,
`page_word` VARCHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`word_source` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`word_frequency` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
PRIMARY KEY (`id`) USING BTREE,
INDEX `FK_inner_pages_content_inner_pages` (`page_id`) USING BTREE,
INDEX `page_word` (`page_word`) USING BTREE,
CONSTRAINT `FK_inner_pages_content_inner_pages` FOREIGN KEY (`page_id`) REFERENCES `rocket_test`.`inner_pages` (`id`) ON UPDATE CASCADE ON DELETE CASCADE
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB
AUTO_INCREMENT=1629374935
I'm then running queries similar to this against it to build up relevancy scores about specific pages against specific keywords.
SELECT page_id, word_source, word_frequency FROM inner_pages_content WHERE page_word IN
('docker', 'software development', 'linux', 'microsoft', 'computer software');
Everything is working well apart from it being painfully slow. One solution I've found is to move everything into GCP's BigQuery but before I pursue that route permanently and write off CloudSQL/MySQL I'd like to see if there is anything I can do to keep my solution using CloudSQL/MySQL.
Both locally on my SSD and on CloudSQL with 500GB SSD these queries are taking 120-200 seconds to complete.
I believe one issue (strong chance I'm wrong!) is that I'm not putting the entire table in memory locally or in CloudSQL due to the costs of 100GB of memory so it's always pulling the results from the disk.
Interestingly when I COUNT(*) the matching results rather than SELECT it's very quick. Typically under 0.5 seconds.
Is there anything I can do to make this query quicker?
Additional information
You don't have a index / query issue (because of the count(*) quick time.
It's IO / network issue, so you can try to add selected fields into the index to put data in memory :
https://mariadb.com/kb/en/building-the-best-index-for-a-given-select/
part : "Covering" indexes
create index page_word on inner_pages_content (page_word , page_id, word_source, word_frequency)

MySQL - just adding ORDER BY an indexed field adds 5 minutes for just 52 records. Where to start?

EDIT 2: now that we have optimized the db and narrowed down in MySQL - Why is phpMyAdmin extremely slow with this query that is super fast in php/mysqli?
EDIT 1: there are two solutions that helped us. One on database level (configuration) and one on query level. I could of course only accept one as the best answer, but if you are having similar problems, look at both.
We have a database that has been running perfectly fine for years. However, right now, we have a problem that I don't understand. Is it a mysql/InnoDB configuration problem? And we currently have nobody for system maintenance (I am a programmer).
The tabel TitelDaggegevens is a few Gigs in size, about 12,000,000 records, so nothing extraordinary.
If we do:
SELECT *
FROM TitelDaggegevens
WHERE fondskosten IS NULL
AND (datum BETWEEN 20200401 AND 20200430)
it runs fine, within a few tenths of a second.
The result: 52 records.
Also if we add ORDER BY datum or if we order by any other non-indexed field: all is well, same speed.
However, if I add ORDER BY id (id being the primary key), suddenly the query takes 15 seconds for the same 52 records.
And when I ORDER BY another indexed field, the query-time increases tot 4-6 minutes. For ordering 52 records. On an indexed field.
I have no clue what is going on. EXPLAIN doesn't help me. I optimized/recreated the table, checked it, and restarted the server. All to no avail. I am absolutely no expert on configuring MySQL or InnoDB, so I have no clue where to start the search.
I am just hoping that maybe someone recognises this and can point me into the right direction.
SHOW TABLE STATUS WHERE Name = 'TitelDaggegevens'
Gives me:
I know this is a very vague problem, but I am not able to pin it down more specifically. I enabled the logging for slow queries but the table slow_log stays empty. I'm lost.
Thank you for any ideas where to look.
This might be a help to someone who knows something about it, but not really to me, phpmyadmins 'Advisor':
In the comments and a reaction were asked for EXPLAIN outputs:
1) Without ORDER BY and with ORDER BY datum (which is in the WHERE and has an index):
2) With ORDER BY plus any field other than datum (indexed or not, so the same for both quick and slow queries).
The table structure:
CREATE TABLE `TitelDaggegevens` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`isbn` decimal(13,0) NOT NULL,
`datum` date NOT NULL,
`volgendeDatum` date DEFAULT NULL,
`prijs` decimal(8,2) DEFAULT NULL,
`prijsExclLaag` decimal(8,2) DEFAULT NULL,
`prijsExclHoog` decimal(8,2) DEFAULT NULL,
`stadiumDienstverlening` char(2) COLLATE utf8mb4_unicode_520_ci DEFAULT NULL,
`stadiumLevenscyclus` char(1) COLLATE utf8mb4_unicode_520_ci DEFAULT NULL,
`gewicht` double(7,3) DEFAULT NULL,
`volume` double(7,3) DEFAULT NULL,
`24uurs` tinyint(1) DEFAULT NULL,
`UitgeverCode` varchar(4) COLLATE utf8mb4_unicode_520_ci DEFAULT NULL,
`imprintId` int(11) DEFAULT NULL,
`distributievormId` tinyint(4) DEFAULT NULL,
`boeksoort` char(1) COLLATE utf8mb4_unicode_520_ci DEFAULT NULL,
`publishingStatus` tinyint(4) DEFAULT NULL,
`productAvailability` tinyint(4) DEFAULT NULL,
`voorraadAlles` mediumint(8) unsigned DEFAULT NULL,
`voorraadBeschikbaar` mediumint(8) unsigned DEFAULT NULL,
`voorraadGeblokkeerdEigenaar` smallint(5) unsigned DEFAULT NULL,
`voorraadGeblokkeerdCB` smallint(5) unsigned DEFAULT NULL,
`voorraadGereserveerd` smallint(5) unsigned DEFAULT NULL,
`fondskosten` enum('depot leverbaar','depot onleverbaar','POD','BOV','eBoek','geen') COLLATE utf8mb4_unicode_520_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `ISBN+datum` (`isbn`,`datum`) USING BTREE,
KEY `UitgeverCode` (`UitgeverCode`),
KEY `Imprint` (`imprintId`),
KEY `VolgendeDatum` (`volgendeDatum`),
KEY `Index op voorraad om maxima snel te vinden` (`isbn`,`voorraadAlles`) USING BTREE,
KEY `fondskosten` (`fondskosten`),
KEY `Datum+isbn+fondskosten` (`datum`,`isbn`,`fondskosten`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=16519430 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci
Have this to handle the WHERE entirely:
INDEX(fondskosten, Datum)
Note: the = is first, then the range.
Fetch the *. Note: If there are big TEXT or BLOB columns that you don't need, spell out the SELECT list so you can avoid them. They may be stored "off-record", hence take longer to fetch.
An optional ORDER BY. If it is on Datum, then there is no extra effort. If it is on any other column, then there will be a sort. But a sort of 52 rows will be quite fast (milliseconds).
Notes:
If you don't have fondskosten IS NULL or you have some other test, then all bets are off. We have to start over in designing the optimal composite index.
USE/FORCE INDEX -- use this as a last resort.
Always provide SHOW CREATE TABLE when needing to discuss a query.
The Advisor has some good stuff, but without any clues of what is "too big", it is rather useless.
I suspect all the other discussions failed to realize that there are far more than 52 rows for the given Datum range. That is fondskosten IS NULL is really part of the problem and solution.
For people searching for tweaks in similar cases, these are the tweaks the specialist made to the db that sped it up considerably (mind you this is for a database with 100s of tables and MANY very complex and large queries sometimes joining over 15 tables but not super massive number of records. The database is only 37 gigabytes.
[mysqld]
innodb_buffer_pool_size=2G
innodb_buffer_pool_instances=4
innodb_flush_log_at_trx_commit=2
tmp_table_size=64M
max_heap_table_size=64M
join_buffer_size=4M
sort_buffer_size=8M
optimizer_search_depth=5
The optimizer_search_depth was DECREASED to minimize the time the optimizer needs for the complex queries.
After restarting the server, (regularly) run all queries that are the result of running this query:
SELECT CONCAT('OPTIMIZE TABLE `', TABLE_SCHEMA , '`.`', TABLE_NAME ,'`;') AS query
FROM INFORMATION_SCHEMA.TABLES
WHERE DATA_FREE/DATA_LENGTH > 2 AND DATA_LENGTH > 4*1024*1024
(This first one better when the server is off line or has low use if you have large tables. It rebuilds and thus optimizes the tables that need it.)
And then:
SELECT CONCAT('ANALYZE TABLE `', TABLE_SCHEMA , '`.`', TABLE_NAME ,'`;') AS query
FROM INFORMATION_SCHEMA.TABLES
WHERE DATA_FREE/DATA_LENGTH > 2 AND DATA_LENGTH > 1*1024*1024
(This second querie-series is much lighter and less infringing but may still help speed up some queries by recalculating query strategies by the server.)
Looks like ORDER BY uses 3 different optimization plans
ORDER BY id - Extra: Using index condition; Using where; Using filesort. MySQL uses filesort to resolve the ORDER BY. But rows are sorted already. So, it takes 15 second.
ORDER BY Datum or other non-indexed field - Extra: Using index condition; Using where. MySQL uses Datum index to resolve the ORDER BY. It takes few seconds.
ORDER BY index_field - Extra: Using index condition; Using where; Using filesort. MySQL uses filesort to resolve the ORDER BY. Rows are unsorted. It takes few minutes.
It's my suggestion. Only EXPLAIN can tells what's going on
Influencing ORDER BY Optimization
UPD:
Could you check this query with every ORDER BY clauses?
SELECT *
FROM TitelDaggegevens USE INDEX FOR ORDER BY (Datum)
WHERE fondskosten IS NULL
AND (Datum BETWEEN 20200401 AND 20200430)
Also you may try to increasing the sort_buffer_size
If you see many Sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization or improved indexing.
On Linux, there are thresholds of 256KB and 2MB where larger values may significantly slow down memory allocation, so you should consider staying below one of those values.

MySQL GROUP BY on large tables

I have a table with over 75 millions registers. I want to run a group by to summarize this registries.
The table structure is:
CREATE TABLE `output_medicos_full` (
`name` varchar(100) NOT NULL DEFAULT '',
`term` varchar(50) NOT NULL DEFAULT '',
`hash` varchar(40) NOT NULL DEFAULT '',
`url` varchar(2000) DEFAULT NULL,
PRIMARY KEY (`name`,`term`,`hash`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I want execute the query bellow, but is taking so long using a dedicate mysql server 5.5 with 4GB RAM:
INSERT INTO TABLE report
SELECT
`hash`
,CASE UPPER(SUBSTRING_INDEX(url, ':', 1))
WHEN 'HTTP' THEN 1
WHEN 'HTTPS' THEN 2
WHEN 'FTP' THEN 3
WHEN 'FTPS' THEN 4
ELSE 0 end
,url
FROM output_medicos_full
GROUP BY `hash`;
On table report there is an unique index on hash column
Any help to speed it up?
Thank's
The main cost here is all the I/O. The entire table needs to be read.
innodb_buffer_pool_size = 2G is dangerously high for 4GB of RAM. If swapping occurs, performance will suffer terribly.
Since the hash is a SHA1, it is extremely likely to be unique across a mere 75M urls. So that GROUP BY will yield 75M rows. This is probably not what you wanted. Once you rewrite the query, we can discuss optimizations.

Slow MySQL InnoDB Inserts and Updates

I am using magento and having a lot of slowness on the site. There is very, very light load on the server. I have verified cpu, disk i/o, and memory is light- less than 30% of available at all times. APC caching is enabled- I am using new relic to monitor the server and the issue is very clearly insert/updates.
I have isolated the slowness to all insert and update statements. SELECT is fast. Very simple insert / updates into tables take 2-3 seconds whether run from my application or the command line mysql.
Example:
UPDATE `index_process` SET `status` = 'working', `started_at` = '2012-02-10 19:08:31' WHERE (process_id='8');
This table has 9 rows, a primary key, and 1 index on it.
The slowness occurs with all insert / updates. I have run mysqltuner and everything looks good. Also, changed innodb_flush_log_at_trx_commit to 2.
The activity on this server is very light- it's a dv box with 1 GB RAM. I have magento installs that run 100x better with 5x the load on a similar setup.
I started logging all queries over 2 seconds and it seems to be all inserts and full text searches.
Anyone have suggestions?
Here is table structure:
CREATE TABLE IF NOT EXISTS `index_process` (
`process_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`indexer_code` varchar(32) NOT NULL,
`status` enum('pending','working','require_reindex') NOT NULL DEFAULT 'pending',
`started_at` datetime DEFAULT NULL,
`ended_at` datetime DEFAULT NULL,
`mode` enum('real_time','manual') NOT NULL DEFAULT 'real_time',
PRIMARY KEY (`process_id`),
UNIQUE KEY `IDX_CODE` (`indexer_code`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=10 ;
First: (process_id='8') - '8' is char/varchar, not int, so mysql convert value first.
On my system, I had long times (greater than one second) to update users.last_active_time.
The reason was that I had a few queries that long to perform. As I joined them for the users table. This resulted in blocking of the table to read. Death lock by SELECT.
I rewrote query from: JOIN to: sub-queries and porblem gone.

Optimizing mySQL database on large table with frequent writes and fulltext columns?

My team runs a shopping information site and as we've started to grow, we're starting to experience issues with query response time on our product table impacting display speed.
The main issue we experience is when a user "saves" an item triggering an update query as other users are searching on the FULLTEXT indexes that exist as well:
UPDATE product SET listed = listed+1 WHERE product_id = XX
For example, I just ran the update in 0.01 seconds with no other queries hitting but a few minutes ago, with a large FULLTEXT request also going, the same request took 23 seconds.
I assume this is because the table is MYISAM and can't do row-level locks.
Our product table contains over 3.5 million records and will double by the end of the month. After that it should level off to 2-5% monthly increases.
CREATE TABLE product (
product_id INT UNSIGNED NOT NULL AUTO_INCREMENT,
category_id INT UNSIGNED NOT NULL DEFAULT '0',
title VARCHAR (100) NOT NULL DEFAULT '',
short_desc VARCHAR (255) NOT NULL DEFAULT '',
description TEXT NOT NULL,
msrp DECIMAL (6,2) NOT NULL DEFAULT '000.00',
rating DECIMAL(3,2) NOT NULL DEFAULT '0.0',
reviews INT UNSIGNED NOT NULL DEFAULT '0',
listed INT UNSIGNED NOT NULL DEFAULT '0',
sku VARCHAR(75) NOT NULL DEFAULT '0',
upc VARCHAR(20) NOT NULL DEFAULT '0',
updateddate DATETIME NOT NULL,
PRIMARY KEY (product_id),
KEY title (title),
KEY category_id (category_id),
KEY listed (listed),
KEY mfrg_id (mfrg_id),
KEY identifier (identifier),
FULLTEXT INDEX (title),
FULLTEXT INDEX (description)
) ENGINE = MYISAM;
The database runs on a dedicated server that only hosts our site. We are planning to move the database into a replication structure with a Dual Proc, 16gb RAM server for the query box [slave] and the current "web" server handling the writes [dual proc, 4gb ram].
I'm not DB expert [clearly] and from researching have become warry of running InnoDB at the same time as MYISAM [replication & backup implications?] but it does seem like splitting the product table to house main information [innodb] and the fulltext descriptions [myisam] seperately may help dramatically?
If you have an idea and need more info please comment and I will provide back more details.
Thank you
You're exact about MyIsam. It's table-lock instead of row-lock like innoDB.
So from what you say, the query (full text) is taking a lot of time, so the update need to wait before being done.
You should really consider to switch to innoDB ( easy solution ) or switch your full text search to somewhere else like solr , elastic search, sphinx.
Also, you should check your slow_query log and optimize all those query.