I have looked all over the MySQL website and found no definitive answers.
Is the size of the database determined by the operating system or is there 4GB limit?
Where can I find perforamnce statistics against other databases (sqlServer, oracle ect)
According to the MySQL Manual:
E.10.3. Limits on Table Size
The effective maximum table size for MySQL databases is usually determined by operating system constraints on file sizes, not by MySQL internal limits. The following table lists some examples of operating system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-to-date information, be sure to check the documentation specific to your operating system.
Operating System File-size Limit
Win32 w/ FAT/FAT32 2GB/4GB
Win32 w/ NTFS 2TB (possibly larger)
Linux 2.2-Intel 32-bit 2GB (LFS: 4GB)
Linux 2.4+ (using ext3 file system) 4TB
Solaris 9/10 16TB
MacOS X w/ HFS+ 2TB
Windows users, please note that FAT and VFAT (FAT32) are not considered suitable for production use with MySQL. Use NTFS instead.
On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS) patch for the ext2 file system. Most current Linux distributions are based on kernel 2.4 or higher and include all the required LFS patches. On Linux 2.4, patches also exist for ReiserFS to get support for big files (up to 2TB). With JFS and XFS, petabyte and larger files are possible on Linux.
As for the other part of your question, a few thoughts:
It's a broad, complex, multi-factorial question. Consider narrowing the scope of the question to MySQL and one other RDBMS (eg. SQL Server) and probably even one particular feature.
Google is your friend.
Vendors tend to publish their own biased comparisons. Take vendor numbers with a grain of salt.
1- With respect to database size, the limit is dependent on Operating System file size. Please see this article
2- The effective maximum table size for MySQL databases is usually determined by operating system constraints on file sizes, not by MySQL internal limits. (Source)
3- You may google for MySQL vs SQL Server Vs Oracle, also check this link
default mysql is 256 TB for myd file with 6 byte pointer size. i know this is ridiculous answer, but that is what you wanted to know. in real life all depends on the queries, indexes, column count, row count, etc.. i guess.
With today's hardware and OS, together with MySQL's preferred Engine InnoDB, the table size limitation is 64TB. With PARTITIONing, that can be stretched to over a hundred petabytes.
A database is a collection of tables. So, to answer the title question literally, we need to go beyond the max table size. Since there can be thousands of tables in a database, we are now into the exabyte stratosphere.
See also Hard limits in MySQL .
Related
I'm wondering if MySQL has any capability to specify that data belonging to a certain account (representing e.g., a particular app, or a particular corporate program) be stored at at some particular place in the filesystem (such as a particular drive or RAID), instead of bundling it inside the same physical file structure that is shared by every other account, table, and data element managed by MySQL for everybody on that server.
I'm aware that I can jigger MySQL to store its entire data bundle at a place other than the default place, but I was hoping there might be a way to do this by function, for "some data but not all data."
In MySQL 8.0, there are options to specify the location for each table or tablespace. See https://dev.mysql.com/doc/refman/8.0/en/innodb-create-table-external.html
In earlier versions of MySQL, these options didn't work consistently. You could specify the directory for individual table partitions, if your table was partitioned, but not for a non-partitioned table. Go figure. :-)
That said, I've never encountered a situation where it was worth the time to specify the physical location of tables. Basically, if your performance depends on the difference between one RAID filesystem vs. carefully choosing among different drives, you're already losing.
Instead, I've always done this approach:
Use one datadir on a fast RAID filesystem. Use the default configuration of all tables and logs under this datadir.
Allocate as much RAM as I can afford to the InnoDB buffer pool (up to the size of the database of course - no need to use more RAM than that). RAM is orders of magnitude faster than any disk, even an SSD. So you'd prefer to be reading data out of RAM.
If that's not enough performance, there are other things you can do to optimize, like creating indexes, or modifying the application code to do more caching to reduce database reads, or using a message queue to postpone database writes.
If that's still not enough performance, then scale out to multiple database servers. In other words, sharding.
Mysql 8.0 vs Mysql 5.7 performance comparison
At present in my production , I'm using mysql 5.7 version.
I want to upgrade my database to mysql8 .
I need a performance comparison among these three.
After searching Mysql5.7 vs Mysql 8.
My findings are the followings :
Creating Role
Invisible index
...etc
Insert operations are slower because of binary logging
and we can disable it.
Use slightly more RAM than 5.7 version
But what i really need is a comparison based on performance .
Connection handling, thread, pooling , max user , max connection, processing,CPU usage,memory something like this.
Thanks in advance.
To be honest, here's no clear cut answer for you. 90% of the performance is defined by database design and implementation, not the actual version of MySQL used.
The ability to handle load and stress is mostly the same between versions. Some things under the same configurations may be performing better on the newer version, some other things on older.
Most metric in your list (handling, thread, pooling, max user, max connection, processing, CPU usage, memory, etc) are usually fine tuneable on both my.cnf and OS sysctl level. Even kernel tweaks can have an effect.
So, in general, 90% of people can't see the performance difference. On the other hand, 90% also fail at doing proper performance tests too.
I have high end 48 CPU core server with latest MySQL 5.7 installed
I'm bit surprised that when executing queries with GROUP BY, no matter how I format my query I get only 1 CPU core used for that query, clearly illustrated by htop output
Of course I can perform multiple queries at the same time, and that's how I can use all the cores but it seems not too convenient and not every query can be split to use full server's power
Are there any MySQL extensions or SQL hints that allow to use multiple cores while processing data with GROUP BY?
As the accepted answer by RolandoMySQLDBA to Possible to make MySQL use more than one core? question says:
I actually discussed innodb_thread_concurrency with a MySQL Expert at
the Percona Live NYC conference back in May 2011.
I learned something surprising: In spite of the documentation, it is
best to leave innodb_thread_concurrency at 0 (infinite concurrency).
That way, InnoDB decides the best number of innodb_concurrency_tickets
to open for a given MySQL instance setup.
Once you set innodb_thread_concurrency to 0, you can set
innodb_read_io_threads and innodb_write_io_threads (both since MySQL
5.1.38) to the maximum value of 64. This should engage more cores.
This is the best guidance I ever found on how to make MySQL use more cores in general.
Aside from background threads, one connection will use only one CPU core. The applies to GROUP BY, UNION, PARTITION, and anything else you might think it should manage to do in parallel.
The statement applies for InnoDB at least through MySQL 8.0 and MariaDB 10.2.
A third-party software can make use of multiple cores: https://www.percona.com/blog/2014/05/01/parallel-query-mysql-shard-query/ .
Also, the "column store" in MariaDB 10.2 probably uses multiple cores.
This is a very known limitation of mySQL and thats why mysql is not commonly used for analytics purpose.
As a workaround, you can use Apache Spark (well known for analytics) in front of mysql, here:
Apache Spark will be the "SQL Engine", you can run it as a cluster of workers
mysql will be a simple data storage
I let you discovery how to use Apache Spark, and how to use the native JDBC connector.
Also, it seems this is possible also with PrestoDB (analytics engine from Facebook) (https://prestodb.github.io/docs/current/connector/mysql.html).
We know that MUMPS does not allow to create files bigger than 2 GB.
A Volume Group accepts 16GB, but only with 2 GB for each VG file.
How can I fix it?
Please, check the OS your implementation of MUMPS DB is installed on, in particular check the user setup (user's ulimit in case of Unix) I don't want to guess around since you did not specify any details about your implementation (M DB type and version, OS type and version).
MUMPS definitely allows to create files greater than 2GB. MUMPS Database region files are usually way over 20GB, MUMPS Database Journal Files are kept at 2GB for ease of maintenance, but could be greater, output files MUMPS system at hand may produce (in a batch on demand) can grow to whatever size the file system allows it.
To extent my understanding on MySQL.
1) Is going for 64bit help? Do I go with installation or stand alone?
2) If I am going to use 64-bit in MySQL Community Service, will it affect the MySQL Workbench which only available in 32-bit? If yes, in what sense?
3) Does the size of the file affect the speed of MySQL when doing calculation, analysis, and graph? I get a lot of big trouble using Excel in big size.
Please guide.
Note: I am working on excel database (which I found out on this website that it is not a good way to do it) with size of 1.34GB 63 files and growing. I would like to change and continue working in MySQL. I need the database for calculation, analysis and graph.
You are confusing a few concepts. But let me first answer your questions.
1/ 64-bit is more powerful than 32-bit, if your server supports 64-bit. So yes, it should "help".
2/ You can use a 64-bit server with a 32-bit client without any problems. They just communicate with eachother using MySQL-statements, bitsize makes no difference there.
3/ The size of the file (table/row/...) does matter. A lot of small tables can outperform one big table in certain cases, especially with multicore/raid/...