I have a many-to-many relationship database with 3 tables. It's very slow to load the data into the tables, especially the join table. Several hours for 3 millions rows.
I was suggested to create the tables first without creating index. I am using Hibernate. If I don't annotate index in classes, then what's the best time and way to add index? Should I do it directly on MySql database using SQL statement? Or the index should be added somewhere in Hibernate, without affecting loading performance?
You should add indexes directly to MySQL database using CREATE INDEX statement.
If you have very big table you can use pt-online-schema-change to prevent blocking your application
Related
I'm experimenting with various indexing settings for my mysql database.
I wonder though, by removing or adding indexes is there any possibility to damage data rows in any way? Obviously I realise that if I make any application queries fail, that can cause bad rows. I'm more talking just about the structural queries themselves.
Or will I simply affect the efficiency of the database?
I just want to know if I have safety to experiment or if I have to be cautious?
The data isn't in phpmyadmin, it's in mysql. Adding/removing an index will not affect your data integrity by default. With a unique index, and using the ignore keyword it can.
That said - you should always have a backup of your data, it's easy to run a test like:
CREATE TABLE t1 LIKE t;
INSERT INTO t1 SELECT * FROM t;
ALTER TABLE t1 CREATE INDEX ...;
Then compare the difference in tables (perhaps a COUNT is fine in your case).
Adding/removing indexes is safe in terms of the rows in your table. However as you note, too many indexes or poorly constructed indexes can be (very) detrimental to performance. Likewise, adding indexes on large tables can be a very expensive process, and can bring a MySQL server to its knees, so you're better off not "experimenting" on production tables.
I'm using MySQL MyISAM and I have 7 tables in my database linked by a primary key called ID. I want to PARTITION the data on one of these tables by its timestamp. When I want to delete a partition, I'd like to delete all records on the other tables with the same ID as the ones I deleted from the partition.
Can this be done at a similar speed as dropping a partition? I don't particularly want to go to each table and search for the right ID to delete as it would defeat the purpose of partitioning in the first place.
Cannot achieve what you want as you stated it.
Are all the tables the same size? Or perhaps the other tables are "normalization" tables? In that case, just leave the data there?
Please elaborate on what kind of data you have and the relationships (1:1, 1:many, etc) between the tables.
You can create triggers for those tables:
http://dev.mysql.com/doc/refman/5.7/en/create-trigger.html
We have a very huge Mysql table which is MyISAM. Whenever we run optimize table command, the table is locked and performance is getting impacted. The table is not read only and hence creating temporary tables and swapping them may not work out. We are not able to partition the table also.
Is there any other way/tool to achieve optimize table functionality without degrading the performance. Any suggestion would be of great help.
Thanks in advance.
http://dev.mysql.com/doc/refman/5.5/en/optimize-table.html
For InnoDB tables, OPTIMIZE TABLE is mapped to ALTER TABLE, which
rebuilds the table (...)
Therefore, I would not expect any improvement in switching to InnoDB, as Quassnoi probably suggests.
By definition, OPTIMIZE TABLE needs some exclusive access to the table, hence the degraded performances during OPTIMIZE'ation
Nevertheless, there could be some steps to take to reduce the time taken by OPTIMIZE, depending on how your table is "huge" :
if your table has many fields, your table might need to be normalized. Conversely, you might want to de-normalize your table by spreading your columns into several "narrower" tables, and establish one-to-one relations.
if your table has many records, implement a "manual" partitionning in your application code. A simple step would be to create an "archive" table that holds rarely updated records. This way you only need to optimize a smaller set of records (the non-archive table).
optimize table command lock the table,it decrease the performance.
you download percona tool kit command to optimize table.
this command not lock the table during optimize table.
use below link :
https://www.percona.com/doc/percona-toolkit/2.1/pt-online-schema-change.html
Here is a screenshot of mysql explain command on a common query:
http://cl.ly/3r34251M320A1P2s3e1Y
I have 3 different tables I have to join together to extract the data I want. This is the main CI model code using activerecord
$this->db->select('articles.id, articles.title, articles.link, articles.updated_time, articles.created_time, shows.network,shows.show_id, shows.name');
$this->db->from('articles')->order_by('updated_time','desc')->offset($offset)->limit($limit);
$this->db->join('labels', 'articles.remote_id = labels.articleid');
$this->db->join('shows', 'shows.show_id = labels.showid');
Can anyone suggest any ways to improve the schema or query performance?
Thanks
What makes your query slower is mysql use of temporary tables and filesort which means it can't efficiently use the index and creates a temporary table (many times on disk!).
It usually happen when you join a table using one index and sort it by another index or use a condition on another index.
First thing you can do is read about this issue and see if you can, at least, avoid the use of temporary disk tables.
How mysql uses temp tables
MySQL docs say that tables that use the MERGE storage engine can only union underlying MyISAM tables, which don't allow use of transactions.
Is there an alternative or workaround so that I can have a table that contains the data of several transactional tables in MySQL?
Also, MySQL 4... I know, I know, but it's what I'm stuck with.
Perhaps you could use a view to accomplish this. I'm not too sure if you need the full insert, update, delete functionality or if you just want to select from many tables.