How do I add indexes to a database that is already created?
I'm a newbie to MYSQL and built a DB without any indexes in Django that is really really slow.
I'm trying to add indexes. The problem is that I need to filter by a foreign key. The query is painfully slow (think 20-25 seconds), even though the tables are not big (product = 1 million, another is 10k).
Here is my Django code:
#large db (Product) filtered by foreign key (another__id)
Product.objects.filter(another__id='x')
Should I put an index on the "id" in the another table or the "another" column in the product table?
Do I create an index like this:
CREATE INDEX "another" ON Product
Since this is on the server, what kind of lock should I do?
Related
I am using Talend Open Studio for data migration as I am upgrading my existing application architecture to a new one. I am using MySQL in both the applications but with different schema. I have migrated data successfully between single tables but while I am transferring data from a single table to a parent-child relationship table with a foreign key constraint, the data transfer is tremendously slow.
For e.g. I am migrating my Cities table to Cities and Citiesi18n and below is the schema for them:
My old schema :
CITIES (
id
city_name
status
created_at
)
The newly created schema where I need to migrate the data :
CITIES (
id
status
created_at
)
CITIESI18N (
id
lang_code
name
fk_city_id (// foreign key of cities table)
)
Below are the snapshots from my Talend jobs:
And here is the tmap configuration :
Now when I transfer the data without the foreign key the result are super fast. See below :
But the same when I transfer with a foreign key, my transfer is super slow :
(Note: I have taken province table for example as it is similar to cities table)
I think with Foreign key constraint it must be indexing the columns while transferring the data making it slower, but I am not sure. Is there any way I can fix this as I have a lot of tables similar to this which needs to be migrated. I am just curious to know the reason.
I don't know why you have this behaviour : you can try to redirect 'provience_i18n' to a tHashOutput (cache component), then link to a subjob with tHashInput (refering to your tHashOutput)-->tMySQLOutput. You'll have 2 subjobs, one for each insertion.
You are loading data to the parent & child at the same time. Using one tmap. When you are inserting foreign key in the second table, there is also insertion being made in the foreign/parent table. What you could alternatively do is: Load the data in the main CITIES table first, then onSubJobOk, load into child/CITIESI18N table. It would be faster. Let me know if it works.
I'm developing an Android application in which the data is stored in a SQLite database.
I have made sync with a MySQL database, in the web, to where I'm sending the data stored in the SQLite in the device.
The problem is that I don't know how to maintain the relations between tables, because the primary keys are going to be updated with AUTO_INCREMENT, and the foreign keys remain the same, breaking the relations between tables.
If this is a full migration, don't use auto increment during migration - create tables with normal columns. Use ALTER TABLE to change the model after import.
For incremental sync, the easiest way I see is additional column in each MySQL table called sqlite_id and filled with original id. Then you can update references using UPDATE (with joins).
Alternatives involve temporary tables for storing data and an auxiliary table used for pairing. Tedious for bigger data model.
The approach I tend to use, if possible, is to avoid auto increment in such situations. I have usaully an auxiliary table with four columns like this: t_import(tablename, operationid, sqlite_id, mysqlid).
Process is the following:
Import the primary keys into t_import. Use operationid to separate parallel imports if needed.
Generate new keys for data tables and store them into t_import table. This can be combined with step one.
Import the actual data and use t_import for setting new primary keys and restore relations.
That should work for most scenarios I know about.
Thanks or the help, you have given me some ideas.
I will try to add a id2 field to the tables that will store the same value as the primary key (_id)
When I send the information from SQLite to MySQL and the primary key is incremented I will have the id2 field with the original value of the primary key so that I can compare it with the foreign key of the other tables and update it.
Let’s see if it works.
Thanks
I've got a pretty big relational database and am working on the admin backend. I want to know which tables reference a column in one particular table.
For example: let's say I've got a table products with id as the primary index. I can have a lot of tables that reference this index, such as an orders table, a user_bookmarks table, and a product_reviews table. If I want to delete a particular product, I first need to do some work with the other tables--a simple "cascade" or "delete" directive wouldn't be enough. How do I get MySQL to tell me exactly which columns in which tables are referencing the products.id index?
Bonus question: is there a built-in way to get this info using phpmyadmin?
SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME
FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE (REFERENCED_TABLE_SCHEMA, REFERENCED_TABLE_NAME) = ('mydatabase', 'products')
Re your comment:
MySQL doesn't support recursive queries, so unfortunately there's no way to find the complete graph of foreign keys in a single query. The best you could do is for each of the results of the above query, run the query again, substituting the TABLE_SCHEMA and TABLE_NAME as the string constants in the WHERE clause (be careful not to enter into infinite loops if you have circular foreign keys).
Could you not read the Schema from the database into MySQL Workbench, and then use the tool to plot out the relationships between the tables? I have not tried this myself, as I do the design in MySQL Workbench and then the mappings, etc... and then export to MySQL to create the database.
I work with SQL Server and haven't had such an issue adding a FK to a table like I am having with MySQL. The datatypes match up, the foreign key table contains 300K rows, and is taking forever to execute and eventually times out. Am I doing something wrong? I did this in Workbench and Toad, same thing happens.
The relationship is a one to one, with the User table primary key ID being used a a FK for the District Admins table primary key ID.
I dont know how to find the MySQL version, and can't tell you the amount of memory.
Found out the issue was the table collations were different. My new tables were using latin1, but the existing were using UTF8. That is what I get for relying on Workbench to muchy.
Ihave been developing an app for some time. This involves entering and deleteing alot of useless data in the tables. Now that I want to go to production I want to get rid of all the data but also restore all the 'IDs' ( primary keys ) to 0 so that the live system can start fresh with sensible ID's like 1,2,3 etc.
Using MySQL and PHP / Codeigniter
Many Many Thanks for yoru help !
I would normally use TRUNCATE - this both removes the data and resets the AUTO_INCREMENT.
Note that MySQL will perform a row by row deletion if there is a foreign key relationship, which is quite convenient (compared to SQL Server).
If your pk is autoincrement, you can do
ALTER TABLE tbl AUTO_INCREMENT =1
Make sure table is empty before executing the query.