Is there any alternative to add new column to table that has 50 millions data?
I tried it with migration, and my database is down after trying to add 45 mins.
Is there any other solution to handle this operation?
This is not a good choice to alter a table which is populated with a large data set. As alternative you can always create a new table with the required column and make a relationship on that column to the table. It will be a effective way to do this.
Related
I have a table products that has over 50M records. I want to track who uploaded given product in my system but simply adding uploaded_by_id to such a huge table isn't the solutions I'm looking for. What else than a join table can I create to be able to query for products uploaded by given id in given time range?
Product.where(uploaded_by_id: #user.id, created_at: time_range) is what I need to do but I need more efficient way.
You might want to look into tools like
Soundscloud's Large Hadron Migrator or
Percona's pt-online-schema-change.
Both tools allow altering tables without locking them.
Instead of touching the main table, add another table (Vertical Partitioning). The new table would have the same PRIMARY KEY, but not AUTO_INCREMENT. The new column(s) would go into this table.
Create rows in the new table only when the new column(s) have a value.
When you don't need the new column(s), continue to read only from the old table.
When you also need the new column(s), use LEFT JOIN.
I am working with a existing database's table from another application. I want to query newly inserted records in a fixed interval.
Normally, in a table with AUTO INCREMENT id, I can store the last fetched id and use it in the query like WHERE id > :last_id. However, this table doesn't use AUTO INCREMENT id but use uuid as primary key. So is there any way to fetch new records only?
This DB is using MySQL. I can't change the database structure. The data size is quite huge so I don't think passing fetched uuids in query like WHERE uuid NOT IN (:fetch_uuids) will be a viable solution.
Edit:
There is created field, but unfortunately there is no warranty that the records with smaller created will be inserted first. So there is the risk of missing records using it.
The data were inserted by other application, and I only have read permission in this database.
Your question doesn't state whether there is a column containing the creation time of the record. If this exists then you could use this.
You have stated you cannot change the table structure, but are you sure you cannot add columns onto the existing structure? Your problem could be solved by adding an auto-increment 'secondary' ID and/or record creation timestamp. If you cannot modify the existing tables, could you perhaps create a new table with this additional information?
A solution to your problem may be in this answer. You may be able to either add an additional column to the existing table, or alternatively insert ids into a new table where you create an ID based on a TRIGGER from the original table
I want to create database table using mysql for transport application. Here I want to add columns which are not fixed for every record. They are added dynamically for every record. For example, record 1 contains PoliceFees & StateBoundry whereas record 2 does not have these fields. record 3 might have some others fields and so on. So how to design table for such data??
Dynamic fields and MySQL (relational database)? I think no-SQL is a better solution to your problem.
But if that fields is all known you can create a table with all of them and set as nullable. So you only insert needed data.
I have a list of records to be stored in database table, but I'm facing some difficulty in designing the database. The following would be the data to be stored:
The Class (Rows) and The Day (Column) will be continue to grow in future. My initial idea have 2 designs.
The table design for the database design will be exactly same with the current table. But the problem would be how if want to add Day13? It would be suffer in future in the column keep continue to grow.
Add 1 column as result:
It look better to solve the problem of Day column to be growing in future, but the problem is it will keep large amount of data records in database which make query become slower when more and more data insert.
Any idea or technique on how to optimize the database design? Thank you.
So a Class can have a result and a date. Just make sure to have a unique primary key on your Class table and make the correct data types for your fields.
What I think you need is a ClassId for primary key and then make a ClassName field (varchar) to store the class name. Don't write Day1 it should be a date format.
Maybe something simple like this.
One table for Classes (ClassID, ClassName)
One table for Day/Period/whatever you call it (DayID, DayName, DayInMonth etc..)
One table for Results(ResultId,DayId,ClassId, Result)
Is it possible to add new columns to an existing table without using alter statement?
Other people are answering unequivocally "no, it is not possible." This is the answer to your literal question. But I'm wondering why you ask the question.
One of the biggest pain points of MySQL is that using ALTER TABLE locks the table while you're making a change like adding a column, and the more data in your table, the longer this lasts while it restructures the table. I'm guessing this is the issue you have, and you're trying to get an alternative that doesn't block access to the table while you're adding a new column.
(In the future, it would help folks give you the best answers if you explain more about what you're trying to do.)
The answer to this question is yes, there is a solution: pt-online-schema-change is a free tool that accomplishes this.
You use it just like you would use ALTER TABLE, but you use it at the command-line instead of in an SQL query.
pt-online-schema-change --alter "ADD COLUMN c1 INT" D=sakila,t=actor
In this example, the database name is sakila and the table name is actor. The script does a lot of work behind the scenes:
Create a table like the original table, but empty of rows
ALTER TABLE to add the column or whatever other alteration you told it. You can do anything you would normally do with ALTER TABLE. In fact, it's doing ALTER TABLE for you, against the empty copy table.
Copy rows from the original table to the new table in the background.
Create triggers to capture any changes made to the original table while it's gradually copying the bulk of the data.
Swap the names of the new table (with the extra column) and the original table, once all data has been copied.
Drop the original table.
This has a few caveats, like the original table must have a primary key, and must not have existing triggers.
It tends to take longer than doing a traditional ALTER TABLE, but since it's not blocking access to the original table, it's still more convenient.
Does this help?
Is it possible to add new columns to an existing table without using the alter statement?
No.
Is it possible to add new columns to an existing table without using alter statement?
I don't think it's impossible.
However I'm not sure what you want to do.
lets say you have a table
select * from Store
and you want just export the data or perhaps you want to do something with that data like a selection. but you don't want to STORE the data in your Database
you can just fill a value and give it a name
select
'Test' as name,
*
from Store
this will populate your column with the value your entered.
data results