I removed, what at the time I thought, was an unnecessary field/column from a table in mysql and now would like to import it back into the table. Just before I dropped the column, I did a backup of the whole table. Since then, close to 1000 new rows have been added.
How can I add the column and information back to the table?
I have a sandbox that I can "play" with. I have tried but have yet to get the proper end result. Thank you in advance.
Not sure if it matters but, the system is fedora 16.
What I would do is to load your dump into another table (e.g. table_bak, you might need to do this in your sandbox and redump). Then add your column back in to your live table with a sensible default value. Then you should be able to bring the old data back like this
UPDATE `table`,`table_bak` SET `table`.restored_column=`table_bak`.restored_column WHERE `table`.pk_id=`table_bak`.pk_id
Related
Every week I import a table of our clients into MySQL and every week a random client is missing from that list. That same client might have been imported without problems the week before and then next week again imported perfectly fine, however MySQL will always omit one client from the import which makes my job very frustrating in trying to find who it was and manually fix it.
Has anyone run into similar issues in the past or have any idea why they might be happening?
I am using the MySQL table import wizard. Normally I don't have issues, but this specific client table export I get from another platform is having this issue every single week.
I would do the import from the Excel into a TEMPORARY (ie: not the final table), but one of the same structure. Then, once that is done and the import of all names and columns are parsed, you can left-join from LIVE table to the TEMP table where the entry exists on LIVE but not in TEMP. This would help find the missing import. THEN, since the data is already in a valid table structure, you can pull in all new values from the temp table to the production, including adding any new that did not yet exist.
If you are exporting to excel using the Export wizard, it creates a sheet with the name of the table, but also creates a named range with the same name. The named range then contains only the region with data. Then when you add new data, this named region is not extended. And when you are later importing the data, you probably chose the name without the $ character on end. The name without the $ in the name represents the named region, which probably doesn't contain your newly inserted data. To load all the date chose the name with $ character or extent the named region to cover all the data in the sheet.
today I come to you for inspiration or maybe ideas how to solve a task not killing my laptop with massive and repetitive code.
I have a CSV file with around 10k records. I also have a database with respective records in it. I have four fields inside both of these structures: destination, countryCode,prefix and cost
Every time I update a database with this .csv file I have to check if the record with given destination, countryCode and prefix exist and if so, I have to update the cost. That is pretty easy and it works fine.
But here comes the tricky part: there is a possibility that the destination may be deleted from one .csv file to another and I need to be aware of that and delete that unused record from the database. What is the most efficient way of handling that kind of situation?
I really wouldn't want to check every record from the database with every row in a .csv file: that sounds like a very bad idea.
I was thinking about some time_stamp or just a bool variable which will tell me if the record was modified during the last update of the DB BUT: there is also a chance that neither of params within the record change, thus: no need to touch that record and mark it as modified.
For that task, I use Python 3 and mysql.connector lib.
Any ideas and advice will be appreciated :)
If you're keeping a time stamp why do you care if it's updated even if nothing was changed in the record? If the reason is that you want to save the date of the latest update you can add another column saving a time stamp of the last time the record appeared in the csv and afterwords delete all the records that the value of this column in them is smaller than the date of the last csv.
If the .CSV is a replacement for the existing table:
CREATE TABLE new LIKE real;
load the .csv into `new` (Probably use LOAD DATA...)
RENAME TABLE real TO old, new TO real;
DROP TABLE old;
If you have good reason to keep the old table and patch it, then...
load the .csv into a table
add suitable indexes
do one SQL to do deletes (no loop needed). It is probably a multi-table DELETE.
do one sql to update the prices (no loop needed). It is probably a multi-table UPDATE.
You can probably do the entire task (either way) without touching Python.
As the title says...
How do I determine the time an entry was made into a mysql table without adding a new column? I realize that I could add a table.created TIMESTAMP column but I'd rather not do this. I'm using MySQL 5.1
I don't think you can do that. If you could, then timestamp columns would be unnecessary.
Why the reluctance to use a column?
Well, you first need to figure out where you want this data to be stored. mySql doesn't just automatically track when rows are created or updated, so that means it's up to you to store it.
Your first option is to store it in the database. This means altering your table and adding a new column, or storing it elsewhere in the database. If you want to store the information in another table, you have to modify the code that does the insert to also log the data - or use a TRIGGER to automatically log the data.
If you don't want to store the data in the database, you could perhaps use a logging library to write the information to an event log or file. You'd have to modify the code that does the insert to also log this data through that mechanism.
Hope this helps.
I currently have a table of 3m records that needs updating nightly.
The data that populates this table comes from ~100 APIs that all get normalised into one jumbo table.
Problem:
How to reflect new records being added, and records being deleted at the source?
Facts:
I can't truncate the table every night and reinsert.
Each API provides a constant ID for each record (so I can keep track of what's what).
Some fields will be updated each night.
Solutions:
New records are easy, I just add them to my table with an AvailableFrom date.
Updates are also easy, for each record I check if it exists and if data has changed (performance will suck).
Deleted records are where I'm stuck.
The APIs just dump me a load of data, how do I tell if a record has "dropped off"?
I'm thinking a swap table of some sort - any ideas?
If the only way to tell whether a record has been deleted is to check whether the api delivers it any more without knowing what record exactly you are looking for you will need to keep track on the iports. If you always do a full import:
Solution 1:
set a flag for for every row in database, then do the import and update the flag for every row you get, then delete everything that hasn't been updated.
Solution 2:
Set an import ID (bound to the date?) for every import and write it to the database entries. so you know which row originates from which import. Overriding existing data with the import id from the latest import.
Then you can work only with the data from the last import.
but if you always do a full import, dropping everything before should be faster shouldnt it?
I have a table that has 7000 rows,
I added a new column to this table
The table has a mysql DateTime so.
When i updated the table to fill in this new table it updated the datetime,
I took an sql dump just before i did the update so now i need to use the sql dump to revert the datetime back (and only that column).
How do i do that?
There are a couple ways I can think of to do this off the top of my head.
First is to create another mysql database and load the dump into that database (make sure it's not going to load into the first database from a use commmand in the dump), and then use the data from that database to construct the update queries for the first.
The second, easier, more hackish way, is to open the dump in a text editor, pull out just that table, and find and replace to make update statements for just that column based on primary key instead of inserts. You'd need to be able to find and replace on patterns.
A third way would be to load the dump in an abstract sql tool letting it do the parsing for you, and write new queries from the data in the abstract syntax trees.
A fourth, again hackish, possibility, if this isn't a live system, is to rollback and re-perform the more recent transformations (only if they are simple).
Restore the dump to a second table. Select the ID and datetime from that table. Use those results to update the rows in the original table corresponding to the IDs you got.