I have a table with all columns of type varchar, but the data are actually numbers, i.e. decimal(6,2) would be more appropriate. How can I change all columns from varchar to decimal if there are a lot of columns.
Thanks a lot in advance for your help.
You can change an individual column to decimal by using ALTER TABLE tablename MODIFY COLUMN columnname DECIMAL(6,2); . Any strings that can be converted to numbers will be converted, and others will be changed to zero.
If you want to be certain of doing this non-destructively, you could instead do ALTER TABLE ADD COLUMN to add a decimal column, and then UPDATE tablename SET decimalcolumn = textcolumn , and then use a SELECT to check for any rows where textcolumn and decimalcolumn aren't equal (it does type conversion as part of the comparison, so "5" and 5.00 are equal, as you'd want).
I don't know of a way to automatically apply the same conversion to multiple columns at once, though you could do it in PHP or another programming language by selecting a row from the table, looping over the columns that are returned, and running MODIFY for each one. If there are only a few columns, it's probably easier to do it by hand.
MySQL's ALTER TABLE statement supports changing multiple columns at once. In fact, doing as many changes to a table's schema as you can in one statement is preferred and highly recommended. This is because MySQL copies the whole table to do a schema change, but only does the copy once per ALTER TABLE statement. This is an important time saver when modifying a very large table!
That said, you can rehearse your changes in a couple of ways.
Firstly, I would use a development database to test all this, not a production one. You can then use CREATE TABLE ... LIKE ... to create a structurally identical table and then use INSERT INTO ... SELECT * FROM ... to copy the data. Now you can experiment with ALTER TABLE ... MODIFY COLUMN ... DECIMAL(6,2). If you do this on one column and get the message 0 Warnings, then that column will convert without incident and you can test the next. If you do get warnings, then SHOW WARNINGS will show a number of them so you know what problem MySQL encountered.
Depending on how well you know the data, you can also do a number of different SELECTs to find and filter it to see how much of it might be out of range or unconvertable (e.g. blank or text instead of numbers).
Yes, this approach will take some time, but once you're happy with this, you can assemble all the MODIFY COLUMN clauses into the one statement and run it on the real table.
Related
Have a large table that will be getting updated with new records daily or weekly, that has a lot of formatting issues that have to be resolved once it's loaded... trimming, removing characters, etc. One of them is changing all the empty strings to null, and right now I just have an update statement that has a CASE WHEN listed for every single column (and there are many). It seems so inefficient and clunky... is there a better way?
If you are allowed to use them, then use a before insert or before update trigger on that table. It will check if the new.field = '' and set its value to null. Repeat for all columns.
This is the best way to ensure your database integrity.
I have a table with two numeric values and I will query the difference between those two values over thousands of entries. I have two alternatives:
To query the difference directly like SELECT (column_1 - column_2) as 'DIFFERENCE' FROM 'Table_Name'
To have an UPDATE Trigger that automatically updates a fixed 'difference' column so I can then just query the difference column directly
As I mentioned before I will be querying potentially tens of thousands of entries so maybe option 1 would represent some kind of overload while option 2 will perform the subtraction only when necessary.
I am no expert on database performance optimization though, so maybe there's something I'm missing that somebody with more experience could point out.
Thanks in advance.
An alternative would be a generated column e.g.
ALTER TABLE table_name ADD difference GENERATED ALWAYS AS (column_1 - column_2)
You can choose to add the STORED keyword to the end of the ALTER TABLE command to ensure that the value is computed once (on INSERT and UPDATE operations), or you can omit it (or add the VIRTUAL keyword) to indicate that the column should be computed when read. Using VIRTUAL is like your option 1; STORED is like your option 2.
Here's a small generated column demo on dbfiddle.
If your version of MySQL/MariaDB does not have "generated columns", then don't bother pre-computing the difference. The cost and complication of a Trigger (or whatever) far exceeds simply recomputing the difference in every SELECT that needs it. GENERATED VIRTUAL is probably just a fancy way of doing the difference on the fly.
In general, do not worry about the cost of any expression or builtin function call. It is insignificant compared to the effort of locating the row, parsing it, etc.
I need to insert rows of data in a specific order. Sometimes I forget to insert the row on time and I have to insert it later. Other rows have taken up its place though and till now I manually (programmatically of course) change the index of different number of rows - it could be a couple of rows or hundreds of rows. This is not very efficient and I was looking for another way to go. My thought was to order by date and create a "day's index" number to reorder only the day's records but I was wandering... is there any mysql way to reorder the rows? That is to inform mysql about the required row position and then let it update the primary keys?
I think you need to look at your table design. This is actually a non-problem for most applications because it would have been addressed at the start.
Now, you need to add a DateTime column to your table, and initialise it with some sensible values for the data that's already there. As new rows are added, set the DateTime column in each new row to the actual DateTime. If you have to add a row late, set the DateTime to the time the record should have been added.
When you query your table, use ORDER BY myDateTime (or whatever you decide to call it). Your rows should appear in the correct order.
For small tables (less than a few thousand rows) an index might not help much. For larger tables you should index your DateTime column. You'd have to run some tests to see what works best.
What you think is actually the solution. Create a Date column if not already, and then Create Index on that field, also use Order by in your Query. There is no way other than manual, and even if there is it is not recommended to play with MYSQL way of storing rows, because row storage is done by DB Engine and it is not ideal to play with them, as they store row in best optimal way, so why mess their efficiency for such a small thing.
I have a table with one column whose encoding is cp1252 and collation is latin_swedish_ci, and I need to change it to utf8_general_ci.
I'd like to check if I'm not going to end up with weird characters in one of the rows due to the conversion.
This column stores domain names, and I'm unsure whether or not I have swedish characters in one of the rows.
I've been researching this but I haven't been able to find a way to check for data's integrity before changing the collection.
My best guess so far is to write a script to check if there's a column that doesn't contain any of the english alphabet characters, but I'm pretty sure that there's a better way to do this.
Any help would be great!
UPDATE
I've found multiple rows with garbage like this:
ÜZìp;ìê+ØeÞ{/e¼ðP;
Is there a way to ged rid of that junk without examining row per row?
The canonical way for this is to try it out:
Use SHOW CREATE TABLE to create an identically-structured testing table
Use INSERT INTO .. SELECT .. to populate the testing table with the primary key and relevant column(s) of the original
Try out conversion, noting necessary steps to fix problems
Rinse and repeat
Is there any easy way to find out all characters used in a specific column of a table in MySQL?
For example, these records:
"title"
"DP&E"
"UI/O"
"B,B#M"
All the characters used in the "title" column would be: DPEUIOBM&/#,
I'm not aware of any means to do this easily using MySQL. The best you'll be able to do is to test each potential character one by one with exists statements. This will be very slow, too, since it'll lead to reading your whole table as many times as there are characters that are not present.
If you've the possibility, create a temporary table that aggregates your needed data into a huge text field, dump it, and populate a compatible table in PostgreSQL. This will allow you to extract the needed data using a query that looks like this:
select distinct regexp_split_to_table(yourfield, '') as letter
from yourtable;
It'll still be very slow, but at least you'll go through the data a single time.