I'm trying to convert PostgreSQL database to MySQL with MySQL Workbench tool. It seems that schema migration works fine - there are no errors, but when I'm trying to migrate data - I get huge list of errors and it looks like they all about the same thing - Postgres boolean field is fetched as extremely big integer number that can't be saved as TINYINT in MySQL:
ERROR: `mydb`.`my_table`:Range error fetching field 9 (value 140406775873536, target is MYSQL_TYPE_TINY)
Is there any way to fix that?
OK, I have a solution now.
Reason:
By default when migrate data from Postgres to MySQL using the wizard in MySQL Workbench, if the Postgres field is boolean, it will create TINYINT(1) column in MySQL, that's why we meet this error.
Solution:
alter the boolean field to INT
ALTER TABLEtargetdb.sample_tableCHANGE COLUMNfieldfieldINT NULL DEFAULT '1' COMMENT '' ;
use MySQL Migration wizard will succeed.
alter the field back to TINYINT(1).
If one of you guys has to do it on a large scale i suggest you take a look at this answer in order to generate a bulk script:
bulk changing column types in MySQL
or just check the script below :
SELECT
CONCAT('ALTER TABLE ',
TABLE_NAME,
' CHANGE COLUMN ',
COLUMN_NAME,
' ',
column_name,
' TARGET_TYPE ',
CASE
WHEN IS_NULLABLE = 'NO' THEN ' NOT '
ELSE ''
END,
' NULL;') AS que
FROM
information_schema.columns
WHERE
table_schema = 'MY DB'
AND data_type = 'SOURCE_TYPE';
Related
I maintain Beekeeper Studio, an open source SQL GUI.
I have a problem with information_schema.columns in MySQL 5.7.
when running:
select * from information_schema.columns where table_name = ?`
It returns the COLUMN_DEFAULT, but it doesn't seem to differentiate between variables (eg CURRENT_TIMESTAMP), and a string 'foo'.
Is there a place I can get the 'real' default value with appropriate quoting?
What I expect:
CURRENT_TIMESTAMP
'foo'
Unfortunately, there's no metadata in the INFORMATION_SCHEMA that can tell you this in MySQL 5.7. You just have to know that for a DATETIME or TIMESTAMP column, CURRENT_TIMESTAMP is a special case.
It is treated as a special case in the code, so if you use SHOW CREATE TABLE, it outputs the literal string "CURRENT_TIMESTAMP" instead of a quoted datetime value.
https://github.com/mysql/mysql-server/blob/5.7/sql/sql_show.cc#L1402-L1411
I have about 12 databases, each with 50 tables and most of them with 30+ columns; the db was running in strict mode as OFF, but now we had to migrate the db to cleardb service which by default has strict mode as ON.
all the tables that had "Not Null" constraint, the inserts have stopped working, just because the default values are not being passed; while in case of strict mode as OFF if the value are not provided, the MYSQL will presume the default value of the column datatype.
Is there a script I can use to get the metadata about all the columns of all tables and generate a script to alter all the tables with such columns to change the default to "Null"
You should consider using the information_schema tables to generate DDL statements to alter the tables. This kind of query will get you the list of offending columns.
SELECT CONCAT_WS('.',TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME) col
FROM information_schema.COLUMNS
WHERE IS_NULLABLE = 0
AND LENGTH(COLUMN_DEFAULT) = 0
AND TABLE_SCHEMA IN ('db1', 'db2', 'db3')
You can do similar things to generate ALTER statements to change the tables. But beware, MySQL likes to rewrite tables when you alter certain things. It might take time.
DO NOT attempt to UPDATE the information_schema directly!
You could try changing the strict_mode setting when you connect to the SaaS service, so your software will work compatibly.
This is a large project and is probably important business for cleardb. Why not ask them for help in changing the strict_mode setting?
This is what I came up with on the base of #Ollie-jones script
https://gist.github.com/brijrajsingh/efd3c273440dfebcb99a62119af2ecd5
SELECT CONCAT_WS('.',TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME) col,CONCAT('alter table ',TABLE_NAME,' MODIFY COLUMN ', COLUMN_NAME,' ',DATA_TYPE,'(',CHARACTER_MAXIMUM_LENGTH,') NULL DEFAULT NULL') as script_col
FROM information_schema.COLUMNS
WHERE is_nullable=0
and length(COLUMN_DEFAULT) is NULL and
CHARACTER_MAXIMUM_LENGTH is not NULL and
table_schema = 'immh'
Union
SELECT CONCAT_WS('.',TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME) col,CONCAT('alter table ',TABLE_NAME,' MODIFY COLUMN ', COLUMN_NAME,' ',DATA_TYPE,' NULL DEFAULT NULL') as script_col
FROM information_schema.COLUMNS
WHERE is_nullable=0
and length(COLUMN_DEFAULT) is NULL and
CHARACTER_MAXIMUM_LENGTH is NULL and
table_schema = 'immh'
I have a Microsoft stored procedure that queries two MySQL databases using OpenQuery. The two MySQL databases should be have the same schemas, so I can run the same query on both.
However, we will soon alter the MySQL schemas, and add a column to a table. But the two MySQL databases won't happen at the same time, and I don't know the exact date of the releases.
I therefore want to write the query so that if the new column exists, then I use it in my select. If not, then I use a default value.
Is this possible? (That is have a query that handles differences in the table schema?)
(Not to be confused with 'coelesce' where the field definitely exists, but is simply null.)
You can use the following SELECT statement:
SELECT *
FROM information_schema.COLUMNS
WHERE
TABLE_SCHEMA = 'database name' AND TABLE_NAME = 'your table name'
AND COLUMN_NAME = 'the column name you want to check for'
If the above returns a value, your column is there. If not, then run your alternative SELECT statement
Updated statement:
IF EXISTS (SELECT *
FROM OPENQUERY(servername, 'SELECT *
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = ''database name''
AND TABLE_NAME = ''your table name''
AND COLUMN_NAME = ''the column name you want to check for'' ))
I am migrating a database from Access to MySQL, using Xataface as a frontend. Alas, Xataface is unable to modify a record if the column name contains a space and the people who made the Access database have tons of spaces in the column name.
The database is, alas, rather large, and thus manually modifying it seems rather problematic. I have looked through the MySQL manual and have only found things on how to remove whitespace from individual rows.
Perhaps modifying the INFORMATION_SCHEMA table would be the way to do this?
You could create a statement where each output is an ALTER TABLE statement and run those statements afterwards.
SQL Statement
SELECT 'ALTER TABLE '
+ Table_Name
+ ' CHANGE COLUMN `'
+ Column_Name
+ '` `'
+ REPLACE(Column_Name, ' ', '')
+ '`'
FROM INFORMATION_SCHEMA_COLUMNS
WHERE Column_Name LIKE '% %'
Output
ALTER TABLE tableX CHANGE COLUMN [column with spaces] [columnwithspaces]
I am looking for a single MYSQL script to convert ALL column names in a database to lowercase in one go...
I have inherited a MYSQL database that has a lot of mixed case column names (150 tables with a strange naming convention) and I don't want to go through manually each table by table to do this.
Has anyone got such a script?
Thanks
In case anybody else wants this below is an example of the completed query, please test before you use.....
EDIT COMPLETED SOLUTION AS REQUESTED
SELECT CONCAT(
'ALTER TABLE ', table_name,
' CHANGE ', column_name, ' ',
LOWER(column_name), ' ', column_type, ' ', extra,
CASE WHEN IS_NULLABLE = 'YES' THEN ' NULL' ELSE ' NOT NULL' END, ';') AS line
FROM information_schema.columns
WHERE table_schema = '<DBNAME>'
AND data_type IN ('char', 'varchar','INT', 'TINYINT', 'datetime','text','double','decimal')
ORDER BY line;
HTH somebody in the future...
BTW views are also scripted here so you may need to take them out of your final SQL code
You can rename all table and column names to lowercase by applying the following regular expression to a SQL dump (for example, the dump generated by mysqldump):
s/`\(\w\+\)`/\L&/g
This works because all table and column names are wrapped by `` (backticks). It is better to do this on the schema only, separate from the data (work with table structures only and then do the inserts).
To do this in Vim, open the SQL dump and enter the following command:
:%s/`\(\w\+\)`/\L&/g
Or do it from the command line using sed:
sed 's/`\(\w\+\)`/\L&/g' input.sql > output.sql
If you need to do it repeatedly, store the expression in a text file, and invoke it as follows:
sed -f regex.txt input.sql > output.sql
The solution proposed by lepe is really the only safe way to go. The scripting methods are too dangerous, easy to export or process the wrong data definition. All of the example scripts above leave out several data types, so they are incomplete.
I did a sqldump which places backticks around the table and column names, then used Notepad++ to Search on (`.*`) and Replace With \L\1. That rendered all of my table and column names to lower case.
Then I backed up my database, wiped out all of the tables and then executed my .sql file to rebuild. I did not worry about doing structure separate from data as I found no occurances of the backtick symbol in any of my data.
In my case, I need my column names all lower case because my development environment automatically converts first_name to First Name: as my field label for data entry. If I left them as caps (which I inherited), they would convert to FIRST NAME which is not what I want, I'd have to alter all of my field labels.
You can solve this task by building a script, starting with the output from this statement:
SELECT table_name, column_name, data_type
FROM information_schema.columns
WHERE table_schema = 'dbname';
ORDER BY table_name
Details about this feature can be found here "MYSQL::The INFORMATION_SCHEMA COLUMNS Table"
Then you can use the ALTER TABLE .. CHANGE feature to change the name of the columns
e.g.
ALTER TABLE mytable CHANGE old_name new_name varchar(5);
See also "MYSQL::ALTER TABLE Syntax"
Different datatype have different requirements so you need the UNIONs:
SELECT 'ALTER TABLE '||table_name||' CHANGE '|| column_name||' '||lower(column_name)||' '||datatype||'('||CHAR(character_maximum_length)||');' AS Line
FROM information_schema.columns
WHERE table_schema = dbname and datatype in ( 'CHAR', 'VARCHAR' )
ORDER BY table_name
UNION
SELECT 'ALTER TABLE '||table_name||' CHANGE '|| column_name||' '||lower(column_name)||' '||datatype||'('||CHAR(numeric_precision)||');' AS Line
FROM information_schema.columns
WHERE table_schema = dbname and datatype in ( 'INTEGER' )
ORDER BY table_name
UNION
SELECT 'ALTER TABLE '||table_name||' CHANGE '|| column_name||' '||lower(column_name)||' '||datatype||'('||CHAR(numeric_precision)||','||CHAR(numeric_scale)|');' AS Line
FROM information_schema.columns
WHERE table_schema = dbname and datatype in ( 'FLOAT' )
ORDER BY table_name
UNION
SELECT 'ALTER TABLE '||table_name||' CHANGE '|| column_name||' '||lower(column_name)||' '||datatype||');' AS Line
FROM information_schema.columns
WHERE table_schema = dbname and datatype in ( 'DATE' )
ORDER BY table_name