Is there a simple tool to convert mysql to postgresql syntax? - mysql

I've tried the tools listed here, some with more success than others, but none gave me valid postgres syntax I could use (tinyint errors etc.)

There's a mysqldump option which makes it output PostgreSQL code:
mysqldump --compatible=postgresql ...
But that doesn't work too well.
Instead, please see the mysql-to-postgres tool as described in Linus Oleander's answer.

Try this one , it works like charm !!
http://www.sqlines.com/online

After some time on Google I found this post.
Install the mysql2psql gem using [sudo] gem install mysql2psql.
Create a config file by running mysql2psql. You'll see an error but a mysql2psql.yml file should have been created.
Edit mysql2psql.yml
Run mysql2psql again to migrate you data.
Tip: Set force_truncate to true in your mysql2psql.yml config file if you want the postgresql database to be cleared before migrating your data.

Install pgloader on Debian or Ubuntu:
sudo apt install pgloader
Login as the postgres user and create a database
sudo su postgres
createdb -O user db_migrated
Transfer data from the mysql database to postgresql
pgloader mysql://user#localhost/db postgresql:///db_migrated
Check also Dimitri Fontaine's rewrite of pgloader from python to common lisp so that he could implement real threading.
Installation on other platforms
To install pgloader on Windows, you can use the Windows Subsystem for Linux.
To install pgloader on Mac, you can use: brew install --HEAD pgloader.

I've used py-mysql2pgsql. After installation it needs only simple configuration file in yml format (source, destination), e.g.:
# if a socket is specified we will use that
# if tcp is chosen you can use compression
mysql:
hostname: localhost
port: 3306
socket: /tmp/mysql.sock
username: mysql2psql
password:
database: mysql2psql_test
compress: false
destination:
# if file is given, output goes to file, else postgres
file:
postgres:
hostname: localhost
port: 5432
username: mysql2psql
password:
database: mysql2psql_test
Usage:
> py-mysql2pgsql -h
usage: py-mysql2pgsql [-h] [-v] [-f FILE]
Tool for migrating/converting data from mysql to postgresql.
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show progress of data migration.
-f FILE, --file FILE Location of configuration file (default:
mysql2pgsql.yml). If none exists at that path,
one will be created for you.
More on its home page https://github.com/philipsoutham/py-mysql2pgsql.

Have a look at PG Foundry, extra utilities for Postgres tend to live there. I believe that the tool you're looking for does exist though.

There is one piece of pay software listed on this postgresql page:
http://www.postgresql.org/download/products/1
and this is on pgFoundry:
http://pgfoundry.org/projects/mysql2pgsql/

This page lists the syntax differences, but a simple working query converter i haven't found yet. Using an ORM package instead of raw SQL could prevent these issues.
I'm currently hacking up a converter for a legacy codebase:
function mysql2pgsql($mysql){
return preg_replace("/limit (\d+), *(\d+)/i", "limit $1 offset $2", preg_replace("/as '([^']+)'/i", 'as "$1"', $mysql)); // Note: limit needs order
}
For CREATE statements, SQLines converts most of them online. I still had to edit the mysqldump afterwards, though:
"mediumtext" -> "text", "^LOCK.*" -> "", "^UNLOCK.*" -> "", "`" -> '"', "'" -> "''" in 'data', "0000-00-00" -> "2000-01-01", deduplicate constraint names, " CHARACTER SET utf8 " -> " ".
"int(10)" -> "int" was missed in the last table, so pass that part of the mysqldump through http://www.sqlines.com/online again.

you will most likely never get a tool for such task which would do all of your job for you. be prepared to do some refactoring work yourself.

Related

Configure MySQL in SublimeText 3

I want to configure MySQL in SublimeText 3, I found that the configuration is from Build System / New Build System ...
My written code is as follows:
{
"cmd": ["C: /xampp/mysql/bin/mysql.exe", "-u", "dorbezo", "-P", "Dorbezo123", "-h", "192.168.1.99", " -e "," source $ file "," -t "],
"selector": "source.sql",
"quiet": true}
My credentials to access MySQL from Workbench are the following:
user: dorbezo,
pass: Dorbezo123,
host: port: 192.168.1.99:3306
Am I entering the connection correctly? It is worth mentioning that I connect via VPN and I have ** xampp ** started when I try to run a query, getting the following error:
** show databases; **
Unknown suffix 'D' used for variable 'port' (value 'Dorbezo123')
C: /xampp/mysql/bin/mysql.exe: Error while setting value 'Dorbezo123' to 'port'
I also mention that the port where ** xampp ** connects is 3307, since 3306 (which I use in Workbench) causes me conflict.
There are several problems with your .sublime-build file.
Your command path "C: /xampp/mysql/bin/mysql.exe" has a space in it but perhaps that was pasted into your post incorrectly. I would have expected something more like this on Windows: "C:\\xampp\\mysql\\bin\\mysql.exe".
You are using an uppercase -P for your password, it should be a lowercase -p.
You possibly need to add the port number with the uppercase "-P", "3306". I say 'possibly' because 3306 is the default port for MySQL so you may not need to specify it at all.
Using long form options is generally a good idea because they prevent letter case mistakes. e.g. --user, --password, --host, --port, --execute, --table.
Here is a MySQL.sublime-build file for you to try, the long options (with my details) work for me on Linux:
{
"cmd": ["C:\\xampp\\mysql\\bin\\mysql.exe", "--user=dorbezo", "--password=Dorbezo123", "--host=192.168.1.99", "--port=3306", "--execute=source $file", "--table"],
"selector": "source.sql"
}
Clearly storing a password in a .sublime-build file is a security risk. You should consider creating a MySQL user with an insecure password which has limited privileges.
There is also the SQLTools Sublime Text plugin which you could install, see here for the documentation. Instead of your connection details being stored in a .sublime-build file you would add them in a SQLToolsConnections.sublime-settings settings file. The documentation link above has detailed examples. I suspect you might find using this plugin easier than managing the build file. Using this plugin means there is less of a password security issue; if you use null in the password field (not in quotes) then the plugin will prompt for the password and then remember it for the session (I think).

MySQLdump backup script no longer works, getting "mysqldump: unknown variable 'local-infile=0'"

I've recently upgrade a server to Debian 9 and MySQL to the latest version. I have a simple backup script that I run before performing any work on a production site but this time, when running my script, I encounter the following:
mysqldump: unknown variable 'local-infile=0'
Here is my script. What's going on?
#!/bin/bash
# [skipping commentary]
SITE=prod
# Set the directory that the Drupal root is IN, no trailing slashes
DROOT=[website_root]
# Set the directory for storing backups, no trailing slashes
BUD=/$DROOT/notes/backups
# Don't edit; End of defining variables
echo Doing a full back up...
echo Prepare to enter MySQL password...
# tar -czf $BUD/$SITE-files-$(date +'%Y%m%d%H%M%S').tgz $DROOT/docroot
mysqldump -u mysql_user -p drupal > $BUD/$SITE-drupal-$(date +'%Y%m%d%H%M%S').sql
mysqldump -u mysql_user -p civicrm > $BUD/$SITE-civicrm-$(date +'%Y%m%d%H%M%S').sql
ls -lh $BUD
pwd
echo Finished with backups...
MySQL version 10.1.37-MariaDB-0+deb9u1 Debian 9.6
Edit: When I ssh and run mysqldump with correct permissions I get the same issue. Weirdest thing, cron that runs similar process is backing up my databases as ordered.
The best way to solve this is simply to rename the variable to:
loose-local-infile=1
This will allow mysqldump to merely throw a warning, rather than a fatal error.
The suggestion to comment out the variable is not an option if you want LOAD DATA INFILE functionality out of the box, and MySQL 8+ for security reasons requires you to set this variable for both server (mysqld) and client. It is the [client] variable grouping in your config that chokes mysqldump if you don't add the "loose-" prefix to local-infile.
Seems like the new version you install is compiled without support of local-infile parameter. And because package management system (usually) keep your current configuration file you can try to find this parameter in my.ini file and comment it.
This parameter manage LOAD DATA LOCAL functionality. But seems like this have some potential security issues (more here)

Facing issue in Postgres loading extension

I am trying to to load pgrouting in extension in postgis database named as "routing" the problem I am facing is I type in osgeo shell in "C:\Program Files (x86)\Boundless\OpenGeo\bin" "psql -U postgres routing" and it gives the error of "psql : illegal option --u"
Are you sure, that your wrote "-U" and not "-u" in lower case? Because this is what the error tells you. If you connect to the database routing using: psql -U postgres routing then it should be all right.

MySQL: Enable LOAD DATA LOCAL INFILE

I'm running Mysql 5.5 on Ubuntu 12 LTS. How should I enable LOAD DATA LOCAL INFILE in my.cnf?
I've tried adding local-infile in my config at various places but I'm still getting the "The used command is not allowed with this MySQL version"
From the MySQL 5.5 manual page:
LOCAL works only if your server and your client both have been
configured to permit it. For example, if mysqld was started with
--local-infile=0, LOCAL does not work. See Section 6.1.6, “Security Issues with LOAD DATA LOCAL”.
You should set the option:
local-infile=1
into your [mysql] entry of my.cnf file or call mysql client with the --local-infile option:
mysql --local-infile -uroot -pyourpwd yourdbname
You have to be sure that the same parameter is defined into your [mysqld] section too to enable the "local infile" feature server side.
It's a security restriction.
The my.cnf file you should edit is the /etc/mysql/my.cnf file. Just:
sudo nano /etc/mysql/my.cnf
Then add:
[mysqld]
local-infile
[mysql]
local-infile
The headers [mysqld] and [mysql] are already given, just locate them in the file and add local-infile underneath each of them.
It works for me on MySQL 5.5 on Ubuntu 12.04 LTS.
I solved this problem on MySQL 8.0.11 with the mysql terminal command:
SET GLOBAL local_infile = true;
I mean I logged in first with the usual:
mysql -u user -p*
After that you can see the status with the command:
SHOW GLOBAL VARIABLES LIKE 'local_infile';
It should be ON. I will not be writing about security issued with loading local files into database here.
Replace the driver php5-mysql by the native driver
On debian
apt-get install php5-mysqlnd
in case your flavor of mysql on ubuntu does NOT under any circumstances work and you still get the 1148 error, you can run the load data infile command via command line
open a terminal window
run mysql -u YOURUSERNAME -p --local-infile YOURDBNAME
you will be requested to insert mysqluser password
you will be running MySQLMonitor and your command prompt will be mysql>
run your load data infile command (dont forget to end with a semicolon ; )
like this:
load data local infile '/home/tony/Desktop/2013Mini.csv' into table Reading_Table FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
See below image...
I've added --local-infile=1 to normal mysql command mysql -u root -p
So total line would be :
mysql --local-infile=1 -u root -p
Also, for other readers, if you are trying to do this in Django AND your server allows local_infile (you can check by typing SHOW VARIABLES via a mysql client) then you can add this to your settings.py file (since python MySQLdb doesn't by default read the .my.cnf file):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mydb',
'USER': 'myname',
'PASSWORD': 'mypass',
'HOST': 'myserver',
'PORT': '3306',
'OPTIONS' : {
'local_infile':1,
},
}
}
Add local_infile in both mysql and mysqld section.
[mysql]
local_infile=1
...
[mysqld]
local_infile=1
...
Tested in MySQL 8.x both in Windows and Linux.
I know this is not exactly what the OP is asking, but as this thread is quite old and none of the solutions proposed here worked for me, I decided to share this.
If someone is having trouble enabling local_infile in the version 8 of MySql, this command here did the trick for me:
SET PERSIST local_infile = 1;
It persists the configuration on the "mysqld-auto.cnf" config file and then the change will be remembered after service or server restart.
You have to take care how you establish your mysqli connection. Full credit for this solution goes to Jorge Albarenque, source
In order to fix it I had to:
Add local-infile=1 to the [mysqld] and [mysql] sections of my.cnf (as explained in the comments above)
Use mysqli_real_connect function (PHP documentation).
The catch is that with that function you can explicitly enable the support for LOAD DATA LOCAL INFILE. For example (procedural style):
$link = mysqli_init();
mysqli_options($link, MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_real_connect($link, $host, $username, $password, $database);
or object oriented
$mysqli = mysqli_init();
$mysqli->options(MYSQLI_OPT_LOCAL_INFILE, true);
$mysqli->real_connect($host, $username, $password, $database);
if your csv file located same with db, you need to remove LOCAL in LOAD DATA INFILE,
or you will get the error
The used command is not allowed with this MySQL version
Another way is to use the mysqlimport client program.
You invoke it as follows:
mysqlimport -uTheUsername -pThePassword --local yourDatabaseName tableName.txt
This generates a LOAD DATA statement which loads tableName.txt into the tableName table.
Keep in mind the following:
mysqlimport determines the table name from the file you provide; using all text from the start of the file name up to the first period as the table name. So, if you wish to load several files to the same table you could distinguish them like tableName.1.txt, tableName.2.txt,..., etc, for example.
This went a little weird for me, from one day to the next one the script that have been working since days just stop working. There wasn´t a newer version of mysql or any kind of upgrade but I was getting the same error, so I give a last try to the CSV file and notice that the end of lines were using \n instead of the expected ( per my script ) \r\n so I save it with the right EOL and run the script again without any trouble.
I think is kind of odd for mysql to tell me The used command is not allowed with this MySQL version since the reason was completely different.
My working command looks like this:
LOAD DATA LOCAL INFILE 'file-name' IGNORE INTO TABLE table-name CHARACTER SET latin1 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES.
I used below method, which doesn't require any change in config, tested on mysql-5.5.51-winx64 and 5.5.50-MariaDB:
put 'load data...' in .sql file (ex: LoadTableName.sql)
LOAD DATA INFILE 'D:\\Work\\TableRecords.csv' INTO TABLE tbl1 FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES (col1,col2);
then:
mysql -uroot -pStr0ngP#ss -Ddatabasename -e "source D:\Work\LoadTableName.sql"
In case if Mysql 5.7 you can use "show global variables like "local_infile" ;" which will give the local infile status ,You can turn it on using "set global local_infile=ON ; ".
I am using xampp v3.2.4 and mysql server 8.0.20.
I added local-infile=1 to [mysql] and [mysqld] in the file "my.ini". The file is located at "C:\xampp\mysql\bin\my.ini".
Then I inserted the data from csv file using the following code LOAD DATA INFILE .... It is important to move LOCAL. Otherwise it won't work.
Thanks for all suggestions above. A combination finally worked out for me.
Ok, something odd is happening here. To make this work, do NOT need to make any configuration changes in /etc/mysql/my.cnf . All you need to do is to restart the current mysql service in terminal:
sudo service mysql restart
Then if I want to "recreate" the bug, I simply restart the apache service:
sudo service apache2 restart
Which can then be fixed again by entering the following command:
sudo service mysql restart
So, it appears that the apache2 is doing something to not allow this feature
when it starts up (which is then reversed/corrected if restart the mysql service).
Valid in Debian based distributions.
service mysqld restart
service httpd restart
Valid in RedHat based distributions
For those of you looking for answers to make LOAD DATA LOCAL INFILE work like me, this might probably work. Well it worked for me, so here it goes. Install percona as your mysql server and client by following the steps from the link. A password will be prompted for during the installation, so provide one that you'll remember and use it later. One the installation is done, reboot your system and test if the server is up and running by going to the terminal and typing mysql -u root -p and then the password. Try running the command LOAD DATA LOCAL INFILE now.. Hope it works :)
BTW I was working on Rails 2.3 with Ruby 1.9.3 on Ubuntu 12.04.
All: Evidently this is working as designed. Please see new ref man dated 2019-7-23, Section 6.1.6, Security Issues with LOAD DATA LOCAL.
Please note that for newer MySQL servers, like version 8.0, this setting is not a boolean value, but ON|OFF as you can read here in the docs: https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile
If you set it to 1 MySQL will not recognize it and will warn this error on your logfiles:
# /var/log/mysql/mysql.err
2022-04-08T14:45:06.532132Z 0 [Warning] [MY-000076] [Server] option 'local_infile': boolean value '1;' was not recognized. Set to OFF.

connecting to mysql from cygwin

I can successfully connect to MySQL from a DOS prompt, but when I try to connect from cygwin, it just hangs.
$/cygdrive/c/Program\ Files/MySQL/MySQL\ Server\ 5.1/bin/mysql -u root -p
What's wrong?
I just came across this, and when I read someone's mention of it being a windows/DOS command that you run in cygwin I did a which mysql and that gave me:
$ which mysql
/cygdrive/c/Program Files/MySQL/MySQL Server 5.5/bin/mysql
So I ran the cygwin Setup.exe searched for "mysql" and installed the latest "mysql client". Now which mysql looks like:
$ which mysql
/usr/bin/mysql
And the MySQL command works in cygwin :)
Though it's an old question, it would be nice to have the actual answer here, as people (like myself) might still stumble across it.
If your attempts to run the MySQL client from Cygwin return the following error:
$ mysql -u root -p
Enter password:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysql.sock' (2)
Then you can fix it by adding the explicit -h 127.0.0.1 options to the command line, as in:
$ mysql -u root -p -h 127.0.0.1
Updates based on comments:
To avoid specifying -h 127.0.0.1 on the command line every time you connect, you can add:
[client]
host=127.0.0.1
to the file /etc/my.cnf
On some installations of Cygwin, specifying the host with -h might not be enough. If so, try also specifying:
--protocol=tcp
or add
protocol=tcp
to the config file.
Assuming that you have a native Windows build of MySQL, there is a terminal emulation incompatibility between DOS (command prompt) windows and bash. The prompt for mysql isn't showing up.
To confirm this, type a command and return - it will probably work, but the prompt and the echo of the command (what you're typing) is getting lost.
There may be a workaround in either the CYGWIN sytem properties or in bash, but I've never taken the time to work this one out.
Other answers lack the following key detail:
Cygwin has two shells:
Default: c:\cygwin\bin\mintty.exe
Basic: c:\cygwin\Cygwin.bat (which launches c:\cygwin\bin\bash.exe)
The Win32 MySQL can write properly to #2, but not #1, because Win32 MySQL cannot probe stdin properly (thanks #PeterNore)
Want to know if you're using Win32 MySQL? Use which, e.g.
$ which mysql
/cygdrive/c/Program Files/MySQL/MySQL Server 5.1/bin/mysql
Bonus: Cygwin guide to overcoming path problems (thanks #Dustin)
I posted a solution/workaround here:
enter key sometimes not recognized in windows apps under cygwin
Run bash from the cmd.exe executable and then mysql will work inside bash.
Create a shortcut for cmd.exe on your desktop.
Open up the properties for the shortcut and change the startup directory to the cygwin bin directory (usually C:\cygwin\bin).
Add "/c bash.exe" to the end of the command in the target parameter.
This will run bash under the windows cmd.exe environment and when you attempt to run mysql it will execute as you would expect. This is working under windows 7 but has not been tested in any other version.
Put cygwin bin directory in path env variable.
Use command window by running cmd
Run bash -l in cmd window
Then MySQL can be run without problem.
Svend Hansen's answer is the right one:
Install windows mysql server files (from mysql-5.5.25-win32.msi for example)
Install Cygwin mysql client with cygwin installer (setup.exe)
Connect to your server in a cygwin window using cygwin client "mysql -u[user] -p[Password] -h[host]", in my case "mysql -uroot -pXXXX -h127.0.0.1"
I think that when the question was posted, the cygwin setup did not provide mysql components, which is solved now.
Althoug Svend Hansen answer has some points, another thing is the PATH in Environment variables - if the path to mysql is before that of cygwin
which mysql
will show
/cygdrive/c/Program Files/MySQL/MySQL Server 5.5/bin/mysql
otherwise it will show the cygwin client.
As reference Wikipedia says:
Some programs may add their directory to the front of the PATH
variable's content during installation, to speed up the search process
and/or override OS commands.
Download Cygwin
Install mysql client app
create an alias in .bashrc file
alias mysql='mysql -h 127.0.0.1'
execute source .bashrc
Now you can connect to mysql
mysql -u user -p
I have created a semi-fix for this that satisfies me.
I ran cygwin.bat in cmd.exe, then typing mysql in- everything worked fine.
I realized right there that the problem was mintty.
Easy solution? Download Console2, and under settings you can point
it to the cygwin shell. Restart Console2, run mysql and the output
appears.
This is advantageous anyways, because Console2 has a more robust interface/customization than Mintty. I really like the transparency and color mapping options.
Do This:
just copy ur mysql.exe from C:\Program Files\MySQL\MySQL Server 5.5\bin
paste this mysql.exe in C:\cygwin\usr\local\bin
now run which mysql, It will
Disclaimer: The following solved this issue for me under MinTTY on MinGW/MSYS. From research, I believe this same root cause affects Cygwin as well.
Answer is posted here: https://stackoverflow.com/a/23164362/1034436
In a nutshell, you'll need to prepend your mysql command with winpty's console.exe (or have aliases that does so). This solution worked with native Windows MySQL executables and not a special cygwin/mingw build. You do, however, have to compile winpty, but that was simple and painless, and worked as per their documentation for me.
Note: This also solved my issue with several other native Windows console applications, namely Python and Mercurial with OpenSSH.
Reinstall cygwin and during reinstallation search for mysql in packages, install the mysql client and then it would work fine.
Found this question today 2018-03-18 looking for some answers to
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysql.sock' (2 "No such file or directory")
The file /etc/my.conf references config files in /etc/my.cnf.d
I added this to /etc/my.cnf.d/client.cnf:
[client]
host=127.0.0.1
protocol=tcp
After that I was able to access the local windows MySQL instance from a cygwin terminal using mysql -u root -p