I want to connect MATLAB with MYSQL.I dont know the procedure.In MATLAB help it says about some drivers which confuses me alot.Can someone please guide me!please tell me the complete process.I shall be very very thankful!!!
I use JDBC to connect from MATLAB to mySQL database. Works seamlessly.
First download JDBC driver for mySQL from here:
http://www.mysql.com/downloads/connector/j/
Unpack mysql-connector-java-x.x.xx-bin.jar (the latest version) file from the archive into a folder
In the beginning of your script add the path to this jar file, then you can connect to a database and so on.
Here is an example of connecting to and querying public human genome database:
%# add path to the JAR file you just installed to Java dynamic classpath
javaaddpath('h:\Documents\MATLAB\myJavaClasses\mysql-connector-java-5.1.12-bin.jar')
%# connection parameteres
host = 'genome-mysql.cse.ucsc.edu';
user = 'genome';
password = '';
dbName = 'hg18';
%# JDBC parameters
jdbcString = sprintf('jdbc:mysql://%s/%s', host, dbName);
jdbcDriver = 'com.mysql.jdbc.Driver';
%# Create the database connection object
conn = database(dbName, user , password, jdbcDriver, jdbcString);
gene = 'NF1';
if isconnection(conn) % check to make sure that we successfully connected
qry = sprintf('SELECT geneName, chrom, txStart, txEnd FROM refFlat WHERE geneName=''%s''',gene);
rs = fetch(exec(conn, qry));
rsdata = get(rs, 'Data');
end
Related
I have been trying to connect to my SQL that I am trying to create. I recently downloaded MySQL, the workbench, the connector ODBC, and the ODBC Manager, but I can't find the solution to solve the error for the connection.
Do I need to download anything else? I can't find a solution on internet or youtube for Mac.
packages_required = c("quantmod", "RSQLite", "data.table", "lubridate", "pbapply", "DBI")
install.packages(packages_required)
library("quantmod")
library("RSQLite")
library("data.table")
library("lubridate")
library("pbapply")
library("odbc")
PASS <- new.env()
assign("pwd","My Password",envir=PASS)
library("DBI")
con <- dbConnect(odbc(), Driver = "/usr/local/mysql-connector-odbc-8.0.28-macos11-x86-64bit/lib/libmyodbc8w.so",
Server = "localhost", Database = "data", UID = "root", PWD = PASS$pwd,
Port = 3306)
-----------------------------------------------------------------------------------------
> con <- dbConnect(odbc(), Driver = "/usr/local/mysql-connector-odbc-8.0.28-macos11-x86-64bit/lib/libmyodbc8w.so",
+ Server = "localhost", Database = "data", UID = "root", PWD = PASS$pwd,
+ Port = 3306)
Error: nanodbc/nanodbc.cpp:1021: 00000: [
>
Thank you
Presuming you're on Windows, try creating an ODBC connection using the most recent driver. The ODBC data sources tool should already be installed, you just need to open it and create a new one.
Press the windows key (or click the search spyglass) and type in "ODBC." The "ODBC Data Sources (64-bit)" tool should come up.
How to Create an ODBC Connection in Windows
Open the "ODBC Data Sources (64-bit)" application
Click "Add"
Choose"MySQL ODBC 8.0 Unicode Driver" (or whatever the newest version you
have is). If you don't have it, you can download it here:
https://dev.mysql.com/downloads/connector/odbc/
Enter the following information:
Data source name (the example code below uses "my_odbc_connection"), TCP/IP Server, Port, User and Password
Click "Details" to expand the box.
In the "Connection" tab, you may need to check the "Enable Cleartext
Authentication" box. Could depend on your system configuratoin.
Click "Test" to test the connection. If everything went right you
should get "Connection Successful" message. If you aren't able to get a
successful connection, make sure that you have access and that your
connection information doesn't have any typos.
After making a successful connection, perform these 2 additional steps (the
drop-downs won't populate until you connect successfully):
Click the "Database" drop down to choose the default database that you'll
be writing data to. If you will be writing to more than 1 database
then you may need to create a separate connection for each database
that you'll be writing to, specifying the default database
differently for each one.
Click the "Character Set" drop down and choose utf8.
You should now able to use the "DBI" and "ODBC" packages to read, write, etc. any data directly from R. Specific settings listed above may or may not apply depending on situation.
See the example code below.
Further reading: https://www.r-bloggers.com/setting-up-an-odbc-connection-with-ms-sql-server-on-windows/
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Load or install packages
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Load or install librarian
if(require(librarian) == FALSE){
install.packages("librarian")
if(require(librarian)== FALSE){stop("Unable to install and load librarian")}
}
# Load multiple packages using the librarian package
librarian::shelf(tidyverse, readxl, DBI, lubridate, ODBC, quiet = TRUE)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Read
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Connect to a pre-defined ODBC connection named "my_odbc_connection"
conn <- DBI::dbConnect(odbc::odbc(), "my_odbc_connection")
# Create a query
query <- "
SELECT *
FROM YOUR_SCHEMA.YOUR_TABLE;
"
# Run the query
df_data <- DBI::dbGetQuery(conn,query)
# Close the open connection
try(DBI::dbDisconnect(conn), silent = TRUE)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Write
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Define the connection and database you'll be writing to
conn <- DBI::dbConnect(odbc::odbc(), "my_odbc_connection", db ="YOUR_DEFAULT_DB")
# Define variable types for your data frame. As a general rule, it's a good idea to define your data types rather than let the package guess.
field_types <- c("INTEGER","VARCHAR(20)","DATE","DATETIME","VARCHAR(20)","VARCHAR(50)","VARCHAR(20)")
names(field_types) <- names(df_data)
# Record start time
start_time <- Sys.time()
# Example write statement
DBI::dbWriteTable(conn,"YOUR_TABLE_NAME",YOUR_DATA_FRAME,overwrite=TRUE,field.types=field_types, row.names = FALSE)
# Print time difference
print("Writing complete.")
print(Sys.time() - start_time)
# Close the open connection
try(DBI::dbDisconnect(conn), silent = TRUE)
I have been using Heroku for a while to host my Discord bot. It has been connecting to a MySQL database hosted on ClearDB successfully. However, very recently, whenever I use the bot and it tries to connect to the database, it throws this error:
2026 (HY000): SSL connection error: error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
It has been working completely fine until now, and I haven't changed anything. For background, all I did was delete a pipeline and make my app a standalone app without any pipeline. Just in case this helps.
Is this because Heroku has been updated? How can I fix my bot? Let me know if you need any more information.
Any help is appreciated, and Thank You in advance!
EDIT:
Database connection code:
import mysql.connector
def create_conn():
conn = None
try:
conn = mysql.connector.connect(host="HOST",
database="DB",
user="USER",
password="PWD")
except Exception as e:
print(e)
return conn
def execute_query(query, params, fetchall=True):
conn = create_conn()
if conn:
cursor = conn.cursor()
cursor.execute(query % params)
try:
if fetchall:
results = cursor.fetchall()
else:
results = cursor.fetchone()
except:
results = None
conn.commit()
cursor.close()
conn.close()
return results
else:
return False
The database connection used to work, and still works when I run it on my testing machine, a raspberry pi.
EDIT 2:
requirements.txt:
aiohttp==3.6.3
async-timeout==3.0.1
attrs==20.3.0
CacheControl==0.12.6
cachetools==4.2.0
certifi==2020.12.5
cffi==1.14.4
chardet==3.0.4
click==7.1.2
cryptography==3.3.1
cssselect==1.1.0
cssutils==1.0.2
discord==1.0.1
discord-pretty-help==1.2.0
discord.py==1.6.0
emoji==0.6.0
Flask==1.1.2
google-api-core==1.24.1
google-api-python-client==1.12.8
google-auth==1.24.0
google-auth-httplib2==0.0.4
google-cloud-core==1.5.0
google-cloud-firestore==2.0.2
google-cloud-storage==1.35.0
google-crc32c==1.1.0
google-resumable-media==1.2.0
googleapis-common-protos==1.52.0
grpcio==1.34.0
gunicorn==20.0.4
httplib2==0.18.1
idna==2.8
importlib-metadata==3.3.0
itsdangerous==1.1.0
jeepney==0.6.0
Jinja2==2.11.2
keyring==21.8.0
lxml==4.6.2
MarkupSafe==1.1.1
msgpack==1.0.2
multidict==4.7.6
mysql-connector-python==8.0.22
numpy==1.19.4
pandas==1.1.5
premailer==3.7.0
proto-plus==1.13.0
protobuf==3.14.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
python-dateutil==2.8.1
python-dotenv==0.15.0
pytz==2020.4
requests==2.25.1
rsa==4.7
schedule==0.6.0
SecretStorage==3.3.0
six==1.15.0
typing-extensions==3.7.4.3
uritemplate==3.0.1
urllib3==1.26.2
Werkzeug==1.0.1
yagmail==0.14.245
yarl==1.5.1
zipp==3.4.0
Just in case you can turn of ssl by:
conn = mysql.connector.connect(host="HOST",
database="DB",
user="USER",
password="PWD",
ssl_disabled=True)
i'm not quite sure how to do this, but i'm pretty sure you have to disable SSL for it to work, hope this helps.
Clearly, you need to enforce an SSL connection between your app and MySQL.
If you are using ruby stack then follow the given options and your SSL error problem will be solved.
Download the CA, Client, and Private Key files from your ClearDB dashboard and place them in the root of the application’s filesystem.
Make sure you have OpenSSL installed, which you can find here for Unix/Linux/OS X and here for Windows.
*Due to the MySQL client library configuration used on Heroku, you will need to strip the password from the private key file, which can be done like this:
$ openssl rsa -in cleardb_id-key.pem -out cleardb_id-key-no-password.pem
You can now delete the cleardb_id-key.pem and rename cleardb_id-key-no-password.pem to cleardb_id-key.pem, which you will use with your app.
*Set the DATABASE_URL config variable with the value of your modified CLEARDB_DATABASE_URL, like this:
$ heroku config:add DATABASE_URL="mysql2://abc1223:dfk243#us-cdbr-east.cleardb.com/my_heroku_db?
sslca=cleardb-ca-cert.pem&sslcert=cleardb_id-cert.pem&sslkey=cleardb_id-key.pem&reconnect=true"
notice how we added the “reconnect=true” parameters to the end of the URL? This is so that your application will automatically reconnect to ClearDB in the event of a connection timeout.
From here, simply restart your application (if Heroku didn’t already do that for you), and as long as you specified the correct file names and paths to the certificates in your DATABASE_URL, your app will now connect via SSL to ClearDB.
I have a SSIS package that will load data from a set of excel files then archive those files to a specified folder.
The excel files are stored in a folder and inside that folder I have the archive folder.
Below is my script task code for reference.
public void Main()
{
// TODO: Add your code here
string sourceDir = Dts.Variables["User::strFilePath"].Value.ToString();
string destDir = Dts.Variables["User::strArchivePath"].Value.ToString();
DirectoryInfo di = new DirectoryInfo(sourceDir);
string[] sDirFiles = Directory.GetFiles(sourceDir);
FileInfo[] fi = di.GetFiles("*.xls");
int fileCnt = fi.Length;
for (int i = 0; i < fileCnt; i++)
{
String filename = fi[i].Name;
string[] splitFilename = filename.Split('.');
DateTime dt = DateTime.Now;
string ArchiveDate = String.Format("{0:ddMMMyyyy}", dt);
string sourceFileName = filename;
string sourceFilePath = sourceDir + filename;
string destinationFileName = splitFilename[0] + '_' + ArchiveDate + '.' + splitFilename[1];
string destinationPath = destDir + destinationFileName;
//MessageBox.Show("Source File " + sourceFilePath + " to destination " + destinationPath);
if (File.Exists(destinationPath))
File.Delete(destinationPath);
// To move a file or folder to a new location:
System.IO.File.Move(sourceFilePath, destinationPath);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
The sourceDir and destDir are variables that provides the path of source files folder and archive folder. The package works fine when I run it from visual studio.
I have deployed it to run as a job using deployment utility by creating manifest file. When I run the job I'm getting error in my archive script task. Below is the screenshot of it.
I searched for solution in stackoverflow however the provided solutions does not solve my problem.
Problem using SQL Agent to run SSIS Packages - fails with “DTSER_FAILURE(1)”
DTSX package runs in Visual Studio but not when called from a Database Job
I have granted read write access to SQL server for both the folders. Still getting the same error?
Any help is highly appreciated.
Try choosing 64bit on your Project --> Properties --> Dubugging--> Run64BitTime this could solve the problem.
I replaced the script task with a file system task and it worked. I also created a proxy account to run the package with my system credentials so that it will have the access to the folder specified.
Putting the steps in creating a proxy account for running the SQL job so that it will be helpful for users looking out for it..
Create a User account in SQL under which you need to run the Job
Create a credentials for the user account created.
--Script #1 - Creating a credential to be used by proxy
USE MASTER
GO
--Drop the credential if it is already existing
IF EXISTS (SELECT 1 FROM sys.credentials WHERE name = N'SSISProxyCredentials')
BEGIN
DROP CREDENTIAL [SSISProxyCredentials]
END
GO
CREATE CREDENTIAL [SSISProxyCredentials]
WITH IDENTITY = N'<Insert the Username>',
SECRET = N'abcd#0987'
GO
Create Proxy account and associate the credentials created
--Script #2 - Creating a proxy account
USE msdb
GO
--Drop the proxy if it is already existing
IF EXISTS (SELECT 1 FROM msdb.dbo.sysproxies WHERE name = N'SSISProxyDemo')
BEGIN
EXEC dbo.sp_delete_proxy
#proxy_name = N'SSISProxyDemo'
END
GO
--Create a proxy and use the same credential as created above
EXEC msdb.dbo.sp_add_proxy
#proxy_name = N'SSISProxyDemo',
#credential_name=N'SSISProxyCredentials',
#enabled=1
GO
--To enable or disable you can use this command
EXEC msdb.dbo.sp_update_proxy
#proxy_name = N'SSISProxyDemo',
#enabled = 1 --#enabled = 0
GO
Granting proxy account to SQL Server Agent Sub-systems
USE msdb
GO
--You can view all the sub systems of SQL Server Agent with this command
--You can notice for SSIS Subsystem id is 11
EXEC sp_enum_sqlagent_subsystems
GO
--Grant created proxy to SQL Agent subsystem
--You can grant created proxy to as many as available subsystems
EXEC msdb.dbo.sp_grant_proxy_to_subsystem
#proxy_name=N'SSISProxyDemo',
#subsystem_id=11 --subsystem 11 is for SSIS as you can see in the above image
GO
--View all the proxies granted to all the subsystems
EXEC dbo.sp_enum_proxy_for_subsystem
Granting proxy access to security principals
USE msdb
GO
--Grant proxy account access to security principals that could be
--either login name or fixed server role or msdb role
--Please note, Members of sysadmin server role are allowed to use any proxy
EXEC msdb.dbo.sp_grant_login_to_proxy
#proxy_name=N'SSISProxyDemo'
,#login_name=N'<Insert the Username>'
--,#fixed_server_role=N''
--,#msdb_role=N''
GO
--View logins provided access to proxies
EXEC dbo.sp_enum_login_for_proxy
GO
Finally associate the proxy account to the package step. This can be also done through job wizard.
EXEC msdb.dbo.sp_add_jobstep #job_id=#jobId, #step_name=N'SSISPackageCall',
#step_id=1,
#cmdexec_success_code=0,
#on_success_action=1,
#on_success_step_id=0,
#on_fail_action=2,
#on_fail_step_id=0,
#retry_attempts=0,
#retry_interval=0,
#os_run_priority=0, #subsystem=N'SSIS',
#command=N'/FILE "C:\Package.dtsx" /CHECKPOINTING OFF /REPORTING E',
#database_name=N'master',
#flags=0,
#proxy_name = N'SSISProxyDemo';
Thanks for the valuable replies for my post..
I tried to connect to an existing mySQL connection with webmatrix, but I cannot specify a port different from 3306.
I tried to put as server:
xxx.xxx.xxx.xxx:myport
or
xxx.xxx.xxx.xxx myport
but it cannot connect.
I also tried to edit web.config file, writing a regular mysql connection specifying port=myport, but it seems that Webmatrix ignore that.
I don't try it yet on the "port" parameter, but it should work if you connect the database and set the connection string like this:
#{
var cs = "Server=myServerAddress;Port=1234;Database=myDataBase;Uid=myUsername;Pwd=myPassword;";
var db = Database.OpenConnectionString(cs, "MySql.Data.MySqlClient");
}
I used this workaround to add extra params that webmatrix just ignored when I put them into the web.config. You can alter the connectionString after getting it from the configuration file like this:
#{
var cs = System.Configuration.ConfigurationManager
.ConnectionStrings["connectionString_name"];
var cs_Altered = cs.ConnectionString + ";Command Timeout=120";
var db = Database.OpenConnectionString(cs_Altered, cs.ProviderName);
}
I can not get a valid string from an MSSQL server into python. I believe there is an encoding mismatch somewhere. I believe it is between the ODBC layer and python because I am able to get readable results in tsql and isql.
What character encoding does pyodbc expect? What do I need to change in the chain to get this to work?
Specific Example
Here is a simplified python script as an example:
#!/usr/bin/env python
import pyodbc
dsn = 'yourdb'
user = 'import'
password = 'get0lddata'
database = 'YourDb'
def get_cursor():
con_string = 'DSN=%s;UID=%s;PWD=%s;DATABASE=%s;' % (dsn, user, password, database)
conn = pyodbc.connect(con_string)
return conn.cursor()
if __name__ == '__main__':
c = get_cursor()
c.execute("select id, name from recipe where id = 4140567")
row = c.fetchone()
if row:
print row
The output of this script is:
(Decimal('4140567'), u'\U0072006f\U006e0061\U00650067')
Alternatively, if the last line of the script is changed to:
print "{0}, '{1}'".format(row.id, row.name)
Then the result is:
Traceback (most recent call last):
File "/home/mdenson/projects/test.py", line 20, in <module>
print "{0}, '{1}'".format(row.id, row.name)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)
A transcript using tsql to execute the same query:
root#luke:~# tsql -S cmw -U import -P get0lddata
locale is "C"
locale charset is "ANSI_X3.4-1968"
using default charset "UTF-8"
1> select id, name from recipe where id = 4140567
2> go
id name
4140567 orange2
(1 row affected)
and also in isql:
root#luke:~# isql -v yourdb import get0lddata
SQL> select id, name from recipe where id = 4140567
+----------------------+--------------------------+
| id | name |
+----------------------+--------------------------+
| 4140567 | orange2 |
+----------------------+--------------------------+
SQLRowCount returns 1
1 rows fetched
So I have worked at this for the morning and looked high and low and haven't figured out what is amiss.
Details
Here are version details:
Client is Ubuntu 12.04
freetds v0.91
unixodbc 2.2.14
python 2.7.3
pyodbc 2.1.7-1 (from ubuntu package) & 3.0.7-beta06 (compiled from source)
Server is XP with SQL Server Express 2008 R2
Here are the contents of a few configuration files on the client.
/etc/freetds/freetds.conf
[global]
tds version = 8.0
text size = 64512
[cmw]
host = 192.168.90.104
port = 1433
tds version = 8.0
client charset = UTF-8
/etc/odbcinst.ini
[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
CPTimeout =
CPReuse =
FileUsage = 1
/etc/odbc.ini
[yourdb]
Driver = FreeTDS
Description = ODBC connection via FreeTDS
Trace = No
Servername = cmw
Database = YourDB
Charset = UTF-8
So after continued work I am now getting unicode characters into python. Unfortunately the solution I've stumbled upon is about as satisfying as kissing your cousin.
I solved the problem by installing the python3 and python3-dev packages and then rebuilding pyodbc with python3.
Now that I've done this my scripts now work even though I am still running them with python 2.7.
So I don't know what was fixed by doing this, but it now works and I can move on to the project I started with.
Any chance you're having a problem with a BOM (Byte Order Marker)? If so, maybe this snippet of code will help:
import codecs
if s.beginswith( codecs.BOM_UTF8 ):
# The byte string s begins with the BOM: Do something.
# For example, decode the string as UTF-8
if u[0] == unicode( codecs.BOM_UTF8, "utf8" ):
# The unicode string begins with the BOM: Do something.
# For example, remove the character.
# Strip the BOM from the beginning of the Unicode string, if it exists
u.lstrip( unicode( codecs.BOM_UTF8, "utf8" ) )
I found that snippet on this page.
If you upgrade the pyodbc to version 3 the problem will be solved.