I have been trying to connect to my SQL that I am trying to create. I recently downloaded MySQL, the workbench, the connector ODBC, and the ODBC Manager, but I can't find the solution to solve the error for the connection.
Do I need to download anything else? I can't find a solution on internet or youtube for Mac.
packages_required = c("quantmod", "RSQLite", "data.table", "lubridate", "pbapply", "DBI")
install.packages(packages_required)
library("quantmod")
library("RSQLite")
library("data.table")
library("lubridate")
library("pbapply")
library("odbc")
PASS <- new.env()
assign("pwd","My Password",envir=PASS)
library("DBI")
con <- dbConnect(odbc(), Driver = "/usr/local/mysql-connector-odbc-8.0.28-macos11-x86-64bit/lib/libmyodbc8w.so",
Server = "localhost", Database = "data", UID = "root", PWD = PASS$pwd,
Port = 3306)
-----------------------------------------------------------------------------------------
> con <- dbConnect(odbc(), Driver = "/usr/local/mysql-connector-odbc-8.0.28-macos11-x86-64bit/lib/libmyodbc8w.so",
+ Server = "localhost", Database = "data", UID = "root", PWD = PASS$pwd,
+ Port = 3306)
Error: nanodbc/nanodbc.cpp:1021: 00000: [
>
Thank you
Presuming you're on Windows, try creating an ODBC connection using the most recent driver. The ODBC data sources tool should already be installed, you just need to open it and create a new one.
Press the windows key (or click the search spyglass) and type in "ODBC." The "ODBC Data Sources (64-bit)" tool should come up.
How to Create an ODBC Connection in Windows
Open the "ODBC Data Sources (64-bit)" application
Click "Add"
Choose"MySQL ODBC 8.0 Unicode Driver" (or whatever the newest version you
have is). If you don't have it, you can download it here:
https://dev.mysql.com/downloads/connector/odbc/
Enter the following information:
Data source name (the example code below uses "my_odbc_connection"), TCP/IP Server, Port, User and Password
Click "Details" to expand the box.
In the "Connection" tab, you may need to check the "Enable Cleartext
Authentication" box. Could depend on your system configuratoin.
Click "Test" to test the connection. If everything went right you
should get "Connection Successful" message. If you aren't able to get a
successful connection, make sure that you have access and that your
connection information doesn't have any typos.
After making a successful connection, perform these 2 additional steps (the
drop-downs won't populate until you connect successfully):
Click the "Database" drop down to choose the default database that you'll
be writing data to. If you will be writing to more than 1 database
then you may need to create a separate connection for each database
that you'll be writing to, specifying the default database
differently for each one.
Click the "Character Set" drop down and choose utf8.
You should now able to use the "DBI" and "ODBC" packages to read, write, etc. any data directly from R. Specific settings listed above may or may not apply depending on situation.
See the example code below.
Further reading: https://www.r-bloggers.com/setting-up-an-odbc-connection-with-ms-sql-server-on-windows/
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Load or install packages
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Load or install librarian
if(require(librarian) == FALSE){
install.packages("librarian")
if(require(librarian)== FALSE){stop("Unable to install and load librarian")}
}
# Load multiple packages using the librarian package
librarian::shelf(tidyverse, readxl, DBI, lubridate, ODBC, quiet = TRUE)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Read
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Connect to a pre-defined ODBC connection named "my_odbc_connection"
conn <- DBI::dbConnect(odbc::odbc(), "my_odbc_connection")
# Create a query
query <- "
SELECT *
FROM YOUR_SCHEMA.YOUR_TABLE;
"
# Run the query
df_data <- DBI::dbGetQuery(conn,query)
# Close the open connection
try(DBI::dbDisconnect(conn), silent = TRUE)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Write
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Define the connection and database you'll be writing to
conn <- DBI::dbConnect(odbc::odbc(), "my_odbc_connection", db ="YOUR_DEFAULT_DB")
# Define variable types for your data frame. As a general rule, it's a good idea to define your data types rather than let the package guess.
field_types <- c("INTEGER","VARCHAR(20)","DATE","DATETIME","VARCHAR(20)","VARCHAR(50)","VARCHAR(20)")
names(field_types) <- names(df_data)
# Record start time
start_time <- Sys.time()
# Example write statement
DBI::dbWriteTable(conn,"YOUR_TABLE_NAME",YOUR_DATA_FRAME,overwrite=TRUE,field.types=field_types, row.names = FALSE)
# Print time difference
print("Writing complete.")
print(Sys.time() - start_time)
# Close the open connection
try(DBI::dbDisconnect(conn), silent = TRUE)
Related
I am trying to connect to a MySQL database and run an SQL query using hdbc and hdbc-odbc
main :: IO ()
main = do
mysqlSettings <- readMySQLSettings
putStr "Connecting to MySQL database..."
mysqlConn <- connectODBC $ buildMySQLConnectionString mysqlSettings
putStrLn "Connected"
_ <- run mysqlConn "USE np" []
putStrLn " Done."
The database connects fine but subsequently when it runs an SQL query (_ <- run mysqlConn "USE np" []) I get the following error.
SqlError {seState = "", seNativeError = -1, seErrorMsg = "Tried to use a disposed ODBC Connection handle"}
To my understanding it seems like the error says that the connection gets immediately freed as soon as it's created. This problem only happens when connecting to a remote database (Amazon RDS in this case) and does not for my local MySQL instance.
I guess this happens when you're using an ODBC driver which internally uses libmysqlclient. The thing is, libmysqlclient doesn't restart system calls interrupted by signals, and GHC runtime uses signals internally.
You can see how you can avoid this by looking at Database.HDBC.MySQL.withRTSSignalsBlocked. Simply put, this function blocks SIGALRM and SIGVTALRM which are used by GHC runtime while executing your code block. You may directly use this function, or you can just copy it if you don't want to depend on HDBC-mysql.
Turns out the problem is in the cpp-options I added blindly to the cabal file.
...
cpp-options: -DDCABAL_BUILD_DEVELOPER
build-depends:
base >=4.14.2.0
...
I have no idea what DDCABAL_BUILD_DEVELOPER is used for. Disabling the cpp-options fixed the issue.
I'm working on the front end of a webapp, and my co-developer is using Pyramid and SQAlchemy. We've just moved from SQLite to MySQL. I installed MySQL 5.6.15 (via Homebrew) on my OS X machine to get the Python MySQLdb install to work (via pip in a virtualenv).
Because in MySQL >= 5.6.5 secure_auth is now ON by default I can only connect to the remote database (pre 5.6.5) with the --skip-secure-auth flag, which works fine in a terminal.
However, in the Python Pyramid code, it only seems possible to add this flag as an argument to create_engine(), but I can't find create_engine() in my co-dev's code, only the connection string below in an initialisation config file. He's not available, this isn't my area of expertise, and we launch next week :(
sqlalchemy.url = mysql+mysqldb://gooddeeds:deeds808letme1now#146.227.24.38/gooddeeds_development?charset=utf8
I've tried appending various "secure auth" strings to the above with no success. Am I looking in the wrong place? Has MySQLdb set secure_auth to ON because I'm running MySQL 5.6.15? If so, how can I change that?
If you are forced to use the old passwords (bah!) when using MySQL 5.6, and using MySQLdb with SQLAlchemy, you'll have to add the --skip-secure-auth to an option file and use URL:
from sqlalchemy.engine.url import URL
..
dialect_options = {
'read_default_file': '/path/to/your/mysql.cnf',
}
engine = create_engine(URL(
'mysql',
username='..', password='..',
host='..', database='..',
query=dialect_options
))
The mysql.cnf would contain:
[client]
skip-secure-auth
For Pyramid, you can do the following. Add a line in your configuration ini-file that holds the connection arguments:
sqlalchemy.url = mysql://scott:tiger#localhost/test
sqlalchemy.connect_args = { 'read_default_file': '/path/to/foo' }
Now you need to change a bit the way the settings are read and used. In the file that launches your Pyramic app, do the following:
def main(global_config, **settings):
try:
settings['sqlalchemy.connect_args'] = eval(settings['sqlalchemy.connect_args'])
except KeyError:
settings['sqlalchemy.connect_args'] = {}
engine = engine_from_config(settings, 'sqlalchemy.')
# rest of code..
The trick is to evaluate the string in the ini file which contains a dictionary with the extra options for the SQLAlchemy dialect.
I am deploying sql express with my application. I will like that database engine to accept remote connections. I know how to configure that manual by launching the sql server configuration manager, enabling tcp/ip connections, specifying the ports etc.. I am wondering if it will be possible to do the same thing from the command line.
Or maybe I will have to create a "SQL Server 2008 Server Project" in visual studio.
EDIT 1
I posted the same question in here but I will like to do the same thing on a instance of sql express that is already installed. Take a look at the question in here
EDIT 2
I found these links that claim on doing something similar and I still cannot make it work.
1) http://support.microsoft.com/kb/839980
2) http://social.msdn.microsoft.com/Forums/en-US/sqlexpress/thread/c7d3c3af-2b1e-4273-afe9-0669dcb7bd02/
3) http://www.sql-questions.com/microsoft/SQL-Server/34211977/can-not-connect-to-sql-2008-express-on-same-lan.aspx
4) http://datazulu.com/blog/post/Enable_sql_server_tcp_via_script.aspx
EDIT 3
As Krzysztof stated in his response I need (plus other things that I know that are required)
1 - enable TCP/IP
I have managed to do this when installing a new instance of SQLExpress passing the parameter /TCPENABLED=1. When I install sql express like in this example. that instance of sql express will have TCP/IP enabled
2 - Open the right ports in the firewall
(I have done this manualy but I belive I will be able to figure it out how to do it with c#)
So far I have to play aroud with this console command:
netsh firewall set portopening protocol = TCP port = 1433 name = SQLPort mode = ENABLE scope = SUBNET profile = CURRENT
3 - Modify TCP/IP properties enable a IP address
I have not been able to figure out how to enable a IP, change a port etc.. I think this will be the step more complicated to solve
4 - Enable mixed mode authentication in sql server
I have managed to do this when installing SQL Express passing the parameter /SECURITYMODE=SQL refer to step 1's link.
SQL Server express requires this authentication type to accept remote connections.
5 - Change user (sa) default passowrd
By default the sa account has a NULL passowrd. In order to accept connections this user must have a password. I changed the default passowrd of the sa with the script:
ALTER LOGIN [sa] WITH PASSWORD='*****newPassword****'
6 - finally
will be able to connecto if all the last steps are satisied as:
SQLCMD -U sa -P newPassword -S 192.168.0.120\SQLEXPRESS,1433
by typing that in the command line: the connection string in C# will be very similar. I will have to replace -U for user , -P for password and -S for data source. I dont recall the exact names.
I tested below code with SQL Server 2008 R2 Express and I believe we should have solution for all 6 steps you outlined. Let's take on them one-by-one:
1 - Enable TCP/IP
We can enable TCP/IP protocol with WMI:
set wmiComputer = GetObject( _
"winmgmts:" _
& "\\.\root\Microsoft\SqlServer\ComputerManagement10")
set tcpProtocols = wmiComputer.ExecQuery( _
"select * from ServerNetworkProtocol " _
& "where InstanceName = 'SQLEXPRESS' and ProtocolName = 'Tcp'")
if tcpProtocols.Count = 1 then
' set tcpProtocol = tcpProtocols(0)
' I wish this worked, but unfortunately
' there's no int-indexed Item property in this type
' Doing this instead
for each tcpProtocol in tcpProtocols
dim setEnableResult
setEnableResult = tcpProtocol.SetEnable()
if setEnableResult <> 0 then
Wscript.Echo "Failed!"
end if
next
end if
2 - Open the right ports in the firewall
I believe your solution will work, just make sure you specify the right port. I suggest we pick a different port than 1433 and make it a static port SQL Server Express will be listening on. I will be using 3456 in this post, but please pick a different number in the real implementation (I feel that we will see a lot of applications using 3456 soon :-)
3 - Modify TCP/IP properties enable a IP address
We can use WMI again. Since we are using static port 3456, we just need to update two properties in IPAll section: disable dynamic ports and set the listening port to 3456:
set wmiComputer = GetObject( _
"winmgmts:" _
& "\\.\root\Microsoft\SqlServer\ComputerManagement10")
set tcpProperties = wmiComputer.ExecQuery( _
"select * from ServerNetworkProtocolProperty " _
& "where InstanceName='SQLEXPRESS' and " _
& "ProtocolName='Tcp' and IPAddressName='IPAll'")
for each tcpProperty in tcpProperties
dim setValueResult, requestedValue
if tcpProperty.PropertyName = "TcpPort" then
requestedValue = "3456"
elseif tcpProperty.PropertyName ="TcpDynamicPorts" then
requestedValue = ""
end if
setValueResult = tcpProperty.SetStringValue(requestedValue)
if setValueResult = 0 then
Wscript.Echo "" & tcpProperty.PropertyName & " set."
else
Wscript.Echo "" & tcpProperty.PropertyName & " failed!"
end if
next
Note that I didn't have to enable any of the individual addresses to make it work, but if it is required in your case, you should be able to extend this script easily to do so.
Just a reminder that when working with WMI, WBEMTest.exe is your best friend!
4 - Enable mixed mode authentication in sql server
I wish we could use WMI again, but unfortunately this setting is not exposed through WMI. There are two other options:
Use LoginMode property of Microsoft.SqlServer.Management.Smo.Server class, as described here.
Use LoginMode value in SQL Server registry, as described in this post. Note that by default the SQL Server Express instance is named SQLEXPRESS, so for my SQL Server 2008 R2 Express instance the right registry key was
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS\MSSQLServer.
5 - Change user (sa) default password
You got this one covered.
6 - Finally (connect to the instance)
Since we are using a static port assigned to our SQL Server Express instance, there's no need to use instance name in the server address anymore.
SQLCMD -U sa -P newPassword -S 192.168.0.120,3456
Please let me know if this works for you (fingers crossed!).
I recommend to use SMO (Enable TCP/IP Network Protocol for SQL Server).
However, it was not available in my case.
I rewrote the WMI commands from Krzysztof Kozielczyk to PowerShell.
# Enable TCP/IP
Get-CimInstance -Namespace root/Microsoft/SqlServer/ComputerManagement10 -ClassName ServerNetworkProtocol -Filter "InstanceName = 'SQLEXPRESS' and ProtocolName = 'Tcp'" |
Invoke-CimMethod -Name SetEnable
# Open the right ports in the firewall
New-NetFirewallRule -DisplayName 'MSSQL$SQLEXPRESS' -Direction Inbound -Action Allow -Protocol TCP -LocalPort 1433
# Modify TCP/IP properties to enable an IP address
$properties = Get-CimInstance -Namespace root/Microsoft/SqlServer/ComputerManagement10 -ClassName ServerNetworkProtocolProperty -Filter "InstanceName='SQLEXPRESS' and ProtocolName = 'Tcp' and IPAddressName='IPAll'"
$properties | ? { $_.PropertyName -eq 'TcpPort' } | Invoke-CimMethod -Name SetStringValue -Arguments #{ StrValue = '1433' }
$properties | ? { $_.PropertyName -eq 'TcpPortDynamic' } | Invoke-CimMethod -Name SetStringValue -Arguments #{ StrValue = '' }
# Restart SQL Server
Restart-Service 'MSSQL$SQLEXPRESS'
Is there a way to connect to a network drive that requires a different username/password than the username/password of the user running the package?
I need to copy files from a remote server. Right now I map the network drive in Windows Explorer then do I filesystem task. However, eventually this package will be ran automatically, from a different machine, and will need to map the network drive on its own. Is this possible?
You can use the Execute Process task with the "net use" command to create the mapped drive. Here's how the properties of the task should be set:
Executable: net
Arguments: use \Server\SomeShare YourPassword /user:Domain\YourUser
Any File System tasks following the Execute Process will be able to access the files.
Alternative Method
This Sql Server Select Article covers the steps in details but the basics are:
1) Create a "Execute Process Task" to map the network drive (this maps to the z:)
Executable: cmd.exe
Arguments: /c "NET USE Z: "\\servername\shareddrivename" /user:mydomain\myusername mypassword"
2) Then run a "File System Task" to perform the copy. Remember that the destination "Flat File Connection" must have "DelayValidation" set to True as z:\suchandsuch.csv won't exist at design time.
3) Finally, unmap the drive when you're done with another "Execute Process Task"
Executable: cmd.exe
Arguments: /c "NET USE Z: /delete"
Why not use an FTP task to GET the files over to the local machine? Run SSIS on the local machine. When transferring using FTP in binary, its real fast. Just remember that the ROW delimter for SSIS should be LF, not CRLF, as binary FTp does not convert LF (unix) to CRLF (windows)
You have to map the network drive, here's an example that I'm using now:
profile = "false"
landingPadDir = Dts.Variables("strLandingPadDir").Value.ToString
resultsDir = Dts.Variables("strResultsDir").Value.ToString
user = Dts.Variables("strUserName").Value.ToString
pass = Dts.Variables("strPassword").Value.ToString
driveLetter = Dts.Variables("strDriveLetter").Value.ToString
objNetwork = CreateObject("WScript.Network")
CheckDrive = objNetwork.EnumNetworkDrives()
If CheckDrive.Count > 0 Then
For intcount = 0 To CheckDrive.Count - 1 Step 2 'if drive is already mapped, then disconnect it
If CheckDrive.Item(intcount) = driveLetter Then
objNetwork.RemoveNetworkDrive(driveLetter)
End If
Next
End If
objNetwork.MapNetworkDrive(driveLetter, landingPadDir, profile, user, pass)
From There just use that driveLetter and access the file via the mapped drive.
I'm having one issue (which led me here) with a new script that accesses two share drives and performs some copy/move operations between the drives and I get an error from SSIS that says:
This network connection has files open or requests pending.
at Microsoft.VisualBasic.CompilerServices.LateBinding.InternalLateCall(Object o, Type objType, String name, Object[] args, String[] paramnames, Boolean[] CopyBack, Boolean IgnoreReturn)
at Microsoft.VisualBasic.CompilerServices.NewLateBinding.LateCall(Object Instance, Type Type, String MemberName, Object[] Arguments, String[] ArgumentNames, Type[] TypeArguments, Boolean[] CopyBack, Boolean IgnoreReturn)
at ScriptTask_3c0c366598174ec2b6a217c43470f581.ScriptMain.Main()
This is only on the "2nd run" of the process and if I run it a 3rd time it all works fine so I'm guessing the connection isn't being properly closed or it is not waiting for the copy/move to complete before moving forward or some such, but I'm unable to find a "close" or "flush" command that prevents this error. If you have any solution, please let me know, but the above code should work for getting the drive mapped using your alternate credentials and allow you to access that share.
Zach
To make the package more robust, you can do the following;
In the first Execute Process Task, set - FailTaskIfReturnCodeNotSuccessValue = False
This will let the package run if the last disconnect has not worked.
This is an older question but more recent versions of SQL Server with SSIS databases allow you to use a proxy to execute SQ Server jobs.
In SSMS Under Security<Credentials set up a credential in the database mapped to the AD account you want to use.
Under SQL Server Agent create a new proxy giving it the credential from step 1 and permissions to execute SSIS packages.
Under the SQL Server Agent jobs create a new job that executes your package
Select the step that executes the package and click EDIT. In the Run As dropdown select the Proxy you created in step 2
I want to connect MATLAB with MYSQL.I dont know the procedure.In MATLAB help it says about some drivers which confuses me alot.Can someone please guide me!please tell me the complete process.I shall be very very thankful!!!
I use JDBC to connect from MATLAB to mySQL database. Works seamlessly.
First download JDBC driver for mySQL from here:
http://www.mysql.com/downloads/connector/j/
Unpack mysql-connector-java-x.x.xx-bin.jar (the latest version) file from the archive into a folder
In the beginning of your script add the path to this jar file, then you can connect to a database and so on.
Here is an example of connecting to and querying public human genome database:
%# add path to the JAR file you just installed to Java dynamic classpath
javaaddpath('h:\Documents\MATLAB\myJavaClasses\mysql-connector-java-5.1.12-bin.jar')
%# connection parameteres
host = 'genome-mysql.cse.ucsc.edu';
user = 'genome';
password = '';
dbName = 'hg18';
%# JDBC parameters
jdbcString = sprintf('jdbc:mysql://%s/%s', host, dbName);
jdbcDriver = 'com.mysql.jdbc.Driver';
%# Create the database connection object
conn = database(dbName, user , password, jdbcDriver, jdbcString);
gene = 'NF1';
if isconnection(conn) % check to make sure that we successfully connected
qry = sprintf('SELECT geneName, chrom, txStart, txEnd FROM refFlat WHERE geneName=''%s''',gene);
rs = fetch(exec(conn, qry));
rsdata = get(rs, 'Data');
end