I'm trying to connect to a Google Cloud SQL second generation in Python from AppEngine standard (Python 2.7).
Until now, I was using MySQLDB driver directly and it was fine.
I've tried to switch to SQLAlchemy, but now I'm always having this error when the code is deployed (it seems to work fine in local) resulting in a error 500 (It's not just some connections which are lost, it constantly fails) :
OperationalError: (_mysql_exceptions.OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 38") (Background on this error at: http://sqlalche.me/e/e3q8)
I don't understand because the setup doesn't differ from before, so it must be related to the way I use SQLAlchemy.
I use something like this :
create_engine("mysql+mysqldb://appuser:password#x.x.x.x/db_name?unix_socket=/cloudsql/gcpProject:europe-west1:instanceName")
I've tried different values (with, without the ip, ...). But it is still the same. Is is a version compatibility problem ?
I use
MySQL-python in the app.yaml and SQLAlchemy 1.2.4 :
app.yaml :
- name: MySQLdb
version: "latest"
requirements.txt :
SQLAlchemy==1.2.4
It was a problem in the url. I was adding in a specific part of the code "/dbname" at the end of the connection string, resulting in something like this :
mysql+mysqldb://appuser:password#/db_name?unix_socket=/cloudsql/gcpProject:europe-west1:instanceName/dbname
So in the end, the meaning of this error can also be that the unix socket is wrong.
There are a number of causes for connection loss to Google CloudSQL server but quite rightly, you have to ensure that your setup is appropriate first. I don't think this issue is about version compatibility.
According to the documentation, for your application to be able to connect to your Cloud SQL instance when the app is deployed, you require to add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the app.yaml file(Your displayed app.yaml does not seem to contain these environment variables).
I recommend you review the details in the link for details on how to set up your CloudSQL instance and connecting to the instance.
Related
I have an application in .net core 3.1 and I am using MySql with that. To connect to MySql I am using Pomelo.EntityFrameworkCore.MySql nuget.
My .net core application is deployed as a docker container on AWS ECS and I am using Aurora MySql RDS to store data. I have granted "Publicly accessible" access to the Aurora MySql and I can connect to the db using MySql workbench and also with my .net core application using localhost. But when I deploy the application and try to perform any db action then it starts giving throwing exception:
An exception has been raised that is likely due to a transient
failure. Consider enabling transient error resiliency by adding
'EnableRetryOnFailure()' to the 'UseMySql' call.
Then I have added retry pattern like this:
services.AddDbContextPool<DataContext>(options =>
options.UseMySql(Configuration.GetConnectionString("DefaultConnection"), builder =>
{
builder.EnableRetryOnFailure(5, TimeSpan.FromSeconds(5), null);
}
);
and my connection string is something like this:
"DefaultConnection": "Server=db-cluster-1-instance-1.cqb2fsjwx78p.us-east-2.rds.amazonaws.com;Database=dbName;User ID=admin;Password=password;port=3306"
After adding retry pattern. I am getting this error:
"Maximum number of retries (5) exceeded while executing database
operations with 'MySqlRetryingExecutionStrategy'. See inner exception
for the most recent failure."
So I suspect, it's something else. What am I doing wrong here? Or it might be something wrong at aws side
So the issue was related to docker image I was pulling.
I was using mcr.microsoft.com/dotnet/core/sdk:3.1. Changing it to mcr.microsoft.com/dotnet/core/sdk:3.1-bionic worked.
https://github.com/dotnet/SqlClient/issues/222
Add SslMode=None to your connection string and see if the connection works (see MySqlConnector's Connection String Options). The transient exception just means, that Pomelo (i.e. MySqlConnector) was unable to connect to the database server.
So this is related to connection problems (either fixable by altering the connection string or by changing your Aurora/firewall configuration).
Backstory, I would like to build shiny apps to give to some of our data collectors so they can review what has been collected. We currently house all of our data in a cloud based MySQL server. Ideally, I would like the shiny app to pull data directly from the MySQL server so it can be fully automated without any data pulls and up 24/7.
I have been trying to first just build the connection between R and MySQL using the RMySQL package and can't seem to get it working. I have set up a specific username/password for this connection that is read only(however I have also tried my regular username which has all privileges granted). This is the code I am running;
mydb=dbConnect(
MySQL(),
user='myuser',
password='mypass',
dbname='vgtg',
host='ipaddress',
port=3306,
)
Obviously the 'ipaddress' of the server has been changed for the sake of posting here but it is a generic looking address like
'192.168.1.1'
When I run the code above I get this error message;
Error in .local(drv, ...) :
Failed to connect to database: Error: Can't connect to MySQL server on
'ipaddress' (0)
I have tried looking for previous questions posted but none seem to be exactly this error message that I am receiving. It makes me think that for some reason RMySQL is looking locally for the server when it is actually a cloud based, remote, server.
Also, is there anything more I should set up server side to allow the connection? I do have a server admin to help out but I am not sure how familiar he is with R and likewise I am not particularly familiar with working with servers. He has opened port 3306 for me and is able to see my attempts to connect through the port.
Your syntax is correct with the exception of
port=3306,
You need to drop the comma. That said, the error you received is unrelated to the syntax.
Without knowing the details of your setup, it is hard to diagnose. Where does the MySQL DB reside? For example, if it is on an AWS RDS instance, then the host isn't a standard IP address, it is something like this
mydb.cm1abc2v4mod.us-west-1.rds.amazonaws.com
Assuming that the IP address you used is correct, then the problem is most likely on the server. You need to ensure that port 3306 is open to traffic. Otherwise, R will not be able to connect to the DB.
I am working with ServiceStack evaluating the ORMLite provider with MySql for use on a.NET Core 1.1 project. I am running into a curious issue I am not sure what the problem is and looking for some guidance on how to troubleshoot this further or possible fixes.
I am running the .Net Core app in a standard docker container and MySql in the standard MySql container. I have been able to successfully run MySQL and connect with the workbench tool.
What I cannot do is get the executing code of the web application to successfully connect to the MySQL container. I have tried various combinations. Those where the IP Address or hostname were wrong I received a meaningful error saying the host was unknown.
But under scenarios where server name or IP address values that are used that are appropriate for the MySQL container, I receive a generic MySQLException error. The username and password are correct. I have even used the root account to ensure that there aren't any potential security hangups.
Here is a code snippet of where I am attempting to connect. This setup was working well using a local SQL db file, there is just something I am missing with either the configuration of Docker or MySQL that I haven't been able to isolate.
const string connectionString = "Server=3400f112c973:3306;Database=ServiceDB;Uid=root;Pwd=my-secret-pw;";
container.Register<IDbConnectionFactory>(
new OrmLiteConnectionFactory(connectionString, MySqlDialect.Provider ));
using (var db = container.Resolve<IDbConnectionFactory>().Open())
{
if (db.CreateTableIfNotExists<TypeExample>())
{
//Add seed data
}
}
I have figured out my scenario.
In the MySQL Connection string, if you are using a non-standard port, you need to use the port parameter to specify the different port and omit the port entirely if using the standard port.
My issue was I was always using "Server=servername:port;" which is an incorrect format and should have been "Server=servername;Port=port;" in the cases where I was using a nonstandard port number, otherwise Port= can be omitted.
I have seen many stack overflows questions and some blogs tried workarounds, but nothings helped - hence re-posting the question with more details.
I am seeing the weird behaviour with MySQL and Python application, details are as follows:
1) My application works perfectly fine with MySQL (tried and tested on many platforms) but on this particular machine it fails to connect to MySQL.
structure of application is :
Windows service -> parent process -> Mysql(child process)
and when application tries to connect to MySQL it get this error:
ERROR 2013 , Lost connection to MySQL server at 'waiting for initial communication packet' - system error 0
I tried:
- connect_timeout=300
- skip-name-resolve=0
- firewall is OFF
- use 17.0.0.1, localhost , IP of machine to connect to but it still fails with same error.
2) Now the weird thing is -
If I manually follow all the steps which application does, It works perfectly fine, details are follows:
a) Start MySQL with same command (which application uses) with administrator privileges
mysql --default-file = xxx --basedir =xxx
b) Connect with same credentials ( -u root -P 6075 -h 127.0.0.1) and
It works perfectly fine, I double checked all the steps which application does, there is no difference between manually steps and application code.
AM I missing something here ? Any suggestions ?
MySQL version : 5.5.35
Python : 2.7
Base OS : Windows 2012 R2
Thanks in advance..
Found a reason - answering my question:
When I used to run MySQL from my application - it was running under system user privileges - so it used to pick "C:\WINDOWS\TEMP" as a temp directory- this directory was messed up - has lot of unnecessary files .. and MySQL was stuck while processing files under this directory...
But when I ran it manually under My administrator account it was using his temp directory... C:\Users\USER_NAME\AppData\Local\Temp and everything was working like magic...
To fix this permanently I changed tmp directory through MySQL conf file and now My Application runs like the Wind.... :)
[mysqld]
tmpdir = 'PATH_TO_THE_DIRECTORY'
I was getting this same error trying to set up a SQL Server Linked Server
Cannot initialize the data source object of OLE DB provider "MSDASQL" for linked server "DBLINKED".
OLE DB provider "MSDASQL" for linked server "DBLINKED" returned message "[MySQL][ODBC 8.0(w) Driver]Lost connection to MySQL server at 'waiting for initial communication packet', system error: 10060". (Microsoft SQL Server, Error: 7303)
You mentioned it in your initial question - the Connection Timeout was the issue for me.
The default is 0 - raised it to 300. I thought default of 0 would mean no timeout, but it's obviously something reasonably short, and I was trying to connect to a remote database on a slow internet connection. A lot of other question and answers out there relate to connecting within the same machine, so this error isn't reported much.
I downloaded and installed the driver setup file, DB2OLEDB.exe, from here:
http://download.microsoft.com/mwg-internal/de5fs23hu73ds/progress?id=HYLbKUfGNl
Using the connection string that worked on another PC, I tried to create a Connection Object in an SSIS package. When I tested the connection I got this error:
Test connection failed because of an error in initializing provider. A TCPIP socket error has occurred (10057): A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
Any suggestions on what the cause of this error is and how I might resolve this issue?
By the way, when I use the DB2 Configuration set up utility and test a connection from within that, I am able to successfully connect.
What other info can I provide to help you answer this question?
Thank you
Could this be related to a blocked port?
If you follow all the steps illustrated here: http://www.bidn.com/blogs/PatrickLeBlanc/ssis/700/connecting-to-db2-using-ssis do you still get the same result?
Maybe a silly question, did you restart the computer after the installation?
Are you an admin user on one machine and not on the other?
You could try to verify the port connectivity with a quick telnet command:
telnet your-db-host your-db-listening-port
If it connects, that one is off the list.
Doing some research I've found two possible fixes.
The first link suggests calling BeginReceive after the EndAccept logic is complete. Are you using script code, or just using the GUI without any scripting?
TCP async sockets throwing 10057
The second link points to drivers / software on the PC. It could be that you are missing a windows update or have faulty hardware / drivers.
I think this is less likely the case since you could connect to a different machine with the same connection string(?). Can you verify this is a valid statement?
http://social.msdn.microsoft.com/Forums/en-US/1bc3df95-c86d-4d25-aa20-30f61ed00c63/odd-socket-errors
If you could show the connection strings used for both the working and non working, and give a little more detail about The "Other PC" in comparison to the non-working PC... that would be helpful =]
If neither of the posts I've linked are the solution, this specific Google search has proven to yield some seemingly helpful results
"socket" "10057" "no address was supplied."