I have been working on a java application that has a connection to a mysql database. I can connect and run queries but when I try to take a string and run it as a sql query I get this error.
Exception in thread "main" com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO usageData (`COMID`,`Year`,`Month`,`kwhr`,`co2`)
VALUES ('15650', '2' at line 3
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.Util.getInstance(Util.java:384)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3566)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3498)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2113)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2562)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1664)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1583)
Bellow is the code that I'm using to connect to the database
public static void main(String[] args) throws Exception {
String dbms = "mysql";
String serverName = "localhost";
String portNumber = "8889";
String DBName = "ConnectDatabase";
String user = "root";
String password = "root";
ArrayList<Integer> yearList = new ArrayList<Integer>();
ArrayList<CustomerRecord> customerRecordList = getCustomerRecords(args[0], yearList);
ArrayList<ClimateRecord> climateRecordList = getClimateRecords(args[1]);
StringBuffer buf = new StringBuffer();
for (CustomerRecord record : customerRecordList) {
buf.append(customerRecord2SQL(record));
}
for (int i = (climateRecordList.size() - 1); i >= 0; i--) {
//for (ClimateRecord record : climateRecordList) {
ClimateRecord record = climateRecordList.get(i);
buf.append(climateRecord2SQL(record));
}
buf.append(cityStats(dbms,serverName,portNumber,DBName,user,password));
buf.append(zipStats(dbms,serverName,portNumber,DBName,user,password));
System.out.println(buf.toString());
//here is the code to go ahead and update the database
Connection con = null;
con = DriverManager.getConnection("jdbc:" + dbms + "://" + serverName + ":" + portNumber + "/" + DBName + "?user="+user+"&password=" + password);
Statement stmt = con.createStatement();
stmt.executeUpdate(buf.toString());
BufferedWriter out = new BufferedWriter(new FileWriter(args[2]));
out.write(buf.toString());
out.close();
}
This is not a connection issue. It means that something is wrong with your SQL statement. Try copying the statement as is and executing it directly in the database. See what is wrong and then correct it in the Java code.
I notice that the single quotes around the field names look fancy. That might be a problem.
Here's what was happening and what everyone else failed to realize. JDBC will not do more than one query at a time I was trying to run several million queries contained in a string buffer which wouldn't work.
Your query should be 'INSERT INTO usageData VALUES (COMID,Year,Month,kwhr,co2);
and btw COMID,year...what are these..?? and why fancy quotes...??
Related
My databricks notebook randomly gives error 107: Transport connection error connection timed out for MySQL server. Once in 50 runs the system receives this type of an error.
From databricks I am trying to access the SQL server. I am only using one SQL server. The SQL server has configurations stored in tables that I need in data ricks to do compute. Also audit tables are present on SQL server, so through databricks I'm inserting records in SQL database after every process.
When I reached out to Microsoft with this error they suggested to use connection pooling to mitigate the error. Will connection pooling help or should I try something else?
The code used to connect to MySQL is below:
def execute_query_mysql(self, query, get_single_result=None):
""" Module for executing queries"""
mysql_connection = None
try:
status_message = "Starting function to execute a MySQLquery : "+str(query)
logger.info(status_message, extra=self.execution_context.get_context())
if query:
query = query.strip()
mysql_connection = self.get_my_sql_connection()
cursor = mysql_connection.cursor(dictionary=True)
status_message = "Created connection to MySQL"
#logger.debug(status_message, extra=self.execution_context.get_context())
status_message = "Executing query on MySQL"
#logger.debug(status_message, extra=self.execution_context.get_context())
if query.lower().startswith("select"):
cursor.execute(query)
if get_single_result:
result = cursor.fetchone()
else:
result = cursor.fetchall()
elif query.lower().startswith("insert into"):
result = cursor.execute(query)
if result != 0:
result = cursor.lastrowid
cursor.execute("commit")
else:
cursor.execute(query)
cursor.execute("commit")
result = ""
#closing Mysql connections
mysql_connection.close()
cursor.close()
status_message = "Executed query on MySQL with result : " + str(result)
logger.debug(status_message, extra=self.execution_context.get_context())
return result
except Exception as exception:
error = "ERROR in " + self.execution_context.get_context_param("current_module") + \
" ERROR MESSAGE: " + str(traceback.format_exc())
self.execution_context.set_context({"traceback": error})
logger.error(error, extra=self.execution_context.get_context())
raise exception
def get_my_sql_connection(self):
"""Module for getting mySQL connection"""
try:
status_message = "Starting function to fetch my-sql connection"
logger.info(status_message, extra=self.execution_context.get_context())
secret = json.loads(json.dumps(self.configuration.get_configuration([CommonConstants.ENVIRONMENT_PARAMS_KEY, "di_keys"])))
host = secret["mysql_host"]
user = secret["mysql_username"]
encoded_password = secret["mysql_password"]
password = self.decrypt_value(encoded_password)
port = secret["mysql_port"]
db = secret["mysql_db"]
ssl_ca_file_path = self.configuration.get_configuration(
[CommonConstants.ENVIRONMENT_PARAMS_KEY, "mysql_certificate_file_path"])
if len(host) == 0 or len(user) == 0 or len(password) == 0 or len(db) == 0 \
or len(port) == 0:
status_message = "Invalid configurations for " + MODULE_NAME + " in " + \
CommonConstants.ENVIRONMENT_CONFIG_FILE
logger.error(status_message, extra=self.execution_context.get_context())
raise Exception
if ssl_ca_file_path:
mysql_connection = connection.MySQLConnection(
host=host, user=user, passwd=password, db=db, port=int(port),
ssl_verify_cert=True, ssl_ca=ssl_ca_file_path, connect_timeout=6000)
else:
mysql_connection = connection.MySQLConnection(
host=host, user=user, passwd=password, db=db, port=int(port))
status_message = "Connection to MySQL successful"
return mysql_connection
except Exception as exception:
error = "ERROR in " + self.execution_context.get_context_param("current_module") + \
" ERROR MESSAGE: " + str(exception)
self.execution_context.set_context({"traceback": error})
logger.error(error, extra=self.execution_context.get_context())
raise exception
def decrypt_value(self, encoded_string):
"""
Purpose : This method decrypts encoded string
Input : encoded value
Output : decrypted value
"""
try:
status_message = "Started decoding for value:" + encoded_string
#logger.debug(status_message, extra=self.execution_context.get_context())
decoded_string = base64.b64decode(encoded_string).decode()
status_message = "Completed decoding value"
#logger.info(status_message, extra=self.execution_context.get_context())
return decoded_string
except Exception as exception:
status_message = "Error occured in decrypting value"
error = "ERROR in " + self.execution_context.get_context_param("module_name") + \
" ERROR MESSAGE: " + str(traceback.format_exc())
self.execution_context.set_context({"traceback": error})
logger.error(status_message, extra=self.execution_context.get_context())
raise exception
I have this code for connect to MySQL through a SSH, inside of a python class:
def executeQuery(self, query_string):
print("connecting to database " + self.sql_main_database)
with SSHTunnelForwarder(
(
self.ssh_host,
self.ssh_port),
ssh_username = self.ssh_user,
ssh_pkey = self.pkey,
remote_bind_address=(self.sql_hostname, self.sql_port)
) as tunnel:
print("performing connection")
conn = pymysql.connect(
host="127.0.0.1",
user=self.sql_username,
password=self.sql_password,
db=self.sql_main_database,
port=tunnel.local_bind_port)
query = query_string
print("Querying")
data = pd.read_sql_query(query, conn)
print("Done!")
conn.close()
return data
The code is working well, but when the query is not well defined, the notebook freezes.
Then, I tried to use a try/catch, and the code ended like this
def executeQuery(self, query_string):
try:
with SSHTunnelForwarder(
(
self.ssh_host,
self.ssh_port
),
ssh_username = self.ssh_user,
ssh_pkey = self.pkey,
remote_bind_address=(self.sql_hostname, self.sql_port)
) as tunnel:
try:
conn = pymysql.connect(
host = "127.0.0.1",
user = self.sql_username,
password = self.sql_password,
db = self.sql_main_database,
port = tunnel.local_bind_port
)
try:
query = query_string
data = pd.read_sql_query(query, conn)
return data
except DatabaseError as e:
Log.debug(self,str(e))
raise DatabaseError
except pymysql.err.InternalError as e:
Log.debug(self, str(e))
raise DataError
except Exception as e:
Log.debug(self, "[Error]Setting up database: \'" + self.sql_main_database + "\'")
raise DataError
The issue is that pd.read_sql_query never stops so the except is never called, the try won't fail, and the process will just continue forever
The timeout workaround is not possible, because the queries don't have defined execution times and some of them can stay in processing during a couple of hours.
I'm not sure how to fix it.
Indeed the problem was not on the connector, just updating the jupyter version was needed.
I have a function that receives parameters (schema name, column name etc) and updates a Mysql table, the problem is that when I use two Mysql commands inside this function (below), one to set the schema and one to update the table, the Close connection command at the end `(conDataBase3->Close();) does not work.
I am checking the number of open connections in the Mysql console (SHOW FULLPROCESSLIST) before and after running the function. any solutions or explanations? thanks
int simple_1::update_table_with_value(gcroot<String^ > schema, gcroot<String^ > table_name, int numerator, gcroot<String^ > field_to_update, double value_to_update)
{
gcroot<MySqlConnection^ > conDataBase3;
conDataBase3 = gcnew MySqlConnection(constring);
conDataBase3->Open();
try{
String ^ schema_name = "Use " + schema + " ;";
MySqlCommand ^cmdDataBase3 = gcnew MySqlCommand(schema_name, conDataBase3);
MySqlCommand ^cmdDataBase4 = gcnew MySqlCommand(schema_name, conDataBase3);
cmdDataBase3->ExecuteNonQuery();
String ^ temp1 = "UPDATE ";
String ^ temp2 = table_name;
String ^ temp3 = " SET ";
String ^ temp4 = field_to_update;
String ^ temp6 = "=(#value1) WHERE numerator = (#value2)";
String ^ temp8 = temp1 + temp2 + temp3 + temp4 + temp6;
// end of the writing part
cmdDataBase4 = gcnew MySqlCommand(temp8, conDataBase3);
cmdDataBase4->Parameters->AddWithValue("#value1", value_to_update);
cmdDataBase4->Parameters->AddWithValue("#value2", numerator);
cmdDataBase4->Prepare();
cmdDataBase4->ExecuteNonQuery();
}//try
catch (Exception^ ex)
{
System::Windows::Forms::MessageBox::Show(ex->Message);
}
conDataBase3->Close();
int answer = 0;
return (answer);
}
ok found the answer, I needed to disable the pooling option otherwise closing the connection still keeps the socket open..
found it here: http://bugs.mysql.com/bug.php?id=24138 see the last 2 lines.
The way to disable the pooling :
http://www.connectionstrings.com/mysql-connector-net-mysqlconnection/disable-connection-pooling/
I am writing a scala application using the Playframework. I don't want to write the database configuration in the application.conf file. Instead I have a function similar to the one given below. I want to send the configuration parameters as arguments of a function. Then I want create a connection and execute the statement. BTW, the database is a mysql database.
def run_sql(sql: String, host: String, user: String, pass: String, port: String, dbname: String): Unit = {
// **** create the connection ****
DB.withConnection {
conn =>
{
val statement = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)
try {
statement.execute(sql)
}
catch {
case e: Exception => Logger.warn("Error executing DML %s. %s".format(sql, e.getMessage))
}
}
}
}
Is it possible? If so then how? I hope I have made my question clear. If you have any confusion please ask. Thanks in advance.
// **** create the connection ****
here we can write
val driver = Play.current.configuration.getString("rds.driver").getOrElse("")
val host = Play.current.configuration.getString("rds.host").getOrElse("")
val port = Play.current.configuration.getString("rds.port").getOrElse("")
val user = Play.current.configuration.getString("rds.user").getOrElse("")
val password = Play.current.configuration.getString("rds.password").getOrElse("")
val url = "mysql://" + user + ":" + password + "#" + host + ":" + port + "/" + dbname
// there's probably a better way to do this
Class.forName(driver)
val connection: Connection = DriverManager.getConnection(url, user, password)
I try to use this code as a console application so that I can back up mydatabase automatics
I am not sure what wrong with it
static void Main(string[] args)
{
try
{
DateTime backupTime = DateTime.Now;
int year = backupTime.Year;
int month = backupTime.Month;
int day = backupTime.Day;
int hour = backupTime.Hour;
int minute = backupTime.Minute;
int second = backupTime.Second;
int ms = backupTime.Millisecond;
String tmestr = backupTime.ToString();
tmestr = "C:\\" + year + "-" + month + "-" + day + "-" + hour + "-" + minute + ".sql";
StreamWriter file = new StreamWriter(tmestr);
ProcessStartInfo proc = new ProcessStartInfo();
string cmd = string.Format(#"-u{0} -p{1} -h{2} {3} > {4};", "root", "", "localhost", "dbfile", "backup.sql");
proc.FileName = "mysqldump";
proc.RedirectStandardInput = false;
proc.RedirectStandardOutput = true;
proc.Arguments = cmd;//"-u root -p smartdb > testdb.sql";
proc.UseShellExecute = false;
Process p = Process.Start(proc);
string res;
res = p.StandardOutput.ReadToEnd();
file.WriteLine(res);
p.WaitForExit();
file.Close();
}
catch (IOException ex)
{
MessageBox.Show("Disk full or other IO error , unable to backup!");
}
}
Since I'm not sure what kind of error you're geting, atleast this could be changed.
string cmd = string.Format(#"-u{0} -p{1} -h{2} {3} > {4};", "root", "", "localhost", "dbfile", "backup.sql");
Later you commented it should be -u root -p smartdb > testdb.sql";
except the above, is missing the space after the -u so I'd change it to:
string cmd = string.Format(#"-u {0} -p {1} -h {2} {3} > {4};", "root", "", "localhost", "dbfile", "backup.sql");
Why are you creating a StreamWriter and trying to get the output of mysqldump, Why not just tell mysqldump to write to the backup file itself?
string cmd = string.Format(#"-u{0} -p{1} -h{2} {3} > ""{4}"";",
"root", "", "localhost", "dbfile", tmestr);
for starters i would change how you're outputting the information. You're essentially opening a file so you can redirect the backup utility's output to that file. The > in your process call is for that very purpose. Just change your "backup.sql" parameter to tmestr.
Also, because the output is already being redirected to a file, your process won't have anything to return. But because we're now having it dump to the correct path, this should be irrelevant.
One final change, add a space between your -u{0} so it's -u {0}. And the same with -h{2}.
In summary:
remove StreamWriter and all variables associated to it
modify the String.Format in your process arguments to use `tmestr
add spaces in your cmd variable for -u&-h
And you should be good-to-go (assuming it's not a problem with locating the mysqldump executable.