Dropping MySQL table with SparkSQL - mysql

Question

I know that we can query or create a Mysql table from SparkSQL through the below commands.
val data = sqlContext.read.jdbc(urlstring, tablename, properties)
data.write.format("com.databricks.spark.csv").save(result_location)
val dataframe = sqlContext.read.json("users.json")
dataframe.write.jdbc(urlstring, table, properties)
Like that is there any way to drop a table ?

Answer 1

You can try a basic DROP operation with the JDBC driver :
val DB_URL: String = ???
val USER: String = ???
val PASS: String = ???
def dropTable(tableName: String) = {
import java.sql._;
var conn: Connection = null;
var stmt: Statement = null;
try {
Class.forName("com.mysql.jdbc.Driver");
println("Connecting to a selected database...");
conn = DriverManager.getConnection(DB_URL, USER, PASS);
println("Connected database successfully...");
println("Deleting table in given database...");
stmt = conn.createStatement();
val sql: String = s"DROP TABLE ${tableName} ";
stmt.executeUpdate(sql);
println(s"Table ${tableName} deleted in given database...");
} catch {
case e: Exception => println("exception caught: " + e);
} finally {
???
}
}
dropTable("test")
You can do that with Spark using JDBCUtils but this is quite straightforward.

Answer 2

You can have a look at the write mode method
dataframe.write.mode('overwrite').jdbc(urlstring, table, properties)
Overwrite mode means that when saving a DataFrame to a data source, if data/table already exists, existing data is expected to be overwritten by the contents of the DataFrame.
from: https://spark.apache.org/docs/latest/sql-data-sources-load-save-functions.html#save-modes
Also, you can put in the properties to truncate if you don't want to delete the definition too.
This is a JDBC writer-related option. When SaveMode.Overwrite is enabled. This option causes Spark to truncate an existing table instead of dropping and recreating it. This can be more efficient and prevents the table metadata (e.g., indices) from being removed. However, it will not work in some cases, such as when the new data has a different schema. It defaults to false. This option applies only to writing.
from: https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html