Execute MySQL script in grails app - mysql

I have an MySQL Script I want to execute in a controller when my Grails 3.0.9 application is running. I've tried it this way:
import groovy.sql.Sql
import grails.util.Holders
def void clearDatabase() {
String sqlFilePath = 'path/to/file/clear_database.sql'
String sqlString = new File(sqlFilePath).text
def sql = Sql.newInstance(Holders.config.dataSource.url, Holders.config.dataSource.username, Holders.config.dataSource.password, Holders.config.dataSource.driverClassName)
sql.execute(sqlString)
}
Thats how my clear_database.sql file looks like:
SET FOREIGN_KEY_CHECKS = 0;
TRUNCATE table_a;
TRUNCATE table_b;
TRUNCATE table_c;
SET FOREIGN_KEY_CHECKS = 1;
Thats the error message I get:
WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper - SQL Warning Code: 1064, SQLState: 42000
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TRUNCATE table_a;
Is my MySQL syntax wrong or am I missing something else?
edit:
When I run the script manually it works. So I think the script is correct but the way I execute it not.

The problem was that sql.execute(sqlString) wants a GString and I've given it a normal String so it added quotes and the result was an incorrect MySQL syntax like described here.
That's how it works for me:
def sql = Sql.newInstance(Holders.config.dataSource.url, Holders.config.dataSource.username, Holders.config.dataSource.password, Holders.config.dataSource.driverClassName)
sql.execute "SET FOREIGN_KEY_CHECKS = 0;"
sql.execute "truncate ${Sql.expand("table_a")}"
sql.execute "truncate ${Sql.expand("table_b")}"
sql.execute "truncate ${Sql.expand("table_c")}"
sql.execute "SET FOREIGN_KEY_CHECKS = 1;"
There is no clear_database.sql file needed anymore.

Not sure but could you try: as your clear_database.sql
SET FOREIGN_KEY_CHECKS = 0;
TRUNCATE TABLE table_a;
TRUNCATE TABLE table_b;
TRUNCATE TABLE table_c;
SET FOREIGN_KEY_CHECKS = 1;
Good luck!!

Related

Saving dbplyr query (tbl_sql object) to MySQL without saving data locally

This question expands on this question
Here, I'm using the custom function created by #Simon.S.A. shown in the answer to this question. I'm attempting to save a tbl_sql object in R to MySQL as a new table without first saving it locally. Here, the database and schema in my MySQL are named "test." The tbl_sql object in R is my_data, and I want to save this is a new table in MySQL labeled "car_data".
library(DBI)
library(tidyverse)
library(dbplyr)
#establish connection and import data from MySQL
con <- DBI::dbConnect(RMariaDB::MariaDB(),
dbname = "test",
host = "127.0.0.1",
user = "user",
password = "password")
my_data <- tbl(con, "mtcars")
my_data <- my_data %>% filter(mpg >= 22)
# write function to save tbl_sql as a new table in SQL
write_to_database <- function(input_tbl, db, schema, tbl_name){
# connection
tbl_connection <- input_tbl$src$con
# SQL query
sql_query <- glue::glue(
"SELECT *\n",
"INTO {db}.{schema}.{tbl_name}\n",
"FROM (\n",
dbplyr::sql_render(input_tbl),
"\n) AS sub_query"
)
result <- dbExecute(tbl_connection, as.character(sql_query))
}
# execute function
write_to_database(my_data, "test", "test", "car_data")
After running final line, I get the following error. I'm not sure how I can fix this.
Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.test.car_data
FROM (
SELECT *
FROM `mtcars`
WHERE (`mpg` >= 22.0)
) AS sub_quer' at line 2 [1064]
12.
stop(structure(list(message = "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.test.car_data\nFROM (\nSELECT *\nFROM `mtcars`\nWHERE (`mpg` >= 22.0)\n) AS sub_quer' at line 2 [1064]",
call = NULL, cppstack = NULL), class = c("Rcpp::exception",
"C++Error", "error", "condition")))
11.
result_create(conn#ptr, statement, is_statement)
10.
initialize(value, ...)
9.
initialize(value, ...)
8.
new("MariaDBResult", sql = statement, ptr = result_create(conn#ptr,
statement, is_statement), bigint = conn#bigint, conn = conn)
7.
dbSend(conn, statement, params, is_statement = TRUE)
6.
.local(conn, statement, ...)
5.
dbSendStatement(conn, statement, ...)
4.
dbSendStatement(conn, statement, ...)
3.
dbExecute(tbl_connection, as.character(sql_query))
2.
dbExecute(tbl_connection, as.character(sql_query))
1.
write_to_database(my_data, "test", "test", "car_data")
Creating a table with INTO command is an SQL Server (even MS Access) specific syntax and not supported in MySQL. Instead, consider the counterpart statement: CREATE TABLE...SELECT. Also, schema differs between RDBMS's. For MySQL, database is synonymous to schema.
Therefore, consider adjusted version of SQL build:
sql_query <- glue::glue(
"CREATE TABLE {db}.{tbl_name}\n AS \n",
"SELECT * \n",
"FROM (\n",
dbplyr::sql_render(input_tbl),
"\n) AS sub_query"
)

How to write entire dataframe into mySql table in R

I have a data frame containing columns 'Quarter' having values like "16/17 Q1", "16/17 Q2"... and 'Vendor' having values like "a", "b"... .
I am trying to write this data frame into database using
query <- paste("INSERT INTO cc_demo (Quarter,Vendor) VALUES(dd$FY_QUARTER,dd$VENDOR.x)")
but it is throwing error :
Error in .local(conn, statement, ...) :
could not run statement: Unknown column 'dd$FY_QUARTER' in 'field list'
I am new to Rmysql, Please provide me some solution to write entire dataframe?
To write a data frame to mySQL DB you need to:
Create a connection to your database, you need to specify:
MySQL connection
User
Password
Host
Database name
library("RMySQL")
connection <- dbConnect(MySQL(), user = 'root', password = 'password', host = 'localhost', dbname = 'TheDB')
Using the connection create a table and then export data to the database
dbWriteTable(connection, "testTable", testTable)
You can overwrite an existing table like this:
dbWriteTable(connection, "testTable", testTable_2, overwrite=TRUE)
I would advise against writing sql query when you can actually use very handy functions such as dbWriteTable from the RMySQL package. But for the sake of practice, below is an example of how you should go about writing the sql query that does multiple inserts for a MySQL database:
# Set up a data.frame
dd <- data.frame(Quarter = c("16/17 Q1", "16/17 Q2"), Vendors = c("a","b"))
# Begin the query
sql_qry <- "insert into cc_demo (Quarter,Vendor) VALUES"
# Finish it with
sql_qry <- paste0(sql_qry, paste(sprintf("('%s', '%s')", dd$Quarter, dd$Vendors), collapse = ","))
You should get:
"insert into cc_demo (Quarter,Vendor) VALUES('16/17 Q1', 'a'),('16/17 Q2', 'b')"
You can provide this query to your database connection in order to run it.
I hope this helps.

SQL % statement in python returns error

I created 5 tables in mysql workbench 5.7 in which I will pull data from APIgraph queries for a given facebook page.
However, when I run the code, it throws an error:
ProgrammingError: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s, %s)' at line 1
Here is the part of the code which I think contains the error :
#create connection to db
connection = connect_db()
cursor = connection.cursor()
#SQL request for inserting the date of the page into the database
insert_page = ("INSERT INTO page"
"(fb_id, name)"
"VALUES (%s, %s)")
insert_posts = ("INSERT INTO posts "
"(page_id, fb_post_id, message, time_created)"
"VALUES (%s, %s, %s, %s)")
And I finally put the data at the end of the code:
cursor.execute(insert_page, json_pageiddata)
Any ideas? Thanks for helping
EDIT: here is my json_pageiddtata variable, obtained from a URL query with APIgraph:
pageid_url = create_pageid_url(current_page, APP_ID, APP_SECRET)
json_pageiddata = render_to_json(pageid_url)
print json_pageiddata["name"], json_pageiddata["id"]
If you are using data as dictionary u need to specify index name, try to use %(name)s.

Generating pyodbc SQL syntax within a loop

Can you do such a thing? I have the following but cursor.execute does not like the syntax of selectSQL. Ultimately I'm looking to iterate through all tables in a .accdb and insert records from each table into a another .accdb with the same tables and fields. Reason being, bringing over new records from field data collection on TabletPCs to master database on server.
import pyodbc
connOtherDB = pyodbc.connect("Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ='path to my dbase;")
otherDBtbls = connOtherDB.cursor().tables()
for t in otherDBtbls:
if t.table_name.startswith("tbl"): #ignores MS sys tables
cursor = connOtherDB.cursor()
#selectSQL = '"SELECT * from '+ str(t.table_name) + '"'
cursor.execute("SELECT * from tblDatabaseComments") #this works if I refer to a table by name
cursor.execute(selectSQL) #this does not work. throws SQL syntax error
row = cursor.fetchone()
if row:
print t.table_name
print row
Use str.format() to ease building of SQL statements:
import pyodbc
connOtherDB = pyodbc.connect("Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ='path to my dbase;")
otherDBtbls = connOtherDB.cursor().tables()
for t in otherDBtbls:
if t.table_name.startswith("tbl"): #ignores MS sys tables
cursor = connOtherDB.cursor()
selectSQL = 'SELECT * FROM {}'.format(t.table_name)
cursor.execute(selectSQL)
row = cursor.fetchone()
if row:
print t.table_name
print row
As an aside, take a look a PEP 8 -- Style Guide for Python Code for guidance on maximum line length and variable naming, among other coding conventions.

MySQL Exception: Execute SQL Transaction

I am trying to commit a sql transaction to MySQL but I get myself past an MySQLSyntaxErrorException.
The code I am using is:
implicit connection =>
SQL("""
start transaction;
insert into projects(id_user, name, description) values({idUser}, {name}, {description});
set #last_id = last_insert_id();
insert into assigned(id_user, id_project) values({idUser}, #last_id);
commit;
""")
.on('idUser -> idUser,
'name -> project.name,
'description -> project.description
).execute()
The exception I get:
[MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'insert into projects(id_user, name, description) values(1, 'First inserted proje' at line 1]
I am starting to think that I can't execute such statements at all with Anorm.
You cannot use transaction that way. You have to understand that anorm is simply a wrapper around existing jdbc libraries. By default, when using the withConnection and SQL :
DB.withConnection { conn =>
SQL("...
}
Your query is transformed using a PreparedStatement. Meaning the ; chars are causing errors.
Thus, if you want to use transaction, you have to use anorm's mecanism for that.
DB.withTransaction { conn =>
SQL("...
}