Engine = create_engine("mysql+mysqldb://blah-blah-blah", encoding="utf-8")
Session = sessionmaker(bind = Engine)
ses = Session()
Meta = MetaData(bind = Engine, reflect = True)
PersonTable = Meta.tables["person"]
class Person(object):
pass
mapper(Person, PersonTable)
APerson = Person("1111", "2222", "1.01.1980")
ses.add(APerson)
ses.commit()
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s, %s, %s)' at line 1") b'INSERT INTO person (Name, OriginalName, DoB) VALUES (%s, %s, %s)' ('1111', '2222', '25.01.1980')
What is the %s? What do I wrong?
Python 3.1
SQLAlchemy 0.6.5
MySQL 5.1
Windows 7 Ultimate
Thank you.
You sqlalchemy commit is trying to issue an insert query that is not compatible with the schema. near '%s, %s, %s)' means your trying to insert invalid data. I can only speculate that it is because the date format - this is not the proper mysql date format YYYY-MM-DD.
I migrated back to Python 2.7. Now it works fine.
Related
i want to put a Panda Dataframe to a IBM i Series / AS400. I already researched a much, but now I am stuck.
I already made a lot of queries, where I use pyodbc. For df.to_sql() I should use, as readed on other stacks, sqlalchemy with the ibm_db_sa dialect.
My actual code is:
CONNECTION_STRING = (
"driver={iSeries Access ODBC Driver};"
"System=111.111.111.111;"
"database=TESTDB;"
"uid=USER;"
"pwd=PASSW;"
)
quoted = urllib.parse.quote_plus(CONNECTION_STRING)
engine = create_engine('ibm_db_sa+pyodbc:///?odbc_connect={}'.format(quoted))
create_statement = df.to_sql("TABLETEST", engine, if_exists="append")
the following packages are installed
python 3.9
ibm-db 3.1.3
ibm-db-sa 0.3.7
ibm-db-sa-py3 0.3.1.post1
pandas 1.3.5
pip 22.0.4
setuptools 57.0.0
SQLAlchemy 1.4.39
when I run, i get the following error:
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42S02', '[42S02] [IBM][System i Access ODBC Driver][DB2 for i5/OS]SQL0204 - COLUMNS in SYSCAT type *FILE not found. (-204) (SQLPrepare)')
[SQL: SELECT "SYSCAT"."COLUMNS"."COLNAME", "SYSCAT"."COLUMNS"."TYPENAME", "SYSCAT"."COLUMNS"."DEFAULT", "SYSCAT"."COLUMNS"."NULLS", "SYSCAT"."COLUMNS"."LENGTH", "SYSCAT"."COLUMNS"."SCALE", "SYSCAT"."COLUMNS"."IDENTITY", "SYSCAT"."COLUMNS"."GENERATED"
FROM "SYSCAT"."COLUMNS"
WHERE "SYSCAT"."COLUMNS"."TABSCHEMA" = ? AND "SYSCAT"."COLUMNS"."TABNAME" = ? ORDER BY "SYSCAT"."COLUMNS"."COLNO"]
[parameters: ('USER', 'TABLETEST')]
(Background on this error at: https://sqlalche.me/e/14/f405)
I think, the dialect could be wrong, because the parameters are the username and the table for the ODBC connection?
AND: I am not really sure, whats the difference between ibm_db_sa and ibm_db?
I tried a few days again, before someone is trying to do this via sqlalchemy should do it via pyodbc.
Here is my working example
refering the df_to_sql_bulk_insert function to this
(and now I am currently using my system-DSN):
def df_to_sql_bulk_insert(df: pd.DataFrame, table: str, **kwargs) -> str:
df = df.copy().assign(**kwargs)
columns = ", ".join(df.columns)
tuples = map(str, df.itertuples(index=False, name=None))
values = re.sub(r"(?<=\W)(nan|None)(?=\W)", "NULL", (",\n" + " " * 7).join(tuples))
return f"INSERT INTO {table} ({columns})\nVALUES {values}"
cnxn = pyodbc.connect("DSN=XXX")
cursor = cnxn.cursor()
sqlstr = df_to_sql_bulk_insert(df,"DBXXX.TBLXXX")
cursor.execute(sqlstr)
cnxn.commit()
This question expands on this question
Here, I'm using the custom function created by #Simon.S.A. shown in the answer to this question. I'm attempting to save a tbl_sql object in R to MySQL as a new table without first saving it locally. Here, the database and schema in my MySQL are named "test." The tbl_sql object in R is my_data, and I want to save this is a new table in MySQL labeled "car_data".
library(DBI)
library(tidyverse)
library(dbplyr)
#establish connection and import data from MySQL
con <- DBI::dbConnect(RMariaDB::MariaDB(),
dbname = "test",
host = "127.0.0.1",
user = "user",
password = "password")
my_data <- tbl(con, "mtcars")
my_data <- my_data %>% filter(mpg >= 22)
# write function to save tbl_sql as a new table in SQL
write_to_database <- function(input_tbl, db, schema, tbl_name){
# connection
tbl_connection <- input_tbl$src$con
# SQL query
sql_query <- glue::glue(
"SELECT *\n",
"INTO {db}.{schema}.{tbl_name}\n",
"FROM (\n",
dbplyr::sql_render(input_tbl),
"\n) AS sub_query"
)
result <- dbExecute(tbl_connection, as.character(sql_query))
}
# execute function
write_to_database(my_data, "test", "test", "car_data")
After running final line, I get the following error. I'm not sure how I can fix this.
Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.test.car_data
FROM (
SELECT *
FROM `mtcars`
WHERE (`mpg` >= 22.0)
) AS sub_quer' at line 2 [1064]
12.
stop(structure(list(message = "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.test.car_data\nFROM (\nSELECT *\nFROM `mtcars`\nWHERE (`mpg` >= 22.0)\n) AS sub_quer' at line 2 [1064]",
call = NULL, cppstack = NULL), class = c("Rcpp::exception",
"C++Error", "error", "condition")))
11.
result_create(conn#ptr, statement, is_statement)
10.
initialize(value, ...)
9.
initialize(value, ...)
8.
new("MariaDBResult", sql = statement, ptr = result_create(conn#ptr,
statement, is_statement), bigint = conn#bigint, conn = conn)
7.
dbSend(conn, statement, params, is_statement = TRUE)
6.
.local(conn, statement, ...)
5.
dbSendStatement(conn, statement, ...)
4.
dbSendStatement(conn, statement, ...)
3.
dbExecute(tbl_connection, as.character(sql_query))
2.
dbExecute(tbl_connection, as.character(sql_query))
1.
write_to_database(my_data, "test", "test", "car_data")
Creating a table with INTO command is an SQL Server (even MS Access) specific syntax and not supported in MySQL. Instead, consider the counterpart statement: CREATE TABLE...SELECT. Also, schema differs between RDBMS's. For MySQL, database is synonymous to schema.
Therefore, consider adjusted version of SQL build:
sql_query <- glue::glue(
"CREATE TABLE {db}.{tbl_name}\n AS \n",
"SELECT * \n",
"FROM (\n",
dbplyr::sql_render(input_tbl),
"\n) AS sub_query"
)
I created 5 tables in mysql workbench 5.7 in which I will pull data from APIgraph queries for a given facebook page.
However, when I run the code, it throws an error:
ProgrammingError: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s, %s)' at line 1
Here is the part of the code which I think contains the error :
#create connection to db
connection = connect_db()
cursor = connection.cursor()
#SQL request for inserting the date of the page into the database
insert_page = ("INSERT INTO page"
"(fb_id, name)"
"VALUES (%s, %s)")
insert_posts = ("INSERT INTO posts "
"(page_id, fb_post_id, message, time_created)"
"VALUES (%s, %s, %s, %s)")
And I finally put the data at the end of the code:
cursor.execute(insert_page, json_pageiddata)
Any ideas? Thanks for helping
EDIT: here is my json_pageiddtata variable, obtained from a URL query with APIgraph:
pageid_url = create_pageid_url(current_page, APP_ID, APP_SECRET)
json_pageiddata = render_to_json(pageid_url)
print json_pageiddata["name"], json_pageiddata["id"]
If you are using data as dictionary u need to specify index name, try to use %(name)s.
I am trying to use Ruby to insert values into MySQL on localhost. The value i want to insert is the result from Twitter search. My program can successfully write the result to a file, so now i want to insert the result to MySQL. Here is part of my code:
results = #search.perform("yahoo", 100)
client = Mysql2::Client.new(:host => "localhost", :username => "root", :password => "123", :database => "db1")
results.map do |status|
insert = client.query ("INSERT INTO table1 (fromuser, tweet) VALUES (#{status.from_user},#{status.text})")
end
The error is "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near " tweet " at line 1 (Mysql2::Error).
What caused the error?
Another problem i have found is that when i used following code to insert value to MySQL ,i got another error: "Unknown column 'a' in 'field list' (Mysql::ServerError::BadFieldError)"
require 'mysql'
require 'rubygems'
sql = Mysql.real_connect("localhost", "root", "123", "db1")
st1 = "a"
st2 = "b"
user_string = "(#{st1},#{st2})"
query="INSERT INTO table1 (fromuser, tweet) VALUES" + user_string
sql.query(query)
I want to insert "a" and "b" into table.
How to solve this?
Thanks in advance,
Like Andrew said, you definitely want to escape your data.
I think you also need to quote the values:
insert = client.query("INSERT INTO tweets (from_user, tweet_text)
VALUES ('#{client.escape(status.from_user)}',
'#{client.escape(status.text)}')")
You need to use CREATE TABLE to create a table in your database to insert the data into. At the moment you are saying you want to insert the data into the database name ("db1") itself.
Also, you must escape your data first:
insert = client.query("INSERT INTO tweets (from_user, tweet_text)
VALUES (#{client.escape(status.from_user)},
#{client.escape(status.text)})")
I am trying to commit a sql transaction to MySQL but I get myself past an MySQLSyntaxErrorException.
The code I am using is:
implicit connection =>
SQL("""
start transaction;
insert into projects(id_user, name, description) values({idUser}, {name}, {description});
set #last_id = last_insert_id();
insert into assigned(id_user, id_project) values({idUser}, #last_id);
commit;
""")
.on('idUser -> idUser,
'name -> project.name,
'description -> project.description
).execute()
The exception I get:
[MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'insert into projects(id_user, name, description) values(1, 'First inserted proje' at line 1]
I am starting to think that I can't execute such statements at all with Anorm.
You cannot use transaction that way. You have to understand that anorm is simply a wrapper around existing jdbc libraries. By default, when using the withConnection and SQL :
DB.withConnection { conn =>
SQL("...
}
Your query is transformed using a PreparedStatement. Meaning the ; chars are causing errors.
Thus, if you want to use transaction, you have to use anorm's mecanism for that.
DB.withTransaction { conn =>
SQL("...
}