I created a table using dbWriteTable in R (and then appended more data to it), using two commands:
dbWriteTable(my_db, name="dallas_flights", value=nov_trim, overwrite = TRUE, row.names = FALSE)
dbWriteTable(my_db, name="dallas_flights", value=dec_trim, append = TRUE, row.names = FALSE)
After I ran my script, I deleted everything using rm(list=ls()). When I type, ls() into R, I get character(0).
However, when I try to rerun the script, and reach these two commands, I get an error indicating that the table still exists:
Error in sqliteExecStatement(con, statement, bind.data) :
RS-DBI driver: (error in statement: table dallas_flights already exists)
I do not know how to delete this table. Any advice?
Related
I have connected to Spotify's API in Python to extract the top twenty tracks of a searched artist. I am trying to store the data in MySQL Workbench in a database named 'spotify_api', I created called 'spotify'. Before I added my code to connect to MySQL Workbench, my code worked correctly and was able to extract the list of tracks, but I have run into issues in getting my code to connect to my database. Below is the code I have written to both extract the data and store it into my database:
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
import mysql.connector
mydb = mysql.connector.connect(
host = "localhost",
user = "root",
password = "(removed for question)",
database = "spotify_api"
)
mycursor = mydb.cursor()
sql = 'DROP TABLE IF EXISTS spotify_api.spotify;'
mycursor.execute(sql)
sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials(client_id="(removed for question)",
client_secret="(removed for question)"))
results = sp.search(q='sza', limit=20)
for idx, track in enumerate(results['tracks']['items']):
print(idx, track['name'])
sql = "INSERT INTO spotify_api.spotify (tracks, items) VALUES (" + \
str(idx) + ", '" + track['name'] + "');"
mycursor.execute(sql)
mydb.commit()
print(mycursor.rowcount, "record inserted.")
mycursor.execute("SELECT * FROM spotify_api.spotify;")
myresult = mycursor.fetchall()
for x in myresult:
print(x)
mycursor.close()
Every time I run my code in the VS Code terminal, I receive an error stating that my table doesn't exist. This is what it states:
"mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'spotify_api.spotify' doesn't exist"
I'm not sure what I need to fix in my code or in my database in order to eliminate this error and get my data stored into my table. In my table I have created two columns 'tracks' and 'items', but I'm not sure if my issues lie in my database or in my code.
Well, it seems pretty clear. You ran
DROP TABLE IF EXISTS spotify_api.spotify;
...
INSERT INTO spotify_api.spotify (tracks, items) VALUES ...
We won't even raise the spectre of the Chuck Berry
track titled little ol' Bobby Tables here.
You DROP'd it, then tried to INSERT into it.
That won't work.
You'll need to CREATE TABLE prior to the INSERT.
I have the following issue with my R tests. I have test functions that need to alter the database, compute the results, check if these results are equal to the test values and clean the database. I am trying to do using transactions from MySQL and dbBegin(con) and dbRollback(con) functions from RMySQL.
I was trying to run the following code:
.setUp <- function() {
dbBegin(con)
}
.tearDown <- function() {
dbRollback(con)
}
test.function1 <- function() {
....
}
test.function2 <- function() {
....
}
with the test suite
test.suite <- defineTestSuite("example",
dirs = file.path("tests"),
testFileRegexp = '*.r')
test.result <- runTestSuite(test.suite)
printTextProtocol(test.result)
However, when I run multiple functions I get the
Error in .local(conn, statement, ...) :
could not run statement: Duplicate entry '-1' for key 'PRIMARY'
Which means that I never rollback what I write in my database.
Could anyone indicate what is wrong in the code above and/or how to write tests in R + RMySQL to test inside transactions?
Thanks,
Vladimir
This is a proper way of testing R functions that alter the MySQL database. I was getting the error because of a typo.
Very similar to this question MySQL Dynamic Query Statement in Python
However what I am looking to do instead of two lists is to use a dictionary
Let's say i have this dictionary
instance_insert = {
# sql column variable value
'instance_id' : 'instnace.id',
'customer_id' : 'customer.id',
'os' : 'instance.platform',
}
And I want to populate a mysql database with an insert statement using sql column as the sql column name and the variable name as the variable that will hold the value that is to be inserted into the mysql table.
Kind of lost because I don't understand exactly what this statement does, but was pulled from the question that I posted where he was using two lists to do what he wanted.
sql = "INSERT INTO instance_info_test VALUES (%s);" % ', '.join('?' for _ in instance_insert)
cur.execute (sql, instance_insert)
Also I would like it to be dynamic in the sense that I can add/remove columns to the dictionary
Before you post, you might want to try searching for something more specific to your question. For instance, when I Googled "python mysqldb insert dictionary", I found a good answer on the first page, at http://mail.python.org/pipermail/tutor/2010-December/080701.html. Relevant part:
Here's what I came up with when I tried to make a generalized version
of the above:
def add_row(cursor, tablename, rowdict):
# XXX tablename not sanitized
# XXX test for allowed keys is case-sensitive
# filter out keys that are not column names
cursor.execute("describe %s" % tablename)
allowed_keys = set(row[0] for row in cursor.fetchall())
keys = allowed_keys.intersection(rowdict)
if len(rowdict) > len(keys):
unknown_keys = set(rowdict) - allowed_keys
print >> sys.stderr, "skipping keys:", ", ".join(unknown_keys)
columns = ", ".join(keys)
values_template = ", ".join(["%s"] * len(keys))
sql = "insert into %s (%s) values (%s)" % (
tablename, columns, values_template)
values = tuple(rowdict[key] for key in keys)
cursor.execute(sql, values)
filename = ...
tablename = ...
db = MySQLdb.connect(...)
cursor = db.cursor()
with open(filename) as instream:
row = json.load(instream)
add_row(cursor, tablename, row)
Peter
If you know your inputs will always be valid (table name is valid, columns are present in the table), and you're not importing from a JSON file as the example is, you can simplify this function. But it'll accomplish what you want to accomplish. While it may initially seem like DictCursor would be helpful, it looks like DictCursor is useful for returning a dictionary of values, but it can't execute from a dict.
I need to insert some data into a mysql database. Db is connected and working.
I am running the following code :
a = sprintf('%s',hashedStr);
sqlQueryStr = 'insert into products (security_code) values (a)'
QueryDB(sqlQueryStr);
I have a database called test and a table named products with 2 fields id and security_code.
When I run this, I get :
Unknown column 'a' in fieldlist ...
Why is this happening ? i dont have and dont need this column ...
Any help ?
Try with:
sqlQueryStr = sprintf('insert into products (security_code) values ("%s")',hashedStr);
QueryDB(sqlQueryStr);
problem is that you are not replacing "a" variable into sql expression
I am experimenting with ErlyDB in a non-erlyweb environment and I am not having much luck.
I have a 'Thing' table for testing, and a corresponding Thing module:
-module(thing).
-export([table/0, fields/0]).
table() ->
thing.
fields() ->
[name, value].
The module itself works - I can query the database fine using ([Thing] = thing:find({name, '=', "test"})).
When I try and save a new record, however things aren't so good.
I consistently see the following error:
mysql_conn:426: fetch <<"BEGIN">> (id <0.97.0>)
mysql_conn:426: fetch <<"INSERT INTO thing(value,name) VALUES ('vtha','blah')">> (id <0.97.0>)
mysql_conn:426: fetch <<"ROLLBACK">> (id <0.97.0>)
** exception exit: {{'EXIT',{badarg,[{erlang,hd,[[]]},
{erlydb_base,'-do_save/1-fun-0-',4},
{mysql_conn,do_transaction,2},
{mysql_conn,loop,1}]}},
{rollback_result,{updated,{mysql_result,[],[],0,[]}}}}
in function erlydb_base:do_save/1
in call from erlydb_base:hook/4
in call from test_thing:test/0
called as test_thing:test()
The table exists, the credentials work, and the SQL itself is fine, as I can execute the command directly on the database.
The code I am using to save is:
erlydb:start(mysql, Database),
Thing = thing:new(<<"hello">>, <<"world">>),
thing:save(Thing),
Is there something I am missing?
Is there some way of viewing some more helpful error messages from the database?
Looking at the source of erlydb_base, the exception happens when erlydb calls your thing module's db_pk_fields() function. That function should return a list, but apparently it doesn't.
I can confirm that altering the code in erlydb.erl fixes this problem (from this reference on the mailing list).
Change Line 561 of erlydb.erl from
lists:map(fun({_Name, _Atts} = F) -> F;
(Name) -> {Name, []}
end, lists:usort(DefinedFields)),
To:
lists:map(fun({_Name, _Atts} = F) -> F;
(Name) -> {Name, []}
end, DefinedFields),