I have connected to Spotify's API in Python to extract the top twenty tracks of a searched artist. I am trying to store the data in MySQL Workbench in a database named 'spotify_api', I created called 'spotify'. Before I added my code to connect to MySQL Workbench, my code worked correctly and was able to extract the list of tracks, but I have run into issues in getting my code to connect to my database. Below is the code I have written to both extract the data and store it into my database:
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
import mysql.connector
mydb = mysql.connector.connect(
host = "localhost",
user = "root",
password = "(removed for question)",
database = "spotify_api"
)
mycursor = mydb.cursor()
sql = 'DROP TABLE IF EXISTS spotify_api.spotify;'
mycursor.execute(sql)
sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials(client_id="(removed for question)",
client_secret="(removed for question)"))
results = sp.search(q='sza', limit=20)
for idx, track in enumerate(results['tracks']['items']):
print(idx, track['name'])
sql = "INSERT INTO spotify_api.spotify (tracks, items) VALUES (" + \
str(idx) + ", '" + track['name'] + "');"
mycursor.execute(sql)
mydb.commit()
print(mycursor.rowcount, "record inserted.")
mycursor.execute("SELECT * FROM spotify_api.spotify;")
myresult = mycursor.fetchall()
for x in myresult:
print(x)
mycursor.close()
Every time I run my code in the VS Code terminal, I receive an error stating that my table doesn't exist. This is what it states:
"mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'spotify_api.spotify' doesn't exist"
I'm not sure what I need to fix in my code or in my database in order to eliminate this error and get my data stored into my table. In my table I have created two columns 'tracks' and 'items', but I'm not sure if my issues lie in my database or in my code.
Well, it seems pretty clear. You ran
DROP TABLE IF EXISTS spotify_api.spotify;
...
INSERT INTO spotify_api.spotify (tracks, items) VALUES ...
We won't even raise the spectre of the Chuck Berry
track titled little ol' Bobby Tables here.
You DROP'd it, then tried to INSERT into it.
That won't work.
You'll need to CREATE TABLE prior to the INSERT.
I have come across an Oracle problem for which I have so far been unable to find the cause.
The query below works in Oracle SQL developer, but when running in .NET it throws:
ORA-01008: not all variables bound
I've tried:
Changing the Oracle data type for lot_priority (Varchar2 or int32).
Changing the .NET data type for lot_priority (string or int).
One bind variable name is used twice in the query. This is not a problem in my
other queries that use the same bound variable in more than one
location, but just to be sure I tried making the second instance its
own variable with a different :name and binding it separately.
Several different ways of binding the variables (see commented code;
also others).
Moving the bindByName() call around.
Replacing each bound variable with a literal. I've had two separate variables cause the problem (:lot_pri and :lot_priprc). There were some minor changes I can't remember between the two. Changing to literals made the query work, but they do need to work with binding.
Query and code follow. Variable names have been changed to protect the innocent:
SELECT rf.myrow floworder, rf.stage, rf.prss,
rf.pin instnum, rf.prid, r_history.rt, r_history.wt
FROM
(
SELECT sub2.myrow, sub2.stage, sub2.prss, sub2.pin, sub2.prid
FROM (
SELECT sub.myrow, sub.stage, sub.prss, sub.pin,
sub.prid, MAX(sub.target_rn) OVER (ORDER BY sub.myrow) target_row
,sub.hflag
FROM (
WITH floc AS
(
SELECT flow.prss, flow.seq_num
FROM rpf#mydblink flow
WHERE flow.parent_p = :lapp
AND flow.prss IN (
SELECT r_priprc.prss
FROM r_priprc#mydblink r_priprc
WHERE priprc = :lot_priprc
)
AND rownum = 1
)
SELECT row_number() OVER (ORDER BY pp.seq_num, rpf.seq_num) myrow,
rpf.stage, rpf.prss, rpf.pin,
rpf.itype, hflag,
CASE WHEN rpf.itype = 'SpecialValue'
THEN rpf.instruction
ELSE rpf.parent_p
END prid,
CASE WHEN rpf.prss = floc.prss
AND rpf.seq_num = floc.seq_num
THEN row_number() OVER (ORDER BY pp.seq_num, rpf.seq_num)
END target_rn
FROM floc, rpf#mydblink rpf
LEFT OUTER JOIN r_priprc#mydblink pp
ON (pp.prss = rpf.prss)
WHERE pp.priprc = :lot_priprc
ORDER BY pp.seq_num, rpf.seq_num
) sub
) sub2
WHERE sub2.myrow >= sub2.target_row
AND sub2.hflag = 'true'
) rf
LEFT OUTER JOIN r_history#mydblink r_history
ON (r_history.lt = :lt
AND r_history.pri = :lot_pri
AND r_history.stage = rf.stage
AND r_history.curp = rf.prid
)
ORDER BY myrow
public void runMyQuery(string lot_priprc, string lapp, string lt, int lot_pri) {
Dictionary<int, foo> bar = new Dictionary<int, foo>();
using(var con = new OracleConnection(connStr)) {
con.Open();
using(var cmd = new OracleCommand(sql.rtd_get_flow_for_lot, con)) { // Query stored in sql.resx
try {
cmd.BindByName = true;
cmd.Prepare();
cmd.Parameters.Add(new OracleParameter("lapp", OracleDbType.Varchar2)).Value = lapp;
cmd.Parameters.Add(new OracleParameter("lot_priprc", OracleDbType.Varchar2)).Value = lot_priprc;
cmd.Parameters.Add(new OracleParameter("lt", OracleDbType.Varchar2)).Value = lt;
// Also tried OracleDbType.Varchar2 below, and tried passing lot_pri as an integer
cmd.Parameters.Add(new OracleParameter("lot_pri", OracleDbType.Int32)).Value = lot_pri.ToString();
/*********** Also tried the following, more explicit code rather than the 4 lines above: **
OracleParameter param_lapp
= cmd.Parameters.Add(new OracleParameter("lapp", OracleDbType.Varchar2));
OracleParameter param_priprc
= cmd.Parameters.Add(new OracleParameter("lot_priprc", OracleDbType.Varchar2));
OracleParameter param_lt
= cmd.Parameters.Add(new OracleParameter("lt", OracleDbType.Varchar2));
OracleParameter param_lot_pri
= cmd.Parameters.Add(new OracleParameter("lot_pri", OracleDbType.Varchar2));
param_lapp.Value = lastProcedureStackProcedureId;
param_priprc.Value = lotPrimaryProcedure;
param_lt.Value = lotType;
param_lot_pri.Value = lotPriority.ToString();
//***************************************************************/
var reader = cmd.ExecuteReader();
while(reader.Read()) {
// Get values from table (Never reached)
}
}
catch(OracleException e) {
// ORA-01008: not all variables bound
}
}
}
Why is Oracle claiming that not all variables are bound?
I know this is an old question, but it hasn't been correctly addressed, so I'm answering it for others who may run into this problem.
By default Oracle's ODP.net binds variables by position, and treats each position as a new variable.
Treating each copy as a different variable and setting it's value multiple times is a workaround and a pain, as furman87 mentioned, and could lead to bugs, if you are trying to rewrite the query and move things around.
The correct way is to set the BindByName property of OracleCommand to true as below:
var cmd = new OracleCommand(cmdtxt, conn);
cmd.BindByName = true;
You could also create a new class to encapsulate OracleCommand setting the BindByName to true on instantiation, so you don't have to set the value each time. This is discussed in this post
I found how to run the query without error, but I hesitate to call it a "solution" without really understanding the underlying cause.
This more closely resembles the beginning of my actual query:
-- Comment
-- More comment
SELECT rf.flowrow, rf.stage, rf.process,
rf.instr instnum, rf.procedure_id, rtd_history.runtime, rtd_history.waittime
FROM
(
-- Comment at beginning of subquery
-- These two comment lines are the problem
SELECT sub2.flowrow, sub2.stage, sub2.process, sub2.instr, sub2.pid
FROM ( ...
The second set of comments above, at the beginning of the subquery, were the problem. When removed, the query executes. Other comments are fine.
This is not a matter of some rogue or missing newline causing the following line to be commented, because the following line is a SELECT. A missing select would yield a different error than "not all variables bound."
I asked around and found one co-worker who has run into this -- comments causing query failures -- several times.
Does anyone know how this can be the cause? It is my understanding that the very first thing a DBMS would do with comments is see if they contain hints, and if not, remove them during parsing. How can an ordinary comment containing no unusual characters (just letters and a period) cause an error? Bizarre.
You have two references to the :lot_priprc binding variable -- while it should require you to only set the variable's value once and bind it in both places, I've had problems where this didn't work and had to treat each copy as a different variable. A pain, but it worked.
On Charles' comment problem: to make things worse, let
:p1 = 'TRIALDEV'
via a Command Parameter, then execute
select T.table_name as NAME, COALESCE(C.comments, '===') as DESCRIPTION
from all_all_tables T
Inner Join all_tab_comments C on T.owner = C.owner and T.table_name = C.table_name
where Upper(T.owner)=:p1
order by T.table_name
558 line(s) affected. Processing time: 00:00:00.6535711
and when changing the literal string from === to ---
select T.table_name as NAME, COALESCE(C.comments, '---') as DESCRIPTION
[...from...same-as-above...]
ORA-01008: not all variables bound
Both statements execute fine in SQL Developer. The shortened code:
Using con = New OracleConnection(cs)
con.Open()
Using cmd = con.CreateCommand()
cmd.CommandText = cmdText
cmd.Parameters.Add(pn, OracleDbType.NVarchar2, 250).Value = p
Dim tbl = New DataTable
Dim da = New OracleDataAdapter(cmd)
da.Fill(tbl)
Return tbl
End Using
End Using
using Oracle.ManagedDataAccess.dll Version 4.121.2.0 with the default settings in VS2015 on the .Net 4.61 platform.
So somewhere in the call chain, there might be a parser that is a bit too aggressively looking for one-line-comments started by -- in the commandText. But even if this would be true, the error message "not all variables bound" is at least misleading.
The solution in my situation was similar answer to Charles Burns; and the problem was related to SQL code comments.
I was building (or updating, rather) an already-functioning SSRS report with Oracle datasource. I added some more parameters to the report, tested it in Visual Studio, it works great, so I deployed it to the report server, and then when the report is executed the report on the server I got the error message:
"ORA-01008: not all variables bound"
I tried quite a few different things (TNSNames.ora file installed on the server, Removed single line comments, Validate dataset query mapping). What it came down to was I had to remove a comment block directly after the WHERE keyword. The error message was resolved after moving the comment block after the WHERE CLAUSE conditions. I have other comments in the code also. It was just the one after the WHERE keyword causing the error.
SQL with error: "ORA-01008: not all variables bound"...
WHERE
/*
OHH.SHIP_DATE BETWEEN TO_DATE('10/1/2018', 'MM/DD/YYYY') AND TO_DATE('10/31/2018', 'MM/DD/YYYY')
AND OHH.STATUS_CODE<>'DL'
AND OHH.BILL_COMP_CODE=100
AND OHH.MASTER_ORDER_NBR IS NULL
*/
OHH.SHIP_DATE BETWEEN :paramStartDate AND :paramEndDate
AND OHH.STATUS_CODE<>'DL'
AND OHH.BILL_COMP_CODE IN (:paramCompany)
AND LOAD.DEPART_FROM_WHSE_CODE IN (:paramWarehouse)
AND OHH.MASTER_ORDER_NBR IS NULL
AND LOAD.CLASS_CODE IN (:paramClassCode)
AND CUST.CUST_CODE || '-' || CUST.CUST_SHIPTO_CODE IN (:paramShipto)
SQL executes successfully on the report server...
WHERE
OHH.SHIP_DATE BETWEEN :paramStartDate AND :paramEndDate
AND OHH.STATUS_CODE<>'DL'
AND OHH.BILL_COMP_CODE IN (:paramCompany)
AND LOAD.DEPART_FROM_WHSE_CODE IN (:paramWarehouse)
AND OHH.MASTER_ORDER_NBR IS NULL
AND LOAD.CLASS_CODE IN (:paramClassCode)
AND CUST.CUST_CODE || '-' || CUST.CUST_SHIPTO_CODE IN (:paramShipto)
/*
OHH.SHIP_DATE BETWEEN TO_DATE('10/1/2018', 'MM/DD/YYYY') AND TO_DATE('10/31/2018', 'MM/DD/YYYY')
AND OHH.STATUS_CODE<>'DL'
AND OHH.BILL_COMP_CODE=100
AND OHH.MASTER_ORDER_NBR IS NULL
*/
Here is what the dataset parameter mapping screen looks like.
It's a bug in Managed ODP.net - 'Bug 21113901 : MANAGED ODP.NET RAISE ORA-1008 USING SINGLE QUOTED CONST + BIND VAR IN SELECT' fixed in patch 23530387 superseded by patch 24591642
Came here looking for help as got same error running a statement listed below while going through a Udemy course:
INSERT INTO departments (department_id, department_name)
values( &dpet_id, '&dname');
I'd been able to run statements with substitution variables before. Comment by Charles Burns about possibility of server reaching some threshold while recreating the variables prompted me to log out and restart the SQL Developer. The statement ran fine after logging back in.
Thought I'd share for anyone else venturing here with a limited scope issue as mine.
I'd a similar problem in a legacy application, but de "--" was string parameter.
Ex.:
Dim cmd As New OracleCommand("INSERT INTO USER (name, address, photo) VALUES ('User1', '--', :photo)", oracleConnection)
Dim fs As IO.FileStream = New IO.FileStream("c:\img.jpg", IO.FileMode.Open)
Dim br As New IO.BinaryReader(fs)
cmd.Parameters.Add(New OracleParameter("photo", OracleDbType.Blob)).Value = br.ReadBytes(fs.Length)
cmd.ExecuteNonQuery() 'here throws ORA-01008
Changing address parameter value '--' to '00' or other thing, works.
I need to
1. run a select query on MYSQL DB and fetch the records.
2. Records are processed by python script.
I am unsure about the way I should proceed. Is xcom the way to go here? Also, MYSQLOperator only executes the query, doesn't fetch the records. Is there any inbuilt transfer operator I can use? How can I use a MYSQL hook here?
you may want to use a PythonOperator that uses the hook to get the data,
apply transformation and ship the (now scored) rows back some other place.
Can someone explain how to proceed regarding the same.
Refer - http://markmail.org/message/x6nfeo6zhjfeakfe
def do_work():
mysqlserver = MySqlHook(connection_id)
sql = "SELECT * from table where col > 100 "
row_count = mysqlserver.get_records(sql, schema='testdb')
print row_count[0][0]
callMYSQLHook = PythonOperator(
task_id='fetch_from_testdb',
python_callable=mysqlHook,
dag=dag
)
Is this the correct way to proceed?
Also how do we use xcoms to store the records for the following MySqlOperator?'
t = MySqlOperator(
conn_id='mysql_default',
task_id='basic_mysql',
sql="SELECT count(*) from table1 where id > 10",
dag=dag)
I was really struggling with this for the past 90 minutes, here is a more declarative way to follow for newcomers:
from airflow.hooks.mysql_hook import MySqlHook
def fetch_records():
request = "SELECT * FROM your_table"
mysql_hook = MySqlHook(mysql_conn_id = 'the_connection_name_sourced_from_the_ui', schema = 'specific_db')
connection = mysql_hook.get_conn()
cursor = connection.cursor()
cursor.execute(request)
sources = cursor.fetchall()
print(sources)
...your DAG() as dag: code
task = PythonOperator(
task_id = 'fetch_records',
python_callable = fetch_records
)
This returns to the logs the contents of your DB query.
I hope this is of use to someone else.
Sure, just create a hook or operator and call the get_records() method: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/hooks/dbapi.html
INSERT INTO voucher (voucher_no, account, party_name, rece_amt, particulars, voucher_date, voucher_type, cuid, cdt)
SELECT voucher_rec_no, #account, #party_name, #rece_amt, #particulars, #voucher_date, #voucher_type, #cuid, #cdt
FROM auto_number
WHERE (auto_no = 1)
Error:
A parameter is not allowed in this location. Ensure that the '#' sign is in a valid location or that parameters are valid at all in this SQL statement.
I've just stumbled upon this whilst trying to fix the same issue. I know it's late but, assuming that you're getting this error when attempting to execute the query via .net, ensure that you are setting the SqlCeParameter.DbType - if this is not specified, you get the exception you listed above.
Example (assume cmd is a SqlCeCommand - all the stuff is in the System.Data.SqlServerCe namespace):
SqlCeParameter param = new SqlCeParameter();
param.ParameterName = "#SomeParameterName";
param.Direction = ParameterDirection.Input;
param.DbType = DbType.String; // this is the important bit to avoid the exception
param.Value = kvp.Value;
cmd.Parameters.Add(param);
Obviously, you'd want to set the DB type to match the type of your parameter.
Isn't my coding typing wrong way? I need create an update button so user can edit the information by using Matlab. After update, the button need connect to mySQL server 5.6 and ODBC connector.
This is my code:
% --- Executes on button press in update.
function update_Callback(hObject, eventdata, handles)
% hObject handle to update (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
%Display dialog box to confirm save
choice = questdlg('Confirm update to database?', ...
'', ...
'Yes','No','Yes');
% Handle dialog box response
switch choice
case 'Yes'
%Set preferences with setdbprefs.
setdbprefs('DataReturnFormat', 'cellarray');
%Make connection to database.
conn = database('animal_cbir', '', '');
%Test if database connection is valid
testConnection = isconnection(conn);
disp(testConnection);
fileID = getappdata(0,'namevalue');
imageID = fileID;
name = get(handles.edit11,'String');
commonName = get(handles.edit1,'String');
scientificName = get(handles.edit2,'String');
class = get(handles.edit3,'String');
diet = get(handles.edit4,'String');
habitat = get(handles.edit5,'String');
lifeSpan = get(handles.edit6,'String');
size = get(handles.edit7,'String');
weight = get(handles.edit8,'String');
characteristic = get(handles.edit10,'String');
tablename = 'animal';
colnames ={'imageID','name','commonName','scientificName','class','diet','habitat','lifeSpan','size','weight','characteristic'};
data = {imageID,name,commonName,scientificName,class,diet,habitat,lifeSpan,size,weight,characteristic};
disp (data);
whereClause = sprintf(['where imageID = "%s"'],fileID);
update(conn,tablename,colnames,data,whereClause);
updateSuccess = helpdlg('Existing animal species successfully updated in database.');
commit(conn);
case 'No'
end
Error I am getting:
No method 'setInt' with matching signature found for class 'sun.jdbc.odbc.JdbcOdbcPreparedStatement'.
Hope that anyone can help me solve it.