Issues printing a Dataframe after collecting Data from MySQL - mysql

I hope you can help me with my Issue. I connected python to my database using pyodbc and I think I was able to save the data into a pandas Dataframe, but unfortunatly I cant work with the Dataframe (for example simply print it) aftertwards.
The error Message says "undefined name "DataFrame"".
How do I need to change my Code so I can get the data from MySQL and use the Dataframe afterwards regularly.
Just as a side Information I want to calculate with the dataframe a little using pandas (optional) and then create a plot using Plotnine and add a UI later. just in Case that matters.
#This function I call
def SQLtoPandas(Connection,SQLString,DataFrame):
DataFrame = pd.DataFrame(
pd.read_sql(SQLString,
con=Connection)
)
#If i call this function it works just fine
def SQL_ReadFunction(Connection,SQLString):
cursor=Connection.cursor()
cursor.execute(
SQLString
)
rows = cursor.fetchall()
for row in rows:
print(row)
SQLString = 'select * from employees'
SQL_ReadFunction(Connection,SQLString)
Connection.close
#Doesnt work, moving it inside the connection also doesnt help.
print (DataFrame)

you don't need additional function for this. just use
df=pd.read_sql('select * from employees',con=con)
print(df)
and manipulate df as you wish using pandas.
i would reccomend using jupyter notebook as it displays dataframe nicely.
also note pd.read_sql() already returns pandas DataFrame, no need to reconvert

You have a few things to take care:
Your function can directly have pd.read_sql as it will load your table as a dataframe. You do not need an extra pd.DataFrame.
You have to print your dataframe inside the function, or assign the dataframe outside like df = SQLtoPandas(Connection,SQLString) and have a return df inside your function
Avoid using the keyword DataFrame to name your DataFrame, use df or something else that is not reserved.
Method 1:
Inside your function:
def SQLtoPandas(Connection,SQLString):
df= pd.read_sql(SQLString, con=Connection)
print(df)
Now call your function outside:
SQLtoPandas(Connection, SQLString)
Method 2:
Inside your function:
def SQLtoPandas(Connection,SQLString):
df = pd.read_sql(SQLString, con=Connection)
return df
Now outside your function do:
df = SQLtoPandas(Connection, SQLString)
print(df)

Related

ValueError: Cannot accept list of column references or list of columns for both `x` and `y`

I am not able to get the heatmap in Plotly and getting ValueError: Cannot accept list of column references
def update_graph(xaxis_column_name, yaxis_column_name,value):
dff = df[df['Year'] == value]
fig = px.density_heatmap(
x=dff[dff['Population'] == xaxis_column_name]['Pop. Density (per sq. mi.)'],
y=dff[dff['Area (sq. mi.)'] == yaxis_column_name]['Pop. Density (per sq. mi.)'],
text_auto=True,
hover_name=dff[dff['Country'] == yaxis_column_name]['Region'])
return fig
I had the same error when trying to plot a px.line chart by passing a dataframe to px.line, and I could not find an answer online that solved my specific problem. I'm not sure if your problem was caused by the same thing as mine, but I found that my error was occurring when I tried to pass an empty dataframe to px.line - the data in the dataframe that I am passing to the px.line chart changes each time the script is re-run, and sometimes there is no data in the dataframe, that's why I get an empty dataframe sometimes. To solve the problem, I simply wrote an if statement that stated: if the dataframe that I was trying to pass to px.line was empty, then I passed a "dummy" dataframe to px.line with the same column headers but with only one row of data that was all zeros, else, pass the originally intended dataframe, like this:
# orig_df is the dataframe I am passing to px.line
empty_df = orig_df.empty
if empty_df = True:
orig_df = pandas.DataFrame({'Col1':[0], 'Col2':[0], 'Col3':[0]}) # column names are same as in orig_df
else:
orig_df = orig_df
It solved the problem everywhere I was passing a dynamic dataframe into a px.line chart. I hope this helps.

Code to replace a function to complete a Ticker

THIS IS WHAT I coded :
def statement(ticker,statement):
if statement =='IS':
temp='income-statement'
elif statement =='BS':
temp='balance-sheet-statement'
else:
temp='cash-flow-statement'
df = tickers.temp(frequency='q')
return df
tickers=['AAPL','GOOG','TSLA']
I want the temp from the function to replace in the df = tickers.temp(frequency='q') and finally giving me the correct financial statement when i replace the tickers with the stock code. If anyone has similar suitable code , Please do post it below
I am not really sure what you want to achieve.
So i assume you have something like
tickers.income-statement()
tickers.balance-sheet-statement()
...
and you want to call the method according to the temp string in that function.
so
myfunc = getattr(tickers, temp)
should give you the desired function out of the string you achieved.

In Python, how to concisely get nested values in json data?

I have data loaded from JSON and am trying to extract arbitrary nested values using a list as input, where the list corresponds to the names of successive children. I want a function get_value(data,lookup) that returns the value from data by treating each entry in lookup as a nested child.
In the example below, when lookup=['alldata','TimeSeries','rates'], the return value should be [1.3241,1.3233].
json_data = {'alldata':{'name':'CAD/USD','TimeSeries':{'dates':['2018-01-01','2018-01-02'],'rates':[1.3241,1.3233]}}}
def get_value(data,lookup):
res = data
for item in lookup:
res = res[item]
return res
lookup = ['alldata','TimeSeries','rates']
get_value(json_data,lookup)
My example works, but there are two problems:
It's inefficient - In my for loop, I copy the whole TimeSeries object to res, only to then replace it with the rates list. As #Andrej Kesely explained, res is a reference at each iteration, so data isn't being copied.
It's not concise - I was hoping to be able to find a concise (eg one or two line) way of extracting the data using something like list comprehension syntax
If you want one-liner and you are using Python 3.8, you can use assignment expression ("walrus operator"):
json_data = {'alldata':{'name':'CAD/USD','TimeSeries':{'dates':['2018-01-01','2018-01-02'],'rates':[1.3241,1.3233]}}}
def get_value(data,lookup):
return [data:=data[item] for item in lookup][-1]
lookup = ['alldata','TimeSeries','rates']
print( get_value(json_data,lookup) )
Prints:
[1.3241, 1.3233]
I don't think you can do it without a loop, but you could use a reducer here to increase readability.
functools.reduce(dict.get, lookup, json_data)

DynamoDB JSON response parsing prints vertically

I have a script that scans a DynamoDB table that stores my instance IDs. Then I try to query another table to see if it also has that same instance and get all of the metadata attributes in a master table. When I iterate through the query using the instance ID from the initial scan of the first table, I am noticing each character of the instance id string is being printed to a new line, instead of the entire string on one line. I am confused how to fix this. Below is my code, sample output, and the expected output.
CODE:
import boto3
import json
from boto3.dynamodb.conditions import Key, Attr
def table_diff():
dynamo = boto3.client('dynamodb')
dynamodb = boto3.resource('dynamodb')
table_missing = dynamodb.Table('RunningInstances')
missing_response = dynamo.scan(TableName='CWPMissingAgent')
for instances in missing_response['Items']:
instance_id = instances['missing_instances']['S']
# This works how I want, prints i-xxxxx
print(instance_id)
for id in instance_id:
# This does not print how I want (vertically)
print(id)
query_response = table_missing.query(KeyConditionExpression=Key('ID').eq(id))
OUTPUT:
i
-
x
x
x
x
x
EXPECTED OUTPUT:
i-xxxxx
etc etc
instance_id is a string. Thus, when you loop over it (for id in instance_id), you are actually looping over each character in the string, and printing them out individually.
Why do you try to loop over it, when you say that just printing it produces the correct result?

save list to CSV - python

I try to save my output as csv using the "import csv" and only get errors. Any reason why?
Since I can not make it run will it also notify if the file already exists?
Thanks a lot
from tkinter import *
from tkinter.filedialog import asksaveasfilename
from tkinter import ttk
import csv
def data():
...
output= <class 'list'> #just an example
...
def savefile():
name= asksaveasfilename()
create = csv.writer(open(name, "wb"))
create.writerow(output)
for x in output:
create.writerow(x)
root = Tk()
Mframe = ttk.Frame(root)
Mframe.grid(column=0, row=0, sticky=(N, W, E, S))
bSave=ttk.Button(Mframe, text='Save File', command=savefile)
bSave.grid(column=1, row=0)
root.mainloop()
You are opening the file, but not closing it. A good practise is to use a with statement make sure you close it. By the way, I don't know how is the output list, but if it isn't a list of lists, it makes more sense to me to call writerow once.
Besides, make sure this list is also a global variable, otherwise it won't be available within the scope of savefile. However, global variables are not a very good solution, so consider to pass it as an argument to savefile or use a class to hold all this data:
def savefile():
name = asksaveasfilename()
with open(name, 'w', newline='') as csvfile:
create = csv.writer(csvfile)
create.writerow(output)