I am not able to get the heatmap in Plotly and getting ValueError: Cannot accept list of column references
def update_graph(xaxis_column_name, yaxis_column_name,value):
dff = df[df['Year'] == value]
fig = px.density_heatmap(
x=dff[dff['Population'] == xaxis_column_name]['Pop. Density (per sq. mi.)'],
y=dff[dff['Area (sq. mi.)'] == yaxis_column_name]['Pop. Density (per sq. mi.)'],
text_auto=True,
hover_name=dff[dff['Country'] == yaxis_column_name]['Region'])
return fig
I had the same error when trying to plot a px.line chart by passing a dataframe to px.line, and I could not find an answer online that solved my specific problem. I'm not sure if your problem was caused by the same thing as mine, but I found that my error was occurring when I tried to pass an empty dataframe to px.line - the data in the dataframe that I am passing to the px.line chart changes each time the script is re-run, and sometimes there is no data in the dataframe, that's why I get an empty dataframe sometimes. To solve the problem, I simply wrote an if statement that stated: if the dataframe that I was trying to pass to px.line was empty, then I passed a "dummy" dataframe to px.line with the same column headers but with only one row of data that was all zeros, else, pass the originally intended dataframe, like this:
# orig_df is the dataframe I am passing to px.line
empty_df = orig_df.empty
if empty_df = True:
orig_df = pandas.DataFrame({'Col1':[0], 'Col2':[0], 'Col3':[0]}) # column names are same as in orig_df
else:
orig_df = orig_df
It solved the problem everywhere I was passing a dynamic dataframe into a px.line chart. I hope this helps.
Related
I have data loaded from JSON and am trying to extract arbitrary nested values using a list as input, where the list corresponds to the names of successive children. I want a function get_value(data,lookup) that returns the value from data by treating each entry in lookup as a nested child.
In the example below, when lookup=['alldata','TimeSeries','rates'], the return value should be [1.3241,1.3233].
json_data = {'alldata':{'name':'CAD/USD','TimeSeries':{'dates':['2018-01-01','2018-01-02'],'rates':[1.3241,1.3233]}}}
def get_value(data,lookup):
res = data
for item in lookup:
res = res[item]
return res
lookup = ['alldata','TimeSeries','rates']
get_value(json_data,lookup)
My example works, but there are two problems:
It's inefficient - In my for loop, I copy the whole TimeSeries object to res, only to then replace it with the rates list. As #Andrej Kesely explained, res is a reference at each iteration, so data isn't being copied.
It's not concise - I was hoping to be able to find a concise (eg one or two line) way of extracting the data using something like list comprehension syntax
If you want one-liner and you are using Python 3.8, you can use assignment expression ("walrus operator"):
json_data = {'alldata':{'name':'CAD/USD','TimeSeries':{'dates':['2018-01-01','2018-01-02'],'rates':[1.3241,1.3233]}}}
def get_value(data,lookup):
return [data:=data[item] for item in lookup][-1]
lookup = ['alldata','TimeSeries','rates']
print( get_value(json_data,lookup) )
Prints:
[1.3241, 1.3233]
I don't think you can do it without a loop, but you could use a reducer here to increase readability.
functools.reduce(dict.get, lookup, json_data)
Following the basic steps to create Prophet model and forecast
m = Prophet(daily_seasonality=True)
m.fit(data)
forecast = m.make_future_dataframe(periods=2)
forecast.tail().T
the result is as following (no yhat value ??)
The data passed in to fit the model has two columns (date and value).
Not sure what I have missed out here.
I managed to get it works by creating a new dataframe:
df_p = pd.DataFrame({'ds': d.index, 'y': d.values})
I hope you can help me with my Issue. I connected python to my database using pyodbc and I think I was able to save the data into a pandas Dataframe, but unfortunatly I cant work with the Dataframe (for example simply print it) aftertwards.
The error Message says "undefined name "DataFrame"".
How do I need to change my Code so I can get the data from MySQL and use the Dataframe afterwards regularly.
Just as a side Information I want to calculate with the dataframe a little using pandas (optional) and then create a plot using Plotnine and add a UI later. just in Case that matters.
#This function I call
def SQLtoPandas(Connection,SQLString,DataFrame):
DataFrame = pd.DataFrame(
pd.read_sql(SQLString,
con=Connection)
)
#If i call this function it works just fine
def SQL_ReadFunction(Connection,SQLString):
cursor=Connection.cursor()
cursor.execute(
SQLString
)
rows = cursor.fetchall()
for row in rows:
print(row)
SQLString = 'select * from employees'
SQL_ReadFunction(Connection,SQLString)
Connection.close
#Doesnt work, moving it inside the connection also doesnt help.
print (DataFrame)
you don't need additional function for this. just use
df=pd.read_sql('select * from employees',con=con)
print(df)
and manipulate df as you wish using pandas.
i would reccomend using jupyter notebook as it displays dataframe nicely.
also note pd.read_sql() already returns pandas DataFrame, no need to reconvert
You have a few things to take care:
Your function can directly have pd.read_sql as it will load your table as a dataframe. You do not need an extra pd.DataFrame.
You have to print your dataframe inside the function, or assign the dataframe outside like df = SQLtoPandas(Connection,SQLString) and have a return df inside your function
Avoid using the keyword DataFrame to name your DataFrame, use df or something else that is not reserved.
Method 1:
Inside your function:
def SQLtoPandas(Connection,SQLString):
df= pd.read_sql(SQLString, con=Connection)
print(df)
Now call your function outside:
SQLtoPandas(Connection, SQLString)
Method 2:
Inside your function:
def SQLtoPandas(Connection,SQLString):
df = pd.read_sql(SQLString, con=Connection)
return df
Now outside your function do:
df = SQLtoPandas(Connection, SQLString)
print(df)
I have a script that scans a DynamoDB table that stores my instance IDs. Then I try to query another table to see if it also has that same instance and get all of the metadata attributes in a master table. When I iterate through the query using the instance ID from the initial scan of the first table, I am noticing each character of the instance id string is being printed to a new line, instead of the entire string on one line. I am confused how to fix this. Below is my code, sample output, and the expected output.
CODE:
import boto3
import json
from boto3.dynamodb.conditions import Key, Attr
def table_diff():
dynamo = boto3.client('dynamodb')
dynamodb = boto3.resource('dynamodb')
table_missing = dynamodb.Table('RunningInstances')
missing_response = dynamo.scan(TableName='CWPMissingAgent')
for instances in missing_response['Items']:
instance_id = instances['missing_instances']['S']
# This works how I want, prints i-xxxxx
print(instance_id)
for id in instance_id:
# This does not print how I want (vertically)
print(id)
query_response = table_missing.query(KeyConditionExpression=Key('ID').eq(id))
OUTPUT:
i
-
x
x
x
x
x
EXPECTED OUTPUT:
i-xxxxx
etc etc
instance_id is a string. Thus, when you loop over it (for id in instance_id), you are actually looping over each character in the string, and printing them out individually.
Why do you try to loop over it, when you say that just printing it produces the correct result?
Is it possible to read and access the solver properties in Pycaffe?
I need to use some of the information stored in the solver file, but apparently the solver object which is created using
import caffe
solver = caffe.get_solver(solver_path)
is of no use in this case. Is there any other way to get around this problem?
I couldn't find what I was after using solver object and I ended up writing a quick function to get around this issue:
def retrieve_field(solver_path, field=None):
'''
Returns a specific solver parameter value using the specified field
or the whole content of the solver file, when no field is provided.
returns:
a string, as a field value or the whole content as list
'''
lines = []
field_segments = []
with open(solver_path, 'r') as file:
for line in file:
line = line.strip()
lines.append(line)
field_segments = line.split(':')
if (field_segments[0] == field):
#if that line contains # marks (for comments)
if('#' in field_segments[-1]):
idx = field_segments[-1].index('#')
return field_segments[-1][0:idx]
else:
return field_segments[-1]
return lines