I try to use date_format in Jupyter notebook in my sql query to get the correct data but Jupyter throws the following error:
sql = "SELECT DATE_FORMAT(issues.created_on, "%Y") as YEAR, issues.tracker_id as Ticketgrund, count(*) FROM issues where issues.project_id = '2' group by YEAR, Ticketgrund;"
^
SyntaxError: invalid syntax"
Here is my cell:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import sqlalchemy
session = sqlalchemy.create_engine("mysql+pymysql://user:pwj#ip/DB")
sql = "SELECT DATE_FORMAT(issues.created_on, "%Y") as YEAR, issues.tracker_id as Ticketgrund, count(*) FROM issues where issues.project_id = '2' group by YEAR, Ticketgrund;"
df = pd.read_sql_query(sql,session)
df.head(20)
The sql query runs without errors but not in jupyter. Escaping the %, which I assume to be the reason for error, does not change anything.
What can I do?
Escape the % with % (and use single quotes inside double quotes):
sql = "SELECT DATE_FORMAT(issues.created_on, '%%Y') as YEAR, issues.tracker_id as Ticketgrund, count(*) FROM issues where issues.project_id = '2' group by YEAR, Ticketgrund;"
Alternatively, use the date_format function available in SQLAlchemy. Here is a brief example:
from sqlalchemy import func
from sqlalchemy import MetaData, Table, Column
from sqlalchemy import select
from sqlalchemy import Integer, String, DateTime
md = MetaData()
mytable = Table(
"employees",
md,
Column("emp_no", Integer, primary_key=True),
Column("first_name", String(30)),
Column("last_name", String(30)),
Column("hire_date", DateTime),
)
row = session.execute( select(mytable.c.first_name, func.date_format(mytable.c.hire_date, "%Y") ) ).first()
print(row)
Related
I am new to SQLAlchemy & wanted to create an SQLAlchemy query equivalent to "order by exact match first".
Below is the SQL:
select word from dictionary where word like '%Time%' order by (word = 'Time') desc;
This is my SQLAlchemy equivalent.
Dictionary.query.with_entities ( Dictionary.word )
.filter(Dictionary.word.like("%{}%".format("Time")))
.order_by(Dictionary.word == "Time")
But it throws an error at "order_by": SyntaxError: keyword can't be an expression. How to solve it ?
Solved it.
from sqlalchemy.sql import func
.order_by(Dictionary.word == q,func.length(Dictionary.word))
I want to put randomised (will later be substituted with real numbers) numbers and a timestamp in a mySQL database.
I am running the SQL Server on my PC, if I copy & paste the comand in the SQL Terminal it works, but with python it generates errors
import pymysql
import random
import time
import datetime
def dynamic_data_entry():
date = "10.07.19"
Temperatur = str(random.randint(0, 100))
Feuchtigkeit = str(random.randint(20, 80))
Operation = 'INSERT INTO messwerte_sensor1 (Time, Temperatur, Luftfeuchtigkeit) VALUES (%s, %s, %s);' , (date, Temperatur, Feuchtigkeit)
db = pymysql.connect("127.0.0.1","root","Root","tests")
cursor = db.cursor()
cursor.execute(Operation)
data = cursor.fetchone
print(data)
db.close()
dynamic_data_entry()
The problem is with your date format. In mysql, standar date format is aaaa-mm-dd, sou you will need to change it. Also, i modify your code to use prepared statements:
import pymysql
import random
import time
import datetime
def dynamic_data_entry():
date = "2019-07-10"
Temperatur = str(random.randint(0, 100))
Feuchtigkeit = str(random.randint(20, 80))
Operation = 'INSERT INTO messwerte_sensor1 (Time, Temperatur, Luftfeuchtigkeit) VALUES (%s, %s, %s);'
db = pymysql.connect("127.0.0.1","root","Root","tests")
cursor = db.cursor()
cursor.execute(Operation,(date, Temperatur, Feuchtigkeit))
data = cursor.fetchone()
print(data)
db.close()
dynamic_data_entry()
I thin you need quotes around each %s, like "%s"
We want to wrtie this query in django please
SELECT sum(recommended='1') AS YES,sum(recommended='0') AS NO FROM `rating` WHERE applied_users = 32500
we have no idea how to use in sum "= 1"
Rating.objects.filter(applied_id = 32500).aggregate(YES=Sum('recommended'))
Use aggregate with case in django, it will return you single result with annotate it will return you a multiple records
https://docs.djangoproject.com/en/1.11/ref/models/conditional-expressions/
from django.db.models import Sum, Case, When, IntegerField
Rating.objects.filter(applied_id = 32500)
.aggregate(
yes=Sum(
Case(
When(recommended='1', then=1),
output_field=IntegerField()
)
),
no=Sum(
Case(
When(recommended=0,then=1),
output_field=IntegerField()
)
)
)
First refer this : https://docs.djangoproject.com/en/1.11/ref/models/conditional-expressions/
from django.db.models import Sum, Case, When, IntegerField
OR
from django.db.models.aggregates import Sum
from django.db.models.expressions import Case, When
ratings = Rating.objects
.annotate(yes_count=Sum(Case(When(recommended='1', then=1),output_field=IntegerField())))
.annotate(no_count=Sum(Case(When(recommended='0', then=1),output_field=IntegerField())))
.filter(applied_id = 32500)
What is the most effective way to insert data dumped on the database A into the database B? Normally I would use mysqldump for the task like this, but because of the complex query I had to take a different approach. At present I have the following inefficient solution:
from sqlalchemy import create_engine, Column, INTEGER, CHAR, VARCHAR
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
SessFactory = sessionmaker()
print('## Configure database connections')
db_one = create_engine('mysql://root:pwd1#127.0.0.1/db_one', echo=True).connect()
sess_one = SessFactory(bind=db_one)
db_two = create_engine('mysql://root:pwd2#127.0.0.2/db_two', echo=True).connect()
sess_two = SessFactory(bind=db_two)
## Declare query to dump data
dump_query = (
'SELECT A.id, A.name, B.address '
'FROM table_a A JOIN table_b B '
'ON A.id = B.id_c WHERE '
'A.deleted = 0'
)
print('## Fetch data on db_one')
data = db_one.execute(dump_query).fetchall()
## Declare table on db_two
class cstm_table(Base):
__tablename__ = 'cstm_table'
pk = Column(INTEGER, primary_key=True)
id = Column(CHAR(36), nullable=False)
name = Column(VARCHAR(150), default=None)
address = Column(VARCHAR(150), default=None)
print('## Recreate "cstm_table" on db_two')
cstm_table.__table__.drop(bind=db_two, checkfirst=True)
cstm_table.__table__.create(bind=db_two)
print('## Insert dumped data into the "cstm_table" on db_two')
for row in data:
insert = cstm_table.__table__.insert().values(row)
db_two.execute(insert)
This execute sequentially over a 100K inserts (horrible).
I also tried:
with db_two.connect() as conn:
with conn.begin() as trans:
row_as_dict = [dict(row.items()) for row in data]
try:
conn.execute(cstm_table.__table__.insert(), row_as_dict)
except:
trans.rollback()
raise
else:
trans.commit()
But then after inserting ~20 rows I get error:
OperationalError: (_mysql_exceptions.OperationalError) (2006, 'MySQL server has gone away')
The following also does the job, but I'm not so sure it's the most efficient:
sess_two.add_all([cstm_table(**dict(row.items())) for row in data])
sess_two.flush()
sess_two.commit()
How do I get the MIN() of a datetime column and a literal datetime in SQL Alchemy v0.6.4? I want to clamp a datetime result to a specific range.
I've been using sqlalchemy.func.min(column, literal_datetime). This seems to work fine in SQLite but not at all with MySQL, which no doubt means I'm going about this wrong. The error from MySQL is:
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' '2011-06-14 12:30:00') AS min_1 \nFROM example' at line 1") 'SELECT min(example.my_datetime, %s) AS min_1 \nFROM example' (datetime.datetime(2011, 6, 14, 12, 30),)
How can I clamp datetime results in a portable way?
(Do I really have to do it in my application?)
Here's what I've been using to explore the problem - works fine as presented here (using a memory-based SQLite DB), doesn't work with MySQL:
#!/usr/bin/env python
import random
from datetime import datetime
import sqlalchemy as sa
import sqlalchemy.orm as orm
from sqlalchemy.ext.declarative import declarative_base
orm_base = declarative_base()
class MyClass( orm_base ):
__tablename__ = "example"
pri_key = sa.Column(sa.Integer(), primary_key=True)
my_datetime = sa.Column(sa.DateTime(), index=True)
engine = sa.create_engine("sqlite:///:memory:", echo=True)
Session = orm.sessionmaker( bind=engine )
orm_base.metadata.bind = engine
orm_base.metadata.create_all()
# Create test-data
session = Session()
random.seed(1234567980)
for i in range(100):
month = random.randint(1, 12)
day = random.randint(1, 28)
hour = random.randint(0, 23)
minute = random.randint(0, 59)
my_instance = MyClass()
my_instance.my_datetime = datetime(2011, month, day, hour, minute)
session.add( my_instance )
session.commit()
session.close()
# Problem starts here
session = Session()
literal_datetime = datetime(2011, 06, 14, 12, 30)
print session.query( sa.func.min(MyClass.my_datetime) ).one() # OK
print session.query( sa.func.min(MyClass.my_datetime, literal_datetime) ).all() # Error in MySQL
I'm not an SQL expert but database portability is important to me, so any tips on how to avoid pitfalls like this in the future are welcome.
Use sqlalchemy.sql.expression.case expression, which is supported on both mySql and SQLite.
The min/max functions of SQLite support non-aggregate operation on multiple values. This is not supported on mySql though.