SQL to SQLALCHEMY - sqlalchemy

Base on this post, I have this SQL query:
SELECT
id, (6371.009 * acos(cos(radians(37)) * cos(radians(lat)) *
cos(radians(lng) - radians(-122)) + sin(radians(37)) *
sin(radians(lat)))) AS distance
FROM
company
HAVING
distance < 25
ORDER BY
distance
LIMIT
0 , 20;
and I want to convert this query to SQLALCHEMY. How I can convert SQL queries that are included of functions (e.g. acos, cos, sin, radians, ...) to SQLALCHEMY queries?
I have tried this:
company_nearby = s.query(Company).filter(
(6371.009 * acos(
cos(radians(37)) * cos(radians(Company.latitude)) *
cos(radians(Company.longtitude) - radians(-122)) +
sin(radians(37)) * sin(radians(Company.latitude))
)) < 25
).limit(20).all()
But I have this error:
TypeError: a float is required
Probably because Company.latitude returns None, but company table has 16000 record with latitude and longitude.
This is Company table:
class Company(db.Model):
__tablename__ = 'company'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(255))
latitude = db.Column(db.Float)
longtitude = db.Column(db.Float)
Any help will be much appreciated.

To use functions within your database you can use the func method: http://docs.sqlalchemy.org/en/latest/core/functions.html#sqlalchemy.sql.functions.func
Something like this for example:
print select([func.cos(func.radians(table.c.lng))

Using raw query execution is one of the methods to run SQL queries in SQLALCHEMY. for this example this is how I proceed:
from math import sqrt, radians, cos, acos, sin, pow
...
raw_con = engine.raw_connection()
raw_con.create_function("cos", 1, cos)
raw_con.create_function("acos", 1, acos)
raw_con.create_function("sin", 1, sin)
raw_con.create_function("radians", 1, radians)
raw_con.create_function("sqrt", 1, sqrt)
raw_con.create_function("pow", 1, pow)
results = []
try:
cursor = raw_con.cursor()
cursor.execute('SELECT id, '
'( 6371.009 * acos( cos( radians(37) ) * '
'cos( radians( lat ) ) * cos( radians( lng ) - radians(-122) ) + '
'sin( radians(37) ) * sin( radians( lat ) ) ) ) AS distance '
'FROM company '
'WHERE distance < 25 '
'ORDER BY distance '
'LIMIT 0 , 20 ;')
results = cursor.fetchall()
cursor.close()
finally:
raw_con.close()
and:
print results
gives:
[(68, 4958.2844864897925), (50, 4958.399769765616), (17, 4958.417639125849), (13, 4958.701413904964), (16, 4958.720521740769), (46, 4958.758729888283), (54, 4958.769050291066), (47, 4958.789468071686), (57, 4958.833125409799), (14, 4958.843715312685), (45, 4958.853
383812306), (60, 4958.870656980226), (35, 4958.901153544977), (65, 4958.901643917998), (37, 4958.941355727554), (33, 4958.974325724986), (41, 4958.985977481861), (1, 4959.002398001085), (38, 4959.093650178002), (8, 4959.142373411243)]

Related

Converting row sql sub query into Codeigniter Active record

I have a sql which is running perfectly okie. I want to convert it into active record query but don't know how to convert these sub query into active record
below is my sql:
Select A.*, B.*
FROM
( SELECT ( 6371 * acos( cos( radians(24.8964089) ) * cos( radians(RB.`lat`) )
* cos( radians(RB.`lng`) - radians(67.06749)) + sin(radians(24.8964089))
* sin( radians(RB.`lat`)))) AS distance, RB.`id` as BRID , R.`name` as NAME
FROM
`rent_br` RB
JOIN `rent` R ON R.`id` = RB.`restId`
JOIN `rent_category` RC on R.`id` = RC.`restId`
WHERE
TRUNCATE(RB.`lat`, 0) >= TRUNCATE(24.8964089,0)
AND TRUNCATE(RB.`lat`, 0) < TRUNCATE(25.8964089,0)
AND RB.`isdelivery` = '1'
AND RC.`catId` = 11
/*AND ( rent.`tags` like '%burger%' OR rent.`name` like '%burger%' OR info.`name` like '%burger%' )*/
HAVING
`distance` <= 50
ORDER BY
`distance` ASC ) A
JOIN
(select max(radius) as maxRadius, rent_br_delivery_radius.`rest_brId` as bbrId from `rent_br_delivery_radius`
group by bbrId) B on `A`.BRID = B.bbrId
WHERE A.distance <= B.maxRadius
ORDER BY A.NAME
the only thing you have to consider here is the use of the get_compiled_select function of the query builder - once you understand that - its pretty easy. You just have to split your queries and merge them together to one query
the following should do the job
your subselect
$strSubQuery = $this->db
->select('
( 6371 * acos( cos( radians(24.8964089) ) * cos( radians(RB.`lat`) )
* cos( radians(RB.`lng`) - radians(67.06749)) + sin(radians(24.8964089))
* sin( radians(RB.`lat`)))) AS distance, RB.`id` as BRID , R.`name` as NAME
')
->from('rent_br RB')
->join('rent r', 'r.id = RB.restId')
->join('rent_category RC', 'R.id = RC.restId')
->where('TRUNCATE(RB.`lat`, 0) >= TRUNCATE(24.8964089,0)', NULL, false)
->where('TRUNCATE(RB.`lat`, 0) < TRUNCATE(25.8964089,0)', NULL, false)
->where('RB.isdelivery`', 1)
->where('RC.catId', 11)
->having('distance <=', 50)
->order_by('distance', 'ASC')
->get_compiled_select();
your subjoin query
$strSubQueryJoin = $this->db
->select_max('radius', 'maxRadius')
->select('rent_br_delivery_radius.rest_brId AS bbrId')
->from('rent_br_delivery_radius')
->group_by('bbrId')
->get_compiled_select();
and finally we put this pieces together
$query = $this->db
->select('A.*, B.*')
->from($strSubQuery.' A', false)
->join('('.$strSubQueryJoin.') B', 'A.BRID = B.bbrId', 'INNER', false)
->where('A.distance <= B.maxRadius', NULL, false)
->order_by('A.NAME')
->get();
That should pretty much do the job.

MySQL check if row with category exists

I'm using the Haversine formula with this query and it works until. The goal is to check if each row has certain categories in an value of $deflin which looks like $deflin = category1, category2, category3. The results will show rows within 50km and if it contains any of the categories defined by $deflin. Not sure how to approach this either with WHERE IN or LIKE. Any help is appreciated.
MySQL for query
$awaka = "SELECT *,
( 6371 * acos( cos( radians(?) ) * cos( radians(job_latitude) ) *
cos(radians(?) - radians(job_longitude) ) + sin( radians(?) ) *
sin( radians(job_latitude) ) ) ) AS distance FROM job, users
WHERE job.listee_id = users.user_id AND job.job_category LIKE ?
HAVING distance < 50";
$result = $this->db->query($awaka, array($conlat, $conlong, $conlat, $deflin));
I don't think you like to use the LIKE operator since it only searches for patterns within a column and you only wan't to return rows that really has one of your categories. Instead you should use an IN clause to check if a job has one of your categories:
// contains the id for each category
$categories = array(1, 4, 5, 6);
$deflin = implode(',', $categories);
// if job_category is a text column you could do like this instead
$categories = array('category1', 'category2', 'category3');
$deflin = implode(',', $categories);
$awaka = "SELECT *,
( 6371 * acos( cos( radians(?) ) * cos( radians(job_latitude) ) *
cos(radians(?) - radians(job_longitude) ) + sin( radians(?) ) *
sin( radians(job_latitude) ) ) ) AS distance FROM job, users
WHERE job.listee_id = users.user_id AND job.job_category IN ($deflin)
HAVING distance < 50";

SQL UNION not working with 2 HAVING clauses

I want to use UNION to join two SQL SELECT queries. I need the final data to use the HAVING clause to filter the entire query. Here is my statement:
SELECT CLIENT,
BIZNAME,
BIZSTREET,
BIZCITY,
BIZSTATE,
BIZZIP,
BIZPHONE,
URL,
LAT,
LNG,
CONSOLIDATED,
( 3959 * ACOS(COS(RADIANS('%s')) * COS(RADIANS(LAT)) * COS(
RADIANS(LNG) - RADIANS('%s'))
+ SIN
(RADIANS('%s')) * SIN(RADIANS(LAT))) ) AS distance
FROM BizInfo
INNER JOIN WebSites
ON WebSites.CUSTOMER = BizInfo.CUSTOMER
WHERE BizInfo.CLIENT = 'GCB'
AND WebSites.STATUS <> 'Cancel'
AND WebSites.STATUS <> 'In Progress'
AND WebSites.STATUS <> 'Review'
AND WebSites.STATUS <> 'Testing'
UNION
SELECT CLIENT,
BIZNAME,
BIZSTREET,
BIZCITY,
BIZSTATE,
BIZZIP,
BIZPHONE,
'http://www.abc-site.com',
LAT,
LNG,
'0',
( 3959 * ACOS(COS(RADIANS('%s')) * COS(RADIANS(LAT)) * COS(
RADIANS(LNG) - RADIANS('%s'))
+ SIN
(RADIANS('%s')) * SIN(RADIANS(LAT))) ) AS distance
FROM BizInfo
WHERE CLIENT = 'GCB'
AND BIZNAME = 'Acme'
HAVING DISTANCE < '%s'
ORDER BY DISTANCE
LIMIT 0, 200
I read on this site http://www.really-fine.com/SQL_union.html (GROUP BY and HAVING clauses can be used only within individual queries and cannot be used to affect the final results set. ), but I don't understand how to implement this or if it is correct.
How do I properly write this SQL query?
You can probably solve that very easily by wrapping everything into a subquery. Something like :
SELECT
*
FROM
(
SELECT Client, BizName, BizStreet, BizCity, BizState, BizZip, BizPhone, url, lat, lng, Consolidated,
( 3959 * acos( cos( radians('%s') ) * cos( radians( lat ) ) * cos( radians( lng ) - radians('%s') ) + sin( radians('%s') ) * sin( radians( lat ) ) ) )
AS distance FROM BizInfo
INNER JOIN WebSites ON WebSites.Customer = BizInfo.Customer
WHERE BizInfo.Client = 'GCB'
AND WebSites.Status <> 'Cancel' AND WebSites.Status <> 'In Progress' AND WebSites.Status <> 'Review' AND WebSites.Status <> 'Testing'
UNION SELECT Client, BizName, BizStreet, BizCity, BizState, BizZip, BizPhone, 'http://www.abc-site.com', lat, lng, '0',
( 3959 * acos( cos( radians('%s') ) * cos( radians( lat ) ) * cos( radians( lng ) - radians('%s') ) + sin( radians('%s') ) * sin( radians( lat ) ) ) )
AS distance FROM BizInfo WHERE Client = 'GCB' AND BizName = 'Acme'
) AS ClientInfo
WHERE
distance < '%s'
ORDER BY
distance
LIMIT 0 , 200
This is the fastest path to what you want but it is not very clean.
Also please tell me that all those parameters are not vulnerable to SQL injection...?
HAVING can only be used in an aggregate query with a GROUP BY clause. Your queries do not. Use WHERE instead.
Those are scalar functions, not aggregate functions. So to filter on the result, use the WHERE clause, not the HAVING clause. You are filtering horizontally, not vertically.
You cannot use a column alias in the WHERE clause, so you have to restate the functions used in the SELECT list (I've done that in the query below).
Try the following. There are 2 issues at play.
SELECT Client,
BizName,
BizStreet,
BizCity,
BizState,
BizZip,
BizPhone,
url,
lat,
lng,
Consolidated,
(3959 * acos(cos(radians('%s')) * cos(radians(lat)) *
cos(radians(lng) - radians('%s')) +
sin(radians('%s')) * sin(radians(lat)))) AS distance
FROM BizInfo
INNER JOIN WebSites
ON WebSites.Customer = BizInfo.Customer
WHERE BizInfo.Client = 'GCB'
AND WebSites.Status <> 'Cancel'
AND WebSites.Status <> 'In Progress'
AND WebSites.Status <> 'Review'
AND WebSites.Status <> 'Testing'
UNION
SELECT Client,
BizName,
BizStreet,
BizCity,
BizState,
BizZip,
BizPhone,
'http://www.abc-site.com',
lat,
lng,
'0',
(3959 * acos(cos(radians('%s')) * cos(radians(lat)) *
cos(radians(lng) - radians('%s')) +
sin(radians('%s')) * sin(radians(lat)))) AS distance
FROM BizInfo
WHERE Client = 'GCB'
AND BizName = 'Acme'
and (3959 * acos(cos(radians('%s')) * cos(radians(lat)) *
cos(radians(lng) - radians('%s')) +
sin(radians('%s')) * sin(radians(lat)))) < '%s'
ORDER BY distance LIMIT 0, 200

Mysql select from inner select with join

Can I do something like this ? Bassically I user inner select to select lowest date that is bigger than NOW(), for example 9.4.2014 would mathc this, for each event ids. And now I would like to return this, here is the code :
SELECT id_event , event_title, event_details, dates.event_date, id_show, id_category, distance FROM(
SELECT event.id_event id_event, event_title, event_details, min(event_date_time) event_date_time, event_showtime.id_show , event_category.id_category id_category, ( 6371 * ACOS( COS( RADIANS( '49.20513921227407' ) ) * COS( RADIANS( event_showtime.latitude ) ) * COS( RADIANS( event_showtime.longitude ) - RADIANS( '18.762441839599678' ) ) + SIN( RADIANS( '49.20513921227407' ) ) * SIN( RADIANS( event_showtime.latitude ) ) ) ) AS distance
FROM event
JOIN event_showtime ON event.id_event = event_showtime.id_event
JOIN event_category ON event.id_category = event_category.id_category
JOIN ( SELECT id_event, min(event_date_time) as event_date_time FROM event_showtime
WHERE event_date_time > NOW()
GROUP BY id_event ) AS dates ON event.id_event = dates.id_event
WHERE event_date_time > NOW()
GROUP BY event.id_event
HAVING distance < '5'
ORDER BY distance
LIMIT 0 , 20
)t
Here is the SQLfiddle http://www.sqlfiddle.com/#!2/6545ab/51 with 3 tables :)
How could I access those inner details in the top select ?
You need to give the table aliases to your columns if there are multiple columns that have same names
SELECT
es.id_show,
e.id_event ,
e.event_title,
e.event_details,
d.event_date_time,
ec.id_category,
( 6371 * ACOS( COS( RADIANS( '49.20513921227407' ) ) * COS( RADIANS( es.latitude ) ) * COS( RADIANS( es.longitude ) - RADIANS( '18.762441839599678' ) ) + SIN( RADIANS( '49.20513921227407' ) ) * SIN( RADIANS( es.latitude ) ) ) ) AS distance
FROM event e
JOIN event_showtime es ON e.id_event = es.id_event
JOIN event_category ec ON e.id_category = ec.id_category
JOIN ( SELECT id_event,
min(event_date_time) as event_date_time
FROM event_showtime
WHERE event_date_time > NOW()
GROUP BY id_event ) AS d
ON (es.id_event = d.id_event AND es.event_date_time =d.event_date_time)
WHERE d.event_date_time > NOW()
GROUP BY e.id_event
-- HAVING distance < '5'
ORDER BY distance
LIMIT 0 , 20
Here is the simplified version of your query you are using HAVING clause to check the distance should be less than 5 but for second event the distance is 112.773868864733 in my query i have commented the having clause just to show your the results,second thing you asked for the nearest date to NOW() but not include past dates so in your provided sample data set for event_showtime
`id_show`, `id_event`, `latitude`, `longitude`, `event_date_time`
(1, 1, 49.2016762922894, 18.7615620750428, '2014-03-31 16:13:17'),
(2, 1, 49.2016762922894, 18.7615620750428, '2014-04-01 20:00:00'),
(3, 2, 49.2113914818564, 18.7520992416382, '2014-03-31 15:00:00'),
(4, 2, 49.0545135142313, 20.2952223676682, '2014-04-16 11:00:00'),
(5, 2, 49.2113914818564, 18.7520992416382, '2014-04-23 11:00:00'),
(6, 2, 49.0545135142313, 20.2952223676682, '2014-04-30 11:00:00'),
(7, 2, 49.2016762922894, 18.7615620750428, '2014-04-29 12:00:00'),
(8, 1, 49.2016762922894, 18.7615620750428, '2014-04-24 12:00:00');
For event id 1 there are 3 datetimes 2014-03-31 16:13:17 ,2014-04-01 20:00:00,2014-04-24 12:00:00 the date 2014-04-24 12:00:00 is the nearest one for event id 2 the date 2014-04-16 11:00:00 is the nearest one not 23rd one ,third thing you asked in comments that id_show is wrong for this your last join from subquery needs another condition with event_date_time to be joined to get the exact id you need
Hope it makes sense
Fiddle Demo
For your specific requirement to select nearest dates with distance limitation you can do so
SELECT t.* FROM (
SELECT
es.id_show,
e.id_event,
e.event_title,
e.event_details,
MIN(es.event_date_time) event_date_time,
ec.id_category,
es.distance
FROM
event e
JOIN event_category ec
ON e.id_category = ec.id_category
JOIN
(SELECT
`id_show`,
`id_event`,
`latitude`,
`longitude`,
event_date_time,
(
6371 * ACOS(
COS(RADIANS('49.20513921227407')) * COS(RADIANS(event_showtime.latitude)) * COS(
RADIANS(event_showtime.longitude) - RADIANS('18.762441839599678')
) + SIN(RADIANS('49.20513921227407')) * SIN(RADIANS(event_showtime.latitude))
)
) AS distance
FROM
event_showtime
WHERE `event_date_time` > NOW()
HAVING distance < 5) es
ON (e.id_event = es.id_event)
GROUP BY e.id_event
) t
JOIN `event_showtime` es USING(id_event,event_date_time)
ORDER BY t.distance
LIMIT 0, 20
Fiddle Demo

Use result of substring function to compute values for multiple columns

I am trying to run to find the closest point with a long and lat. Which works fine is i have separate value stored in my db, however i have a single piped string, which i am able to split with a substring index. However when i try and combine these functions in my select query i am not getting any joy.
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 1), '|', -1) as 'lat',
SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 2), '|', -1) as 'lng',
title,
( 6371 * acos( cos( radians(51.527351) ) * cos( radians( 'lat') ) * cos( radians( 'lng' ) - radians(0.765544) ) + sin( radians(51.527351) ) * sin( radians( 'lat' ) ) ) ) AS distance
FROM locations HAVING distance < 500
ORDER BY distance
LIMIT 0 , 20
Can anyone shed some light?
radians( 'lat')
In this part of the query, 'lat' is a string, not a column name or similar. When passing that string to radians, it will be converted to a number, in this case to zero. So you're converting 0 to radians.
You cannot use one column as the input to the computation of the value of another column. You either have to repeat the expression, or use a subquery like this:
SELECT lat, lng, title,
(6371 * acos(cos(radians(51.527351)) *
cos(radians(lat)) * cos(radians(lng) - radians(0.765544)) +
sin(radians(51.527351)) * sin(radians(lat)))
) AS distance
FROM
(SELECT title,
SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 1), '|', -1) + 0 as lat,
SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 2), '|', -1) + 0 as lng
FROM locations
) AS sub
HAVING distance < 500
ORDER BY distance
LIMIT 0 , 20
Here was my solution which repeats the query. I haven;t had a chance to test of benchmark the differing solutions, but the subquery solution is more pleasant
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 1), '|', -1) as 'latitude',
SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 2), '|', -1) as 'longitude',
(
3959 * acos(
cos(radians({MYLAT})) * cos(radians(SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 1), '|', -1)))
* cos(radians(SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 2), '|', -1))) - radians({MYLONG}) + sin(radians({MYLAT}))
* sin( radians(SUBSTRING_INDEX(SUBSTRING_INDEX(longlat, '|', 1), '|', -1)))
)
)
AS distance,
title
FROM locations
ORDER BY distance
LIMIT 0 , 20
Turns out i'm going to be using this equation everywhere now, so glad to get it nailed :)
I'm using the averaged 3959 as the earth's radius, however i wonder if i might be better to use some kind regional latitude for different global locations.