I want to know an efficient way to solve the following query. Essentially I have the following two classes
CREATE TABLE `example`.`doc` (
`id` INT NOT NULL AUTO_INCREMENT,
`uuid` INT NOT NULL,
`creator` VARCHAR(32) NOT NULL,
PRIMARY KEY (`id`);
CREATE TABLE `example`.`pic` (
`id` INT NOT NULL AUTO_INCREMENT,
`docuuid` INT NOT NULL,
`taken_at` DATETIME DEFAULT CURRENT_TIMESTAMP,
`lat` DECIMAL(3,10),
`lon` DECIMAL(3,10),
PRIMARY KEY (`id`);
this two tables are related by uuid (I know this is not the best idea to link two tables, but the table is like this). uuid is unique per doc. With the following query
SELECT
`doc`.`id` AS `docId`,
`doc`.`uuid` AS `uuid`,
`doc`.`creator` AS `creator`,
COUNT(`pic`.`id`) AS `pics`,
MIN(`pic`.`taken_at`) AS `min_date`,
MAX(`pic`.`taken_at`) AS `max_date`
FROM
`doc` INNER JOIN
`pic` ON (`doc`.`uuid` = `pic`.`docuuid`)
WHERE (
`pic`.`docuuid` IS NOT NULL AND
`doc`.`uuid` IS NOT NULL)
GROUP BY `doc`.`uuid`
I get, for each doc, the date at witch the first and last pics was taken. Additionally I want to get in this query, for each doc, the latitude and longitude of the first and last pic taken at that doc.
For example, If I have
doc1 = (id=0, uuid=1)
doc2 = (id=1, uuid=2)
pic1 = (taken_at=2021-01-01, docuuid=1, lat=1, lon=2)
pic2 = (taken_at=2021-01-02, docuuid=1, lat=3, lon=4)
pic3 = (taken_at=2021-01-03, docuuid=2, lat=5, lon=6)
pic4 = (taken_at=2021-01-04, docuuid=2, lat=7, lon=8)
pic5 = (taken_at=2021-01-05, docuuid=2, lat=9, lon=10)
then I want to get for doc1 minLat=1, minLon=2, maxLat=3, maxLon=4 and for doc2 minLat=5, minLon=6, maxLat=9, maxLon=10
You mat continue by joining the results of your current query with the pics table and extracting the desired details of each pic, I've included a sample below:
SELECT
t.*,
earliest_pic.lat as minLat,
earliest_pic.lon as minLon,
latest_pic.lat as maxLat,
latest_pic.lon as maxLon
FROM
(
SELECT
`doc`.`id` AS `docId`,
`doc`.`uuid` AS `uuid`,
`doc`.`creator` AS `creator`,
COUNT(`pic`.`id`) AS `pics`,
MIN(`pic`.`taken_at`) AS `min_date`,
MAX(`pic`.`taken_at`) AS `max_date`
FROM
`doc` INNER JOIN
`pic` ON (`doc`.`uuid` = `pic`.`docuuid`)
GROUP BY `doc`.`uuid`
) AS t
INNER JOIN `pic` as earliest_pic ON earliest_pic.taken_at = t.min_date AND
earliest_pic.docuuid = t.docId
INNER JOIN `pic` as latest_pic ON latest_pic.taken_at = t.max_date AND
latest_pic.docuuid = t.docId
Let me know if this works for you
Related
Is this query optimised?
I'm trying to get the list of people personA follows who follow personB.
Few thousand of rows right now in the table but growing fast.
Want to make sure the query is performant enough for mysql.
Query:
select
*
from
(
select
*
from
creator_followers cf
where
cf.follower_user_id = 'personA'
and cf.current_active = 1
and cf.current_following = 1
) as fo
join creator_followers cf
where
fo.creator_user_id = cf.follower_user_id
and cf.creator_user_id = 'personB'
and cf.current_following = 1
and cf.current_active = 1
order by
cf.created_at desc
limit
10 offset 0;
Schema:
create table social.creator_followers
(
creator_user_id varchar(16) charset utf8 null,
follower_user_id varchar(16) charset utf8 null,
current_following bit null,
current_active bit null,
created_at bigint null,
id bigint auto_increment
primary key
)
It seems that it must be
SELECT *
FROM creator_followers cf1
join creator_followers cf2 ON cf1.creator_user_id = cf2.follower_user_id
where cf1.follower_user_id = 'personA'
and cf2.creator_user_id = 'personB'
and cf1.current_active = 1
and cf1.current_following = 1
and cf2.current_following = 1
and cf2.current_active = 1
order by cf2.created_at desc
limit 10 offset 0;
Based on a Comment, this might provide another speedup. Replace
id bigint auto_increment
primary key
with
PRIMARY KEY(creator_user_id, follower_user_id)
I have three separate tables, bs_products, td_products, and fv_products.
SELECT
bso.order_id, bso.order_num, bso.Salesman, bso.salesman_name, bso.date, bso.status,
bsp.product_id, bsp.product_branch, bsp.product_name, bsp.description, bsp.vatable, bsp.critical_stock,
bsp.quantity, bsp.sell_price, bsp.category_id, bsp.expiry_date, bsp.date, bsp.isEmpty, bsp.empties,
bss.sws_id, bss.sws_proname, bss.sws_isEmpty, bss.sws_category, bss.sws_unitprice, bss.sws_prodesc,
bss.sws_proexp, bss.sws_vat, bss.sws_number, bss.date, bss.sws_salesman, bss.sws_route, bss.sws_smname,
bss.sws_driver, bss.sws_plate, bss.sws_vehicle, bss.sws_load, bss.sws_productid, bss.sws_quantity
FROM bs_orders as bso INNER JOIN bs_products as bsp
INNER JOIN bs_sws as bss WHERE status = 'Completed'
AND bso.date = '11/30/2016'
AND bso.order_num = bss.sws_number
AND bss.sws_productid = bsp.product_id
UNION
SELECT
tdo.order_id, tdo.order_num, tdo.Salesman, tdo.salesman_name, tdo.date, tdo.status,
tdp.product_id, tdp.product_branch, tdp.product_name, tdp.description, tdp.vatable, tdp.critical_stock,
tdp.quantity, tdp.sell_price, tdp.category_id, tdp.expiry_date, tdp.date, tdp.isEmpty, tdp.empties,
tds.sws_id, tds.sws_proname, tds.sws_isEmpty, tds.sws_category, tds.sws_unitprice, tds.sws_prodesc,
tds.sws_proexp, tds.sws_vat, tds.sws_number, tds.date, tds.sws_salesman, tds.sws_route, tds.sws_smname,
tds.sws_driver, tds.sws_plate, tds.sws_vehicle, tds.sws_load, tds.sws_productid, tds.sws_quantity
FROM td_orders as tdo INNER JOIN td_products as tdp
INNER JOIN td_sws as tds WHERE status = 'Completed'
AND tdo.date = '11/30/2016'
AND tdo.order_num = tds.sws_number
AND tds.sws_productid = tdp.product_id
UNION
SELECT
fvo.order_id, fvo.order_num, fvo.Salesman, fvo.salesman_name, fvo.date, fvo.status,
fvp.product_id, fvp.product_branch, fvp.product_name, fvp.description, fvp.vatable, fvp.critical_stock,
fvp.quantity, fvp.sell_price, fvp.category_id, fvp.expiry_date, fvp.date, fvp.isEmpty, fvp.empties,
fvs.sws_id, fvs.sws_proname, fvs.sws_isEmpty, fvs.sws_category, fvs.sws_unitprice, fvs.sws_prodesc,
fvs.sws_proexp, fvs.sws_vat, fvs.sws_number, fvs.date, fvs.sws_salesman, fvs.sws_route, fvs.sws_smname,
fvs.sws_driver, fvs.sws_plate, fvs.sws_vehicle, fvs.sws_load, fvs.sws_productid, fvs.sws_quantity
FROM fv_orders as fvo INNER JOIN fv_products as fvp
INNER JOIN fv_sws as fvs WHERE status = 'Completed'
AND fvo.date = '11/30/2016'
AND fvo.order_num = fvs.sws_number
AND fvs.sws_productid = fvp.product_id
I tried running this on phpMyAdmin but it gives me an error of
1221 - Incorrect usage of UNION and LIMIT
What is the correct query for this?
_orders Table
order_id (Primary) int(11)
order_num int(255)
Salesman varchar(255)
salesman_name varchar(255)
date varchar(255)
status varchar(255)
_products Table
product_id (Primary) int(11)
product_branch varchar(255)
product_name varchar(255)
description varchar(255)
vatable tinyint(1)
critical_stock int(11)
quantity int(11)
sell_price double
category_id int(11)
expiry_date varchar(255)
date varchar(255)
isEmpty tinyint(1)
empties int(11)
_sws Table
sws_id (Primary) int(11) No
sws_proname varchar(255)
sws_isEmpty tinyint(1)
sws_category int(11)
sws_unitprice double
sws_prodesc varchar(255)
sws_proexp varchar(255)
sws_vat tinyint(1)
sws_number int(255)
date varchar(255)
sws_salesman varchar(255)
sws_route varchar(255)
sws_smname varchar(255)
sws_driver varchar(255)
sws_plate varchar(255)
sws_vehicle varchar(255)
sws_load int(255)
sws_productid int(255)
sws_quantity int(255)
When using UNION the number of columns that are in each subquery has to be the same and they have to be of the same data type.
You will need to adjust your query so that you are not using * but defining each column from each table.
See the following for more info:
http://www.mysqltutorial.org/sql-union-mysql.aspx
https://dev.mysql.com/doc/refman/5.7/en/union.html
There are some rules that you need to follow in order to use the UNION operator:
The number of columns appears in the corresponding SELECT statements must be equal.
The columns appear in the corresponding positions of each SELECT statement must have the same data type or, at least, convertible data type.
You need to re-think you plan.
To do a union you need fields of the same names, and data types. You can't simply select * and expect different tables to combine in a single result set.
You may need to change names or add dummy columns in different parts of the select to make it work.
If you post the table structures and field names people may be able to help you more.
Table 1 - this table is completely populated from an XLSX file...
Tables 2 and 3 - contain 1-1 references for a couple of the columns in the final table.
Table 3 - the on I am trying to get populated with first three tables....
As you can see, the tables are sensible, there is no einsteinic equations or conversions going on. Here is the code that I have already tried, unsuccessfully:
INSERT INTO att_oem_orders SELECT NULL, ost.om_or_po, (SELECT j.job_id FROM jobs j WHERE j.project_number = project_no), NULL, (SELECT ao.id FROM att_oem WHERE ao.item_no = item_no), ost.po_number, (SELECT ol.id FROM order_lsc WHERE STATUS = ol.line_status_code), ost.ordered_date, ost.shipment_date, NULL, NULL, ost.item_qty, NULL, NULL, NULL, NULL, ost.shipping_to, ost.tracking_number, ost.carrier) FROM oem_temp_sync WHERE ost.item_qty > 0
Try limiting the number of possible values to put into a field using FIRST() like so:
INSERT INTO att_oem_orders SELECT
NULL,
ost.om_or_po,
(SELECT FIRST(j.job_id) FROM jobs j WHERE j.project_number = project_no),
NULL,
(SELECT FIRST(ao.id) FROM att_oem WHERE ao.item_no = item_no),
ost.po_number,
(SELECT FIRST(ol.id) FROM order_lsc WHERE STATUS = ol.line_status_code),
ost.ordered_date,
ost.shipment_date,
NULL,
NULL,
ost.item_qty,
NULL,
NULL,
NULL,
NULL,
ost.shipping_to,
ost.tracking_number,
ost.carrier
FROM oem_temp_sync WHERE ost.item_qty > 0
You also appear to have an extra ) just before FROM
With a little bit of help from everyone, I've found a few errors and fixed a few sub-queries. Here is the final version:
INSERT INTO att_oem_orders
SELECT
NULL,
ost.om_or_po,
(SELECT
FIRST(j.id)
FROM
jobs j
WHERE j.project_number = ost.project_no),
NULL,
(SELECT
FIRST(ao.id)
FROM
att_oem ao
WHERE ao.item_no = ost.item_no),
ost.po_number,
(SELECT
FIRST(ol.id)
FROM
att_oem_lsc ol
WHERE ol.status = ost.line_status_code),
ost.ordered_date,
ost.shipment_date,
NULL,
NULL,
ost.item_qty,
NULL,
NULL,
NULL,
NULL,
ost.ship_to,
ost.tracking_no,
ost.carrier
FROM
oem_sync_temp ost
WHERE ost.item_qty > 0
I'm having trouble with the following query:
SELECT
job.name,
job_element.label,
job_element_role_hours.role,
job_element_role_hours.hours_budgeted,
latest_rate.hourly_rate,
( job_element_role_hours.hours_budgeted * latest_rate.hourly_rate ) AS line_cost
FROM
job_element
INNER JOIN job ON job_element.job = job.id
INNER JOIN job_element_role_hours ON job_element_role_hours.element = job_element.id
LEFT JOIN(
SELECT
rate.*
FROM
rate
LEFT JOIN rate AS newest ON (
rate.role = newest.role
AND COALESCE(rate.client_company, 1) = COALESCE(newest.client_company, 1)
AND COALESCE(rate.client_group, 1) = COALESCE(newest.client_group, 1)
AND COALESCE(rate.client_contact, 1) = COALESCE(newest.client_contact, 1)
AND newest.date_from > rate.date_from
)
WHERE newest.id IS NULL
) AS latest_rate ON (
latest_rate.role = job_element_role_hours.role
AND (
COALESCE(latest_rate.client_company, 1) = COALESCE(job.client_company, 1)
OR latest_rate.client_company IS NULL
)
AND (
COALESCE(latest_rate.client_group, 1) = COALESCE(job.client_group, 1)
OR latest_rate.client_group IS NULL
)
AND (
COALESCE(latest_rate.client_contact, 1) = COALESCE(job.client_contact, 1)
OR latest_rate.client_contact IS NULL
)
)
WHERE job.id = 4
So... this query fetches a list of Elements for a Job. Each Element is broken down by "Role Hours" - an Element called "Build an HTML email", for example, might have two hours of Designer, and two hours of Developer - and then the calculates a cost based on the Hourly Rate for each Role.
The issue is where I join the latest_rate subquery back to the main query. Here's my rates table:
'CREATE TABLE `rate` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`client_company` int(11) DEFAULT NULL,
`client_group` int(11) DEFAULT NULL,
`client_contact` int(11) DEFAULT NULL,
`role` int(11) DEFAULT NULL,
`date_from` datetime DEFAULT NULL,
`hourly_rate` decimal(18,2) DEFAULT NULL,
`last_edited_by` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
)
There might be several rates for a role: a "global" rate where the role is set but the client_company, client_group and client_contact fields are NULL; a company-specific rate where the role and client_company are set but everything else is NULL, and a client-specific rate where the role, client_company and client_contact is set but the client_group is NULL.
I want to get the one rate for the role that "most closely" matches the client_company, client_group and client_contact set for the Job. So if there's a Rate record with the role and client_company set (and the client_company matches the one for the Job), I want that one. Otherwise, I'll fall back to the one with NULL client_company.
My attempt at joining is clearly wrong:
AND (
COALESCE(latest_rate.client_company, 1) = COALESCE(job.client_company, 1)
OR latest_rate.client_company IS NULL
)
It'll return all the records that either match the client_company, or have a NULL client_company.
How can I get it to just get the one with the most matching fields?
Thanks :)
What you could try is have one query that basically gets all matching rates against one parameter that you absolutely want to match - seems to be role in your case.
Then you can do soemthing like:
MAX(
CASE WHEN <conditions 1> THEN <calculation 1> END,
CASE WHEN <conditions 2> THEN <calculation 2> END,
...
)
This would give you the maximum rate possible, i.e, the rate applicable if you had the most matching fields.
I've been thinking about this one for quite some time now, I need a way to add replies to comments in the database but I'm not sure how to proceed.
This is my currently comment table (doesn't say much but its a start):
CREATE TABLE IF NOT EXISTS `comments` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`comment` text,
`user_id` int(12) DEFAULT NULL,
`topic_id` int(12) NOT NULL,
`ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `topic_id` (`topic_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=27 ;
and here is my current query:
SELECT c.id, c.comment, c.user_id, u.username, u.photo
FROM (comments c)
JOIN users u ON c.user_id = u.id
WHERE c.topic_id = 9
One option would be to create a new table called "comment_replies" but I'm not sure If I'm able to select all the comments and comment replies in one query, and If I add a new column called "reply" I'm not sure how to sort them to get each comment with each reply.
I would love to get some advice on how to deal with this.
Edit:
Following the below answers about adding parent_comment_id result in this kind of array from 1 comment and 2 replies:
array(2) {
[0]=>
object(stdClass)#17 (7) {
["id"]=>
string(2) "26"
["comment"]=>
string(36) "adding a comment from the admin page"
["user_id"]=>
string(2) "16"
["ts"]=>
string(10) "1249869350"
["username"]=>
string(5) "Admin"
["photo"]=>
string(13) "gravatar2.png"
["reply"]=>
string(23) "There is no admin page!"
}
[1]=>
object(stdClass)#18 (7) {
["id"]=>
string(2) "26"
["comment"]=>
string(36) "adding a comment from the admin page"
["user_id"]=>
string(2) "16"
["ts"]=>
string(10) "1249869350"
["username"]=>
string(5) "Admin"
["photo"]=>
string(13) "gravatar2.png"
["reply"]=>
string(13) "Yes there is!"
}
}
How should I process this array to work with it, Is it possible to separate the comment from the replies?
If you want people to be able to reply to the replies (i.e. have a hierarchy of replies such as you would see in, say, an online message forum), then I would add an optional parent_comment_id field to the comments table.
Your table would look like this
`CREATE TABLE IF NOT EXISTS `comments` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`parent_comment_id` int(12) NULL,
`comment` text,
`user_id` int(12) DEFAULT NULL,
`topic_id` int(12) NOT NULL,
`ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `topic_id` (`topic_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=27 ;`
Your query showing all comments and replies would be something like:
SELECT c.id, c.comment, r.comment as reply, c.user_id, u.username, u.photo
FROM (comments c)
JOIN users u ON c.user_id = u.id
LEFT JOIN comments r ON c.id = r.parent_comment_id
WHERE c.topic_id = 9
Note however that with this query your replies would also show up not only in the 'reply' column, but also in the 'comment' column as additional rows each with zero or more replies.
To show the username of the users who replied to a comment, you will need to join twice to the users table (first for the user who posted the original comment, and again for the user(s) who replied). Try this query to show the usernames of the users who replied:
SELECT c.id, c.comment, c.user_id, u.username, u.photo, r.comment as reply, r.user_id as reply_user_id,
u2.username as reply_username, u2.photo as reply_photo
FROM (comment c)
JOIN users u ON c.user_id = u.id
LEFT JOIN comments r ON c.id = r.parent_comment_id
JOIN users u2 ON r.user_id = u2.id
WHERE c.topic_id = 9
Add a parent_comment_id column to your comments table. Make it optional. When you query, you can join all child comments to each parent. As a bit of selective denormalization (slight redundancy) you can make sure topic_id is set on the child comments as well, letting you pull them all a bit easier (assuming you're going to display all child comments in the main comment thread and not via smaller ajax requests).
Build the presentation however you need, toss the results into memcached (or a flat file, or memory... however you're caching) and you're set.
I decided to add the parent_id column in the database and instead of left joining the replies I just selected all the comments at once to later on sort the comments and replies with server-side code, heres the query:
SELECT c.*, u.username, u.photo
FROM (comments c)
JOIN users u ON c.user_id = u.id
WHERE c.topic_id = 9
ORDER BY c.id ASC
Now I pass the query result to the below function so that every reply will be added as an array inside the comment array, so basically it returns a multidimensional array.
function sort_comments($ar)
{
$comments = array();
foreach($ar as $item)
{
if(is_null($item['parent_id'])) $comments[] = $item;
else
{
$parent_array = array_search_key($item['parent_id'],$comments,'id');
if($parent_array !== false) $comments[$parent_array]['replies'][] = $item;
}
}
return $comments;
}
this seems all good but what about if the table contained over a million rows? and some comments could be hundreds of thousands of rows apart. how will these queries perform?
A comment reply is a comment with a parent comment_id. Try adding comment_id as a field to your comments table. What you will get is a tree-like structure.
If you wish to retrieve an entire tree of comments, your best bet is to use a nested set (https://wiki.htc.lan/Hierarchy_model). But that's a more complicated solution.
Here's some more info from MySQL: http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/
Looks like you are working with WordPress, adding a parent_comment_id would have been an ideal solution, but not in this case.
Firstly, I don't think modifying the WordPress basic tables is a good idea. Secondly, you'll end-up with a complex code that will break with wordpress updates.
Best is use a plugin like Intense Comments
Still if you want to create your own solution, I would say create another table for comment replies.
a) Your new table would look like this
`CREATE TABLE IF NOT EXISTS `comment_replies` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`parent_comment_id` int(12) NULL,
`comment` text,
`user_id` int(12) DEFAULT NULL,
`topic_id` int(12) NOT NULL,
`ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `topic_id` (`topic_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
b) You'd fetch them this way
$comment_ids = array_map( 'intval', array_keys( $comments ) );
sort( $comment_ids );
$comments_id_list = join( ',', $comment_ids );
$query = "SELECT c.id, c.comment, c.user_id, u.username, u.photo
FROM (comment_replies c)
JOIN users u ON c.user_id = u.id
WHERE c.parent_comment_id IN ( $comments_id_list )"
$replies = $wpdb->get_results($query);
$replies_by_parent = array();
foreach ( array_keys( $replies ) as $i ) {
$replies_by_parent [$replies[$i]->id] = array();
}
foreach ( array_keys( $replies ) as $i ) {
$replies_by_parent [$replies[$i]->id][] =& $replies[$i];
}
c) Now within your comments loop you can get the replies like this
foreach ( $replies_by_parent [$parent_id] as $reply ) {
echo $reply->comment;
}
function sort_comments($ar)
{
$comments = [];
$i=0;
foreach($ar as $co){
if(!empty($co['comment_replyof'])) {
$comments[$co['comment_replyof']]['replies'] = $co;
}else{
foreach($co as $c => $o) $comments[$co['comment_id']][$c] = $o;
}
$i++;
}
return $comments;
}
SELECT C.*, U.id,U.fname, U.lname FROM (comment C) JOIN users U ON `enter code here`C.comment_user = U.id where C.comment_content='10'