MySQL join on substring is slow - mysql

I have a query where I do a join on a substring, the problem is that this is really slow to complete. Is there a more effecient way to write this?
SELECT *, SUM(s.pris*s.antall) AS total, SUM(s.antall) AS antall
FROM ecs_statistikk AS s
JOIN butikk_ordre AS bo ON ordreId=bo.ecs_ordre_id AND butikkNr=bo.site_id
JOIN ecs_supplier AS l ON SUBSTRING( s.artikkelId, 1,2 )=l.lev_id
WHERE s.salgsDato>='2016-6-01' AND s.salgsDato<='2016-09-30'
GROUP BY l.lev_id ORDER BY total DESC

First, I would check indexes. For this query:
SELECT *, SUM(s.pris*s.antall) AS total, SUM(s.antall) AS antall
FROM ecs_statistikk s JOIN
butikk_ordre bo
ON s.ordreId = bo.ecs_ordre_id AND
s.butikkNr = bo.site_id JOIN
ecs_supplier l
ON SUBSTRING(s.artikkelId, 1, 2 ) = l.lev_id
WHERE s.salgsDato >= '2016-06-01' AND s.salgsDato <= '2016-09-30'
GROUP BY l.lev_id
ORDER BY total DESC ;
You want indexes on ecs_statistikk(salgsDato, ordreId, butikkNr, artikkelId), butikk_ordre(ecs_ordre_id, site_id), and ecs_supplier(lev_id)`.
Next, I would question whether you need the last JOIN at all. Does this do what you want?
SELECT LEFT(s.artikkelId, 2) as lev_id, *,
SUM(s.pris*s.antall) AS total, SUM(s.antall) AS antall
FROM ecs_statistikk s JOIN
butikk_ordre bo
ON s.ordreId = bo.ecs_ordre_id AND
s.butikkNr = bo.site_id
WHERE s.salgsDato >= '2016-06-01' AND s.salgsDato <= '2016-09-30'
GROUP BY LEFT(s.artikkelId, 2)
ORDER BY total DESC ;

Related

a sum() function with aritmatich 4 table

this is sample data in table pengiriman_supply.
and this is for data_barang
this is for data_supplier and table masuk.
if I'm not using 3 tables the sum is no a problem but if I'm using 4 tables and using subtraction with (sum(table1.a)-ifnull(table2.b)). here is the result with just sum
and this is the picture with subtraction
the code is like this
SELECT DISTINCT
row_number() over(
order by pengiriman_supply.po_nomor desc) as no,
pengiriman_supply.po_nomor as PO,
data_supplier.nama_supplier,
data_barang.nama_barang,
((sum( pengiriman_supply.jumlah ))- (sum( COALESCE ( masuk.terima, 0 )) over ( PARTITION BY masuk.refrence ))) as total
FROM
pengiriman_supply
LEFT JOIN masuk ON pengiriman_supply.po_nomor = masuk.refrence
INNER JOIN data_supplier ON data_supplier.id_supplier = pengiriman_supply.idsupplier
INNER JOIN data_barang ON data_barang.idbarang = pengiriman_supply.idbarang
WHERE
pengiriman_supply.tanggal between date_sub(curdate(), interval 60 day) and curdate()
GROUP BY
pengiriman_supply.po_nomor,masuk.po_nomor,data_supplier.nama_supplier
ORDER BY
GROUP_CONCAT(DISTINCT pengiriman_supply.po_nomor) DESC
this the code that SQL statement that I can find. but the group by not make the SQL statement just pengiriman_supply.po_nomor. can I make the group by just the pengiriman_supply.po_nomor .
can the number 31194 make in one group?
it seems you need to include ifnull(masuk.terima,0) inside sum()
SELECT
pengiriman_supply.po_nomor AS po,
data_supplier.nama_supplier,
data_barang.nama_barang,
Sum((pengiriman_supply.jumlah)-ifnull(masuk.terima,0)) as total
FROM
pengiriman_supply
INNER JOIN data_barang ON pengiriman_supply.idbarang = data_barang.idbarang
INNER JOIN data_supplier ON pengiriman_supply.idsupplier = data_supplier.id_supplier
LEFT JOIN masuk ON masuk.refrence = pengiriman_supply.po_nomor
GROUP BY
pengiriman_supply.po_nomor
ORDER BY
po DESC

Optimizing MySQL query with multiple joins and Sub query

I am using the following query to get data from 10 table, It is working fine but quite slow, Is there any way to Optimizing the query.
Query: SELECT emi.emi_due_date,users.usr_mobile,users.usr_id,concat_ws(" ",users.usr_fname,users.usr_mname,users.usr_lname) as borrower,users.usr_status,users.usr_curnt_city, users.usr_email,emi.loan_id,emi.emi_show_date,sum(emi.emi_amount)-sum(ifnull(emi.settled_amount,0)) as due_amount,cb.cb_type,blr.bloan_collection_executive_id,blr.pp_allow,blr.bloan_legal_team_id,blr.bloan_legal_team_status,concat_ws(" ",cp.cp_fname,cp.cp_lname) as cp_name,cp.cp_mobile,cp.cp_firm_name,cp.cp_type,bg.guarantor_name,bg.guarantor_contact,pl.ecs_date,pd.p2p_date,
(SELECT instrument FROM borrower_payment_master WHERE loan_id = emi.loan_id order by id desc limit 0,1) as last_pmode,
(SELECT IFNULL(DATE_FORMAT(emi_show_date - INTERVAL 1 MONTH,"%m-%Y"),"") FROM emi AS e WHERE e.loan_id=emi.loan_id and e.emi_status < 2 ORDER by e.emi_show_date ASC limit 1) as paid_till,
(select payment_date from borrower_payment_master as bp where bp.loan_id=emi.loan_id order by bp.id desc limit 1) as last_emi_paid FROM emi AS emi
INNER JOIN borrower_loan_reg_requests AS blr ON emi.loan_id=blr.bloan_id
INNER JOIN users AS users ON users.usr_id=blr.bloan_user_id
INNER JOIN borrower_loan_disbursed_funds AS blf ON blf.df_bloan_id=emi.loan_id
LEFT JOIN channel_partners AS cp ON cp.cp_id=users.usr_cp_referral_id
LEFT JOIN borrower_posted_loans AS pl ON pl.pl_bloan_id=emi.loan_id
LEFT JOIN collection_bucket AS cb ON cb.cb_loan_id=emi.loan_id AND cb.cb_status = 1
LEFT JOIN borrower_guarantors AS bg ON bg.guarantor_borrower_id=users.usr_id
LEFT JOIN p2p_dates AS pd ON pd.p2p_loan_id=emi.loan_id AND pd.p2p_status = 1
WHERE emi.emi_status<2 AND emi.emi_amount != 0
AND (SELECT count(*) FROM borrower_payment_master as pm WHERE pm.loan_id = emi.loan_id
AND MONTH(pm.payment_date) = "'.date('m').'" AND YEAR(pm.payment_date) = "'.date('Y').'") = 0
AND (select s.settlement_date as sdate from settlement as s WHERE emi.loan_id=s.loan_id limit 1) !=""
group by emi.loan_id order by emi.loan_id desc

How to optimize the product fetching query

I have used below query for product listing. Query is working fine but it takes approximately 0.4534 seconds. How can I optimize the same query.
SELECT SQL_CALC_FOUND_ROWS DISTINCT tp.prod_id, tp.prod_name, tp.prod_shop, tp.prod_retail_price, tp.prod_sale_price, tp.prod_initial_price, tp.prod_stock, ts.shop_id, ts.shop_name, ts.shop_logo, ts.shop_description, ts.shop_title, tu.user_profile_image, ( SELECT pdiscount_price FROM tbl_product_discounts tpd WHERE tpd.pdiscount_product_id = tp.prod_id AND tpd.pdiscount_qty = '1' AND( ( tpd.pdiscount_start_date = '0000-00-00' OR tpd.pdiscount_start_date < NOW()) AND( tpd.pdiscount_end_date = '0000-00-00' OR tpd.pdiscount_end_date > NOW()) ) ORDER BY tpd.pdiscount_priority ASC, tpd.pdiscount_price ASC LIMIT 1 ) AS discount FROM tbl_products tp LEFT JOIN tbl_shops ts ON tp.prod_shop = ts.shop_id AND ts.shop_is_deleted = 0 INNER JOIN tbl_users tu ON ts.shop_user_id = tu.user_id WHERE tp.prod_is_deleted = '0' LIMIT 0, 20
Without checking you table or Requirement :
Try to use group by instead of DISTINCT
Do not use sub query if possible .
Try To use indexing in you table .
This will help you to optimize you query.

Optimize Query with JOINS and Subqueries

I want to speed up one of my slower queries.
The problem is that I can't access the outer colum value within a subquery.
What I have:
SELECT r.id AS room_id, r.room_name, coalesce(d.score,0) AS total_messages, d.latest
FROM cf_rooms_time_frames tf
INNER JOIN cf_rooms r on r.id = tf.room_id
INNER JOIN(
SELECT cf.room_id, count(*) as score, max(cf.id) as latest
FROM cf_rooms_messages cf
WHERE EXISTS(
SELECT NULL FROM cf_rooms_time_frames tf
WHERE tf.start <= cf.id AND ( tf.end IS NULL OR tf.end >= cf.id )
AND tf.room_id = cf.room_id AND tf.uid = 8
)
GROUP BY cf.room_id
ORDER BY latest
DESC ) d on d.room_id = r.id
WHERE tf.uid = 8
ORDER BY coalesce(latest, score) DESC LIMIT 0, 20
What I want:
SELECT r.id AS room_id, r.room_name, coalesce(d.score,0) AS total_messages, d.latest
FROM cf_rooms_time_frames tf
INNER JOIN cf_rooms r on r.id = tf.room_id
INNER JOIN(
SELECT cf.room_id, count(*) as score, max(cf.id) as latest
FROM cf_rooms_messages cf
/* line added here */
WHERE cf.room_id = tf.room_id
/* */
AND EXISTS(
SELECT NULL FROM cf_rooms_time_frames tf
WHERE tf.start <= cf.id AND ( tf.end IS NULL OR tf.end >= cf.id )
AND tf.room_id = cf.room_id AND tf.uid = 8
)
GROUP BY cf.room_id
ORDER BY latest
DESC ) d on d.room_id = r.id
WHERE tf.uid = 8
ORDER BY coalesce(latest, score) DESC LIMIT 0, 20
I think the markup explains what the query does.
It searches for "chatrooms" for a given user and orders them by the last message, gets the number of total message which ids are in a given range ( timeframes ), and the last message id.
I don't know why, but the first query returns all rows within the chatmessage table ( cf ) if I can trust EXPLAIN. It delivers the correct results but is kind of slow on a huge table.
I tested the second one with a "hardcoded" room_id and this one was very fast and doesn't "touched" the whole table.

SQL order by the result of one operation

I need to order the result of this query by the result of (LIKES (puntuacion=1) - DISLIKE (puntuacion=0).
This is the old query where I order by the sum of likes (puntuacion=1).
"SELECT entradas.* , SUM(puntuacion) AS total_likes
FROM entradas
LEFT JOIN valoraciones ON valoraciones.entradas_id = entradas.id
and valoraciones.puntuacion=1
WHERE fecha>=:fecha1 AND aceptada=1
GROUP BY entradas.id
ORDER BY `total_likes` DESC
limit 5";
Tried this, but total_likes / total_dislikes are temporal variables and can't operation with them.
SELECT entradas.* , SUM(puntuacion=1) AS total_likes, SUM(puntuacion=0) AS total_dislikes, total_likes-total_dislikes AS TOTAL
FROM entradas
LEFT JOIN valoraciones ON valoraciones.entradas_id = entradas.id
WHERE aceptada=1
GROUP BY entradas.id
ORDER BY `total_likes` DESC
limit 5
SELECT entradas.* , (SUM(v1.puntuacion) - SUM(v0.puntuacion)) AS total_likes
FROM entradas
LEFT JOIN valoraciones v1 ON v1.entradas_id = entradas.id and v1.puntuacion=1
LEFT JOIN valoraciones v0 ON v0.entradas_id = entradas.id and v0.puntuacion=0
WHERE fecha >= :fecha1 AND aceptada=1
GROUP BY entradas.id
ORDER BY `total_likes` DESC
limit 5
[EDIT]
Sorry mate, the query abover is not quite alright. I think the right answer for what you are looking for is this one below:
SELECT entradas.* , SUM(IF(v.puntuacion = 1, 1, -1)) AS total_likes
FROM entradas
LEFT JOIN valoraciones v ON v.entradas_id = entradas.id
WHERE fecha >= :fecha1 AND aceptada=1
GROUP BY entradas.id
ORDER BY `total_likes` DESC
LIMIT 5