Speed up an SQL query which uses a FullText Join - mysql

I have a situation where I need to JOIN and inner query from a products database, grouping items by name and then using the name column to perform the join, thus:
SELECT p.*, p3.variants, sl.id, l.name as locationName FROM products p
LEFT JOIN (SELECT item, count(DISTINCT id) as variants FROM products GROUP BY item) p3
ON p3.item = p.item
LEFT JOIN stockLevels sl ON sl.id=p.id
LEFT JOIN locations l ON l.id=sl.stockLocation AND l.showStock='1'
WHERE p.id='10459' GROUP BY p.id
Having messed around with the order of JOIN and the query itself, I have determined that having this inner table being created first and performing the first required join makes a difference to the speed of the query, but the inner join (SELECT item, count(DISTINCT id) as variants FROM products GROUP BY item) is giving (at present) 7034 rows, which is needless as only about 5 are required.
If I modify the inner join to:
(SELECT item, count(DISTINCT id) as variants
FROM products *WHERE item LIKE '%SOME VALUE%'* GROUP BY item)
This obviously reduces the number of rows returned by that inner join, and it is roughly twice as fast.
But, I can't do this as I don't have a known value to use for the inner where clause.
Is there any way I can bring the results from the outer table into the inner and produce an inner query that references the outer, ie:
SELECT p.*, p3.variants, sl.id, l.name as locationName FROM products p
LEFT JOIN (SELECT item, count(DISTINCT id) as variants
FROM products *WHERE item LIKE p.item * GROUP BY item) p3 << New Where Clause
ON p3.item = p.item
LEFT JOIN stockLevels sl ON sl.id=p.id
LEFT JOIN locations l ON l.id=sl.stockLocation AND l.showStock='1'
WHERE p.id='10459' GROUP BY p.id
I know that what I have written above won't work, and understand why, but is there a way I could achieve something on these lines?
With the number of records in my database at the moment, this actually isn't an issue, the data is returned very quickly, but I can imagine that in time, when the data grows, it could be.
Note, I did look at making a parentId integer column, instead of grouping by the FullText item but there are lots of other issues within my application, and it didn't seem to speed the query up anyway, for the same reasons as listed above, the inner still had to return all rows
Also Note, this is MySQL
If it helps at all here's the output from EXPLAIN

You can put a correlated subquery in the SELECT clause:
SELECT p.*, sl.id, l.name as locationName,
(SELECT COUNT(distinct id)
FROM products p3
WHERE p3.item = p.item) as variants
FROM products p
LEFT JOIN stockLevels sl ON sl.id=p.id
LEFT JOIN locations l ON l.id=sl.stockLocation AND l.showStock='1'
WHERE p.id='10459' GROUP BY p.id

If only 5 are required, just do it the classic way:
SELECT
p.*,
p3.variants,
sl.id,
l.name as locationName,
(SELECT count(DISTINCT id)
FROM products
WHERE item LIKE p.item GROUP BY item) total
FROM products p
LEFT JOIN stockLevels sl
ON sl.id=p.id
LEFT JOIN locations l
ON l.id=sl.stockLocation AND l.showStock='1'
WHERE p.id='10459' GROUP BY p.id
Here you can access the parent table.

Related

How to avoid Left join table show duplicate row?

I have some problem with the query issue when trying to sum up the quantity.
Table
This cart item table stored id_cart and id product
This order table stored id_cart and other id may be included such as supplier. This table is used to track order record and send notification to supplier.
Wrong result. Expected output = 1, 1, 1
SELECT id, id_product, SUM(qty)
from cart_item
left join Orderp using(id_cart)
group by id_product
http://sqlfiddle.com/#!9/07bf57/1
The issue caused by duplicate id_cart in order table as well. How can i handle this? Any solution to make it works? Thanks.
There is something wrong in your data, or in your data model
INSERT INTO OrderP(`id_order`,`id_cart`)VALUES(1, 1);
INSERT INTO OrderP(`id_order`,`id_cart`)VALUES(2, 1);
There are 2 rows for id_cart = 1, so the "natural join" will double every row when joining cart_item to orderp.
Using an inner join to a different column in orderp works better because now there is only one row in orederp for each cart_item.
SELECT id_product, sum(qty)
from cart_item ci
left join Orderp o on ci.id_cart = o.id_order
GROUP BY id_product
http://sqlfiddle.com/#!9/07bf57/13
Try the following query
SELECT
i.id_product,
p.name productname,
b.id_branch,
b.branchname,
SUM(i.qty)
from cart_item i
left join (SELECT DISTINCT id_cart,id_branch FROM Orderp) o on o.id_cart=i.id_cart
left join product p on i.id_product=p.id_product
left join catalog c on c.id_product=p.id_product and c.id_branch=o.id_branch
left join branch b on b.id_branch=o.id_branch
group by
i.id_product,
p.name,
b.id_branch,
b.branchname
The main problem in Orderp table because it containts two different orders for one cart (DISTINCT id_cart,id_branch helps here). And you need to use the second condition by id_branch for catalog (and c.id_branch=o.id_branch).
SQL Fiddle - http://sqlfiddle.com/#!9/f32d5f/16
And I think you can use everywhere INNER JOIN instead LEFT JOIN
SELECT
i.id_product,
p.name productname,
b.id_branch,
b.branchname,
SUM(i.qty)
from cart_item i
join (SELECT DISTINCT id_cart,id_branch FROM Orderp) o on o.id_cart=i.id_cart
join product p on i.id_product=p.id_product
join catalog c on c.id_product=p.id_product and c.id_branch=o.id_branch
join branch b on b.id_branch=o.id_branch
group by
i.id_product,
p.name,
b.id_branch,
b.branchname

MySQL JOIN tables with COUNT values

I have the following tables in my database.I only listed the important columns which can be used for joining.
I need to get the following output
Currently I'm using two seperate queries for each COUNT value
For assigned licenses
select
products.id,products.name,COUNT(assigned_licenses.id)
from
deployment_users
inner join
assigned_licenses
on
deployment_users.id = assigned_licenses.deployment_user_id
inner join
products
on
assigned_licenses.id = products.id
and
deployment_users.customer_id = 10
group by
assigned_licenses.id
;
For total licenses
select
products.id,products.name,COUNT(total_licenses.id)
from
customers
inner join
total_licenses
on
customers.iccode = licenses.iccode
inner join
products
on
total_licenses.id = products.id
and
customers.id = 10
group by
total_licenses.id
;
Since there are more than a 1,000 products that need to be listed,I want to combine them into a single query.How can I do that?
Your specification leaves some room for interpretation (e.g. can a user have assigned licenses without total licenses? if yes my query will fail.) but I would go with this.
SELECT
products.id,
products.name,
Count(Distinct total_licenses.id) As CountTotalLicenses,
Count(Distinct assigned_liceses.deployment_users_id) As CountAssignedLicenses
FROM products
LEFT JOIN total_licenses ON total_licenses.products_id = products.id
LEFT JOIN customers ON customers.iccode = total_licenses.customers_iccode
LEFT JOIN assigned_licenses ON assigned_liceses.total_licenses_id = total_licenses.id
WHERE
customers.id = 10
GROUP BY
products.id,
products.name
For the future it would be awesome if you could paste code as code and not as an image. People cannot simple copy paste snippets of your code and have to type everything again...
Try joining Both of your query
SELECT * FROM (
(First Query) as assigned_licn
INNER JOIN
(Second Query) as total_licn
USING (id)
);

Need to join MySql query to 3 tables

SELECT products.acctnum,products.subacctnum,NOW(),
items.amount,items.id,items.invoice_id,items.product_id,
items.po_id, invoices.customer_id, purchaseorders.vendor_id FROM items
INNER JOIN (products, invoices, purchaseorders)
ON (items.product_id=products.product_id AND items.invoice_id=invoices.id
AND items.po_id=purchaseorders.id)
This returns nothing... however..
SELECT products.acctnum,products.subacctnum,NOW(),
items.amount,items.id,items.invoice_id,items.product_id,
items.po_id, purchaseorders.vendor_id FROM items
INNER JOIN (products, purchaseorders)
ON (products.product_id=items.product_id AND purchaseorders.id=items.po_id)
Works...
SELECT products.acctnum,products.subacctnum,NOW(),
items.amount,items.id,items.invoice_id,items.product_id,
items.po_id, invoices.customer_id FROM items
INNER JOIN (products, invoices)
ON (products.product_id=items.product_id AND invoices.id=items.invoice_id)
Works...
Works for the rows I need in the result but when I join the 3rd table it doesn't work. LEFT JOIN displayed all the columns I needed but some rows were NULL.
I imagine the join clause you want looks more like this:
FROM items LEFT JOIN
invoices
ON invoices.id = items.invoice_id LEFT JOIN
purchaseorders
ON purchaseorders.id = items.po_id LEFT JOIN
products
ON products.product_id = items.product_id
I'm not sure which fields are not valid when you select them, but you can probably fix such issues by using coalesce() with appropriate fields from invoices and purchaseorders.

mySQL, need help getting many sub-queries into 1 SELECT query (displays 13 columns)

this is my first post. I know that this is horribly inefficient and repetitive code that won't actually work, what I need to do is combine all these outputs into 1 select statement. I am obviously fairly new at this, but I've been at it all day and I just can't get started in the right direction, each snippet works on its own...please help!
Essentially I'm working with a DB with many tables, and to get the right data for each column, I often have to account for 3 tables with joins.
Thanks for any insight or help!
SELECT
Product.ProductID,
(
SELECT Abbreviation AS Country
FROM Product
LEFT JOIN ProductCountry
ON Product.ProductID = ProductCountry.ProductID
LEFT JOIN Location
ON ProductCountry.LocationID = Location.LocationID
GROUP BY Product.ProductID
),
(
SELECT r.ResourceName AS Manufacturer, rr.ResourceName AS Brand
FROM Product p
LEFT JOIN Resource r
ON p.ManufactureCode = r.ResourceID
INNER JOIN Resource rr
ON p.BrandCode = rr.ResourceID
),
Product.Name,
Product.UPC,
Product.Size,
(
SELECT Unit.abbreviation AS Measure
FROM Product
LEFT JOIN Unit
ON Product.Unit = Unit.UnitID
),
(
SELECT Category.ParentID AS Category, Category.Description AS Sub_Category
FROM Product
LEFT JOIN ProductCategory
ON Product.ProductID = ProductCategory.ProductID
LEFT JOIN Category
ON ProductCategory.CategoryID = Category.CategoryID
),
(
SELECT i.Description AS INGREDIENTS, i.MayContain AS Allergen_Statement
FROM Product
LEFT JOIN Ingredient i
ON Product.ProductID = i.IngredientID
),
(
SELECT GROUP_CONCAT( Special.Description SEPARATOR ', ' ) AS Free_From
FROM Product
LEFT JOIN ProductSpecial
ON Product.ProductID = ProductSpecial.ProductID
LEFT JOIN Special
ON ProductSpecial.SpecialID = Special.SpecialID
GROUP BY Product.ProductID
)
FROM Product, ProductStatus
WHERE ProductStatus.ProductStatusID = 1
First, some notes and assumptions...
I'm assuming the Country column is in the Location table, otherwise why would you bother joining it.
If you have trouble with this part, change the second join to a LEFT JOIN. I've had occasional trouble doing left join a to b followed by inner join b to c. I've found it's easier to keep the LEFT JOIN going, so my example left-joins both tables:
LEFT JOIN Resource r
ON p.ManufactureCode = r.ResourceID
INNER JOIN Resource rr
ON p.BrandCode = rr.ResourceID
You're joining to Resource in two different ways. That's completely OK with MySQL (and all the mainstream databases). You just have to alias one or both of the joins. I've aliased one of them:
LEFT JOIN Resource ON Product.ManufactureCode ...
LEFT JOIN Resource BrandResource ON Product.BrandCode...
I don't know from your example how ProductStatus is joined. You'll have to supply that.
Start small using the example below. Join in the Country, then when you've got that working, join in Manufacturer, then Brand, then Measure, etc. The query isn't doing a lot of advanced stuff; it's complicated mostly due to the sheer number of tables. Tackle them one at a time and you'll win :)
Finally, as #Bohemian noted in the comments, the GROUP BY can't be brought up to the top. Actually, it probably can, but it will complicate things beyond belief. I've left that as a subquery.
Here's the final result. Note that it's not tested because it's huge and I don't have table structures or sample data. But mostly because it's huge :) At any rate, this is meant as an example only.
SELECT
Product.ProductID,
Location.Country,
Resource.ResourceName AS Manufacturer,
BrandResource.ResourceName AS Brand,
Product.Name,
Product.UPC,
Product.Size,
Unit.Abbreviation AS Measure,
Category.ParentID AS Category,
Category.Description AS Sub_Category,
Ingredient.Description AS Ingredients,
Ingredient.MayContain AS Allergen_Statement,
(SELECT GROUP_CONCAT( Special.Description SEPARATOR ', ' ) AS Free_From
FROM Product
LEFT JOIN ProductSpecial
ON Product.ProductID = ProductSpecial.ProductID
LEFT JOIN Special
ON ProductSpecial.SpecialID = Special.SpecialID
GROUP BY Product.ProductID
)
FROM Product
INNER JOIN ProductStatus ON ... however it's joined
LEFT JOIN ProductCountry ON Product.ProductID = ProductCountry.ProductID
LEFT JOIN Location ON ProductCountry.LocationID = Location.LocationID
LEFT JOIN Resource ON Product.ManufactureCode = Resource.ResourceID
LEFT JOIN Resource BrandResource ON Product.BrandCode = BrandResource.ResourceID
LEFT JOIN Unit ON Product.Unit = Unit.UnitID
LEFT JOIN ProductCategory ON Product.ProductID = ProductCategory.ProductID
LEFT JOIN Category ON ProductCategory.CategoryID = Category.CategoryID
LEFT JOIN Ingredient ON Product.ProductID = i.IngredientID
WHERE ProductStatus.ProductStatusID = 1

MySql query to get count of days spent in each country for each purpose? (Get count of all record in second table present in first table)

I have three tables tl_log, tl_geo_countries,tl_purpose. I am trying to get the count of number of days spent in each country in table 'tl_log' for each purpose in table 'tl_purpose'.
I tried below mysql query
SELECT t.country_id AS countryID,t.reason_id AS reasonID,count(t.reason_id) AS
days,c.name AS country, p.purpose AS purpose
FROM `tl_log` AS t
LEFT JOIN tl_geo_countries AS c ON t.country_id=c.id
LEFT JOIN tl_purpose AS p ON t.reason_id=p.id
GROUP BY t.reason_id,t.country_id ORDER BY days DESC
But landed up with.
I am not able to get the count for purpose for each country in 'tl_log' that is not present in table 'tl_log'. Any help is greatly appreciated. Also, Please let me know if the question is difficult to understand.
Expected Output:
Below is the structure of these three tables
tl_log
tl_geo_countries
tl_purpose
If you want all possible combination of countries and purposes, even those that do not appear on the log table (these will be shown with a count of 0), you can do first a cartesian product of the two tables (a CROSS join) and then LEFT join to the log table:
SELECT
c.id AS countryID,
p.id AS reasonID,
COUNT(t.reason_id) AS days,
c.name AS country,
p.purpose AS purpose
FROM
tl_geo_countries AS c
CROSS JOIN
tl_purpose AS p
LEFT JOIN
tl_log AS t
ON t.country_id = c.id
AND t.reason_id = p.id
GROUP BY
p.id,
c.id
ORDER BY
days DESC ;
If you want the records for only the countries that are present in the log table (but still all possible reason/purposes), a slight modification is needed:
SELECT
c.id AS countryID,
p.id AS reasonID,
COUNT(t.reason_id) AS days,
c.name AS country,
p.purpose AS purpose
FROM
( SELECT DISTINCT
country_id
FROM
tl_log
) AS dc
JOIN
tl_geo_countries AS c
ON c.id = dc.country_id
CROSS JOIN
tl_purpose AS p
LEFT JOIN
tl_log AS t
ON t.country_id = c.id
AND t.reason_id = p.id
GROUP BY
p.id,
c.id
ORDER BY
days DESC ;
LEFT JOIN should be replaced by RIGHT JOIN