i have a large dataset and i need help to make a few queries faster.
So far i am using subquery to retrieve the product ids and get the different persons of this products (something like filtering)
Here is my query
SELECT assoc.*, count(assoc.product_id) as count FROM ws_products_persons_assoc as assoc
WHERE
assoc.product_id
IN
( SELECT c.id
FROM ws_products as c
WHERE c.status = '1'
AND ( ( product_name LIKE '%960%' ) OR ( ( code LIKE '%960%' OR isbn13 LIKE '%960%' OR parent_codes LIKE '%960%') ) OR ( publisher_name LIKE '%960%' ) OR ( author_name LIKE '%960%' ) )
ORDER BY c.year desc,c.product_name ASC
)
GROUP BY assoc.person_id
ORDER BY count DESC LIMIT 0,30
Query Time =1.7937450408936 seconds
The subquery search in a few fields for the given keyword
The subquery returns 183473 rows and takes 1.7 secs to run.
Any ideas for making subquery faster is appriciated
Thanks
Why not an inner join? (assuming product_id is unique)
SELECT a.*, count(a.product_id) as count
FROM ws_products_persons_assoc a INNER JOIN ws_products p
ON p.id = a.product_id
WHERE
p.status = 1 AND
( (p.product_name LIKE '%960%' ) OR
(p.code LIKE '%960%') OR
(p.isbn13 LIKE '%960%' OR
(p.parent_codes LIKE '%960%') OR
(p.publisher_name LIKE '%960%') OR
(p.author_name LIKE '%960%' )
)
GROUP BY a.person_id
ORDER BY count DESC
LIMIT 0, 30;
I would start by switching to EXISTS:
SELECT a.person_id, count(a.product_id) as count
FROM ws_products_persons_assoc a
WHERE EXISTS (SELECT 1
FROM ws_products p
WHERE p.id = a.product_id AND
p.status = 1 AND
( (p.product_name LIKE '%960%' ) OR
(p.code LIKE '%960%') OR
(p.isbn13 LIKE '%960%' OR
(p.parent_codes LIKE '%960%') OR
(p.publisher_name LIKE '%960%') OR
(p.author_name LIKE '%960%' )
)
)
GROUP BY a.person_id
ORDER BY count DESC
LIMIT 0, 30;
For this version, you want an index on ws_products(id, status).
The select columns should match the group by columns.
Related
I have a table of keywords (ID,ean,keyword) and another table with product details. I want the search to return EANs where all keywords match at least once, however the closest I have got is the following, but this returns matches that have the first term in them 3 times for example.
To give an example, let's say I have a product called 'Generic headphones - iPhone, iPad, iPod' and I searched 'gen%' 'hea%' 'ip%' it would come back as a match, but it would also match 'Apple headphones - iPhone, iPad, iPod' due to the 3 ip words, which is not desired.
SQL Fiddle
I want EAN 1 to match only, so matches need to be at least 1 for each term.
Any help would be much appreciated.
SELECT Count(keywords.ean) AS cc,
products.*
FROM keywords
INNER JOIN products
ON products.ean = keywords.ean
WHERE (
keyword LIKE 'gen%'
|| keyword like 'ip%'
|| keyword LIKE 'hea%')
GROUP BY (keywords.ean)
HAVING cc>=3
ORDER BY `products`.`ean` ASC
UPDATE: This gets the desired results, but there must be more efficient ways to do this.
SELECT products.*
FROM products
INNER JOIN (SELECT ean, count(*) as tc1
FROM keywords
WHERE ( keyword like 'gen%' )
GROUP BY ean
HAVING tc1 > 0 ) as t1 ON t1.ean = products.ean
INNER JOIN (SELECT ean, count(*) as tc2
FROM keywords
WHERE ( keyword like 'ip%' )
GROUP BY ean
HAVING tc2 > 0 ) as t2 ON t2.ean = products.ean
INNER JOIN (SELECT ean, count(*) as tc3
FROM keywords
WHERE ( keyword like 'hea%' )
GROUP BY ean
HAVING tc3 > 0 ) as t3 ON t3.ean = products.ean
ORDER BY products.ean
Perhaps you're after something more like this...
SELECT p.ean
, p.description
FROM products p
JOIN keywords k
ON k.ean = p.ean
WHERE k.keyword LIKE 'iP%'
OR k.keyword LIKE 'hea%'
OR k.keyword LIKE 'gen%'
GROUP
BY p.ean
HAVING COUNT(DISTINCT CASE WHEN k.keyword LIKE 'iP%' THEN 'iP'
WHEN k.keyword LIKE 'hea%' THEN 'hea'
WHEN k.keyword LIKE 'gen%' THEN 'gen'
ELSE keyword END) = 3;
http://sqlfiddle.com/#!9/270f9/25
This is how I'd do it in PostgreSQL. MySQL may have a slightly different syntax.
SELECT kc.cc AS cc,
products.*
FROM products
INNER JOIN ( SELECT ean, count(*) AS cc
FROM keywords
WHERE ( keyword like 'ip%'
OR keyword like 'ai%'
OR keyword like 'bei%' )
GROUP BY ean
HAVING count(*) >= 3 ) AS kc
ON kc.ean = products.ean
ORDER BY Products.ean;
I want to add data from table b in table a but unfortunately full outer join do not work in mysql . I have also tried union but it is throwing errors because my statement has group by and order by keyword
SELECT COUNT( ReviewedBy ) AS TotalReviews, OrganizationId, SUM( Rating ) AS TotalStars, COUNT( Rating ) AS TotalRatings, (
SUM( Rating ) / COUNT( Rating )
) AS AverageRating
FROM `tbl_reviews`
WHERE ReviewType = 'shopper'
AND ReviewFor = 'org'
AND OrganizationId
IN (
SELECT OrganizationId
FROM tbl_organizations
WHERE CategoryID =79
)
GROUP BY OrganizationId
ORDER BY AverageRating DESC
This is what i'm getting from the above statement
I want to get organizationId 21 data in the result but i'm not getting result because it's not present in 'tbl_review' table
click here to see the table b
How can i get Desired result ?
You don't need a FULL, but a LEFT join:
SELECT COUNT( ReviewedBy ) AS TotalReviews, o.OrganizationId,
SUM( Rating ) AS TotalStars, COUNT( Rating ) AS TotalRatings,
(SUM( Rating ) / COUNT( Rating )) AS AverageRating
FROM tbl_organizations AS o
LEFT JOIN `tbl_reviews` AS r
ON o.OrganizationId = r.OrganizationId
AND ReviewType = 'shopper' -- conditions on inner table
AND ReviewFor = 'org' -- must be moved to ON
WHERE CategoryID =79
GROUP BY o.OrganizationId
ORDER BY AverageRating DESC
Why don't you use AVG instead of SUM/COUNT?
Have you tried:
from organization
left outer join tbl_reviews
on organization.ID = tbl_reviews.organization is
for your where clause? I don't think you need a full outer join in this case... A left outer join should do
I have this huge query that filters results out of a series of keywords.
select distinct textures.id from textures
WHERE ((textures.id in (
select tt.texture_id
from tag_texture tt join tags t
on t.id = tt.tag_id
where t.name in ('tag1', 'tag2')
group by tt.texture_id
HAVING COUNT(DISTINCT t.id) = 2
)
) OR (textures.id in (
select ct.texture_id
from category_texture ct join categories c
on c.id = ct.category_id
where c.name in ('category1', 'category2')
group by ct.texture_id
HAVING COUNT(DISTINCT c.id) = 2
)
) OR (textures.id in (
select tex.id
from textures tex
where tex.name LIKE 'texturename'
group by tex.id
HAVING COUNT(DISTINCT tex.id) = 1
)
) ) AND textures.is_published = 1
The problem is that if I search for texturename tag1, all texturename results will be found, even if they have nothing to do with tags. However, if I search for "tag1 tag2", the resulting list is filtered out (less results than just searching tag1). Changing those ORs to AND widens the results even more, obviously.
What's the best way to merge these results so that each time a word is filtered the result set is narrowed down?
Changing all the OR to AND should solve the problem:
SELECT id, name
FROM textures
WHERE ((textures.id in (
select tt.texture_id
from tag_texture tt join tags t
on t.id = tt.tag_id
where t.name in ('1k', 'test')
group by tt.texture_id
HAVING COUNT(DISTINCT t.id) = 2
)
) AND (textures.id in (
select ct.texture_id
from category_texture ct join categories c
on c.id = ct.category_id
where c.name in ('mine')
group by ct.texture_id
HAVING COUNT(DISTINCT c.id) = 1
)
) AND (textures.id in (
select tex.id
from textures tex
where tex.name LIKE '%apple%'
group by tex.id
HAVING COUNT(DISTINCT tex.id) = 1
)
) ) AND textures.is_published = 1
SqlFiddle
There's no need to use DISTINCT in this query. You're not joining with any other tables, so nothing is going to cause the results to multiply.
If you want to search for the same keywords in all the fields, and require that at least one of them match each field, get rid of the GROUP BY and HAVING clauses.
select textures.id, textures.name from textures
WHERE ((textures.id in (
select tt.texture_id
from tag_texture tt join tags t
on t.id = tt.tag_id
where t.name in ('1k', 'test', 'apple', 'mine')
)
) AND (textures.id in (
select ct.texture_id
from category_texture ct join categories c
on c.id = ct.category_id
where c.name in ('1k' 'test', 'apple', 'mine')
)
) AND (textures.id in (
select tex.id
from textures tex
where tex.name LIKE '%1k%' OR tex.name LIKE '%test%' OR tex.name LIKE '%apple%'
OR tex.name LIKE '%mine%'
)
) ) AND textures.is_published = 1
I added mine to the list of keywords, because otherwise there was no match in the categories table.
SqlFiddle
users:
uid int(11) - userid(primary key, auto_increment)
name varchar(255)
pass varchar(64)
created int(11)
projects:
pid int(11) .....
name varchar(150)
description varchar(255)
created int(11)
users_projects:
uid int(11) - user id
pid int(11) - product id
How can i select the project with the most ussers assigned to it?
sql query.
You could use something like this:
select p.pid,
p.name,
up.TotalUsers
from projects p
inner join
(
select pid, count(uid) TotalUsers
from users_projects
group by pid
) up
on p.pid = up.pid
order by TotalUsers Desc
-- limit 1
See SQL Fiddle with Demo
This will return the list of all projects and a count of the Total Users per project. If you want to return the project will the most users, then you will include the limit 1 that is commented out.
If you have more than one project that has the same number of users, then you would want to use something similar to this:
select p.pid,
p.name,
up.TotalUsers
from projects p
inner join
(
select pid, count(uid) TotalUsers
from users_projects
group by pid
) up
on p.pid = up.pid
where totalusers = (select count(*) Total
from users_projects
group by pid
order by total desc
limit 1)
See SQL Fiddle with Demo
Thanks #JW for the Fiddle
The following query will include multiple projects having the same number of users and happens to be the most number of users.
SELECT a.name userName, c.name ProjectName
FROM users a
INNER JOIN users_projects b
ON a.uid = b.uid
INNER JOIN projects c
ON b.pid = c.pid
INNER JOIN
(
SELECT pid, COUNT(*) totalCount
FROM users_projects
GROUP BY pid
HAVING COUNT(*) = (SELECT COUNT(*) x
FROM users_projects
GROUP BY pid
ORDER BY x DESC
LIMIT 1)
) d ON b.pid = d.pid
ORDER BY a.Name ASC
SQLFiddle Demo (with duplicate project)
SQLFiddle Demo
If you are looking for only one project, the following is the fastest way:
select project_id, count(*) as NumUsers
from user_projects
group by project_id
order by count(*) desc
limit 1
(I am assuming that "product" = "project").
I have a MySQL query like this:
SELECT *
FROM backstage
WHERE backstage_id IN (
SELECT backstage_id
FROM visitor_counter
WHERE backstage_id !=0
GROUP BY backstage_id
ORDER BY COUNT( DISTINCT ( ip_address ) ) DESC
)
LIMIT 0 , 100
I get the results I want, but I would like to order it by COUNT( DISTINCT ( ip_address ) ) DESC as the inner question does.
Any tips on how to do this?
Give this a go and see if it gives you what you're after:
select bs.*
from backstage bs
inner join
(
select backstage_id,count(distinct ip_address) as distIpCount
from visitor_counter
where backstage_id !=0
group by backstage_id
) vc on vc.backstage_id = bs.backstage_id
order by vc.distIpCount desc
limit 0,100;