how to get better performance, with not in query - mysql

here is my custom sql query ;
SELECT id,
post_title,
post_content,
comment_count,
post_date,
post_status,
post_name
FROM wp_posts
WHERE post_type = 'post'
AND post_status = 'publish'
AND id NOT IN (SELECT DISTINCT post_id
FROM wp_postmeta
WHERE meta_key = 'visible_headlane'
AND meta_value = 'On')
AND id NOT IN (SELECT DISTINCT post_id
FROM wp_postmeta
WHERE meta_key = 'visible_homepage'
AND meta_value = 'On')
ORDER BY post_date DESC
LIMIT 11, 32
i can not use, LIMIT in NOT IN Query so, there is any way to use, or limit SELECT DISTINCT sql query, or any idea ?
Note : I need to optimise this query because, it takes so long , like a slow query.
Thanks.

Switch to NOT EXISTS( SELECT 1 FROM ... ) -- this variation does not need to compute the entire list of things; it only needs to check for the absence. LEFT JOIN .. ON .. IS NULL is also more efficient.
Furthermore, the standard schema for wp_postmeta is quite inefficient; see my suggestions.
Meanwhile, keep it as two tests, unlike the suggestion by 'holder'.
The EXISTS would look something like
AND NOT EXISTS ( SELECT 1 FROM wp_postmeta
WHERE post_id = wp_posts.id
AND meta_key = 'visible_headlane'
AND meta_value = 'On' )
which would make good use of the PRIMARY KEY(post_id, meta_key) that I recommend.
(Is it really 'headlane', not 'headline'?)

Maybe something like this as jarlh suggested
SELECT
ID,post_title,post_content,comment_count,post_date,post_status,post_name
FROM wp_posts t1
left join (
SELECT DISTINCT post_id from wp_postmeta
WHERE meta_key IN ('visible_headlane','visible_homepage') AND meta_value = 'On' ) t2 on t1.ID = t2.post_id
WHERE post_type = 'post'
AND post_status = 'publish'
AND t2.post_id is null
ORDER BY post_date DESC
LIMIT 11, 32

Related

SQL - Slow SQL Query

I have a sql query (see below) for wordpress which is taking around 4-5secs to get results. It gives all order ids which have a product/variation id in it.
I want to make it more fast, any help?
SELECT p.ID order_id
FROM wp_posts p
INNER JOIN wp_woocommerce_order_items i ON p.ID=i.order_id
INNER JOIN wp_woocommerce_order_itemmeta im ON i.order_item_id=im.order_item_id
WHERE im.meta_key IN ('_product_id','_variation_id')
AND im.meta_value IN ('703899','981273','981274','981275')
AND p.post_status IN ('wc-completed')
GROUP BY p.ID HAVING COUNT(p.ID)>1
ORDER BY p.post_date desc
LIMIT 0, 20
Above query EXPLAIN:
Why do you join when you only want to select IDs from wp_posts anyway?
SELECT p.ID order_id
FROM wp_posts p
WHERE p.post_status = 'wc-completed'
AND p.ID IN
(
SELECT i.order_id
FROM wp_woocommerce_order_items i
JOIN wp_woocommerce_order_itemmeta im ON im.order_item_id = i.order_item_id
WHERE im.meta_key IN ('_product_id','_variation_id')
AND im.meta_value IN ('703899','981273','981274','981275')
GROUP BY i.order_id
HAVING COUNT(*) > 1
)
ORDER BY p.post_date DESC
LIMIT 0, 20;
Now let's think about how the DBMS can address this. It can look for posts with status 'wc-completed', if there are only few such rows and then check whether they represent an order with more than one of the desired items. This would ask for these indexes:
create index idx1 on wp_posts(post_status, id, post_date);
create index idx2 on wp_woocommerce_order_items(order_id, order_item_id);
create index idx3 on wp_woocommerce_order_itemmeta(order_item_id, meta_key, meta_value);
Or it could look for the desired products, see whether an order contains more than one of them and then check whther this relates to a post with status = 'wc-completed'. That would ask for these indexes:
create index idx4 on wp_woocommerce_order_itemmeta(meta_key, meta_value, order_item_id);
create index idx5 on wp_woocommerce_order_items(order_item_id, order_id);
create index idx6 on wp_posts(id, post_status, post_date);
We don't know which way the DBMS will prefer, so we create all six indexes. Then we look at the explain plan to see which are being used and remove the others. Maybe the DBMS even sees no advantage in using indexes here at all, but I find this unlikely.
The first thing you can try doing is trimming what data you fetch.
That means:
Not fetching fields that you don't need/check
Implementing our constrains before joining
SELECT
p.ID order_id
FROM
(SELECT id, post_status, post_date FROM wp_posts WHERE post_status = 'wc-completed') p,
(SELECT order_id, order_item_id FROM wp_woocommerce_order_items) i,
(
SELECT
order_item_id,
meta_key,
meta_value
FROM
wp_woocommerce_order_itemmeta
WHERE
meta_key IN ('_product_id','_variation_id')
AND meta_value IN ('703899','981273','981274','981275')
) im
WHERE
p.ID = i.order_id
AND i.order_item_id = im.order_item_id
GROUP BY
p.ID
HAVING
COUNT(p.ID)>1
ORDER BY
p.post_date desc
LIMIT
0, 20
Edit:
If Inner joins are necessary, you can try:
SELECT
p.ID order_id
FROM
(SELECT id, post_status, post_date FROM wp_posts WHERE post_status = 'wc-completed') p
INNER JOIN
(SELECT order_id, order_item_id FROM wp_woocommerce_order_items) i
ON
p.ID = i.order_id
INNER JOIN
(
SELECT
order_item_id,
meta_key,
meta_value
FROM
wp_woocommerce_order_itemmeta
WHERE
meta_key IN ('_product_id','_variation_id')
AND meta_value IN ('703899','981273','981274','981275')
) im
ON
i.order_item_id = im.order_item_id
GROUP BY
p.ID
HAVING
COUNT(p.ID)>1
ORDER BY
p.post_date desc
LIMIT
0, 20
ps* I hope my syntax is correct ˙ my SQL is quite rusty

MySQL query stalling - is there a more efficient solution for this MySQL Query?

SELECT post_title,
count(*) AS c
FROM wp_posts
WHERE post_type = "product"
GROUP BY post_title
HAVING c > 1
ORDER BY c DESC
runs no problem, returns result in < 1 sec. Yet
select * from wp_posts where post_title in (
select post_title from wp_posts WHERE post_type = "product"
group by post_title having count(*) > 1
)
hangs up.
Yet they are fundamentally the same query except for the fact that in the second query I'm trying to pull out the entire record rather than just the post_title.
Have I erred? Is there a more efficient way to achieve the equivalent?
Edit: EXPLAIN query and SHOW CREATE TABLE wp_posts has been appended for your information.
you could avoid the IN clause on the subquery and use an inner join
select a.*
from wp_posts a
INNER JOIN (
select post_title
from wp_posts
WHERE post_type = "product"
group by post_title
having count(*) > 1
) t ON t.post_title = a.post_title
this should be more performant
The most efficient way to write this query is probably using exists . . . assuming wp_posts has a primary key:
select p.*
from wp_posts p
where p.post_type = 'product' and
exists (select 1
from wp_posts p2
where p2.post_title = p.post_title and
p2.post_type = p.post_type and
p2.id <> p.id
);
For performance, yu want an index on wp_posts(post_type, title, id).

Why is MySQL COUNT returning double of what I expect when counting joined columns?

I am trying to build a user notification system in wordpress. I have a query that joins rows in the wp_posts table that have a post_status of publish, a post_author of 1 and a post_type of user_notification. There are two rows that meet these requirements. The total_unread column is always 4 and the entire query only returns 1 row. What am I doing wrong? Here is a SqlFiddle
Here is the table structure
CREATE TABLE `wp_posts` (
`ID` bigint(20),
`post_author` bigint(20) DEFAULT '0',
`post_content` longtext NULL,
`post_status` varchar(20) DEFAULT 'publish',
`post_type` varchar(20) DEFAULT 'post'
) ENGINE=InnoDB;
INSERT INTO wp_posts
(ID, post_author, post_content, post_status, post_type)
VALUES(1, 1, 'John Smith would like to be friends!', 'publish', 'user_notification');
INSERT INTO wp_posts
(ID, post_author, post_content, post_status, post_type)
VALUES(2, 1, 'Sally Miller shared your post!', 'publish', 'user_notification');
Here is the query:
SELECT SQL_CALC_FOUND_ROWS wp_posts.ID, wp_posts.post_status, COUNT(unread.post_status) as total_unread
FROM wp_posts
JOIN wp_posts AS unread ON unread.post_author = 1 AND unread.post_status = 'publish' AND unread.post_type = 'user_notification'
WHERE wp_posts.post_author IN (1)
AND wp_posts.post_type = 'user_notification'
ORDER BY wp_posts.ID DESC LIMIT 0, 5
The problem is your JOIN is creating a cross-product of each post with each other post, so you get a count of 4 (2x2). If you had 3 similar rows in the table you would get a count of 9. Try running the query as SELECT * instead of your current fields and you will see the 4 rows.
I don't really understand why you are using a JOIN? What are you trying to achieve? If all you want to do is count the "unread" posts (indicated by post_status='publish') this query will suffice:
SELECT post_author, SUM(IF(post_status='publish',1,0)) as total_unread
FROM wp_posts
WHERE post_type = 'user_notification'
GROUP BY post_author
Or even simpler:
SELECT post_author, COUNT(*) as total_unread
FROM wp_posts
WHERE post_type = 'user_notification' AND post_status='publish'
GROUP BY post_author
I ran your SQL Fiddle added another record which was unpublished to see how the results would be and it was still doubling the value.
ALSO When using JOIN your query should be structured more like this :
SELECT * FROM wp_posts t1
JOIN wp_posts t2 ON t1.ID = t2.ID
WHERE...
//attempted this in your fiddle and it actually resolves your issue.
Since your JOIN has no such relationship this can all be done with one single query. I kept all your parameters because you might want the statistic for a unique user. You can view the result in this SQL Fiddle where I have added another post_author.
Here is the query I used:
SELECT post_author, post_status, COUNT(post_author) as total_unread
FROM wp_posts
WHERE post_type = 'user_notification'
AND post_status = 'publish'
AND post_author = 1
GROUP BY post_author

Return up to 5 of each 'post_type' in a DB table

I have a query that returns all posts from a DB table that match the given criteia, but I'm looking for a way to only return up to 5 posts from each 'post_type'. Currently the query is selecting every single post that matches and I am having to limit the numbers from each 'post_type' in PHP, which is not particularly efficient.
Can this be done? Thanks.
SELECT ID, post_title, post_type
FROM wp_posts
WHERE 1=1
AND post_status = "publish"
AND (
post_type IN (
"staff"
)
AND post_name LIKE "%The%"
)
OR (
post_type NOT IN (
"staff",
"Attachment"
)
AND (
post_name LIKE "%The%"
OR post_title LIKE "%The%"
)
)
ORDER BY post_type, post_name ASC
This solution will select the five most recent (based on id) posts per post_type:
SELECT a.id, a.post_title, a.post_type
FROM wp_posts a
INNER JOIN wp_posts b ON a.post_type = b.post_type AND a.id <= b.id
WHERE a.post_status = 'publish' AND
(a.post_type = 'staff' AND a.post_name LIKE '%The%') OR
(a.post_type NOT IN ('staff', 'Attachment') AND (a.post_name LIKE '%The%' OR a.post_title LIKE '%The%'))
GROUP BY a.id, a.post_title, a.post_type
ORDER BY a.post_type, a.post_name
HAVING COUNT(1) <= 5

MYSQL - Multiple Select and LIKE Method

What i want is:
Get posts with date greater then 2010-03-02 and with the meta_value 'something' + like '2010-'
because there are other values like 239048192304 or 293811743
$query = "SELECT DISTINCT wp_postmeta.meta_key, wp_postmeta.meta_value, wp_posts.ID, wp_posts.guid, wp_postmeta.post_id, wp_posts.post_title
FROM wp_postmeta
INNER JOIN wp_posts
ON wp_postmeta.post_id=wp_posts.ID
WHERE wp_postmeta.meta_value >='2010-03-02'
AND wp_postmeta.meta_value = 'something'
AND wp_postmeta.meta_value LIKE '2010-'
ORDER BY wp_postmeta.meta_value ASC
LIMIT 0,10";
can you help me out please? thank you!
Update2:
table wp_postmeta
post_id | meta_value
5 | 2010-12-30
5 | Berlin
3 | 2010-12-29
3 | Paris
2 | 2009-12-29
2 | Paris
14 | 12232456521
14 | Berlin
Output:
2010-12-30 Berlin ID 5
2010-12-29 Paris ID 3
Maybe you mean an OR instead of an AND?
...
WHERE wp_postmeta.meta_value >= '2010-03-02' OR
wp_postmeta.meta_value = 'something' OR
wp_postmeta.meta_value LIKE '2010-'
Unfortunately in the English language, AND and OR can be used interchangeably in certain cases:
"I always carry an umbrella for when it rains and snows."
"I always carry an umbrella for when it rains or snows."
The above wouldn't be equivalent for computers :)
A larger data set and sample answer would help clarify the question but here is my interpretation of what you are looking for. It's not elegant but if you've got the buffer space allocated it works.
SELECT DISTINCT wp_postmeta.meta_key, wp_postmeta.meta_value, wp_posts.ID, wp_posts.guid, wp_postmeta.post_id, wp_posts.post_title
FROM wp_postmeta
INNER JOIN wp_posts
ON wp_postmeta.post_id=wp_posts.ID
WHERE wp_postmeta.post_id IN (
select post_id from wp_postmeta where str_to_date(meta_value, '%Y-%m-%d') >= 2010-03-02' and post_id in (select post_id from wp_postmeta where meta_value = 'something')
);
For getting post information and their post meta values you need the following query:
SELECT DISTINCT wp_postmeta.meta_key, wp_postmeta.meta_value, wp_posts.ID,
wp_posts.guid, wp_postmeta.post_id, wp_posts.post_title
FROM wp_postmeta JOIN wp_posts
ON wp_postmeta.post_id = wp_posts.ID
WHERE wp_posts.post_date >= '2010-03-02'
AND EXISTS (SELECT 1 from wp_postmeta m1
WHERE m1.post_ised = wp_posts.ID
AND wp_postmeta.meta_value = 'something'
AND EXISTS (SELECT 1 from wp_postmeta m2
WHERE m2.post_ised = wp_posts.ID
AND wp_postmeta.meta_value LIKE '2010-%')
ORDER BY wp_postmeta.meta_value
ASC LIMIT 0, 10
Try this:
$query = "SELECT p.ID, m1.meta_value, m2.meta_value, p.post_title FROM
wp_posts p, wp_postmeta m1, wp_postmeta m2
WHERE p.post_date > '2010-03-02' AND
m1.post_id=p.ID AND m2.post_id=p.ID AND
m2.meta_value LIKE '2010-%' AND
m1.meta_value = 'something'
ORDER BY m1.meta_value, m2.meta_value
LIMIT 0,10";
No need for the distinct, since we're showing everything on one row anyway.
Those statements contradict themselves. First you are asking for it to be >= a date. So is meta_value a date? Then you say it must be equaled to 'something', so now it is a string and not a date. Finally you say have it be like 2010-, so now we are back to a string or a date, but no wild card % in the like so you are basically saying it has to be equaled to 2010- as well.
What is the value stored in meta_value?
Are you sure you do not mean to query multiple fields?
UPDATE
Based on the new information, I think this is what you want:
$query = "SELECT DISTINCT wp_postmeta.meta_key, wp_postmeta.meta_value, wp_posts.ID, wp_posts.guid, wp_postmeta.post_id, wp_posts.post_title
FROM wp_postmeta
INNER JOIN wp_posts
ON wp_postmeta.post_id=wp_posts.ID
WHERE (wp_postmeta.meta_value = 'something'
OR wp_postmeta.meta_value LIKE '2010-%')
ORDER BY wp_postmeta.meta_value ASC
LIMIT 0,10";
Hopefully that is what you were after. I removed the 2010 >= condition given that the 2010 LIKE condition will pull that same data.