Join on multiple rows - mysql

I'm trying to load rows form a posts table based on whether they have multiple rows in another table. Take the below table structures:
posts
post_id post_title
-------------------
1 My Post
2 Another Post
post_tags
post_tag_id post_tag_name
--------------------------
1 My Tag
2 Another Tag
postTags
postTag_id postTag_tag_id postTag_post_id
------------------------------------------
1 1 1
2 2 1
Unsurprisingly, post and post_tags stores the posts and tags, and postTags joins which posts have which tags.
What I'd normally do to join the tables is this:
SELECT * FROM (`posts`)
JOIN `postTags` ON (`postTag_post_id` = `post_id`)
JOIN `post_tags` ON (`post_tag_id` = `postTag_tag_id`)
Then I'd have information on the tags, and can have additional stuff later in the query to search tag names for search terms etc, and then GROUP once I have posts that match the search terms.
What I'm trying to do is only select from posts where a post has both tag 1 AND tag 2, and I can't work out the SQL for it. I think it needs to be done in the actual JOIN rather than having a WHERE clause for it as when I run the join above I'd obviously get two rows back, so I can't have something like
WHERE post_tag_id = 1 AND post_tag_id = 2
as each row will only have one post_tag_id, and I can't check different values for the same column in one row.
What I've tried to do is something like this:
SELECT * FROM (`posts`)
JOIN `postTags` ON (postTag_tag_id = 1 AND postTag_tag_id = 2)
JOIN `post_tags` ON (`post_tag_id` = `postTag_tag_id`)
but this is returning 0 results when I run it; I've put conditions like this in JOINS before for similar things and I'm sure it's close but can't quite work out what to do if this doesn't work.
Am I at least on the right track? Hopefully I'm not missing something obvious.
Thanks.

You are trying to ask the postTags row to be at the same time one thing and another.
You either need to do two joins to post_tags and postTags so you get both. Or you can say that the post can have whatever tag between those two and the total amount of tags must equal two (assuming a post cannot related to the same tag more than once).
First approach:
SELECT *
FROM `posts` as p
WHERE p.`post_id` IN (SELECT pt.`postTag_post_id`
FROM `postTags` as pt
WHERE pt.`postTag_tag_id` = 1)
AND p.`post_id` IN (SELECT pt.`postTag_post_id`
FROM `postTags` as pt
WHERE pt.`postTag_tag_id` = 2);
Second approach:
SELECT *
FROM posts as p
WHERE p.post_id IN (SELECT pt.postTag_post_id
FROM (SELECT count(0) as c, pt.postTag_post_id
FROM postTags as pt
WHERE pt.postTag_tag_id IN (1, 2)
GROUP BY pt.postTag_post_id
HAVING c = 2) as pt);
I want also to add that if you use IN or EXISTS in the first approach then you won't have multiple lines for the same post row just because you have more than one tag. This way you save one DISTINCT later that would make your query slower.
I've used an IN in the second approach just as a rule of thumb I use: if you don't need to show the data you don't need to do a JOIN in the FROM section.

SELECT p.*, t1.*, t2.* FROM posts p
INNER JOIN postTags pt1 ON pt1.postTag_post_id = p.id AND pt1.postTag_tag_id = 1
INNER JOIN postTags pt2 ON pt2.postTag_post_id = p.id AND pt2.postTag_tag_id = 2
INNER JOIN post_tags t1 ON t1.post_tag_id = pt1.postTag_tag_id
INNER JOIN post_tags t2 ON t2.post_tag_id = pt2.postTag_tag_id

Without actually building a db the same as yours this is hard to verify but it should work.
Let me start by saying that this type of query is much easier and much more performant in a database that supports analytic queries (Oracle, MS SQL Server). So in MySQL you have to do it the old, crappy, aggregate way.
I also want to say that having a table that stores the names of the tags in post_tags and then the mapping of post tags to posts in postTags is confusing. If it were me, I would change the name of the mapping table to post_tags_map or post_tags_to_post_map. So you would have posts with post_id, post_tags with post_tags_id, and post_tags_map with post_tags_map_id. And those id columns would be named the same in every table. Having the same column that is named differently in other tables is also confusing.
Anyways, let's solve your problem.
First you want a result set that is 1 post id per row, and only the posts that have tags 1 & 2.
select postTag_post_id, count(1) cnt from (
select postTag_post_id from postTags where postTag_tag_id in (1, 2)
) group by postTag_post_id;`
That should give you back data like this:
postTag_post_id | cnt
1 | 2
Then you can join that result set back to your posts table.
select * from posts p,
(
select postTag_post_id, count(1) cnt from (
select postTag_post_id from postTags where postTag_tag_id in (1, 2)
) group by postTag_post_id;
) t
where p.post_id = t.postTag_post_id
and t.cnt >= 2;
If you need to do another join to the post_tags table in order to get the postTag_tag_id from the post_tag_name, your inner most query would change like so:
select postTag_post_id
from postTags a,
post_tags b
where a.postTag_tag_id = b.post_tag_id
and b.post_tag_name in ('tag 1', 'tag 2');
That should do the trick.

Assuming you already know tag IDs (1 and 2), you could do something like this:
SELECT post_id, post_title
FROM posts JOIN postTags ON (postTag_post_id = post_id)
WHERE postTag_tag_id IN (1, 2)
GROUP BY post_id, post_title
HAVING COUNT(DISTINCT postTag_tag_id) = 2
NOTE: DISTINCT is not necessary if there is an alternate key on postTags {postTag_tag_id, postTag_post_id}, as it should be.
NOTE: If you don't have tag IDs (and just have tag names), you'll need another JOIN (towards the post_tags table).
BTW, you should seriously consider ditching the surrogate PK in the junction table (postTags.postTag_id) and just having the natural PK {postTag_tag_id, postTag_post_id}. InnoDB tables are clustered, and secondary indexes in clustered tables are fatter and slower than in heap-based tables. Also, some queries can benefit from storing posts tagged by the same tag physically close together (or storing tags of the same post close together, if you reverse the PK).

Related

MySQL - Merging two queries that have different conditions and limits

I'm implementing a Tag System for my website, using PHP + MySQL.
In my database, I have three tables:
Posts
Id
Title
DateTime
Primary Key: Id
Tags
Id
Tag
Slug
1
First Tag
first-tag
Primary Key: Id | Key: Slug
TagsMap
Id
Tag
Primary Key: both
(Id = post's Id in Posts; Tag = Tag's Id in Tags)
Given, for instance, the url www. ... .net/tag/first-tag, I need to show:
the tag's name (in this case: "First Tag");
the last 30 published posts having that tag.
In order to achieve this, I'm using two different queries:
firstly
SELECT Tag FROM Tags WHERE Slug = ? LIMIT 1
then
SELECT p.Title FROM Posts p, Tags t, TagsMap tm
WHERE p.Id = tm.Id
AND p.DateTime <= NOW()
AND t.Id = tm.Tag
AND t.Slug = ?
ORDER BY p.Id DESC
LIMIT 30
But I don't think it's a good solution in terms of performance (please, correct me if I'm wrong).
So, my question is: how (if possible) to merge those two queries into just one?
Thanks in advance for Your suggestions.
The query that you have shown above is not a optimal solution as first it creates a cartesian product of all the tables and then filters out the data based on the conditions. If these tables become heavier in future, then your query will start slowing down (SLOW QUERIES).
Please use joins over this approach. ex. INNER JOIN, LEFT JOIN, RIGHT JOIN etc.
Try this SQL:
SELECT t.*, p.* FROM Tags t
INNER JOIN TagsMap tm ON (tm.Tag = t.Id )
INNER JOIN Posts p ON (p.Id = tm.Id AND p.DateTime <= NOW())
WHERE t.slug LIKE 'First Tag'
ORDER BY p.Id DESC
LIMIT 30
Given that you have structured your tables in a manner where you can utilize foreign keys and match them with their counterparts, then you can make use of JOIN's in your query.
SELECT
Tags.Tag,
Posts.title
FROM
Tags
LEFT JOIN
TagsMap ON Tags.id = TagsMap.tag
LEFT JOIN
Posts ON TagsMap.id = Posts.id AND
Posts.DateTime <= NOW()
WHERE
Posts.id = TagsMap.id AND
Tags.Slug = ?
ORDER BY
Posts.id DESC
LIMIT 30
The idea is that the query is optimized, but you will need to filter your result set programmatically in the view, in order to display the Tag only once.
If there is at most one "slug" per "post", include slug as a column in Posts.
If there can be any number of "tags" per "post", then have a table
CREATE Tags (
post_id ... NOT NULL,
tag VARCHAR(..)... NOT NULL,
post_dt DATETIME NOT NULL,
PRIMARY KEY(post_id),
INDEX(tag, dt)
) ENGINE=InnoDB
And you may want to use LEFT JOIN Tags and GROUP_CONCAT(tag).
I don't know what you mean by "first" in "first_tag". Maybe you should get rid of "first"?
The last 30 posts for a given tag:
SELECT p.*,
( SELECT GROUP_CONCAT(tag) FROM Tags ) AS tags
FROM ( SELECT post_id FROM tags WHERE tag = ?
ORDER BY post_dt DESC LIMIT 30 ) AS x
JOIN posts AS p ON p.id = x.post_id

Select only entities with n relations from an n:m relationship

I have an m:n relationship of images and tags in my database, which uses a crosstable to model this.
Table imgs contains much more information than just img_id, but that is all that's required to uniquely identify an image.
I want to find every img_id which has both tagA and tagB (and tagC and so on, I'll build this string so it won't really matter whether its two or ten tags).
Now, where I'm stuck is, of course first you'll join imgs with img_tags with tags, add a where clause for the tags;
SELECT *
FROM imgs
INNER JOIN img_tags ON imgs.img_id = img_tags.img_id
INNER JOIN tags ON img_tags.tag_id = tags.tag_id
WHERE tag = 'tagA' OR tag = 'tagB';
and then you'll get rows with identical imgs information, only differing in tag and tag_id. Now I should be able to count those, targeting only those which appear in the same amount as tags were supplied (Count(*) = n), and then using group by to aggregate them? But I can't quite figure it out.
In case it might be of relevance, you might assume the fields in img_tags are both foreign keys referencing the other tables, however that is not the case, they are not linked in any way.
You can use aggregation like this:
SELECT i.*
FROM imgs i JOIN
img_tags it
ON i.img_id = it.img_id JOIN
tags t
ON it.tag_id = t.tag_id
WHERE tag IN ('tagA', 'tagB')
GROUP BY i.img_id
HAVING COUNT(*) = 2;
Aggregating by i.img_id is safe -- and supported by the SQL standard -- assuming that img_id is the primary key in the table.
Here is on approach using a correlated subquery:
SELECT i.*
FROM imgs i
WHERE (
SELECT COUNT(*)
FROM img_tags it
INNER JOIN tags t ON it.tag_id = t.tag_id
WHERE i.img_id = it.img_id AND t.tag IN('tagA', 'tagB')
) = 2
This assumes no duplicate tags in your data structure. Otherwise, you can use COUNT(DISTINCT t.tag) instead of COUNT(*).
You can also use aggregation:
SELECT i.id
FROM imgs i
INNER JOIN img_tags it ON i.img_id = it.img_id
INNER JOIN tags t ON it.tag_id = t.tag_id
WHERE t.tag IN('tagA', 'tagB')
GROUP BY i.id
HAVING COUNT(*) = 2
If there are not many tags involved, I would just use exists (or not exists if you want to exclude some tags) for this
select *
from imgs
where
exists(select 1 from img_tags it where it.tag_id=(select tag_id from tags where tag='tagA') and it.img_id=imgs.img_id)
and exists(select 1 from img_tags it where it.tag_id=(select tag_id from tags where tag='tagB') and it.img_id=imgs.img_id);
especially if you end up wanting to do more complicated boolean expressions e.g. (A and (B or not C)).

MySQL - match all tags rather than any

I have an SQL setup akin to the following:
ARTICLES
id (PK)
name
TAGS
id (PK)
tag
...and a third table logging associations between the two, since there can be multiple tags to each article:
ARTICLE_TAG_ASSOCS
id (PK)
article_id (FK)
tag_id (FK)
Via this question I managed to construct a query that would find articles that were tagged with at least one of a number of tags, e.g.
SELECT articles.*
FROM articles
JOIN article_tag_assocs ata ON articles.id = ata.article_id
JOIN tags ON tags.id = ata.tag_id
WHERE tags.tag = 'budgie' OR tags.tag = 'parrot';
Question: How can I alter the above to find articles that match ALL tags, i.e. both 'budgie' and 'parrot', not just one?
Clearly modifying the logic to
WHERE tags.tag = 'budgie' && tags.tag = 'parrot';
...is logically flawed, since MySQL is considering each tag in isolation, one at a time, but hopefully you get what I mean.
There are several workable approaches.
One approach is to perform separate JOIN operations for each tag. For example:
SELECT articles.*
FROM articles
JOIN article_tag_assocs ata
ON ata.article_id = articles.id
JOIN tags ta
ON ta.id = ata.tag_id
AND ta.tag = 'budgie'
JOIN article_tag_assocs atb
ON atb.article_id = articles.id
JOIN tags tb
ON tb.id = atb.tag_id
AND tb.tag = 'parrot'
Note that this can return "duplicate" rows if a given articles is associated to the same tag value more than once. (Adding the DISTINCT keyword or a GROUP BY clause are ways to eliminate the duplicates.)
Another approach, if we are guaranteed that a given article has no duplicate tag values, is to use an inline view to get the list of article_id that are associated with both tags, and then JOIN that set to the articles table. For example:
SELECT a.*
FROM ( SELECT ata.article_id
FROM article_tag_assocs ata
JOIN tags t
ON t.id = ata.tag_id
WHERE t.tag IN ('budgie','parrot')
GROUP BY ata.article_id
HAVING COUNT(1) = 2
) s
JOIN articles a
ON a.id = s.article_id
Note that the literal "2" in the HAVING clause matches the number of values in the predicate on the tag column. The inline view (aliased as s) returns a distinct list of article_id, and we can join that to the articles table.
This approach is useful if you wanted to match, for example, at least three out of four tags. We could use lines like this in the inline view query.
WHERE t.tag IN ('fee','fi','fo','fum')
HAVING COUNT(1) >= 3
Then, any article that matched at least three of those four tags would be returned.
These aren't the only ways to return the specified result, there are several other approaches.
As Roland's answer pointed out, you can also do something like this:
FROM articles a
WHERE a.id IN ( <select article id values related to tag 'parrot'> )
AND a.id IN ( <select article id values related to tag 'bungie'> )
You could also use an EXISTS clause with a correlated subquery, though this approach doesn't usually perform as well with large sets, due to the number of executions of the subquery
FROM articles a
WHERE EXISTS ( SELECT 1
FROM article_tag_assocs s1
JOIN tags t1 ON t1.tag = 'bungie'
WHERE s1.article_id = a.id
)
AND EXISTS ( SELECT 1
FROM article_tag_assocs s2
JOIN tags t2 ON t2.tag = 'parrot'
WHERE s2.article_id = a.id
)
NOTE: in this case, it is possible to reuse the same table aliases within each subquery, because it doesn't lead to ambiguity, though I still prefer distinct aliases because the table aliases show up in the EXPLAIN output, and the distinct aliases make it easier to match the rows in the EXPLAIN output to the references in the query.)
What about this?
Will this give bad performance like EXISTS for large data sets?
This query is to check which rows of 'a1' table has some specified tags and not has some other specified tags
SELECT * FROM a1 WHERE a1.id IN
(SELECT taggables.taggable_id FROM taggables WHERE taggables.taggable_type = 'a1' AND taggables.tag_id IN (1))
AND a1.id NOT IN
(SELECT taggables.taggable_id FROM taggables WHERE taggables.taggable_type = 'a1' AND taggables.tag_id IN (2))
ORDER BY a1.file_count DESC LIMIT 0, 5

Where in with mysql or better?

Here is my actual query:
$result = mysqli_query($link,"select ids_billets from pref_tags where id='$_GET[id]'");
$tags_ids = $result->fetch_object();
$result = mysqli_query($link,"select id, html from pref_posts where id in ($tags_ids->ids_billets)");
while($posts = $result->fetch_object()) {
.....
}
I have ids in one varchar field of the pref_tags table (ids_billets) - example : "12,16,158"
Is there a better way to query this?
Thanks
Instead of one row with a comma-separated list, I would create a new table linking posts to tags, with one row per post/tag combo, i.e.:
posts
---------------------
post_id | html | etc.
posts_tags
----------------
post_id | tag_id
tags
------------------------
tag_id | tag_name | etc.
Then do something like this:
SELECT p.post_id, p.html
FROM posts p
INNER JOIN posts_tags pt
ON p.post_id = pt.post_id
INNER JOIN tags t
ON pt.tag_id = t.tag_id
WHERE t.tag_name = ?
Or if you already have the tag_id, like you seem to:
SELECT p.post_id, p.html
FROM posts p
INNER JOIN posts_tags pt
ON p.post_id = pt.post_id
WHERE pt.tag_id = ?
You can also do the same query in a different form, using a subquery:
SELECT post_id, html
FROM posts
WHERE post_id IN (SELECT post_id FROM posts_tags WHERE tag_id = ?)
Also, look at prepared statements, which will make it easy to avoid the serious SQL injection problems your current code has.
You should not put multiple values in a single column, because doing so breaks first normal form. Have a look at that link for examples of the problems you're likely to come across, and how to fix them.
Rather, create a separate table where you can have the ID and the Tag ID in separate columns. Then you can pull back the IDs in a subquery in your second example query, and get the benefits of being able to search for and manipulate individual IDs in other queries.

How to write mysql subselect properly with conditions and limiting

I have three Tables:
Posts:
id, title, authorId, text
authors:
id, name, country
Comments:
id, authorId, text, postId
I want to run a mysql command which selects the first 5 posts which were written by authors, whose country is 'Ireland'. In the same call, I want to retrieve all the comments for those five posts, and also the author info.
I've tried the following:
SELECT posts.id as 'posts.id', posts.title as 'posts.title' (etc. etc. list all fields in three table)
FROM
(SELECT * FROM posts, authors WHERE authors.country = 'ireland' AND authors.id = posts.authorId LIMIT 0, 5 ) as posts
LEFT JOIN
comments ON comments.postId = posts.id,
authors
WHERE
authors.id = posts.authorId
I had to include every field with an alias ^ because there was a duplicate for id, and more fields in future may become duplicates as I'm looking for a generic solution.
My two questions are:
1) I am getting a duplicate field entry from within my subselect for id, so do I have to list out all my fields as aliases again within the subselect or is there only one field I need for a subselect
2) Is there a way to auto-alias my call? At the moment I've just aliased every field in the main select but can it do this for me so there are no duplicates?
Sorry if this isn't very clear it's a bit of a messy problem! Thanks.
You are doing an unnecessary join back to the author table in your query. You get all the fields you want in the posts subquery. I would rename this to something other than an existing table, perhaps pa to indicate posts and authors.
You say you want the first 5 posts, but have no order clause. A better form of the query is:
SELECT pa.id as 'posts.id', pa.title as 'posts.title' (etc. etc. list all fields in three table)
FROM (SELECT *
FROM posts join
authors
on authors.id = posts.authorId
WHERE authors.country = 'ireland'
order by post.date
LIMIT 0, 5
) pa LEFT JOIN
comments c
ON c.postId = pa.id
Note that this returns the first five posts and their authors (as specified in the question). But one author may be responsible for all five posts.
In MySQL, you can use * and it will get rid of duplicate aliases in the from clause. I think this is dangerous. It is better to list all the columns you want.
To answer your questions:
You can select as many (or as few) columns as you need from a sub-query
You do not need to join the authors table again since you already selected all fields in the sub-query (and so get rid of duplicate columns names).
A few additional remarks...
... about the JOIN syntax
Prefer the form
FROM t1 JOIN t2 ON (t1.fk = t2.pk)
to the obsolete, obscure
FROM t1, t2 WHERE t1.fk = t2.pk
... about the use of a LIMIT clause without an ORDER BY clause
The order in which rows are returned by a SELECT statement without an ORDER BY clause is undefined. Therefore, a LIMIT n clause without an ORDER BY clause could return any n rows in theory.
Your final query should look like this:
SELECT *
FROM (
SELECT *
FROM posts
JOIN authors ON (authors.id = posts.authorId )
WHERE authors.country = 'ireland'
ORDER BY posts.id DESC -- assuming this column is monotonically increasing
LIMIT 5
) AS last_posts
LEFT JOIN comments ON ( comments.postId = last_posts .id )