I'm trying to write a script that will export modx users to CSV, fairly straightforward stuff, but in modx users can belong to many groups. Simply joining the modx_member_groups table will result in several rows for different users.
What I would like to do is somehow rewrite the query below so that the join on the modx_member_groups would return a list or array of group ids that the user belongs to.
For example, I would like the returned data to look like:
user_group | id | username | ...the rest
1,3,5,7 | 12 | johndoe | ...
here is the query I have.
SELECT mg.user_group, u.id, u.username, ua.*
FROM modx_users u
LEFT JOIN modx_user_attributes ua ON u.id = ua.internalKey
LEFT JOIN modx_member_groups mg ON u.id = mg.member
LIMIT 10
Ideally it would be awesome to somehow select the actual group names as columns. and then just force a true or false in the group name column.
UPDATE
I've updated the query after shtever's answer but have performance issues:
- GROUP_CONCAT was returning a BLOB type so I had to convert it, setting the group_concat_max_length to below 512 was not working
SELECT GROUP_CONCAT(CONVERT(mg.user_group, CHAR(10)) ORDER BY mg.user_group SEPARATOR ',') AS groups, u.id, u.username, ua.*
FROM modx_users u
LEFT JOIN modx_user_attributes ua ON u.id = ua.internalKey
LEFT JOIN modx_member_groups mg ON mg.member = u.id
GROUP BY u.id
The query now takes 27.5 seconds to execute if I limit it to 10 results or let it run on the entire 6000 users it always takes 27.5 seconds. If I remove the GROUP_CONCAT ~ same amount of time.
For MySQL, take a look at the GROUP_CONCAT function. Mysql GROUP_CONCAT Description
Your query might look something like:
SELECT GROUP_CONCAT(DISTINCT mg.user_group ASC SEPARATOR ',') , u.id, u.username, ua.*
FROM modx_users u
LEFT JOIN modx_user_attributes ua ON u.id = ua.internalKey
LEFT JOIN modx_member_groups mg ON u.id = mg.member
GROUP BY u.id, u.username
LIMIT 10
You might have to fiddle with the GROUP BY fields depending on the relation between the modx_user_attributes and modx_users table.
Related
SELECT DISTINCT u.id AS userId,u.type AS userType
FROM User AS u,Personal AS p,Company AS c
WHERE (p.realName LIKE '%adf%' AND u.type=1 AND u.id=p.userId)
OR (c.name LIKE '%grge%' AND u.id=c.userId)
LIMIT 0 , 10000
You can write your query as:
SELECT DISTINCT u.id AS userId,u.type AS userType
FROM User AS u inner join Personal AS p on u.id=p.userId
inner join Company AS c on u.id=c.userId
where p.realName LIKE '%adf%' or c.name LIKE '%grge%'
LIMIT 0 , 10000
Try to avoid comma seperated JOINS
You appear to be doing a quite hideous cross join, and then selectively narrowing down the records in the WHERE clause.
It is probably better to do 2 queries and union the results together. Each query can do one proper join. It is still going to have to access one column using the LIKE, and with a leading wild card that is not going to be quick (it can't use indexes).
SELECT u.id AS userId,
u.type AS userType
FROM User AS u
INNER JOIN Personal AS p
ON u.id = p.userId
WHERE p.realName LIKE '%adf%'
AND u.type = 1
UNION
SELECT u.id AS userId,
u.type AS userType
FROM User AS u
INNER JOIN Company AS c
ON u.id=c.userId
WHERE c.name LIKE '%grge%'
LIMIT 0 , 10000
I have a wordpress user database table 100,000+ users. As part of a plugin I need to list the subscribers. Obviously getting 100,000 users needs to be paginated. To get the total number of users to work out the pagination, I am running the main query without a limit and doing a PHP count() on the results:
SELECT role.umeta_id, role.user_id, role.meta_key, role.meta_value role, u.ID, u.user_login, u.user_email, u.user_registered
FROM wp_users AS u
LEFT JOIN wp_usermeta role ON role.user_id = u.ID
AND role.meta_key = 'wp_capabilities'
WHERE role.meta_value LIKE '%subscriber%'
GROUP BY u.ID
ORDER BY u.ID ASC
I am (unsurprisingly) running out of memory doing this. I have tried just doing a count similar to
SELECT COUNT( u.ID )
FROM wp_users AS u
LEFT JOIN wp_usermeta role ON role.user_id = u.ID
AND role.meta_key = 'wp_capabilities'
WHERE role.meta_value LIKE '%subscriber%'
GROUP BY u.ID
ORDER BY u.ID ASC
but rather than returning a single value, this returns rows and rows of count = 1.
I know that there are get_user functions in Wordpress to do this - I am just using this as a simplified example (the query is actually more complex)
So the question is "How can I efficiently get the total number of rows in such a situation as this?"
The problem with your query is that you're grouping by u.ID and count is an aggregate function
Edited:
I suggest getting rid of the group and the order by to where you're left with this
SELECT COUNT( u.ID )
FROM wp_users AS u
LEFT JOIN wp_usermeta role ON role.user_id = u.ID
AND role.meta_key = 'wp_capabilities'
WHERE role.meta_value LIKE '%subscriber%'
I need to get the count from one of the sub tables / joined tables involved in the query. I will demonstrate with a simple example:
Table: user
id name etc
-------------------------------------------
1 u1
2 u2
Table: exercise
id userId etc
-------------------------------------------
1 1
2 1
Now I need to select from user table various fields like id, name, etc along with the count of various user id in exercise table.
For example, in the above case I need the output:
id name count
-------------------------------------------
1 u1 2 --since two u1's are present in exercise
2 u2 0 --since no u2's are present in exercise
I tried this: but I get syntax error:
--actual query is very complex
SELECT u.id, u.name, COUNT(e.*)
FROM user AS u
JOIN exercise AS e ON u.id = e.userId
I tried this: but I get syntax error again:
--actual query is very complex
SELECT u.id, u.name, (SELECT COUNT(*) FROM e)
FROM user AS u
JOIN exercise AS e ON u.id = e.userId
This works, as a sub query, but is painfully slow (5 to 13 seconds for about 10000 rows in each table):
--actual query is very complex
SELECT u.id, u.name, (SELECT COUNT(*) FROM exercise WHERE e.userId = u.id)
FROM user AS u
Is there a way I can get the count info in one single query, with the help of join or so? Sub query is very slow for my needs.
Try using a GROUP BY, like this:
SELECT u.id, u.name, COUNT(e.userId)
FROM user AS u
LEFT JOIN exercise AS e
ON u.id = e.userId
GROUP BY u.id
Try this:
SELECT u.id, u.name, COUNT(e.userId)
FROM user AS u
LEFT JOIN exercise AS e
ON u.id = e.userId
GROUP BY u.id,u.name
Left join will still return you row from user table even if there are no records in exercise table.
I usually go with the join approach but in this case I am a bit confused. I am not even sure that it is possible at all. I wonder if the following query can be converted to a left join query instead of the multiple select in used:
select
users.id, users.first_name, users.last_name, users.description, users.email
from users
where id in (
select assigned.id_user from assigned where id_project in (
select assigned.id_project from assigned where id_user = 1
)
)
or id in (
select projects.id_user from projects where projects.id in (
select assigned.id_project from assigned where id_user = 1
)
)
This query returns the correct result set. However, I guess the repetition of the query that selects assigned.id_project is a waste.
You could start with the project assignments of user 1 a1. Then find all assignments of other people to those projects a2, and the user in the project table p. The users you are looking for are then in either a2 or p. I added distinct to remove users who can be reached in both ways.
select distinct u.*
from assigned a1
left join
assigned a2
on a1.id_project = a2.id_project
left join
project p
on a1.id_project = p.id
join user u
on u.id = a2.id_user
or u.id = p.id_user
where a1.id_user = 1
Since both subqueries have a condition where assigned.id_user = 1, I start with that query. Let's call that assignment(s) the 'leading assignment'.
Then join the rest, using left joins for the 'optional' tables.
Use an inner join on user that matches either users of assignments linked to the leading assignment or users of projects linked to the leading project.
I use distinct, because I assumen you'd want each user once, event if they have an assignment and a project (or multiple projects).
select distinct
u.id, u.first_name, u.last_name, u.description, u.email
from
assigned a
left join assigned ap on ap.id_project = a.id_project
left join projects p on p.id = a.id_project
inner join users u on u.id = ap.id_user or u.id = p.id_user
where
a.id_user = 1
Here's an alternative way to get rid of the repetition:
SELECT
users.id,
users.first_name,
users.last_name,
users.description,
users.email
FROM users
WHERE id IN (
SELECT up.id_user
FROM (
SELECT id_user, id_project FROM assigned
UNION ALL
SELECT id_user, id FROM projects
) up
INNER JOIN assigned a
ON a.id_project = up.id_project
WHERE a.id_user = 1
)
;
That is, the assigned table's pairs of id_user, id_project are UNIONed with those of projects. The resulting set is then joined with the user_id = 1 projects to obtain the list of all users who share the projects with the ID 1 user. And now it only remains to retrieve the details for those users, which in this case is done in the same way as in your query, i.e. using an IN clause.
I'm sorry to say that I don't have MySQL to thoroughly test the performance of this query and so cannot be quite sure if it is in any way better or worse than your original query or than the one suggested both by #GolezTrol and by #Andomar. Generally I tend to agree with #GolezTrol's comment that a query with simple (semi- or whatever-) joins and repetitive parts might turn out more efficient than an equivalent sophisticated query that doesn't have repetitions. In the end, however, it is testing that must reveal the final answer for you.
The following query does what I want. It returns all the resuls in the users table and then if there is a match in the details tble, returns the relevant data
users
id|username
details
id|userid|firstname|lastname
$sql = "SELECT u.*, d.*
FROM `users` u
LEFT JOIN `details` d on
u.id = d.userid
ORDER BY $strorder";
However, when I try to join an additonal table where I want to do the same thing--return all the results of the users table and if there is a match in the third table, return the relevant data (total followers of this user)--it only returns one record.
3rd table
follow
id|followerid|followedid
$sql = "SELECT u.*, d.*, COUNT(f.id)
FROM `users` u
LEFT JOIN `details` d on
u.id = d.userid
LEFT JOIN `follow` f on
u.id = f.followedid
ORDER BY $strorder";
Can anyone see what I am doing wrong? Thanks in advance for any suggestions.
Many thanks.
Try to avoid * to select fields, it will be clearer to group your datas (even if mysql is quite permissive with groupings).
When you have an aggregate function (like COUNT, SUM), the other "non aggregated" requested fields should be in a GROUP BY clause.
Mysql don't force you to GROUP BY all the fields, but... I think it's quite a good habit to be "as ANSI as possible" (usefull when you use another DBMS)
SELECT u.id, u.username, d.firstname, d.lastname, count(*) as numberfollowers
FROM user u
LEFT JOIN details d on u.id = d.userid
LEFT JOIN follow f on u.id = f.followedid
GROUP BY u.id, u.username, d.firstname, d.lastname --or just GROUP BY u.id with Mysql
ORDER BY count(*) desc
COUNT being an aggregate function, when selected with other columns, requires you to group your results by those other columns in the select list.
You should rewrite your query with columns that you want to select from users and details and group by those columns.