I have a wordpress user database table 100,000+ users. As part of a plugin I need to list the subscribers. Obviously getting 100,000 users needs to be paginated. To get the total number of users to work out the pagination, I am running the main query without a limit and doing a PHP count() on the results:
SELECT role.umeta_id, role.user_id, role.meta_key, role.meta_value role, u.ID, u.user_login, u.user_email, u.user_registered
FROM wp_users AS u
LEFT JOIN wp_usermeta role ON role.user_id = u.ID
AND role.meta_key = 'wp_capabilities'
WHERE role.meta_value LIKE '%subscriber%'
GROUP BY u.ID
ORDER BY u.ID ASC
I am (unsurprisingly) running out of memory doing this. I have tried just doing a count similar to
SELECT COUNT( u.ID )
FROM wp_users AS u
LEFT JOIN wp_usermeta role ON role.user_id = u.ID
AND role.meta_key = 'wp_capabilities'
WHERE role.meta_value LIKE '%subscriber%'
GROUP BY u.ID
ORDER BY u.ID ASC
but rather than returning a single value, this returns rows and rows of count = 1.
I know that there are get_user functions in Wordpress to do this - I am just using this as a simplified example (the query is actually more complex)
So the question is "How can I efficiently get the total number of rows in such a situation as this?"
The problem with your query is that you're grouping by u.ID and count is an aggregate function
Edited:
I suggest getting rid of the group and the order by to where you're left with this
SELECT COUNT( u.ID )
FROM wp_users AS u
LEFT JOIN wp_usermeta role ON role.user_id = u.ID
AND role.meta_key = 'wp_capabilities'
WHERE role.meta_value LIKE '%subscriber%'
Related
Database Tables:
The below tables contain the information about users and their licenses.
Note: One user can have multiple licenses.
users table
Users( 'user_id', 'user_email', 'name' )
license table
Licenses( 'license_id', 'user_id', 'license_key', 'license_status' )
Write an optimized MySql query to generate the output like
Desired output
Edit:
I have tried this SELECT u.user_email, u.name as user_name, l.license_status as status, l.license_key FROM users u left join licenses l on u.user_id = l.user_id but it's not showing expected result and I'm not sure how can I get the Desired output
GROUP_CONCAT to the rescue:
SELECT u.user_email,
u.name as user_name,
GROUP_CONCAT(l.license_status ORDER BY license_id) AS statuses,
GROUP_CONCAT(l.license_key ORDER BY license_id) AS "keys"
FROM users u
left join licenses l ON u.user_id = l.user_id
GROUP BY u.user_id
If desired change to
COALESCE(GROUP_CONCAT(...), "none") AS ...
If you don't want the users with no licenses, change LEFT JOIN to JOIN.
In my database, I have users, apps, and releases. A user can have 0..n apps through a permissions table and an app can have 0..n releases.
I'm trying to get a list of users who have at least 1 app, but none of that user's apps have any releases.
The schema is roughly
users permissions apps releases
----- ----------- ---- --------
id user_id id id
email app_id app_id
I think I've got something working with this, but it appears inefficient to me because I mention the permissions table twice and I'm using nested exists clauses. Is there a more efficient way to write this query?
select u.email from users u
join permissions p on p.user_id = u.id
where not exists (
select a.id from apps a
join permissions p on p.app_id = a.id
where p.user_id = u.id and exists (
select r.id from releases r
where r.app_id = a.id
)
);
You just need to use a LEFT JOIN on releases, and then look for the case where the number of released apps (r.app_id is non-NULL) is 0. If all you want is a list of users, I don't think you need to JOIN the apps table at all, as JOINing on permissions will ensure that only users that have permission for 1 or more apps are included.
SELECT u.email
FROM users u
JOIN permissions p ON p.user_id = u.id
LEFT JOIN releases r ON r.app_id = p.app_id
GROUP BY u.email
HAVING COUNT(r.app_id) = 0
The first Join seems to be correct between users and permissions table. You just need to check whether the app_id from joined result-set exists in releases table or not. You can try this query -
select u.email from users u
join permissions p on p.user_id = u.id
where not exists ( Select 1 from releases r where r.App_id = p.app_id)
I will do something like this, hope this helps:
SELECT
u.id, u.email
FROM
users AS u
INNER JOIN
permissions AS p ON p.user_id = u.id
LEFT JOIN
releases AS r ON r.app_id = p.app_id
GROUP BY
u.id, u.email
HAVING
SUM(CASE WHEN r.id IS NOT NULL THEN 1 ELSE 0 END) = 0
Another thing you could try is a combination of left and inner joins like this:
Select
email
From users u
Inner Join (
Select
p.user_id
, p.app_id
From permissions p
Left Join releases r
on p.app_id = r.app_id
Where r.app_id is null) a
on u.user_id = a.user_id
Group by email
It's hard to tell which is faster between this and the previous posted solution without knowing the size of the different tables (and hence how many rows SQL will be trying to join).
One thing that is clear - without the 'Group by email' line at the end, you might see users' email repeated multiple times in your list. Generally, literature on SQL states that using a "group by" statement at the end of your query is a faster way to get a distinct set than a "select distinct" statement at the beginning of your query.
I'm trying to write a script that will export modx users to CSV, fairly straightforward stuff, but in modx users can belong to many groups. Simply joining the modx_member_groups table will result in several rows for different users.
What I would like to do is somehow rewrite the query below so that the join on the modx_member_groups would return a list or array of group ids that the user belongs to.
For example, I would like the returned data to look like:
user_group | id | username | ...the rest
1,3,5,7 | 12 | johndoe | ...
here is the query I have.
SELECT mg.user_group, u.id, u.username, ua.*
FROM modx_users u
LEFT JOIN modx_user_attributes ua ON u.id = ua.internalKey
LEFT JOIN modx_member_groups mg ON u.id = mg.member
LIMIT 10
Ideally it would be awesome to somehow select the actual group names as columns. and then just force a true or false in the group name column.
UPDATE
I've updated the query after shtever's answer but have performance issues:
- GROUP_CONCAT was returning a BLOB type so I had to convert it, setting the group_concat_max_length to below 512 was not working
SELECT GROUP_CONCAT(CONVERT(mg.user_group, CHAR(10)) ORDER BY mg.user_group SEPARATOR ',') AS groups, u.id, u.username, ua.*
FROM modx_users u
LEFT JOIN modx_user_attributes ua ON u.id = ua.internalKey
LEFT JOIN modx_member_groups mg ON mg.member = u.id
GROUP BY u.id
The query now takes 27.5 seconds to execute if I limit it to 10 results or let it run on the entire 6000 users it always takes 27.5 seconds. If I remove the GROUP_CONCAT ~ same amount of time.
For MySQL, take a look at the GROUP_CONCAT function. Mysql GROUP_CONCAT Description
Your query might look something like:
SELECT GROUP_CONCAT(DISTINCT mg.user_group ASC SEPARATOR ',') , u.id, u.username, ua.*
FROM modx_users u
LEFT JOIN modx_user_attributes ua ON u.id = ua.internalKey
LEFT JOIN modx_member_groups mg ON u.id = mg.member
GROUP BY u.id, u.username
LIMIT 10
You might have to fiddle with the GROUP BY fields depending on the relation between the modx_user_attributes and modx_users table.
I've never been all that great with much more then regular select queries. I have a new project that has users, roles and assigned_roles (lookup table for users with roles).
I want to group_concat the roles.name so that my result shows me what roles each user has assigned.
I've tried several things:
select users.id, users.displayname,users.email, rolenames from `users`
left join `assigned_roles` on `assigned_roles`.`user_id` = `users`.`id`
left join (SELECT `id`, group_concat(`roles`.`name`) as `rolenames` FROM `roles`) as uroles ON `assigned_roles`.`role_id` = `uroles`.`id`
This gives me the grouped role names but shows me duplicate entries if a user has two roles, so the second row in the result shows the same user but no role names.
select users.id, users.displayname,users.email, rolenames from `users`
join `assigned_roles` on `assigned_roles`.`user_id` = `users`.`id`
join (SELECT `id`, group_concat(`roles`.`name`) as `rolenames` FROM `roles`) as uroles ON `assigned_roles`.`role_id` = `uroles`.`id`
Just regular joins shows me what I want but wont lists users who do not have any assigned.roles, so its not complete.
I'll keep plugging away but I thought stack could help, hopefully I'll learn a bit more about joins today.
Thank you.
For GROUP CONCAT to work in this scenario, you'll need a GROUP BY to get the group info per user, something like;
SELECT u.id, u.displayname, u.email, GROUP_CONCAT(r.name) rolenames
FROM users u
LEFT JOIN assigned_roles ar ON ar.user_id = u.id
LEFT JOIN roles r ON r.id = ar.role_id
GROUP BY u.id, u.displayname, u.email
The following query does what I want. It returns all the resuls in the users table and then if there is a match in the details tble, returns the relevant data
users
id|username
details
id|userid|firstname|lastname
$sql = "SELECT u.*, d.*
FROM `users` u
LEFT JOIN `details` d on
u.id = d.userid
ORDER BY $strorder";
However, when I try to join an additonal table where I want to do the same thing--return all the results of the users table and if there is a match in the third table, return the relevant data (total followers of this user)--it only returns one record.
3rd table
follow
id|followerid|followedid
$sql = "SELECT u.*, d.*, COUNT(f.id)
FROM `users` u
LEFT JOIN `details` d on
u.id = d.userid
LEFT JOIN `follow` f on
u.id = f.followedid
ORDER BY $strorder";
Can anyone see what I am doing wrong? Thanks in advance for any suggestions.
Many thanks.
Try to avoid * to select fields, it will be clearer to group your datas (even if mysql is quite permissive with groupings).
When you have an aggregate function (like COUNT, SUM), the other "non aggregated" requested fields should be in a GROUP BY clause.
Mysql don't force you to GROUP BY all the fields, but... I think it's quite a good habit to be "as ANSI as possible" (usefull when you use another DBMS)
SELECT u.id, u.username, d.firstname, d.lastname, count(*) as numberfollowers
FROM user u
LEFT JOIN details d on u.id = d.userid
LEFT JOIN follow f on u.id = f.followedid
GROUP BY u.id, u.username, d.firstname, d.lastname --or just GROUP BY u.id with Mysql
ORDER BY count(*) desc
COUNT being an aggregate function, when selected with other columns, requires you to group your results by those other columns in the select list.
You should rewrite your query with columns that you want to select from users and details and group by those columns.