Mysql-> Group after rand() - mysql

I have the following table in Mysql
Name Age Group
abel 7 A
joe 6 A
Rick 7 A
Diana 5 B
Billy 6 B
Pat 5 B
I want to randomize the rows, but they should still remain grouped by the Group column.
For exmaple i want my result to look something like this.
Name Age Group
joe 6 A
abel 7 A
Rick 7 A
Billy 6 B
Pat 5 B
Diana 5 B
What query should i use to get this result? The entire table should be randomised and then grouped by "Group" column.

What you describe in your question as GROUPing is more correctly described as sorting. This is a particular issue when talking about SQL databases where "GROUP" means something quite different and determines the scope of aggregation operations.
Indeed "group" is a reserved word in SQL, so although mysql and some other SQL databases can work around this, it is a poor choice as an attribute name.
SELECT *
FROM yourtable
ORDER BY `group`
Using random values also has a lot of semantic confusion. A truly random number would have a different value every time it is retrieved - which would make any sorting impossible (and databases do a lot of sorting which is normally invisible to the user). As long as the implementation uses a finite time algorithm such as quicksort that shouldn't be a problem - but a bubble sort would never finish, and a merge sort could get very confused.
There are also degrees of randomness. There are different algorithms for generating random numbers. For encryption it's critical than the random numbers be evenly distributed and completely unpredictable - often these will use hardware events (sometimes even dedicated hardware) but I don't expect you would need that. But do you want the ordering to be repeatable across invocations?
SELECT *
FROM yourtable
ORDER BY `group`, RAND()
...will give different results each time.
OTOH
SELECT
FROM yourtable
ORDER BY `group`, MD5(CONCAT(age, name, `group`))
...would give the results always sorted in the same order. While
SELECT
FROM yourtable
ORDER BY `group`, MD5(CONCAT(DATE(), age, name, `group`))
...will give different results on different days.

DROP TABLE my_table;
CREATE TABLE my_table
(name VARCHAR(12) NOT NULL
,age INT NOT NULL
,my_group CHAR(1) NOT NULL
);
INSERT INTO my_table VALUES
('Abel',7,'A'),
('Joe',6,'A'),
('Rick',7,'A'),
('Diana',5,'B'),
('Billy',6,'B'),
('Pat',5,'B');
SELECT * FROM my_table ORDER BY my_group,RAND();
+-------+-----+----------+
| name | age | my_group |
+-------+-----+----------+
| Joe | 6 | A |
| Abel | 7 | A |
| Rick | 7 | A |
| Pat | 5 | B |
| Diana | 5 | B |
| Billy | 6 | B |
+-------+-----+----------+

Do the random first then sort by column group.
select Name, Age, Group
from (
select *
FROM yourtable
order by RAND()
) t
order by Group

Try this:
SELECT * FROM table order by Group,rand()

Related

MYSQL - Query to extract all columns from the top N distinct elements

I have design an event where you register multiple fishes and I wanted a query to extract the top 3 heaviest fishes from different people. In case of tie, it should be decided by a third parameter: who registered it first. I've tested several ways I found here on stack overflow but none of them worked the way I needed.
My schema is the following:
id | playerid | playername | itemid | weight | date | received | isCurrent
Where:
id = PK, AUTO_INCREMENT - it's basically an index
playerid = the unique code of the person who registered the fish
playername = name of the person who registered the fish
itemid = the code of the fish
weight = the weight of the fish
date = pre-defined as CURRENT_TIMESTAMP, the exact time the fish was registered
received = pre-defined as 0, it really don't matter for this analysis
isCurrent = pre-defined as 1, basically every time this event runs it updates this field to 0, meaning the registers don't belong to the current version of the event.
Here you can see the data I'm testing with
my problem is: How to avoid counting the same playerid for this rank more than once?
Query 1:
SELECT `playerid`, `playername`, `itemid`, `weight`
FROM `event_fishing`
WHERE `isCurrent` = 1 AND `weight` IN (
SELECT * FROM
(SELECT MAX(`weight`) as `fishWeight`
FROM `event_fishing`
WHERE `isCurrent` = 1
GROUP BY `playerid`
LIMIT 3) as t)
ORDER BY `weight` DESC, `date` ASC
LIMIT 3
Query 2:
SELECT * FROM `event_fishing`
INNER JOIN
(SELECT playerid, MAX(`weight`) as `fishWeight`
FROM `event_fishing`
WHERE `isCurrent` = 1
GROUP BY `playerid`
LIMIT 3) as t
ON t.playerid = `event_fishing`.playerid AND t.fishWeight = `event_fishing`.weight
WHERE `isCurrent` = 1
ORDER BY weight DESC, date ASC
LIMIT 3
Keep in mind that I must return at least the fields: playerid, playername, itemid, weight, that the version of the event must be the actual (isCurrent = 1), one playerid per line with the heaviest weight he registered for this version of the event and the date is registered.
Expected output for the data I've sent:
id |playerid|playername|itemid|weight| date |received| isCurrent
7 | 3734 |Mago Xxx | 7963 | 1850 | 2018-07-26 00:17:41 | 0 | 1
14 | 228 |Night Wolf| 7963 | 1750 | 2018-07-26 19:45:49 | 0 | 1
8 | 3646 |Test Spell| 7159 | 1690 | 2018-07-26 01:16:51 | 0 | 1
Output I'm getting (with both queries):
playerid|playername|itemid|weight
3734 |Mago Xxx | 7963 | 1850
228 |Night Wolf| 7963 | 1750
228 |Night Wolf| 7963 | 1750
Thank you for the attention.
EDIT: I've followed How can I SELECT rows with MAX(Column value), DISTINCT by another column in SQL? since my query is very similar to the accepted answer, in the comments I've found something that at a first glance seem to have solved my problem but I've found a case where the accepted answer fail. Check http://sqlfiddle.com/#!9/72aeef/1
If you take a look at data you'll notice that the id 14 was the first input of 1750 and therefore should be second place, but the MAX(id) returns the last input of the same playerid and therefore give us a wrong result.
Despite the problems seems alike, mine has a greater complexity and therefore the queries that were suggested doesn't work
EDIT 2:
I've managed to solve my problem with the following query:
http://sqlfiddle.com/#!9/d711c7/6
But I'll leave this question open because of two things:
1- I don't know if there's a case where this query might fail
2- Despite we limit a lot the first query, I still think this can be more optimized, so I'll leave it open to any one that might know a better way to solve the issue.

mySQL Multi Join from 2 Statements

I've found many similar questions but have not been able to understand / apply the answers; and I don't really know what to search for...
I have 2 tables (docs and words) which have a many to many relationship. I am trying to generate a list of the top 5 most frequently used words that DO NOT appear in a specified docs.
To this end I have 2 mySQL queries, each of which takes me part way to achieving my goal:
Query #1 - returns words sorted by frequency of use, falls short because it also returns ALL words (SQLFiddle.com)
SELECT `words_idwords` as wdID, COUNT(*) as freq
FROM docs_has_words
GROUP BY `words_idwords`
ORDER BY freq DESC, wdID ASC
Query #2 - returns words that are missing from specified document, falls short because it does not sort by frequency of use (SQLFiddle.com)
SELECT wordscol as wrd, idwords as wID
FROM `words` where NOT `idwords`
IN (SELECT `words_idwords` FROM `docs_has_words` WHERE `docs_iddocs` = 1)
But what I want the output to look like is:
idwords | wordscol | freq
-------------------------
| 8 | Dog | 3 |
| 3 | Ape | 2 |
| 4 | Bear | 1 |
| 6 | Cat | 1 |
| 7 | Cheetah | 1 |
| 5 | Beaver | 0 |
Note: `Dolphin`, one of the most frequently used words, is NOT in the
list because it is already in the document iddocs = 1
Note: `Beaver`, is a "never used word" BUT is in the list because it is
in the main word list
And the question is: how can I combine these to queries, or otherwise, get my desired output?
Basic requirements:
- 3 column output
- results sorted by frequency of use, even if use is zero
Updates:
In light of some comments, the approach that I was thinking of when I came up with the 2 queries was:
Step 1 - find all the words that are in the main word list but not used in document 1
Step 2 - rank words from Step 1 according to how many documents use them
Once I had the 2 queries I thought it would be easy to combine them with a where clause, but I just can't get it working.
A hack solution could be based on adding a dummy document that contains all the words and then subtract 1 from freq (but I'm not that much of a hack!).
I see now what the problem is. I was mislead by your statement regarding the results of the 1st query (emphasis is mine):
returns words sorted by frequency of use, falls short because it also returns ALL words
This query does not return all words, it only returns all used words.
So, you need to left join the words table on docs_has_words table to get all words and eliminate the words that are associated with doc 1:
SELECT w.idwords as wdID, w.wordscol, COUNT(d.words_idwords) as freq
FROM words w
LEFT JOIN `docs_has_words` d on w.idwords=d.words_idwords
WHERE w.idwords not in (SELECT `words_idwords` FROM `docs_has_words` WHERE `docs_iddocs` = 1)
GROUP BY w.idwords
ORDER BY freq DESC, wdID ASC;
See sqlfiddle
I think #Shadow has it right in his comment, you just need to add the where clause like this: sqlFiddle
SELECT
`words_idwords` as wdID,
COUNT(*) as freq
FROM docs_has_words
WHERE NOT `words_idwords` IN (SELECT `words_idwords` FROM `docs_has_words` WHERE `docs_iddocs` = 1)
GROUP BY `words_idwords`
ORDER BY freq DESC, wdID ASC
Does this produce the output you need?

Fewest grouped by distinct - SQL

Ok, I think the answer of this is somewhere but I can't find it...
(and even my title is bad)
To be short, I want to get the fewest number of group I can make from a part of an association table
1st, Keep in mind this is already a result of a 5 table (+1k line) join with filter and grouping, that I'll have to run many time on a prod server as powerful as a banana...
2nd, This is a fake case that picture you my problem
After some Querying, I've got this data result :
+--------------------+
|id_course|id_teacher|
+--------------------+
| 6 | 1 |
| 6 | 4 |
| 6 | 14 |
| 33 | 1 |
| 33 | 4 |
| 34 | 1 |
| 34 | 4 |
| 34 | 10 |
+--------------------+
As you can see, I've got 3 courses, witch are teach by up to 3 teacher. I need to attend at one of every course, but I want as few different teacher as possible (I'm shy...).
My first query
Should answer : what is the smallest number of teacher I need to cover every unique course ?
With this data, it's a 1, cause Teacher 1 or Teacher 4 make courses for these 3 one.
Second query
Now that I've already get these courses, I want to go to two other courses, the 32 and the 50, with this schedule :
+--------------------+
|id_course|id_teacher|
+--------------------+
| 32 | 1 |
| 32 | 12 |
| 50 | 12 |
+--------------------+
My question is : For id_course N, will I have to get one more teacher ?
I want to check course by course, so "check for course 32", no need to check many at the same time
The best way I think is to count an inner join with a list of teacher of same fewest rank from the first query, so with our data we got only two : Teacher(1, 4).
For the Course 32, Teacher2 don't do this one, but as the Teacher1 do Courses(6, 33, 34, 32) I don't have to get another teacher.
For the Course 50, the only teacher to do it is the Teacher12, so I'll not find a match in my choice of teacher, and I'll have to get one more (so two in total with these data)
Here is a base [SQLFiddle
Best regards, Blag
You want to get a distinct count of ID_Teachers with the least count then... get a distinct count and limit the results to 1 record.
So perhaps something like...
SELECT count(Distinct ID_Teacher), Group_concat(ID_Teacher) as TeachersIDs
FROM Table
WHERE ID_Course in ('Your List')
ORDER BY count(Distinct ID_Teacher) ASC Limit 1
However this will randomly select if a tie exists... so do you want to provide the option to select which group of teachers and classes should ties exist? Meaning there are multiple paths to fulfill all classes involving the same number of teachers... For example teachers A, B and A, C fulfill all required classes.... should both records return in the result or is 1 sufficient?
So I've finally found a way to do what I want !
For the first query, as my underlying real need was "is there a single teacher to do everything", I've lower a bit my expectation and go for this one (58 lines on my true case u_u") :
SELECT
(
SELECT count(s.id_teacher) nb
FROM t AS m
INNER JOIN t AS s
ON m.id_teacher = s.id_teacher
GROUP BY m.id_course, m.id_teacher
ORDER BY nb DESC
LIMIT 1
) AS nbMaxBySingleTeacher,
(
SELECT COUNT(DISTINCT id_course) nb
FROM t
) AS nbTotalCourseToDo
[SQLFiddle
And I get back two value that answer my question "is one teacher enough ?"
+--------------------------------------+
|nbMaxBySingleTeacher|nbTotalCourseToDo|
+--------------------------------------+
| 4 | 5 |
+--------------------------------------+
The 2nd query use the schedule of new course, and take the id of one I want to check. It should tell me if I need to get one more teacher, or if it's ok with my actual(s) one.
SELECT COUNT(*) nb
FROM (
SELECT
z.id_teacher
FROM z
WHERE
z.id_course = 50
) t1
WHERE
FIND_IN_SET(t1.id_teacher, (
SELECT GROUP_CONCAT(t2.id_teacher) lst
FROM (
SELECT DISTINCT COUNT(s.id_teacher) nb, m.id_teacher
FROM t AS m
INNER JOIN t AS s
ON m.id_teacher = s.id_teacher
GROUP BY m.id_course, m.id_teacher
ORDER BY nb DESC
) t2
GROUP BY t2.nb
ORDER BY nb DESC
LIMIT 1
));
[SQLFiddle
This tell me the number of teacher that are able to teach the courses I already have AND the new one I want. So if it's over zero, then I don't need a new teacher :
+--+
|nb|
+--+
|1 |
+--+

MySQL counting number of max groups

I asked a similar question earlier today, but I've run into another issue that I need assistance with.
I have a logging system that scans a server and catalogs every user that's online at that given moment. Here is how my table looks like:
-----------------
| ab_logs |
-----------------
| id |
| scan_id |
| found_user |
-----------------
id is an autoincrementing primary key. Has no real value other than that.
scan_id is an integer that is incremented after each successful scan of all users. It so I can separate results from different scans.
found_user. Stores which user was found online during the scan.
The above will generate a table that could look like this:
id | scan_id | found_user
----------------------------
1 | 1 | Nick
2 | 2 | Nick
3 | 2 | John
4 | 3 | John
So on the first scan the system found only Nick online. On the 2nd it found both Nick and John. On the 3rd only John was still online.
My problem is that I want to get the total amount of unique users connected to the server at the time of each scan. In other words, I want the aggregate number of users that have connected at each scan. Think counter.
From the example above, the result I want from the sql is:
1
2
2
EDIT:
This is what I have tried so far, but it's wrong:
SELECT COUNT(DISTINCT(found_user)) FROM ab_logs WHERE DATE(timestamp) = CURDATE() GROUP BY scan_id
What I tried returns this:
1
2
1
The code below should give you the results you are looking for
select s.scan_id, count(*) from
(select distinct
t.scan_id
,t1.found_user
from
tblScans t
inner join tblScans t1 on t.scan_id >= t1.scan_id) s
group by
s.scan_id;
Here is sqlFiddle
It assumes the names are unique and includes current and every previous scans in the count
Try with group by clause:
SELECT scan_id, count(*)
FROM mytable
GROUP BY scan_id

What can an aggregate function do in the ORDER BY clause?

Lets say I have a plant table:
id fruit
1 banana
2 apple
3 orange
I can do these
SELECT * FROM plant ORDER BY id;
SELECT * FROM plant ORDER BY fruit DESC;
which does the obvious thing.
But I was bitten by this, what does this do?
SELECT * FROM plant ORDER BY SUM(id);
SELECT * FROM plant ORDER BY COUNT(fruit);
SELECT * FROM plant ORDER BY COUNT(*);
SELECT * FROM plant ORDER BY SUM(1) DESC;
All these return just the first row (which is with id = 1).
What's happening underhood?
What are the scenarios where aggregate function will come in handy in ORDER BY?
Your results are more clear if you actually select the aggregate values instead of columns from the table:
SELECT SUM(id) FROM plant ORDER BY SUM(id)
This will return the sum of all id's. This is of course a useless example because the aggregation will always create only one row, hence no need for ordering. The reason you get a row qith columns in your query is because MySQL picks one row, not at random but not deterministic either. It just so happens that it is the first column in the table in your case, but others may get another row depending on storage engine, primary keys and so on. Aggregation only in the ORDER BY clause is thus not very useful.
What you usually want to do is grouping by a certain field and then order the result set in some way:
SELECT fruit, COUNT(*)
FROM plant
GROUP BY fruit
ORDER BY COUNT(*)
Now that's a more interesting query! This will give you one row for each fruit together with the total count for that fruit. Try adding some more apples and the ordering will actually start making sense:
Complete table:
+----+--------+
| id | fruit |
+----+--------+
| 1 | banana |
| 2 | apple |
| 3 | orange |
| 4 | apple |
| 5 | apple |
| 6 | banana |
+----+--------+
The query above:
+--------+----------+
| fruit | COUNT(*) |
+--------+----------+
| orange | 1 |
| banana | 2 |
| apple | 3 |
+--------+----------+
All these queries will all give you a syntax error on any SQL platform that complies with SQL standards.
SELECT * FROM plant ORDER BY SUM(id);
SELECT * FROM plant ORDER BY COUNT(fruit);
SELECT * FROM plant ORDER BY COUNT(*);
SELECT * FROM plant ORDER BY SUM(1) DESC;
On PostgreSQL, for example, all those queries will raise the same error.
ERROR: column "plant.id" must appear in the GROUP BY clause or be
used in an aggregate function
That means you're using a domain aggregate function without using GROUP BY. SQL Server and Oracle return similar error messages.
MySQL's GROUP BY is known to be broken in several respects, at least as far as standard behavior is concerned. But the queries you posted were a new broken behavior to me, so +1 for that.
Instead of trying to understand what it's doing under the hood, you're probably better off learning to write standard GROUP BY queries. MySQL will process standard GROUP BY statements correctly, as far as I know.
Earlier versions of MySQL docs warned you about GROUP BY and hidden columns. (I don't have a reference, but this text is cited all over the place.)
Do not use this feature if the columns you omit from the GROUP BY part
are not constant in the group. The server is free to return any value
from the group, so the results are indeterminate unless all values are
the same.
More recent versions are a little different.
You can use this feature to get better performance by avoiding
unnecessary column sorting and grouping. However, this is useful
primarily when all values in each nonaggregated column not named in
the GROUP BY are the same for each group. The server is free to choose
any value from each group, so unless they are the same, the values
chosen are indeterminate.
Personally, I don't consider indeterminate a feature in SQL.
When you use an aggregate like that, the query gets an implicit group by where the entire result is a single group.
Using an aggregate in order by is only useful if you also have a group by, so that you can have more than one row in the result.