As part of a pruning process, I am trying to delete data from a table which is not used anymore. There are search jobs (table Search) which have results (table Res), and people receive the search results (table Abo) via email. For every Abonnement, the last result sent via email is saved in the column "last_result".
What I want to do: For every search job, I want to delete all results which have been sent to the users.
This is a simplified variant of the structure:
http://sqlfiddle.com/#!9/5ef8a3
Abo
---
user search last_result
A 1 100
B 1 200
C 2 50
D 3 100
Res
---
id search
50 1 <---- this should be deleted
300 1
51 2
Search
------
1
2
3
What I tried to do is something like this:
DELETE r
FROM Res r
JOIN Abo a ON r.search = a.search
WHERE
r.id < min(a.last_res)
For every "search" value, I want to delete all results ("Res") which are below the "last_res" field in the "Abo" table.
You need to group MIN(last_res) by search, so the way to do this is to write a subquery that calculates this, then join with that subquery.
DELETE r
FROM Res AS r
JOIN (SELECT search, MIN(last_res) AS min_res
FROM Abo
GROUP BY search) AS a
ON r.search = a.search AND r.id < a.min_res
Related
I have the following MySQL 8 tables:
[submissions]
===
id
submission_type
name
[reject_reasons]
===
id
name
[submission_reject_reasons] -- crosswalk joining the first 2 tables
===
id
submission_id
reject_reason_id
In my application, users can submit submissions, and other users can request changes to those submissions. When they request these rejections, 1+ entries get saved to the submission_reject_reasons table (which stores the ID of the submission for which rejections are requested, as well as the ID of the reason for why the rejection is being made). So a typical entry in the table might look like:
id submission_id reject_reason_id
==============================================
45 384 294
Where submission_id = 384 is the "Fizz Buzz" submission and reject_reason_id = 294 is the "Missing Required Field" reason.
I currently have a query that fetches all the reject_reasons out of the DB:
SELECT * FROM reject_reasons
I now want to modify this query to sort the results based on their usage frequency. Meaning the query might currently return:
294 | Missing Required Field
14 | Malformed Entry
1885 | Makes No Sense
etc. But lets say there are 5 entries in the submission_reject_reasons table where 294 (Missing Required Field) is the reject_reason_id, and say there are 15 enries where 1885 (Makes No Sense) is present, and 120 entries where 14 (Malformed Entry) are present. I need a query that returns all reject_reasons sorted by their count in the submission_reject_reasons (SRR) table, descending, so that the most frequently used appear earlier in the sort. Hence the result set would be:
14 | Malformed Entry --> because there are 120 instances of this in the SRR table
1885 | Makes No Sense --> because there are 15 instances in the SRR
294 | Missing Required Field --> because there are only 5 instances in the SRR
Furthermore, I need a ranking from most-used to least-used. If a reason doesn't exist in the SRR table it should have a default "count" of zero (0) but should still come back in the query. If 2+ reason counts are tied, then I don't care how they are sorted. Any ideas here? I need the final result set to only contain the rr.id and rr.name field/values.
My best attempt is not getting me anywhere:
SELECT rr.id, rr.name
FROM reject_reasons AS rr
LEFT JOIN submission_reject_reasons AS srr on rr.id = srr.reject_reason_id
GROUP BY rr.id
ORDER BY COUNT(*) DESC
Can anyone help me over the finish line here? Can anyone spot where I'm goin awry? Thanks in advance!
You should be grouping by the reject reason ID. COUNT(*) is what you want to count in each group.
SELECT rr.id, rr.name
FROM reject_reasons AS rr
JOIN submission_reject_reasons AS srr on rr.id = srr.reject_reason_id
GROUP BY rr.id
ORDER BY COUNT(*) DESC
There's no need for any EXISTS check, since the INNER JOIN won't return any reject reasons that don't exist in submission_reject_reasons.
I have two tables in my database
Table A with columns user_id, free_data, used_data
Table B with columns donor_id, receptor_id, share_data
Basically, a user (lets call x) has some data in his account which is represented by his entry in table A. The data is stored in free_data column. He can donate data to any other user (lets call y), which will show up as an entry in Table B. The same amount of data gets deducted from the user x free_data column.
While entry in Table B gets created, an entry in Table A for user y is also created with free_data value equal to share_data. Now user y can give away data to user z & the process continues.
Each user keep using their data & the entry used_data in Table A keeps on adding up to indicate how much data each user has used.
This is like a tree structure where there is a an entry with all the data (root node) who eventually gives data to others who in-turn give data to other nodes.
Now I would like to write an sql query such that, given a node x (id of entry in Table A), I should be able to sum up total data x has given & who all are beneficiaries at multiple level, all of their used_data need to be collated & showed against x.
Basically, I want to collate
Overall data x has donated.
How much of the donated data from x has been used up.
While the implementation is more graph-like, I am more interested to know if we assume it to be a tree below node x & can come up with a single sql query to be able to get the data I need.
Example
Table A
user_id, free_data, used_data
1 50 10
2 30 20
3 20 20
Table B
donor_id, receptor_id, share_data
1 2 30
1 3 20
Total data donated by 1 - 30 + 20 = 50
Total donated data used - 20 + 20 = 40
This is just one level where 1 donated to 2 & 3. 2 in turn could donated to 4 & all that data needed to be collated in a bubbled up fashion for calculating the overall donated data usage.
Yes its possible using a nested set model. There's a book by Joe Celko that describes but if you want to get straight into it there's an article that talks about it. Both the collated data that you need can be retrieved by a single select statement like this:
SELECT * FROM TableB where left > some_value1 and right < some_value2
In the above example to get all the child nodes of "Portable Electronics" the query will be:
SELECT * FROM Electronics WHERE `left` > 10 and `right` < 19
The article describes how the left and right columns should be initialised.
If I understand the problem correctly, the following should give you the desired results:
SELECT B.donor_id AS donor_id, SUM(A.used_data) AS total_used_data FROM A
INNER JOIN B ON A.user_id = B.receptor_id GROUP BY B.donor_id;
Hope this will solve your problem now.
Try below query(note that you will have to pass userid at 2 places):
SELECT SUM(share_data) as total_donated, sum(used_data) as total_used FROM tablea
LEFT JOIN tableB
ON tableA.user_id = tableB.donor_id
WHERE user_id IN (select receptor_id as id
from (select * from tableb
order by donor_id, receptor_id) u_sorted,
(select #pv := '1') initialisation
where find_in_set(donor_id, #pv) > 0
and #pv := concat(#pv, ',', receptor_id)) OR user_id = 1;
Please help by writing a SQL-script that will collate data.
A key difficulty - need to create an additional column on which sorting will take place.
I tried to describe the situation as detailed as possible.
Let's get started. There is a table of the following form:
We will receive a user ID and return data, only those who do not have he, but there are others.
Next step: sort by artificially created column.
Next, I'll step by step.
So what do I mean by artificial column:
This column will contain the difference between the estimates. So to get it - you need to first perform a number of actions:
According to the information which is like set the user and at other user to calculate the difference in assessment, and get an average score.
The following two pictures show the same data and then the calculation itself, it seems to me - it's pretty simple.
Calculation of this column is as follows:
User with 2nd id:
1: 5 - 1 = 4;
2: 2 - 9 = -7;
3: next data what is in user 1 - absent in user 2, and we ease pass it;
User with 3rd id:
1: 3 - 1 = 2;
2: the next data's is absent in user with 3rt id;
3: 8 – 9 = -1;
4: 6 – 2 = 4;
5: passed;
End in the end:
User_2 will have new mark = -1.5
User_3 will have new mark = 1.66666
And in the end I need to return the table:
But that's not all. Often, the data will be duplicated and I'd like to get average results from the data obtained. Please look at the following example:
And this is the end. I really need your help, experts. I teach sql code myself, but it is very difficult for me.
Had the idea of making the script as follows:
SELECT d.data, (d.mark + myCount(d.user, 1)) newOrder
FROM info d
WHERE -- data from user_1 NOT equal data from other users
ORDER BY newOrder;
But the script will execute a lot of time, because it uses its own function that could do with a query to each user, and not to record. I hope someone will be able to cope with this task.
Following your steps:
First, we need to isolate the data from the selected user (let's assume it's 1):
CREATE TEMP TABLE sel_user AS
SELECT data, mark FROM info d WHERE user = 1;
Now, we calculate the mark for every other user (again, the selected user is 1):
SELECT d.user user, d.mark - s.mark mark
FROM info d JOIN sel_user s USING (data)
WHERE d.user <> 1;
Result:
user mark
---------- ----------
2 4
2 -7
3 2
3 -1
3 4
We can query just the average:
SELECT d.user user, AVG(d.mark - s.mark) mark
FROM info d JOIN sel_user s USING (data)
WHERE d.user <> 1 GROUP BY user;
user mark
---------- ----------
2 -1.5
3 1.66666666
But you still want to do calculations with the marks that do not correspond to user 1:
SELECT d.user user, mark FROM info d
WHERE d.user <> 1 AND d.data NOT IN (SELECT data FROM sel_user);
user mark
---------- ----------
2 4
3 3
3 10
Specifically, you want to add the previously calculated average to each row:
SELECT d.user user, d.data, d.mark + d2.mark AS neworder FROM info d JOIN (
SELECT d.user user, AVG(d.mark - s.mark) mark
FROM info d JOIN sel_user s USING (data)
WHERE d.user <> 1 GROUP BY user
) d2 USING (user)
WHERE d.data NOT IN (SELECT data FROM sel_user)
ORDER BY neworder DESC;
user data neworder
---------- ---------- ----------------
3 6 11.6666666666667
3 3 4.66666666666667
2 5 2.5
And your last request is to get the average for each data:
SELECT data, AVG(neworder) final FROM (
SELECT d.user user, d.data, d.mark + d2.mark AS neworder FROM info d JOIN (
SELECT d.user user, AVG(d.mark - s.mark) mark
FROM info d JOIN sel_user s USING (data)
WHERE d.user <> 1 GROUP BY user
) d2 USING (user)
WHERE d.data NOT IN (SELECT data FROM sel_user)
)
GROUP BY data
ORDER BY final DESC;
data final
---------- ----------------
6 11.6666666666667
3 4.66666666666667
5 2.5
I have a table of surveys which contains (amongst others) the following columns
survey_id - unique id
user_id - the id of the person the survey relates to
created - datetime
ip_address - of the submission
ip_count - the number of duplicates
Due to a large record set, its impractical to run this query on the fly, so trying to create an update statement which will periodically store a "cached" result in ip_count.
The purpose of the ip_count is to show the number of duplicate ip_address survey submissions have been recieved for the same user_id with a 12 month period (+/- 6months of created date).
Using the following dataset, this is the expected result.
survey_id user_id created ip_address ip_count #counted duplicates survey_id
1 1 01-Jan-12 123.132.123 1 # 2
2 1 01-Apr-12 123.132.123 2 # 1, 3
3 2 01-Jul-12 123.132.123 0 #
4 1 01-Aug-12 123.132.123 3 # 2, 6
6 1 01-Dec-12 123.132.123 1 # 4
This is the closest solution I have come up with so far but this query is failing to take into account the date restriction and struggling to come up with an alternative method.
UPDATE surveys
JOIN(
SELECT ip_address, created, user_id, COUNT(*) AS total
FROM surveys
WHERE surveys.state IN (1, 3) # survey is marked as completed and confirmed
GROUP BY ip_address, user_id
) AS ipCount
ON (
ipCount.ip_address = surveys.ip_address
AND ipCount.user_id = surveys.user_id
AND ipCount.created BETWEEN (surveys.created - INTERVAL 6 MONTH) AND (surveys.created + INTERVAL 6 MONTH)
)
SET surveys.ip_count = ipCount.total - 1 # minus 1 as this query will match on its own id.
WHERE surveys.ip_address IS NOT NULL # ignore surveys where we have no ip_address
Thank you for you help in advance :)
A few (very) minor tweaks to what is shown above. Thank you again!
UPDATE surveys AS s
INNER JOIN (
SELECT x, count(*) c
FROM (
SELECT s1.id AS x, s2.id AS y
FROM surveys AS s1, surveys AS s2
WHERE s1.state IN (1, 3) # completed and verified
AND s1.id != s2.id # dont self join
AND s1.ip_address != "" AND s1.ip_address IS NOT NULL # not interested in blank entries
AND s1.ip_address = s2.ip_address
AND (s2.created BETWEEN (s1.created - INTERVAL 6 MONTH) AND (s1.created + INTERVAL 6 MONTH))
AND s1.user_id = s2.user_id # where completed for the same user
) AS ipCount
GROUP BY x
) n on s.id = n.x
SET s.ip_count = n.c
I don't have your table with me, so its hard for me to form correct sql that definitely works, but I can take a shot at this, and hopefully be able to help you..
First I would need to take the cartesian product of surveys against itself and filter out the rows I don't want
select s1.survey_id x, s2.survey_id y from surveys s1, surveys s2 where s1.survey_id != s2.survey_id and s1.ip_address = s2.ip_address and (s1.created and s2.created fall 6 months within each other)
The output of this should contain every pair of surveys that match (according to your rules) TWICE (once for each id in the 1st position and once for it to be in the 2nd position)
Then we can do a GROUP BY on the output of this to get a table that basically gives me the correct ip_count for each survey_id
(select x, count(*) c from (select s1.survey_id x, s2.survey_id y from surveys s1, surveys s2 where s1.survey_id != s2.survey_id and s1.ip_address = s2.ip_address and (s1.created and s2.created fall 6 months within each other)) group by x)
So now we have a table mapping each survey_id to its correct ip_count. To update the original table, we need to join that against this and copy the values over
So that should look something like
UPDATE surveys SET s.ip_count = n.c from surveys s inner join (ABOVE QUERY) n on s.survey_id = n.x
There is some pseudo code in there, but I think the general idea should work
I have never had to update a table based on the output of another query myself before.. Tried to guess the right syntax for doing this from this question - How do I UPDATE from a SELECT in SQL Server?
Also if I needed to do something like this for my own work, I wouldn't attempt to do it in a single query.. This would be a pain to maintain and might have memory/performance issues. It would be best have a script traverse the table row by row, update on a single row in a transaction before moving on to the next row. Much slower, but simpler to understand and possibly lighter on your database.
I have a watchlist system that I've coded, in the overview of the users' watchlist, they would see a list of records, however the list shows duplicates when in the database it only shows the exact, correct number.
I've tried GROUP BY watch.watch_id, GROUP BY rec.record_id, none of any types of group I've tried seems to remove duplicates. I'm not sure what I'm doing wrong.
SELECT watch.watch_date,
rec.street_number,
rec.street_name,
rec.city,
rec.state,
rec.country,
usr.username
FROM
(
watchlist watch
LEFT OUTER JOIN records rec ON rec.record_id = watch.record_id
LEFT OUTER JOIN members usr ON rec.user_id = usr.user_id
)
WHERE watch.user_id = 1
GROUP BY watch.watch_id
LIMIT 0, 25
The watchlist table looks like this:
+----------+---------+-----------+------------+
| watch_id | user_id | record_id | watch_date |
+----------+---------+-----------+------------+
| 13 | 1 | 22 | 1314038274 |
| 14 | 1 | 25 | 1314038995 |
+----------+---------+-----------+------------+
GROUP BY does not "remove duplicates". GROUP BY allows for aggregation. If all you want is to combine duplicated rows, use SELECT DISTINCT.
If you need to combine rows that are duplicate in some columns, use GROUP BY but you need to to specify what to do with the other columns. You can either omit them (by not listing them in the SELECT clause) or aggregate them (using functions like SUM, MIN, and AVG). For example:
SELECT watch.watch_id, COUNT(rec.street_number), MAX(watch.watch_date)
... GROUP by watch.watch_id
EDIT
The OP asked for some clarification.
Consider the "view" -- all the data put together by the FROMs and JOINs and the WHEREs -- call that V. There are two things you might want to do.
First, you might have completely duplicate rows that you wish to combine:
a b c
- - -
1 2 3
1 2 3
3 4 5
Then simply use DISTINCT
SELECT DISTINCT * FROM V;
a b c
- - -
1 2 3
3 4 5
Or, you might have partially duplicate rows that you wish to combine:
a b c
- - -
1 2 3
1 2 6
3 4 5
Those first two rows are "the same" in some sense, but clearly different in another sense (in particular, they would not be combined by SELECT DISTINCT). You have to decide how to combine them. You could discard column c as unimportant:
SELECT DISTINCT a,b FROM V;
a b
- -
1 2
3 4
Or you could perform some kind of aggregation on them. You could add them up:
SELECT a,b, SUM(c) "tot" FROM V GROUP BY a,b;
a b tot
- - ---
1 2 9
3 4 5
You could add pick the smallest value:
SELECT a,b, MIN(c) "first" FROM V GROUP BY a,b;
a b first
- - -----
1 2 3
3 4 5
Or you could take the mean (AVG), the standard deviation (STD), and any of a bunch of other functions that take a bunch of values for c and combine them into one.
What isn't really an option is just doing nothing. If you just list the ungrouped columns, the DBMS will either throw an error (Oracle does that -- the right choice, imo) or pick one value more or less at random (MySQL). But as Dr. Peart said, "When you choose not to decide, you still have made a choice."
While SELECT DISTINCT may indeed work in your case, it's important to note why what you have is not working.
You're selecting fields that are outside of the GROUP BY. Although MySQL allows this, the exact rows it returns for the non-GROUP BY fields is undefined.
If you wanted to do this with a GROUP BY try something more like the following:
SELECT watch.watch_date,
rec.street_number,
rec.street_name,
rec.city,
rec.state,
rec.country,
usr.username
FROM
(
watchlist watch
LEFT OUTER JOIN est8_records rec ON rec.record_id = watch.record_id
LEFT OUTER JOIN est8_members usr ON rec.user_id = usr.user_id
)
WHERE watch.watch_id IN (
SELECT watch_id FROM watch WHERE user_id = 1
GROUP BY watch.watch_id)
LIMIT 0, 25
I Would never recommend using SELECT DISTINCT, it's really slow on big datasets.
Try using things like EXISTS.
You are grouping by watch.watch_id and you have two results, which have different watch IDs, so naturally they would not be grouped.
Also, from the results displayed they have different records. That looks like a perfectly valid expected results. If you are trying to only select distinct values, then you don't want ot GROUP, but you want to select by distinct values.
SELECT DISTINCT()...
If you say your watchlist table is unique, then one (or both) of the other tables either (a) has duplicates, or (b) is not unique by the key you are using.
To suppress duplicates in your results, either use DISTINCT as #Laykes says, or try
GROUP BY watch.watch_date,
rec.street_number,
rec.street_name,
rec.city,
rec.state,
rec.country,
usr.username
It sort of sounds like you expect all 3 tables to be unique by their keys, though. If that is the case, you are simply masking some other problem with your SQL by trying to retrieve distinct values.