NOTE: I'm using SQLAlchemy but I've chosen to translate my idea into regular SQL so that more people can answer / read this question.
I am attempting to select all entries in a table which are not in a filter. I'm getting the filter by associating a user_id to a table_id. I currently have a working implementation using,
select * from cleared_table where user_id=user_id
and then I construct a second select statement saying,
select * from table where table_id != first_id and table_id != second_id ....
I'm looking for a way to condense this down into a single SQL statement rather then having to make two trips to the database.
Example cleared_table,
cleared_table
user_id | table_id
1 | 1
1 | 2
1 | 3
2 | 2
3 | 3
You are looking for a nested query.
select * from table where table_id not in (select table_id from cleared_table where user_id=user_id)
You can use NOT IN and a subquery:
select
*
from
table
where
table_id not in (select
table_id
from
cleared_table where user_id=user_id)
Related
I have a table T with data like this:
ID | Status
-----------------
1 | X
2 | Y
1 | Z
3 | P
4 | Q
3 | Z
I need to retrieve all the rows where the ID does not have status Z in that or any other rows.
So, for the above example table, I need my query to to return the following table:
ID | Status
-----------------
2 | Y
4 | Q
Rows with ID 1 and 3 were discarded because they had at least one row with the value Z as Status.
What's a good and efficient query for this?
Thanks in advance!
You can exclude the ids with the Z status using the subquery .
select * from datas
where id not in ( select id from datas
where status ='Z' );
Demo: https://www.db-fiddle.com/f/7yUJcuMJPncBBnrExKbzYz/2
As #IVO GELOV suggested you can use EXISTS;
select d.id,d.status
from datas d
where not exists ( select id from datas e
where status ='Z' and d.id=e.id);
Here the inner query will return 1,3 as the id and while checking the returned data the outer query will return 2,4 as the returned id with status
NOT In Query
select id,status from T
where id not in
(select id from T where status='Z');
**OR**
NOT Exists Query
select ext_alias.id,ext_alias.status from T ext_alias
where not exists
(select id from T inn_alias where inn_alias.status='Z' and inn_alias.Id=ext_alias.Id);
You can use any of the above queries but NOT Exists query is faster than Not In query.
I have multiple SELECT statements that all return the same columns but may return different resultsets. Is there any way to select all rows that are in all resultsets on database level?
E.g.
|---------------------|------------------|---------|
| ID | Name | Age |
|---------------------|------------------|---------|
| 1 | Paul | 50 |
| 2 | Peter | 40 |
| 3 | Frank | 20 |
| 4 | Pascal | 60 |
|---------------------|------------------|---------|
SELECT 1
SELECT name FROM table WHERE age > 40
Result: Paul, Pascal
SELECT 2
SELECT name FROM table where name like 'P%'
Result: Paul, Peter, Pascal
SELECT 3
SELECT name FROM table where id > 3
Result: Pascal
EDIT: This is a very simplified example of my problem. The statements can get very complex (joins over multiple tables), so a simple AND in the WHERE part is not the final solution.
The result should be Pascal. What I am looking for is something like a "reverse UNION".
Alternatively it would be possible to achieve that programatically (NodeJS), but I would like to avoid to iterate over all resultsets, because they might be quite huge.
Thanks in advance!
Is there any way to select all rows that are in all resultsets?
You seem to want and:
select name
from table
where age > 40 and name like 'P%' and id < 3
If using AND between the WHERE conditions is not possible, you could use multiple IN expressions on subqueries using your initial queries.
SELECT name
FROM table
WHERE id IN (SELECT id FROM table WHERE age > 40)
AND id IN (SELECT id FROM table where name like 'P%')
AND id IN (SELECT id FROM table where id < 3)
If you have different result sets and you want to see the intersection, you can use join:
select q1.id
from (<query 1>) q1 join
(<query 2>) q2
on q1.id = q2.id join
(<query 3>) q3
on q1.id = q3.id;
That said, I think GMB has the most concise answer to the question that you actually asked.
If your statements are complex, what you could do is to use a procedure where each of the statements put the matching id's into a temp table. Then select those rows where id's match the number of statements. This will also most likely be more efficient than one huge query with all complex statements combined into one.
create procedure sp_match_all()
begin
drop temporary table if exists match_tmp;
create temporary table match_tmp (
id int
);
insert into match_tmp
SELECT id FROM table WHERE age > 40;
insert into match_tmp
SELECT id FROM table where name like 'P%';
insert into match_tmp
SELECT id FROM table where id < 3;
select t.name
from table t
join (
select id
from match_tmp
group by id
having count(*)=3
) q on q.id=t.id;
drop temporary table match_tmp;
end
i have a large MySQL Database with more than 1 Million rows. How can i find the missing eid's?
+----+-----+
| id | eid |
+----+-----+
| 1 | 1 |
+----+-----+
| 2 | 2 |
+----+-----+
| 3 | 4 |
+----+-----+
I like to list all missing eid's, the 3 in this example. I've tried many things but everything what i do need to much time.
I hope someone can help me.
Thanks
You can use NOT EXISTS to find the required rows.
create table t(id integer, eid integer);
insert into t values(1,1);
insert into t values(2,2);
insert into t values(3,4);
SELECT id
FROM t a
WHERE NOT EXISTS
( SELECT 1
FROM t b
WHERE b.eid = a.id );
or use NOT IN:
SELECT ID
FROM t
WHERE ID NOT IN
(SELECT EID
FROM t);
produces:
| id |
|----|
| 3 |
Try the below query
SELECT ID FROM table WHERE ID NOT IN(SELECT EID FROM table );
Finding duplicate numbers is easy:
select id, count() from sequence
group by id
having count() > 1;
In this case there are no duplicates, since I’m not concentrating on that in this post (finding duplicates is straightforward enough that I hope you can see how it’s done). I had to scratch my head for a second to find missing numbers in the sequence, though. Here is my first shot at it:
select l.id + 1 as start
from sequence as l
left outer join sequence as r on l.id + 1 = r.id
where r.id is null;
The idea is to exclusion join against the same sequence, but shifted by one position. Any number with an adjacent number will join successfully, and the WHERE clause will eliminate successful matches, leaving the missing numbers. Here is the result:
https://www.xaprb.com/blog/2005/12/06/find-missing-numbers-in-a-sequence-with-sql/
if you want a lighter way to search millions of rows of data,
I was try for search in more than 23 millions rows with old CPU (12.6Gb data need about 1gb of free ram):
Affected rows: 0 Found rows: 346.764 Warnings: 0 Duration for 2 queries: 00:04:48.0 (+ 2,656 sec. network)
SET #idBefore=0, #st=0,#diffSt=0,#diffEnd=0;
SELECT res.idBefore `betweenID`, res.ID `andNextID`
, res.startEID, res.endEID
, res.diff `diffEID`
-- DON'T USE this missingEIDfor more than a thousand of rows
-- this is just for sample view
, GROUP_CONCAT(b.aNum) `missingEID`
FROM (
SELECT
#idBefore `idBefore`
, #idBefore:=(a.id) `ID`
, #diffSt:=(#st) `startEID`
, #diffEnd:=(a.eid) `endEID`
, #st:=a.eid `end`
, #diffEnd-#diffSt-1 `diff`
FROM eid a
ORDER BY a.ID
) res
-- DON'T USE this integers for more than a thousand of rows
-- this is just for sample view
CROSS JOIN (SELECT a.ID + (b.ID * 10) + (c.ID * 100) AS aNum FROM integers a, integers b, integers c) b
WHERE res.diff>0 AND b.aNum BETWEEN res.startEID+1 AND res.endEID-1
GROUP BY res.ID;
check out this http://sqlfiddle.com/#!9/33deb3/9
and this is for missing ID http://sqlfiddle.com/#!9/3ea00c/9
UPDATED
id | id_list
1 | 2,3,5,7
2 | 1,4,5,6
3 | 1,4,6,7
4 | 2,3,5,8
5 | 1,2,4,8
6 | 2,3,7,8
7 | 1,3,6,9
8 | 4,5,6,9
9 | 7,8
let's say I'm up to the content of id=1
I wanted to select all the rows where id is in id_list of id=1 PLUS the row where id=1
so the result would be
rows with id = 1,2,3,5,7
How to do this query guys?
You can also use a self join
Using IN()
select * from atable a
join atable b on (a.id = b.id )
where 1 IN (a.id_list) or b.id =1
Fiddle with IN()
Using FIND_IN_SET()
select * from atable a
join atable b on (a.id = b.id )
where FIND_IN_SET('1', a.id_list) or b.id =1
Fiddle with FIND_IN_SET()
Using UNION ALL
select * from atable
where id =1
UNION ALL
select * from atable
where 1 IN (id_list)
Fiddle with UNION ALL
Your database design is broken; id_list should be represented as a join table instead of as a string. However, to solve your immediate problem:
select * from table where id=1
or id_list like '1%'
or id_list like '%,1,%'
or id_list like '%,1'
Adjust as needed for PreparedStatement. You have to provide all three cases because if you just did
or id_list like '%1%'
and id_list contained the value 11, you'd get an incorrect match
Try this (see SQL-Fiddle):
SELECT * FROM tbl
WHERE id = 1
OR INSTR((SELECT id_list FROM tbl WHERE id = '1'), id) > 0;
Tested with MySQL 5.5.30
try this one
select *
from tbl
where id=1
or id_list like '%1%'
This appears to call for a union of two sets. The one set would be the single row whose id matches the specified value:
SELECT
id
FROM
atable
WHERE
id = #id
The other set would be the result of this self-join:
SELECT
item.id
FROM
atable AS item
INNER JOIN
atable AS list
ON
FIND_IN_SET(item.id, list.id_list)
WHERE
list.id = #id
That is, the row with the specified id is matched against every row in the table on the condition that the other row's id is found in the specified row's id_list.
You can try the complete query at SQL Fiddle.
Please note that lists aren't a very good design feature. In your situation, it might be better to use a many-to-many table as suggested by #Othman. Only I would probably use a slightly different query to get the desired output, because his doesn't include the specified row itself:
SELECT
id
FROM
manytomany
WHERE
id = #id
UNION
SELECT
linked_id
FROM
manytomany
WHERE
id = #id
;
While the entries in manytomany are assumed to be unique, the query uses the UNION DISTINCT operator because of the potential duplicates returned by the first subquery, although it is possible to move the application of DISTINCT to the first subquery only like this:
SELECT DISTINCT
id
FROM
manytomany
WHERE
id = #id
UNION ALL
SELECT
linked_id
FROM
manytomany
WHERE
id = #id
;
That first subquery could actually be rewritten simply as SELECT #id AS id, but the rewrite would only make sense if the passed value was guaranteed to be valid, i.e. that it would definitely be found in manytomany.id.
Here's a demo for the other approach too (all three variations, including the SELECT #id AS id one).
I don't have an answer for your question but I encourage you to redesign your table like this I think this called many to many relation
id | friend
1 | 2
1 | 3
1 | 5
1 | 7
2 | 1
2 | 4
2 | 5
2 | 6
3 | 1
3 | 4
3 | 6
3 | 7
And then your query will be like this
SELECT DISTINCT(friend) FROM `new` WHERE id = 1
I am assuming you are using php..
My suggestion is to grab the id_list for id 1.
Explode that id_list on the comma, and then do another mysql query to grab the remaining results for 5|3|6|8
ex) $idarray = explode(",", $result);
select * from your_table where id in ('5','3','6','8')
OPTION 2:
SELECT * FROM your_table
WHERE id = '1'
OR id IN ('\''+(SELECT REPLACE(id_list,',','\',\'') FROM your_table WHERE id = '1')+'\'')
EDIT: Oops, sorry, that should be an OR instead.
I have a table containing several fields. The primary key is userId. Currently the user id column contains values '1,2,3,4...etc' like so:
+------+
|userId|
+------+
| 1 |
| 2 |
| 3 |
| 4 |
...etc
I now want to add new rows ending in a,b,c, like so:
+------+
|userId|
+------+
| 1 |
| 1a |
| 1b |
| 1c |
| 2 |
| 2a |
| 2b |
| 2c |
...etc
The new rows should be identical to their parent row, except for the userId. (i.e. 1a,1b & 1c should match 1)
Also I can't guarantee that there won't already be a few 'a', 'b' or 'c's in userid column.
Is there a way to write an sql query to do this quickly and easily?
DON'T DO IT you will run into more problems than the one you are trying to solve!
add a new column to store the letter and make the primary key cover the original UserId and this new column.
If you ever just want the userId, you need to split the letter portion off, which will be expensive for your query and be a real pain.
I agree with KM. I'm not sure why you're creating these duplicate/composite IDs, but it feels like an uncomfortable direction to take.
That said, there is only really one obsticle to overcome; Apparently you can't select from and insert into the same table in MySQL.
So, you need to insert into a Temporary Table first, then insert into the real table...
CREATE Temporary TABLE MyNewUserIDs (
UserID VARCHAR(32)
)
INSERT INTO
myNewUserIDs
SELECT
CONCAT(myTable.UserID, suffix.val)
FROM
myTable
INNER JOIN
(SELECT 'A' as val UNION ALL SELECT 'B' UNION ALL SELECT 'C' UNION ALL SELECT 'D') AS suffix
ON RIGHT(myTable.UserID, 1) <> Suffix.val
WHERE
NOT EXISTS (SELECT * FROM myTable AS lookup WHERE UserID = CONCAT(myTable.UserID, suffix.val))
INSERT INTO
myTable
SELECT
UserID
FROM
MyNewUserIDs
Depending on your environment, you may want to look into locking the tables, so that changes are not made between creating the list of IDs and inserting them into your table.
This is quite simple from a SQL perspective to generate the extra rows: I'll do that here
#Km's answer tells you how to store it as 2 distinct values which I've assumed here. Feel free to concatenate userid and suffix if you prefer.
INSERT myTable (userid, suffix, co11, col2, ...coln)
SELECT M.userid, X.suffix, M.col1, M.col2, ..., M.coln
FROM
myTable M
CROSS JOIN
(SELECT 'a' AS Suffix UNION ALL SELECT 'b' UNION ALL SELECT 'c') X
WHERE
NOT EXISTS (SELECT *
FROM
MyTable M2
WHERE
M2.userid = M.userid ANS M2.Suffix = X.Suffix)