Get list of IDs not present in table - mysql

Say I have a list of ids, e.g. (1, 3, 9, 2, 4, 86), and a table with a column id. I want to find all of the numbers in my list where there is not a matching row.
i.e. if the mysql table was like this:
id letter
1 a
2 b
3 c
4 d
5 e
6 f
7 g
And I have the list (1, 3, 9, 2, 4, 86), I want a query that will return (9, 86).
The only thing I can think of, is to build a really big virtual table, like:
select 1 as n union select 3 as n union select 9 as n union ....
Which I can then join against. Is there a better way? I would like to be able to do this all within mysql. As a side note (although I don't expect it to be relevant), my table has around 10,000 rows, and the list I'm using has ~100 numbers in it.


You have to first create a table that will contain the elements of the LIST
i.e (1, 3, 9, 2, 4, 86)
create table t
(
num int
)
insert into t
values
(1),(3),(9),(2),(4),(86)
Now you can use NOT IN
SELECT num
FROM t
WHERE num not in (select id from letter_table);
SQL Fiddle
From Comments.
Edit:
There is a way in which you don't have to create a table
select N from
(select 1 as N
union all
select 3 as N
union all
select 9 as N
union all
select 2 as N
union all
select 4 as N
union all
select 86 as N)t1
where t1.N
not in (select id from letter_table)
Please refer the New SQL Fiddle.
I think OP want's the Edited Part.
P.S. Make Sure table t1 doesn't exists in your DB


Create a table which contains IDs and than you can do it eaasily. See a demonstration here
SELECT
S.id,
'' AS `letter`
FROM sequence S
WHERE S.id NOT IN(SELECT
id
FROM mytable)
SQL Fiddle Demo


Assuming you use the temp table or the UNION method in #Luv's answer, consider replacing the NOT IN with an outer join as it'll likely perform better (test with your actual environment & data, of course):
SELECT num
FROM t
LEFT OUTER JOIN letter_table
ON t.num = letter_table.id
WHERE letter_table.id IS NULL;
If you use the UNION method, replace FROM t with FROM ([big UNION here]) t.

Related

How to count the number of entries in the list when requesting Select?

I apologize for the possible incorrectness in the presentation, I use a translator. Let's say there is a users table in which there is an id field. And there is a list that lists the id numbers and some of them are repeated. My query
select id, count(*)
from users
where id in (3, 10, 10, 10)
group by id;
returns the following 3 - 1, 10 - 1. And I would like to get 3 - 1, 10 - 3, and so on. Is it possible to get it somehow?
UPD.
The data in the list (3, 10, 10, 10) is just an example, the exact number of digits is not known because they are returned from another question.

You would need to use a join. You can put the values in a derived table for this:
select id, count(*)
from users u join
(select 3 as id union all
select 10 as id union all
select 10 as id union all
select 10 as id union all
) i
using(id)
group by id;

How to combine data from one table where there are duplicate values into a single row?

I have tried what i thought was a pretty simple sql with php to do the following but it is not turning out correctly and i cant find a good way to "google" the solutions. I have a table valoreguide2 that holds over 200,000 records with multiple columns in this table are duplicate rows that have different values in some columns but same value in other columns. I want to take all of the values and put them in a new table, while doing this i want to combine the rows that have the same value in the column labeled isbn and keep all of the values for those rows in one row in the new table.
example is
if you cant see the image it is a table with 20 columns including a auto increment field. I would like to take all of the information from this table and insert it into another table with different column titles but combine them based on isbn. the code i have written is a follows:
`$results = $conn->query("select * from valoreguide2 group by isbn");
while ($row12 = $results->fetch_assoc()) {
$isbn = $row12['isbn'];
$APrice =$row12['Acomm'];
$AQty=$row12['Aqty'];
$IPrice =$row12['Icomm'];
$IQty=$row12['Iqty'];
$SPrice =$row12['Scomm'];
$SQty=$row12['Sqty'];
$TPrice =$row12['Tcomm'];
$TQty=$row12['Tqty'];
$NPrice =$row12['Ncomm'];
$NQty=$row12['Nqty'];
$BBPrice =$row12['BBcomm'];
$BBQty=$row12['BBqty'];
$guide_prices = array('amtext'=>(float)($APrice), 'ingram'=>(float)($IPrice), 'sterling'=>(float)($SPrice), 'tichenor'=>(float)($TPrice), 'nebraska'=>(float)($NPrice), 'BB'=>(float)($BBPrice), );
$bestGuidePrice = max($guide_prices);
$bestGuidePrice = number_format($bestGuidePrice,2,'.','');
foreach ($guide_prices as $key => $val) {
if ($val == max($guide_prices))
{
$bestGuide = $key;
}
}
$valoreprice=(($bestGuidePrice/1.15)-5);
$conn->query("insert into valorebest2 (isbn, aprice, iprice, sprice, tprice, nprice, bookbyte, bestprice, valore, bestguide) values ('$isbn','$APrice','$IPrice','$SPrice','$TPrice','$NPrice','$BBPrice','$bestGuidePrice','$valoreprice','$bestGuide') ");
}`
but the result is not combining the rows it is just picking one.. If i have not provided enough please let me know.. i did not want to type on and on when someone is going to say hey you missed a comma.
Edited:
Using MySQL
Edited: This is the output i am looking for

Can you be specific about the database, you use: Oracle, MS SQL, MySql ..?
Here is an answer that it based on ansi SQL. Hope it helps
Testdata.
CREATE TABLE tbs_test (ID, ISBN, A, B, C, D) AS
SELECT 1,'0001','A','B','C',10 FROM DUAL UNION ALL
SELECT 2,'0002','A',null,null,null FROM DUAL UNION ALL
SELECT 3,'0002',null,'B',null,null FROM DUAL UNION ALL
SELECT 4,'0002',null,null,'C',null FROM DUAL UNION ALL
SELECT 5,'0003','A',null,'C',10 FROM DUAL UNION ALL
SELECT 6,'0003',null,null,null,10 FROM DUAL UNION ALL
SELECT 7,'0004',null,null,'C',10 FROM DUAL;
This select groups by isbn, taking the min-value of each column, that resides in a row with the same isbn.
select isbn, min(A), min(B), min(C), min(D)
from tbs_test
group by isbn
order by isbn;
This select groups by isbn, takes min-value of A,B,C and sums D
select isbn, min(A), min(B), min(C), sum(D)
from tbs_test
group by isbn
order by isbn;
Hope this helps :-)

If it wont let you select, how on earth can you get any values to work with? :-)
It you'll succeed in getting acces to the data, this would do it (notice: the fake_time column is just a way to emulate two different situations in the input table. You can remove it from the select/update if you'll go live with the code):
CREATE TABLE tbs_test_input (ID, ISBN, A, B, C, D, Fake_time) AS
SELECT 1,'0001','A','B','C',10,1 FROM DUAL UNION ALL
SELECT 2,'0002','A',null,null,null,1 FROM DUAL UNION ALL
SELECT 3,'0002',null,'B',null,null,1 FROM DUAL UNION ALL
SELECT 4,'0002',null,null,'C',null,1 FROM DUAL UNION ALL
SELECT 5,'0003','A',null,'C',10,1 FROM DUAL UNION ALL
SELECT 6,'0003',null,null,null,10,1 FROM DUAL UNION ALL
SELECT 7,'0004',null,null,'C',10,1 FROM DUAL UNION ALL
SELECT 8,'0003',null,'B',null,null,2 FROM DUAL UNION ALL
SELECT 9,'0003',null,'B',null,20,2 FROM DUAL UNION ALL
SELECT 10,'0004','A',null,null,null,2 FROM DUAL
;
create table tbs_test_output as select ISBN, A, B, C, D
from tbs_test_input where 1=2;
insert into tbs_test_output
select isbn, min(A), min(B), min(C), min(D)
from tbs_test_input
where isbn not in (select isbn from tbs_test_output)
and fake_time = 1
group by isbn
order by isbn;
select * from tbs_test_output;
update tbs_test_output o
set (A,B,C,D) = (
select min(i.A), min(i.B), min(i.C), min(i.D)
from tbs_test_input i
where i.fake_time = 2
group by i.isbn
having i.isbn = o.isbn
)
WHERE EXISTS (
SELECT 1
FROM tbs_test_input i
WHERE i.isbn = o.isbn
and i.fake_time = 2)
;
select * from tbs_test_output;
The result:
4 rows inserted.
ISBN A B C D
\---- - - - ----------
0001 A B C 10
0002 A B C
0003 A C 10
0004 C 10
2 rows updated.
ISBN A B C D
\---- - - - ----------
0001 A B C 10
0002 A B C
0003 B 20
0004 A

Convert columns into rows with inner join in mysql

Please take a look at this fiddle.
I'm working on a search filter select box and I want to insert the field names of a table as rows.
Here's the table schemea:
CREATE TABLE general
(`ID` int, `letter` varchar(21), `double-letters` varchar(21))
;
INSERT INTO general
(`ID`,`letter`,`double-letters`)
VALUES
(1, 'A','BB'),
(2, 'A','CC'),
(3, 'C','BB'),
(4, 'D','DD'),
(5, 'D','EE'),
(6, 'F','TT'),
(7, 'G','UU'),
(8, 'G','ZZ'),
(9, 'I','UU')
;
CREATE TABLE options
(`ID` int, `options` varchar(15))
;
INSERT INTO options
(`ID`,`options`)
VALUES
(1, 'letter'),
(2, 'double-letters')
;
The ID field in options table acts as a foreign key, and I want to get an output like the following and insert into a new table:
id field value
1 1 A
2 1 C
3 1 D
4 1 F
5 1 G
6 1 I
7 2 BB
8 2 CC
9 2 DD
10 2 EE
11 2 TT
12 2 UU
13 2 ZZ
My failed attempt:
select DISTINCT(a.letter),'letter' AS field
from general a
INNER JOIN
options b ON b.options = field
union all
select DISTINCT(a.double-letters), 'double-letters' AS field
from general a
INNER JOIN
options b ON b.options = field

Pretty sure you want this:
select distinct a.letter, 'letter' AS field
from general a
cross JOIN options b
where b.options = 'letter'
union all
select distinct a.`double-letters`, 'double-letters' AS field
from general a
cross JOIN options b
where b.options = 'double-letters'
Fiddle: http://sqlfiddle.com/#!2/bbf0b/18/0
A couple to things to point out, you can't join on a column alias. Because that column you're aliasing is a literal that you're selecting you can specify that literal as criteria in the WHERE clause.
You're not really joining on anything between GENERAL and OPTIONS, so what you really want is a CROSS JOIN; the criteria that you're putting into the ON clause actually belongs in the WHERE clause.

I just made this query on Oracle.
It works and produces the output you described :
SELECT ID, CASE WHEN LENGTH(VALUE)=2THEN 2 ELSE 1 END AS FIELD, VALUE
FROM (
SELECT rownum AS ID, letter AS VALUE FROM (SELECT DISTINCT letter FROM general ORDER BY letter)
UNION
SELECT (SELECT COUNT(DISTINCT LETTER) FROM general) +rownum AS ID, double_letters AS VALUE
FROM (
SELECT DISTINCT double_letters FROM general ORDER BY double_letters)
)
It should also run on Mysql.
I did not used the options table. I do not understand his role. And for this example, and this type of output it seems unnecessary
Hope this could help you to.

GROUP_CONCAT on two tables

I have two tables, with independent ids (can't be connected via joins), I want to query and get a GROUP_CONCAT of both columns.
Example: table "a" has ids: 1, 2, 3. table "b" has the ids: 10, 11.
End result should be: 1, 2, 3, 10, 11
I have tried a few queries:
SELECT CONCAT_WS(',', GROUP_CONCAT(a.id), GROUP_CONCAT(b.id)) AS combined FROM a, b
SELECT GROUP_CONCAT(a.id, b.id) AS combined FROM a, b
These queries are returning me duplicate results though 8as in, all results from a twice and all results from b twice as well)

Try union all:
select group_concat(ab.id) as ids
from ((select id from a
) union all
(select id from b
)
) ab;
Your queries are doing cross join's between the tables, so data after the cross join is:
a.id b.id
1 10
1 11
2 10
2 11
3 10
3 11
After the union all, the data is:
ab.id
1
2
3
10
11

GROUP_CONCAT(DISTINCT [])
will help
https://dev.mysql.com/doc/refman/5.0/en/group-by-functions.html#function_group-concat

The following query will generate that you want.
You can play with the table_position dynamic column for deciding which table goes first.
Select group_concat(id order by table_position) from
(
select id, 1 as table_position from a
union all
select id, 2 as table_position from b
)

If you want duplicates, use union all. If you don't want duplicates, use union.
In either case, the query you need is as follows:
select group_concat(id) from
(select id from a
union
select id from b) as ids;

Inserting duplicate records based on a value without using cursor

I had a problem in database. I have to insert duplicate records of a particular record on a another table based on a value.
First i used cursor to fetch each records and get the number of duplication i wants and after that used another cursor for duplication. Everything worked fine. But if the records in more than 500, i went dead slow. Then i did some research and found a way to insert without cursor.
INSERT INTO report(id, Name)
SELECT i.id,i.Name FROM (SELECT 1 AS id
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
UNION SELECT 7
UNION SELECT 8
UNION SELECT 9
UNION SELECT 10) AS o
INNER JOIN table i WHERE o.id<=i.frequence;
where frequence is the number of duplication. Please drop your idea to improve your query.

You could try creating a table with a record for each value from 1 to 10 and then join to that. I'm not sure it would be any faster though. You would have to experiment with it.
In this example the table with the values from 1 to 10 is called "dup" and the field containing these values is called "id".
INSERT INTO report(id, Name)
SELECT i.id, i.Name
FROM table i
JOIN dup d
ON d.id <= i.frequence;

If you have any table that contains a row number that goes at least as high as the maximum frequence, you could to this:
INSERT INTO report(id, Name)
SELECT i.id,i.Name FROM table i
inner join (
select distinct some_row_number_column from some_table
) o on o.some_row_number_column <= i.frequence;
This is basically the same as what you were doing, but it avoids the messy union all statements.
Or you could make a cursor that inserts numbers from 1 to the maximum frequence into a temporary table, then use that in your join. Or you could use a row numbering variable to generate the necessary sequence. Basically, do anything that will generate a list of consecutive numbers from 1 to the maximum that you need.
I would normally use recursion for this (DB2 syntax):
INSERT INTO report(id, Name)
with num_list (num) as (
values (1)
union all
select num + 1 from num_list
where num < (select max(frequence) from table)
)
SELECT i.id,i.Name FROM table i
inner join num_list on num_list.num <= i.frequence;
However, MySQL doesn't support recursion, apparently.