I have tried what i thought was a pretty simple sql with php to do the following but it is not turning out correctly and i cant find a good way to "google" the solutions. I have a table valoreguide2 that holds over 200,000 records with multiple columns in this table are duplicate rows that have different values in some columns but same value in other columns. I want to take all of the values and put them in a new table, while doing this i want to combine the rows that have the same value in the column labeled isbn and keep all of the values for those rows in one row in the new table.
example is
if you cant see the image it is a table with 20 columns including a auto increment field. I would like to take all of the information from this table and insert it into another table with different column titles but combine them based on isbn. the code i have written is a follows:
`$results = $conn->query("select * from valoreguide2 group by isbn");
while ($row12 = $results->fetch_assoc()) {
$isbn = $row12['isbn'];
$APrice =$row12['Acomm'];
$AQty=$row12['Aqty'];
$IPrice =$row12['Icomm'];
$IQty=$row12['Iqty'];
$SPrice =$row12['Scomm'];
$SQty=$row12['Sqty'];
$TPrice =$row12['Tcomm'];
$TQty=$row12['Tqty'];
$NPrice =$row12['Ncomm'];
$NQty=$row12['Nqty'];
$BBPrice =$row12['BBcomm'];
$BBQty=$row12['BBqty'];
$guide_prices = array('amtext'=>(float)($APrice), 'ingram'=>(float)($IPrice), 'sterling'=>(float)($SPrice), 'tichenor'=>(float)($TPrice), 'nebraska'=>(float)($NPrice), 'BB'=>(float)($BBPrice), );
$bestGuidePrice = max($guide_prices);
$bestGuidePrice = number_format($bestGuidePrice,2,'.','');
foreach ($guide_prices as $key => $val) {
if ($val == max($guide_prices))
{
$bestGuide = $key;
}
}
$valoreprice=(($bestGuidePrice/1.15)-5);
$conn->query("insert into valorebest2 (isbn, aprice, iprice, sprice, tprice, nprice, bookbyte, bestprice, valore, bestguide) values ('$isbn','$APrice','$IPrice','$SPrice','$TPrice','$NPrice','$BBPrice','$bestGuidePrice','$valoreprice','$bestGuide') ");
}`
but the result is not combining the rows it is just picking one.. If i have not provided enough please let me know.. i did not want to type on and on when someone is going to say hey you missed a comma.
Edited:
Using MySQL
Edited: This is the output i am looking for
Can you be specific about the database, you use: Oracle, MS SQL, MySql ..?
Here is an answer that it based on ansi SQL. Hope it helps
Testdata.
CREATE TABLE tbs_test (ID, ISBN, A, B, C, D) AS
SELECT 1,'0001','A','B','C',10 FROM DUAL UNION ALL
SELECT 2,'0002','A',null,null,null FROM DUAL UNION ALL
SELECT 3,'0002',null,'B',null,null FROM DUAL UNION ALL
SELECT 4,'0002',null,null,'C',null FROM DUAL UNION ALL
SELECT 5,'0003','A',null,'C',10 FROM DUAL UNION ALL
SELECT 6,'0003',null,null,null,10 FROM DUAL UNION ALL
SELECT 7,'0004',null,null,'C',10 FROM DUAL;
This select groups by isbn, taking the min-value of each column, that resides in a row with the same isbn.
select isbn, min(A), min(B), min(C), min(D)
from tbs_test
group by isbn
order by isbn;
This select groups by isbn, takes min-value of A,B,C and sums D
select isbn, min(A), min(B), min(C), sum(D)
from tbs_test
group by isbn
order by isbn;
Hope this helps :-)
If it wont let you select, how on earth can you get any values to work with? :-)
It you'll succeed in getting acces to the data, this would do it (notice: the fake_time column is just a way to emulate two different situations in the input table. You can remove it from the select/update if you'll go live with the code):
CREATE TABLE tbs_test_input (ID, ISBN, A, B, C, D, Fake_time) AS
SELECT 1,'0001','A','B','C',10,1 FROM DUAL UNION ALL
SELECT 2,'0002','A',null,null,null,1 FROM DUAL UNION ALL
SELECT 3,'0002',null,'B',null,null,1 FROM DUAL UNION ALL
SELECT 4,'0002',null,null,'C',null,1 FROM DUAL UNION ALL
SELECT 5,'0003','A',null,'C',10,1 FROM DUAL UNION ALL
SELECT 6,'0003',null,null,null,10,1 FROM DUAL UNION ALL
SELECT 7,'0004',null,null,'C',10,1 FROM DUAL UNION ALL
SELECT 8,'0003',null,'B',null,null,2 FROM DUAL UNION ALL
SELECT 9,'0003',null,'B',null,20,2 FROM DUAL UNION ALL
SELECT 10,'0004','A',null,null,null,2 FROM DUAL
;
create table tbs_test_output as select ISBN, A, B, C, D
from tbs_test_input where 1=2;
insert into tbs_test_output
select isbn, min(A), min(B), min(C), min(D)
from tbs_test_input
where isbn not in (select isbn from tbs_test_output)
and fake_time = 1
group by isbn
order by isbn;
select * from tbs_test_output;
update tbs_test_output o
set (A,B,C,D) = (
select min(i.A), min(i.B), min(i.C), min(i.D)
from tbs_test_input i
where i.fake_time = 2
group by i.isbn
having i.isbn = o.isbn
)
WHERE EXISTS (
SELECT 1
FROM tbs_test_input i
WHERE i.isbn = o.isbn
and i.fake_time = 2)
;
select * from tbs_test_output;
The result:
4 rows inserted.
ISBN A B C D
\---- - - - ----------
0001 A B C 10
0002 A B C
0003 A C 10
0004 C 10
2 rows updated.
ISBN A B C D
\---- - - - ----------
0001 A B C 10
0002 A B C
0003 B 20
0004 A
Here is the updated scenario, hope this is clearer than the last version.
Table 1 is where all data for this process is housed and contains around 5 million records.
`table1`
ID Forename Surname Tel Source Optin DistributedDate
1 A Test 0131 TL037 NULL NULL
2 B Test 0141 TL035 NULL NULL
v v v v v v v
Table 2 is updated via an insert script and contains a distinct list of sources used within a period of time, this could just as easily be created by a view.
INSERT INTO table2(`Sourcecode`)
(SELECT DISTINCT(a.`Sourcecode`) FROM `table1` a)
`table2 - structure`
ID Sourcecode
1 TL037
2 TL031
3 TL004
4 TL029
5 TL035
I am now trying to pull back the details of 5 random records from each distinct source from table 2 and insert these details into table 3.
`table3 - structure`
ID Forename Surname Tel Source Optin DistributedDate
NULL NULL NULL NULL NULL NULL NULL
This is the code I've created so far:
SET #Sourcecode =
(SELECT b.`Sourcecode`
FROM `table2` b
WHERE b.id = b.id
AND b.`Sourcecode` NOT IN (SELECT DISTINCT(source) FROM `table3`)
LIMIT 1);
INSERT INTO table3.*
(SELECT
a.id AS 'ID',
a.`FirstName` AS 'Forename',
a.`Surname` AS 'Surname',
a.`TelephoneNumber2` AS 'Tel',
a.`SourceCode` AS 'Source',
a.`optin` AS 'Optin',
a.`DateExported` AS 'DistributedDate'
FROM `table1`
WHERE a.`SourceCode` = #SourceCode
ORDER BY RAND(a.sourcecode)
LIMIT 5
Basically the end result I'm looking for is to run the above queries multiple times automatically from the result-set against #Sourcecode until value is NULL.
I've tried this using the loop function but I'm not having much luck with it.
You can create a View like this:
CREATE VIEW query1 AS
SELECT DISTINCT(b.`source`)
FROM `table2` b
Then collect from the View as:
SELECT *
FROM `table1` a
WHERE a.`source` IN(DISTINCT(b.`source`) FROM `query1` b)
ORDER BY RAND(a.id)
LIMIT 5)
I haven't tested it myself, if this doesn't work, please give some source and expected data.
Try this:
SELECT a.id, A2.RAND_NUMBER
FROM `table1` a, (SELECT DISTINCT(b.`source`)
,ABS(CHECKSUM(NewId())) % #INT AS RAND_NUMBER
FROM `table2` b) AS A2
WHERE A2.RAND_NUMBER BETWEEN 1 AND 5
GROUP BY a.id, A2.RAND_NUMBER
ORDER BY RAND(a.id)
Say I have a list of ids, e.g. (1, 3, 9, 2, 4, 86), and a table with a column id. I want to find all of the numbers in my list where there is not a matching row.
i.e. if the mysql table was like this:
id letter
1 a
2 b
3 c
4 d
5 e
6 f
7 g
And I have the list (1, 3, 9, 2, 4, 86), I want a query that will return (9, 86).
The only thing I can think of, is to build a really big virtual table, like:
select 1 as n union select 3 as n union select 9 as n union ....
Which I can then join against. Is there a better way? I would like to be able to do this all within mysql. As a side note (although I don't expect it to be relevant), my table has around 10,000 rows, and the list I'm using has ~100 numbers in it.
You have to first create a table that will contain the elements of the LIST
i.e (1, 3, 9, 2, 4, 86)
create table t
(
num int
)
insert into t
values
(1),(3),(9),(2),(4),(86)
Now you can use NOT IN
SELECT num
FROM t
WHERE num not in (select id from letter_table);
SQL Fiddle
From Comments.
Edit:
There is a way in which you don't have to create a table
select N from
(select 1 as N
union all
select 3 as N
union all
select 9 as N
union all
select 2 as N
union all
select 4 as N
union all
select 86 as N)t1
where t1.N
not in (select id from letter_table)
Please refer the New SQL Fiddle.
I think OP want's the Edited Part.
P.S. Make Sure table t1 doesn't exists in your DB
Create a table which contains IDs and than you can do it eaasily. See a demonstration here
SELECT
S.id,
'' AS `letter`
FROM sequence S
WHERE S.id NOT IN(SELECT
id
FROM mytable)
SQL Fiddle Demo
Assuming you use the temp table or the UNION method in #Luv's answer, consider replacing the NOT IN with an outer join as it'll likely perform better (test with your actual environment & data, of course):
SELECT num
FROM t
LEFT OUTER JOIN letter_table
ON t.num = letter_table.id
WHERE letter_table.id IS NULL;
If you use the UNION method, replace FROM t with FROM ([big UNION here]) t.
I had a problem in database. I have to insert duplicate records of a particular record on a another table based on a value.
First i used cursor to fetch each records and get the number of duplication i wants and after that used another cursor for duplication. Everything worked fine. But if the records in more than 500, i went dead slow. Then i did some research and found a way to insert without cursor.
INSERT INTO report(id, Name)
SELECT i.id,i.Name FROM (SELECT 1 AS id
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
UNION SELECT 7
UNION SELECT 8
UNION SELECT 9
UNION SELECT 10) AS o
INNER JOIN table i WHERE o.id<=i.frequence;
where frequence is the number of duplication. Please drop your idea to improve your query.
You could try creating a table with a record for each value from 1 to 10 and then join to that. I'm not sure it would be any faster though. You would have to experiment with it.
In this example the table with the values from 1 to 10 is called "dup" and the field containing these values is called "id".
INSERT INTO report(id, Name)
SELECT i.id, i.Name
FROM table i
JOIN dup d
ON d.id <= i.frequence;
If you have any table that contains a row number that goes at least as high as the maximum frequence, you could to this:
INSERT INTO report(id, Name)
SELECT i.id,i.Name FROM table i
inner join (
select distinct some_row_number_column from some_table
) o on o.some_row_number_column <= i.frequence;
This is basically the same as what you were doing, but it avoids the messy union all statements.
Or you could make a cursor that inserts numbers from 1 to the maximum frequence into a temporary table, then use that in your join. Or you could use a row numbering variable to generate the necessary sequence. Basically, do anything that will generate a list of consecutive numbers from 1 to the maximum that you need.
I would normally use recursion for this (DB2 syntax):
INSERT INTO report(id, Name)
with num_list (num) as (
values (1)
union all
select num + 1 from num_list
where num < (select max(frequence) from table)
)
SELECT i.id,i.Name FROM table i
inner join num_list on num_list.num <= i.frequence;
However, MySQL doesn't support recursion, apparently.
I have a table with a column (registration_no varchar(9)). Here is a sample:
id registration no
1 42400065
2 483877668
3 019000702
4 837478848
5 464657588
6 19000702
7 042400065
Please take note of registration numbers like (042400065) and (42400065), they are almost the same, the difference is just the leading zero.
I want to select all registration numbers that have the same case as above and delete the ones without a leading zero i.e (42400065)
pls, also note that before i delete the ones without leading zeros (42400065), i need to be sure that there is an equivalent with leading zeros(042400065)
declare #T table
(
id int,
[registration no] varchar(9)
)
insert into #T values
(1, '42400065'),
(2, '483877668'),
(3, '019000702'),
(4, '837478848'),
(5, '464657588'),
(6, '19000702'),
(7, '042400065')
;with C as
(
select row_number() over(partition by cast([registration no] as int)
order by [registration no]) as rn
from #T
)
delete from C
where rn > 1
create table temp id int;
insert into temp select id from your_table a where left (registration_no, ) = '0' and
exists select id from your_table
where a.registration_no = concat ('0', registration_no)
delete from your_table where id in (select id from temp);
drop table temp;
I think you can do this with a single DELETE statement. The JOIN ensures that only duplicates can get deleted, and the constraint limits it further by the registration numbers that don't start with a '0'.
DELETE
r1
FROM
Registration r1
JOIN
Registration r2 ON RIGHT(r1.RegistrationNumber, 8) = r2.RegistrationNumber
WHERE
LEFT(r1.RegistrationNumber, 1) <> '0'
Your table looks like this after running the above DELETE. I tested it on a SQL Server 2008 instance.
ID RegistrationNumber
----------- ------------------
2 483877668
3 019000702
4 837478848
5 464657588
7 042400065
This solution won't depend on the registration numbers being a particular length, it just looks for the ones that are the same integer, yet not the same value (because of the leading zeroes) and selects for the entry that has a '0' as the first character.
DELETE r
FROM Registration AS r
JOIN Registration AS r1 ON r.RegistrationNo = CAST(r1.RegistrationNo AS INT)
AND r.RegistrationNo <> r1.RegistrationNo
WHERE CHARINDEX('0',r.registrationno) = 1