I have a problem with my database, which has many tables but i am focused on the main named TblLivroPorta(t1) and the 2nd one named Tblp_h(t2).
so t1 comunicate with the application while t2 stores everything that happens with t1, I can say that t2 is the t1's backup.
I want to find the data I am looking for in t2 and copy them to t1 so they can be accessed by the application.
The statement below gives me all data I want to copy back
select NOrdem, Num_Oficio from tblp_h where Num_Oficio != '3469/3ª V/TAPS/2012'
and Data_Saida between '2012-01-01' and '2012-11-30'
union select NOrdem, Num_Oficio from TbLivroPorta
where Num_Oficio = null and Data_Saida between '2012-01-01' and '2012-11-30'
so my difficult is to copy them back to t1.
I hope I have been clear.
You need to use INSERT INTO SELECT to copy the values from one table to another
http://www.w3schools.com/sql/sql_insert_into_select.asp
You need to ensure that your SELECT statement returns the columns that will be inserted into the target table
Related
I have a select request in MySQL that takes between 25-30s, which is extremely long and I was wondering if you could help me fasten it.
CREATE TEMPORARY TABLE results(
id VARCHAR(30),
secondid VARCHAR(5),
allele VARCHAR(30),
translation VARCHAR(10),
level VARCHAR(20),
subgroup VARCHAR(20),
subgroup2 VARCHAR(20)
);
INSERT INTO results(id, secondid, allele, level) SELECT DISTINCT t1.id, t1.secondid, t1.texte, t3.texte
FROM database t1
JOIN database t2 ON t1.id=t2.id
JOIN database t3 ON t1.id=t3.id AND t1.secondid=t3.secondid
WHERE (t1.qualifier,t2.qualifier) = ("allele","organism") AND t3.qualifier = "level_length" AND t3.texte NOT REGEXP "X" AND t3.texte IS NOT NULL
AND t2.texte = ? AND t1.texte REGEXP ?
GROUP BY t1.texte;
UPDATE results SET translation = (SELECT t1.qualifier
FROM database t1
JOIN database t2 ON t1.id=t2.id AND t1.secondid=t2.secondid
JOIN database t3 ON t1.id=t3.id AND t1.secondid=t3.secondid
WHERE t1.qualifier IN ("protein","ncRNA","rRNA") AND t2.texte=results.allele AND t3.texte=results.level LIMIT 1);
UPDATE results SET subgroup = (SELECT t2.subgrp
FROM alleledb.alleleSubgroups t1
JOIN alleledb.subgroups t2 ON t1.subgroup=t2.subgroup
WHERE t1.gene=SUBSTRING_INDEX(results.allele, "*", 1) AND t1.species=? LIMIT 1);
ALTER TABLE results DROP id, DROP secondid;
SELECT * FROM results ORDER BY subgroup ASC, level ASC;
DROP TABLE results;
I need to go through many dbs to get join (same id), database are huge but results to extract are quite low (less than 1% of all the database). The majority of the results are stored in the same db, in different rows (with the same id and secondid). However, id and secondid are not unique to the rows I need to select, only the combinaison of two is.
Thank you.
I would start by having a proper composite index on your database table
First on
(qualifier, id, secondid, texte)
This will help your joins, the where testing and NOT have to go back to the actual raw data tables for the records as the index has the data you are interested in.
Next, I would adjust the query/joins. Since you are specifically looking for the "allele" and "organism" from t1 and t2 respectively, make them as such.
I have no idea what you are doing with your REGEXP "X" or "?" values for texte, but you'll figure that out after.
Here is how I would revise the queries
insert into ...
SELECT DISTINCT
t1.id,
t1.secondid,
t1.texte,
t3.texte
FROM
database t1
JOIN database t2
ON t1.id = t2.id
AND t2.qualifier = 'organism'
JOIN database t3 ON
t1.id = t3.id
AND t1.secondid = t3.secondid
AND t3.qualifier = 'level_length'
WHERE
t1.qualifier = 'allele'
AND t1.texte REGEXP ?
-- I would move these t2 and t3 into the respective JOINs above directly.
AND t3.texte NOT REGEXP "X"
AND t3.texte IS NOT NULL
AND t2.texte = ?
GROUP BY
t1.texte;
As for your UPDATE commands, having a second index on (id, secondid) will help on the join to t2 and t3 since there is no qualifier context to the join.
As for your UPDATE commands, as Rick mentioned, without some context of an ORDER BY clause, you have no guarantee WHICH record is returned back by the LIMIT 1.
First of all, thank you for all your help.
My first table (The insert to and the first update, database named) looks like this :
I want all things in red. In others words, I need some parameters which has the same id and secondid as the "level" which is unique among the id. Whereas others parameters may be repeated within the same id (but different second id).
I am filtering using the allele name (ECK in EC locus) with thé REGEXP and species. For example, all allèles from EC locus of human.
Then (last update), I take one parameter (allele), substring it and go to a database that gives me one id (one row -> one id). And I use this id on annoter database that gives me one or two rows (one subgroup or two subgroups/rare). So as in my example I only has one group, the absence of ORDER BY was not seen. But yes I want to order (get the subgroup that contains the allele in first). I don't know how to do that.
Finally, I can try to make an index but due to the size of the db, I'm wondering the time and the size of such an index. Would it significally improve time and can I remove it ?
The REGEXP "X" is to remove every matches that are not relevant regarding this parameter (I don't want them).
The ? is user input (for the species/2 occurrences this one and the locus).
The operations on the first database takes 30s, last operation on the two databases lasts 1-2s. Others (drop , select) are <20ms (not the problem).
This is a hard one. A third party has been sending us data from a fourth party. But they have done that in a horrible format and they messed up and duplicated many of the data.
Now the data is all in one table, even though it should have been in much more than one. This has to do with a historical data format.
Now what SHOULD be each record with multiple related records in other tables, is actually put into our database as follows:
Id HistoricalId Field1 Field2 Field3 Field4 FieldX ...
1 327
2 data data data
3 data data data
4 data data
5 data data
6 328
7 data data data (etc etc)
Everything grossly simplified. So you always first have a sort of "header record". Then records with the data. Until there is a new header. Let's call all the records from one header to the next together a "superrecord" (for instance in the example ID 1 t/m 5 form together the first superrecord, the next superrecord stats at Id 6).
Problem is: there are MANY duplicate "superrecords", easily identified by their duplicate HistoricalId in the header record. But they can be anywhere in the database (the records that form the superrecord will be well sorted and not mixed up, but the superrecords are mixed up).
So the puzzle: remove all duplicate superrecords. We are talking 10.000s here if not more.
So, how would you, in MySQL:
Find a Id from a duplicate superrecord (easy)
Find the Id from the next header record (i.e. the following superrecord)
Delete everything between (and including) the first Id and the second Id minus 1
And do this for all duplicate superrecords.
My head starts spinning. It must be possible with just mySQL, but how? I am just not experienced enough. Even though I am not bad at MySQL, here I cannot even see where to start. Or should I program something in php?
Anyone likes a challenge? Thank you in advance!
UPDATE: Solved it thanks to you and two hours of hard work. See solution.
If you're open to copying to a different table etc., then...
You can figure which records you want to delete. All records where the historical-id exists in some other record with a higher ID
SELECT id, HISTORICAL_ID
FROM tbl t1
WHERE historical_id>0
AND exists
(SELECT 1 FROM tbl t2
WHERE T2.hISTORICAL_id=T1.HISTORICAL_ID and T2.ID>T1.ID)
Since each record has an ID, for each record, you could compute the ID of the Header Record. (This is what you mention in your comment). It would be the Max. ID from any "previous" record where historical id is filled in.
Select ID, HISTORICAL_ID
,(Select MAX(ID) FROM T2 Where T1.ID <T2.ID and T1.HistoricalId<>0) As PARENT_ID
From TBL T1
You can then match the PARENT_ID with the first query to get all the IDs you wish to delete
I finally solved it. Thanks everyone, you all put me into the right direction.
Three queries are needed:
First mark all duplicate header records by setting HistoricalID to -1
UPDATE
t1 INNER JOIN
(SELECT MIN(id) AS keep, HistoricalID FROM t1
GROUP BY HistoricalID
HAVING count(*) > 1 AND HistoricalID > 0) t2
ON t1.HistoricalID = t2.HistoricalID
SET HistoricalID = IF(t1.id=t2.keep, t1.HistoricalID , -1)
WHERE t1.HistoricalID > 0
Secondly copy HistoricalID from the header record to all other records below it (in the same superrecord). I can undo this later easily if needed.
UPDATE
t1 JOIN
( SELECT Id, #s:=IF(HistoricalID='', #s, HistoricalID) HistoricalID FROM
(SELECT * FROM t1 ORDER BY Id) r, (SELECT #s:='') t ) t2
ON t1.Id = t2.Id
SET t1.HistoricalID= t2.HistoricalID
Delete all duplicates:
DELETE FROM t1 WHERE HistoricalID = -1
It worked. Couldn't have done it without you!
I need to select the data from two tables and insert into one table. The same kind of the question was asked and answered many times but I have kind of requirement.
I have total three tables T1,T2,T3.
My final goal is to insert the data into table T3. I have total 15 columns in table T3. Among that 15 columns I need to fill the 14 columns data from table T2 and the data for the last column I need to join table T1 and T2 from that I need to fetch the data for that column. Please find the below query
CREATE procedure proc_name
BEGIN
Insert into T3(
id,
col1,
col2,
....
...
col14)
select
(select id from T1 INNER JOIN T2 ON
(T1.somecol1=T2.somecol1,
T1. somecol2= T2.somecol2,
T1.somecol3 = T2.somecol3,
T1.somecol4= T2.somecol4)
ORDER BY T2.somecol5 LIMIT 1),
T2.col1,
T2.col2,
...
...
T2.col14 from T2;
END;
Here the rest of the fourteen columns of T3 have the relation first column id.
Whenever I call the above stored procedure all the records in into T3 is inserting with top 1 id in table T1 time even though I have total 10 id's in T1.
After close observation I came to know that the reason for that is as I'm mentioning the limit 1 so it is fetch only first id every time.
If I didn't mention limit 1 it is returning all the 10 id's and the query it self is failing.
Is there anyway I can get all id's in table T3. Please suggest me.
Thanks in Advance.
To elaborate or the comment by #jarlh (who is absolutely correct) you need to have a join between the two tables rather than selecting a value in a subquery
INSERT INTO T3
(id
,col1
,col2
,...
,col14)
SELECT T1.id
,T2.col1
,T2.col2
,...
FROM T1
INNER JOIN T2 ON
(T1.somecol1=T2.somecol1,
T1. somecol2= T2.somecol2,
T1.somecol3 = T2.somecol3,
T1.somecol4= T2.somecol4)
With your code as it is, you would only ever expect a single value from T1.
I have a large table with 1 mln records. I get this table from another company. All records have an autonumber id as PRIMARY KEY. A lot of the fields are empty, because some records "belong together" (as a group) and these fields are only filled in for the first record (sort of header record).
I want to fill in the same values in all following records until that field is not empty (that is where a new "header" from a new group of records starts). (I know, bad database design, but it is what I get and I want to turn it into better database design as soon as it gets in, and this is just the first necessary step into a longer step process to get there.)
I am having a hard time getting this right. I want to UPDATE the table, in order of id, where, if the specific fields are empty, they are filled in with the values of the previous record.
I tried different solutions, all of them turn out not to be working.
My last one:
UPDATE t1
SET f1 =
(SELECT t2.f1 FROM t1 AS t2
WHERE t2.id = t1.id-1) ,
Verzend_tijd=
(SELECT t3.f2 FROM t1 as t3
WHERE t3.id = t1.id-1)
WHERE t2.f1 = ''
ORDER BY t1.id
I get the error:
You can't specify target table 't1' for update in FROM clause
Anyone an idea how to get this done? I also tried with an INNER JOIN but it turns our I cannot do an ORDER BY in UPDATE+INNER JOIN, and the order is important!
I am a bit at a loss and Googling didn't bring me anything either.
After more and more Googling I found the answer right here on this website:
Mysql Updating a record with a value from the previous record
Sorry for the inconvenience
I'm not sure if this is considered as a noob question, but i'm trying to understand, how come when i execute a query like this in phpmyadmin
SELECT t1.id, FROM_UNIXTIME(t1.t_stamp/1000) as time_stamp, t1.value
FROM table1 t1
INNER JOIN table2 t2
ON t1.id = t2.colid
WHERE t1.colid = "123"
AND
DATE(from_unixtime(t1.t_stamp/1000)) BETWEEN "2014-03-24" AND "2014-03-25"
I can't directly edit the rows, I mean that the Edit, Copy, Delete icons are not showing and I tried editing the rows but the change didn't apply
Then I tried another query like this
SELECT id, value, t_stamp
FROM table2
WHERE DATE(from_unixtime(t_stamp/1000))
BETWEEN "2014-03-24" AND "2014-03-25"
AND id = "123"
and then with that query i could edit the rows. Is there an option that I need to activate so that the Edit, Copy, and Delete icons is always showing and that I can always directly edit the rows?