I have a query that goes something like this :
INSERT IGNORE INTO `destination_table` (`id`, `field1`, `field2`, `field3`)
SELECT `id`, `field1`, `field2`, `field3`
FROM `source_table`
WHERE `source_table`.`id` IN (
SELECT DISTINCT `id` FROM `some_table`
UNION DISTICT SELECT DISTINCT `id` FROM `some_other_table`
);
This does not work -- the query hangs indefinitely. The size of the tables is definitely not the problem, all tables have a fairly small number of records ( < 100k records). The query is fine and quite fast if I run it without the UNION :
INSERT IGNORE INTO `destination_table` (`id`, `field1`, `field2`, `field3`)
SELECT `id`, `field1`, `field2`, `field3`
FROM `source_table`
WHERE `source_table`.`id` IN (
SELECT DISTINCT `id` FROM `some_table` -- I tried with `some_other_table` too, same result
);
or
INSERT IGNORE INTO `destination_table` (`id`, `field1`, `field2`, `field3`)
SELECT `id`, `field1`, `field2`, `field3`
FROM `source_table`
both work and are nice and fast (well under a second). So I imagine that the UNION DISTICT SELECT ... is the culprit here, but I don't know why.
What's wrong with that query and why does it hang ?
Using mysql 5.7 is that makes a difference
Your first query seems to have a few typos, but I would suggest using exists logic here:
INSERT IGNORE INTO destination_table (id, field1, field2, field3)
SELECT id, field1, field2, field3
FROM source_table t1
WHERE
EXISTS (SELECT 1 FROM some_table s1 WHERE s1.id = t1.id) OR
EXISTS (SELECT 1 FROM some_other_table s2 WHERE s2.id = t1.id);
The possible advantage of using exists in this way is that MySQL can stop searching as soon as it finds the first matching id in either of the subqueries on the two tables. You may find that adding an index on the id columns in the two other would help (assuming that id be not already indexed):
CREATE INDEX some_idx_1 ON some_table (id);
CREATE INDEX some_idx_2 ON some_other_table (id);
This should speed up the lookup of the id in the two dependent tables.
You could work around the problem by rephrasing the query:
INSERT IGNORE INTO `destination_table` (`id`, `field1`, `field2`, `field3`)
SELECT `id`, `field1`, `field2`, `field3`
FROM `source_table`
WHERE `source_table`.`id` IN (
SELECT DISTINCT `id` FROM `some_table`
)
OR `source_table`.`id` IN (
SELECT DISTINCT `id` FROM `some_other_table`
);
Related
I want to copy data from one table to another but only which has processed='1' in the column value after a specific date.
I have code which could do it but its taking a long time to execute.
"INSERT INTO eamglo5_billingsystem.`consignment` (
`consignment_status`,
`account`,
`awb`,
`hawb`,
`service`,
`handling`,
`reference`,
`date_submitted`,
`date_imported`,
`date_printed`,
`printed_file_id`,
`date_received`,
`date_booked`,
`booked_file_id`,
`date_exported`,
`export_file_id`,
`company`,
`contact`,
`address_line_1`,
`address_line_2`,
`address_line_3`,
`id`
)
SELECT
'Y',
`account`,
`awb`,
`hawb`,
`service`,
`handling`,
`reference`,
`date_submitted`,
`date_imported`,
`date_printed`,
`printed_file_id`,
`date_received`,
`date_booked`,
`booked_file_id`,
`date_exported`,
`export_file_id`,
`company`,
`contact`,
`address_line_1`,
`address_line_2`,
`address_line_3`,
`id`
FROM `eamglo5_singaporelive`.`consignment`
left join (
SELECT eamglo5_billingsystem.`consignment`.`id` as id1
FROM eamglo5_billingsystem.`consignment`
) t ON `eamglo5_singaporelive`.`consignment`.id >id1
WHERE `eamglo5_singaporelive`.`consignment`.`processed`=1
and `eamglo5_singaporelive`.`consignment`.date_booked>'2018-07-17'
Expected: Should copy data from eamglo5_singaporelive.consignment table into eamglo5_billingsystem.consignment table with only processed=1 values.
Actual: Taking an infinite time to execute and fetch the rows.
Your LEFT JOIN with the condition consignment.id >id1 is almost creating a catesian product. What you probably want, is to insert only rows with a higher id from the source table than the highest id1 in the destination table. You should use a SELECT MAX(id) subquery instead:
SELECT [..]
FROM `eamglo5_singaporelive`.`consignment`
WHERE `eamglo5_singaporelive`.`consignment`.`processed`=1
and `eamglo5_singaporelive`.`consignment`.date_booked>'2018-07-17'
and `eamglo5_singaporelive`.`consignment`.id > (
SELECT MAX(id1) FROM eamglo5_billingsystem.`consignment`
)
So here is the perfectly working query I need to run though short of the necessary condition:
INSERT INTO content (`id`,`id_pages`,`content`, `date`)
SELECT `id`, `id`, `content`, `date_modified` FROM `pages`;
Unfortunately not all the databases are synced properly so some of the tables are populated and some are not.
How do I INSERT data from table A to table B IF table A is empty?
A couple queries I've tried:
IF (
SELECT count(id) FROM content='0',
INTO content (`id`,`id_pages`,`content`, `date`)
SELECT `id`, `id`, `content`, `date_modified` FROM `pages`)
...as well as:
IF (SELECT count(id) FROM content)=0
THEN (INSERT INTO content (`id`,`id_pages`,`content`, `date`)
SELECT `id`, `id`, `content`, `date_modified` FROM `pages`);
Try this:
INSERT INTO content (`id`,`id_pages`,`content`, `date`)
SELECT `id`, `id`, `content`, `date_modified`
FROM `pages`
WHERE NOT EXISTS (SELECT 1 FROM content)
The SELECT of the above INSERT statement will return all pages records unless there is at least on record in content table.
Demo with empty table | Demo with not empty table
I am using UNION to get joined output from 2 rows. A and B, following is the code.
"SELECT `ent_id` as `id`, `owner_id`, `category_id`, `ent_name` as `name`, `ent_details` as `details` FROM `A` WHERE `category_id` = '$cat'
UNION
SELECT `service_id` as `id`, `owner_id`, `category_id`, `service_name` as `name`, `service_details` as `details` FROM `B` WHERE `category_id` = '$cat'
The query works absolutely fine however I now want to know which output is from table A and which from B.
Is there a way to do this? If so how?
Thanks for your time. :)
Add a constant value to each select:
select 'table a' as source_table, ... from A where ...
union all
select 'table b' as source_table, ... from B where ...
Also, union eliminates duplicates between the two sets that form the union which union all doesn't. If there can be no duplicates you should use union all for better performance.
I need to find multiple rows related to users and then insert into another table or update if record exists for current day.
I am doing this way
SELECT CASE WHEN
(
SELECT
DISTINCT `userid`,
COUNT(DISTINCT `userip`,`userid`) AS `count`,
#date:=UNIX_TIMESTAMP(CURDATE())
FROM `tablename`
WHERE (`date` >= UNIX_TIMESTAMP(CURDATE())) GROUP BY `userid`
)
THEN
(
UPDATE `tablename2` SET `count`=`count`,`userid`=`userid`,`date`=`date` WHERE `date` >= UNIX_TIMESTAMP(CURDATE()))
)
ELSE
(
INSERT INTO `tablename2` (`count`,`userid`,`date`) VALUES(`count`,`userid`,`date`);
)
END
But this is giving me syntax error near UPDATE..
How can I fix this?
I am guessing that you want one row per user and date in tablename2. If so, enforce this rule with a unique index:
CREATE UNIQUE INDEX idx_tablename2(userid, date)
Then the database enforces it.
Your SQL is a mess, but I think I can see what you are trying to do. The basic idea is INSERT . . . ON DUPLICATE KEY UPDATE. I think the following does what you want:
INSERT INTO `tablename2` (`count`, `userid`, `date`)
SELECT `userid`, COUNT(DISTINCT `userip`, `userid`) AS `count`,
UNIX_TIMESTAMP(CURDATE())
FROM `tablename`
WHERE `date` >= UNIX_TIMESTAMP(CURDATE())
GROUP BY `userid`
ON DUPLICATE KEY UPDATE `count` = VALUES(`count`);
i am trying to do a insert into a with SELECT, bit this following query doesnt seems to be working for some reason,
INSERT INTO `employer_data`
(`employer_id`, `data`, `datetime`)
VALUES
( SELECT employer_id, employer_id AS data, NOW() AS `datetime` FROM employer );
Any ideas ?
remove values keyword
INSERT INTO `employer_data`
(`employer_id`, `data`, `datetime`)
( SELECT employer_id, employer_id AS data, NOW() AS `datetime` FROM employer );
Syntax was wrong.
Try:
INSERT INTO `employer_data`
(`employer_id`, `data`, `datetime`)
SELECT employer_id, employer_id AS data, NOW() AS `datetime` FROM employer;
Refer To:
MySQL: INSERT ... SELECT Syntax