I have two queries on the same table and I need to get results which do not appear in the first resultset.
There are several ways to get there (I do not ask for one), my first try was this:
SELECT * FROM
(
SELECT
o.custom_order_id AS 'order',
'yes' AS 'test'
FROM orders
WHERE <first criteria>
UNION
SELECT
o.custom_order_id AS 'order',
'no' AS 'test'
FROM orders
WHERE <second criteria>
) x
WHERE x.test = 'no'
UNION does not append rows which already appeared in the first resultset.
Actually I do get rows like
12345 no
but 12345 does appear in the first resultset (query before UNION).
Why that ?
Edit:
custom_order_id has no index and is not primary key (although it actually is unique) - does UNION need a (unique) index or pk to recognize a row as already-in-first-resultset?
UNION uses the entire tuple to determine if a row is unique. In your case that is (order, test).
As one half of your answers has test set to "yes", and one "no", you can end up with multiple orders with the same id (one for yes, one for no).
You only get rows with "no", because that's what you specified in your WHERE clause at the end:
WHERE x.test = 'no'
Eliminate that and UNION will return unique rows from unioned queries.
UNION doesn't necesarrly need keys, although optimizing query is almost always a good idea :) Try running EXPLAIN ( above query with UNION goes here) and see what is produced.
Related
I have created the below query. but the below query taken lots of time to fetch result. i have added two queries and combined with union statement. how to optimize the below query in my sql. it consume much amount of time.
select count(*) as count
from(
select id,createdUser,patientName,patientFirstName,patientLastName,tagnames
FROM vw_tagged_forms v1 where v1.tenant_id = 91 AND
CASE WHEN v1.patsiteId IS NOT NULL THEN v1.patsiteId IN
(151,2937,1450,1430,2746,1431,1472,1438,2431,1428) ELSE v1.patsiteId IS NULL END group by
COALESCE(`message_grp_id`, `id`)
UNION
select
id,createdUser,patientName,patientFirstName,patientLastName,tagnames
FROM vw_tagged_forms_logs v2 where tenant_id = 91 AND CASE WHEN v2.patsiteId IS NOT
NULL THEN v2.patsiteId IN (151,2937,1450,1430,2746,1431,1472,1431) ELSE v2.patsiteId IS
NULL END) a
Simplification: I think that
CASE WHEN x IS NOT NULL
THEN x IN (...)
ELSE x IS NULL
END
can be simplified to either of these:
x IN (...) OR x IS NULL
or
COALESCE(x IN (...), TRUE)
Bug? (The IN lists are not the same; was this an oversight?)
INDEX (for performance): Add this composite index, with the columns in the order given:
INDEX(tenant_id, patsiteId)
When adding a composite index, DROP index(es) with the same leading columns.
That is, when you have both INDEX(a) and INDEX(a,b), toss the former.
DISTINCT, id, etc:
Are the id values different for the two tables? If they are AUTO_INCREMENT and the rows were independently INSERTed, I would expect them to be different. If, on the other hand, you copy a row from v1 to v2, then I would expect the ids to match.
That leads to the question of "what are you counting"? If the ids are expected to match, then do only SELECT id. If they they were independently generated, then leave id out of the SELECTs; keep the other 5 columns.
I want to remove duplicates based on the combination of listings.product_id and listings.channel_listing_id
This simple query returns 400.000 rows (the id's of the rows I want to keep):
SELECT id
FROM `listings`
WHERE is_verified = 0
GROUP BY product_id, channel_listing_id
While this variation returns 1.600.000 rows, which are all records on the table, not only is_verified = 0:
SELECT *
FROM (
SELECT id
FROM `listings`
WHERE is_verified = 0
GROUP BY product_id, channel_listing_id
) AS keepem
I'd expect them to return the same amount of rows.
What's the reason for this? How can I avoid it (in order to use the subselect in the where condition of the DELETE statement)?
EDIT: I found that doing a SELECT DISTINCT in the outer SELECT "fixes" it (it returns 400.000 records as it should). I'm still not sure if I should trust this subquery, for there is no DISTINCT in the DELETE statement.
EDIT 2: Seems to be just a bug in the way phpMyAdmin reports the total count of the rows.
Your query as it stands is ambiguous. Suppose you have two listings with the same product_id and channel_id. Then what id is supposed to be returned? The first, the second? Or both, ignoring the GROUP request?
What if there is more than one id with different product and channel ids?
Try removing the ambiguity by selecting MAX(id) AS id and adding DISTINCT.
Are there any foreign keys to worry about? If not, you could pour the original table into a copy, empty the original and copy back in it the non-duplicates only. Messier, but you only do SELECTs or DELETEs guaranteed to succeed, and you also get to keep a backup.
Assign aliases in order to avoid field reference ambiguity:
SELECT
keepem.*
FROM
(
SELECT
innerStat.id
FROM
`listings` AS innerStat
WHERE
innerStat.is_verified = 0
GROUP BY
innerStat.product_id,
innerStat.channel_listing_id
) AS keepem
This is what I have now.
SELECT distinct ClientID
FROM Table
WHERE PAmt = '' and ClientID not in
(select distinct ClientID from Table where PAmt != '')
ORDER BY ID ASC
ClientID can be inside the Table more then once and some of them have PAmt value some don't.
I am trying to get only the clientid's that never had a PAmt value. The table has about 12000 entry's and only 2700 are unique clientid's
I think this could be easier solved by
SELECT
ClientID,
MAX(IF(PAmt='',0,1)) AS HasPAmt
FROM `Table`
GROUP BY ClientID
HAVING HasPAmt=0
Edit
Some words on the rationale behind this:
Subqueries in MySQL are a beasty thing: If the resultset is either too large (Original SQL) or intertwined with the driving query (#DavidFleeman's answer), the inner query is looped, i.e. it is repeated for every row of the driving query. This ofcourse gives bad performance.
So we try to reformulate the query in a way, that will avoid looping. My suggestion works by running only two queries: The first (everything before the HAVING) will create a temp table, that marks each distinct ClientID as having at least one non-empty PAmt (or not), the second selects only those rows of the temp table, that are marked as having none, into the final result set.
try to reorganize your query to something like this:
select clientID
from Table
group by clientID
having max(length(PAmt)) == 0
of course you should add index (clientID, PAmt)
if this query will still work slow, add column with pre-calculated length, and replace PAmt with this column
Take away the subquery:
$clientIds = runQuery("select distinct ClientID from Table where PAmt != ''");
$clientIds = implode(",", $clientIds);
runQuery("SELECT distinct ClientID
FROM Table
WHERE PAmt = '' and ClientID not in
({$clientIds})
ORDER BY ID ASC")
I know it looks like MySQL should do that optimisation step for you, but it doesn't. You will find your query is about 12000 times faster if you do the sub query as a separate query.
Try using not exists instead of not in.
http://sqlfiddle.com/#!2/249cd5/26
I need help with CONCAT function. I have two select queries and the result of every query is one column. I need to merge this two columns in one. Is that possible? Beacuse, I can't get result even if I try with simple select queries like:
SELECT owner FROM table WHERE number="value1";
SELECT number FROM table WHERE owner="value2" AND number IS NOT null;
These queries work and throw 3 rows like result. But, if I want to merge them in one column using CONCAT - that doesn't work. Do you know why?
SELECT CONCAT(SELECT owner FROM table WHERE number="value1",
SELECT number FROM table WHERE owner="value2" AND number IS NOT null
) as NEW_COLUMN FROM table;
I think you want this:
SELECT CONCAT(owner, number) newCol1
FROM yourTable
WHERE number="value1"
OR (owner="value2" AND number IS NOT null)
SELECT
CONCAT(owner, number) as NEW_COLUMN
FROM
table
WHERE
owner = "value2"
AND number = "value1"
AND number IS NOT NULL
The fundamental reason is that the DB cannot concatenate two different SELECTs which might have a different number of rows.
What you need to do is to re-formulate your query in terms of a JOIN.
For example suppose we have this table:
owner number
John value1
value2 123456
Your first query:
SELECT owner FROM table WHERE number="value1";
will return "John". The second one
SELECT number FROM table WHERE owner="value2" AND number IS NOT null;
will return "123456".
If you CONCAT the two values you would therefore get "John 123456".
First of all, is this the expected behaviour of the query you want? What happens is there is a third row with owner=Jack and number=value1, so that the first query returns TWO rows "John" and "Jack"?
One thing you could look into is the CROSS JOIN syntax.
SELECT CONCAT (table1.owner, ', ', table2.number) AS new_column
FROM ( SELECT owner FROM table WHERE number="value1" ) AS tablel1
CROSS JOIN
(SELECT number FROM table WHERE owner="value2" AND number IS NOT null ) AS table2;
Note that if the first query returns three rows and the second four rows, the combined query will return 3*4 = 12 rows.
I have two tables.
I query like this:
SELECT * FROM (
Select requester_name,receiver_name from poem_authors_follow_requests as one
UNION
Select requester_name,receiver_name from poem_authors_friend_requests as two
) as u
where (LOWER(requester_name)=LOWER('user1') or LOWER(receiver_name)=LOWER('user1'))
I am using UNION because i want to get distinct values for each user if a user exists in the first table and in the second.
For example:
table1
nameofuser
peter
table2
nameofuser
peter
if peter is on either table i should get the name one time because it exists on both tables.
Still i get one row from first table and a second from table number two. What is wrong?
Any help appreciated.
There are two problems with your SQL:
(THis is not the question, but should be considered) by using WHERE over the UNION instead of the tables, you create a performance nightmare: MySQL will create a temporary table containing the UNION, then query it over the WHERE. Using a calculation on a field (LOWER(requester_name)) makes this even worse.
The reason you get two rows is, that UNION DISTINCT will only suppress real duplicates, so the tuple (someuser,peter) and the tuple (someotheruser, peter) will result in duplication.
Edit
To make (someuser, peter) a duplicate of (peter, someuser) you could use:
SELECT
IF(requester_name='peter', receiver_name, requester_name) AS otheruser
FROM
...
UNION
SELECT
IF(requester_name='peter', receiver_name, requester_name) AS otheruser
FROM
...
So you only select someuser which you already know : peter
You need the where clause on both selects:
select requester_name, receiver_name
from poem_authors_follow_requests
where LOWER(requester_name) = LOWER('user1') or LOWER(receiver_name) = LOWER('user1')
union
select requester_name, receiver_name
from poem_authors_friend_requests
where LOWER(requester_name) = LOWER('user1') or LOWER(receiver_name) = LOWER('user1')
The two queries are independent of each other, so you shouldn't try to connect them other than by union.
You can use UNION if you want to select rows one after the other from several tables or several sets of rows from a single table all as a single result set.
UNION is available as of MySQL 4.0. This section illustrates how to use it.
Suppose you have two tables that list prospective and actual customers, a third that lists vendors from whom you purchase supplies, and you want to create a single mailing list by merging names and addresses from all three tables. UNION provides a way to do this. Assume the three tables have the following contents:
http://w3webtutorial.blogspot.com/2013/11/union-in-mysql.html
You are doing the union before and then applying the where clause. So you would get a unique combination of "requester_name,receiver_name" and then the where clause would apply. Apply the where clause in each select...
Select requester_name,receiver_name from poem_authors_follow_requests
where (LOWER(requester_name)=LOWER('user1')
or LOWER(receiver_name)=LOWER('user1'))
UNION
Select requester_name,receiver_name from poem_authors_friend_requests
where (LOWER(requester_name)=LOWER('user1')
or LOWER(receiver_name)=LOWER('user1'))
In your where statement, reference the alias "u" for each field refence in your where statement.
So the beginning of your where statement would be like: where (LOWER(u.requester_name) = ...
This is simlar to the answer you can see in: WHERE statement after a UNION in SQL?
You should be able to use the INTERSECT keyword instead of doing a nested query on a UNION.
SELECT member_id, name FROM a
INTERSECT
SELECT member_id, name FROM b
can simply be rewritten to
SELECT a.member_id, a.name
FROM a INNER JOIN b
USING (member_id, name)
http://www.bitbybit.dk/carsten/blog/?p=71