I have the following query that retrieves all rows (x:text,y:text) from table1 where text in column x is not yet in table2.x
Given:
We are using MariaDB;
the column x is defined as not nullable text in both tables;
No indices are there (the application is very dynamic that we do not want to use indices);
we need the column y to be retrieved as well (EXCEPT won't work).
The execution plan always shows the inner query as a Dependent Subquery and the performance is super bad.
SELECT DISTINCT(table1.x), table1.y
FROM table1
WHERE table1.x IS NOT NULL
AND table1.x NOT IN (
SELECT DISTINCT table2.x from table2
WHERE table2.x IS NOT NULL
)
Could you please suggest some better solution to this issue?
Thanks
See if the following is any better for you:
select table1.x, table1.y -- use Distinct only if x & y have duplicates
from table1 t1
where not exists (select * from table2 t2 where t2.x = t1.x);
You obviously don't need to check for NULL is columns do not allow NULL. Probably the optimizer would ignore this anyway.
I have scenario where I have to find the sum of multiple columns from table2 and I have the value in where clause from another table1.
I wrote mySql query for the same as follow. I need to write it in the jooq.
select (sum(t2.column1)+sum(t2.column2)+sum(t2.column3)) as total_amount
from db.table1 t1, db.table2 t2
where t1.column1 = ‘value1’ and t1.column2 = t2.column4;
As a general rule of thumb, all functions are available from org.jooq.impl.DSL by the same name, and all operators are available from org.jooq.Field by a name that reflects the way the operator is pronounced. In your case, use:
DSL.sum(Field)
Field.plus(Field)
Specifically, assuming this static import:
import static org.jooq.impl.DSL.*;
Write:
Table1 t1 = TABLE1.as("t1");
Table2 t2 = TABLE2.as("t2");
ctx.select(sum(t2.COLUMN1).plus(sum(t2.COLUMN2)).plus(t2.COLUMN3)).as("total_amount"))
.from(t1, t2)
.where(t1.COLUMN1.eq("value1"))
.and(t1.COLUMN2.eq(t2.COLUMN4))
.fetch();
I have a uuid that's stored in one table as a human-readable guid, but in another table it's split into upper and lower bits. How can I write a query to join the tables on the uuid?
Edit: table2 will only have 1 result with the given upper & lower bits, so hopefully efficiency shouldn't be too terrible, but please factor this into answers.
table1.uuid = 'b33ac8a9-ae45-4120-bb6e-7537e271808e'
table2.upper_bits = -5531888561172430560, table2.lower_bits = -4940882858296115058
I need to retrieve table2.status along with table1.* where table2.upper_bits + table2.lower_bits = table1.uuid (pseudo where statement) but I don't know how to either sum the table2 upper & lower values for the join, or how to convert table1's uuid to bits for the join.
Thanks!
Something like this might work... but would obviously be highly inefficient.
SELECT ...
FROM table1 AS t1
INNER JOIN table2 AS t2 ON REPLACE(t1.uuid, '-', '')
= CONCAT(HEX(t2.upper_bits), HEX(t2.lower_bits))
...
...you might to force upper/lower case depending on collation/comparison.
I'd kind of lean toward it being "absolutely necessary" to change your database structure (to respond to a comment you made on another answer). To minimize the impact on existing queries and logic, you could change one of the tables to have additional matching fields to the other, and add triggers to the table to automatically populate/update the new fields; and then do a one time update to set all the old records' values.
I'd try to go with modifying t1 first, since an index on two ints is likely "better" than on one string; but I'm not sure how straight-forward converting the string to the upper and lower bits would be.
Modifying t2 would be easier, the triggers would be little more than SET NEW.uuid = CONCAT(HEX(NEW.upper_bits), HEX(NEW.lower_bits)); ...I say "little more than" because it would be best for the trigger to also insert the -'s at the expected points as well so the join condition could eliminate all function use.
Edit: I found a way to calculate the bits in pure SQL:
SELECT #uuid := REPLACE('b33ac8a9-ae45-4120-bb6e-7537e271808e', '-', '') AS uuid
, -1 * CAST((~CAST(CONV(SUBSTRING(#uuid, 1, 16), 16, 10) AS SIGNED) + 1) AS SIGNED) AS upper_bits
, -1 * CAST((~CAST(CONV(SUBSTRING(#uuid, 17, 16), 16, 10) AS SIGNED) + 1) AS SIGNED) AS lower_bits
;
You may be able to use something like this in triggers for t1, and a one time update for t1 for the new fields.
...it might even help with the join:
ON -1 * CAST((~CAST(CONV(SUBSTRING(REPLACE(t1.uuid, '-', ''), 1, 16), 16, 10) AS SIGNED) + 1) AS SIGNED)
= t2.upper_bits
AND -1 * CAST((~CAST(CONV(SUBSTRING(REPLACE(t1.uuid, '-', ''), 17, 16), 16, 10) AS SIGNED) + 1) AS SIGNED)
= t2.lower_bits
Note: Yes, the excessive casting in both of these appears to be necessary (at least on the older version of MySQL I tested the calculation against.)
Here's a variation of #Uueerdo's solution that should be more efficient (sort of a decorate-join-undecorate), but I haven't run the EXPLAINs to know for sure:
SELECT t1.*, t2.status
FROM (
SELECT UUID_TO_BIN(uuid) AS tmpid, *
FROM table1
) AS t1 INNER JOIN (
SELECT UUID_TO_BIN(CONCAT(HEX(upper_bits), HEX(lower_bits))) AS tmpid, status
FROM table2
) AS t2 ON t1.tmpid = t2.tmpid
It might use a bit more memory, which is something to keep in mind if the tables have many rows and/or if table1 is very wide.
If you only need the records from table1 and table2 matching a single UUID, you should just execute two queries, not a join:
SELECT *
FROM table1
WHERE UUID_TO_BIN(uuid) = UUID_TO_BIN(?)
SELECT status
FROM table2
WHERE UUID_TO_BIN(CONCAT(HEX(upper_bits), HEX(lower_bits))) = UUID_TO_BIN(?)
If upper_bits and lower_bits are indexed, this would be a better way to query table2:
SET #tmpid = UUID_TO_BIN(?);
SELECT status
FROM table2
WHERE upper_bits = #tmpid >> 64, lower_bits = _binary X'FFFFFFFFFFFFFFFF' & #tmpid
And you could apply similar logic to my first solution (I think):
SELECT t1.*, t2.status
FROM (
SELECT
UUID_TO_BIN(uuid) >> 64 AS upper_bits,
_binary X'FFFFFFFFFFFFFFFF' & UUID_TO_BIN(uuid) AS lower_bits,
*
FROM table1
) AS t1 INNER JOIN (
SELECT upper_bits, lower_bits, status
FROM table2
) AS t2 ON t1.upper_bits = t2.upper_bits AND t1.lower_bits = t2.lower_bits
None of this is tested but hopefully it gives you some ideas to play around with.
The easiest way is to store lower_bits & upper_bits in table1 together with the uuid. Then join the tables on lower_bits & upper_bits.
Edit: If you need only a single row, and you are sure, that there will be a single matching row in the other table, then calculate the uuid= #uuid, lower_bits =#lbits & upper_bits= #ubits, and then run the following:
Select t1.*, t2.status
From
(select * from table1 where uuid = #uuid) as t1
Cross join
(select status from table2 where lower_bits =#lbits and upper_bits= #ubits) as t2;
I have a query:
SELECT DISTINCT field1 FROM table1 WHERE field2 = something
(table1 contains 1 million records, execution time:0.106sec, returns: 20 records)
Another query
SELECT DISTINCT similarField1 FROM table2 WHERE similarField2 = somethingElse
(table2 contains half million records, execution time:0.078sec, returns: 20 records)
Now if I run a query, by combining above both:
SELECT DISTINCT field1 FROM table1 WHERE field2 = something AND field1 NOT IN (SELECT DISTINCT similarField1 FROM table2 WHERE similarField2 = somethingElse)
It does't give result even running for 10mins. Why it has became dramatically slow, and what could be a potential solution.
edit: I am using MySQL with dbvisualizer 6.5
You don't need to use DISTINCT on the sub-query. Try to use NOT EXISTS which probably is more efficient in SQL-Server:
SELECT DISTINCT field1
FROM table1
WHERE field2 = #something
AND NOT EXISTS
(
SELECT 1 FROM table2
WHERE table2.similarfield1 = table1.field2
AND table2.similarfield2 = #somethingelse
)
Edit: Since you have updated the tags, i'm not sure if this is more efficient in MySql. However, i'd prefer NOT EXISTS anyway since it also works with NULL values(if you use IS NULL) and is easier to read and to maintain.
my query and advice are similar to #TimSchmelter.
In fact you should not use distinct at all. First you should remove distinct and check if you are getting duplicate records you have just ask part of your problem.Table design are not clear.
You should post your complete problem and query here without any hesitant. Also don't forget to apply index on feild2, feild1,similarField1,similarField2.
SELECT DISTINCT field1
FROM table1 tbl1
WHERE field2 = something
AND NOT EXISTS (
SELECT similarField1
FROM table2 tbl2
WHERE tbl1.field1 = tbl2.similarField1
AND similarField2 = somethingElse
)
Warning: vagueness & unclear questioning will abound because I know squat about databases.
I just discovered that I need to use views as surrogates for a cronned update statement. I can somewhat get the view to work, but I'm having trouble with rows.
This post helped me to bang out the update I need, but now that I know that views can run that update whenever it's needed rather than on a cron schedule, how can I set the view's column value based upon the view's row id or equivalent?
I've got the select I need:
SELECT SUM( table2.column1/ (
SELECT table2constant
FROM table3
)
FROM table2
WHERE table2table1id = table1id
table1id is the AI id column for table1. table2table1id is PKd to table1id. I'd like the view to have a column PKd to table1id like with table2, and the view needs to have every distinct table1id represented.
I'm sure the jargon's way off, but hopefully you can see what I need.
Will provide as many edits as necessary for clarity.
Many thanks in advance!
EDIT1
Should I create a trigger that creates the view upon insert to table1? Just found about materialization which is what I need/want?
Clarity
I need a summed value for each table1.table1id
Progress
With this code, I'm getting the first id from table1 and only the total sum. I need a sum for each table1.id.
CREATE VIEW db1.sums as
SELECT SUM( table2.column1/ (
SELECT table2constant
FROM table3
) as theSum, table1id
FROM table1, table2
WHERE table2.table2table1id = table1.table1id
To be clear I'm still not sure what you're trying to accomplish here but if what you posted works, try
SELECT table1.table1id,
SUM( table2.collumn1 ) / (SELECT table2constant FROM table3 ) as theSum
FROM table1, table2
WHERE table2.table2table1id = table1.table1id GROUP BY table1.table1id
you can replace (SELECT table2constant FROM table3 ) with your constant if it has no reason to otherwise be in the database (if it's not updated)
Its actually very simple. Here is an example of how you can do it.
SELECT SUM( table1.column / table2.column ), table1.*, table2.*
FROM table1, table2
WHERE table1.id = table2.column_id