Converting three-table FULL OUTER JOIN query to MySQL - mysql

I need to translate this query which works in SQL Server to MySQL. I've circled around it at least 50 times and am stumped and cannot even find a starting point.
The issues in a nutshell:
It has not one but two FULL OUTER joins which MySQL does not do. I
know this can be emulated some how using UNION in combination with LEFT and RIGHT joins.
It joins the tables on multiple conditions and I am not sure how to add exclusion
criteria in the ON if doing a typical RIGHT JOIN
SELECT COALESCE(ca.reporting_period, s_p.reporting_period, s_uy.reporting_period) AS reporting_period,
COALESCE(ca.state, s_p.state, s_uy.state) AS state,
COALESCE(ca.servicer, s_p.servicer, s_uy.servicer) AS servicer,
COALESCE(ca.product, s_p.product, s_uy.product) AS product,
COALESCE(ca.product_group, s_p.product_group, s_uy.product_group) AS product_group,
COALESCE(ca.portfolio, s_p.portfolio, s_uy.portfolio) AS portfolio,
COALESCE(ca.channel, s_p.channel, s_uy.channel) AS channel,
ca.Gross,
ca.Costs,
ca.commission,
ca.commissionable,
s_p.WAVG_placed_numerator,
s_p.place_balance AS WAVG_placed_denominator,
SUM(s_uy.cum_gross) AS gross_uy_num,
SUM(s_uy.cum_netnet) AS netnet_uy_num,
SUM(s_uy.cum_commission) AS servicer_uy_num,
SUM(s_uy.num_accounts) AS uy_den,
ca.costs_recovered
# INTO adhoc_work.RevenueReport_Legal
FROM adhoc_work.Cash ca
FULL OUTER JOIN adhoc_work.Summary_placed s_p
ON ca.reporting_period = s_p.reporting_period
AND ca.state = s_p.state
AND ca.servicer = s_p.servicer
AND ca.product = s_p.product
AND ca.product_group = s_p.product_group
AND ca.portfolio = s_p.portfolio
AND ca.channel = s_p.channel
FULL OUTER JOIN adhoc_work.Summary_uy s_uy
ON ca.reporting_period = s_uy.reporting_period
AND ca.state = s_uy.state
AND s_p.state = s_uy.state
AND ca.servicer = s_uy.servicer
AND s_p.servicer = s_uy.servicer
AND ca.product = s_uy.product
AND s_p.product = s_uy.product
AND ca.product_group = s_uy.product_group
AND s_p.product_group = s_uy.product_group
AND ca.portfolio = s_uy.portfolio
AND s_p.portfolio = s_uy.portfolio
AND ca.channel = s_uy.channel
AND s_p.channel = s_uy.channel
WHERE COALESCE(ca.reporting_period,s_p.reporting_period,s_uy.reporting_period) BETWEEN DATE_FORMAT(TIMESTAMPADD(MONTH,-13,NOW(3)),'%Y%m%d') AND DATE_FORMAT(NOW(3),'%Y%m%d')
GROUP BY COALESCE(ca.reporting_period,s_p.reporting_period,s_uy.reporting_period)
,COALESCE(ca.state,s_p.state,s_uy.state)
,COALESCE(ca.servicer,s_p.servicer,s_uy.servicer)
,COALESCE(ca.product,s_p.product,s_uy.product)
,COALESCE(ca.product_group,s_p.product_group,s_uy.product_group)
,COALESCE(ca.portfolio,s_p.portfolio,s_uy.portfolio)
,COALESCE(ca.channel,s_p.channel,s_uy.channel)
,ca.gross
,ca.Costs
,ca.commission
,ca.commissionable
,s_p.WAVG_placed_numerator
,s_p.place_balance
,ca.costs_recovered;

Actually, your query doesn't quite do a full outer join, because a condition like this:
ca.state = s_uy.state
requires that ca.state be matched. That is, the outer join is turned to an inner join.
So, to answer your question more theoretically, the simplest method is probably to generate all the rows by using UNION among the the table and then use left join:
FROM (SELECT reporting_period, . . .
FROM adhoc_work.Cash ca
UNION -- on purpose to remove duplicates
SELECT reporting_period, . . .
FROM adhoc_work.Summary_placed
UNION -- on purpose to remove duplicates
SELECT reporting_period, . . .
FROM adhoc_work.Summary_uy
) x LEFT JOIN
adhoc_work.Cash ca
ON x.reporting_period = ca.reporting_period AND
. . . LEFT JOIN
adhoc_work.Summary_placed s_p
ON x.reporting_period = s_p.reporting_period AND
. . . LEFT JOIN
adhoc_work.Summary_uy s_uy
ON x.reporting_period = s_uy.reporting_period AND
. . .

Related

Magento 2 filterBuilder left join

I'm trying to filter product collection with multiple OR filter with this code :
$values = ['xxx','xxx'];
$filters = [];
$filters[] = $this->_filterBuilder
->setField('field_a')->setConditionType('in')
->setValue($values)
->create();
$filters[] = $this->_filterBuilder
->setField('field_b')
->setConditionType('in')
->setValue($values)
->create();
// two more filter like that
$filterGroup = $this->_filterGroupBuilder
->setFilters($filters)
->create();
$searchCriteria = $this->_searchCriteriaBuilder
->setFilterGroups([$filterGroup])
->create();
$products = $this->_productRepository->getList($searchCriteria)->getItems();
Problem is collection return 0 result instead of two. After analyze sql query generated by Magento eav table are joined with INNER JOIN like that :
INNER JOIN `catalog_product_entity_text` AS `at_field_b` ON (`at_field_b`.`row_id` = `e`.`row_id`) AND (`at_field_b`.`attribute_id` = '204') AND (`at_field_b`.`store_id` = 0)
If on raw sql query I execute it by replacing INNER JOIN by LEFT JOIN it works, i've got my results.
So my question is how can I "force" magento to left join instead of inner ? Or maybe it's a pure coincidence and real reason isn't the left/inner join
I didn't precise but field_a,field_b,etc... aren't not required so they could be empty for products

Select * from view more than 10x slower than executing the views create statement directly with MySQL 5.7

I have created a view as a join of several tables to hide the complexity.
Now when I execute a select * from view it takes more than 10x of the execution time it takes to execute the select statement directly, that defines the view (about 70 ms vs 900 ms).
This behaviour occurs on a MySQL 5.7.23 server.
I compared the execution plans for both queries and the only difference (as expected) is the derived table:
Is this a normal behaviour of views or how can I fix it?
P.S.:
The create view statement as requested (names where obfuscated):
create view vrs as select
rs."id" as "id",
l."language" as "language",
rss."t" as "discriminator",
rss."type" as "type",
coalesce(lt3."text", concat('!',s3."textkey",'(',l."language",')')) as "typeText",
rss."model" as "model",
coalesce(lt2."text", concat('!',s2."textkey",'(',l."language",')')) as "modelText",
rs."name" as "name",
rs."reqCATGroup" as "CATGroup",
rs."devID" as "devID",
case s."builtIn"
when 0 then s."displayName"
else (select lt1."text"
from loctext lt1
where lt1."language"=l."language"
and lt1."textkey"=s."displayName")
end as "sName",
s."id" as "sId",
s."postcode" as "sPostcode",
s."place" as "sPlace",
s."street" as "sStreet",
s."streetNumber" as "sStreetNumber",
s."tId" as "tId",
t."tName" as "tName",
rss."CCVMajor" as "CCVMajor",
rss."CCVMinor" as "CCVMinor",
rss."CCVPatch" as "CCVPatch",
rss."CCHV" as "CCHV",
coalesce(lHw."text", concat('!',sHw."textkey",'(',l."language",')')) as "CCHVText",
rss."MCVMajor" as "MCVMajor",
rss."MCVMinor" as "MCVMinor",
rss."MCVPatch" as "MCVPatch",
rss."level",
coalesce(lLevel."text", concat('!', sLevel."textkey", '(', l."language", ')')) as "levelText",
rss."event",
coalesce(lEvent."text", concat('!', sEvent."textkey", '(', l."language", ')')) as "eventText",
rss."isBusy" as "isBusy",
rss."tsLastComIn" as "tsLastCom",
rss."tsBT" AS "tsBT",
b1."ordinal" as "isSending",
coalesce(ltSend."text", concat('!',b1."textkey",'(',l."language",')')) as "isSendingText",
b2."ordinal" as "isReceiving",
coalesce(ltReceive."text", concat('!',b2."textkey",'(',l."language",')')) as "isReceivingText",
rs."historyCreateDevTs",
rs."historyCreateDevUserId",
rs."historyCreateDevUserLoginName",
rs."historyModifyDevLinkTs",
rs."historyModifyDevLinkUserId",
rs."historyModifyDevLinkUserLoginName",
rs."monitorStartTs",
rs."monitorEndTs"
from synch rss
cross join loclanguages l
join cfgrs rs on rs."id" = rss."id"
join cfgs s on s."id" = rs."sId"
join cfgt t on t."id" = s."tId"
left join locmodel s2 on s2."ordinal" = rss."model"
left join loctext lt2 on lt2."textkey" = s2."textkey" and lt2."language" = l."language"
left join loctype s3 on s3."ordinal" = rss."type"
left join loctext lt3 on lt3."textkey" = s3."textkey" and lt3."language" = l."language"
left join locbooleanrange b1 on rs."sending"+1 = b1."ordinal"
left join loctext ltSend on ltSend."textkey" = b1."textkey" and ltSend."language" = l."language"
left join locbooleanrange b2 on rs."receiving"+1 = b2."ordinal"
left join loctext ltReceive on ltReceive."textkey" = b2."textkey" and ltReceive."language" = l."language"
left join loceventlevel sLevel on sLevel."ordinal" = rss."level"
left join loctext lLevel on lLevel."textkey" = sLevel."textkey" and lLevel."language" = l."language"
left join locevent sEvent on sEvent."ordinal" = rss."event" and sEvent."type" = rss."type"
left join loctext lEvent on lEvent."textkey" = sEvent."textkey" and lEvent."language" = l."language"
left join loccchv sHw on sHw."ordinal" = rss."CCHV"
left join loctext lHw on lHw."textkey" = sHw."textkey" and lHw."language" = l."language"
where rs."deleted" = 0
;
P.P.S.: The complete execution plans look like this:
direct call:
view:
This link is for MySQL 8.0, but when I last had performance issues with Views in MySQL 5.6, 5.7, and went digging, I got the same answer: You basically cannot take advantage of Indexes when using Views.
https://dev.mysql.com/doc/refman/8.0/en/view-restrictions.html
I personally found Views in MySQL to Not be useful overall, due to limitations like the ones described above, particularly due to the performance hit.
Readers - if somehow Views in MySQL can perform, I certainly missed a key memo somewhere. Please correct me if this is the case.

Chain of SQL subqueries within large query of JOINS

I am trying to structure a SQL query with complex nested select operation!
My original SQL query successfully JOINS around 30 tables, the data are retrieved as wanted! However every fetched record in the 30 tables
has many records in another table called (Comments)! What I want to do is to atribute every record in the (Comments table) to its record in the other
30 tables by IDs and retrieve them all together in one query. Yet this is not the only challenge, some of the 30 tables have in addition to the records in
(Comments table) more records in another table called (extras), so i am looking for additional subquery within the main subquery within
a LEFT JOIN inside the outter main query.
To make the idea more clear; without subquery the script will be as following:
$query = $mysqli->query("
SELECT
parent1.parent1_id,
parent1.child1_id,
parent1.child2_id,
parent1.child3_id,
parent2.parent2_id,
parent2.child1_id,
parent2.child2_id,
parent2.child3_id,
child1.child1_id,
child1.child1_content,
child2.child2_id,
child2.child2_content,
child3.child3_id,
child3.child3_content
FROM
parent1
LEFT JOIN child1
ON child1.child1_id = parent1.child1_id
LEFT JOIN child2
ON child2.child2_id = parent1.child2_id
LEFT JOIN child3
ON child3.child3_id = parent1.child3_id
LEFT JOIN followers
ON parent1.user_id = followers.followed_id
AND parent1.parent1_timestamp > followers.followed_timestamp
AND parent1.parent1_id NOT IN (SELECT removed.isub_rmv FROM removed)
AND parent1.parent1_hide = false
WHERE
followers.follower_id = {$_SESSION['info']}
{$portname_clause}
ORDER BY
parent1.parent1_timestamp DESC
LIMIT
{$postnumbers}
OFFSET
{$offset}
")
// Now fetching and looping through the retrieved data
while($row = $query->fetch_assoc()){
echo $row['child1_content'];
$subquery1 = $mysqli->query("SELECT extras.child1_id,
extras.extrasContent FROM extras WHERE extras.child1_id =
{$row['child1_id']}");
while($row1 = $subquery1->fetch_assoc()){
echo $row1['extrasContent'];
}
echo $row['child2_content'];
$subquery2 = $mysqli->query("SELECT extras.child2_id,
extras.extrasContent FROM extras WHERE extras.child2_id =
{$row['child2_id']}");
while($row2 = $subquery2->fetch_assoc()){
echo $row2['extrasContent'];
}
echo $row['child3_content'];
$subquery3 = $mysqli->query("SELECT extras.child3_id,
extras.extrasContent FROM extras WHERE extras.child3_id =
{$row['child3_id']}");
while($row3 = $subquery3->fetch_assoc()){
echo $row3['extrasContent'];
// Here i need to run additional query inside the subquery 3 to retrieve the (Comments table) data beside (extras table)
$subquery4 = $mysqli->query("SELECT comments.comment_id, comments.comment FROM comments WHERE comments.child3_id = {$row['child3_id']} OR comments.child3_id = {$row3['child3_id']}");
while($row4 = $subquery4->fetch_assoc()){
echo $row4['comment'];
}
}
} // No sane person would make such code
Because the code above would be totally rediclious i searched for a better way to carry it out, and thats where i came across the subquery
concept, but i do not know anything about subqueries, and shortly after i studied it i came up with this messy code, check it below!
I am not posting the origianl code here because it is too long, i am including a virtual example of the tables i want to apply the
query on in order to demonstrate the process.
SELECT
parent1.parent1_id,
parent1.child1_id,
parent1.child2_id,
parent1.child3_id,
parent2.parent2_id,
parent2.child1_id,
parent2.child2_id,
parent2.child3_id
FROM
parent1
LEFT JOIN
( SELECT
child1.child1_id,
child1.child1_content
FROM
child1
WHERE
child1.child1_id = parent1.child1_id ) child1
( SELECT extras.extrasID, extras.extrasContent
FROM
extras
WHERE
extras.child1_id = child1.child1_id )
ON parent1.child1_id = child1.child1_id
LEFT JOIN child2
( SELECT
child2.child2_id,
child2.child2_content
FROM
child2
WHERE
child2.child2_id = parent1.child2_id )
( SELECT
extras.extrasID,
extras.extrasContent
FROM
extras
WHERE
extras.child2_id = child2.child2_id )
ON parent1.child2_id = child2.child2_id
LEFT JOIN child3
( SELECT
child3.child3_id,
child3.child3_content
FROM
child3
WHERE
child3.child3_id = parent1.child3_id )
( SELECT
extras.extrasID,
extras.extrasContent
FROM
( SELECT
comments.comment_id,
comments.comment
FROM
comments
WHERE
comments.child3_id = extras.child3_id ) extras
JOIN child3
ON extras.child3_id = child3.child3_id )
ON parent1.child3_id = child3.child3_id
LEFT JOIN followers
ON parent1.user_id = followers.followed_id
AND parent1.parent1_timestamp > followers.follower_timestamp
AND parent1.parent1_id NOT IN (SELECT removed.isub_rmv FROM removed)
AND parent1.parent1_hide = false
WHERE
followers.follower_id = {$_SESSION['info']}
{$portname_clause}
ORDER BY
parent1.parent1_timestamp DESC
LIMIT
{$postnumbers}
OFFSET
{$offset} // <-- Sorry for the bad code formatting!
I am using MySql 5.6.37
I did not get the hang of the subquery concept yet, frankly i got lost and confused as i was studying it and for another reason too mentioned in the note below.
Note: I apologize in advance that i might not be on instant reply because where i live there is no electrecity or ADSL or phones and my
USB modem hardly gets signal, i have only two hours of averege three hours a day of electericity generated by a desil generator. i recharge
my laptop and check internet and the remaining one-two hours are for other life stuff.
I know the joke is on me as i am developing a web project without electricity or permanent internet. BUT LIFE DOES NOT GIVE EVERYTHING! lol.
This is how i solved the problem!
SELECT
parent1.parent1_id,
parent1.child1_id,
child1.child1_id,
child1.child1_content,
comments.comment_id,
comments.comment,
comments.child1_id
FROM parent1 LEFT JOIN
(
SELECT comments.comment_id, comments.comment, comments.child1_id
FROM
(
SELECT comments.comment_id,comments. comment, comments.child1_id
FROM comments
) comments JOIN child1
ON comments.child1_id = child1.child1_id
) comments
ON child1.child1_id = comments.
It needs some aliases and then it's good to go.

How to search exact string using MySql Query which avoids extended values?

My Query looks like
$search_query = db_query("SELECT nd.nid, users.name, nd.type FROM node as nd
LEFT JOIN node_revisions as nd_rev ON nd_rev.nid = nd.nid AND nd_rev.vid = nd.vid
LEFT JOIN users ON nd.uid = users.uid
WHERE nd.status = 1 AND nd_rev.body LIKE LOWER('%node/100%')
AND nd.nid NOT IN(SELECT DISTINCT nid FROM term_node WHERE tid = 293)");
This query actually returns all the matches from node_revisions.body field, Which includes
node/1000, node/1001.... Etc.,
I want to get only the result of exact match where possible like
"node/100"
"node/100/"
"/node/100"
"/node/100/"
'node/100'
'node/100/'
'/node/100'
'/node/100/'
and not like
"node/1006"
"node/10064/"
"/node/1000"
"/node/10001/"
'node/10023'
'node/1005/'
'/node/1001'
'/node/10069/'
This above query returned me result which has string like below..
..a href="/node/1006"
How to avoid this kind of errors? Please help..
Try removing the % after 100 so the search won't consider any digit after 100, like this:
LOWER('%node/100')
Then consider the following Regular Expression
Example:
`nd_rev.body` REGEXP "^/?node/100/?$"
Oh ya... I got an resolution for this.. I redefined my query like below and it gives me result as expected..
$search_query = db_query("SELECT nd.nid, users.name, nd.type FROM node as nd
LEFT JOIN node_revisions as nd_rev ON nd_rev.nid = nd.nid AND nd_rev.vid = nd.vid
LEFT JOIN users ON nd.uid = users.uid
WHERE nd.status = 1 AND nd_rev.body RLIKE '[[:<:]]" . $search_string . "[[:>:]]'
AND nd.nid NOT IN(SELECT DISTINCT nid FROM term_node WHERE tid = 293)");
Look at
nd_rev.body RLIKE '[[:<:]]" . $search_string . "[[:>:]]'
This is what i expected

configuring how to pull data out of 3 tables with mysql

Here is how the the database is layout. I can connect to DB fine.
I had it pullling from database two things but adding a third i can not get it to pull. I just need some help if i could ..
database - mmoore_drupal
table 1
table name = content_type_uprofile
data
vid= 19723
nid =19674
field_name_value = matthew moore
table 2
table name = location_instance
data
lid = 1521
vid = 19723
nid = 19674
table 3
table name = location
data
lid = 1521
street =
city =
country =
latitude =
longitude =
I am trying to pull name and then other info from the other two tables. But mainly i need to have name and other information from location. I thought i had to have the other table to associate the connection. Any help is appreciated.
$query = "SELECT content_type_uprofile.field_name_value,location.street,location.city
FROM location_instance,location,content_type_uprofile
WHERE location_instance.lid = location.lid and location_instance.nid=content_type_uprofile.nid"
$result = mysql_query($query) or die(mysql_error());
// Print out the contents of each row into a table
while($row = mysql_fetch_array($result)){
echo $row['nid']."-".$row['street']. " - ". $row['city'];
echo "<br />";
}
?>
Use this SQL (explicit join syntax):
SELECT
content_type_uprofile.nid,
location.street,
location.city
FROM
content_type_uprofile
INNER JOIN location_instance
ON (content_type_uprofile.nid = location_instance.nid)
INNER JOIN location
ON (location_instance.lid = location.lid)
The SQL that you posted is using implicit join SQL syntax.
I think for some reason, I think the line in your SQL:
WHERE location_instance.lid = location.lid and location_instance.nid=content_type_uprofile.nid
is filtering out all the rows from your result set. I'm not sure because I avoid the implicit syntax.
You were also missing the nid field which your PHP code is looking for in the result set.
As long as your data is correct (i.e. the fields that you are joining on have the right values), the SQL that I posted will work for you.
You ever done something with join's?
select *.locaition,
*.content_type_uprofile
from location_instance li
inner join location l on l.lid = li.lid
inner join content_type_uprofile ctu on ctu.vid = li.vid