I am using Sequalize ORM to fetch data from the database. I have below query to execute which runs fine in Postgres DB and I want to sequalize it, specifically, my query is related to how to sequalize the Sub-query part in FROM clause.
SELECT
"users"."id",
"users"."user_id" AS "peopleId",
"users"."first_name" AS "firstName",
"users"."last_name" AS "lastName",
"users"."email",
"users"."mobile",
"users"."status"
FROM (select * from users where address_id in (select addresses_id from location_view where location = 'Some, Location, India')) as "users"
LEFT OUTER JOIN "addresses" AS "addresses" ON "users"."address_id" = "addresses"."id"
LEFT OUTER JOIN "countries" AS "addresses->countries" ON "addresses"."country_id" = "addresses->countries"."id"
LEFT OUTER JOIN "work_info" AS "work_info" ON "users"."work_info_id" = "work_info"."id"
WHERE
"users"."status" = 'Active'
ORDER BY
"users"."first_name" ASC
LIMIT
5 OFFSET 0;
I know we do have Sequalize.query and literal methods to achieve this where I can directly specify my sub-query in raw format, but whatever examples I have googled out basically provides a way of how I can write a raw query in WHERE or select attributes sub-query conditions.
I am unable to find a way, where I have the subquery involved of the main model I will be using to execute findAndCountAll like, in this case, it will be let's say usersModel.findAndCountAll.
Also, I need to have JOIN for some other tables from which I need to select data (Not mentioned in the query).
How can I achieve the same using Sequalize?
Maybe you do not need a subquery at all? Here is the same query but flattened.
SELECT
"users"."id",
"users"."user_id" AS "peopleId",
"users"."first_name" AS "firstName",
"users"."last_name" AS "lastName",
"users"."email",
"users"."mobile",
"users"."status"
FROM "users"
LEFT OUTER JOIN location_view lv on "users".address_id = lv.addresses_id
LEFT OUTER JOIN "addresses" AS "addresses" ON "users"."address_id" = "addresses"."id"
LEFT OUTER JOIN "countries" AS "addresses->countries" ON "addresses"."country_id" = "addresses->countries"."id"
LEFT OUTER JOIN "work_info" AS "work_info" ON "users"."work_info_id" = "work_info"."id"
WHERE
"users"."status" = 'Active' AND lv.location = 'Some, Location, India'
ORDER BY
"users"."first_name" ASC
LIMIT
5 OFFSET 0;
I have preserved double quoting in order to be as close to your query as possible but please note that double quotes are only needed if the name contains capital letters and/or non-alphanumeric characters.
Related
I want to optimize these SQL queries using if-else but how I should use it? .
if this query result contain 'ALL'
SELECT
bdsubcategory.subcategoryID as ID,
bdsubcategory.subcategoryName as Name
FROM
phonebook.newsms_subscription
INNER JOIN bdsubcategory ON bdsubcategory.subcategoryID = newsms_subscription.subcategoryID
INNER JOIN newsms_client ON newsms_subscription.clientID =newsms_client.clientID
INNER JOIN newsms_person ON newsms_subscription.personID = newsms_person.personID
WHERE
newsms_subscription.isActive = 1 AND
newsms_person.personID = '856'
Then i want to query this
SELECT
bdsubcategory.subcategoryID as ID,
bdsubcategory.subcategoryName as Name
FROM
phonebook.newsms_subscription
INNER JOIN bdsubcategory ON bdsubcategory.subcategoryID = newsms_subscription.subcategoryID
INNER JOIN newsms_person ON newsms_subscription.personID = newsms_person.personID
WHERE
newsms_subscription.isActive = 1
GROUP BY subcategoryName
ORDER BY subcategoryName
otherwise take query1 result .
The problem is that if we do not refactor your project, then you always have to evaluate query1 and see whether it contains All or not. If it does not contain All, then you need to evaluate query2 as well. This can hardly be optimized, let's see a few approaches:
Quickening query1
Since All might be not be the very last evaluated element, adding it to the filter and limiting it is a good idea to quicken query1:
SELECT
COUNT(*)
FROM
phonebook.newsms_subscription
INNER JOIN bdsubcategory ON bdsubcategory.subcategoryID = newsms_subscription.subcategoryID
INNER JOIN newsms_client ON newsms_subscription.clientID =newsms_client.clientID
INNER JOIN newsms_person ON newsms_subscription.personID = newsms_person.personID
WHERE
newsms_subscription.isActive = 1 AND
newsms_person.personID = '856' AND
bdsubcategory.subcategoryName = 'ALL'
LIMIT 0, 1
So, you could create a stored procedure which evaluates query1' (query1' is the quickened version of query1, as seen above) and if there is a result, then we need to execute query1. Otherwise we need to execute query2. This way you still execute two queries, but the first query is optimized.
Refactoring
Note that the second query does not change. You could create a table where you could cache its results, using a periodic job. Then, you could skip the second table to
SELECT ID, Name
FROM MyNewTable;
without the many joins. You would also cache the results of the first query into a table where the items having ALL would be stored and query that table.
One option would be to use a CASE.
Change this:
newsms_person.personID = '856'
To this:
'Y' = CASE WHEN UPPER('856') = 'ALL' THEN 'Y'
WHEN newsms_person.personID = '856' THEN 'Y'
ELSE 'N' END
Alternatively, a stored procedure could be used to first validate whether the personID seems valid, then returns the appropriate data.
SELECT
TC_TXN.BOOKING_REF_ID,
TC_TXN.CREATION_TIME,
IFNULL(MAN_REQ.SUB_COMPANY_CODE,"") AS "SUB_COMPANY_CODE",
IFNULL(CONCAT(' / ',SUBSTRING_INDEX(SUB_COMP.BV_GROUP_FQN_NAME,'/',-1)),"") AS SUB_COMPANY_NAME,
"Company" AS "BILL_TO",
"" AS "REF",
BV_ENUM.VALUE,
MI_TXN.PRODUCT_CHARGES,
IFNULL(TC_TXN.VENDOR_PAYABLE_AMOUNT,0) AS VENDOR_AMOUNT,
IFNULL(MAN_REQ.TICKET_NO,(select TICKET_NO from TC_FLIGHTS_BOOK tfb join TC_TRANSACTION tcx on(tcx.TRANSACTION_ID=tfb.TRANSACTION_ID) join TC_FB_FARE_DETAILS tf on (tf.OID=tfb.OID) limit 1)) AS "TICKET_NO",
IFNULL(TC_FB.SP_PNR_NO,IFNULL(MAN_REQ.PNR_NO,"")) AS "PNR_NO",
MAN_REQ.TOUR_CODE,
TC_TXN.TRANSACTION_ID,
TC_TXN.REQUEST_ID,
MAN_REQ.EMPLOYE_ID,
"" AS "TCID"
FROm TC_TRANSACTION TC_TXN
LEFT JOIN MANUAL_INVOICE_TRANSACTION MI_TXN ON (TC_TXN.TRANSACTION_ID = MI_TXN.TRANSACTION_ID)
LEFT JOIN MANUAL_INVOICE_REQUEST MAN_REQ ON (MAN_REQ.REQUEST_ID = MI_TXN.REQUEST_ID)
LEFT JOIN TCP_ORGANIZATION SUB_COMP ON (SUB_COMP.ORGANIZATION_ID = TC_TXN.CUSTOMER_CODE)
LEFT JOIN BV_ENUM_VALUES BV_ENUM ON (TC_TXN.BOOKING_TYPE = BV_ENUM.INT_CODE AND BV_ENUM.TYPE_NAME = 'BOOKING_TYPE')
LEFT JOIN TC_FB_FLIGHT_DETAILS TC_FB ON (TC_FB.OID=TC_TXN.OID)
WHERE
TC_TXN.CREATION_TIME>=? and TC_TXN.CREATION_TIME<= ? AND TC_TXN.CURRENT_BOOKING_STATUS=0
GROUP BY TC_TXN.TRANSACTION_ID
when I am running this query for ticket_no column only 1st data is fetched. Remaining columns are fetching properly. i am using trying to solve this error for more than 2 days.
Thanks in advance
In the light of the comment by the OP ("for every row the ticket_no value is same"), the issue is that the subquery for the ticket_no field is not a correlated one. This means that its value does not depend on any value from the outer query:
...
IFNULL(MAN_REQ.TICKET_NO,(select TICKET_NO from TC_FLIGHTS_BOOK tfb join TC_TRANSACTION tcx on(tcx.TRANSACTION_ID=tfb.TRANSACTION_ID) join TC_FB_FARE_DETAILS tf on (tf.OID=tfb.OID) limit 1)) AS "TICKET_NO",
...
This also means that MAN_REQ.TICKET_NO field is always null, which begs the question why it is there in the first place.
Since the issue is with the data and the question does not describe how the tables are related, nor provides any sample data, I cannot provide an exact solution, only point out the root cause.
Can you use Doctrine QueryBuilder to INNER JOIN a temporary table from a full SELECT statement that includes a GROUP BY?
The ultimate goal is to select the best version of a record. I have a viewVersion table that has multiple versions with the same viewId value but different timeMod. I want to find the version with the latest timeMod (and do a lot of other complex joins and filters on the query).
Initially people assume you can do a GROUP BY viewId and then ORDER BY timeMod, but ORDER BY has no effect on GROUP BY, and MySQL will return random results. There are a ton of answers out there (e.g. here) that explain the problem with using GROUP and offer a solution, but I am having trouble interpreting the Doctrine docs to find a way to implement the SQL with Doctrine QueryBuilder (if it's even possible). Why don't I just use DQL? I may have to, but I have a lot of dynamic filters and joins that are much easier to do with QueryBuilder, so I wanted to see if that's possible.
Sample MySQL to Reproduce in Doctrine QueryBuilder
SELECT vv.*
FROM view_version vv
#inner join only returns where the result sets overlap, i.e. one record
INNER JOIN (
SELECT MAX(timeMod) maxTimeMod, viewId
FROM view_version
GROUP BY viewId
) version ON version.viewId = vv.viewId AND vv.timeMod = version.maxTimeMod
#join other tables for filter, etc
INNER JOIN view v ON v.id = vv.viewId
INNER JOIN content_type c ON c.id = v.contentTypeId
WHERE vv.siteId=1
AND v.contentTypeId IN (2)
ORDER BY vv.title ASC;
Theoretical Solution via Query Builder (not working)
I am thinking that the JOIN needs to inject a DQL statement, e.g.
$em = $this->getDoctrine()->getManager();
$viewVersionRepo = $em->getRepository('GutensiteCmsBundle:View\ViewVersion');
$queryMax = $viewVersionRepo->createQueryBuilder()
->addSelect('MAX(timeMod) AS timeModMax')
->addSelect('viewId')
->groupBy('viewId');
$queryBuilder = $viewVersionRepo->createQueryBuilder('vv')
// I tried putting the query in a parenthesis, to no avail
->join('('.$queryMax->getDQL().')', 'version', 'WITH', 'vv.viewId = version.viewId AND vv.timeMod = version.timeModMax')
// Join other Entities
->join('e.view', 'view')
->addSelect('view')
->join('view.contentType', 'contentType')
->addSelect('contentType')
// Perform random filters
->andWhere('vv.siteId = :siteId')->setParameter('siteId', 1)
->andWhere('view.contentTypeId IN(:contentTypeId)')->setParameter('contentTypeId', $contentTypeIds)
->addOrderBy('e.title', 'ASC');
$query = $queryBuilder->getQuery();
$results = $query->getResult();
My code (which may not match the above example perfectly) outputs:
SELECT e, view, contentType
FROM Gutensite\CmsBundle\Entity\View\ViewVersion e
INNER JOIN (
SELECT MAX(v.timeMod) AS timeModMax, v.viewId
FROM Gutensite\CmsBundle\Entity\View\ViewVersion v
GROUP BY v.viewId
) version WITH vv.viewId = version.viewId AND vv.timeMod = version.timeModMax
INNER JOIN e.view view
INNER JOIN view.contentType contentType
WHERE e.siteId = :siteId
AND view.contentTypeId IN (:contentTypeId)
ORDER BY e.title ASC
This Answer seems to indicate that it's possible in other contexts like IN statements, but when I try the above method in the JOIN, I get the error:
[Semantical Error] line 0, col 90 near '(SELECT MAX(v.timeMod)': Error: Class '(' is not defined.
A big thanks to #AdrienCarniero for his alternative query structure for sorting the highest version with a simple JOIN where the entity's timeMod is less than the joined table timeMod.
Alternative Query
SELECT view_version.*
FROM view_version
#inner join to get the best version
LEFT JOIN view_version AS best_version ON best_version.viewId = view_version.viewId AND best_version.timeMod > view_version.timeMod
#join other tables for filter, etc
INNER JOIN view ON view.id = view_version.viewId
INNER JOIN content_type ON content_type.id = view.contentTypeId
WHERE view_version.siteId=1
# LIMIT Best Version
AND best_version.timeMod IS NULL
AND view.contentTypeId IN (2)
ORDER BY view_version.title ASC;
Using Doctrine QueryBuilder
$em = $this->getDoctrine()->getManager();
$viewVersionRepo = $em->getRepository('GutensiteCmsBundle:View\ViewVersion');
$queryBuilder = $viewVersionRepo->createQueryBuilder('vv')
// Join Best Version
->leftJoin('GutensiteCmsBundle:View\ViewVersion', 'bestVersion', 'WITH', 'bestVersion.viewId = e.viewId AND bestVersion.timeMod > e.timeMod')
// Join other Entities
->join('e.view', 'view')
->addSelect('view')
->join('view.contentType', 'contentType')
->addSelect('contentType')
// Perform random filters
->andWhere('vv.siteId = :siteId')->setParameter('siteId', 1)
// LIMIT Joined Best Version
->andWhere('bestVersion.timeMod IS NULL')
->andWhere('view.contentTypeId IN(:contentTypeId)')->setParameter('contentTypeId', $contentTypeIds)
->addOrderBy('e.title', 'ASC');
$query = $queryBuilder->getQuery();
$results = $query->getResult();
In terms of performance, it really depends on the dataset. See this discussion for details.
TIP: The table should include indexes on both these values (viewId and timeMod) to speed up results. I don't know if it would also benefit from a single index on both fields.
A native SQL query using the original JOIN method may be better in some cases, but compiling the query over an extended range of code that dynamically creates it, and getting the mappings correct is a pain. So this is at least an alternative solution that I hope helps others.
I have the following query (and it works fine):
SELECT cd.id AS card_id,
ct.id AS category_id,
COUNT(cc.user_id) AS cnt
FROM uiCards AS cd
JOIN uiCardCategories AS ct USING (project_id)
LEFT JOIN uiCategories2Cards AS cc ON (cc.card_id = cd.id AND cc.stack_id = ct.id)
WHERE cd.project_id = $projID
GROUP BY cd.id, ct.id
ORDER BY cd.id, ct.id
I also have a sting of numbers:
$exclude = '100,122,345';
I need to modify the string too exclude results found in the string. So I added:
AND cc.user_id NOT IN ($exclude)
below WHERE
WHERE cd.project_id = $projID
AND cc.user_id NOT IN ($exclude)
It did not seem to work, so I tried to modify more, and the whole query collapsed on me.
UPDATE:
I got it! I added quotes:
AND (FIND_IN_SET(cc.user_id, '$exclude') = 0 OR FIND_IN_SET(cc.user_id, '$exclude') IS NULL)
The SQL IN clause doesn't allow a single variable to represent a list of values. The query, as-is, can only be run as dynamic SQL -- on any database. Even run dynamically, SQL will only interpret this example as a single string.
Secondly, because of using an OUTER JOIN (LEFT in this example), placement of criteria can drastically affect the results returned. Specifying criteria in the JOIN's ON clause will apply the criteria before the JOIN is made; using the WHERE clause means the criteria is applied after the JOIN, which could mean additional records you did not want included.
You could use the FIND_IN_SET function instead:
WHERE cd.project_id = $projID
AND (FIND_IN_SET(cc.user_id, $excluded) = 0 OR
FIND_IN_SET(cc.user_id, $excluded) IS NULL)
..vs in the LEFT JOIN criteria:
LEFT JOIN uiCategories2Cards AS cc ON cc.card_id = cd.id
AND cc.stack_id = ct.id
AND FIND_IN_SET(cc.user_id, $excluded) = 0
IN() requires a row set, but you are providing a string, so this won't work.
Use the function FIND_IN_SET() instead.
I am currently running this SQL
SELECT jm_recipe.name, jm_recipe.slug
FROM jm_recipe
LEFT JOIN jm_category_recipe ON jm_category_recipe.recipe_id = jm_recipe.id
WHERE jm_category_recipe.category_id = $cat"
This returns the desired results except that I also need to return the name of the category that the recipe I am looking for is in, to do this I tried to add the field in to my SELECT statement and also add the table into the FROM clause,
SELECT jm_recipe.name, jm_recipe.slug, jm_category_name
FROM jm_recipe, jm_category
LEFT JOIN jm_category_recipe ON jm_category_recipe.recipe_id = jm_recipe.id
WHERE jm_category_recipe.category_id = $cat"
However this just returns no results, what am i doing wrong?
You need to join both tables:
SELECT jm_recipe.name, jm_recipe.slug, jm.category_name
FROM jm_recipe
INNER JOIN jm_category_recipe ON jm_category_recipe.recipe_id = jm_recipe.id
INNER JOIN jm_category ON jm_recipe.recipe_id = jm_category.recipe_id
WHERE jm_category_recipe.category_id = $cat
I've changed the joins to inner joins as well. You might want to make them both LEFT joins if you have NULLs and want them in the result.
Also, you're vulnerable to SQL Injection by simply copying over $cat.
Here's some PHP specific info for you (I'm assuming you're using PHP.)