Ransack: how to merge sub-queries? - mysql

I need to filter according to key + value in a polymorphic association. In this example, an Incident can have many CustomFieldsValues, and I need to filter Incidents by CustomFieldsValue.custom_field_id and CustomFieldsValue.value.
I tried creating a search like the following SQL query:
SELECT id FROM incidents
WHERE id IN (
SELECT customfieldable_id from custom_fields_values
WHERE (custom_fields_values.custom_field_id = 1091 AND (custom_fields_values.value >= '2015-06-29 10:00:00') AND (custom_fields_values.value <= '2015-07-03 09:59:59'))
)
AND id IN (
SELECT customfieldable_id from custom_fields_values
WHERE (custom_fields_values.custom_field_id = 1099 AND (custom_fields_values.value = 'bbb'))
);
Or this one with LEFT OUTER JOIN:
SELECT distinct incidents.id FROM incidents
LEFT OUTER JOIN custom_fields_values ON custom_fields_values.customfieldable_id = incidents.id AND custom_fields_values.customfieldable_type = 'Incident'
WHERE incidents.id IN (
SELECT customfieldable_id from custom_fields_values
WHERE (custom_fields_values.custom_field_id = 1091 AND (custom_fields_values.value >= '2015-06-29 10:00:00') AND (custom_fields_values.value <= '2015-07-03 09:59:59'))
)
AND incidents.id IN (
SELECT customfieldable_id from custom_fields_values
WHERE (custom_fields_values.custom_field_id = 1099 AND (custom_fields_values.value = 'bbb'))
);
I thought the Ransack query would look something like this:
Incident.ransack(:m=>"and", :g=>{
"0"=>{"id_in"=>{
"0"=>{:m=>"and", :g=>{
"0"=>{"custom_fields_values_custom_field_id_is"=>1091},
"1"=>{"custom_fields_values_value_gte_any"=>["2015-06-29 00:00:00"]},
"2"=>{"custom_fields_values_value_lte_any"=>["2015-07-02 23:59:59"]}
}}}},
"1"=>{"id_in"=>{
"0"=>{:m=>"and", :g=>{
"0"=>{"custom_fields_values_custom_field_id_is"=>1099},
"1"=>{"custom_fields_values_value_is_any"=>["bbb"]}
}}}}
})
Is there a way to include nested queries in Ransack? I tried merging searches, but got NoMethodError: undefined method 'base' for #<CustomFieldsValue::ActiveRecord_Relation:0x0000010fc45eb8>.
Is there another way of doing this?

Related

Problems with query speed when using a nested query for item count

When I add the nested query for invCount, my query time goes from .03 sec to 14 sec. The query works and I get correct values, but it is very, very slow in comparison. Is that just because I have to many conditions in that query? When I take it out and still have the second nested query, the time is still .03 secs. There is clearly something about the first nested query the database doesn't like, but I am not seeing what it is. I have a foreign key set for all the inner join lines too. Any help or ideas would be appreciated.
SELECT a.*,
f.name,
f.partNumber,
f.showInAdminStore,
f.showInPublicStore,
f.productImage,
r.mastCatID,
(SELECT COUNT(b.inventoryID)
FROM storeInventory b
INNER JOIN events c ON c.eventID = b.eventID
WHERE b.pluID = a.pluID
AND b.listPrice = a.listPrice
AND b.unlimitedQty = a.unlimitedQty
AND (b.packageID = a.packageID OR (b.packageID IS NULL AND a.packageID IS NULL))
AND b.orderID IS NULL
AND c.isOpen = '1'
AND b.paymentTypeID <= '2'
AND (b.inCart < '$cartTime' OR b.inCart IS NULL) ) AS invCount,
(SELECT COUNT(x.inventoryID)
FROM storeInventory x
WHERE x.packageID = a.inventoryID) AS packageCount
FROM storeInventory a
INNER JOIN storePLUs f ON f.pluID = a.pluID
INNER JOIN storeCategories r ON r.catID = f.catID
INNER JOIN events d ON d.eventID = a.eventID
WHERE a.storeFrontID = '1'
AND a.orderID IS NULL
AND a.paymentTypeID <= '2'
AND d.isOpen = '1'
GROUP BY a.packageID, a.unlimitedQty, a.listPrice, a.pluID
Table from query output
UPDATE: 12/12/2022
I changed the line checking the packageID to "AND (b.packageID <=> a.packageID)" as suggested and that cut my query time down to 7.8 seconds from 14 seconds. Thanks for the pointer. I will definitely use that in the future for NULL comparisons.
using "count(*)" took about half a second off. When I take the first nested query out, it drops down to .05 seconds even with the other nested queries in there, so I feel like there is still something causing issues. I tried running it without the other "AND (b.inCart < '$cartTime' OR b.inCart IS NULL)" line and that did take about a second off, but no where what I was hoping for. Is there an operand that includes NULL on a less than comparison? I also tried running it without the inner join in the nested query and that didn't change much at all. Of course removing any of that, throughs the values off and they become incorrect, so I can't run it that way.
Here is my current query setup that still pulls correct values.
SELECT a.*,
f.name,
f.partNumber,
f.showInAdminStore,
f.showInPublicStore,
f.productImage,
r.mastCatID,
(SELECT COUNT(*)
FROM storeInventory b
INNER JOIN events c ON c.eventID = b.eventID
WHERE b.pluID = a.pluID
AND b.listPrice = a.listPrice
AND b.unlimitedQty = a.unlimitedQty
AND (b.packageID <=> a.packageID)
AND b.orderID IS NULL
AND c.isOpen = '1'
AND b.paymentTypeID <= '2'
AND (b.inCart < '$cartTime' OR b.inCart IS NULL) ) AS invCount,
(SELECT COUNT(x.inventoryID)
FROM storeInventory x
WHERE x.packageID = a.inventoryID) AS packageCount
FROM storeInventory a
INNER JOIN storePLUs f ON f.pluID = a.pluID
INNER JOIN storeCategories r ON r.catID = f.catID
INNER JOIN events d ON d.eventID = a.eventID
WHERE a.storeFrontID = '1'
AND a.orderID IS NULL
AND a.paymentTypeID <= '2'
AND d.isOpen = '1'
GROUP BY a.packageID, a.unlimitedQty, a.listPrice, a.pluID
I am not familiar with the term 'Composite indexes' Is that something different than these?
Screenshot of ForeignKeys on Table a
I think
AND (b.packageID = a.packageID
OR (b.packageID IS NULL
AND a.packageID IS NULL)
)
can be simplified to ( https://dev.mysql.com/doc/refman/8.0/en/comparison-operators.html#operator_equal-to ):
AND ( b.packageID <=> a.packageID )
Use COUNT(*) instead of COUNT(x.inventoryID) unless you check for not-NULL.
The subquery to compute packageCount seems strange; you seem to count inventories but join on packages.
The need to reach into another table to check isOpen is part of the performance problem. If eventID is not the PRIMARY KEYforevents, then add INDEX(eventID, isOpen)`.
Some other indexes that may help:
a: INDEX(storeFrontID, orderID, paymentTypeID)
a: INDEX(packageID, unlimitedQty, listPrice, pluID)
b: INDEX(pluID, listPrice, unlimitedQty, orderID)
f: INDEX(pluID, catID)
r: INDEX(catID, mastCatID)
x: INDEX(packageID, inventoryID)
After OP's Update
There is no way to do (x<y OR x IS NULL) except by switching to a UNION. In your case, it is pretty easy to do the conversion. Replace
( SELECT COUNT(*) ... AND ( b.inCart < '$cartTime'
OR b.inCart IS NULL ) ) AS invCount,
with
( SELECT COUNT(*) ... AND b.inCart < '$cartTime' ) +
( SELECT COUNT(*) ... AND b.inCart IS NULL ) AS invCount,
Revised indexes:
storePLUs:
INDEX(pluID, catID)
storeCategories:
INDEX(catID, mastCatID)
events:
INDEX(isOpen, eventID)
storeInventory:
INDEX(pluID, listPrice, unlimitedQty, orderID, packageID)
INDEX(pluID, listPrice, unlimitedQty, orderID, inCart)
INDEX(packageID, inventoryID)
INDEX(storeFrontID, orderID, paymentTypeID)

Include another select column based on data - MySQL

How do I include SUM((pm.Quantity * bl.TotalQty)) AS NextBOMItemCount WHERE projectbomlist.ParentPartNum = bl.PartNum?
The data should not be changed, the same data should be retrieved, the however additional column has to be included.
VIEW: `NEWprojectBOMItemCount
select
`pm`.`ProjectCode` AS `ProjectCode`,
`bl`.`PartNum` AS `PartNum`,
sum((`pm`.`Quantity` * `bl`.`TotalQty`)) AS `BOMItemCount`,
`bl`.`mp` AS `mp`,
`p`.`complete` AS `complete`,
`bl`.`RMInd` AS `RMInd`,
`bl`.`M_PartNum` AS `M_PartNum`
from
(
(`projectmachine` `pm` join `projectbomlist` `bl`)
join `projects` `p`
)
where
(
(`pm`.`MachineListID` = `bl`.`MachineListID`)
and (`pm`.`ProjectCode` = `bl`.`ProjectCode`)
and (`pm`.`ProjectCode` = `p`.`ProjectCode`)
and (`p`.`AfterProjectHeirarchyInd` = 'Y')
)
and and pm.ProjectCode = 'AB212323'
group by
`pm`.`ProjectCode` ,
`bl`.`PartNum`
order by
`pm`.`ProjectCode` ,
`bl`.`PartNum`
Or, another option can be, please consider above view used in below query, please suggest changes to the below query as shown above (repeating here)
`sum((pm.Quantity * bl.TotalQty)) AS NextBOMItemCount where projectbomlist.ParentPartNum = bl.PartNum` - in place of `(select-NextBOMItemCount)`?
Please see PBLH.ParentPartNum is the column that I should compare with BL.ProjectCode to get NextBOMItemCount value.
QUERY calling view: NEWprojectBOMItemCount
Select
BL.PartNum PartNumber,
PBLH.ParentPartNum NextBOM,
(select-NextBOMItemCount),
BOMItemCount TotalQty,
PL.Description,
BL.MP as PartType,
PL.Vendor,
PL.QBType
from
NEWprojectBOMItemcount BL,
bomwiz.partslist PL,
bomwiz.projectbomlistheirarchy PBLH
Where
BL.PartNum = PL.PartNum
And BL.PartNum = PBLH.PartNum
And BL.ProjectCode = PBLH.ProjectCode
And BL.projectCode = 'AB212323'
Order By PartNumber
I think that you are looking for conditional aggregation. Your requirement could be expressed as follows:
SUM(
CASE WHEN blh.ParentPartNum = bl.PartNum
THEN pm.Quantity * bl.TotalQty
ELSE 0
END
) AS NextBOMItemCount
Let me pinpoint other issues with your query:
you have unwanted parentheses all around, and I am suspicious about the syntax of the JOINs ; you need to move conditions to the ON clause of the relevant JOIN.
every non-aggregated column must appear in the GROUP BY clause - you have missing columns there
backquotes are usually not needed
Here is an updated version of the query:
SELECT
pm.ProjectCode AS ProjectCode,
bl.PartNum AS PartNum,
SUM(pm.Quantity * bl.TotalQty) AS BOMItemCount,
SUM(
CASE WHEN blh.ParentPartNum = bl.PartNum
THEN pm.Quantity * bl.TotalQty
ELSE 0
END
) AS NextBOMItemCount,
bl.mp AS mp,
p.complete AS complete,
bl.RMInd AS RMInd,
bl.M_PartNum AS M_PartNum
FROM
projectmachine AS pm
INNER JOIN projectbomlist AS bl
ON pm.MachineListID = bl.MachineListID
AND pm.ProjectCode = bl.ProjectCode
INNER JOIN join projects AS p
ON pm.ProjectCode = p.ProjectCode
AND p.AfterProjectHeirarchyInd = 'Y'
INNER JOIN projectbomlistheirarchy blh
ON bl.ProjectCode = blh.ProjectCode
WHERE
pm.ProjectCode = 'AB212323'
GROUP BY
pm.ProjectCode,
bl.PartNum,
bl.mp,
p.complete,
bl.RMInd,
bl.M_PartNum
ORDER BY
pm.ProjectCode,
bl.PartNum

Not able to get the required ID's from a MySQL GROUP BY + HAVING query

I'm using a query various IN() clauses to get the price of product combinations. I am actually correctly getting the prices, so thats OK. But I also need some other ID's to identify to which ID's the prices belong.
As you can see IN's, they all start with a different bundle_variant_id's but the rest of the bundle_variant_id's are the same. I want to return that first (or all) of those bundle_variant_id's in the result too, but using this query, i always get '1620' as bundle_variant_id. How can i get the first bundle_variant_id in those IN's (1616, 1655, 1677, etc) or ALL of the bundle_variant_id's?
It's not an option to remove the GROUP and the HAVING or any of the WHERE clauses.
SELECT `HardwareProductSubscriptionInstanceLink`.`subscription_instance_id`
,`BundleVariantLink`.`bundle_variant_id`
,`HardwareProductSubscriptionInstanceLink`.`price`
FROM `wax`.`prd_providers` AS `Provider`
INNER JOIN `wax`.`prd_subscription_types` AS `SubscriptionType` ON (`SubscriptionType`.`provider_id` = `Provider`.`id`)
INNER JOIN `wax`.`prd_subscriptions` AS `Subscription` ON (`Subscription`.`subscription_type_id` = `SubscriptionType`.`id`)
INNER JOIN `wax`.`prd_subscription_instances` AS `SubscriptionInstance` ON (`SubscriptionInstance`.`subscription_id` = `Subscription`.`id`)
INNER JOIN `wax`.`prd_bundle_variants_subscription_instances` AS `BundleVariantLink` ON (`BundleVariantLink`.`subscription_instance_id` = `SubscriptionInstance`.`id`)
INNER JOIN `wax`.`prd_hardware_products_subscription_instances` AS `HardwareProductSubscriptionInstanceLink` ON (
`HardwareProductSubscriptionInstanceLink`.`subscription_instance_id` = `SubscriptionInstance`.`id`
AND `HardwareProductSubscriptionInstanceLink`.`hardware_product_id` = 317
)
WHERE (
(
`BundleVariantLink`.`bundle_variant_id` IN (
'1616'
,'1618'
,'1600'
,'1620'
)
)
OR (
`BundleVariantLink`.`bundle_variant_id` IN (
'1655'
,'1618'
,'1600'
,'1620'
)
)
OR (
`BundleVariantLink`.`bundle_variant_id` IN (
'1677'
,'1618'
,'1600'
,'1620'
)
)
OR (
`BundleVariantLink`.`bundle_variant_id` IN (
'1691'
,'1618'
,'1600'
,'1620'
)
)
AND `SubscriptionInstance`.`number_of_bundle_variants` = 4
AND (
(`Provider`.`enabled` = '1')
AND (`Provider`.`visible` = '1')
)
GROUP BY `SubscriptionInstance`.`id`
HAVING count(`BundleVariantLink`.`bundle_variant_id`) = 4
Current results:
(column 1 = subscription_instance_id, 2=bundle_variant_id,3=price)
9213 1620 331.405
9214 1620 311.57
9215 1620 291.736
9219 1620 390.909
Required results:
9213 1616 331.405
9214 1655 311.57
9215 1677 291.736
9219 1691 390.909
Or alternate required results:
9213 1616,1618,1600,1620 331.405
9214 1655,1618,1600,1620 311.57
9215 1677,1618,1600,1620 291.736
9219 1691,1618,1600,1620 390.909

SQL Query behavior

I'm bogged in trying to figure out why query a is returning different records than query b. Both queries have seemingly same purpose yet a is returning 500 and b 3500.
this is query a:
SELECT DISTINCT ODE.OrderBillToID
FROM APTIFY.dbo.vwVwOrderDetailsKGExtended ODE
WHERE ProductID IN (2022, 1393)
AND LTRIM(RTRIM(ODE.OrderStatus)) <> 'Cancelled'
AND LTRIM(RTRIM(ODE.OrderType)) <> 'Cancellation'
AND LTRIM(RTRIM(ODE.cancellationStatus)) <> 'FULLY CANCELLED'
UNION
SELECT DISTINCT ID
FROM APTIFY.dbo.vwPersons WHERE City = 'A'
UNION
SELECT DISTINCT RecordID
FROM APTIFY.dbo.vwTopicCodeLinks WHERE TopicCodeID = 16 AND Value = 'Yes, Please'
query b:
SELECT
APTIFY..vwPersons.ID
FROM
APTIFY..vwPersons
WHERE
( APTIFY..vwPersons.ID IN (
SELECT
vwMeetingRegistrants.ID
FROM
APTIFY.dbo.vwMeetings vwMeetings
INNER JOIN APTIFY.dbo.vwMeetingRegistrants vwMeetingRegistrants
ON vwMeetings.ID=vwMeetingRegistrants.ActualMeetingID WHERE
vwMeetings.ProductID = 2022
)
OR
APTIFY..vwPersons.ID IN (
SELECT
vwMeetingRegistrants.ID
FROM
APTIFY.dbo.vwMeetings vwMeetings
INNER JOIN APTIFY.dbo.vwMeetingRegistrants vwMeetingRegistrants
ON vwMeetings.ID=vwMeetingRegistrants.ActualMeetingID WHERE
vwMeetings.ProductID = 1393
)
OR
APTIFY..vwPersons.City = N'Albany' )
OR
((
APTIFY..vwPersons.ID IN (
SELECT
RecordID
FROM
APTIFY.dbo.vwTopicCodeLinks vwTopicCodeLinks
WHERE
vwTopicCodeLinks.TopicCodeID = 16
)
AND
APTIFY..vwPersons.ID IN (
SELECT
RecordID
FROM
APTIFY.dbo.vwTopicCodeLinks vwTopicCodeLinks
WHERE
vwTopicCodeLinks.Value = N'Yes, Please'
) )
)
vwMeetingsRegistrants from the b query are producing the same records as orderkgdetailsextended from query. I cannot see ANY difference in those queries - which perhaps shows my lack of understanding the query behaviour.
BIG Thanks for any points guys! :)
As it came out, incorrectly structured query is a result of badly configured application, Aptify.

mysql query - how to find items not within certain dates

I'm pretty useless at SQL it seems and I'm trying to figure out what is the correct query to use.
I have a table Items and a table Reservations. An item can have many reservations and reservations are made between two dates.
I'm trying to create a search query which will return all the Items which don't have reservations between two user inputted dates.
the SQL I have at the minute looks like:
SELECT `Item`.`id`, `Item`.`user_id`, `Item`.`name`, `Item`.`description`, `User`.`id`, `User`.`username`, `User`.`password`, `User`.`email`
FROM `database`.`items` AS `Item`
LEFT JOIN `database`.`reservations` AS `ReservationJoin` ON (`ReservationJoin`.`item_id` = `Item`.`id` AND `ReservationJoin`.`start` >= '2013-07-17' and `ReservationJoin`.`finnish` <= '2013-07-20')
LEFT JOIN `database`.`users` AS `User` ON (`Item`.`user_id` = `User`.`id`)
WHERE ((`Item`.`name` LIKE '%projector%') OR (`Item`.`description` LIKE '%projector%')
AND `ReservationJoin`.`id` IS NULL
LIMIT 10
I'm doing this in cakephp 2.3 so the code looks like:
$this->paginate = array(
"conditions" => array(
"or" => array(
"Item.name LIKE" => '%'.$projector.'%',
"Item.description LIKE" => '%'.$projector.'%',
),
"ReservationJoin.id" => null,
),
"joins" => array(
array(
"table" => "reservations",
"alias" => "ReservationJoin",
"type" => "LEFT",
"conditions" => array(
"ReservationJoin.item_id = Item.id",
"ReservationJoin.checkin >= '{$start}' and ReservationJoin.checkout <= '{$finnish}'",
)
)
),
"limit"=>10
);
$data = $this->paginate('Item');
This isn't working and I think it's to do with the join not excluding the reservations properly. But I've not been able to figure out what the correct mysql is. Can a kind soul tell me what I should be using?
thanks
If something has a reservation between two dates, then one of the following is true:
It has a start date between the dates
It has an end date between the dates
It has a start date before the earlier date and an end date after the second one
The following query uses this logic in a having clause. The approach is to aggregate at the item level and ensure that the three above conditions are true:
SELECT i.`id`, i.`user_id`, i.`name`, i.`description`
FROM `database`.`items`i LEFT JOIN
`database`.`reservations` r
ON r.`item_id` = i.`id`
WHERE ((i.`name` LIKE '%projector%') OR (i.`description` LIKE '%projector%')
group by i.id
having max(start between '2013-07-17' and '2013-07-20') = 0 and
max(finish between '2013-07-17' and '2013-07-20') = 0 and
max(start < '2013-07-17' and finished > '2013-07-20') = 0
LIMIT 10;
Note that none matches are returned, because the conditions are treated as false when start and ned are NULL.
I think it may be easier for you in CakePHP to put all of the conditions in a WHERE clause. (This could also be done with some OUTER JOINs but it may be difficult to transcribe into CakePHP)
SELECT id, user_id, name, description
FROM items
WHERE ((name LIKE '%projector%') OR (description LIKE '%projector%'))
AND NOT EXISTS(
SELECT *
FROM reservations
WHERE items.id = reservations.item_id
AND (
'2013-07-17' BETWEEN start and finnish
OR
'2013-07-20' BETWEEN start and finnish
OR
start BETWEEN '2013-07-17' AND '2013-07-20'));
Or, using the same logic the WHERE clause can be cleaned up to be
SELECT id, user_id, name, description
FROM items
WHERE ((name LIKE '%projector%') OR (description LIKE '%projector%'))
AND NOT EXISTS(
SELECT *
FROM reservations
WHERE items.id = reservations.item_id
AND NOT(
'2013-07-17' > finnish
OR
'2013-07-20' < start
));
you can try this.
SELECT `Item`.`id`, `Item`.`user_id`, `Item`.`name`, `Item`.`description`, `User`.`id`, `User`.`username`, `User`.`password`, `User`.`email`
FROM `database`.`items` AS `Item`
LEFT JOIN `database`.`reservations` AS `ReservationJoin` ON (`ReservationJoin`.`item_id` = `Item`.`id`)
LEFT JOIN `database`.`users` AS `User` ON (`Item`.`user_id` = `User`.`id`)
WHERE ((`Item`.`name` LIKE '%projector%') OR (`Item`.`description` LIKE '%projector%'))
AND `ReservationJoin`.`start` >= '2013-07-17'
AND `ReservationJoin`.`finnish` <= '2013-07-20'
AND `ReservationJoin`.`id` IS NULL
LIMIT 10
i.e just move the date clauses into the WHERE clause as oppose to being in the JOIN CLAUSE