Optimizing Sub Query Getting Last Entry Based On Group - mysql

We had a bug in our code that cached the wrong values to the last_order_id (expected_previous_order) column. The query I wrote properly finds the correct last order id but is too slow for our data set.
I want to select this data into another table but I cannot because the query will take too long
I have setup a simple example here with a smaller data set. Original table has about 170k rows.
SQL Fiddle of my Example
In the example:
original_artwork_id is how these rows are grouped.
order_id is the current rows order id
actual_previous_order is the corrected last order id
expected_previous_order is the currently stored last order id. This is the wrong value as it does not actually reference the last order id
EXPLAIN Results
EDIT
Every time a reorder is placed a new entry is placed into the order_artwork table with a reference to the original_artwork_id and last_order_id.
The reference in the current data set to the last_order_id is wrong.
I need to update all records to properly indicate the last order id.
I am doing this by trying to find each artwork and joining it with the previous entry of the same original_artwork_id. Then I can pull the order_id from the last entry to update the current entries last_order_id
Join the current row with the previous row created before the current row with the same original_artwork_id or the current row original_artwork_id = the previous rows id

Not sure if this will be faster than your current query. But anyway.
SQL DEMO
First you need add a new field
`pos` int DEFAULT 0,
And update your base case so can do the JOIN.
update `order_artwork` o
SET `original_artwork_id` = `id`
WHERE `original_artwork_id` IS NULL;
You could use COALESCE(original_artwork_id, id) but cant use index on that case.
Then assign a row_number to each order based in original_artwork_id and date
update `order_artwork` o
left join (
SELECT o.id,
#rn := if(#order_id = `original_artwork_id`,
#rn + 1,
if(#order_id := `original_artwork_id`, 1, 1)
) as rn
FROM `order_artwork` o
CROSS JOIN (SELECT #id := 0, #order_id := 0, #rn := 0) as var
ORDER BY `original_artwork_id`,
`created`
) b on
o.id = b.id
set
o.pos = b.rn;
Finally update the last order.
UPDATE `order_artwork` o
JOIN (
SELECT o1.original_artwork_id,
o2.order_id,
o1.order_id as last_order_id
FROM `order_artwork` o1
LEFT JOIN `order_artwork` o2
ON o1.pos = o2.pos - 1
AND o1.original_artwork_id = o2.`original_artwork_id`
WHERE o2.pos IS NOT NULL
) as b
ON o.original_artwork_id = b.original_artwork_id
AND o.order_id = b.order_id
SET o.last_order_id = b.last_order_id;

I found that the created time column was not reliable. So I decided to just find the last highest order id with the same original_artwork_id.
Create a table that has the corrected values
CREATE TABLE order_artwork_two AS
select
d1.id,
d1.order_id,
max(d2.order_id) last_order_id,
d1.original_artwork_id
from order_artwork d1
left join order_artwork d2
ON d1.original_artwork_id = d2.original_artwork_id
and d1.order_id > d2.order_id
group by d1.original_artwork_id, d1.order_id;
Add an index to the new table. Otherwise the update would be way too slow
alter table order_artwork_two add primary KEY(id);
Update our original table.
update order_artwork d1
left join order_artwork_two d2 on d2.id = d1.id
set d1.last_order_id = d2.last_order_id;

Related

update an ordered sql table and set an incresing value

I use this code to order some virtuemart products how i want and the result is like the photo
SELECT m8jnx_virtuemart_product_categories.* ,m8jnx_virtuemart_products_el_gr.product_s_desc
FROM m8jnx_virtuemart_product_categories
INNER JOIN m8jnx_virtuemart_products_el_gr ON m8jnx_virtuemart_product_categories.virtuemart_product_id = m8jnx_virtuemart_products_el_gr.virtuemart_product_id
where `virtuemart_category_id` =272
GROUP BY `m8jnx_virtuemart_product_categories`.`virtuemart_product_id`
ORDER BY `m8jnx_virtuemart_products_el_gr`.`product_s_desc` desc
I want to set to the ordering an increasing value for this order I have do is this possible?
edit - I mean set in the first row ordering =1 in the second ordering=2, in third 3 etc -
Edit i found this sql queries in an other post but i don't know how to make work in my case that i use allready an inner join
UPDATE Test
SET Number = rowNumber
FROM Test
INNER JOIN
(SELECT ID, row_number() OVER (ORDER BY ID DESC) as rowNumber
FROM Test) drRowNumbers ON drRowNumbers.ID = Test.ID
edit The question is not duplicate I want to update the 'ordering' column that already exist not display only a column with an incresing value

MySQL to update subsequent duplicates of a row

We're going through a bit of a clean-up exercise and I need to remove duplicate data that has accidentally been added to our database table. The ID is obviously different, but other fields are the same.
I can use the following query to select the duplicate data sets:
SELECT user_id, start_datetime, count(id) AS dup_count
FROM our_table
WHERE status = 1
GROUP BY user_id, start_datetime
HAVING count(id) > 1;
What I need to do is create a query that would take each of the duplicate IDs APART FROM THE FIRST and use that to update the status to 0.
I'm not sure I can do this is one query, but I think the steps are as follows:
Run a query similar to the one above
Extract all the IDs for the duplicate sets
Ignore the first in the list as we don't want to alter the correctly added first record
Run the update on the remaining set of IDs
Am I out of luck here - or is it possible to do?
Many thanks!
You can do this with an update/join:
UPDATE our_table ot JOIN
(SELECT user_id, start_datetime, count(id) AS dup_count, min(id) as minid
FROM our_table
WHERE status = 1
GROUP BY user_id, start_datetime
HAVING count(id) > 1
) dups
ON ot.user_id = dups.user_id and
ot.start_datetime = dups.start_datetime and
ot.id > dups.minid
SET ot.status = 0;
You can use this update query that will join OUR_TABLE with itself:
UPDATE
our_table o1 INNER JOIN our_table o2
ON o1.status=1
AND o2.status=1
AND o1.user_id = o2.user_id
AND o1.start_datetime = o2.start_datetime
AND o1.ID > o2.ID
SET
o1.status = 0
Please see an example fiddle here.

order, group and search query. too many selects

i am running this query, witch consists of 3 recursive selects.
select idigorUserFields
from ( select *
from ( select *
from igorUserFields f
where f.idigorUsers = 1
order by f.idigorUserFields desc) tbl
group by tbl.idigorUserFieldTemplates ) tbl2
where value="qf" and idigorUserFields = 28
what I am trying to do is simple:
get all fields, order by insert date ( i am using primary key for that )
get the last inserted value for a field (idigorFieldTemplates)
compare the last inserted field with the one i am about to insert, to save some space on the database
some relevant info:
idigorUserFieldTemplates is the primary key for a "html user fields table"
the last inserted value is the one displayed on the program
also, I have a sqlfiddle! with some data to test.
my question is: can I make this query better? and what do I need to use to do that.
Alright so it looks like you want to get the last inserted value for a given user and value and then compare that with the data you are about to insert. Here's how I would tackle that
SELECT a.idigorUserFieldTemplates
FROM igorUserFields AS a
INNER JOIN (SELECT MAX(idigorUserFields) as max_id FROM igorUserFields WHERE idigorUsers = 1 AND value="qf") AS b
ON a.idigorUserFields= b.max_id
The subquery is giving me the largest idigorUserFields for which idigorUsers = 1 AND value="qf." For this to work, you have to assume that that the primary key (idigorUserFields) is incrementing with date. You indicated in your post that it does, so hopefully this assumption is okay.
Once we've got that last updated record, we then join back with igorUserFields to get the corresponding value of idigorUserFieldTemplates
edit:
I want to get the last inserted for a given user and field. then compare it with what i am about to insert.
I am sorry if I was not clear enough about what I wanted.
I modified the above sql and i got what i wanted:
SELECT a.* FROM igorUserFields
AS a INNER JOIN
(SELECT MAX(idigorUserFields) as max_id
FROM igorUserFields WHERE idigorUsers = 1 and idigorUserFieldTemplates =6) AS b
ON a.idigorUserFields= b.max_id
AND value="qf"
thanks! now I got a more efficient solution :)
if all you want is the largest id by specific data just specify that order it and limit it like so
SELECT idigorUserFields
FROM igorUserFields
WHERE idigorUsers = 1 AND value="qf"
ORDER BY idigorUserFields DESC
LIMIT 1
DEMO
if you are trying to get the last inserted value then you can get the largest idigorUserFields assuming its auto incremented.
SELECT MAX(idigorUserFields) FROM idigorUserFields
and then if you want a specific column from that field you could use it as a subquery
SELECT f.idigorUserFieldTemplates
FROM idigorUserFields f
WHERE f.idigorUserFields =
( SELECT MAX(idigorUserFields)
FROM idigorUserFields
WHERE idigorUsers = 1 AND value="qf"
)

using max and limit in a compund MySQL statement

I have a simple process I'm trying to do in a single SQL statement.
I've got a table of players (called tplayers) with columns indicating what their userid and tourneyid are, as well as a "playerpoints" column. I've also got a table called "tscores" which contains scores, a userid and column called "rankpoints" - I want to take the top 3 rows per player with the highest rankpoints and put that value in the corresponding user record in tplayers -- all for a specific tourneyid.
Here's the query:
update tplayers p set playerpoints=
(
select sum(b.mypoints) y from
(
select scorerankpoints as mypoints from tscores t where t.tourneyid=p.tourneyid and p.userid=t.userid and t.scorerankpoints>0 order by scorerankpoints desc limit 3
) as b
) where p.tourneyid='12'
This generates this error: Unknown column 'p.tourneyid' in 'where clause'
I'm basically looking to take the top 3 values of "scorerankpoints" from table tscores and put the summed value into a column in table tplayers called playerpoints,
and I want to do this for all players and scores who have the same tourneyid in their tables.
It appears that the inner reference to p.tourneyid is undefined... Is there a way to do this in a single statement or do I have to break it up?
MySQL has a problem resolving correlated references that are more than one layer deep. This is a hard one to fix.
The following uses variables to enumerate the rows and then choosing the right rows for aggregation in an update/join:
update tplayers p join
(select ts.userid, sum(ts.scorerankpoints) as mypoints
from (select ts.*,
#rn := if(#userid = userid, 1, #rn + 1) as rn,
#userid := #userid
from tscores ts cross join
(select #rn := 0, #userid := '') const
where ts.tourneyid = '12'
order by ts.userid, ts.scorerankpoints desc
) ts
where rn <= 3
) ts
on p.userid = ts.userid
set playerpoints = ts.mypoints
where p.tourneyid = '12' ;

MySQL Inner Join with where clause sorting and limit, subquery?

Everything in the following query results in one line for each invBlueprintTypes row with the correct information. But I'm trying to add something to it. See below the codeblock.
Select
blueprintType.typeID,
blueprintType.typeName Blueprint,
productType.typeID,
productType.typeName Item,
productType.portionSize,
blueprintType.basePrice * 0.9 As bpoPrice,
productGroup.groupName ItemGroup,
productCategory.categoryName ItemCategory,
blueprints.productionTime,
blueprints.techLevel,
blueprints.researchProductivityTime,
blueprints.researchMaterialTime,
blueprints.researchCopyTime,
blueprints.researchTechTime,
blueprints.productivityModifier,
blueprints.materialModifier,
blueprints.wasteFactor,
blueprints.maxProductionLimit,
blueprints.blueprintTypeID
From
invBlueprintTypes As blueprints
Inner Join invTypes As blueprintType On blueprints.blueprintTypeID = blueprintType.typeID
Inner Join invTypes As productType On blueprints.productTypeID = productType.typeID
Inner Join invGroups As productGroup On productType.groupID = productGroup.groupID
Inner Join invCategories As productCategory On productGroup.categoryID = productCategory.categoryID
Where
blueprints.techLevel = 1 And
blueprintType.published = 1 And
productType.marketGroupID Is Not Null And
blueprintType.basePrice > 0
So what I need to get in here is the following table with the columns below it so I can use the values timestamp and sort the entire result by profitHour
tablename: invBlueprintTypesPrices
columns: blueprintTypeID, timestamp, profitHour
I need this information with the following select in mind. Using a select to show my intention of the JOIN/in-query select or whatever that can do this.
SELECT * FROM invBlueprintTypesPrices
WHERE blueprintTypeID = blueprintType.typeID
ORDER BY timestamp DESC LIMIT 1
And I need the main row from table invBlueprintTypes to still show even if there is no result from the invBlueprintTypesPrices. The LIMIT 1 is because I want the newest row possible, but deleting the older data is not a option since history is needed.
If I've understood correctly I think I need a subquery select, but how to do that? I've tired adding the exact query that is above with a AS blueprintPrices after the query's closing ), but did not work with a error with the
WHERE blueprintTypeID = blueprintType.typeID
part being the focus of the error. I have no idea why. Anyone who can solve this?
You'll need to use a LEFT JOIN to check for NULL values in invBlueprintTypesPrices. To mimic the LIMIT 1 per TypeId, you can use the MAX() or to truly make sure you only return a single record, use a row number -- this depends on whether you can have multiple max time stamps for each type id. Assuming not, then this should be close:
Select
...
From
invBlueprintTypes As blueprints
Inner Join invTypes As blueprintType On blueprints.blueprintTypeID = blueprintType.typeID
Inner Join invTypes As productType On blueprints.productTypeID = productType.typeID
Inner Join invGroups As productGroup On productType.groupID = productGroup.groupID
Inner Join invCategories As productCategory On productGroup.categoryID = productCategory.categoryID
Left Join (
SELECT MAX(TimeStamp) MaxTime, TypeId
FROM invBlueprintTypesPrices
GROUP BY TypeId
) blueprintTypePrice On blueprints.blueprintTypeID = blueprintTypePrice.typeID
Left Join invBlueprintTypesPrices blueprintTypePrices On
blueprintTypePrice.TypeId = blueprintTypePrices.TypeId AND
blueprintTypePrice.MaxTime = blueprintTypePrices.TimeStamp
Where
blueprints.techLevel = 1 And
blueprintType.published = 1 And
productType.marketGroupID Is Not Null And
blueprintType.basePrice > 0
Order By
blueprintTypePrices.profitHour
Assuming you might have the same max time stamp with 2 different records, replace the 2 left joins above with something similar to this getting the row number:
Left Join (
SELECT #rn:=IF(#prevTypeId=TypeId,#rn+1,1) rn,
TimeStamp,
TypeId,
profitHour,
#prevTypeId:=TypeId
FROM (SELECT *
FROM invBlueprintTypesPrices
ORDER BY TypeId, TimeStamp DESC) t
JOIN (SELECT #rn:=0) t2
) blueprintTypePrices On blueprints.blueprintTypeID = blueprintTypePrices.typeID AND blueprintTypePrices.rn=1
You don't say where you are putting the subquery. If in the select clause, then you have a problem because you are returning more than one value.
You can't put this into the from clause directly, because you have a correlated subquery (not allowed).
Instead, you can put it in like this:
from . . .
(select *
from invBLueprintTypesPrices ibptp
where ibtp.timestamp = (select ibptp2.timestamp
from invBLueprintTypesPrices ibptp2
where ibptp.blueprintTypeId = ibptp2.blueprintTypeId
order by timestamp desc
limit 1
)
) ibptp
on ibptp.blueprintTypeId = blueprintType.TypeID
This identifies the most recent records for all the blueprintTypeids in the subquery. It then joins in the one that matches.