Poor performance on joining two large tables - mysql

Question

I am struggling since several days with poor performance on joining two large tables. Maybe someone has a hint for me.
The one table is "broker_stock_data" which holds informations about purchases and sells of stocks of a client (this table is currently small but will grow bigger in future). To show the client the current price for his stocks there is the table "stock_data" which holds historical stock prices for a big amount of stocks (currently around 2million rows and growing). It´s a mariaDB/mysql Database and the table uses InnoDB.
Here are some informations about my tables:
broker_stock_data Table
stock_data Table
EXPLAIN Call on the SELECT
Schema of stock_data table
Having that in place I need to somehow get the latest price for each stock which is owned by a client. To do that I have the following query.
SELECT
`brokerStockData`.`id` AS `id`,
`brokerStockData`.`name` AS `name`,
`brokerStockData`.`symbol` AS `symbol`,
`brokerStockData`.`wkn` AS `wkn`,
`brokerStockData`.`modifyDate` AS `modifyDate`,
`brokerStockData`.`addDate` AS `addDate`,
`webApiConfig`.`id` AS `webApiConfigId`,
`webApiConfig`.`name` AS `webApiConfigName`,
`importError`.`msg` AS `importErrorMessage`,
SUM(`brokerStockData`.`purchaseAmount`) AS `purchaseAmount`,
stockData.stock_data_close AS `stockDataClose`,
stockData.stock_data_date AS `purchaseDate`,
stockData.stock_data_close * SUM(purchaseAmount) - SUM(purchasePrice * purchaseAmount) / SUM(purchaseAmount) * SUM(purchaseAmount) AS `difference`,
(
(stockData.stock_data_close * purchaseAmount - SUM(purchasePrice * purchaseAmount) / SUM(purchaseAmount) * purchaseAmount) / (SUM(purchasePrice * purchaseAmount) / SUM(purchaseAmount) * purchaseAmount)
)
* 100 AS `yield`,
SUM(purchasePrice * purchaseAmount) / SUM(purchaseAmount) AS `avgPurchasePrice`
FROM
`broker_stock_data` `brokerStockData`
INNER JOIN
`broker` `broker`
ON `broker`.`id` = `brokerStockData`.`brokerId`
INNER JOIN
`user` `user`
ON `user`.`id` = `broker`.`userId`
INNER JOIN
`webapi_configuration` `webApiConfig`
ON `webApiConfig`.`id` = `brokerStockData`.`webApiConfigId`
LEFT JOIN
(
SELECT
`stockData`.`date` AS `stock_data_date`,
`stockData`.`symbol` AS `stock_data_symbol`,
`stockData`.`close` AS `stock_data_close`,
`stockData`.`webApiConfigId` AS `stock_data_webApiConfigId`
FROM
`stock_data` `stockData`
WHERE
`stockData`.`date` IN
(
SELECT
MAX(`stockDataSQ`.`date`)
FROM
`stock_data` `stockDataSQ`
WHERE
`stockDataSQ`.`symbol` = `stockData`.`symbol`
GROUP BY
`stockDataSQ`.`symbol`
)
GROUP BY
`stockData`.`symbol`
)
`stockData`
ON `brokerStockData`.`symbol` = stock_data_symbol
AND `webApiConfig`.`id` = stock_data_webApiConfigId
LEFT JOIN
`import_log` `importError`
ON `importError`.`symbol` = `brokerStockData`.`symbol`
WHERE
`user`.`id` = 2
AND `broker`.`id` = 2
AND `brokerStockData`.`symbol` != ""
GROUP BY
`brokerStockData`.`symbol`
ORDER BY
`brokerStockData`.`name` ASC LIMIT 12
The problematic part is the LEFT JOIN on the stock_data table. Any Ideas on how to speed this up?
UPDATE
Changed the query since I copied a modified version of me :/
UPDATE2
Updated the EXPLAIN screenshot with the new query, sorry ;)

Answer 1

LEFT JOIN ( SELECT ... ) is likely to be very inefficient. Can you get rid of the LEFT?
Because of the LIMIT 12, a common trick to improve performance is to do the minimum work to find the 12 ids first. Then do whatever JOINs are needed to find all the other data.
After you have done that, I'll look at the indexes.
GROUP BY
`brokerStockData`.`symbol`
ORDER BY
`brokerStockData`.`name` ASC
Because the lists are different, it requires sorting twice. Change to this:
GROUP BY
`brokerStockData`.`name`, `brokerStockData`.`symbol`
ORDER BY
`brokerStockData`.`name` ASC, `brokerStockData`.`symbol` ASC
Assuming symbol and name are 1:1, you will get same results, but faster.