i have an SQL Requests:
SELECT DISTINCT id_tr
FROM planning_requests a
WHERE EXISTS(
SELECT 1 FROM planning_requests b
WHERE a.id_tr = b.id_tr
AND trainer IS NOT NULL
AND trainer != 'FREE'
)
AND EXISTS(
SELECT 1 FROM planning_requests c
WHERE a.id_tr = c.id_tr
AND trainer IS NULL
)
but this requests take 168.9490 sec to execute for returning 23162 rows of 2545088 rows
should i use LEFT JOIN or NOT IN ? and how can i rewrite it thx
You can speed this up by adding indexes. I would suggest: planning_requests(id_tr, trainer).
You can do this as:
create index planning_requests_id_trainer on planning_requests(id_tr, trainer);
Also, I think you are missing an = in the first subquery.
EDIT:
If you have a lot of duplicate values of id_tr, then in addition to the above indexes, it might make sense to phrase the query as:
select id_tr
from (select distinct id_tr
from planning_requests
) a
where . . .
The where conditions are being run on every row of the original table. The distinct is processed after the where.
I think your query can be simplified to this:
SELECT DISTINCT a.id_tr
FROM planning_requests a
JOIN planning_requests b
ON b.id_tr = a.id_tr
AND b.trainer IS NULL
WHERE a.trainer < 'FREE'
If you index planning_requests(trainer), then MySQL can utilize an index range to get all the rows that aren't FREE or NULL. All numeric strings will meet the < 'FREE' criteria, and it also won't return NULL values.
Then, use JOIN to make sure each record from that much smaller result set has a matching NULL record.
For the JOIN, index planning_requests(id_tr, trainer).
It might be simpler if you don't mix types in a column like FREE and 1.
Related
I have a query that selects ~8000 rows. When I execute the query it takes 0.1 sec.
When I copy the query into a view and execute the view it takes about 2 seconds. In the first row of explain it selects ~570K rows, i dont know why.
I dont understand the first Row and why it shows up only in the view explain
1 PRIMARY ALL NULL NULL NULL NULL
This is the query (yes i know im not a mysql pro and the query is not that efficent, but it works ans 0.1 sek would be ok for me. Does anyone know why it is so slow in a view?
MariaDB 10.5.9
select
`xxxxxxx`.`auftraege`.`Zustandigkeit` AS `Zustandigkeit`,
`xxxxxxx`.`auftraege`.`cms` AS `cms`,
`xxxxxxx`.`auftraege`.`auftrag_id` AS `auftrag_id`,
`xxxxxxx`.`angebot`.`angebot_id` AS `angebot_id`,
`xxxxxxx`.`kunden`.`kunde_id` AS `kid`,
`xxxxxxx`.`angebot`.`kunde_id` AS `kunde_id`,
`xxxxxxx`.`kunden`.`firma` AS `firma`,
`xxxxxxx`.`auftraege`.`gekuendigt` AS `gekuendigt`,
`xxxxxxx`.`kunden`.`ansprechpartnerVorname` AS `ansprechpartnerVorname`,
`xxxxxxx`.`kunden`.`ansprechpartner` AS `ansprechpartner`,
`xxxxxxx`.`auftraege`.`ampstatus` AS `ampstatus`,
`xxxxxxx`.`auftraege`.`autoMahnungen` AS `autoMahnungen`,
`xxxxxxx`.`kunden`.`mail` AS `mail`,
`xxxxxxx`.`kunden`.`ansprechpartnerAnrede` AS `ansprechpartnerAnrede`,
case
`xxxxxxx`.`kunden`.`ansprechpartnerAnrede`
when
'm'
then
concat('Herr ', ifnull(`xxxxxxx`.`kunden`.`ansprechpartnerVorname`, ''), ifnull(`xxxxxxx`.`kunden`.`ansprechpartner`, ''))
else
concat('Frau ', ifnull(`xxxxxxx`.`kunden`.`ansprechpartnerVorname`, ''), ifnull(`xxxxxxx`.`kunden`.`ansprechpartner`, ''))
end
AS `ansprechpartnerfullName`, `xxxxxxx`.`kunden`.`website` AS `website`, `xxxxxxx`.`personal`.`name_betrieb` AS `name_betrieb`, `xxxxxxx`.`kunden`.`prioritaet` AS `prioritaet`, `xxxxxxx`.`auftraege`.`infoemail` AS `infoemail`, `xxxxxxx`.`auftraege`.`keywords` AS `keywords`, `xxxxxxx`.`auftraege`.`ftp_h` AS `ftp_h`, `xxxxxxx`.`auftraege`.`ftp_u` AS `ftp_u`, `xxxxxxx`.`auftraege`.`ftp_pw` AS `ftp_pw`, `xxxxxxx`.`auftraege`.`lgi_h` AS `lgi_h`, `xxxxxxx`.`auftraege`.`lgi_u` AS `lgi_u`, `xxxxxxx`.`auftraege`.`lgi_pw` AS `lgi_pw`, `xxxxxxx`.`auftraege`.`autoRemind` AS `autoRemind`, `xxxxxxx`.`kunden`.`telefon` AS `telefon`, `xxxxxxx`.`kunden`.`mobilfunk` AS `mobilfunk`, `xxxxxxx`.`auftraege`.`kommentar` AS `kommentar`, `xxxxxxx`.`auftraege`.`phase` AS `phase`, `xxxxxxx`.`auftraege`.`datum` AS `datum`, `xxxxxxx`.`angebot`.`typ` AS `typ`,
case
`xxxxxxx`.`auftraege`.`gekuendigt`
when
'1'
then
'Ja'
else
'Nein'
end
AS `Gekuendigt ? `,
(
select
count(`xxxxxxx`.`status`.`aenderung`)
from
`xxxxxxx`.`status`
where
`xxxxxxx`.`status`.`auftrag_id` = `xxxxxxx`.`auftraege`.`auftrag_id`
)
AS `aenderungen`,
`xxxxxxx`.`auftraege`.`vertragStart` AS `vertragStart`,
`xxxxxxx`.`auftraege`.`vertragEnde` AS `vertragEnde`,
case
`xxxxxxx`.`auftraege`.`zahlungsart`
when
'U'
then
'Überweisung'
when
'L'
then
'Lastschrift'
else
'Unbekannt'
end
AS `Zahlungsart`, `xxxxxxx`.`kunden`.`yyyyy_piwik` AS `yyyyy_piwik`,
(
select
max(`xxxxxxx`.`status`.`datum`) AS `mxDTst`
from
`xxxxxxx`.`status`
where
`xxxxxxx`.`status`.`auftrag_id` = `xxxxxxx`.`auftraege`.`auftrag_id`
and `xxxxxxx`.`status`.`typ` = 'SEO'
)
AS `mxDTst`,
(
select
case
`xxxxxxx`.`rechnungen`.`beglichen`
when
'YES'
then
'isOk'
else
'isAffe'
end
AS `neuUwe`
from
(
`xxxxxxx`.`zahlungsplanneu`
join
`xxxxxxx`.`rechnungen`
on(`xxxxxxx`.`zahlungsplanneu`.`rechnungsnummer` = `xxxxxxx`.`rechnungen`.`rechnungsnummer`)
)
where
`xxxxxxx`.`zahlungsplanneu`.`auftrag_id` = `xxxxxxx`.`auftraege`.`auftrag_id`
and `xxxxxxx`.`rechnungen`.`beglichen` <> 'STO' limit 1
)
AS `neuer`,
(
select
group_concat(`xxxxxxx`.`kunden_keywords`.`keyword` separator ',')
from
`xxxxxxx`.`kunden_keywords`
where
`xxxxxxx`.`kunden_keywords`.`kunde_id` = `xxxxxxx`.`kunden`.`kunde_id`
)
AS `keyword`,
(
select
case
count(0)
when
0
then
'Cool'
else
'Uncool'
end
AS `AusfallVor`
from
`xxxxxxx`.`rechnungen`
where
`xxxxxxx`.`rechnungen`.`rechnung_tag` < current_timestamp() - interval 15 day
and `xxxxxxx`.`rechnungen`.`kunde_id` = `xxxxxxx`.`kunden`.`kunde_id`
and `xxxxxxx`.`rechnungen`.`beglichen` = 'NO' limit 1
)
AS `Liquidiert`
from
(
((((`xxxxxxx`.`auftraege`
join
`xxxxxxx`.`angebot`
on(`xxxxxxx`.`auftraege`.`angebot_id` = `xxxxxxx`.`angebot`.`angebot_id`))
join
`xxxxxxx`.`kunden`
on(`xxxxxxx`.`angebot`.`kunde_id` = `xxxxxxx`.`kunden`.`kunde_id`))
left join
`xxxxxxx`.`kunden_keywords`
on(`xxxxxxx`.`angebot`.`kunde_id` = `xxxxxxx`.`kunden_keywords`.`kunde_id`))
join
`xxxxxxx`.`personal`
on(`xxxxxxx`.`kunden`.`bearbeiter` = `xxxxxxx`.`personal`.`personal_id`))
left join
`xxxxxxx`.`status`
on(`xxxxxxx`.`auftraege`.`auftrag_id` = `xxxxxxx`.`status`.`auftrag_id`)
)
group by
`xxxxxxx`.`auftraege`.`auftrag_id`
order by
NULL
UPDATE 1
1. The View Itself (Duration 1.83 sec)
1.1 Create the View: This is the View i created, it only contains the query from above.
1.2 Executing the View: It takes 1.83 sek to execute the view
1.3 Analyze the View: This is the explain of the view
2. The view with added where clause (Duration 1.86 sec)
2.1 Analyze the View with added where clause #rick wanted me to add a where clause to the view, if i understood him correctly. This is the explain of the view, where i added a where clause, takes 1.86 sec.
3. The Query, that is the source of the view (Duration: 0.1 sec)
3.1 Execute the query directly This is the query, that is the source of the view, when i execute it directly to the server. It takes ~0.1 - 0.2 seconds.
3.2 Analyze the direct queryAnd this is the explain of the pure query.
Why the view is so much slower, by only cupsuling the query inside of the view?
Update 2
These are the indexes I have set
ALTER TABLE angebot ADD INDEX angebot_idx_angebot_id (angebot_id);
ALTER TABLE auftraege ADD INDEX auftraege_idx_auftrag_id (auftrag_id);
ALTER TABLE kunden ADD INDEX kunden_idx_kunde_id (kunde_id);
ALTER TABLE kunden_keywords ADD INDEX kunden_keywords_idx_kunde_id (kunde_id);
ALTER TABLE personal ADD INDEX personal_idx_personal_id (personal_id);
ALTER TABLE rechnungen ADD INDEX rechnungen_idx_rechnungsnummer_beglichen (rechnungsnummer,beglichen);
ALTER TABLE rechnungen ADD INDEX rechnungen_idx_beglichen_kunde_id_rechnung (beglichen,kunde_id,rechnung_tag);
ALTER TABLE status ADD INDEX status_idx_auftrag_id (auftrag_id);
ALTER TABLE status ADD INDEX status_idx_typ_auftrag_id_datum (typ,auftrag_id,datum);
ALTER TABLE zahlungsplanneu ADD INDEX zahlungsplanneu_idx_auftrag_id (auftrag_id);
Be consistent between tables. kunde_id, for example, seems to be declared differently between tables. This may be preventing some obvious optimizations. (There are 6 JOINs that say func in EXPLAIN`.)
Remove the extra parentheses in JOINs. They may be preventing what the Optimizer is happy to do -- rearrange the tables in a JOIN.
Turn the query inside out. By this, I mean to do the minimum amount of work to do the main JOIN. Collect mostly id(s). Then do the dependent subqueries in an outer select. Something like:
SELECT ... ( SELECT ... ), ...
FROM ( SELECT a1.id
FROM a AS a1
JOIN b ON ..
JOIN c ON .. )
JOIN a AS a2 ON a2.id = a1.id
JOIN d ON ...
The "inside-out" kludge may eliminate the need for the GROUP BY. (Your query is too complex for me to see for sure.) If so, then I call the problem "explode-implode" -- Your query first JOINs, producing a temp table with lots of rows ("explodes"). Then it does a GROUP BY ("implodes").
More
These indexes will probably help:
status: (auftrag_id, typ, datum, aenderung)
rechnungen: (beglichen, kunde_id, rechnung_tag)
rechnungen: (rechnungsnummer, beglichen)
zahlungsplanneu: (auftrag_id, rechnungsnummer)
kunden_keywords: (kunde_id, keyword) -- (unless `kunde_id` is the PK)
(I see from all 3 EXPLAINs that you probably have sufficient indexes on kunden_keywords and status. Show me what indexes you have, so I can see if the existing indexes are as good as my suggestions.) "Using index" == "covering index".
Near the end is this LEFT JOIN, but I did not spot any use for the table; perhaps it can be removed?
left join `kunden_keywords` on(`angebot`.`kunde_id` = `kunden_keywords`.`kunde_id`))
I have a query that gets data and also joins another table (A) and counts the rows in that join table (B). However if the main table (A) is empty I want the query to return nothing. However it is returning a result of null for id and date and an integer value of 0 for users instead of a null row. How do I get an empty result instead of it returning something?
Returning:
id | date | users
null | null | 0
SQL Code
SELECT
`sessions`.`id`,
`sessions`.`date`,
COUNT( sessions_users.id ) AS users
FROM
`sessions`
LEFT JOIN `sessions_users` ON `sessions`.`id` = `sessions_users`.`sessions_id`
An aggregate query without a group by clause always returns a single record, regardless of the content of the underlying result set (and even if it is empty).
But, since you have non-aggregated columns in the select clause (sessions.id and sessions.date), your query is missing a group by clause anyway. In non-ancient versions in MySQL, where sql mode ONLY_FULL_GROUP_BY is enabled by default, this is a syntax error.
Consider:
SELECT
`sessions`.`id`,
`sessions`.`date`,
COUNT( sessions_users.id ) AS users
FROM
`sessions`
LEFT JOIN `sessions_users` ON `sessions`.`id` = `sessions_users`.`sessions_id`
GROUP BY
`sessions`.`id`,
`sessions`.`date`
This will produce one record per session id and date, along with the count of matching records in sessions_users. If there are no records in sessions, the query will return an empty result set.
If I understand correctly, instead of NULL, you want something like this:
id | date | users
| | 0
If so, just simply use IFNULL() in your SELECT as such:
SELECT
IFNULL(`sessions`.`id`,' ') as id,
IFNULL(`sessions`.`date`,' ') as date,
....
There are also a few other ways to achieve this using just IF() or CASE .. but IFNULL is very straight forward.
BUT if you don't want to see any NULL and 0 values, change your LEFT JOIN to INNER JOIN and you're done.
From your description, it sounds like you want an inner join:
SELECT s.id, s.date, COUNT(*) as users
FROM sessions s JOIN
sessions_users su
ON su.id = su.sessions_id;
The query below gives me 2 out of the 3 answers I'm looking for. On the sub-query select I get null instead of no
the 3 possible values for column name isCyl could be blank, yes, no
I'm not sure if the sub-query is the best way to go about it, but I don't know how else to re-state the query.
The schedule table has a series of columns to show what tasks must be completed on an assignment. Related tables store the results of the tasks if they were assigned to be completed. So I need to test if a specific task was scheduled. If so, then I need to see if the results of the task have been recorded in the related table. For brevity I am only showing one of the columns here.
SELECT s.`reckey`,
if(s.cylinders="T",
(select
if(c.areckey is not null,
"yes",
"no"
)
from cylinders c where c.areckey = s.reckey limit 1
)
,""
) as isCyl
from schedule s
where s.assignmentDate between 20161015 and 20161016
order by s.reckey
Use a LEFT JOIN, which returns NULL for columns in the child table when there's no match.
SELECT s.reckey, IF(s.cylinders = "T",
IF(c.areckey IS NOT NULL, 'yes', 'no'),
"") AS isCyl
FROM schedule AS s
LEFT JOIN cylinders AS c ON c.areckey = s.reckey
WHERE s.assignmentDate between 20161015 and 20161016
ORDER BY s.reckey
If there can be multiple rows in cylinders with the same areckey, change it to:
LEFT JOIN (select distinct areckey FROM cylinders) AS c on c.areckey = s.reckey
or use SELECT DISTINCT in the main query.
I have this query:
select *
from transaction_batch
where id IN
(
select MAX(id) as id
from transaction_batch
where status_id IN (1,2)
group by status_id
);
The inner query runs very fast (less than 0.1 seconds) to get two ID's, one for status 1, one for status 2, then it selects based on primary key so it is indexed. The explain query says that it's searching 135k rows using where only, and I cannot for the life of me figure out why this is so slow.
The inner query is run seperatly for every row of your table over and over again.
As there is no reference to the outer query in the inner query, I suggest you split those two queries and just insert the results of the inner query in the WHERE clause.
select b.*
from transaction_batch b
inner join (
select max(id) as id
from transaction_batch
where status_id in (1, 2)
group by status_id
) bm on b.id = bm.id
my first post here.. sorry about the lack of formatting
I had a performance problem shown below:
90sec: WHERE [Column] LIKE (Select [Value] From [Table]) //Dynamic, slow
1sec: WHERE [Column] LIKE ('A','B','C') //Hardcoded, fast
1sec: WHERE #CSV like CONCAT('%',[Column],'%') //Solution, below
I had tried joining rather than subquerying.
I had also tried a hardcoded CTE.
I had lastly tried a temp table.
None of these standard options worked, and I was not willing to dosp_execute option.
The only solution that worked as:
DECLARE #CSV nvarchar(max) = Select STRING_AGG([Value],',') From [Table];
// This yields #CSV = 'A,B,C'
...
WHERE #CSV LIKE CONCAT('%',[Column],'%')
This query will be done in a cached autocomplete text box, possibly by thousands of users at the same time. What I have below works, bit I feel there may be a better way to do what I am doing.
Any advice?
UPDATED -- it can be 'something%':
SELECT a.`object_id`, a.`type`,
IF( b.`name` IS NOT NULL, b.`name`,
IF( c.`name` IS NOT NULL, c.`name`,
IF( d.`name` IS NOT NULL, d.`name`,
IF ( e.`name` IS NOT NULL, e.`name`, f.`name` )
)
)
) AS name
FROM `user_permissions` AS a
LEFT JOIN `divisions` AS b
ON ( a.`object_id` = b.`division_id`
AND a.`type` = 'division'
AND b.`status` = 1 )
LEFT JOIN `departments` AS c
ON ( a.`object_id` = c.`department_id`
AND a.`type` = 'department'
AND c.`status` = 1 )
LEFT JOIN `sections` AS d
ON ( a.`object_id` = d.`section_id`
AND a.`type` = 'section'
AND d.`status` = 1 )
LEFT JOIN `units` AS e
ON ( a.`object_id` = e.`unit_id`
AND a.`type` = 'unit'
AND e.`status` = 1 )
LEFT JOIN `positions` AS f
ON ( a.`object_id` = f.`position_id`
AND a.`type` = 'position'
AND f.`status` = 1 )
WHERE a.`user_id` = 1 AND (
b.`name` LIKE '?%' OR
c.`name` LIKE '?%' OR
d.`name` LIKE '?%' OR
e.`name` LIKE '?%' OR
f.`name` LIKE '?%'
)
Two simple, fast queries is often better than one huge, inefficient query.
Here's how I'd design it:
First, create a table for all your names, in MyISAM format with a FULLTEXT index. That's where your names are stored. Each of the respective object type (e.g. departments, divisions, etc.) are dependent tables whose primary key reference the primary key of the main named objects table.
Now you can search for names with this much simpler query, which runs blazingly fast:
SELECT a.`object_id`, a.`type`, n.name, n.object_type
FROM `user_permissions` AS a
JOIN `named_objects` AS n ON a.`object_id = n.`object_id`
WHERE MATCH(n.name) AGAINST ('name-to-be-searched')
Using the fulltext index will run hundreds of times faster than using LIKE in the way you're doing.
Once you have the object id and type, if you want any other attributes of the respective object type you can do a second SQL query joining to the table for the appropriate object type:
SELECT ... FROM {$object_type} WHERE object_id = ?
This will also go very fast.
Re your comment: Yes, I'd create the table with names even if it's redundant.
Other than changing the nested Ifs to use a Coalesce() function (MySql has Coalesce() doesn't it)? There is not much you can do as long as you need to filter on that input parameter with a like expresion. Putting a filter on a column using a Like expression, where the Like parameter has a wildcard at the begining, as you do, makes the query argument non-SARG-able, which means that the query processor must do a complete table scan of all the rows in the table to evaluate the filter predicate.
It cannot use an index, because an index is based on the column values, and with your Like parameter, it doesn't know which index entries to read from (since the parameter starts with a wild card)
if MySql has Coalesce, you can replace your Select with:
SELECT a.`object_id`, a.`type`,
Coalesce(n.name, c.name, d.Name, e.Name) name
If you can replace the search argument parameter so that it does not start with a wildcard, then just ensure that there is an index on the name column in each of the tables, and (if there are not indices on that column now), the query performance will increase enormously.
There are 500 things you can do. Optimize once you know where your bottlenecks are. Until then, work on getting those users onto your app. Its a much higher priority.