I have this query which pick ups the index I have created successfully.
Query:
select * from optima_contact as individual
unnest individual.individualRoles as individualRole
unnest individualRole.emailContacts as email
where individual.type = 'individual' and lower(email.emailAddress) = "giorgost#hotmail.com"
Index:
CREATE INDEX adv_ALL_individualRoles_emailContacts_lower_email_emailAddress_type ON `optima_contact`((all (array (all (array lower((`email`.`emailAddress`)) for `email` in (`individualRole`.`emailContacts`) end)) for `individualRole` in `individualRoles` end))) WHERE (`type` = 'individual')
Once I change to left unnest, the query doesn't pick up the index anymore.
select * from optima_contact as individual
unnest individual.individualRoles as individualRole
left unnest individualRole.emailContacts as email
where individual.type = 'individual' and lower(email.emailAddress) = "giorgost#hotmail.com"
Why is this happening?
That is expected because LEFT UNNEST. As it needs to project MISSING for email and can't use the index adv_ALL_individualRoles_emailContacts_lower_email_emailAddress_type
Only index it can use is
CREATE INDEX ix1 ON optima_contact(type);
Also if you are applying lower(email.emailAddress) = "giorgost#hotmail.com" filter in WHERE clause so LEFT UNNEST makes sense here.
Related
We have two tables one is properties and another one is property meta when we are getting data from one table "properties" , query only take less then one second in execution but when we are use join to get the data using bellow query from both tables its taking more then 5 second to fetch the data although we have only 12000 record in the tables , i think there is an issue in the sql query any help or suggestion will be appreciated.
SELECT
u.id,
u.property_title,
u.description,
u.city,
u.area,
u.address,
u.slug,
u.latitude,
u.longitude,
u.sync_time,
u.add_date,
u.is_featured,
u.pre_construction,
u.move_in_date,
u.property_status,
u.sale_price,
u.mls_number,
u.bedrooms,
u.bathrooms,
u.kitchens,
u.sub_area,
u.property_type,
u.main_image,
u.area_size as land_area,
pm7.meta_value as company_name,
pm8.meta_value as virtual_tour,
u.year_built,
u.garages
FROM
tbl_properties u
LEFT JOIN tbl_property_meta pm7
ON u.id = pm7.property_id
LEFT JOIN tbl_property_meta pm8
ON u.id = pm8.property_id
WHERE
u.status = 1
AND (pm7.meta_key = 'company_name')
AND (pm8.meta_key = 'virtual_tour')
AND (
(
( u.city = 'Delta'
OR u.post_code LIKE '%Delta%'
OR u.sub_area LIKE '%Delta%'
OR u.state LIKE '%Delta%')
AND country = 'Canada'
)
OR (
( u.city = 'Metro Vancouver Regional District'
OR u.post_code LIKE '%Metro Vancouver Regional District%'
OR u.sub_area LIKE '%Metro Vancouver Regional District%'
OR u.state LIKE '%Metro Vancouver Regional District%' )
AND country = 'Canada'
)
)
AND
u.pre_construction ='0'
GROUP BY
u.id
ORDER BY
u.is_featured DESC,
u.add_date DESC
Try adding this compound index:
ALTER TABLE tbl_property_meta ADD INDEX id_key (property_id, meta_key);
If it doesn't help make things faster, try this one.
ALTER TABLE tbl_property_meta ADD INDEX key_id (meta_key, property_id);
And, you should know that column LIKE '%somevalue' (with a leading %) is a notorious performance antipattern, resistant to optimization via indexes. (There's a way to create indexes for that shape of filter in PostgreSQL, but not in MariaDB / MySQL.)
Add another column with the meta stuff; throw city, post_code, sub_area, and state and probably some other things into it. Then build a FULLTEXT index on that column. Then use MATCH(..) AGAINST("Delta Metro Vancouver Regional District") in the WHERE clause _instead of the LEFT JOINs (which are actually INNER JOINs) and the really messy part of the WHERE clause.
Also, the GROUP BY is probably unnecessary, thereby eliminating extra sort on the intermediate set of rows.
My mysql query is too slow and i don't know how to optimize it. My webapp cant load this query because take too much time to run and the webserver have a limit time to get the result.
SELECT rc.trial_id,
rc.created,
rc.date_registration,
rc.agemin_value,
rc.agemin_unit,
rc.agemax_value,
rc.agemax_unit,
rc.exclusion_criteria,
rc.study_design,
rc.expanded_access_program,
rc.number_of_arms,
rc.enrollment_start_actual,
rc.target_sample_size,
(select name from repository_institution where id = rc.primary_sponsor_id) as
primary_sponsor,
(select label from vocabulary_studytype where id = rc.study_type_id) as study_type,
(select label from vocabulary_interventionassigment where id =
rc.intervention_assignment_id) as intervention_assignment,
(select label from vocabulary_studypurpose where id = rc.purpose_id) as study_purpose,
(select label from vocabulary_studymasking where id = rc.masking_id) as study_mask,
(select label from vocabulary_studyallocation where id = rc.allocation_id) as
study_allocation,
(select label from vocabulary_studyphase where id = rc.phase_id) as phase,
(select label from vocabulary_recruitmentstatus where id = rc.recruitment_status_id) as
recruitment_status,
GROUP_CONCAT(vi.label)
FROM
repository_clinicaltrial rc
inner JOIN repository_clinicaltrial_i_code rcic ON rcic.clinicaltrial_id = rc.id JOIN
vocabulary_interventioncode vi ON vi.id = rcic.interventioncode_id
GROUP BY rc.id;
Using inner join instead join could be a solution?
Changing to JOINs vs continuous selects per every row will definitely improve. Also, since you are using MySQL, using the keyword "STRAIGHT_JOIN" tells MySQL to do the query in the order I provided. Since your "rc" table is the primary and all the others are lookups, this will make MySQL use it in that context rather than hoping some other lookup table be the basis of the rest of the joins.
SELECT STRAIGHT_JOIN
rc.trial_id,
rc.created,
rc.date_registration,
rc.agemin_value,
rc.agemin_unit,
rc.agemax_value,
rc.agemax_unit,
rc.exclusion_criteria,
rc.study_design,
rc.expanded_access_program,
rc.number_of_arms,
rc.enrollment_start_actual,
rc.target_sample_size,
ri.name primary_sponsor,
st.label study_type,
via.label intervention_assignment,
vsp.label study_purpose,
vsm.label study_mask,
vsa.label study_allocation,
vsph.label phase,
vrs.label recruitment_status,
GROUP_CONCAT(vi.label)
FROM
repository_clinicaltrial rc
JOIN repository_clinicaltrial_i_code rcic
ON rc.id = rcic.clinicaltrial_id
JOIN vocabulary_interventioncode vi
ON rcic.interventioncode_id = vi.id
JOIN repository_institution ri
on rc.primary_sponsor_id = ri.id
JOIN vocabulary_studytype st
on rc.study_type_id = st.id
JOIN vocabulary_interventionassigment via
on rc.intervention_assignment_id = via.id
JOIN vocabulary_studypurpose vsp
ON rc.purpose_id = vsp.id
JOIN vocabulary_studymasking vsm
ON rc.masking_id = vsm.id
JOIN vocabulary_studyallocation vsa
ON rc.allocation_id = vsa.id
JOIN vocabulary_studyphase vsph
ON rc.phase_id = vsph.id
JOIN vocabulary_recruitmentstatus vrs
ON rc.recruitment_status_id = vrs.id
GROUP BY
rc.id;
One final note. You are using a GROUP BY and applying to the GROUP_CONCAT() which is ok. However, proper group by says you need to group by all non-aggregate columns, which in this case is every other column in the list. You may know this, and the fact the lookups will be the same based on the "rc" associated columns, but its not good practice to do so.
Your joins and subqueries are probably not the problem. Assuming you have correct indexes on the tables, then these are fast. "Correct indexes" means that the id column is the primary key -- a very reasonable assumption.
My guess is that the GROUP BY is the performance issue. So, I would suggest structuring the query with no `GROUP BY:
select . . .
(select group_concat(vi.label)
from repository_clinicaltrial_i_code rcic
vocabulary_interventioncode vi
on vi.id = rcic.interventioncode_id
where rcic.clinicaltrial_id = rc.id
)
from repository_clinicaltrial rc ;
For this, you want indexes on:
repository_clinicaltrial_i_code(clinicaltrial_id, interventioncode_id)
vocabulary_interventioncode(id, label)
I am using Sequalize ORM to fetch data from the database. I have below query to execute which runs fine in Postgres DB and I want to sequalize it, specifically, my query is related to how to sequalize the Sub-query part in FROM clause.
SELECT
"users"."id",
"users"."user_id" AS "peopleId",
"users"."first_name" AS "firstName",
"users"."last_name" AS "lastName",
"users"."email",
"users"."mobile",
"users"."status"
FROM (select * from users where address_id in (select addresses_id from location_view where location = 'Some, Location, India')) as "users"
LEFT OUTER JOIN "addresses" AS "addresses" ON "users"."address_id" = "addresses"."id"
LEFT OUTER JOIN "countries" AS "addresses->countries" ON "addresses"."country_id" = "addresses->countries"."id"
LEFT OUTER JOIN "work_info" AS "work_info" ON "users"."work_info_id" = "work_info"."id"
WHERE
"users"."status" = 'Active'
ORDER BY
"users"."first_name" ASC
LIMIT
5 OFFSET 0;
I know we do have Sequalize.query and literal methods to achieve this where I can directly specify my sub-query in raw format, but whatever examples I have googled out basically provides a way of how I can write a raw query in WHERE or select attributes sub-query conditions.
I am unable to find a way, where I have the subquery involved of the main model I will be using to execute findAndCountAll like, in this case, it will be let's say usersModel.findAndCountAll.
Also, I need to have JOIN for some other tables from which I need to select data (Not mentioned in the query).
How can I achieve the same using Sequalize?
Maybe you do not need a subquery at all? Here is the same query but flattened.
SELECT
"users"."id",
"users"."user_id" AS "peopleId",
"users"."first_name" AS "firstName",
"users"."last_name" AS "lastName",
"users"."email",
"users"."mobile",
"users"."status"
FROM "users"
LEFT OUTER JOIN location_view lv on "users".address_id = lv.addresses_id
LEFT OUTER JOIN "addresses" AS "addresses" ON "users"."address_id" = "addresses"."id"
LEFT OUTER JOIN "countries" AS "addresses->countries" ON "addresses"."country_id" = "addresses->countries"."id"
LEFT OUTER JOIN "work_info" AS "work_info" ON "users"."work_info_id" = "work_info"."id"
WHERE
"users"."status" = 'Active' AND lv.location = 'Some, Location, India'
ORDER BY
"users"."first_name" ASC
LIMIT
5 OFFSET 0;
I have preserved double quoting in order to be as close to your query as possible but please note that double quotes are only needed if the name contains capital letters and/or non-alphanumeric characters.
I'm thinking about creating an hashed index in (1) because it uses equalities and bit map on (2) because the state can only be 'accepted' or 'not accepted'. What else can i use? And also my problem is that i can only try b-tree indexes on mysql oracle..
(1)select R.user_id from rent as R
inner join supervise S on
R.adress = S.adress
and R.space_id = S.space_id
group by R.user_id
having count(distinct S.supervisor_id) = 1
(2) select distinct P.adress, P.code from space as P where (P.adress, P.code) not in (
select P.adress, P.code from space as P
natural join rent as R
natural join state as E where E.state = ‘accepted’)
Since there is no directly limiting criterias in query #1, it will likely be done using a merge join, and no index will improve that.
For query #2, how selective is the criteria E.state = 'accepted'? If very selective (< 5-15% of query result), then index on E.state, indexes for the joins from E to R and from R to P, and index on P.adress, P.code.
Composite index on each table:
INDEX(space_id, adress)
Don't use WHERE(a,b) IN ... -- it performs very poorly.
Don't use IN ( SELECT ... ) -- it often performs poorly.
Instead, use a JOIN.
For state, have
INDEX(state)
(or is it already the PRIMARY KEY?)
If you need more help after all that, provide SHOW CREATE TABLE and EXPLAIN SELECT ....
I am trying to build a rather complex view in MySQL and want to do a conditional, but it sems always to fail.
My view (simplified) is
Select entry AS Entry, ,(select count(`poitems`.`entry`) AS `count(poitems.entry)` from `poitems` where (`poitems`.`PurchaseOrder` = `purchaseorder`.`entry`)) AS `TotalEntries`, from purchase orders
this is OK but what I am trying to do is add something like
if ((select count(`poitems`.`entry`) = 0),'query.png',NULL) AS Queryflag
or just test the value of TotalEntries.
Help appreciated! Thanks!
I'm not 100% sure on the names of the columns in purchaseorder or poitems tables but the following should get you headed in the right direction:
select t.Entry,
case when t.TotalEntries > 0 then 'query.png' else null end as Queryflag
from
(
select po.entry as Entry,
count(*) as TotalEntries
from purchaseorder po
left outer join poitems poi on poi.purchaseorder = po.entry
group by po.entry
) t;