Mysql Group concat and then queries based on results - mysql

Hi all I have a table business which has alot of many to many relationships. It was suggested i perform a group concat first to get the ideas from the many table and then look at these ids to get the values from the many tables
In the below instance OK i can see i get a list of announcement ids back via the GROUP_CONCAT(DISTINCT ba.announcement_id) as 'announcement', how from here do i set
SELECT * from announcement
where id IN(_______)
where the in represents what was returned from the group_concat
id b
BEGIN
/* Business Information and Categories */
SELECT
b.alias_title, b.title, b.premisis_name,
a.address_line_1, a.address_line_2, a.postal_code,tvc.town_village_city,spc.state_province_county, c.country,
GROUP_CONCAT(DISTINCT be.event_id) as 'event',
GROUP_CONCAT(DISTINCT ba.announcement_id) as 'announcement',
GROUP_CONCAT(DISTINCT bd.document_id) as 'document',
GROUP_CONCAT(DISTINCT bi.image_id) as 'image',
GROUP_CONCAT(DISTINCT bprod.product_id ) as 'product',
GROUP_CONCAT(DISTINCT bt.tag_title_id) as 'tag'
FROM business AS b
INNER JOIN business_category bc_1 ON b.primary_category = bc_1.id
INNER JOIN business_category bc_2 ON b.secondary_category = bc_2.id
LEFT OUTER JOIN business_category bc_3 ON b.tertiary_category = bc_3.id
INNER JOIN address a ON b.address_id = a.id
LEFT OUTER JOIN town_village_city tvc ON a.town_village_city_id = tvc.id
LEFT OUTER JOIN state_province_county spc ON a.state_province_county_id
INNER JOIN country c ON a.country_id = c.id
LEFT OUTER JOIN geolocation g ON b.geolocation_id = g.id
LEFT OUTER JOIN business_event be ON b.id = be.event_id
LEFT OUTER JOIN business_announcement ba ON b.id = ba.announcement_id
LEFT OUTER JOIN business_document bd ON b.id = bd.business_id
LEFT OUTER JOIN business_image bi ON b.id = bi.business_id
LEFT JOIN business_property bp ON b.id= bp.business_id
LEFT JOIN business_product bprod ON b.id= bprod.business_id
LEFT JOIN business_tag bt ON b.id = bt.business_id
WHERE b.id= in_business_id;
SELECT * from announcement
where
END

In your first select statement you may assign the announcementId's to a variable and then use it to get all announcements in the second query:
set #announcementIds = '';
select ...........,
#announcementIds:= GROUP_CONCAT(DISTINCT announcement_id) as 'announcement',
...........;
Select * from announcement
where announcement_id REGEXP REPLACE(#announcementIds,',','|');
Some links:
Replace function
Regexp

Related

SQL: join select multiple tables with missing row

I have a big multiple table query join select where some values are optional.
This is the query:
SELECT a.date_in, a.date_out, b.name, b.phone, b.birthdate,
b.country, b.hotel, b.room_nr, b.passport_nr, c.email,
d.size, e.name, GROUP_CONCAT(DISTINCT g.service), GROUP_CONCAT(DISTINCT h.service)
, GROUP_CONCAT(DISTINCT i.time), GROUP_CONCAT(DISTINCT j.location)
FROM reservation a, rider b, user c,
bike_size d, bike e, services_reservation f, services g,
bike_shipping h, bike_shipping_reservation i
, bike_shipping_location j WHERE a.rider_id = b.id
AND b.user_id = c.id AND a.bike_size_id = d.id AND
d.bike_id = e.id AND a.id = f.reservation_id AND
f.services_id = g.id
AND h.id = i.bike_shipping_id AND a.id = i.reservation_id
AND i.bike_shipping_location_id = j.id
AND a.id = 80
In the tables from the query above, the table named services_reservation with the following columns (id, services_id, reservation_id) is completely empty in this case, which makes the values that I select from the table bike_shipping_reservation NULL.
How can I make some tables that I select from optional in case they are empty?
Here is the SQL Fiddle with 1 empty table, you can see the NULL results at the end (only GROUP_CONCAT(DISTINCT g.service) should be NULL).
http://sqlfiddle.com/#!9/ee31b/6
Here is the SQL Fiddle with all tables having values in there columns, you can see that all values are returned not NULL.
http://sqlfiddle.com/#!9/8bc033/34
Any thoughts?
Use left join where the table (or the row) are empty on don't match
SELECT a.date_in, a.date_out, b.name, b.phone, b.birthdate,
b.country, b.hotel, b.room_nr, b.passport_nr, c.email,
d.size, e.name, GROUP_CONCAT(DISTINCT g.service), GROUP_CONCAT(DISTINCT h.service)
, GROUP_CONCAT(DISTINCT i.time), GROUP_CONCAT(DISTINCT j.location)
FROM reservation a
INNER JOIN rider b on a.rider_id = b.id
INNER JOIN user c on b.user_id = c.id
INNER JOIN bike_size d on a.bike_size_id = d.id
INNER JOIN bike e ON d.bike_id = e.id
LEFT JOIN services_reservation f on a.id = f.reservation_id
INNER JOIN services g on f.services_id = g.id
INNER JOIN bike_shipping_reservation i on a.id = i.reservation_id
INNER JOIN bike_shipping h ON h.id = i.bike_shipping_id
INNER JOIN bike_shipping_location j on i.bike_shipping_location_id = j.id
where a.id = 80

MySQL query performance limit?

We have a home grown document management system and our system is running very slow, particularly on the search. It worked fine at first, but it has gotten progressively slower over time. Its now taking anywhere from 30 to 150 seconds to return results depending upon criteria. This is our search query. We’ve been staring at this thing left and right and can’t see any place to tune this more. All of the joined fields are indexed on their respective tables.
SELECT DISTINCT f.*, ts.*, fo.*, ft.*, p.*, u.*, c.*, co.*, ct.*, fs.*, fd.*, r.*, rt.*, si.*, s.* FROM (
SELECT DISTINCT f.* FROM files f
JOIN folders fo ON(fo.id = f.belongs_to_folder_id)
JOIN projects p ON(p.id = f.belongs_to_project_id)
LEFT OUTER JOIN file_statuses fs ON(fs.id = f.file_status_id)
LEFT OUTER JOIN submittal_items_files sif ON(sif.file_id = f.id)
LEFT OUTER JOIN submittal_items si ON(si.id = sif.submittal_item_id)
LEFT OUTER JOIN submittals s ON(s.id = si.belongs_to_submittal_id)
LEFT OUTER JOIN record_types rt ON(rt.id = f.record_type_id)
LEFT OUTER JOIN companies co ON(co.id = f.company_id)
LEFT JOIN folders_actions_groups ag ON (
f.belongs_to_folder_id = ag.folder_id AND
ag.action_id = 10010
)
LEFT JOIN files_actions_groups fg ON (fg.file_id = f.id)
JOIN users_groups ug ON ((ug.group_id = ag.group_id OR ug.group_id = fg.group_id) AND ug.user_id = 411)
WHERE (
(f.file_generated_name LIKE CONCAT('%', 'the', '%')) OR
(f.record_id LIKE CONCAT('%', 'the', '%')) OR
(f.record_title LIKE CONCAT('%', 'the', '%')) OR
(f.additional_info LIKE CONCAT('%', 'the', '%')) OR
(si.item_number LIKE CONCAT('%', 'the', '%')) OR
(s.element_number LIKE CONCAT('%', 'the', '%'))
) AND f.path LIKE CONCAT('Some Text', '%') AND
f.file_status_id = 3 AND
f.file_revision = 1 AND
f.discipline_id = 1 AND
f.record_type_id = 2 AND
f.triage_status_id = 2 AND
f.deleted = 0
ORDER BY f.created DESC, f.id DESC
LIMIT 100
) AS f
LEFT OUTER JOIN users u ON(f.created_by_user_id = u.id)
LEFT OUTER JOIN contacts c ON(c.user_id = u.id)
LEFT OUTER JOIN companies co ON(co.id = f.company_id)
LEFT OUTER JOIN company_types ct ON(ct.id = co.company_type_id)
JOIN triage_statuses ts ON(f.triage_status_id = ts.id)
JOIN folders fo ON(fo.id = f.belongs_to_folder_id)
JOIN folder_types ft ON(ft.id = fo.folder_type_id)
JOIN projects p ON(p.id = f.belongs_to_project_id)
LEFT OUTER JOIN file_statuses fs ON(fs.id = f.file_status_id)
LEFT OUTER JOIN file_disciplines fd ON(fd.id = f.discipline_id)
LEFT OUTER JOIN revisions r ON(r.id = f.file_revision)
LEFT OUTER JOIN record_types rt ON(rt.id = f.record_type_id)
LEFT OUTER JOIN submittal_items_files sif ON(sif.file_id = f.id)
LEFT OUTER JOIN submittal_items si ON(si.id = sif.submittal_item_id)
LEFT OUTER JOIN submittals s ON(s.id = si.belongs_to_submittal_id)
LEFT OUTER JOIN files_actions_groups ffg ON(ffg.file_id = f.id)
LEFT OUTER JOIN groups g ON(g.id = ffg.group_id)
ORDER BY f.created DESC, f.id DESC
This might be an obvious answer, but have you indexed your database? If you're new to indexing, here's a pretty good rule: just put a unique index on all the columns named "id", such as folders.id or projects.id, then put a standard index on all the columns that reference a foreign id, such as folder.belongs_to_folder_id or folder.record_type_id
Another thing I would change is to try and select only the columns you will actually use rather than your huge list of f.*, ts.*, fo.*, ft.*, p.*, u.*, c.*, co.*, ct.*, fs.*, etc...
You also have TONS of joins, which are very expensive in terms of processing time. Do you really need all those joined tables?

mysql select statement not returning rows where some tables in query aren't populated

I'm having a real mind blank - This code selects rows quite nicely apart from those entries where I change st.station_id from a value of '1' to a different (but still valid) number but where there are no entries for that station_id in either the station_owner_map table, the organisation table or the cap_gen_data_table. I basically need to amend my sql to ignore any table where there are no entries.
Select st.station_id, st.station_name , st.st_town, st.st_state, c1.country_name, o1.organisation_name, som1.equity, st.river_basin, st.cost, st.cost_ref, st.comm_year,cg1.caporgen, ht1.hydro_name, cg1.value, srs1.srs_description, cg1.ref_year
FROM station st
inner join station_country_map scm1 on st.station_id = scm1.station_id
inner join country c1 on scm1.country_id = c1.country_id
inner join station_owner_map som1 on st.station_id = som1.station_id
inner join organisation o1 on som1.owner_id = o1.org_id
inner join cap_gen_data cg1 on st.station_id = cg1.station_id
inner join value_lookup vl1 on cg1.caporgen = vl1.id
inner join hydro_type ht1 on cg1.hydro_type_id = ht1.type_id
inner join station_record_status srs1 on cg1.capacity_status = srs1.st_rec_stat_id
where st.station_id = 1
It's caused by your inner joins. Inner join means there has to be a value in both tables for the record to show up in the result set.
Use left join instead, then only the table 'on the left' has to have a value.
Use left join on tables where the value may not be present.
If you have two tables A and B an inner join will only return the rows from A where the join condition is met. A left join will return all rows from A regardless of if the join condition is satisfied. Columns in the select statement associated with B will be null when a left join is used.
I have only added the left join to the tables you have indicated. If other tables may not satisfy the join condition change the join type from inner to left.
Select st.station_id, st.station_name , st.st_town, st.st_state, c1.country_name, o1.organisation_name, som1.equity, st.river_basin, st.cost, st.cost_ref, st.comm_year,cg1.caporgen, ht1.hydro_name, cg1.value, srs1.srs_description, cg1.ref_year
FROM station st
inner join station_country_map scm1 on st.station_id = scm1.station_id
inner join country c1 on scm1.country_id = c1.country_id
left join station_owner_map som1 on st.station_id = som1.station_id
left join organisation o1 on som1.owner_id = o1.org_id
left join cap_gen_data cg1 on st.station_id = cg1.station_id
inner join value_lookup vl1 on cg1.caporgen = vl1.id
inner join hydro_type ht1 on cg1.hydro_type_id = ht1.type_id
inner join station_record_status srs1 on cg1.capacity_status = srs1.st_rec_stat_id
where st.station_id = 1

MySQL `INNER JOIN` multiples of the same table

Is it possible to INNER JOIN a MySQL query to achieve this result?
I have a table with Strategies and a table with Members. The Strategy table holds the ID of the author that corresponds to their ID in the Member table and the ID of an author that updated the existing author's work. Is it possible to grab a reference to both of these people at the same time? Something like the following, which returns no errors, but also no results...
SELECT * FROM Strategies
INNER JOIN Members AS a
INNER JOIN Members AS b
WHERE Strategies.ID='2'
AND Strategies.AuthorID = a.ID
AND Strategies.UpdateAuthorID = b.ID
Use a LEFT JOIN:
SELECT
s.*,
a.Name AS MemberName,
b.Name AS UpdatedMemberName
FROM Strategies AS s
LEFT JOIN Members AS a ON s.AuthorID = a.ID AND s.ID = 2
LEFT JOIN Members AS b ON s.UpdateAuthorID = b.ID AND s.ID = 2 ;
If you want them in one column use COALESCE:
SELECT
s.*,
COALESCE(a.Name, b.Name) AS MemberName
FROM Strategies AS s
LEFT JOIN Members AS a ON s.AuthorID = a.ID AND s.ID = 2
LEFT JOIN Members AS b ON s.UpdateAuthorID = b.ID AND s.ID = 2
SELECT toD.dom_url AS ToURL,
fromD.dom_url AS FromUrl,
rvw.*
FROM reviews AS rvw
LEFT JOIN domain AS toD
ON toD.Dom_ID = rvw.rev_dom_for
LEFT JOIN domain AS fromD
ON fromD.Dom_ID = rvw.rev_dom_from
if domain is table name

Using GROUP_CONCAT on multiple fields

Here's a picture of my database structure:
With help from users on here I've managed to put together quite a complex SQL statement using GROUP_CONCAT:
SELECT
t1.Name AS Teacher_Name,
t2.Name AS Observer_Name,
o.Datetime AS Datetime,
o.Type AS Type,
o.Year_Group AS Year_Group,
o.Class_Name AS Class_Name,
c.Title AS Course_Name,
GROUP_CONCAT(l.Title) AS Focus,
o.Achievement_Grade AS Achievement_Grade,
o.Behaviour_Grade AS Behaviour_Grade,
o.Teaching_Grade AS Teaching_Grade,
GROUP_CONCAT(cl1.Title) AS Positive,
GROUP_CONCAT(cl2.title) AS Development,
o.Notes AS Notes
FROM observations o
LEFT JOIN teachers t1
ON o.Teacher_ID = t1.Teacher_ID
LEFT JOIN teachers t2
ON o.Observer_ID = t2.Teacher_ID
LEFT JOIN courses c
ON o.Course_ID = c.Course_ID
LEFT JOIN foci f
ON o.ID = f.Observation_ID
LEFT JOIN focus_labels l
on f.focus_id = l.id
LEFT JOIN criteria c1
ON o.ID = c1.Observation_ID
LEFT JOIN criteria_labels cl1
on c1.Criteria_ID = cl1.ID AND c1.Type = 'P'
LEFT JOIN criteria c2
ON o.ID = c2.Observation_ID AND c2.Type = 'D'
LEFT JOIN criteria_labels cl2
on c2.Criteria_ID = cl2.ID
GROUP BY o.id
ORDER BY `Datetime` DESC";
This appears to work OK, apart from the fact that Focus, Positive and Development are each repeated depending on the field that has the highest number of concatenations in.
For example, if Positive has Pace,Progress,Attainment but Focus is only Appraisal, it'll be repeated three times (Appraisal,Appraisal,Appraisal).
I've looked this up and I think it could be because I need to GROUP each of these GROUP_CONCAT JOINs. However, I have no idea how to go about this.
Can anyone help? Thanks in advance,
GROUP_CONCAT has DISTINCT attribute that can be applied to remove duplicates.
SELECT
t1.Name AS Teacher_Name,
t2.Name AS Observer_Name,
o.Datetime AS Datetime,
o.Type AS Type,
o.Year_Group AS Year_Group,
o.Class_Name AS Class_Name,
c.Title AS Course_Name,
GROUP_CONCAT(DISTINCT l.Title) AS Focus,
o.Achievement_Grade AS Achievement_Grade,
o.Behaviour_Grade AS Behaviour_Grade,
o.Teaching_Grade AS Teaching_Grade,
GROUP_CONCAT(cl1.Title) AS Positive,
GROUP_CONCAT(cl2.title) AS Development,
o.Notes AS Notes
FROM observations o
LEFT JOIN teachers t1
ON o.Teacher_ID = t1.Teacher_ID
LEFT JOIN teachers t2
ON o.Observer_ID = t2.Teacher_ID
LEFT JOIN courses c
ON o.Course_ID = c.Course_ID
LEFT JOIN foci f
ON o.ID = f.Observation_ID
LEFT JOIN focus_labels l
on f.focus_id = l.id
LEFT JOIN criteria c1
ON o.ID = c1.Observation_ID
LEFT JOIN criteria_labels cl1
on c1.Criteria_ID = cl1.ID AND c1.Type = 'P'
LEFT JOIN criteria c2
ON o.ID = c2.Observation_ID AND c2.Type = 'D'
LEFT JOIN criteria_labels cl2
on c2.Criteria_ID = cl2.ID
GROUP BY o.id
ORDER BY `Datetime` DESC";