I have a survey app.
Each SurveyResponse has multiple QuestionResponses.
For reporting purposes I want to filter the QuestionResponses based on values answered for other QuestionResponses that are part of the same SurveyResponse - eg. Give me all the question responses of those that selected Male for the Genderquestion.
The following example SQL query is working, but I am struggling to replicate it in Laravel:
SELECT *
FROM ( SELECT *
FROM ( SELECT *
FROM question_responses ) AS A
JOIN ( SELECT survey_response_id AS JOIN_A
FROM question_responses
WHERE (question_short_name = 'Gender') AND (value = 'Male')) AS B
ON A.survey_response_id= JOIN_A) AS C
JOIN ( SELECT survey_response_id AS JOIN_B
FROM question_responses
WHERE (question_short_name = 'Age') AND (value = '45 to 54')) AS D
ON C.survey_response_id=JOIN_B
How would I do this using Eloquent or Laravel's query builder?
You can use DB::select method to retrieve result.
use Illuminate\Support\Facades\DB;
$results = DB::select("SELECT *
FROM ( SELECT *
FROM ( SELECT *
FROM question_responses ) AS A
JOIN ( SELECT survey_response_id AS JOIN_A
FROM question_responses
WHERE (question_short_name = ?) AND (value = ?)) AS B
ON A.survey_response_id= JOIN_A) AS C
JOIN ( SELECT survey_response_id AS JOIN_B
FROM question_responses
WHERE (question_short_name = ?) AND (value = ?)) AS D
ON C.survey_response_id= JOIN_B", ['Gender', 'Male', 'Age', '45 to 54']);
foreach ($results as $key => $row) {
print_r($row);
}
You can use the eloquent relationship, but if you want to use this particular type of query then you can use the DB class in laravel.
use Illuminate\Support\Facades\DB;
$data = DB::select(DB::raw('SELECT *
FROM ( SELECT *
FROM ( SELECT *
FROM question_responses ) AS A
JOIN ( SELECT survey_response_id AS JOIN_A
FROM question_responses
WHERE (question_short_name = 'Gender') AND (value = 'Male')) AS B
ON A.survey_response_id= JOIN_A.your_id) AS C
JOIN ( SELECT survey_response_id AS JOIN_B
FROM question_responses
WHERE (question_short_name = 'Age') AND (value = '45 to 54')) AS D
ON C.survey_response_id=JOIN_B.your_id'))
after getting the response you can retrieve the data by $data[0]['key'] for using '->' or making it collection you can use the Collection class.
use Illuminate\Support\Facades\DB;
use Illuminate\Database\Eloquent\Collection;
$tempData = Collection::make(DB::select(DB::raw('select query'));
($tempData[0])->key;
Related
I have a raw SQL query that I'm using in Laravel framework to get last presence of a user in a convocation table.
Basically, the master query is to get all the convocations of a given user that belongs to a specific ministry.
Like:
Select
`conv_pessoas.`id`,
`conv_pessoas`.`pessoa_id`,
`conv_pessoas`.`created_at`,
`conv_pessoas`.`data_chegada`,
`conv_pessoas`.`data_saida`,
`pessoas`.`nome`,
`ministerios`.`nome` as `ministerio_nome`,
`eventos`.`nome` as `evento_nome`,
`evento_data`.`data_inicial` as `data_evento_inicial`,
`evento_data`.`data_final` as `data_evento_final`,
(
SELECT
max(`convocacao_ministerio_pessoas`.`data_chegada`)
FROM
`convocacao_ministerio_pessoas`
WHERE
`convocacao_ministerio_pessoas`.`pessoa_id` = `conv_pessoas`.`pessoa_id`
AND `convocacao_ministerio_pessoas`.`data_chegada` IS NOT NULL
) as ultima_presenca_geral,
(
SELECT
max(`convocacao_ministerio_pessoas`.`data_chegada`)
FROM
`convocacao_ministerio_pessoas`
WHERE
`convocacao_ministerio_pessoas`.`pessoa_id` = `conv_pessoas`.`pessoa_id`
AND `convocacao_ministerio_pessoas`.`ministerio_id` = `conv_pessoas`.`ministerio_id`
AND `convocacao_ministerio_pessoas`.`data_chegada` IS NOT NULL
) as ultima_presenca_no_ministerio,
(
SELECT
count(*)
FROM
`convocacao_ministerio_pessoas`
WHERE
`convocacao_ministerio_pessoas`.`pessoa_id` = `conv_pessoas`.`pessoa_id`
AND `convocacao_ministerio_pessoas`.`ministerio_id` = `conv_pessoas`.`ministerio_id`
) as qtd_convocacoes_no_ministerio,
(
SELECT
count(*)
FROM
`convocacao_ministerio_pessoas`
WHERE
`convocacao_ministerio_pessoas`.`pessoa_id` = `conv_pessoas`.`pessoa_id`
AND `convocacao_ministerio_pessoas`.`ministerio_id` = `conv_pessoas`.`ministerio_id`
AND `convocacao_ministerio_pessoas`.`data_chegada` IS NOT NULL
) as qtd_presenca_no_ministerio
from
`convocacao_ministerio_pessoas` as `conv_pessoas`
inner join `pessoas` on `conv_pessoas`.`pessoa_id` = `pessoas`.`id`
inner join `convocacaos` on `conv_pessoas`.`convocacao_id` = `convocacaos`.`id`
inner join `ministerios` on `conv_pessoas`.`ministerio_id` = `ministerios`.`id`
inner join `eventos` on `convocacaos`.`evento_id` = `eventos`.`id`
inner join `evento_data` on `convocacaos`.`data_evento_id` = `evento_data`.`id`
where
`conv_pessoas`.`ministerio_id` = '2'
and `convocacaos`.`church_id` = 1
and `evento_data`.`data_final` between '2022-02-19 00:00:00'
and '2022-02-22 23:59:59'
But I need to get the last presence of this user in this same convocation table, but It's not necessary to be in the same ministry.
I've created subqueries for that.
But I'm not sure if it's the best approach.
I need some help to ensure I'm using the bests practices. ( I'm not advanced in SQL ).
**notes:**
Translations:
ministerio = ministry
convocacao = convocation
pessoa = person (user)
evento = event
data chegada = arrival date
I have the following SQL:
SELECT arv.*
FROM article_reference_versions arv
INNER JOIN (SELECT `order`,
Max(`revision`) AS max_revision
FROM article_reference_versions
WHERE `file` = '12338-230180-1-CE.doc'
GROUP BY `file`,
`order`) AS b
ON arv.order = b.order
AND arv.revision = b.max_revision
WHERE arv.file = '12338-230180-1-CE.doc'
I need to convert this to Eloquent, so that I can properly access the data in object form. I tried doing it as such,
$s = Models\EloArticleReferenceVersion::select(
'SELECT arv.*
FROM article_reference_versions arv
INNER JOIN (
SELECT `order`, max(`revision`) as max_revision
FROM article_reference_versions
WHERE file = ? group by `file`, `order`) AS b
ON
arv.order = b.order AND arv.revision = b.max_revision
WHERE arv.file = ?',
[
'12338-230180-1-CE.doc',
'12338-230180-1-CE.doc'
])->get();
dd($s);
But I'm running into a plethora of issues, one after another. I figured it'd be easier to just convert this into an eloquent query, looking for some help with this.
DB Query to Query using Eloquent.
$query = EloArticleReferenceVersion::query()
->join(DB::raw('( SELECT `order`,Max(`revision`) AS max_revision FROM article_reference_versions WHERE `file` = '12338-230180-1-CE.doc' GROUP BY `file`, `order`) as sub_table'), function($join) {
$join->on('sub_table.order', '=', 'article_reference_versions.order');
$join->on('sub_table.max_revision ', '=', 'article_reference_versions.revision');
})
->where('article_reference_versions.file', '=', '12338-230180-1-CE.doc' )
->get();
Not Tested
I have two tables T1 1 000 records and T2 with 500 000 records. I have a query where I run a join between them and fetch data by performing some aggregations. My page seems to be loading slow. Are there any approaches to make this query faster?
I have created indexes on columns for which aggregations are being performed. I think it is a generic statement.
$query = Mymodel::selectRaw("supplier_data.name as distributor,supplier_data.name as name, supplier_data.group_id as group_id, supplier_data.pay,supplier_data.group_id as submitted_group_plan,supplier_data.group_id as group_id_string,
(SELECT sum(t.net_claim) AS trans_number
FROM transactions_data_new as t
JOIN `supplier_data` AS d ON `t`.`member_id` = `d`.`group_id`
WHERE
(
(
t.`submit_date`>= '$date_from' and t.`submit_date`<= '$date_to'
AND t.`member_id` = supplier_data.group_id
)
OR
(
(t.claim_status IS NULL)
AND
(t.submit_date is NULL)
)
)
AND d.id = supplier_data.id
) as trans_number,
(SELECT sum(t.claim) AS trans_number
FROM transactions_data_new as t
JOIN `supplier_data` AS d ON `t`.`member_id` = `d`.`group_id`
WHERE
(
(
t.`submit_date`>= '$date_from' and t.`submit_date`<= '$date_to'
AND t.`member_id` = supplier_data.group_id
)
OR
(
(t.claim_status IS NULL)
AND
(t.submit_date is NULL)
)
)
AND d.id = supplier_data.id
) as claim,
(SELECT sum(t.reversed) AS trans_number
FROM transactions_data_new as t
JOIN `supplier_data` AS d ON `t`.`member_id` = `d`.`group_id`
WHERE
(
(
t.`submit_date`>= '$date_from' and t.`submit_date`<= '$date_to'
AND t.`member_id` = supplier_data.group_id
)
OR
(
(t.claim_status IS NULL)
AND
(t.submit_date is NULL)
)
)
AND d.id = supplier_data.id
) as reversed,
(SELECT sum(t.reversal) AS trans_number
FROM transactions_data_new as t
JOIN `supplier_data` AS d ON `t`.`member_id` = `d`.`group_id`
WHERE
(
(
t.`submit_date`>= '$date_from' and t.`submit_date`<= '$date_to'
AND t.`member_id` = supplier_data.group_id
)
OR
(
(t.claim_status IS NULL)
AND
(t.submit_date is NULL)
)
)
AND d.id = supplier_data.id
) as reversal
");
I don't see the need of this too complex/repeated with same clauses and multiple sub selects for same table which can done using a single left join
SELECT
s.name AS distributor,
s.name AS name,
s.group_id AS group_id,
s.pay,
s.group_id AS submitted_group_plan,
s.group_id AS group_id_string,
SUM(t.net_claim) AS trans_number,
SUM(t.claim) AS claim,
SUM(t.reversed) reversed,
SUM(t.reversal) reversal
FROM
supplier_data s
LEFT JOIN transactions_data_new t
ON `t`.`member_id` = s.`group_id`
AND (
(
t.`submit_date` >= '$date_from'
AND t.`submit_date` <= '$date_to'
)
OR (
t.claim_status IS NULL
AND t.submit_date IS NULL
)
)
GROUP BY s.name,
s.group_id,
s.pay
As I understand it the chunk() method is for use when you need to work with a large dataset and take an action on that data chunk by chunk.
From your question, it sounds like you're performing a query then returning the data as JSON so to me, it doesn't sound like you're taking an action on your dataset that requires chunking.
If you want to break up the returned JSON data you should be instead looking at pagination.
You could apply pagination to your query like so:
$data = Inspector::latest('id')
->select('id', 'firstname', 'status', 'state', 'phone')
->where('firstname', 'LIKE', '%' . $searchtext . '%')
->paginate();
You can specify the size of each set by passing a number to the paginate method:
$data = Inspector::latest('id')
->select('id', 'firstname', 'status', 'state', 'phone')
->where('firstname', 'LIKE', '%' . $searchtext . '%')
->paginate(25);
If I've misunderstood and you did actually want to do the chunking, I believe you could do the following:
$data = Inspector::latest('id')
->select('id', 'firstname', 'status', 'state', 'phone')
->where('firstname', 'LIKE', '%' . $searchtext . '%')
->chunk(50, function($inspectors) {
foreach ($inspectors as $inspector) {
// apply some action to the chunked results here
}
});
Also, if you're returning an eloquent object it will be automatically cast to json so you don't need to perform json_encode() as far as I'm aware.
I have MySQL query currently selecting and joining 13 tables and finally grouping ~60k rows. The query without grouping takes ~0ms but with grouping the query time increases to ~1.7sec. The field, which is used for grouping is primary field and is indexed. Where could be the issue?
I know group by without aggregate is considered invalid query and bad practise but I need distinct base table rows and can not use DISTINCT syntax.
The query itself looks like this:
SELECT `table_a`.*
FROM `table_a`
LEFT JOIN `table_b`
ON `table_b`.`invoice` = `table_a`.`id`
LEFT JOIN `table_c` AS `r1`
ON `r1`.`invoice_1` = `table_a`.`id`
LEFT JOIN `table_c` AS `r2`
ON `r2`.`invoice_2` = `table_a`.`id`
LEFT JOIN `table_a` AS `i1`
ON `i1`.`id` = `r1`.`invoice_2`
LEFT JOIN `table_a` AS `i2`
ON `i2`.`id` = `r2`.`invoice_1`
JOIN `table_d` AS `_u0`
ON `_u0`.`id` = 1
LEFT JOIN `table_e` AS `_ug0`
ON `_ug0`.`user` = `_u0`.`id`
JOIN `table_f` AS `_p0`
ON ( `_p0`.`enabled` = 1
AND ( ( `_p0`.`role` < 2
AND `_p0`.`who` IS NULL )
OR ( `_p0`.`role` = 2
AND ( `_p0`.`who` = '0'
OR `_p0`.`who` = `_u0`.`id` ) )
OR ( `_p0`.`role` = 3
AND ( `_p0`.`who` = '0'
OR `_p0`.`who` = `_ug0`.`group` ) ) ) )
AND ( `_p0`.`action` = '*'
OR `_p0`.`action` = 'read' )
AND ( `_p0`.`related_table` = '*'
OR `_p0`.`related_table` = 'table_name' )
JOIN `table_a` AS `_e0`
ON ( ( `_p0`.`related_id` = 0
OR `_p0`.`related_id` = `_e0`.`id`
OR `_p0`.`related_user` = `_e0`.`user`
OR `_p0`.`related_group` = `_e0`.`group` )
OR ( `_p0`.`role` = 0
AND `_e0`.`user` = `_u0`.`id` )
OR ( `_p0`.`role` = 1
AND `_e0`.`group` = `_ug0`.`group` ) )
AND `_e0`.`id` = `table_a`.`id`
JOIN `table_d` AS `_u1`
ON `_u1`.`id` = 1
LEFT JOIN `table_e` AS `_ug1`
ON `_ug1`.`user` = `_u1`.`id`
JOIN `table_f` AS `_p1`
ON ( `_p1`.`enabled` = 1
AND ( ( `_p1`.`role` < 2
AND `_p1`.`who` IS NULL )
OR ( `_p1`.`role` = 2
AND ( `_p1`.`who` = '0'
OR `_p1`.`who` = `_u1`.`id` ) )
OR ( `_p1`.`role` = 3
AND ( `_p1`.`who` = '0'
OR `_p1`.`who` = `_ug1`.`group` ) ) ) )
AND ( `_p1`.`action` = '*'
OR `_p1`.`action` = 'read' )
AND ( `_p1`.`related_table` = '*'
OR `_p1`.`related_table` = 'table_name' )
JOIN `table_g` AS `_e1`
ON ( ( `_p1`.`related_id` = 0
OR `_p1`.`related_id` = `_e1`.`id`
OR `_p1`.`related_user` = `_e1`.`user`
OR `_p1`.`related_group` = `_e1`.`group` )
OR ( `_p1`.`role` = 0
AND `_e1`.`user` = `_u1`.`id` )
OR ( `_p1`.`role` = 1
AND `_e1`.`group` = `_ug1`.`group` ) )
AND `_e1`.`id` = `table_a`.`company`
WHERE `table_a`.`date_deleted` IS NULL
AND `table_a`.`company` = 4
AND `table_a`.`type` = 1
AND `table_a`.`date_composed` >= '2016-05-04 14:43:55'
GROUP BY `table_a`.`id`
The ORs kill performance.
This composite index may help: INDEX(company, type, date_deleted, date_composed).
LEFT JOIN table_b ON table_b.invoice = table_a.id seems to do absolutely nothing other than slow down the processing. No fields of table_b are used or SELECTed. Since it is a LEFT join, it does not limit the output. Etc. Get rid if it, or justify it.
Ditto for other joins.
What happens with JOIN and GROUP BY: First, all the joins are performed; this explodes the number of rows in the intermediate 'table'. Then the GROUP BY implodes the set of rows.
One technique for avoiding this explode-implode sluggishness is to do
SELECT ...,
( SELECT ... ) AS ...,
...
instead of a JOIN or LEFT JOIN. However, that works only if there is zero or one row in the subquery. Usually this is beneficial when an aggregate (such as SUM) can be moved into the subquery.
For further discussion, please include SHOW CREATE TABLE.
I have this query
$query = "SELECT * FROM items WHERE itemStatus = '1' AND itemAdded > '$timestamp'";
Once this query has returned results I loop through the results
the results array is itemID, itemLocationID, categoryParentID, categoryID, itemName, itemDetails
During the loop I then run three other queries by calling functions within the same class
$locationName = $this->getLocationByName($locationID);
$categoryParentName = $this->getCategoryByName($categoryParentID);
$categoryName = $this->getCategoryByName($categoryID);
the function getLocationByName performs this query;
$q = mysql_query("SELECT * FROM locations WHERE locationID = '$locationID'");
this returns an array of locationID, locationName, locationLink
the function getCategoryByName performs this query;
$q = mysql_query("SELECT * FROM categories WHERE categoryID = '$categoryID'");
this returns an array of categoryID, categoryName, categoryLink
Could someone please help me optimize this query and maybe join them to save doing so many queries.
thanks in advance.
Im now using this query
$q = mysql_query("SELECT
i.itemID,
i.locationID,
i.categoryParentID,
i.categoryID,
i.itemName,
i.itemDetails,
l.*,
c.*
FROM
items i
inner join locations l on i.locationID = l.locationID
inner join categories c on i.categoryID = c.categoryID
WHERE
itemStatus = '1'
AND itemAdded > '$timestamp'")or die(mysql_error());
and the result is
Array
(
[itemID] => 81300
[locationID] => 17
[categoryParentID] => 21
[categoryID] => 183
[itemName] => blah
[itemDetails] => blah
[locationName] => brilliant it pulls in the location as expected.
[locationLink] => blah
[categoryName] => brilliant it pulls in the category as expected.
[categoryLink] => blah
)
[categoryName] => //these are missing for categoryParentID
[categoryLink] => //these are missing for categoryParentID
I would not use * in the select statement
The Query with the joins could be
SELECT
i.itemID,
i.itemLocationID,
i.categoryParentID,
i.categoryID,
i.itemName,
i.itemDetails,
l.*,
c.*
FROM
items i
inner join locations l on i.itemLocationID = l.locationID
inner join categories c on i.categoryID = c.categoryID
WHERE
itemStatus = '1'
AND itemAdded > '$timestamp'
I hope it be useful for you.
Cheers!
I think should be something similar to below query. I do not see where are you using $categoryParentName .
Using your queries and data:
SELECT * FROM items WHERE itemStatus = '1' AND itemAdded > '$timestamp'
SELECT * FROM locations WHERE locationID = '$locationID'
SELECT * FROM categories WHERE categoryID = '$categoryID'
$locationName = $this->getLocationByName($locationID);
$categoryParentName = $this->getCategoryByName($categoryParentID);
$categoryName = $this->getCategoryByName($categoryID);
Please let me know if this returns expected result set. Hope this helps
SELECT
it.itemID, it.itemLocationID, it.categoryParentID, it.categoryID, it.itemName, it.itemDetails,
l.locationID, l.locationName, l.locationLink,
c.categoryID, c.categoryName, c.categoryLink
FROM items it
LEFT JOIN locations l ON l.locationID = it.itemLocationID
LEFT JOIN categories c ON c.categoryID = it.categoryID
WHERE
it.itemStatus = '1'
AND it.itemAdded > '$timestamp'
Update query using categoryParentID - i am not saying is efficient but you can test and optimize as needed.
One option is to update above query - not sure that will work - and for large result sets using OR is not efficient:
LEFT JOIN categories c ON (c.categoryID = it.categoryID OR c.categoryID = it.categoryParentID)
The other option that i see is to get 2 result sets (see below) - one for categId = categId and second for categId = categParentId and combine the result set in one big result set.
SELECT
t.itemID, t.itemLocationID, t.categoryParentID, t.categoryID, t.itemName, t.itemDetails,
l.locationID, l.locationName, l.locationLink,
t.categoryID, t.categoryName, t.categoryLink
FROM
(
SELECT
it.itemID, it.itemLocationID, it.categoryParentID, it.categoryID, it.itemName, it.itemDetails,
c.categoryID, c.categoryName, c.categoryLink
FROM items it
INNER JOIN categories c ON c.categoryID = it.categoryID
WHERE
it.itemStatus = '1'
AND it.itemAdded > '$timestamp'
UNION -- [ALL]
SELECT
it.itemID, it.itemLocationID, it.categoryParentID, it.categoryID, it.itemName, it.itemDetails,
c.categoryID, c.categoryName, c.categoryLink
FROM items it
INNER JOIN categories c ON c.categoryID = it.categoryParentID
WHERE
it.itemStatus = '1' AND
it.itemAdded > '$timestamp'
) AS t
LEFT JOIN locations l ON l.locationID = t.itemLocationID
Other idea - not tested and assuming that id are int - will have to cast as string / char. There are a few options how you can write this query - if you post a structure table and some dummy data i am sure that someone will create a demo / sqlfiddle.
SELECT
it.itemID, it.itemLocationID, it.categoryParentID, it.categoryID, it.itemName, it.itemDetails,
l.locationID, l.locationName, l.locationLink,
c.categoryID, c.categoryName, c.categoryLink
FROM items it
LEFT JOIN locations l ON l.locationID = it.itemLocationID
WHERE
it.itemStatus = '1'
AND it.itemAdded > '$timestamp'
AND c.category ID IN ( SELECT GROUP_CONCAT(categs) FROM (SELECT CONCAT(categoryID, ",", categoryParentID) AS categs FROM items ))
Unless I'm missing something obvious, I'd probably suggest something like this as a first start:
select *
from items i
join locations l
on i.location_id=l.location_id
join categories c
on i.category_id=c.category_id
where item_status='1'
and itemAdded > '$timestamp'