How to count specific column in Entity Framework Core - linq-to-sql

When I use left join to perform a grouped query of two tables, the generated count statement is always count(*), I want to generate the count of the specified column. For example: count(InstrumentCode).
Framework: EF Core 3.0-preview9
var resultQuery = from t1 in query
join t2 in orgCrmQuery on t1.CUSTOMER equals t2._ID
join t3 in orgQuery on t2.NAME equals t3.ORGANIZATION_NAME into t4
from t5 in t4.DefaultIfEmpty()
join t6 in instrumentQuery on t5.ORGANIZATION_ID equals t6.ORGANIZATION_ID into t7
from t8 in t7.DefaultIfEmpty()
where t2.RECORD_TYPE == dto.RecordType && t1.MEETING_NO == dto.MeetingNo
group t8.INSTRUMENT_CODE by new {
t1.PERSON_NAME,
t1.SEX,
t1.POSTION,
t1.PHONE,
t1.IS_CIZHU_KEHU,
t2.NAME,
t2.ORG_LEVEL,
} into g
orderby g.Key.NAME
select new {
g.Key.PERSON_NAME,
g.Key.SEX,
g.Key.POSTION,
g.Key.PHONE,
g.Key.IS_CIZHU_KEHU,
g.Key.NAME,
g.Key.ORG_LEVEL,
INSTRUMENT_COUNT = g.Count()
};
SQL statement generated:
SELECT
[t].[PERSON_NAME],
[t].[SEX],
[t].[POSTION],
[t].[PHONE],
[t].[IS_CIZHU_KEHU],
[t0].[NAME],
[t0].[ORG_LEVEL],
COUNT ( * ) AS [INSTRUMENT_COUNT]
FROM
[dbo].[TB_MEETING_PERSONAL] AS [t] WITH ( NOLOCK )
INNER JOIN [dbo].[TB_ORG_CRM] AS [t0] WITH ( NOLOCK ) ON [t].[CUSTOMER] = [t0].[_ID]
LEFT JOIN [dbo].[MS_ORGANIZATION] AS [m] WITH ( NOLOCK ) ON [t0].[NAME] = [m].[ORGANIZATION_NAME]
LEFT JOIN [dbo].[MS_INSTRUMENT] AS [m0] WITH ( NOLOCK ) ON [m].[ORGANIZATION_ID] = [m0].[ORGANIZATION_ID]
WHERE
(
( ( [t0].[RECORD_TYPE] = #__dto_RecordType_0 ) AND ( [t0].[RECORD_TYPE] IS NOT NULL AND #__dto_RecordType_0 IS NOT NULL ) )
OR ( [t0].[RECORD_TYPE] IS NULL AND #__dto_RecordType_0 IS NULL )
)
AND (
( ( [t].[MEETING_NO] = #__dto_MeetingNo_1 ) AND ( [t].[MEETING_NO] IS NOT NULL AND #__dto_MeetingNo_1 IS NOT NULL ) )
OR ( [t].[MEETING_NO] IS NULL AND #__dto_MeetingNo_1 IS NULL )
)
GROUP BY
[t].[PERSON_NAME],
[t].[SEX],
[t].[POSTION],
[t].[PHONE],
[t].[IS_CIZHU_KEHU],
[t0].[NAME],
[t0].[ORG_LEVEL]
ORDER BY
[t0].[NAME] OFFSET 0 ROWS FETCH NEXT 100 ROWS ONLY
I checked some information, but I didn't find a solution.

Based on your linq, same result for .Count(INSTRUMENT_COUNT) and .Count(*) or .Count(1) because this will return the # of rows regardless of the value of your INSTRUMENT_COUNT.
But if you want to get count specifically for INSTRUMENT_COUNT you can try this.
var resultQuery = from t1 in query
join t2 in orgCrmQuery on t1.CUSTOMER equals t2._ID
join t3 in orgQuery on t2.NAME equals t3.ORGANIZATION_NAME into t4
from t5 in t4.DefaultIfEmpty()
join t6 in instrumentQuery on t5.ORGANIZATION_ID equals t6.ORGANIZATION_ID into t7
from t8 in t7.DefaultIfEmpty()
where t2.RECORD_TYPE == dto.RecordType && t1.MEETING_NO == dto.MeetingNo
group t8.INSTRUMENT_CODE by new {
t1.PERSON_NAME,
t1.SEX,
t1.POSTION,
t1.PHONE,
t1.IS_CIZHU_KEHU,
t2.NAME,
t2.ORG_LEVEL,
t8.INSTRUMENT_CODE,
} into g
orderby g.Key.NAME
select new {
g.Key.PERSON_NAME,
g.Key.SEX,
g.Key.POSTION,
g.Key.PHONE,
g.Key.IS_CIZHU_KEHU,
g.Key.NAME,
g.Key.ORG_LEVEL,
INSTRUMENT_COUNT = g.Select(x => x.INSTRUMENT_CODE).Distinct().Count()
};

Related

Join 2 IDs from different tables

I have 2 queries that I'm union like this. But the problem is that ID from table1 and table2 can be different. how can I join them and make the same?
qry1 = (
select(
func.max(table1.id).label("id"),
table1.warehouse_id,
table1.product_id,
)
.group_by(
table1.warehouse_id,
table1.product_id,
)
.order_by(func.max(table1.id))
qry2 = (
select(func.max(table2.id).label("id"),
table2.warehouse_id,
table2.product_id,
.join(
table3,
and_(
table2.code == table3.code,
),
)
.group_by(
table2.warehouse_id,
table2.product_id,
)
.order_by(KcLpnSerialNumberDao.id)
)
stmt = qry1.union(qry2)
result = session.execute(stmt)

WITH recursive AS Errors were reported in MySQL 5.7

SELECT
p1.res_id,
p1.res_name,
p1.res_type,
p1.file_id,
p1.create_time,
p1.parent_id,
p1.parent_path,
t4.fullname AS create_user,
t5.orgshortname org_name,
t3.label_describe,
( CASE WHEN p1.res_type IN ( 'file' ) THEN NULL ELSE p3.oss_down_path END ) oss_down_path,
p3.file_size,
GROUP_CONCAT( p2.res_name ORDER BY p1.n SEPARATOR '/' ) AS all_parent_name
FROM
(
SELECT
t2.*,
SUBSTRING_INDEX( SUBSTRING_INDEX( t2.parent_path, ',', t1.n ), ',', - 1 ) AS linkid,
t1.n
FROM
tmp t1
CROSS JOIN tb_vk_resources t2
WHERE
t1.n BETWEEN 1
AND (
SELECT
1 + LENGTH( parent_path ) - LENGTH(
REPLACE ( parent_path, ',', '' )))) p1
LEFT OUTER JOIN tb_vk_resources p2 ON ( p1.linkid = p2.res_id )
LEFT OUTER JOIN tb_vk_res_basics t3 ON p1.relation_id = t3.res_basics_id
LEFT OUTER JOIN tb_sys_user t4 ON p1.create_user = t4.userid
LEFT OUTER JOIN tb_idm_org t5 ON t3.org_name = t5.orgid
LEFT OUTER JOIN tb_oss_file_details p3 ON ( p1.file_id = p3.id )
WHERE
p1.parent_id = '1'
GROUP BY
p1.res_id
LIMIT 100 OFFSET 0;
The sql reported a mistake in MySQL5.7 and got the result I wanted in MySQL8.0, which I know is due to 'with recursive as'. But I don't know how to modify it so that he can get the results I want in mysql5.7.

Need to fetch last dates from the query mentioned below

I have latest dates of 2017 in attandance date but its show just 2016 dates kindly plz help me
SELECT
added_on,
types. `type`,
student.`student_id`, student.`roll`, student.`class_id`, student.`name` AS student_name,
SUM(datewise_attandance.`total_classes`) AS SumOfTotal,
datewise_attandance.attandance_date AS attandance_date,
SUM(datewise_attandance.`status`) AS SumOfPresent,
#ROUND(((SUM(datewise_attandance.`status`)/SUM(datewise_attandance.`total_classes`))*100),2) AS percentage,
class.`name` AS class_name, student.`sex`, student.`father_name`, student.`address`,
student.`phone`, subject.`name` AS subject_name,
#types.`type`, types.`type_id`,
subject.`subject_id`,
#MAX( datewise_attandance.`date_to` ) AS MaxOfAtt_Date_To,
student.`session_id`
FROM
(( class
INNER JOIN `subject` ON class.`class_id` = subject.`class_id` )
INNER JOIN datewise_attandance # ( INNER JOIN types ON datewise_attandance.`type_id` = types.`type_id` )
ON ( subject.`subject_id` = datewise_attandance.`subject_id` )
AND ( class.`class_id` = datewise_attandance.`class_id` ))
INNER JOIN `types` ON( types.`type_id` = subject.`subject_type`)
INNER JOIN student ON ( student.`student_id` = datewise_attandance.`std_id` )
AND ( class.`class_id` = student.`class_id` )
GROUP BY student.`student_id`,
student.`roll`, student.`class_id`, student.`name`, class.`name`,
student.`sex`, student.`father_name`, student.`address`,
student.`phone`, subject.`name`,
# types.`type`, types.`type_id`,
subject.`subject_id`, student.`session_id`
HAVING (
(
student.`class_id` = 4
AND student.`roll` = 388
AND student.`session_id` = 5
)
) ORDER BY IFNULL(subject.final_exam,0) DESC
You are using inner join so if there are matching records then only result will be returned for 2017. Why dont you use left join and try the same

MySQL group by kills the query performance

I have MySQL query currently selecting and joining 13 tables and finally grouping ~60k rows. The query without grouping takes ~0ms but with grouping the query time increases to ~1.7sec. The field, which is used for grouping is primary field and is indexed. Where could be the issue?
I know group by without aggregate is considered invalid query and bad practise but I need distinct base table rows and can not use DISTINCT syntax.
The query itself looks like this:
SELECT `table_a`.*
FROM `table_a`
LEFT JOIN `table_b`
ON `table_b`.`invoice` = `table_a`.`id`
LEFT JOIN `table_c` AS `r1`
ON `r1`.`invoice_1` = `table_a`.`id`
LEFT JOIN `table_c` AS `r2`
ON `r2`.`invoice_2` = `table_a`.`id`
LEFT JOIN `table_a` AS `i1`
ON `i1`.`id` = `r1`.`invoice_2`
LEFT JOIN `table_a` AS `i2`
ON `i2`.`id` = `r2`.`invoice_1`
JOIN `table_d` AS `_u0`
ON `_u0`.`id` = 1
LEFT JOIN `table_e` AS `_ug0`
ON `_ug0`.`user` = `_u0`.`id`
JOIN `table_f` AS `_p0`
ON ( `_p0`.`enabled` = 1
AND ( ( `_p0`.`role` < 2
AND `_p0`.`who` IS NULL )
OR ( `_p0`.`role` = 2
AND ( `_p0`.`who` = '0'
OR `_p0`.`who` = `_u0`.`id` ) )
OR ( `_p0`.`role` = 3
AND ( `_p0`.`who` = '0'
OR `_p0`.`who` = `_ug0`.`group` ) ) ) )
AND ( `_p0`.`action` = '*'
OR `_p0`.`action` = 'read' )
AND ( `_p0`.`related_table` = '*'
OR `_p0`.`related_table` = 'table_name' )
JOIN `table_a` AS `_e0`
ON ( ( `_p0`.`related_id` = 0
OR `_p0`.`related_id` = `_e0`.`id`
OR `_p0`.`related_user` = `_e0`.`user`
OR `_p0`.`related_group` = `_e0`.`group` )
OR ( `_p0`.`role` = 0
AND `_e0`.`user` = `_u0`.`id` )
OR ( `_p0`.`role` = 1
AND `_e0`.`group` = `_ug0`.`group` ) )
AND `_e0`.`id` = `table_a`.`id`
JOIN `table_d` AS `_u1`
ON `_u1`.`id` = 1
LEFT JOIN `table_e` AS `_ug1`
ON `_ug1`.`user` = `_u1`.`id`
JOIN `table_f` AS `_p1`
ON ( `_p1`.`enabled` = 1
AND ( ( `_p1`.`role` < 2
AND `_p1`.`who` IS NULL )
OR ( `_p1`.`role` = 2
AND ( `_p1`.`who` = '0'
OR `_p1`.`who` = `_u1`.`id` ) )
OR ( `_p1`.`role` = 3
AND ( `_p1`.`who` = '0'
OR `_p1`.`who` = `_ug1`.`group` ) ) ) )
AND ( `_p1`.`action` = '*'
OR `_p1`.`action` = 'read' )
AND ( `_p1`.`related_table` = '*'
OR `_p1`.`related_table` = 'table_name' )
JOIN `table_g` AS `_e1`
ON ( ( `_p1`.`related_id` = 0
OR `_p1`.`related_id` = `_e1`.`id`
OR `_p1`.`related_user` = `_e1`.`user`
OR `_p1`.`related_group` = `_e1`.`group` )
OR ( `_p1`.`role` = 0
AND `_e1`.`user` = `_u1`.`id` )
OR ( `_p1`.`role` = 1
AND `_e1`.`group` = `_ug1`.`group` ) )
AND `_e1`.`id` = `table_a`.`company`
WHERE `table_a`.`date_deleted` IS NULL
AND `table_a`.`company` = 4
AND `table_a`.`type` = 1
AND `table_a`.`date_composed` >= '2016-05-04 14:43:55'
GROUP BY `table_a`.`id`
The ORs kill performance.
This composite index may help: INDEX(company, type, date_deleted, date_composed).
LEFT JOIN table_b ON table_b.invoice = table_a.id seems to do absolutely nothing other than slow down the processing. No fields of table_b are used or SELECTed. Since it is a LEFT join, it does not limit the output. Etc. Get rid if it, or justify it.
Ditto for other joins.
What happens with JOIN and GROUP BY: First, all the joins are performed; this explodes the number of rows in the intermediate 'table'. Then the GROUP BY implodes the set of rows.
One technique for avoiding this explode-implode sluggishness is to do
SELECT ...,
( SELECT ... ) AS ...,
...
instead of a JOIN or LEFT JOIN. However, that works only if there is zero or one row in the subquery. Usually this is beneficial when an aggregate (such as SUM) can be moved into the subquery.
For further discussion, please include SHOW CREATE TABLE.

OPTIMIZE SQL QUERY WITH NO EXISTS

I have 4 tables, F_NSE, F_LTR, F_TRA, INVENTARIO.
Table INVENTARIO contain data for last stock for a product. F_NSE is table for product trazability (bacth date,..), F_LTR contains data for product selling and F_TRA contains data on the INVOICE. I need calculte the real STOCK for a product from the last date of INVENTARIO table.
One product can be row in INVETARIO table or not.
The problem is with NOT EXISTS clause in the query, is not optimize:
SELECT T0.NSENSE AS LOTE, REPLACE(REPLACE(CONVERT(VARCHAR, T0.FCONSE, 103), '01/01/1900', ''), '/20', '/') AS FCONSE,
SUM(T0.CANNSE) AS UNIDADES, T2.AORTRA AS CODALM, T1.ARTLTR
FROM F_NSE AS T0
INNER JOIN F_LTR AS T1 ON T0.DOCNSE = 'TR' AND T0.TIPNSE = ''
AND T0.CODNSE = T1.DOCLTR AND T0.POSNSE = T1.LINLTR
INNER JOIN F_TRA AS T2 ON T2.DOCTRA = T1.DOCLTR
LEFT OUTER JOIN Stock_Inventario_Pruebas AS T3 ON T3.CODART = T1.ARTLTR
AND T3.CODALM = T2.AORTRA AND T3.NSENSE = T0.NSENSE
WHERE (
NOT EXISTS
(SELECT INVENTARIO
FROM dbo.ERP_Stock_Inventario_Pruebas AS T3
WHERE (T1.ARTLTR = CODART))) AND
T1.ARTLTR ='125554705' OR T2.FECTRA > T3.INVENTARIO
AND T1.ARTLTR = '125554705'
GROUP BY T0.NSENSE, T0.FCONSE, T1.ARTLTR,T2.AORTRA
)
)
I rewrote the query so I find it easier to read:
SELECT T0.NSENSE AS LOTE,
REPLACE(REPLACE(CONVERT(VARCHAR(255), T0.FCONSE, 103), '01/01/1900', ''), '/20', '/') AS FCONSE,
SUM(T0.CANNSE) AS UNIDADES,
T2.AORTRA AS CODALM, T1.ARTLTR
FROM F_NSE T0 INNER JOIN
F_LTR T1
ON T0.DOCNSE = 'TR' AND T0.TIPNSE = '' AND
T0.CODNSE = T1.DOCLTR AND T0.POSNSE = T1.LINLTR INNER JOIN
F_TRA T2
ON T2.DOCTRA = T1.DOCLTR LEFT OUTER JOIN
Stock_Inventario_Pruebas T3
ON T3.CODART = T1.ARTLTR AND T3.CODALM = T2.AORTRA AND T3.NSENSE = T0.NSENSE
WHERE T1.ARTLTR ='125554705' AND
(NOT EXISTS (SELECT 1
FROM dbo.ERP_Stock_Inventario_Pruebas T3
WHERE T1.ARTLTR = T3.CODART
) OR
T2.FECTRA > T3.INVENTARIO
)
GROUP BY T0.NSENSE, T0.FCONSE, T1.ARTLTR,T2.AORTRA;
For this query, I would recommend indexes on F_LTR(ARTLTR) and ERP_Stock_Inventario_Pruebas(CODART).
I do wonder why the left join uses more columns than the NOT EXISTS. Logic more more typically look like this:
. . .
Stock_Inventario_Pruebas T3
ON T3.CODART = T1.ARTLTR AND T2.FECTRA > T3.INVENTARIO
WHERE T1.ARTLTR ='125554705'
. . .
This doesn't do exactly what your query does; it just makes more sense to me.
I think this is equivalent to your restatement of the SQL, but without the correlated subquery. Replace the (primary key) with a mandatory field name so you know if it's missing, the join is missing.
SELECT T0.NSENSE AS LOTE,
REPLACE(REPLACE(CONVERT(VARCHAR(255),
T0.FCONSE, 103), '01/01/1900', ''), '/20', '/') AS FCONSE,
SUM(T0.CANNSE) AS UNIDADES,
T2.AORTRA AS CODALM,
T1.ARTLTR
FROM F_NSE T0
INNER JOIN F_LTR T1
ON T0.DOCNSE = 'TR'
AND T0.TIPNSE = ''
AND T0.CODNSE = T1.DOCLTR
AND T0.POSNSE = T1.LINLTR
INNER JOIN F_TRA T2
ON T2.DOCTRA = T1.DOCLTR
LEFT OUTER JOIN Stock_Inventario_Pruebas T3
ON T3.CODART = T1.ARTLTR
AND T3.CODALM = T2.AORTRA
AND T3.NSENSE = T0.NSENSE
LEFT OUTER JOIN dbo.ERP_Stock_Inventario_Pruebas T4
ON T1.AR√LTR = T4.CODART
WHERE T4.(primary key) IS NULL
OR T2.FECTRA > T3.INVENTARIO
GROUP BY T0.NSENSE, T0.FCONSE, T1.ARTLTR,T2.AORTRA;