Can we discuss how to load the below query result into a destination table ,using ssis.I know we can use this in T-sql and also as an OLEDB source query.But still wondering how to implement it ,only using data flow components
SELECT
CLIENTID = CAST(PER.CLIENTID AS INT)
,CASEID = CAST(CS.CASEID AS INT)
,CAST(RIGHT(ev.oid, 10) as int) AS EventID
,ev.ServiceSubtypeCode
,ev.ServiceSubtypeCode +' - '+ev.ServiceSubTypeDesc as ServiceSubTypeDesc
,WU.ProviderID as WorkunitProviderID
,WU.ProviderName as WorkUnitProviderName
,ev.eventstartdate as AssessmentStartDate
,CONVERT(CHAR(5),ASM.getstarttimestamp,8) as AssessmentStartTime
,ev.EVENTENDDATE as AssessmentEndDate
,ev.EVENTENDTIME as AssessmentEndTime
,CAST(asm.getAssmtTemplateName as nvarchar(200)) as AssessmentTypeDesc
,j.providerid
,j.ProviderName
,j.ProviderRole
,EV.ISCOMPLETED
, EV.ISFINALISED
,EV.ISREVOKED
, EV.REVOKEDDATE AS REVOKEDDATE
,ASM.OID AS ASSESSMENTID
FROM DBO.ASSESSMENT ASM
LEFT OUTER JOIN DBO.INDIVIDUALPERSON PER ON ASM.MYPERSON = PER.OID
LEFT OUTER JOIN DBO.[CASE] CS ON ASM.MYCASE = CS.OID
LEFT OUTER JOIN (
SELECT CAST(ST.CODE AS VARCHAR(8))AS SERVICETYPECODE
, CAST(ST.DESCRIPTION AS VARCHAR(100)) AS SERVICETYPEDESC
, CAST(SST.CODE AS VARCHAR(8)) AS SERVICESUBTYPECODE
, CAST(SST.DESCRIPTION AS VARCHAR(100)) AS SERVICESUBTYPEDESC
, DATEADD(DD,0, DATEDIFF(DD,0,EV.GETRPSSTARTTIMESTAMP)) AS EVENTSTARTDATE
, CONVERT(CHAR(5),EV.GETRPSSTARTTIMESTAMP,8) AS EVENTSTARTTIME
, DATEADD(DD,0, DATEDIFF(DD,0,EV.GETRPSENDTIMESTAMP)) AS EVENTENDDATE
, CONVERT(CHAR(5),EV.GETRPSENDTIMESTAMP,8) AS EVENTENDTIME
,CAST(VEN.DESCRIPTION AS VARCHAR(12)) AS EVENTVENUE
,EV.ISCOMPLETED
, EV.ISFINALISED
,EV.ISREVOKED
, DATEADD(DD,0, DATEDIFF(DD,0,EV.REVOKEDON)) AS REVOKEDDATE
, EV.OID
from Event ev
LEFT OUTER JOIN ServiceType AS st ON ev.myServiceType = st.oid
LEFT OUTER JOIN ServiceSubtype AS sst ON ev.myServiceSubtype = sst.oid
LEFT OUTER JOIN AllCodes AS ven ON ev.myEventVenueCode = ven.oid
)as EV
ON ASM.MYEVENT = EV.OID
LEFT OUTER JOIN (
select wu.oid
,CAST(wu.providerid AS VARCHAR(100)) AS providerid
,CAST(nm.getfullname AS VARCHAR(100)) AS ProviderName
,wu.contactname
,wu.activatedate as StartDate
,wu.deactivatedate as EndDate
,case when wu.deactivatedate is null then 1 else 0 end as IsActiveToday
from workunitprovider wu
LEFT OUTER JOIN dbo.allprovidernames nm ON wu.oid = nm.myprovider
where nm.myNameType in (02245.0000000252)
) as WU
ON ASM.MYWORKUNITPROVIDER = WU.OID
Left join (
select f.myEvent
,f.myProvider
,f.myproviderrolecode
,f.Max_ProvOid
,CAST(g.providerid AS VARCHAR(100)) AS providerid
,CAST( i.description AS VARCHAR(150)) AS ProviderRole
,cast (h.getFullName as nvarchar (150)) as ProviderName
from( select d.myEvent
,myProvider
,myproviderrolecode
,d.Max_ProvOid
from ( select A.myEvent, max(b.oid) as Max_ProvOid
from alleventitems a
left outer join ProviderEventItemRole as b on a.oid = b.myeventitem
group by A.myEvent
) as d
left join
( select A.myEvent,b.myProvider,b.myproviderrolecode,a.oid as a_oid,b.oid as b_oid
from alleventitems a
left outer join ProviderEventItemRole as b on a.oid = b.myeventitem
)as e on d.myevent = e.myevent and max_provOid = b_oid
) as f
left join dbo.allproviders as g on f.myProvider = g.oid
left join (
select *
from dbo.AllProviderNames
where mynametype ='02245.0000000252'
)as h on f.myprovider =h.myprovider
left join dbo.allcodes as i on f.myproviderrolecode = i.oid
)as j on ASM.myevent = j.myevent;
Before we begin, a disclaimer:
Complex SELECT queries are best expressed in T-SQL. SSIS is best used for ETL tasks.
Now... with that out of the way. Let's see what we have. That query has fifteen LEFT JOINS nested across three levels: Five at the top, seven at the middle, and two at the bottom. Peppered throughout are a some CAST()s and GROUP BYs. All of those SQL commands can be done with SSIS components.
JOIN = Merge Join component.
GROUP BY = Aggregate component.
CAST = Derived Column component.
Since you have such a large query, I'd recommend breaking this into smaller chunks. Starting with the inner most join.
select
d.myEvent
,myProvider
,myproviderrolecode
,d.Max_ProvOid
from (
select A.myEvent, max(b.oid) as Max_ProvOid
from alleventitems a
left outer join ProviderEventItemRole as b
on a.oid = b.myeventitem
group by A.myEvent
) as d
left join (
select A.myEvent,b.myProvider,b.myproviderrolecode,a.oid as a_oid,b.oid as b_oid
from alleventitems a
left outer join ProviderEventItemRole as b
on a.oid = b.myeventitem
) as e
Translating that to SSIS would look like this.
Above, we're merging four tables into one. You can learn more about how to configure Merge Joins here. Repeat the above pattern for the remaining JOINS and connect them all together and you will have translated the entire query to SSIS!
Now that we can see how it may be done, may I ask why we'd want to do this in SSIS?
Related
I have this select statement that is taking quite a while to run on a larger dataset
select lookup_svcscat_svcscatnew.SVCSCAT_NEW_DESC as svc_type,
enrolid, msclmid, dx1, dx2, dx3,
proc1,msk_cpt_mapping.surg_length_cd as SL_CD,
msk_cpt_mapping.days as day_window,o.svcdate_form, pay,
table_label
from ccaeo190_ky o
left join lookup_svcscat_svcscatnew on o.svcscat = lookup_svcscat_svcscatnew.svcscat
left join msk_cpt_mapping on o.proc1 = msk_cpt_mapping.cpt_code
where EXISTS
(
select 1
from eoc_op_mapping e
where e.msclmid = o.msclmid
and e.enrolid = o.enrolid
and proc1 =27447
)
ORDER BY svcdate_form, msclmid;
I want to return any row in my ccaeo190_ky table that meets the requirements of the where EXISTS clause on table eoc_op_mapping. Is there any way to achieve these results using joins or select statements?
I was thinking something like:
select lookup_svcscat_svcscatnew.SVCSCAT_NEW_DESC as svc_type,
o.enrolid, o.msclmid, dx1, dx2, dx3,
proc1,msk_cpt_mapping.surg_length_cd as SL_CD,
msk_cpt_mapping.days as day_window,o.svcdate_form, pay,
table_label
from ccaeo190_ky o
left join lookup_svcscat_svcscatnew on o.svcscat = lookup_svcscat_svcscatnew.svcscat
left join msk_cpt_mapping on o.proc1 = msk_cpt_mapping.cpt_code
inner join
(select msclmid, SUM(IF(proc1 = 27447,1,0)) AS cpt
from eoc_op_mapping
group by enrolid
HAVING cpt > 0) e
on e.enrolid = o.enrolid
group by o.enrolid;
But I don't know if this is in the right direction
Usually EXISTS performs better than a join.
If you want to try a join, this the equivalent to your WHERE EXISTS:
.......................................................
inner join (
select distinct msclmid, enrolid
from eoc_op_mapping
where proc1 = 27447
) e on e.msclmid = o.msclmid and e.enrolid = o.enrolid
.......................................................
You can remove distinct if there are no duplicate msclmid, enrolid combinations in eoc_op_mapping.
Using Report Builder 3.0, I have a report using the following dataset:
Declare #Hierarchy nVarChar (1000) = EAC.GetHierarchy (#UserName, DEFAULT)
SELECT TOP(100)
CASE
WHEN #ShowSourceTime = 1 THEN SourceTime
ELSE Dateadd(Minute, #TimeZoneOffset, Time)
END as Time
,Warehouse.EventInfo.TimeZoneShortName
,Warehouse.EventInfo.Category
,Warehouse.EventInfo.[Action]
,Warehouse.EventInfo.[Result]
,Warehouse.EventInfo.Reason
,Warehouse.EventInfo.PersonId
,Warehouse.EventInfo.Title
,Warehouse.EventInfo.FirstName
,Warehouse.EventInfo.MiddleName
,Warehouse.EventInfo.LastName
,Warehouse.EventInfo.Suffix
,Warehouse.EventInfo.Nickname
,Warehouse.EventInfo.DoorName
,Warehouse.EventInfo.DoorBehaviorName
,Warehouse.EventInfo.EnterZone
,Warehouse.EventInfo.LeaveZone
,Warehouse.EventInfo.SiteCode
,Warehouse.EventInfo.CardCode
,Warehouse.EventInfo.SiteCode + '-' + Warehouse.EventInfo.CardCode as Badge
FROM
Warehouse.EventInfo
WHERE
Warehouse.EventInfo.[Time] Between #StartTime and #EndTime
AND (1 = #PersonIdExpr OR EventInfo.PersonId IN (#PersonId))
AND WareHouse.EventInfo.ZoneHierarchy LIKE #Hierarchy
order by Time
The dataset is calling the view below:
SELECT EAC.Event.CreatedUTC AS Time
,DATEADD(MINUTE, EAC.Event.CreatedUTCOffset
, EAC.Event.CreatedUTC) AS SourceTime
, EAC.TimeZoneMap.TimeZoneShortName
, EAC.EventClass.Name AS Class
, EAC.EventCategory.Name AS Category
, EAC.EventAction.Name AS Action
, EAC.EventResult.Name AS Result
, EAC.EventReason.Name AS Reason
, EAC.Person.Id AS PersonId
, EAC.Person.Title
, EAC.Person.FirstName
, EAC.Person.MiddleName
, EAC.Person.LastName
, EAC.Person.Suffix
, EAC.Person.DisplayName AS Nickname
, EAC.Door.Name AS DoorName
, Zone_1.Name AS EnterZone
, (SELECT TOP (1) Name
FROM EAC.Zone
WHERE (Id IN (EAC.Door.Zone1Id, EAC.Door.Zone2Id))
AND (Id <> EAC.Event.ZoneId)) AS LeaveZone
, EAC.DoorBehavior.Name AS DoorBehaviorName
, EAC.Event.CardCode
, EAC.Event.SiteCode
, ISNULL(EAC.Event.Alarmed, 0) AS AlarmType
, Zone_1.ZoneHierarchy
FROM EAC.Event
INNER JOIN EAC.EventType ON EAC.EventType.Id = EAC.Event.EventTypeId
LEFT OUTER JOIN EAC.EventClass ON EAC.EventClass.Id = EAC.Event.EventClassId
LEFT OUTER JOIN EAC.EventCategory ON EAC.EventCategory.Id = EAC.EventType.CategoryId
LEFT OUTER JOIN EAC.EventAction ON EAC.EventAction.Id = EAC.EventType.ActionId
LEFT OUTER JOIN EAC.EventResult ON EAC.EventResult.Id = EAC.EventType.ResultId
LEFT OUTER JOIN EAC.EventReason ON EAC.EventReason.Id = EAC.EventType.ReasonId
LEFT OUTER JOIN EAC.Person ON EAC.Person.Id = EAC.Event.PersonId
LEFT OUTER JOIN EAC.Door ON EAC.Door.Id = EAC.Event.DoorId
LEFT OUTER JOIN EAC.Zone AS Zone_1 ON Zone_1.Id = EAC.Event.ZoneId
LEFT OUTER JOIN EAC.DoorBehavior ON EAC.DoorBehavior.Id = EAC.Door.DoorBehaviorId
LEFT OUTER JOIN EAC.Credential ON EAC.Credential.Id = EAC.Event.CredentialId
LEFT OUTER JOIN EAC.WiegandCredential ON EAC.WiegandCredential.CredentialId = EAC.Credential.Id
LEFT OUTER JOIN EAC.TimeZoneMap on EAC.TimeZoneMap.Id = EAC.Door.TimeZoneMapId
The table it is querying has millions of records but in order to get anything to come back I limited it to 100 records but it still takes 15 minutes.
When I run the report I see this in the database:
Here is the execution plan:
Execution plan
What can I do to make this report run more efficiently?
with that much LEFT OUTER JOIN It is not surprising that your query runs slow. Each join introduces complexity to your code. Besides, LEFT|RIGHT [OUTER] JOIN is much slower then regular join since it has to do all the work of an INNER JOIN (regular join) plus the extra work of null-extending the results.
Also it is impossible to make comment or pinpoint the problem without seeing explain plan of the query. Possibly you have missing indexes on tables(s)
I have the following query:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
p.p_date,
p.p_price AS Unadjusted_Price,
b.region AS Region,
f_splitadjprice(p.fsym_id,p.p_date,p.p_price) AS O_Split_Adjusted_Price,
f_prevunadjprice(p.fsym_id,p.p_date,Previous_Date,p.p_price),
(
SELECT MAX(f.p_date)
FROM fp_v2_fp_basic_prices AS f
WHERE f.fsym_id = p.fsym_id AND f.p_date<p.p_date
) Previous_Date
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id;
I get the error, "Previous_date not on column list" when trying to call the function f_prevundadjprice. Basically what I want to do is create the column previous date using (SELECT MAX..) and then use the value from this column in the function f_prevunadjprice.
If you use p. to reference all the fields coming from fp_v2_fp_basic_prices , you should also do it in the next line:
f_prevunadjprice(p.fsym_id,p.p_date,Previous_Date,p.p_price)
try to change it by:
f_prevunadjprice(p.fsym_id,p.p_date,p.Previous_Date,p.p_price)
If you continue with the same error, you should ensure the table/view fp_v2_fp_basic_prices has a column named Previous_Date
You should also use AS here:
(
SELECT MAX(f.p_date)
FROM fp_v2_fp_basic_prices AS f
WHERE f.fsym_id = p.fsym_id AND f.p_date<p.p_date
) Previous_Date
So:
...) AS Previous_Date
you already have one subquery. If you were using TSQL you could outer apply the second one. MySQL doesn't support apply, so you'll have to left join to it:
SELECT
p.fsym_id,
b.p_co_sec_name_desc AS Company_Name,
p.p_date,
p.p_price AS Unadjusted_Price,
b.region AS Region,
f_splitadjprice(p.fsym_id,p.p_date,p.p_price) AS O_Split_Adjusted_Price,
f_prevunadjprice(p.fsym_id,p.p_date,Previous_Date,p.p_price),
PreviousDate.maxdate
FROM
fp_v2_fp_basic_prices p
LEFT JOIN (
SELECT r2.region, b2.p_co_sec_name_desc, b2.fsym_id
FROM fp_v2_fp_sec_coverage b2
LEFT JOIN sym_v1_sym_region r2 ON b2.fsym_id = r2.fsym_id
WHERE r2.region = "EUR") b
ON b.fsym_id =p.fsym_id;
LEFT JOIN (
SELECT f.fsym_id,f.p_date,MAX(f.p_date) as maxdate
FROM fp_v2_fp_basic_prices AS f
group by f.fsym_id, f.p_date
) Previous_Date
on Previous_Date.fsym_id = p.fsym_id and AND Previous_Date.p_date<p.p_date
How can I access data from an outer table in a SELECT, and use it in an WHERE inside a JOIN estructure?
Below is the current query:
SELECT
cvl.id caracteristica_valor_id,
cvl.nome caracteristica_valor_nome,
cvl.valor caracteristica_valor_valor,
ctp.id caracteristica_tipo_id,
ctp.nome caracteristica_tipo_nome,
ctp.codigo caracteristica_tipo_codigo,
ctp.tipo caracteristica_tipo_tipo,
COUNT(DISTINCT var.id_perfil_produto) quantidade_itens
FROM
caracteristica_variacao cvr
INNER JOIN caracteristica_valor cvl ON cvl.id = cvr.id_caracteristica_valor
INNER JOIN caracteristica_tipo ctp ON ctp.id = cvl.id_caracteristica_tipo
INNER JOIN variacao var ON var.id = cvr.id_variacao
INNER JOIN(
SELECT DISTINCT
ppr.id perfil_produto_id
FROM
perfil_produto ppr
INNER JOIN produto pro ON pro.id = ppr.id_produto
INNER JOIN(
SELECT ppr2.id AS id_perfil_sub,a
COUNT(var.id) AS qtd_variacoes,
SUM(var.quantidade_estoque) AS quantidade_estoque,
COALESCE(SUM(var.quantidade_estoque_reservada),0) AS quantidade_estoque_reservada,
MIN(var.disponibilidade) AS disponibilidade,
MIN(var.frete_gratis) AS frete_gratis,
MIN(var.preco_venda) AS preco_venda,
MAX(var.preco_listagem) AS preco_listagem
FROM
variacao var
LEFT JOIN perfil_produto ppr2 ON ppr2.id = var.id_perfil_produto
LEFT JOIN caracteristica_variacao cvr_1 ON cvr_1.id_variacao = var.id
LEFT JOIN caracteristica_valor cvl_1 ON cvl_1.id = cvr_1.id_caracteristica_valor
LEFT JOIN caracteristica_tipo ctp_1 ON ctp_1.id = cvl_1.id_caracteristica_tipo
WHERE
var.disponibilidade = 1
AND(
ctp_1.codigo = 'tamanho' AND cvl_1.valor IN('p')
)
GROUP BY
ppr2.id
) AS grp_var ON grp_var.id_perfil_sub = ppr.id
INNER JOIN produto_categoria prc ON pro.id = prc.produto_id
INNER JOIN categoria cat ON prc.categoria_id = cat.id
WHERE
pro.disponibilidade = 1 AND prc.categoria_id IN (164, 165, 166)
) AS produto ON produto.perfil_produto_id = var.id_perfil_produto
GROUP BY
cvl.id
ORDER BY
ctp.tipo ASC,
ctp.id
I need the field ctp.codigo from the outer table inside thist part:
WHERE
var.disponibilidade = 1
AND(
ctp_1.codigo = 'tamanho' AND cvl_1.valor IN('p')
)
for this section to be as follows:
WHERE
var.disponibilidade = 1
AND(
(ctp.codigo != 'tamanho' AND ctp_1.codigo = 'tamanho' AND cvl_1.valor IN('p'))
OR
(ctp.codigo = 'tamanho')
)
It's not possible to reference columns from the outer query from inside an inline view query.
In the MySQL venacular, the inline view query is called a "derived table". And that name makes sense, because of the way MySQL processes it. The execution plan first materializes the inline view query into a temporary(-ish) table. Once that is done, then the outer query can run, referencing the contents of the derived table.
MySQL doesn't have available the columns from the outer query at the time the inline view query runs.
It is possible to reference columns from the outer query inside a subquery that appears for example in the SELECT list, or in the WHERE clause. We call a subquery that references columns from outer query a "correlated subquery".
I am trying to write a query that uses a LEFT OUTER JOIN on three tables. I have complete the first part to join two tables but I am stuck on intergarting the third table.
What I need is the "Status" field for the NXLHR_Valid to be included in the first query.
Below are my to queries, how would I include the SECOND query into the FIRST query
FIRST QUERY
SELECT NXLHR_SequenceNo_default.SeqNo, NXLHR_SequenceNo_default.SeqHeader, NXLHR_SequenceNo_default.SeqText, NXLHR_Hist.UniqueID, NXLHR_Hist.Room, NXLHR_Hist.Status, NXLHR_Hist.Water, NXLHR_Hist.AuditBy
FROM NXLHR_SequenceNo_default
LEFT OUTER JOIN NXLHR_Hist
ON NXLHR_SequenceNo_default.SeqID = NXLHR_Hist.SeqID
AND NXLHR_Hist.UniqueID = 'NXLHR01031472477564'
WHERE NXLHR_SequenceNo_default.SeqActive = 1
ORDER BY NXLHR_SequenceNo_default.OrderID
SECOND QUERY
SELECT NXLHR_Valid.UniqueID, NXLHR_Valid.Status
FROM NXLHR_Valid
WHERE NXLHR_Valid.UniqueID = 'NXLHR01031472477564'
Any help would be great. Thank you for your time.
SELECT d.SeqNo
, d.SeqHeader
, d.SeqText
, h.UniqueID
, h.Room
, h.Status
, h.Water
, h.AuditBy
, v.Status
FROM NXLHR_SequenceNo_default d
LEFT
JOIN NXLHR_Hist h
ON h.SeqID = d.SeqID
AND h.UniqueID = 'NXLHR01031472477564'
LEFT
JOIN NXLHR_Valid v
ON v.UniqueID = h.UniqueID
WHERE d.SeqActive = 1
ORDER
BY d.OrderID