my sql query is normally executed in mysql workbench (despite the 15-second wait), but when I try to run the query from the side of another application (phpmyadmin, etc...) it gives me several errors, such as:
Timeout
Error in SELECT clause: expression near '"'.
Missing FROM clause.
Below is the query code:
SELECT investors_positivador.cod_cliente, investors_saldo_financeiro.nome_cliente, investors_base_assessores.squad, investors_base_assessores.nome_investor, investors_saldo_financeiro.saldo_d0,
SUM(investors_posicao_geral.financeiro) AS vencimentos_ate_data, investors_posicao_geral.vencimento,
CASE WHEN investors_diversificacao.produto = "FUNDO" THEN
ROUND(SUM( IF( investors_diversificacao.produto = "FUNDO", investors_diversificacao.net / 6, 0 ) ), 2)
ELSE
"0,00"
END AS fundos_ate_data
,investors_guia_fundos.liquidez_total
,investors_positivador.contatar_liquidity_map
,investors_positivador.id
FROM investors_positivador INNER JOIN
investors_saldo_financeiro ON investors_positivador.cod_cliente = investors_saldo_financeiro.cod_cliente LEFT OUTER JOIN
investors_base_assessores ON investors_positivador.cod_assessor = investors_base_assessores.cod_assessor LEFT OUTER JOIN
investors_posicao_geral ON investors_positivador.cod_cliente = investors_posicao_geral.cod_cliente
LEFT OUTER JOIN investors_diversificacao ON investors_diversificacao.cod_cliente = investors_positivador.cod_cliente AND investors_diversificacao.net != "0,00"
LEFT OUTER JOIN investors_guia_fundos ON investors_guia_fundos.cnpj = investors_diversificacao.cnpj
WHERE investors_base_assessores.nome_investor = "Daiane Costa" AND investors_saldo_financeiro.saldo_d0 > 0 AND investors_posicao_geral.financeiro > 0
GROUP BY cod_cliente
For the waiting time, you could try wrapping your query in a transaction. That may help in optimizing your query
Related
I would like to optimize my sql query. When I put a lot of data in the investors_diversificacao table it gets very slow and doesn't load. But with little data in the investors_diversificacao table it can load the data.
I would like help to optimize query response time.
sql query:
SELECT investors_positivador.cod_cliente, investors_saldo_financeiro.nome_cliente, investors_base_assessores.squad, investors_base_assessores.nome_investor, investors_saldo_financeiro.saldo_d0,
SUM(CASE WHEN investors_posicao_geral.vencimento <= #vencimento THEN investors_posicao_geral.financeiro ELSE 0 END) AS vencimentos_ate_data,
ROUND(SUM(CASE WHEN investors_diversificacao.produto = 'Fundos' THEN investors_diversificacao.net / 6 ELSE 0 END), 2) AS fundos_ate_data, investors_guia_fundos.liquidez_total, investors_positivador.contatar_liquidity_map,
investors_positivador.id, investors_posicao_geral.vencimento, investors_base_assessores.nome_assessor
FROM investors_positivador
INNER JOIN investors_saldo_financeiro ON cod_cliente = investors_saldo_financeiro.cod_cliente
INNER JOIN investors_base_assessores ON cod_assessor = investors_base_assessores.cod_assessor
INNER JOIN investors_posicao_geral ON investors_positivador.cod_cliente = investors_posicao_geral.cod_cliente
LEFT OUTER JOIN investors_diversificacao ON investors_positivador.cod_cliente = investors_diversificacao.cod_cliente
LEFT OUTER JOIN investors_guia_fundos ON investors_diversificacao.cnpj = investors_guia_fundos.cnpj
WHERE (investors_base_assessores.nome_investor = #investor_nome) AND (investors_saldo_financeiro.saldo_d0 > 0)
OR (investors_base_assessores.nome_investor = #investor_nome) AND (investors_posicao_geral.financeiro > 0)
OR (investors_base_assessores.nome_investor = #investor_nome) AND (investors_diversificacao.net > 0)
GROUP BY investors_positivador.cod_cliente
Some of these indexes may help:
investors_positivador: INDEX(cod_cliente, contatar_liquidity_map, id)
investors_saldo_financeiro: INDEX(cod_cliente, saldo_d0, nome_cliente)
investors_base_assessores: INDEX(nome_investor, cod_assessor, squad, nome_assessor)
investors_posicao_geral: INDEX(cod_cliente, financeiro, vencimento)
investors_diversificacao: INDEX(cod_cliente, net, produto, cnpj)
investors_guia_fundos: INDEX(cnpj, liquidez_total)
Are you sure you want that WHERE clause? All the ORs are irrelevant, and Riggs' version simplifies to just
WHERE investors_base_assessores.nome_investor = #investor_nome
AND investors_diversificacao.net > 0
Don't round before summing; you will get extra rounding errors:
ROUND(SUM(IF(investors_diversificacao.produto = 'Fundos',
investors_diversificacao.net, 0
)
) / 6 -- divide after summing and before rounding
, 2) AS fundos_ate_data
i added indexes as well but still it is taking 13 sec
I added compound index for all the columns that i've used here
SELECT carrierbil2_.IDENTITY AS col_0_0_,
carrier4_.CARRIER_NAME AS col_1_0_,
carrier4_.IDENTITY AS col_2_0_,
carrier4_.CARRIER_ID AS col_3_0_,
shipmentor0_.EXTERNAL_REFERENCE_ID AS col_4_0_,
invoicedet5_.INVOICE_NUMBER AS col_5_0_,`enter code here`
shipmentca1_.CARRIER_REFERENCE_NUMBER AS col_6_0_,
SUM(shipmentco9_.RATED_COST) AS col_7_0_,
SUM(shipmentco9_.COST) AS col_8_0_,
invoice6_.TOTAL_PAID_AMOUNT AS col_9_0_,
invoice6_.INVOICE_GENERATED_DATE AS col_10_0_,
shipmentor0_.ACTUAL_SHIP_DATE AS col_11_0_,
bolstatus15_.BOL_STATUS_ID AS col_12_0_,
shipmentlo10_.LOCATION_NAME AS col_13_0_,
country11_.COUNTRY_NAME AS col_14_0_,
postal14_.POSTAL_CODE AS col_15_0_,
state12_.STATE_NAME AS col_16_0_,
city13_.CITY_NAME AS col_17_0_,
shipmentlo16_.LOCATION_NAME AS col_18_0_,
country17_.COUNTRY_NAME AS col_19_0_,
postal20_.POSTAL_CODE AS col_20_0_,
state18_.STATE_NAME AS col_21_0_,
city19_.CITY_NAME AS col_22_0_,
shipmentor0_.IDENTITY AS col_23_0_,
shipmentca1_.IDENTITY AS col_24_0_,
shipmentno7_.NOTE AS col_25_0_
FROM
SHIPMENT_ORDER shipmentor0_
INNER JOIN
SHIPMENT_CARRIER shipmentca1_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentca1_.SHIPMENT_ORDER_ID
AND (shipmentca1_.IS_DELETED = 0)
LEFT OUTER JOIN
CARRIER_BILL_DETAILS carrierbil2_ ON shipmentca1_.SHIPMENT_CARRIER_ID = carrierbil2_.SHIPMENT_CARRIER_ID
LEFT OUTER JOIN
CARRIER_BILLS carrierbil3_ ON carrierbil2_.CARRIER_BILL_ID = carrierbil3_.CARRIER_BILL_ID
INNER JOIN
CARRIER carrier4_ ON shipmentca1_.CARRIER_ID = carrier4_.CARRIER_ID
LEFT OUTER JOIN
INVOICE_DETAILS invoicedet5_ ON shipmentor0_.SHIPMENT_ORDER_ID = invoicedet5_.SHIPMENT_ORDER_ID
LEFT OUTER JOIN
INVOICE invoice6_ ON invoicedet5_.INVOICE_ID = invoice6_.INVOICE_ID
LEFT OUTER JOIN
SHIPMENT_NOTES shipmentno7_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentno7_.SHIPMENT_ORDER_ID
AND (shipmentno7_.NOTE_TYPE = 4)
LEFT OUTER JOIN
SHIPMENT_COST shipmentco8_ ON shipmentor0_.SHIPMENT_ORDER_ID = shipmentco8_.SHIPMENT_ID
LEFT OUTER JOIN
SHIPMENT_COST_DETAILS shipmentco9_ ON shipmentco8_.SHIPMENT_COST_ID = shipmentco9_.SHIPMENT_COST_ID
AND (shipmentco9_.IS_DELETED = 0)
LEFT OUTER JOIN
SHIPMENT_LOCATION shipmentlo10_ ON shipmentor0_.ORIGIN_ID = shipmentlo10_.SHIPMENT_LOCATION_ID
AND (shipmentlo10_.LOCATION_TYPE_ID = 3)
LEFT OUTER JOIN
COUNTRY country11_ ON shipmentlo10_.COUNTRY_ID = country11_.COUNTRY_ID
LEFT OUTER JOIN
STATE state12_ ON shipmentlo10_.STATE_ID = state12_.STATE_ID
LEFT OUTER JOIN
CITY city13_ ON shipmentlo10_.CITY_ID = city13_.CITY_ID
LEFT OUTER JOIN
POSTAL postal14_ ON shipmentlo10_.POSTAL_ID = postal14_.POSTAL_ID
LEFT OUTER JOIN
BOL_STATUS bolstatus15_ ON shipmentor0_.ORDER_STATUS = bolstatus15_.BOL_STATUS_ID
LEFT OUTER JOIN
SHIPMENT_LOCATION shipmentlo16_ ON shipmentor0_.DESTINATION_LOCATION_ID = shipmentlo16_.SHIPMENT_LOCATION_ID
AND (shipmentlo16_.LOCATION_TYPE_ID = 4)
LEFT OUTER JOIN
COUNTRY country17_ ON shipmentlo16_.COUNTRY_ID = country17_.COUNTRY_ID
LEFT OUTER JOIN
STATE state18_ ON shipmentlo16_.STATE_ID = state18_.STATE_ID
LEFT OUTER JOIN
CITY city19_ ON shipmentlo16_.CITY_ID = city19_.CITY_ID
LEFT OUTER JOIN
POSTAL postal20_ ON shipmentlo16_.POSTAL_ID = postal20_.POSTAL_ID
CROSS JOIN
CLIENT client21_
WHERE
shipmentor0_.CLIENT_ID = client21_.CLIENT_ID
AND bolstatus15_.SEQUENCE_ID >= 700
AND (carrierbil3_.IS_APPROVED = 0
OR carrierbil3_.IS_APPROVED IS NULL)
AND (carrierbil3_.IS_DELETED = 0
OR carrierbil3_.IS_DELETED IS NULL)
AND (carrierbil2_.IS_DELETED = 0
OR carrierbil2_.IS_DELETED IS NULL)
AND (shipmentor0_.IS_DELETED = 0
OR shipmentor0_.IS_DELETED IS NULL)
GROUP BY invoice6_.INVOICE_GENERATED_DATE , shipmentca1_.IDENTITY , invoicedet5_.INVOICE_NUMBER , invoice6_.TOTAL_PAID_AMOUNT , shipmentca1_.CARRIER_REFERENCE_NUMBER , carrier4_.CARRIER_ID , CAST(carrier4_.IDENTITY AS SIGNED) , carrier4_.CARRIER_NAME , CAST(carrierbil2_.IDENTITY AS SIGNED) , shipmentor0_.SHIPMENT_ORDER_ID , shipmentno7_.NOTE , shipmentor0_.EXTERNAL_REFERENCE_ID , shipmentlo10_.LOCATION_NAME , country11_.COUNTRY_NAME , postal14_.POSTAL_CODE , state12_.STATE_NAME , city13_.CITY_NAME , shipmentlo16_.LOCATION_NAME , country17_.COUNTRY_NAME , postal20_.POSTAL_CODE , state18_.STATE_NAME , city19_.CITY_NAME , shipmentor0_.IDENTITY
ORDER BY shipmentor0_.SHIPMENT_ORDER_ID DESC;
The indexes are mostly useless because of OR, as in
AND (carrierbil3_.IS_APPROVED = 0
OR carrierbil3_.IS_APPROVED IS NULL)
The simple way to fix that is to pick either 0 or NULL to represent the flag. Then make sure all the data is consistent, and change the WHERE to just check for the one case.
Do you really mean
CROSS JOIN
CLIENT client21_
That is likely to be a performance-killer and generate a huge resultset.
Never mind. You have the ON in WHERE. Please use ON for relations and WHERE for filtering.
WHERE
shipmentor0_.CLIENT_ID = client21_.CLIENT_ID
I see a mixture of LEFT JOIN and JOIN. Check that the LEFT JOINs really need to be LEFT; that is, the 'right' table might have missing data.
To discuss further, please provide EXPLAIN SELECT ....
Eschew over-normalization:
You have 5 tables to describe a location (name, country, postal, state, city). Instead, I recommend a single table with those 5 columns. This, alone, would get rid of 8 JOINs.
CAST(carrier4_.IDENTITY AS SIGNED) -- Can't you fix the datatype to be SIGNED, or allow the value to be UNSIGNED?
But perhaps the main performance-killer is the "explode-implode" syndrone. First, it does a lot of JOINs, building a huge intermediate table, then it collapses that by doing GROUP BY. The remedy is
SELECT ...
FROM ( SELECT SUM(...), SUM(...) FROM ... GROUP BY ... ) AS a
JOIN ((whatever else is needed));
That is, first devise a minimal "derived table" that does the GROUP BY (and/or ORDER BY and/or LIMIT). Then see what else is needed to complete the query (namely all the normalization lookups).
After you have acted on most of my comments, we can discuss whether you have the optimal indexes. (It is premature to do so now.) If so, please start a new Question; it would be too much clutter to add to this one.
First of all, that's a lot of joins. However, the main reason your query is taking a significant time is because you're adding an order by clause. You need to figure out a way to avoid it, or may be come up with a different strategy
The whole point is to get all artist records, and any matching records in 5 other tables.
There are maybe 50,000 artist records.
I plan on creating a view out of this statement and then using it to keep a sugarcrm db up to date.
Any help is hugely appreciated.
here is the SQL statement,
SELECT
/*foreign keys*/
band_profiles.id AS 'BP_ARTIST_ID',
bandsintown_artists.id AS 'BID_ARTIST_ID',
contacts.id AS 'CONTACTS_MGMT_ID',
facebook_stats.id AS 'FB_STATS_ID',
outbound_links.id AS'OUTB_LINKS_ID',
/*high level*/
band_profiles.name AS 'ACCOUNT_NAME',
band_profiles.description AS 'ACCOUNT_DESCRIPTION',
band_profiles.bandsintown_status AS 'BIT_STATUS',
band_profiles.created_at AS 'DR_CREATED_DATETIME',
band_profiles.updated_at AS 'DR_UPDATED_DATETIME',
/*account mgmt fields*/
contacts.description AS 'ACCOUNT_MGMT_CONTACT',
contacts.contact_type AS 'ACCOUNT_MGMT_TYPE',
contacts.custom_contact_type AS 'ACCOUNT_MGMT_TYPE_C',
/*account web & social*/
band_profiles.website_url AS 'WEBSITE_URL',
band_profiles.twitter_url AS 'ACCOUNT_TWITTER_URL',
band_profiles.facebook_url AS 'ACCOUNT_FACEBOOK_URL',
band_profiles.facebook_page_id AS 'ACCOUNT_FACEBOOK_ID',
band_profiles.facebook_like_count AS 'ACCOUNT_FACEBOOK_LIKES',
facebook_stats.like_count AS 'FB_TOTAL_LIKES',
facebook_stats.share_count AS 'FB_TOTAL_SHARES',
facebook_stats.comment_count AS 'FB_TOTAL_COMMENTS',
facebook_stats.click_count AS 'FB_TOTAL_CLICKS',
outbound_links.target_url AS 'OUTBOUND_LINK',
outbound_links.link_type AS 'OUTB_LINK_TYPE',
bandsintown_artists.facebook_tour_dates_url AS 'ACCOUNT_FB_TOUR_DATES'
FROM band_profiles
LEFT JOIN bandsintown_artists
ON band_profiles.id = bandsintown_artists.band_profile_id
LEFT JOIN contacts
ON band_profiles.id = contacts.id
LEFT JOIN facebook_stats
ON band_profiles.id = facebook_stats.band_profile_id
LEFT JOIN outbound_links
on band_profiles.id = outbound_links.band_profile_id
GROUP BY band_profiles.id
LIMIT 10
this is the error code I am getting
Error Code: 2013. Lost connection to MySQL server during query 600.000 sec
If you are the sysadm, you can increase the mysql timeout values. see http://www.rackspace.com/knowledge_center/article/how-to-change-the-mysql-timeout-on-a-server
Depending on the number of deleted entries in your db, you might see significant memory/time savings in this query by pruning deleted entries in your JOIN queries
also limit your final output by band_profiles.deleted = 0
LEFT JOIN bandsintown_artists
ON (band_profiles.id = bandsintown_artists.band_profile_id AND bandsintown_artists.deleted = 0)
LEFT JOIN contacts
ON (band_profiles.id = contacts.id AND contacts.deleted = 0)
LEFT JOIN facebook_stats
ON (band_profiles.id = facebook_stats.band_profile_id AND facebook_stats.deleted = 0)
LEFT JOIN outbound_links
on (band_profiles.id = outbound_links.band_profile_id AND outbound_links.deleted = 0)
WHERE band_profiles.deleted = 0
The below runs fine :
select Runs.runId,Runs.prodId,fail,Owner,
(cast(counts.Count as decimal(4,2))) as PercentAnalysed,Language,Platform
from Runs AS Runs Inner Join Product AS Product On Runs.prodId=Product.prodId
left join (SELECT COUNT(*) AS 'Count', Results.runId
FROM Results inner join Runs on Results.runId=Runs.runId
WHERE Analysed = 'True'
GROUP BY Results.runId ) counts on counts.runId = Runs.runId
I want PercentAnalysed value in outer query to be as below :
= [counts.Count/failCount] if failCount !=0
or
=[0] if failCount=0
How do I do this ?
Any idea anyone ?
You need to use the CASE statement to get what you want.
SELECT
Runs.runId,
Runs.prodId,
fail,
Owner,
CAST(CASE WHEN failcount <> 0 THEN counts.Count/failcount
WHEN failcount = 0 OR counts.Count IS NULL THEN 0
ELSE counts.Count END AS Decimal(4,2)) PercentAnalysed,
Language,
Platform
FROM
.....
p = Patient.find(30)
p.patient_problems
The above code generates the following query
SELECT `patient_problem`.* FROM `patient_problem` WHERE `patient_problem`.`patient_id` = 30 AND (`patient_problem`.`record_status_id` = 1)
But is there any way to assign/use alias table_name like
p.patient_problems(:alias=>'p1') # just for Ex.. This code will not work
p.patient_problems(:alias=>'p2') # just for Ex.. This code will not work
So it will generate the following queries
SELECT `p1`.* FROM `patient_problem` AS `p1` WHERE `p1`.`patient_id` = 30 AND (`p1`.`record_status_id` = 1)
SELECT `p2`.* FROM `patient_problem` AS `p2` WHERE `p2`.`patient_id` = 30 AND (`p2`.`record_status_id` = 1)
Additional Info
My problem is when I try to use joins
p.patient_problems(:all,:joins=>joins)
I get this error
ActionView::Template::Error (Mysql2::Error: Not unique table/alias: 'patient_problem': SELECT `patient_problem`.* FROM `patient_problem` LEFT OUTER JOIN party on party.id = patient_problem.patient_id
LEFT OUTER JOIN party_identifier on party.id = party_identifier.party_id
LEFT OUTER JOIN blood_type on blood_type.id = party.blood_type_id
LEFT OUTER JOIN education_level on education_level.id = party.education_level_id
LEFT OUTER JOIN religion on religion.id = party.religion_id
LEFT OUTER JOIN living_arrangement on living_arrangement.id = party.living_arrangement_id
LEFT OUTER JOIN patient_problem patient_problem on patient_problem.patient_id = party.id and patient_problem.record_status_id = 1
left join (select user_type,username,user_id,auditable_id from (select MAX(id) id from audits where audits.auditable_type = 'PatientProblem' and user_type is not null group by auditable_id ) t inner join audits v on v.id=t.id ) entered_by1 on entered_by1.auditable_id = patient_problem.id
left outer join user user1 on entered_by1.user_id = user1.id
left outer join party as party_user1 on party_user1.id = user1.person_id
LEFT OUTER JOIN patient_patient_search patient_patient_search1 on patient_patient_search1.patient_id = party.id
left join search search1 on patient_patient_search1.patient_search_id = search1.id
and patient_patient_search1.patient_search_id = '75' WHERE `patient_problem`.`patient_id` = 45 AND (`patient_problem`.`record_status_id` = 1) AND ( (patient_problem.occurrence_date > '2013-01-01 00:00:00' and patient_problem.occurrence_date < '2013-06-30 23:59:59' and patient_problem.patient_problem_status_id in (5) and patient_problem.code is not null and patient_problem.code in ('10725009') ) and ( patient_patient_search1.patient_search_id in (75.0) ) ))
Ofcourse I could do some string manipulation on the generated joins query and set alias to patient_problem. But I thought setting alias for associations would be more cleaner since the joins query generated are unpredictable(in my scenario)
I am not sure what the variable joins is or how it was constructed. To alias tables in a join build your query like
Rails 3
PatientProblem.joins("as p1 OUTER JOIN patient_problem as p2 on ...")
or
PatientProblem.find(:all, :joins => "as p1 OUTER JOIN patient_problem as p2 ON ...")
you can make singleton methods for that and write the query one time and use may time like
def self.p1
#your active record query here.
end
and call like
PatientProblem.p1
Update
You can simply change the table name in your code:
Patient.table_name="p2"
I'm not sure if this would break anything else though ... so good luck!
Orignal Answer
One solution may be to define a separate model for each type of patient_problem and then do something like this:
class PatientProblem2 < ActiveRecord::Base
self.set_table_name "p2"
...
end
Another solution may be to use the ActiveRecord query interface which will allows for significant query flexibility:
http://guides.rubyonrails.org/active_record_querying.html
Perhaps you can be more specific on the nature problem you are trying to solve.