I'm trying to pivotise my columns to get from rows to tables.
It's working like expected with the code I have now, but when my fields get updated I have to manually edit the query in order to update it.
I'm trying to automate the process but I'm not getting it to work. Any ideas?
This is the manual code which works:
SELECT
md.entity_guid AS guid, username, e.time_created, time_updated, e.enabled, banned,e.last_action, last_login,
MAX(IF(msn.string = 'question1', msv.string, NULL)) AS question1
FROM exp_metadata md
JOIN exp_metastrings msn ON md.name_id = msn.id
JOIN exp_metastrings msv ON md.value_id = msv.id
JOIN exp_users_entity u ON u.guid = md.entity_guid
JOIN exp_entities e ON e.guid = md.entity_guid
GROUP BY
guid
And this is the query I'm trying to do to automate it:
SET #sql = NULL;
SELECT
md.entity_guid AS guid, username, e.time_created, time_updated, e.enabled, banned, e.last_action, last_login,
GROUP_CONCAT(DISTINCT
CONCAT(
'MAX(IF(msn.string = ''',
msn.string,
''', msv.string, NULL)) AS ',
msn.string
)
) INTO #sql
FROM exp_metadata md
JOIN exp_metastrings msn ON md.name_id = msn.id
JOIN exp_metastrings msv ON md.value_id = msv.id
JOIN exp_users_entity u ON u.guid = md.entity_guid
JOIN exp_entities e ON e.guid = md.entity_guid
SET #sql = CONCAT('SELECT e.guid, ', #sql, ' FROM exp_entities e GROUP BY e.guid');
I get an error in phpmyadmin about the last line.
When I delete the last line to see if it does anything, I get:
#1222 - The used SELECT statements have a different number of columns
Any help would be appreciated.
Thanks a lot, Dries
Related
I've been beating my head against the wall for a while now on this. I've tried following the thought behind Concat in If statement but I can't seem to figure out a way to make my specific need work. I'm now down to a syntax error in my CONCAT statement..'WHERE req.reqCreatedBy = '0' THEN 'Unknown' ELSE users.firstname,' ',users.lastn'. Could anybody give me some help on bringing the first and last name in on this query? I'm at a complete loss.
SELECT req.reqID as Id,
reqDesc.titleText as Title,
req.reqCity as City,
req.reqState as State,
req.areaID as Area,
area.areaname,
reqType.typeTitle as Type,
req.reqCreatedDate as Created,
req.reqEndDate as `End`,
CONCAT((CASE WHERE req.reqCreatedBy = '0' THEN 'Unknown' ELSE users.firstname,' ',users.lastname END))
AS Recruiter
FROM apps_job_request as req
INNER JOIN apps_job_request_description as reqDesc
ON req.reqTitle = reqDesc.titleID
INNER JOIN apps_job_request_type as reqType
ON reqDesc.typeID = reqType.typeID
INNER JOIN `assemble_users`.area AS area
ON area.areaid = req.areaID
INNER JOIN `assemble_users`.users AS users
ON users.username = req.reqCreatedBy
WHERE req.reqID is not null
AND req.reqActive = '1'
If check only one value req.reqCreatedBy, there may be simple IF statement, use CASE for multiple checked values.
SELECT req.reqID as Id,
reqDesc.titleText as Title,
req.reqCity as City,
req.reqState as State,
req.areaID as Area,
area.areaname,
reqType.typeTitle as Type,
req.reqCreatedDate as Created,
req.reqEndDate as `End`,
-- req.reqCreatedBy is '0' OR ''?
IF(req.reqCreatedBy = '0', 'Unknown', CONCAT(users.firstname, ' ', users.lastname)) AS Recruiter
FROM apps_job_request as req
INNER JOIN apps_job_request_description as reqDesc
ON req.reqTitle = reqDesc.titleID
INNER JOIN apps_job_request_type as reqType
ON reqDesc.typeID = reqType.typeID
INNER JOIN `assemble_users`.area AS area
ON area.areaid = req.areaID
INNER JOIN `assemble_users`.users AS users
ON users.username = req.reqCreatedBy
WHERE req.reqID is not null
AND req.reqActive = '1'
I have got three tables that looks as follows.
usedetails [ID,first_name,last_name,telephone,email]
address [ID,streetnumber,streetname,town,county,postcode,userdetailsID]
BOOKING [ID,customerID,pickup_address_id,dropoff_address_id,charge,no_of_passenger]
Address table holds two types of address ie pickoff and dropoff. I would like to display each of the two addresses as one string. The following is my query.
query = "SELECT A.streetnumber,
A.streetname,
A.town,
A.postcode
AS pickup_point
AB.streetnumber,
AB.streetname,
AB.town,
AB.postcode
AS dropoff_point
UD.first_name,
UD.last_name,
UD.telephone,
UD.email
FROM userdetails UD
INNER JOIN booking B
ON B.customerID = UD.ID
INNER JOIN address A
ON B.pickup_address_id = A.ID
INNER JOIN address AB
ON AB.drop_off_address_id = A.ID
WHERE UD.ID = A.userdetailsID OR UD.ID = AB.userdetailsID";
Try CONCAT function:
SELECT CONCAT(A.streetnumber,
' ',
A.streetname,
' ',
A.town,
' ',
A.postcode) AS pickup_point, ...
Or CONCAT_WS function to pass separator as the first argument:
SELECT CONCAT_WS(' ',
A.streetnumber,
A.streetname,
A.town,
A.postcode) AS pickup_point, ...
I have a SQL query which gives the correct result, but performs too slow.
The query operates on the following three tables:
customers contains lots of customer data like name, address, phone
etc. To simplify the table i am only using the name.
customdatas contains certain custom (not customer) data. (The
tables are created in software, which is why the plural form is wrong
for this table)
customercustomdatarels associates custom data with a customer.
customers
Id Name (many more columns)
-----------------------------------------------------------------------
8053c6f4c5c5c631054ddb13d9186117 MyCustomer ...
2efd2aa5711ddfade1f829b12dd88cf3 CheeseFactory ...
customdata
id key
-------------------------------------------------
22deb172c1af6e8e245634a751871564 favoritsport
86eea84d296df9309ad6ff36fd7f856e favoritcheese
customercustomdatarels (relation between customer and custom data - with corresponding value)
customer customdata value
-------------------------------------------------------------------------------------
8053c6f4c5c5c631054ddb13d9186117 22deb172c1af6e8e245634a751871564 cycling
8053c6f4c5c5c631054ddb13d9186117 86eea84d296df9309ad6ff36fd7f856e cheddar
2efd2aa5711ddfade1f829b12dd88cf3 22deb172c1af6e8e245634a751871564 football
2efd2aa5711ddfade1f829b12dd88cf3 86eea84d296df9309ad6ff36fd7f856e mouldy
What i want is a table basically consisting of all data in customers with an variable amount of extra columns, corresponding to the custom data specified in customercustomdatarels.
These columns should be defined somewhere and I have therefore created the following table which defines such extra columns and maps them to a key in the customdata table:
test_customkeymapping
colkey customkey
---------------------
1 favoritsport
2 favoritcheese
The result should then be:
Name ExtraColumn_1 ExtraColumn_2
---------------------------------------------
CheeseFactory football mouldy
MyCustomer cycling cheddar
(ExtraColumn_1 is therefore synonym for a customers' favorite sport and ExtraColumn_2 is a synonym for a customers' favorit cheese.)
This result is achieved by executing the following query:
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT('MAX(CASE
WHEN ckm.colkey = ', colkey, ' THEN
(SELECT value FROM customercustomdatarels ccdr2
LEFT JOIN customdatas cd2
ON cd2.id = ccdr2.customdata
WHERE cd2.key = ckm.customkey AND c.Id = ccdr2.customer)
END) AS ', CONCAT('`ExtraColumn_', colkey, '`'))
) INTO #sql
FROM test_customkeymapping;
SET #sql = CONCAT('SELECT c.Name, ', #sql, '
FROM customers c
LEFT JOIN customercustomdatarels ccdr
ON c.Id = ccdr.customer
LEFT JOIN customdatas cd
ON cd.Id = ccdr.customdata
LEFT JOIN test_customkeymapping ckm
ON cd.key = ckm.customkey
GROUP BY c.Id');
PREPARE stmt FROM #sql;
EXECUTE stmt;
This works. But is too slow (for 7000 customers it takes ~10 seconds).
The query was greatly influenced by the solution in this question:
MySQL Join Multiple Rows as Columns
How do I optimize this query?
I don't understand why you are using a subquery in the group_concat() statement. Wouldn't this generate the code that you really want to run?
SET #sql = NULL;
SELECT
GROUP_CONCAT(DISTINCT
CONCAT('MAX(CASE WHEN ckm.colkey = ', colkey, ' THEN ccd.value END) AS ',
CONCAT('ExtraColumn_', colkey, ''))
) INTO #sql
FROM test_customkeymapping;
SET #sql = CONCAT('SELECT c.Name, ', #sql, '
FROM customers c
LEFT JOIN customercustomdatarels ccdr
ON c.Id = ccdr.customer
LEFT JOIN customdatas cd
ON cd.Id = ccdr.customdata
LEFT JOIN test_customkeymapping ckm
ON cd.key = ckm.customkey
GROUP BY c.Id');
PREPARE stmt FROM #sql;
EXECUTE stmt;
Note: This is untested, but the idea is the same. Use the values from the main from statement for your work rather than the values from some extra, unnecessary subquery.
My query is taking 5 seconds to finish but it sounds to much for something so basic. I did an explain query:
https://www.dropbox.com/s/18pedm9n5fssz4e/localhost%20_%20localhost%20_%20logrede%20_%20phpMyAdmin%203.4.11.pdf
It is possible to you to help me optimize it?
Thanks :)
I've restructured the query using JOINs instead of a less readable WHERE clause. I see everything is based on the primary "material" table. So, my first suggestion is to use the "STRAIGHT_JOIN" clause to tell MySQL to do the query in the order you have listed.... otherwise, sometimes it tries to think for you and do some things in what could be an alternate "primary" table driving the query. All your other tables are basically "lookup values" tables.
Secondly... On each of the tables, I would specifically have the following indexes on the tables. I'm sure you already had on the primary keys of each table... but since your GROUP_CONCAT are getting a corresponding descriptive field from the table, I would include that column in the index. Here's the other reason... When a query is run and using the index, if the query does not have to go back to the raw table for the "other" columns, it doesn't have to. So, by having the description as part of the index, it can do the join AND get the description both from the index...
I would have an index on
material ( cdomaterial )
cliente ( idcliente, nome )
classeobra ( idclasse, sigla )
dma ( iddma, nome )
desenho( iddesenho, nome )
fornecedor ( idfornecedor, nome )
modelo ( idmodelo, nome )
fabricante ( idfabri, nome )
fornecido( codmaterial, preco )
unidade( idunidade, sigla )
tipofornecimento( idtipoforn, sigla )
So, the above said, I would run the following query...
SELECT STRAIGHT_JOIN
material.codmaterial,
cliente.nome as cliente,
tipofornecimento.sigla as fornecimento,
unidade.sigla as unidade,
GROUP_CONCAT(DISTINCT classeobra.sigla ORDER BY classeobra.idclasse SEPARATOR ', ') as classeobra,
GROUP_CONCAT(DISTINCT dma.nome ORDER BY dma.iddma SEPARATOR ', ') as dma,
GROUP_CONCAT(DISTINCT desenho.nome ORDER BY desenho.iddesenho SEPARATOR ', ') as desenho,
GROUP_CONCAT(DISTINCT fornecedor.nome ORDER BY fornecedor.idfornecedor SEPARATOR ', ') as fornecedor,
GROUP_CONCAT(DISTINCT modelo.nome ORDER BY modelo.idmodelo SEPARATOR ', ') as modelo,
GROUP_CONCAT(DISTINCT fabricante.nome ORDER BY fabricante.idfabri SEPARATOR ', ') as marca,
GROUP_CONCAT(DISTINCT fornecido.preco ORDER BY fornecido.preco SEPARATOR ', ') as preco
FROM
material
JOIN unidad
ON material.idunidade = unidade.idunidade
JOIN requisitado
ON material.codmaterial = requisitado.codmaterial
JOIN cliente
ON requisitado.idcliente = cliente.idcliente
JOIN tipofornecimento
ON requisitado.idtipoforn = tipofornecimento.idtipoforn
JOIN possuimodelo
ON material.codmaterial = possuimodelo.codmaterial
JOIN modelo
ON possuimodelo.idmodelo = modelo.idmodelo
JOIN pertence,
ON modelo.idmodelo = pertence.idmodelo
JOIN fabricante
ON pertence.idfabri = fabricante.idfabri
JOIN utilizadoclasseobra
ON material.codmaterial = utilizadoclasseobra.codmaterial
JOIN classeobra
ON utilizadoclasseobra.idclasse = classeobra.idclasse
JOIN possuidma
ON material.codmaterial = possuidma.codmaterial
JOIN dma
ON possuidma.iddma = dma.iddma
JOIN fornecido
ON material.codmaterial = fornecido.codmaterial
JOIN fornecedor
ON fornecido.idfornecedor = fornecedor.idfornecedor
JOIN possuidesenho
ON material.codmaterial = possuidesenho.codmaterial
JOIN desenho
ON possuidesenho.iddesenho = desenho.iddesenho
GROUP BY
material.codmaterial,
cliente.nome
ORDER BY
material.codmaterial ASC,
cliente.nome ASC ;
Is there a way to make MySQL tell which subquery gave the above error in a huge autogenerated query full of subqueries? If not, what would be your strategy for debugging it?
I would start by using either LIMIT or DISTINCT on the query and possibly sub queries.
If that didn't work I'd start going through and running each sub query individually, yes it's time consuming but I find it also helps to avoid the same problem in the future.
Example query:
UPDATE direct_orders do
SET do.dealer_id = (
SELECT d.dealer_id FROM dealers d
INNER JOIN dealer_addresses da
ON da.dealer_id = d.dealer_id
AND da.type = 'M'
AND da.status = 1
INNER JOIN dealer_details dd
ON dd.dealer_id = d.dealer_id
AND dd.status = 1
WHERE ( da.address1 LIKE CONCAT( '%', do.address1, '%' ) AND do.postal LIKE da.postal_code )
OR dd.name LIKE CONCAT( '%', do.company, '%' )
AND do.dealer_id = 0
)
This query was failing, for the same reason as the poster. One way of testing this properly is to do something like the following (note the LEFT JOIN of the table that was previously relied upon for sub-correlation):
SELECT d.dealer_id FROM dealers d
LEFT JOIN direct_orders do
ON do.dealer_id = 0
INNER JOIN dealer_addresses da
ON da.dealer_id = d.dealer_id
AND da.type = 'M'
AND da.status = 1
INNER JOIN dealer_details dd
ON dd.dealer_id = d.dealer_id
AND dd.status = 1
WHERE (da.address1 LIKE CONCAT( '%', do.address1, '%' ) AND do.postal LIKE da.postal_code )
OR dd.name LIKE CONCAT( '%', do.company, '%' )
Doing this revealed that there was in fact one record from the "direct_orders" table that return more than one "dealer_id" from the "dealers" table.