Mysql Join Query optimisation - mysql

Hello i am trying to run this select statement using this query and it is taking over 2 hours to run. I have set up all the index's to be correct. But it still takes forever is there something i am missing or a more efficient way of joining tables together that will speed this query up?
I have indexes set up for all items being joined together and they are the same length and data type.
SELECT
p.sap_article_id,
p.numeric_line_code,
p.uag_linecode,
p.uag_partnum,
p.part_description,
p.jobber_price,
p.jobber_core,
p.discount1,
p.discount2,
p.uom,
p.product_category,
w.as400_warehouse,
w.atp_qty,
p.updated,
t.regular_discount
FROM part p
LEFT JOIN tabjbmaw t ON t.accountnum = '73050'
AND p.numeric_line_code = t.numeric_line_code
AND p.sub_code = t.sub_code
JOIN warehouse w ON w.sap_article_id = p.sap_article_id;
+----+-------------+-----------+------+--------------------------------------------------+-----------------------+---------+----------------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------+------+--------------------------------------------------+-----------------------+---------+----------------------------------+--------+-------------+
| 1 | SIMPLE | part | ALL | PRIMARY,sap_article,part_sap_article_id_fk | NULL | NULL | NULL | 389309 | |
| 1 | SIMPLE | warehouse | ref | article | article | 130 | inventory.part.sap_article_id | 5 | Using where |
| 1 | SIMPLE | tabjbmaw | ref | numeric_line_code_idx,subcode_idx,accountnum_idx | numeric_line_code_idx | 5 | inventory.part.numeric_line_code | 19 | |
+----+-------------+-----------+------+--------------------------------------------------+-----------------------+---------+----------------------------------+--------+-------------+
Thank you for your help
+-----------------------------+--------------+------+-----+---------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------------+--------------+------+-----+---------------------+-----------------------------+
| sap_article_id | varchar(24) | NO | PRI | | |
| sap_brand_id | varchar(20) | NO | | NULL | |
| uag_partnum | varchar(20) | NO | MUL | NULL | |
| uag_linecode | varchar(5) | NO | MUL | NULL | |
| cag_partnum | varchar(20) | NO | MUL | NULL | |
| cag_linecode | varchar(5) | NO | | NULL | |
| product_category_legacy | varchar(20) | NO | | NULL | |
| part_description | varchar(128) | NO | | NULL | |
| abc_indicator | varchar(8) | NO | | NULL | |
| pack_code | varchar(8) | NO | | NULL | |
| case_qty | int(11) | NO | | NULL | |
| per_car_qty | int(11) | NO | | NULL | |
| uom | varchar(6) | NO | | NULL | |
| upc_code | varchar(128) | NO | | NULL | |
| jobber_price | float(14,4) | YES | | NULL | |
| jobber_core | float(14,4) | YES | | NULL | |
| date_last_price_change | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| weight | float(14,4) | YES | | NULL | |
| weight_unit | varchar(6) | NO | | NULL | |
| dimension_type | varchar(6) | NO | | NULL | |
| length | float(14,4) | YES | | NULL | |
| width | float(14,4) | YES | | NULL | |
| height | float(14,4) | YES | | NULL | |
| updated | tinyint(1) | NO | | 0 | |
| superseded_sap_article_id | varchar(24) | YES | | NULL | |
| last_updated | timestamp | NO | | 0000-00-00 00:00:00 | |
| hour_updated | int(11) | YES | | NULL | |
| discount1 | float | YES | | NULL | |
| discount2 | float | YES | | NULL | |
| product_category | varchar(3) | YES | | NULL | |
| superseded_part_number | varchar(20) | YES | | NULL | |
| sub_code | varchar(3) | YES | MUL | NULL | |
| date_effective_price_change | date | YES | | NULL | |
| numeric_line_code | varchar(3) | YES | MUL | NULL | |
| list | float | YES | | NULL | |
+-----------------------------+--------------+------+-----+---------------------+-----------------------------+

I have indexes set up for all items being joined together
Yes, but I am guessing from the names of the indexes that each index only has one field.
Let's look at a few columns in the describe.
| table | possible_keys | key
+-----------+--------------------------------------------------+----------------
| part | PRIMARY,sap_article,part_sap_article_id_fk | NULL
| warehouse | article | article
| tabjbmaw | numeric_line_code_idx,subcode_idx,accountnum_idx | numeric_line_code_idx
It can use an index for numeric_line_code, subcode, and accountnum, but there are only three indexes each with one of the fields, and no index which has all the fields. You are making the optimizer choose one of the one field indexes, instead of providing one index it can use for all three fields.
Add an index on table tabjbmaw with the three fields numeric_line_code, subcode, and accountnum.

Extending #Sebas answer, you should select tabjbmaw first:
SELECT
p.sap_article_id,
p.numeric_line_code,
p.uag_linecode,
p.uag_partnum,
p.part_description,
p.jobber_price,
p.jobber_core,
p.discount1,
p.discount2,
p.uom,
p.product_category,
w.as400_warehouse,
w.atp_qty,
p.updated,
t.regular_discount
FROM tabjbmaw t
LEFT JOIN parts p ON p.numeric_line_code = t.numeric_line_code
AND p.sub_code = t.sub_code
JOIN warehouse w ON w.sap_article_id = p.sap_article_id
WHERE t.accountnum = '73050'
;

You could try to put your Left Join into the SELECT part as a Subselect. That 'may' speed things up a little.
Like this:
SELECT
p.sap_article_id,
p.numeric_line_code,
p.uag_linecode,
p.uag_partnum,
p.part_description,
p.jobber_price,
p.jobber_core,
p.discount1,
p.discount2,
p.uom,
p.product_category,
w.as400_warehouse,
w.atp_qty,
p.updated,
(SELECT t.regular_discount FROM tabjbmaw t WHERE t.accountnum = '73050' AND p.numeric_line_code = t.numeric_line_code AND p.sub_code = t.sub_code LIMIT 1)
FROM
part p
JOIN warehouse w ON w.sap_article_id = p.sap_article_id;

Related

How to decrease this mysql query execution time?

Parent Table,
mysql> desc sattool_testing;
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| circuit_id | varchar(30) | NO | | NULL | |
| coff_id | varchar(100) | NO | | NULL | |
| result | int(11) | YES | | NULL | |
| result_details | text | NO | | NULL | |
| details | longtext | NO | | NULL | |
| reverse_response | varchar(100) | NO | | NULL | |
| start_date | datetime | NO | | NULL | |
| end_date | datetime | YES | | NULL | |
| Modules | varchar(100) | YES | | NULL | |
| rehit | datetime | YES | | NULL | |
| isAuto | tinyint(1) | NO | | 0 | |
+------------------+--------------+------+-----+---------+----------------+
12 rows in set (0.00 sec)
1st Child Table,
mysql> desc sattool_desc;
+--------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+---------+----------------+
| sattool_desc_id | bigint(20) | NO | PRI | NULL | auto_increment |
| ceinterface | varchar(50) | YES | | NULL | |
| cehostname | varchar(50) | YES | | NULL | |
| vprnno | varchar(50) | YES | | NULL | |
| policyname | varchar(50) | YES | | NULL | |
| cosno | varchar(10) | YES | | NULL | |
| peipaddress | varchar(17) | YES | | NULL | |
| router_type | varchar(5) | YES | | NULL | |
| sattool_testing_id | bigint(20) | YES | | NULL | |
| epipeid | varchar(20) | YES | | NULL | |
| cerouter | varchar(10) | YES | | NULL | |
| service_type | varchar(10) | YES | | NULL | |
| scope_of_manage | varchar(50) | YES | | NULL | |
| service_name | varchar(10) | YES | | NULL | |
| ce_vrf_name | varchar(150) | YES | | NULL | |
+--------------------+--------------+------+-----+---------+----------------+
15 rows in set (0.01 sec)
2nd Child Table (Contain multiple records against one parent table record),
mysql> desc sattool_error_log;
+----------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+-------------------+-----------------------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| service_id | varchar(30) | NO | | NULL | |
| coff_id | varchar(50) | NO | | NULL | |
| error_name | varchar(100) | NO | | NULL | |
| error_desc | text | NO | | NULL | |
| error_occurred | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| error_type | int(11) | YES | | NULL | |
| error_code | varchar(10) | YES | | NULL | |
| sattool_id | bigint(20) | YES | | 0 | |
| module_no | int(11) | YES | | 0 | |
+----------------+--------------+------+-----+-------------------+-----------------------------+
10 rows in set (0.00 sec)
SQL Query :-
SELECT GROUP_CONCAT(sattool_error_log.error_desc) AS err_desc, GROUP_CONCAT(sattool_error_log.error_type) AS err_type, sattool_testing.*, sattool_desc.service_type, sattool_desc.service_name, sattool_desc.scope_of_manage
FROM sattool_testing
LEFT JOIN sattool_desc ON sattool_testing.id = sattool_desc.sattool_testing_id
LEFT JOIN sattool_error_log ON sattool_testing.id = sattool_error_log.sattool_id
WHERE sattool_testing.isAuto = 1 GROUP BY sattool_testing.id ORDER BY sattool_testing.id DESC limit 100
Current Execution Time is (22.76 sec)
Explain Query :-
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
| 1 | SIMPLE | sattool_testing | ALL | NULL | NULL | NULL | NULL | 3578 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | sattool_desc | ALL | NULL | NULL | NULL | NULL | 4009 | |
| 1 | SIMPLE | sattool_error_log | ALL | NULL | NULL | NULL | NULL | 8904 | |
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
3 rows in set (0.00 sec)
I want to increase speed to above query execution.
Please give me suggestion on that.
Thanks!!!
I have added index to the columns used in the join/where clauses.
Like :-
ALTER TABLE sattool_desc ADD INDEX sattool_testing_id (sattool_testing_id);
ALTER TABLE sattool_error_log ADD INDEX sattool_id (sattool_id);
ALTER TABLE sattool_testing ADD INDEX isAuto (isAuto);
Now the execution time is 100 rows in set (0.00 sec).
Thanks again.

Optimizing SQL query in MySQL

I would like to know why this query takes is slow (about 10 to 20 seconds), the three tables used have 500,000 records, this is the query:
SELECT *, 'rg_egresos' AS nombre_tabla
FROM rg_detallexml DE
INNER JOIN rg_egresos EG
INNER JOIN rg_emisor EM ON DE.idContador = EG.id
AND DE.idDetalleXml = EG.idDetalleXml
AND DE.idContador = EM.idContador
AND DE.idDetalleXml = EM.idDetalleXml
WHERE DE.idContador = '14894'
AND DATE_FORMAT(dateFechaHora, '%Y-%m-%d') BETWEEN '2017-10-01'
AND '2017-10-31'
AND strTipodeComprobante = 'egreso'
AND version_xml = '3.2'
AND estado_factura = 0
AND modificado = 0;
And this is what it shows when I use EXPLAIN
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: EG
type: index_merge
possible_keys: idx_idDetallexml,idx_estado_factura,idx_modificado,idx_idContador
key: idx_idContador,idx_estado_factura,idx_modificado
key_len: 4,4,4
ref: NULL
rows: 2111
Extra: Using intersect(idx_idContador,idx_estado_factura,idx_modificado); Using where
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: DE
type: eq_ref
possible_keys: PRIMARY,idx_strTipodeComprobante,idx_idContador,idx_version_xml
key: PRIMARY
key_len: 4
ref: db_pwf.EG.idDetalleXml
rows: 1
Extra: Using where
*************************** 3. row ***************************
id: 1
select_type: SIMPLE
table: EM
type: ref
possible_keys: idx_idContador,idx_idDetallexml
key: idx_idDetallexml
key_len: 4
ref: db_pwf.DE.idDetalleXml
rows: 1
Extra: Using where
Can you see a way to improve the query?, I have other queries working with bigger tables and they are faster, all the required fields have its index, thanks.
Table rg_detallexml:
+---------------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------------------+--------------+------+-----+---------+----------------+
| idDetalleXml | int(10) | NO | PRI | NULL | auto_increment |
| UUID | varchar(50) | NO | MUL | NULL | |
| dateFechaSubida | varchar(7) | YES | | NULL | |
| idContador | int(10) | NO | MUL | NULL | |
| dateFechaHora | datetime | YES | MUL | NULL | |
| dateFechaHoraCertificacion | datetime | YES | | NULL | |
| dateFechaPago | datetime | YES | | NULL | |
| intFolio | int(10) | YES | | NULL | |
| strSerie | varchar(2) | YES | | A | |
| doubleDescuento | double | YES | | NULL | |
| doubleTotal | double | YES | | NULL | |
| doubleSubtotal | double | YES | | NULL | |
| duobleTotalImpuestosTrasladados | double | YES | | NULL | |
| doubleTotalImpuestosRetenidos | double | YES | | NULL | |
| doubleTotalRetencionesLocales | double | YES | | NULL | |
| doubleTotalTrasladosLocales | double | YES | | NULL | |
| strTipodeComprobante | varchar(15) | YES | MUL | NULL | |
| strMetodoDePago | varchar(150) | YES | | NULL | |
| strFormaDePago | varchar(150) | YES | | NULL | |
| strMoneda | varchar(10) | YES | | NULL | |
| tipoCambio | double | NO | | NULL | |
| strLugarExpedicion | varchar(150) | YES | | NULL | |
| DIOT | int(1) | YES | | 0 | |
| version_xml | varchar(10) | NO | MUL | NULL | |
+---------------------------------+--------------+------+-----+---------+----------------+
Table rg_egresos:
+---------------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------------+--------------+------+-----+---------+----------------+
| id_egreso | int(11) | NO | PRI | NULL | auto_increment |
| id | int(11) | NO | MUL | NULL | |
| idDetalleXml | int(10) | NO | MUL | NULL | |
| idCatalogo | int(19) | NO | MUL | NULL | |
| tipoCuenta | int(11) | NO | MUL | NULL | |
| intRubro | int(1) | NO | | NULL | |
| RFC | varchar(20) | NO | MUL | NULL | |
| compra_gastos_0_porciento | float | NO | MUL | NULL | |
| deducible | int(1) | NO | | NULL | |
| compra_gastos_exentos | float | NO | | NULL | |
| no_deducibles | float | NO | | NULL | |
| estado_factura | int(11) | NO | MUL | NULL | |
| fecha | date | NO | MUL | NULL | |
| total_xml | double | NO | | NULL | |
| subtotal_xml | double | NO | | NULL | |
| iva_xml | double | NO | | NULL | |
| total_impuestos | double | NO | | NULL | |
| abonado | double | NO | | NULL | |
| subtotal | double | NO | | NULL | |
| iva | double | NO | | NULL | |
| pendiente | double | NO | | NULL | |
| subtotal_sin_iva | double | NO | | NULL | |
| acreditable | int(1) | NO | MUL | 0 | |
| fecha_operacion | datetime | NO | MUL | NULL | |
| modificado | int(1) | NO | MUL | NULL | |
| UUID | varchar(50) | NO | MUL | NULL | |
| IEPS | double | NO | | NULL | |
| retencion_iva | double | NO | | NULL | |
| retencion_isr | double | NO | | NULL | |
| imp_local | double | NO | | 0 | |
| enviado_a | int(11) | NO | MUL | NULL | |
| enviado_al_iva | int(1) | NO | | NULL | |
| EsNomina | int(1) | NO | MUL | 0 | |
| dateFechaPago | date | NO | MUL | NULL | |
| nota_credito | int(1) | NO | MUL | NULL | |
| extranjero | int(1) | NO | MUL | NULL | |
| pago_banco | int(1) | NO | MUL | NULL | |
| idBanco_Pago | int(20) | NO | MUL | NULL | |
| movimientoPago | int(10) | NO | | NULL | |
| saldo_banco | varchar(50) | NO | | NULL | |
| tipo_pago | int(1) | NO | | 0 | |
| responsable | varchar(100) | NO | | NULL | |
+---------------------------+--------------+------+-----+---------+----------------+
Table rg_emisor:
+-----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+--------------+------+-----+---------+----------------+
| idEmisor | int(10) | NO | PRI | NULL | auto_increment |
| idDetalleXml | int(10) | NO | MUL | NULL | |
| idContador | int(10) | NO | MUL | NULL | |
| strRFC | varchar(13) | NO | | NULL | |
| strNombreEmisor | varchar(200) | YES | | NULL | |
| strRegimen | varchar(250) | YES | | NULL | |
| strPais | varchar(40) | YES | | MX | |
| strEstado | varchar(50) | YES | | NULL | |
| intCP | int(5) | YES | | NULL | |
| strMunicipio | varchar(250) | YES | | NULL | |
| strLocalidad | varchar(250) | YES | | NULL | |
| strColonia | varchar(250) | YES | | NULL | |
| intNumExt | int(10) | YES | | NULL | |
| intNumInt | int(10) | YES | | NULL | |
| strCalle | varchar(250) | YES | | NULL | |
| regimenFiscal | varchar(20) | YES | | NULL | |
+-----------------+--------------+------+-----+---------+----------------+
Now that you've shown the tables, we see that rg_egresos.id is not the table's ID. There can hence be multiple records for one contador in the table. Let's look at the tables and the query more closely:
All tables contain a contador ID and a DetalleXml ID. You want to join them all on these two fields. So you start with the rg_detallexml and get all records for the contador. With the idDetalleXml thus found, you search for rg_egresos and rg_emisors.
This is a bit strange. First of all an rg_detallexml is obviously linked to one contador, but in the other tables the rg_detallexml can be linked to another contador. Well, that may be possible (some kind of from/to relation maybe). But with five rg_egresos records and four rg_emisors records for an rg_detallexml/contador, you'd select thirty records, because you are combining rg_egresos records with rg_emisors records that are not really related.
Anyway: you want to find rg_detallexml quickly.
create index idx_de on rg_detallexml(idcontador, strtipodecomprobante, version_xml,
datefechahora, iddetallexml);
Then you look for rg_egresos:
create index idx_eg on rg_egresos(id, iddetallexml, estado_factura, modificad);
At last you look for rg_emisor:
create index idx_em on rg_emisor(idcontador, iddetallexml);
As the columns are present in all tables, we could of course go through them in any order. Starting with rg_detallexml seems most natural and most restrictive, too, but that is not necessarily best. So you may want to offer the DBMS yet another index:
create index idx_eg2 on rg_egresos(id, estado_factura, modificad, iddetallexml);
which would allow the DBMS to look up the contador's records in this table first and with the added criteria find related iddetallexml here.
The biggest problem I see is on this part:
DATE_FORMAT(dateFechaHora, '%Y-%m-%d') BETWEEN '2017-10-01' AND '2017-10-31'
is dateFechaHora a datetime field? Why are you converting a datetime field to a string (DATE_FORMAT)? even if you have an index on the dateFechaHora field, it won't be used.
I would suggest you to use this code instead:
and DateFechaHora >= '2017-10-01' and DateFechaHora < '2017-11-01'
^^^^^^^^^^
yes it's the following day and it won't be included.
So your query might look like this:
select
*,
'rg_egresos' AS nombre_tabla
from
rg_detallexml DE inner join rg_egresos EG
on DE.idContador = EG.id and DE.idDetalleXml = EG.idDetalleXml
inner join rg_emisor EM on DE.idContador = EM.idContador
and DE.idDetalleXml = EM.idDetalleXml
where
DE.idContador = '14894'
and dateFechaHora >= '2017-10-01' and dateFechaHora < '2017-11-01'
and strTipodeComprobante = 'egreso'
and version_xml = '3.2'
and estado_factura = 0
and modificado = 0
;
I see two partial Answers in the other replies. Let's tie them together.
Change
AND DATE_FORMAT(dateFechaHora, '%Y-%m-%d') BETWEEN '2017-10-01'
AND '2017-10-31'
to
AND DE.dateFechaHora >= '2017-10-01'
AND DE.dateFechaHora < '2017-10-01' + INTERVAL 1 MONTH
and
If DE is a good starting table:
DE: INDEX(idContador, strTipodeComprobante, version_xml, dateFechaHora)
-- date last; others in any order
If EG is a better starting table:
EG: INDEX(estado_factura, modificado, id) -- in any order
DE: INDEX(idContador, idDetalleXml,
strTipodeComprobante, version_xml, dateFechaHora)
Also have
EM: INDEX(idContador, idDetalleXml) -- in either order
"Using intersect" almost always is a clue that you should have a composite index instead of separate indexes. (The separate indexes may be useful for other queries.)
(That is, add all those indexes, then let the Optimizer decide.)
Please use SHOW CREATE TABLE, not the less-descriptive DESCRIBE.
Do you really need SELECT *?
The query, after my suggestions:
SELECT DE.*,
EG.*,
EM.*,
'rg_egresos' AS nombre_tabla
FROM rg_detallexml DE
INNER JOIN rg_egresos EG
ON DE.idContador = EG.id
AND DE.idDetalleXml = EG.idDetalleXml
INNER JOIN rg_emisor EM
ON DE.idContador = EM.idContador
AND DE.idDetalleXml = EM.idDetalleXml
WHERE DE.idContador = '14894'
AND DE.dateFechaHora >= '2017-10-01'
AND DE.dateFechaHora < '2017-10-01' + INTERVAL 1 MONTH
AND DE.strTipodeComprobante = 'egreso'
AND DE.version_xml = '3.2'
AND EG.estado_factura = 0
AND EG.modificado = 0;

Merge data from two cells from the same column in different rows in MySQL/MariaDB

I am having trouble searching for an answer to this question, because of my lack of knowledge about the terminology and SQL, even though I know it probably exists.
I have a database with the following tables:
desc pkm;
+-----------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+---------------+------+-----+---------+-------+
| pkm_code | int(11) | NO | PRI | NULL | |
| pkm_name | varchar(32) | NO | UNI | NULL | |
| pkm_category | varchar(32) | NO | | NULL | |
| pkm_description | varchar(1280) | NO | | NULL | |
| pkm_weight | float | NO | | NULL | |
| evolution_code | int(11) | YES | MUL | NULL | |
+-----------------+---------------+------+-----+---------+-------+
desc poketype;
+---------------------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+------------+------+-----+---------+-------+
| pkm_code | int(11) | NO | PRI | NULL | |
| type_code | int(11) | NO | PRI | NULL | |
| poketype_is_primary | tinyint(1) | NO | | NULL | |
+---------------------+------------+------+-----+---------+-------+
desc type;
+-----------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+-------+
| type_code | int(11) | NO | PRI | NULL | |
| type_name | varchar(32) | NO | UNI | NULL | |
+-----------+-------------+------+-----+---------+-------+
And so far I have the following SQL command:
SELECT pkm.pkm_code, pkm.pkm_name,type.type_name FROM poketype
JOIN pkm ON pkm.pkm_code=poketype.pkm_code
JOIN type ON poketype.type_code=type.type_code
WHERE pkm.pkm_code<=151
ORDER BY pkm_code;
Which displays the primary and secondary types on separate lines.
How would I get both types to display on the same row for dual-type pokemon?
My current results:
+-----------+-------------+-----------+
| pkm_code | pkm_name | type_name |
+-----------+-------------+-----------+
| 1 | Bulbasaur | grass |
| 1 | Bulbasaur | poison |
Desired results:
+-----------+-------------+-------------+
| pkm_code | pkm_name | type_name |
+-----------+-------------+-------------+
| 1 | Bulbasaur | grass,poison|
(Yes, bulbasaur is a dual type. I was surprised too!)
Use mysql's group_concat() function to combine values from different records in a single value:
SELECT pkm.pkm_code, pkm.pkm_name, group_concat(type.type_name) as typename FROM poketype
JOIN pkm ON pkm.pkm_code=poketype.pkm_code
JOIN type ON poketype.type_code=type.type_code
WHERE pkm.pkm_code<=151
GROUP BY pkm.pkm_code, pkm.pkm_name;

Determine if a field from one table is like the field from another table and if so count occurrence and if less than 3 return the name

I have two tables images and Interactions in the same DB. I want to determine if the image.images which has a png file name for example
1001_A01_1-4_5mM_3AT_Xgal_7d_W.cropped.resized.grey.png is like plate_name.Interactions which would look like 1001_A01 and then count how many times the images show up. I should get 3 images if there are fewer than 3 images I would like to have plate_name.Interactions returned. I would like to do this using 1 query.
So far I have just tried to count how many occurrences there are but this is failing:
select plate_name.Interactions, count(*) as count from Interactions where plate_name.Interactions like image.images;
Here are the tables in question:
mysql> desc images;
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| image | varchar(100) | NO | MUL | NULL | |
| user_id | varchar(50) | YES | | NULL | |
| project_id | varchar(50) | YES | | NULL | |
+------------+--------------+------+-----+---------+-------+
mysql> desc Interactions;
+----------------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+-------------+------+-----+---------+-------+
| plate_name | varchar(25) | NO | MUL | NULL | |
| plate_number | int(11) | NO | MUL | NULL | |
| bait_sequence_name | varchar(25) | NO | | NULL | |
| bait_gene_promoter | varchar(25) | NO | MUL | NULL | |
| array_coord | varchar(25) | NO | | NULL | |
| transcriptor_factor | varchar(25) | NO | | NULL | |
| orf_name | varchar(25) | NO | | NULL | |
| y_coord | varchar(25) | NO | MUL | NULL | |
| x_coord | varchar(25) | NO | MUL | NULL | |
| orig_intensity_value | varchar(25) | NO | | NULL | |
| rc_intensity_value | varchar(25) | NO | | NULL | |
| ptp_intensity_value | varchar(25) | NO | | NULL | |
| z_score | varchar(25) | NO | MUL | NULL | |
| z_prime | varchar(20) | YES | | NULL | |
| call_type | varchar(25) | NO | | NULL | |
| bleed_over | varchar(25) | NO | MUL | NULL | |
| plate_median | int(11) | YES | MUL | NULL | |
| bait_gene | varchar(25) | NO | | NULL | |
| bait_prey_orf | varchar(25) | NO | | NULL | |
| human_call | varchar(25) | NO | | NULL | |
| modified_call | varchar(25) | NO | | NULL | |
| duplicate_call | varchar(25) | YES | | NULL | |
| user_id | varchar(25) | YES | MUL | NULL | |
| project_id | varchar(25) | YES | MUL | NULL | |
+----------------------+-------------+------+-----+---------+-------+
Added in response to Bobby's answer:
+----+-------------+--------------+-------+---------------+--------------+---------+--- ---+---------+-----------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+-------+---------------+--------------+---------+--- ---+---------+-----------------------------------------------------------+
| 1 | SIMPLE | images | index | NULL | image | 208 | NULL | 19581 | Using where; Using index; Using temporary; Using filesort |
| 1 | SIMPLE | Interactions | range | plate_median | plate_median | 5 | NULL | 3714984 | Using where; Using join buffer |
+----+-------------+--------------+-------+---------------+--------------+---------+------+---------+-----------------------------------------------------------+
I think you write your table name and your column name in incorrect format..
if I'm right, the query should return only plate_name which it has less than 3 images,
so you can do it with a GROUP BY and HAVING method..
SELECT Iteractions.plate_name, count(*) as `count`
FROM Interactions, Images
WHERE Interactions.plate_name LIKE Images.image
GROUP BY Interactions.plate_name
HAVING count(*) < 3
Please try that, and tell me if that's not what you want.. :)

adding an additional item in SQL statement causes slow performance

I have a sql statement that looks like this:
SELECT colID
FROM tableName
WHERE ColDateStart <='$lowerDate'
AND ColDateStart>='$upperDate'
AND ColVcamID='$id1'
AND ColVlviID='$id2'
AND ColSomeID='$id3';
All the columns in the WHERE statment are indexed columns.
When I run this it takes over a second. However when I run this without other Id3, the performance is considerably improved (0.03 seconds).
When I run explain, with otherId3, it uses an index merge using otherId1 and otherId3. However when I remove the otherId3, it uses the single index of otherId2.
Why does adding otherId3 make an impact on the performance?
Table Structure:
+----------------------+-------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+-------------+------+-----+---------------------+----------------+
| ColID | int(11) | NO | PRI | NULL | auto_increment |
| ColCustID | int(11) | NO | MUL | 0 | |
| ColCarrID | int(11) | NO | MUL | NULL | |
| ColTariID | int(11) | NO | MUL | 0 | |
| ColCarrierRef | varchar(30) | NO | MUL | | |
| ColNumbID | int(11) | NO | MUL | 0 | |
| ColVlviID | int(11) | NO | MUL | NULL | |
| ColVcamID | int(11) | NO | MUL | NULL | |
| ColSomeID | int(11) | NO | MUL | NULL | |
| ColVlnsID | int(11) | NO | MUL | NULL | |
| ColNGNumber | varchar(12) | NO | | | |
| ColOrigNumber | varchar(16) | NO | MUL | NULL | |
| ColCLIRestrictedFlag | int(2) | NO | | NULL | |
| ColOrigLocality | varchar(11) | NO | MUL | | |
| ColOrigAreaCode | varchar(11) | NO | MUL | | |
| ColTermNumber | varchar(16) | NO | MUL | NULL | |
| ColBatchNumber | varchar(10) | NO | | | |
| ColDateStart | date | NO | MUL | 0000-00-00 | |
| ColDateClear | date | NO | | 0000-00-00 | |
| ColTimeStart | time | NO | | 00:00:00 | |
| ColTimeClear | time | NO | | 00:00:00 | |
| ColCallLength | time | NO | | 00:00:00 | |
| ColRingLength | time | NO | | 00:00:00 | |
| ColEffectiveFlag | smallint(1) | NO | MUL | NULL | |
| ColUnansweredFlag | smallint(1) | NO | MUL | NULL | |
| ColEngagedFlag | smallint(1) | NO | | NULL | |
| ColRecID | int(11) | NO | MUL | NULL | |
| ColCreatedUserID | int(11) | NO | | 0 | |
| ColCreatedDatetime | datetime | NO | MUL | 0000-00-00 00:00:00 | |
| ColDirection | int(1) | NO | MUL | NULL | |
+----------------------+-------------+------+-----+---------------------+----------------+
Indexes
+-------+------------+-------------------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-------------------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+
| tableName | 0 | PRIMARY | 1 | ColID | A | 18031283 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_ColCustID | 1 | ColCustID | A | 1339 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_ColNumbID | 1 | ColNumbID | A | 24366 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colOrigNumber | 1 | colOrigNumber | A | 4507820 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colOrigLocality | 1 | colOrigLocality | A | 36873 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colOrigAreaCode | 1 | colOrigAreaCode | A | 696 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colTermNumber | 1 | colTermNumber | A | 137643 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colDateStart | 1 | colDateStart | A | 3639 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colEffectiveFlag | 1 | colEffectiveFlag | A | 2 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colUnansweredFlag | 1 | colUnansweredFlag | A | 2 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colEngagedFlag | 1 | colUnansweredFlag | A | 2 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colTariID | 1 | colTariID | A | 91 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_CustID_DateStart | 1 | colCustID | A | 1339 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_CustID_DateStart | 2 | colDateStart | A | 693510 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_NumbID_DateStart | 1 | colNumbID | A | 24366 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_NumbID_DateStart | 2 | colDateStart | A | 4507820 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colRecID | 1 | colRecID | A | 214658 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colCarrierRef | 1 | colCarrierRef | A | 6010427 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colCustID_colTermNumber | 1 | colCustID | A | 1339 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colCustID_colTermNumber | 2 | colTermNumber | A | 143105 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colCreatedDatetime | 1 | colCreatedDatetime | A | 474507 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colDirection | 1 | colDirection | A | 2 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colVlviID | 1 | colVlviID | A | 4133 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colSomeID | 1 | colSomeID | A | 10 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colVcamID | 1 | colVcamID | A | 7 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colVlnsID | 1 | colVlnsID | A | 18 | NULL | NULL | | BTREE | |
| tableName | 1 | idx_colCarrID | 1 | colCarrID | A | 4 | NULL | NULL | | BTREE | |
+-------+------------+-------------------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+
First up, you have far too may indexes. Most of those indexes are likely pointless. If you aren't using a specific index for WHERE criteria, JOINs, or ORDERing, then remove it as it just slows things down.
Next up, for your query you specify 3 columns that are always in the query comprising 1x DATE and 2x INT columns. The DATE column should be first as a date range is pretty fast on an index, and then the two INTs. This gives a starting point of the following 3-column index
CREATE INDEX searchIndex
ON tableName (ColDateStart,ColVlviID,ColVcamID)
USING BTREE;
More information on that available here : http://dev.mysql.com/doc/refman/5.0/en/multiple-column-indexes.html
I ordered the columns that way intentionally. ColDateStart for the date range filtering, followed by the other columns in decreasing order of cardinality. I selected ColVlviID as a second column as it has a cardinality of 4133 compared to ColVcamID having a cardinality of 7. This will allow MySQL to more efficiently reduce the matching rows.
Now, assuming colSomeID is the last column, I might instead do the following
CREATE INDEX searchIndex_someID
ON tableName (ColDateStart,ColVlviID,ColVcamID,ColSomeID)
USING BTREE;
This 3-column index will help MySQL get down to the applicable dataset before checking for that last ID. Optioanlly you could add a 4th column to that index at the end, if you are commonly filtering in another particular INT column.
As an aside, you may want to consider the following instead of the dateCol criteria
SELECT colID
FROM tableName
WHERE ColDateStart BETWEEN DATE('$lowerDate') AND DATE('$upperDate')
AND ColVcamID=$id1
AND ColVlviID=$id2
AND ColSomeID=$id3
The above all of course assumes that you are sanitising the variables before executing the query. I have removed the quotes from the $idx variables, as they should be numeric and therefore do not need to be entered as strings.
SELECT id
FROM tableName FORCE INDEX(`idx_otherId2`)
WHERE dateCol <='$lowerDate'
AND dateCol>='$upperDate'
AND otherId1='$id1'
AND otherId2='$id2'
AND otherId3='$id3';