How to decrease this mysql query execution time? - mysql

Parent Table,
mysql> desc sattool_testing;
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| circuit_id | varchar(30) | NO | | NULL | |
| coff_id | varchar(100) | NO | | NULL | |
| result | int(11) | YES | | NULL | |
| result_details | text | NO | | NULL | |
| details | longtext | NO | | NULL | |
| reverse_response | varchar(100) | NO | | NULL | |
| start_date | datetime | NO | | NULL | |
| end_date | datetime | YES | | NULL | |
| Modules | varchar(100) | YES | | NULL | |
| rehit | datetime | YES | | NULL | |
| isAuto | tinyint(1) | NO | | 0 | |
+------------------+--------------+------+-----+---------+----------------+
12 rows in set (0.00 sec)
1st Child Table,
mysql> desc sattool_desc;
+--------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+---------+----------------+
| sattool_desc_id | bigint(20) | NO | PRI | NULL | auto_increment |
| ceinterface | varchar(50) | YES | | NULL | |
| cehostname | varchar(50) | YES | | NULL | |
| vprnno | varchar(50) | YES | | NULL | |
| policyname | varchar(50) | YES | | NULL | |
| cosno | varchar(10) | YES | | NULL | |
| peipaddress | varchar(17) | YES | | NULL | |
| router_type | varchar(5) | YES | | NULL | |
| sattool_testing_id | bigint(20) | YES | | NULL | |
| epipeid | varchar(20) | YES | | NULL | |
| cerouter | varchar(10) | YES | | NULL | |
| service_type | varchar(10) | YES | | NULL | |
| scope_of_manage | varchar(50) | YES | | NULL | |
| service_name | varchar(10) | YES | | NULL | |
| ce_vrf_name | varchar(150) | YES | | NULL | |
+--------------------+--------------+------+-----+---------+----------------+
15 rows in set (0.01 sec)
2nd Child Table (Contain multiple records against one parent table record),
mysql> desc sattool_error_log;
+----------------+--------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+-------------------+-----------------------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| service_id | varchar(30) | NO | | NULL | |
| coff_id | varchar(50) | NO | | NULL | |
| error_name | varchar(100) | NO | | NULL | |
| error_desc | text | NO | | NULL | |
| error_occurred | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| error_type | int(11) | YES | | NULL | |
| error_code | varchar(10) | YES | | NULL | |
| sattool_id | bigint(20) | YES | | 0 | |
| module_no | int(11) | YES | | 0 | |
+----------------+--------------+------+-----+-------------------+-----------------------------+
10 rows in set (0.00 sec)
SQL Query :-
SELECT GROUP_CONCAT(sattool_error_log.error_desc) AS err_desc, GROUP_CONCAT(sattool_error_log.error_type) AS err_type, sattool_testing.*, sattool_desc.service_type, sattool_desc.service_name, sattool_desc.scope_of_manage
FROM sattool_testing
LEFT JOIN sattool_desc ON sattool_testing.id = sattool_desc.sattool_testing_id
LEFT JOIN sattool_error_log ON sattool_testing.id = sattool_error_log.sattool_id
WHERE sattool_testing.isAuto = 1 GROUP BY sattool_testing.id ORDER BY sattool_testing.id DESC limit 100
Current Execution Time is (22.76 sec)
Explain Query :-
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
| 1 | SIMPLE | sattool_testing | ALL | NULL | NULL | NULL | NULL | 3578 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | sattool_desc | ALL | NULL | NULL | NULL | NULL | 4009 | |
| 1 | SIMPLE | sattool_error_log | ALL | NULL | NULL | NULL | NULL | 8904 | |
+----+-------------+-------------------+------+---------------+------+---------+------+------+----------------------------------------------+
3 rows in set (0.00 sec)
I want to increase speed to above query execution.
Please give me suggestion on that.

Thanks!!!
I have added index to the columns used in the join/where clauses.
Like :-
ALTER TABLE sattool_desc ADD INDEX sattool_testing_id (sattool_testing_id);
ALTER TABLE sattool_error_log ADD INDEX sattool_id (sattool_id);
ALTER TABLE sattool_testing ADD INDEX isAuto (isAuto);
Now the execution time is 100 rows in set (0.00 sec).
Thanks again.

Related

why is MySQL casting varchar as integer when WHERE clause is present?

I have a MySQL database table, student_import_record, with a student_id varchar(50) field that can contain alphanumeric data.
When I run this SELECT statement:
select student_id from student_import_record;
I get the VARCHAR results as expected:
0001546
0001660
0002207
0002349
But when I run this SELECT statement with a WHERE clause:
select student_id from student_import_record where student_import_id = 185;
I get results that appear to be cast as an integer:
1546
1660
2207
2349
How do I prevent this casting when using a WHERE clause?
Full Table schema:
mysql> describe student_import_record;
+--------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| student_id | varchar(50) | NO | | NULL | |
| relationship | varchar(50) | NO | | NULL | |
| date_of_birth | varchar(50) | NO | | NULL | |
| first_name | varchar(255) | NO | | NULL | |
| last_name | varchar(255) | NO | | NULL | |
| gender | varchar(10) | NO | | NULL | |
| student_type | varchar(50) | YES | | NULL | |
| email | varchar(255) | YES | | NULL | |
| phone | varchar(20) | YES | | NULL | |
| address | varchar(255) | YES | | NULL | |
| address2 | varchar(255) | YES | | NULL | |
| city | varchar(255) | YES | | NULL | |
| state | varchar(50) | YES | | NULL | |
| zip | varchar(50) | YES | | NULL | |
| mandate | varchar(10) | YES | | NULL | |
| ssn | varchar(11) | YES | | NULL | |
| suffix | varchar(20) | YES | | NULL | |
| department | varchar(50) | YES | | NULL | |
| password | varchar(255) | YES | | NULL | |
| benefit1 | varchar(255) | YES | | NULL | |
| benefit2 | varchar(255) | YES | | NULL | |
| benefit3 | varchar(255) | YES | | NULL | |
| benefit4 | varchar(255) | YES | | NULL | |
| benefit5 | varchar(255) | YES | | NULL | |
| benefit6 | varchar(255) | YES | | NULL | |
| plan1 | varchar(255) | YES | | NULL | |
| plan2 | varchar(255) | YES | | NULL | |
| plan3 | varchar(255) | YES | | NULL | |
| plan4 | varchar(255) | YES | | NULL | |
| plan5 | varchar(255) | YES | | NULL | |
| plan6 | varchar(255) | YES | | NULL | |
| account | int(11) | NO | | NULL | |
| enrollment_config | int(11) | NO | | NULL | |
| imported_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| status | smallint(6) | YES | | NULL | |
| status_description | longblob | YES | | NULL | |
| student_import_id | int(11) | NO | | NULL | |
Result sets
mysql> select student_id from student_import_record where student_import_id = 185 limit 10;
+------------+
| student_id |
+------------+
| 1546 |
| 1660 |
| 2207 |
| 2349 |
| 3123 |
| 3208 |
| 3319 |
| 3811 |
| 3837 |
| 3842 |
+------------+
10 rows in set (0.02 sec)
mysql> select student_id from student_import_record limit 10;
+------------+
| student_id |
+------------+
| 0001546 |
| 0001660 |
| 0002207 |
| 0002349 |
| 0003123 |
| 0003208 |
| 0003319 |
| 0003811 |
| 0003837 |
| 0003842 |
+------------+
10 rows in set (0.00 sec)
Looks like you have already figured out what was going on. In the hope to close out this question, here's a link that explains type conversion in expression evaluation: https://dev.mysql.com/doc/refman/5.5/en/type-conversion.html
When an operator is used with operands of different types, type
conversion occurs to make the operands compatible. Some conversions
occur implicitly. For example, MySQL automatically converts numbers to
strings as necessary, and vice versa.
mysql> SELECT 1+'1';
-> 2
In your case, MySQL might be seeing arithmetic operation in the WHERE clause and performing implicit conversion to integers in that column. My recommendation is to structure the table, if possible, in a manner that numbers are in numeric column and non-numeric data is in its appropriate column. When there's mixed content, you could force the data to be character.

how to add a foreign key using an existing column in mysql

I have following table called transactions and I want to add foreign key using column account_id. But when I execute the query (see below) 0 rows affected. I am not getting what is wrong with the query.
I have two tables called transactions and accounts. Accounts table has an id as primary key and an account has_many transactions.
transactions table
+---------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| account_number | int(11) | YES | | NULL | |
| m_number | varchar(255) | YES | | NULL | |
| registration_number | varchar(255) | YES | | NULL | |
| page_number | varchar(255) | YES | | NULL | |
| entry_date | datetime | YES | | NULL | |
| narration | varchar(255) | YES | | NULL | |
| voucher_number | varchar(255) | YES | | NULL | |
| debit_credit | float | YES | | NULL | |
| profit | float | YES | | NULL | |
| account_code | int(11) | YES | | NULL | |
| balance | float | YES | | NULL | |
| branch | varchar(255) | YES | | NULL | |
| account_id | int(11) | YES | MUL | NULL | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
+---------------------+--------------+------+-----+---------+----------------+
16 rows in set (0,01 sec)
mysql> ALTER TABLE transactions ADD FOREIGN KEY (account_id) REFERENCES accounts(id);
Query OK, 0 rows affected (0,03 sec)
Records: 0 Duplicates: 0 Warnings: 0
accounts table
+-----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| account_number | int(11) | YES | | NULL | |
| account_code | int(11) | YES | | NULL | |
| branch | varchar(255) | YES | | NULL | |
| name | varchar(255) | YES | | NULL | |
| father_name | varchar(255) | YES | | NULL | |
| nic_number | varchar(255) | YES | | NULL | |
| mohalla | varchar(255) | YES | | NULL | |
| village | varchar(255) | YES | | NULL | |
| nominee | varchar(255) | YES | | NULL | |
| relationship | varchar(255) | YES | | NULL | |
| opening_balance | float | YES | | NULL | |
| opening_date | datetime | YES | | NULL | |
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
+-----------------+--------------+------+-----+---------+----------------+
After running the query shouldn't the column say F_K or something like PRI ?
Any help would be great. Thanks!

Include null values in SQL query

I have the following four tables in my database:
+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| complex_id | int(11) | NO | PRI | NULL | auto_increment |
| complex_name | varchar(45) | NO | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
+--------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+----------------+
| machine_id | int(11) | NO | PRI | NULL | auto_increment |
| complex_id | int(11) | NO | MUL | NULL | |
| machine_name | varchar(45) | NO | | NULL | |
+--------------+-------------+------+-----+---------+----------------+
+-----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+--------------+------+-----+---------+----------------+
| devices_id | int(11) | NO | PRI | NULL | auto_increment |
| machine_id | int(11) | NO | MUL | NULL | |
| description | varchar(255) | NO | | NULL | |
| location | varchar(255) | YES | | NULL | |
| verification | varchar(255) | YES | | NULL | |
| rack_num | varchar(8) | YES | | NULL | |
| section_num | varchar(8) | YES | | NULL | |
| color_or_number | varchar(16) | YES | | NULL | |
| normal_position | varchar(16) | YES | | NULL | |
+-----------------+--------------+------+-----+---------+----------------+
+------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+----------------+
| pnp_id | int(11) | NO | PRI | NULL | auto_increment |
| devices_id | int(11) | NO | MUL | NULL | |
| pnp_num | varchar(45) | NO | | NULL | |
+------------+-------------+------+-----+---------+----------------+
I am trying to get results formatted as follows (NULL values appear as blanks):
+------------+-------------+-----------------------+
| devices_id | description | pnpnum |
+------------+-------------+-----------------------+
| 1 | ex | 1234 |
| 2 | ex2 | 2345 |
| 3 | ex3 | |
| 4 | ex4 | 3456, 4567, 5678, 6879|
+------------+-------------+-----------------------+
Using the following SQL query,
SELECT *, GROUP_CONCAT(pnp.pnp_num separator ', ') pnpnum
FROM devices
JOIN pnp ON devices.devices_id = pnp.devices_id
WHERE devices.machine_id = 1
GROUP BY devices.devices_ID
ORDER BY devices.description;
my results are relatively close, however, I am unable to include a device if it has a null pnpnum.
+------------+-------------+-----------------------+
| devices_id | description | pnpnum |
+------------+-------------+-----------------------+
| 1 | ex | 1234 |
| 2 | ex2 | 2345 |
| 4 | ex4 | 3456, 4567, 5678, 6879|
+------------+-------------+-----------------------+
What is it that I am missing from my SQL statement that will allow me to include null values?
You need to use LEFT JOIN because even if there isn't a match, it will return all the results from the left table leaving the fields from the right table null.

Determine if a field from one table is like the field from another table and if so count occurrence and if less than 3 return the name

I have two tables images and Interactions in the same DB. I want to determine if the image.images which has a png file name for example
1001_A01_1-4_5mM_3AT_Xgal_7d_W.cropped.resized.grey.png is like plate_name.Interactions which would look like 1001_A01 and then count how many times the images show up. I should get 3 images if there are fewer than 3 images I would like to have plate_name.Interactions returned. I would like to do this using 1 query.
So far I have just tried to count how many occurrences there are but this is failing:
select plate_name.Interactions, count(*) as count from Interactions where plate_name.Interactions like image.images;
Here are the tables in question:
mysql> desc images;
+------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| image | varchar(100) | NO | MUL | NULL | |
| user_id | varchar(50) | YES | | NULL | |
| project_id | varchar(50) | YES | | NULL | |
+------------+--------------+------+-----+---------+-------+
mysql> desc Interactions;
+----------------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+-------------+------+-----+---------+-------+
| plate_name | varchar(25) | NO | MUL | NULL | |
| plate_number | int(11) | NO | MUL | NULL | |
| bait_sequence_name | varchar(25) | NO | | NULL | |
| bait_gene_promoter | varchar(25) | NO | MUL | NULL | |
| array_coord | varchar(25) | NO | | NULL | |
| transcriptor_factor | varchar(25) | NO | | NULL | |
| orf_name | varchar(25) | NO | | NULL | |
| y_coord | varchar(25) | NO | MUL | NULL | |
| x_coord | varchar(25) | NO | MUL | NULL | |
| orig_intensity_value | varchar(25) | NO | | NULL | |
| rc_intensity_value | varchar(25) | NO | | NULL | |
| ptp_intensity_value | varchar(25) | NO | | NULL | |
| z_score | varchar(25) | NO | MUL | NULL | |
| z_prime | varchar(20) | YES | | NULL | |
| call_type | varchar(25) | NO | | NULL | |
| bleed_over | varchar(25) | NO | MUL | NULL | |
| plate_median | int(11) | YES | MUL | NULL | |
| bait_gene | varchar(25) | NO | | NULL | |
| bait_prey_orf | varchar(25) | NO | | NULL | |
| human_call | varchar(25) | NO | | NULL | |
| modified_call | varchar(25) | NO | | NULL | |
| duplicate_call | varchar(25) | YES | | NULL | |
| user_id | varchar(25) | YES | MUL | NULL | |
| project_id | varchar(25) | YES | MUL | NULL | |
+----------------------+-------------+------+-----+---------+-------+
Added in response to Bobby's answer:
+----+-------------+--------------+-------+---------------+--------------+---------+--- ---+---------+-----------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+-------+---------------+--------------+---------+--- ---+---------+-----------------------------------------------------------+
| 1 | SIMPLE | images | index | NULL | image | 208 | NULL | 19581 | Using where; Using index; Using temporary; Using filesort |
| 1 | SIMPLE | Interactions | range | plate_median | plate_median | 5 | NULL | 3714984 | Using where; Using join buffer |
+----+-------------+--------------+-------+---------------+--------------+---------+------+---------+-----------------------------------------------------------+
I think you write your table name and your column name in incorrect format..
if I'm right, the query should return only plate_name which it has less than 3 images,
so you can do it with a GROUP BY and HAVING method..
SELECT Iteractions.plate_name, count(*) as `count`
FROM Interactions, Images
WHERE Interactions.plate_name LIKE Images.image
GROUP BY Interactions.plate_name
HAVING count(*) < 3
Please try that, and tell me if that's not what you want.. :)

CakePHP-2.0: Refactor my code editing faster sql query, need a faster sql query

mysql> describe posts;
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | | NULL | |
| title | varchar(255) | NO | | NULL | |
| body | text | YES | | NULL | |
| category_id | int(11) | NO | | NULL | |
| tags | varchar(50) | NO | | NULL | |
| mark | tinyint(4) | NO | | 1 | |
| created | datetime | YES | | NULL | |
| modified | datetime | YES | | NULL | |
+-------------+--------------+------+-----+---------+----------------+
9 rows in set (0.00 sec)
mysql> describe comments;
+----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| post_id | int(11) | NO | MUL | NULL | |
| name | varchar(255) | NO | | NULL | |
| email | varchar(255) | NO | | NULL | |
| body | varchar(500) | NO | | NULL | |
| mark | tinyint(4) | NO | | 1 | |
| created | datetime | YES | | NULL | |
| modified | datetime | YES | | NULL | |
+----------+--------------+------+-----+---------+----------------+
8 rows in set (0.00 sec)
I need the posts.title which has more comments limit 10 or there is no limit.
What i tried so far=>
$conditions=array(
'fields'=>array('Comment.post_id'),
'group'=>array('Comment.post_id'),
'order'=>array('count(Comment.post_id) DESC'),
'limit'=>'5'
);
$mostComments=$this->Post->Comment->find('all',$conditions);
$postId=array();
foreach($mostComments as $val){
array_push($postId,$val['Comment']['post_id']);
}
$postsWithmostComments=$this->Post->find('all',array('conditions'=>array('Post.id '=>$postId)) );
$this->set('postsWithmostComments',$postsWithmostComments);
Can anyone post sql query to find posts.it,posts.title with more comments? Or any cakephp find command?
Try this
$contain = array('Comment');
$posts = $this->Post->find('all', array('contain'=>$contain));
$posts_with_comments = array_filter($posts, 'ten_or_more');
uasort($posts_with_comments, 'order_by_comment_count');
$this->set('postsWithmostComments',$posts_with_comments);
function order_by_comment_count($a, $b) {
if (count($a['Comment'] == $b['Comment'])
return 0;
return ($a['Comment'] < $b['Comment']) ? -1 : 1;
}
function ten_or_more($post) {
return (count($post['Comment']) >= 10);
}