Table and Query optimization - mysql

Simple web app, hosted on standard shared hosting using PHp and MySql backend.
I have two relatively small tables (few hundred rows in each) that when I query with a join the query can take two to three seconds.
Here's one table: 'students':
CREATE TABLE IF NOT EXISTS `students` (
`_id` mediumint(9) NOT NULL AUTO_INCREMENT,
`firstname` char(30) NOT NULL,
`lastname` char(30) NOT NULL,
`dateOfBirth` date DEFAULT NULL,
`gender` varchar(10) NOT NULL,
`knownasname` varchar(30) NOT NULL,
`school` varchar(30) DEFAULT NULL,
`schoolyear` varchar(30) DEFAULT NULL,
`notes` text,
`foundusvia` varchar(30) DEFAULT NULL,
`dateadded` datetime NOT NULL,
`archived` tinyint(1) DEFAULT NULL,
PRIMARY KEY (`_id`),
UNIQUE KEY `_id` (`_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=684 ;
Here's the second:
CREATE TABLE IF NOT EXISTS `contacts` (
`_id` mediumint(9) NOT NULL AUTO_INCREMENT,
`student_id` mediumint(9) NOT NULL,
`studentAddressStreet` varchar(50) DEFAULT NULL,
`studentAddressArea` varchar(30) DEFAULT NULL,
`studentAddressTown` varchar(30) DEFAULT NULL,
`studentAddressPostcode` varchar(30) DEFAULT NULL,
`studentPhoneMobile` varchar(30) DEFAULT NULL,
`studentPhoneLand` varchar(30) DEFAULT NULL,
`studentAddressNotes` text,
`studentemail` varchar(50) DEFAULT NULL,
`billingAddressStreet` varchar(50) DEFAULT NULL,
`billingAddressArea` varchar(30) DEFAULT NULL,
`billingAddressTown` varchar(30) DEFAULT NULL,
`billingAddressPostcode` varchar(30) DEFAULT NULL,
`billingPhoneMobile` varchar(30) DEFAULT NULL,
`billingPhoneLand` varchar(30) DEFAULT NULL,
`billingContactName` varchar(30) DEFAULT NULL,
`billingContactRelationship` varchar(30) DEFAULT NULL,
`billingAddressNotes` text,
`billingemail` varchar(50) DEFAULT NULL,
`caregiverAddressStreet` varchar(50) DEFAULT NULL,
`caregiverAddressArea` varchar(30) DEFAULT NULL,
`caregiverAddressTown` varchar(30) DEFAULT NULL,
`caregiverAddressPostcode` varchar(30) DEFAULT NULL,
`caregiverPhoneMobile` varchar(30) DEFAULT NULL,
`caregiverPhoneLand` varchar(30) DEFAULT NULL,
`caregiverContactName` varchar(30) DEFAULT NULL,
`caregiverContactRelationship` varchar(30) DEFAULT NULL,
`caregiverAddressNotes` text,
`caregiveremail` varchar(50) DEFAULT NULL,
PRIMARY KEY (`_id`),
KEY `contacts_index` (`student_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=290 ;
Here's the query:
Its job is to get all the data from the table "students' and 'contacts' for the student with a given _id.
SELECT *
FROM students, contacts
WHERE students._id = contacts.student_id
AND students._id =99 <=== This value obviously changes
The table contacts has an index on student_id
The table students has an index on firstname and lastname.
What is wrong to make this query take 2 to 3 seconds to run? The tables are small and the query simple. Any help greatly appreciated!!
edit: here's the output of EXPLAIN. Also revised CREATE for contacts table
id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra
===|=============|=========|======|===============|==============|=========|======|=============
1 | SIMPLE | students|const | PRIMARY | PRIMARY | 3 |const | 1 |
1 | SIMPLE | contacts|ref | contacts_index|contacts_index| 3 |const | 1 |

Pro tip: If at all possible avoid SELECT *. It's especially problematic on JOINs.
You'll have an easier time reading your query if you use a standard JOIN, like so. That comma-separated list of tables jumped the shark sometime in the late 1990s.
SELECT students.*,
contacts.*
FROM students
LEFT JOIN contacts ON students._id = contacts.student_id
WHERE students._id =99
Finally, put an index on the student_id column of the contacts table so the join can quickly access that table.

Related

how to calculate the number of a table's rows changed(update or insert) in a day on mysql

I have a trade records table and i want to calculate the rows of this table changed including update or insert.
I have check the information_schema in mysql.There is a table called TABLE and it has a column TABLE_ROWS.But i didn't find anything helpful.
well,this table like this
CREATE TABLE pre_actived_apk_record
(
id INT(11) PRIMARY KEY NOT NULL AUTO_INCREMENT,
appid VARCHAR(255) NOT NULL,
channelid VARCHAR(255) NOT NULL,
uid INT(11) NOT NULL,
appname VARCHAR(255) NOT NULL,
packagename VARCHAR(255) NOT NULL,
versioncode VARCHAR(255) NOT NULL,
versionname VARCHAR(255) NOT NULL,
brand VARCHAR(255) NOT NULL,
model VARCHAR(255) NOT NULL,
imei VARCHAR(255) NOT NULL,
sys_version VARCHAR(255) NOT NULL,
mac_address VARCHAR(255) NOT NULL,
imsi VARCHAR(255) NOT NULL,
iccid VARCHAR(255) NOT NULL,
uploadtime INT(11) NOT NULL,
dateline INT(11) NOT NULL,
rule INT(11) NOT NULL,
price INT(11) NOT NULL
);
And i wonder i could use some built-in method to show the result of this table chanaged.

mysql workbench duplicate key error

I have a data set with a list of country names, and the country names are repeated once for "Males" and then again for "Females".
For example:
c_name gender
China M
Greece M
Algeria M
China F
Greece F
Algeria F
When I create table and import data from a csv file, I get a 'duplicate key' error. I am wondering if this has anything to do with the engine settings? Any ideas how this can be resolved? (I know it works because my friend got it to work on her Mac, and she did not have the option to choose 'Collate' or 'Engine' when creating her tables, but I'm on Windows)
EDIT: Here's how I'm creating the table:
CREATE TABLE dbs.enrollment (
e_id INT NOT NULL,
c_name VARCHAR(45) NOT NULL,
gender VARCHAR(45) NULL,
2001 INT NULL,
2002 INT NULL,
2003 INT NULL,
2004 INT NULL,
2005 INT NULL,
2006 INT NULL,
2007 INT NULL,
2008 INT NULL,
2009 INT NULL,
2010 INT NULL,
PRIMARY KEY (e_id, c_name));
Your friend might set a primary key with c_name in this table ,if you want to have a change ,you can cancel the primary key。
It would be better to answer if you shared the query which is used for the above. I agree with Walker Li.
If your query is something like this..
CREATE TABLE enrollment (
c_name VARCHAR(255) NOT NULL,
gender VARCHAR(255) NOT NULL,
PRIMARY KEY (c_name)
);
You can change it by removing the primary id line as
CREATE TABLE enrollment (
c_name VARCHAR(255) NOT NULL,
gender VARCHAR(255) NOT NULL
);
Hope this solves your problem.
If your csv data contains any unique column as ID you could use it as primary key as,
CREATE TABLE enrollment (
e_id INT NOT NULL,
gender VARCHAR(255) NOT NULL,
PRIMARY KEY (e_id)
);
A table can have only one primary key. Remove c_name in the last line and only have e_id as shown below
PRIMARY KEY (e_id);
The combination of e_id and c_name must be unique. If not you can create an additional artifical primary key as work around.
CREATE TABLE `enrollment` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`e_id` int(11) NOT NULL,
`c_name` varchar(45) NOT NULL,
`gender` varchar(45) DEFAULT NULL,
`2001` int(11) DEFAULT NULL,
`2002` int(11) DEFAULT NULL,
`2003` int(11) DEFAULT NULL,
`2004` int(11) DEFAULT NULL,
`2005` int(11) DEFAULT NULL,
`2006` int(11) DEFAULT NULL,
`2007` int(11) DEFAULT NULL,
`2008` int(11) DEFAULT NULL,
`2009` int(11) DEFAULT NULL,
`2010` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

MySQL inserting same value

I stumbled upon weird behavior in MySQL.
Lets say that I have only one record in table
| id | oib |
|----|-----|
| 1 | 5 |
Field oib is unique.
INSERT INTO `test` (`id`, `oib`) VALUES (NULL, '6')
I get following exception
Duplicate entry '5' for key 'oib_UNIQUE'
And this keeps going on no matter what value I try to save.
Anyone have idea what could cause this. I've never seen it.
UPDATE:
Here is CREATE TABLE statement:
CREATE TABLE IF NOT EXISTS `user` (
`id` int(25) NOT NULL AUTO_INCREMENT,
`email` varchar(45) DEFAULT NULL,
`password` varchar(255) DEFAULT NULL,
`first_name` varchar(45) DEFAULT NULL,
`last_name` varchar(45) DEFAULT NULL,
`dob` int(25) DEFAULT NULL,
`address` varchar(255) DEFAULT NULL,
`zip` int(6) DEFAULT NULL,
`oib` int(11) DEFAULT NULL,
`position` tinyint(4) DEFAULT NULL,
`role` varchar(45) DEFAULT NULL,
`status` tinyint(4) DEFAULT NULL,
`note` mediumtext,
PRIMARY KEY (`id`),
UNIQUE KEY `email_UNIQUE` (`email`),
UNIQUE KEY `oib_UNIQUE` (`oib`)
)
You cannot insert null value in id primary key
Are you sure your table have one record for oib = '5' ?
You cannot insert null into primary key.

making queries run faster

ive run into a bit of a snag here, it seems that my pages are loading slower and slower the most content is being added to the database.
i have already added indexes on the ID and ID_* fields, that fixed the problem a while ago, but is not working anymore.
i've also run OPTIMIZE on all tables.
here is my query:
select
l.id, l.id_infusionsoft, l.name_first, l.name_last, l.postcode, a.website as a_website, l.website as l_website, l.date_added,
a.id_dealership, a.id_lead, a.id as id_assign, a.date_assigned, a.manual_or_auto, a.assigned_by,
d.name as dealership,
COALESCE(a.date_assigned, l.date_added) AS date_sort
from `leads` as l
left join `assignments` as a on (a.id_lead = l.id)
left join `dealerships` as d on (d.id = a.id_dealership)
order by date_sort desc
here are the table structures:
CREATE TABLE assignments (
id int(11) NOT NULL auto_increment,
id_dealership int(11) NOT NULL,
id_lead int(11) NOT NULL,
date_assigned int(11) NOT NULL,
website varchar(255) NOT NULL default '',
make varchar(255) NOT NULL default '',
model varchar(255) NOT NULL default '',
ip_address varchar(255) NOT NULL default '',
is_reassign varchar(255) NOT NULL default 'no',
manual_or_auto varchar(255) NOT NULL default 'N/A',
assigned_by varchar(255) NOT NULL default 'N/A',
PRIMARY KEY (id),
KEY id_dealership (id_dealership),
KEY id_lead (id_lead),
KEY date_assigned (date_assigned)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=24569 ;
CREATE TABLE dealerships (
id int(11) NOT NULL auto_increment,
province varchar(255) NOT NULL default '',
city varchar(255) NOT NULL default '',
`name` varchar(255) NOT NULL,
email text NOT NULL,
email_subject varchar(255) NOT NULL default 'Car Lead',
`type` varchar(255) NOT NULL,
make varchar(255) NOT NULL,
leads int(11) NOT NULL default '0',
`status` varchar(255) NOT NULL,
low_notif int(11) NOT NULL,
reassign_id int(11) NOT NULL,
reassign_days int(11) NOT NULL,
salesman varchar(255) NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=176 ;
CREATE TABLE leads (
id int(11) NOT NULL auto_increment,
id_infusionsoft int(11) NOT NULL,
name_first varchar(255) NOT NULL,
name_last varchar(255) NOT NULL,
email varchar(255) NOT NULL,
phone_home varchar(255) NOT NULL,
phone_cell varchar(255) NOT NULL,
phone_work varchar(255) NOT NULL,
postcode varchar(255) NOT NULL,
website varchar(255) NOT NULL,
address varchar(255) NOT NULL,
province varchar(255) NOT NULL,
employer varchar(255) NOT NULL,
city varchar(255) NOT NULL,
rentorown varchar(255) NOT NULL,
emp_months varchar(255) NOT NULL,
emp_years varchar(255) NOT NULL,
sin varchar(255) NOT NULL,
occupation varchar(255) NOT NULL,
monthly_income varchar(255) NOT NULL,
bankruptcy varchar(255) NOT NULL,
tradein varchar(255) NOT NULL,
cosign varchar(255) NOT NULL,
monthly_payment varchar(255) NOT NULL,
residence_years varchar(255) NOT NULL,
residence_months varchar(255) NOT NULL,
birthday varchar(255) NOT NULL,
make varchar(255) NOT NULL,
model varchar(255) NOT NULL,
date_added int(11) NOT NULL,
ip_address varchar(255) NOT NULL default '',
time_to_call varchar(255) NOT NULL,
referrer varchar(255) NOT NULL default '',
`source` varchar(255) NOT NULL,
PRIMARY KEY (id),
KEY id_infusionsoft (id_infusionsoft),
KEY date_added (date_added)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=20905 ;
what can i do to make my query run faster?
EDIT: here is the explain select output:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE l ALL NULL NULL NULL NULL 15381 Using temporary; Using filesort
1 SIMPLE a ref id_lead id_lead 4 infu.l.id 1
1 SIMPLE d eq_ref PRIMARY PRIMARY 4 infu.a.id_dealership 1
Use InnoDB instead of MyISAM. InnoDB caches both data and indexes, whereas MyISAM only caches indexes. InnoDB has other benefits too.
Do you really need all 15381 leads to be returned by this query? Or are you trying to query only a subset of leads, such as those that have assignments and dealerships?
Make sure you understand the type of join you're using. I suspect you're using left join when you need an inner join. It's probably causing the result set to be larger than it needs to be. Adding overhead for sorting, memory use, etc.
Choose data types more appropriately. Does every string have to be varchar(255)? Are you aware that varchars pad out to their maximum length in memory? You're even using varchar(255) to store an IP address. Also, you're using a signed integer to store dates?
Change the following index:
ALTER TABLE assignments ADD KEY assgn_lead_dealership (id_lead, id_dealership),
DROP KEY id_lead;

Odd MySQL Behavior - Query Optimization Help

We have a central login that we use to support multiple websites. To store our users' data we have an accounts table which stores each user account and then users tables for each site for site specific information.
We noticed that one query that is joining the tables on their primary key user_id is executing slowly. I'm hoping that some SQL expert out there can explain why it's using WHERE to search the users_site1 table and suggest how we can optimize it. Here is the slow query & the explain results:
mysql> explain select a.user_id as 'id',a.username,a.first_name as 'first',a.last_name as 'last',a.sex,u.user_id as 'profile',u.facebook_id as 'fb_id',u.facebook_publish as 'fb_publish',u.facebook_offline as 'fb_offline',u.twitter_id as 'tw_id',u.api_session as 'mobile',a.network from accounts a left join users_site1 u ON a.user_id=u.user_id AND u.status="R" where a.status="R" AND u.status="R" AND a.facebook_id='1234567890';
+----+-------------+-------+--------+----------------+---------+---------+-----------------------+-------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+----------------+---------+---------+-----------------------+-------+-------------+
| 1 | SIMPLE | u | ALL | PRIMARY | NULL | NULL | NULL | 79769 | Using where |
| 1 | SIMPLE | a | eq_ref | PRIMARY,status | PRIMARY | 4 | alltrailsdb.u.user_id | 1 | Using where |
+----+-------------+-------+--------+----------------+---------+---------+-----------------------+-------+-------------+
2 rows in set (0.00 sec)
Here are the definitions for each table:
CREATE TABLE `accounts` (
`user_id` int(9) unsigned NOT NULL AUTO_INCREMENT,
`username` varchar(40) DEFAULT NULL,
`facebook_id` bigint(15) unsigned DEFAULT NULL,
`facebook_username` varchar(30) DEFAULT NULL,
`password` varchar(20) DEFAULT NULL,
`profile_photo` varchar(100) DEFAULT NULL,
`first_name` varchar(40) DEFAULT NULL,
`middle_name` varchar(40) DEFAULT NULL,
`last_name` varchar(40) DEFAULT NULL,
`suffix_name` char(3) DEFAULT NULL,
`organization_name` varchar(100) DEFAULT NULL,
`organization` tinyint(1) unsigned DEFAULT NULL,
`address` varchar(200) DEFAULT NULL,
`city` varchar(40) DEFAULT NULL,
`state` varchar(20) DEFAULT NULL,
`zip` varchar(10) DEFAULT NULL,
`province` varchar(40) DEFAULT NULL,
`country` int(3) DEFAULT NULL,
`latitude` decimal(11,7) DEFAULT NULL,
`longitude` decimal(12,7) DEFAULT NULL,
`phone` varchar(20) DEFAULT NULL,
`sex` char(1) DEFAULT NULL,
`birthday` date DEFAULT NULL,
`about_me` varchar(2000) DEFAULT NULL,
`activities` varchar(300) DEFAULT NULL,
`website` varchar(100) DEFAULT NULL,
`email` varchar(150) DEFAULT NULL,
`referrer` int(4) unsigned DEFAULT NULL,
`referredid` int(9) unsigned DEFAULT NULL,
`verify` int(6) DEFAULT NULL,
`status` char(1) DEFAULT 'R',
`created` datetime DEFAULT NULL,
`verified` datetime DEFAULT NULL,
`activated` datetime DEFAULT NULL,
`network` datetime DEFAULT NULL,
`deleted` datetime DEFAULT NULL,
`logins` int(6) unsigned DEFAULT '0',
`api_logins` int(6) unsigned DEFAULT '0',
`last_login` datetime DEFAULT NULL,
`last_update` datetime DEFAULT NULL,
`private` tinyint(1) unsigned DEFAULT NULL,
`ip` varchar(20) DEFAULT NULL,
PRIMARY KEY (`user_id`),
UNIQUE KEY `username` (`username`),
KEY `status` (`status`),
KEY `state` (`state`)
);
CREATE TABLE `users_site1` (
`user_id` int(9) unsigned NOT NULL,
`facebook_id` bigint(15) unsigned DEFAULT NULL,
`facebook_username` varchar(30) DEFAULT NULL,
`facebook_publish` tinyint(1) unsigned DEFAULT NULL,
`facebook_checkin` tinyint(1) unsigned DEFAULT NULL,
`facebook_offline` varchar(300) DEFAULT NULL,
`twitter_id` varchar(60) DEFAULT NULL,
`twitter_secret` varchar(50) DEFAULT NULL,
`twitter_username` varchar(20) DEFAULT NULL,
`type` char(1) DEFAULT 'M',
`referrer` int(4) unsigned DEFAULT NULL,
`referredid` int(9) unsigned DEFAULT NULL,
`session` varchar(60) DEFAULT NULL,
`api_session` varchar(60) DEFAULT NULL,
`status` char(1) DEFAULT 'R',
`created` datetime DEFAULT NULL,
`verified` datetime DEFAULT NULL,
`activated` datetime DEFAULT NULL,
`deleted` datetime DEFAULT NULL,
`logins` int(6) unsigned DEFAULT '0',
`api_logins` int(6) unsigned DEFAULT '0',
`last_login` datetime DEFAULT NULL,
`last_update` datetime DEFAULT NULL,
`ip` varchar(20) DEFAULT NULL,
PRIMARY KEY (`user_id`)
);
Add a index on the column facebook_id in the accounts table.
Current, MySql is scanning the entire users table, since it cannot find the record directly in the account table.
The least create 3 indexes on accounts.user_id, user_site1.user_id and accounts.facebook_id. It's likely that user_id indexes already exist as they are defined as PKs though.
Your query is looking for rows in table accounts based on the Facebook ID and on the account "status". You don't have any indexes that help with this, so MySQL is doing a table scan. I suggest the following index:
ALTER TABLE accounts ADD INDEX (facebook_id, user_id)
If you wanted, you could even include the status column in the index. Whether this is a good idea or not would really depend on whether or not it would help to make the index an attractive choice for the optimiser for any other queries you plan to run.
PS. The comment "using where" is normal and is to be expected in most queries. The thing to be concerned about here is the fact that MySQL is not using an index, and that it thinks it has to examine a large number of rows (surely this should not be the case when you are passing in a specific ID number).
Maybe because you haven't created indexes on the columns you're searching on??? Try indexing the columns used on the join statements. Without indexing, you're scanning through all the dataset.
CREATE INDEX accounts_user_id_index ON accounts (user_id);
CREATE INDEX accounts.facebook_id_index ON accounts (status);
CREATE INDEX user_site1.user_id_index ON user_site1 (user_id);