I have created one table and inserting values in the table from a csv file using a python code .The createdatetime and updatedatetime columns are set to default CURRENT_TIMESTAMP.But when I am populating the data updatedatetime is populating the correct value but createdatetime is populating as 0000-00-00 00:00:00.
Here is my table definition:
CREATE TABLE `fico_details` (
`adf_contact_id` varchar(100) NOT NULL,
`sf_contact_id` varchar(100) DEFAULT NULL,
`Name` varchar(100) NOT NULL,
`BirthDate` date DEFAULT NULL,
`Address1` varchar(100) DEFAULT NULL,
`City` varchar(100) DEFAULT NULL,
`State` varchar(2) DEFAULT NULL,
`Zipcode` varchar(10) DEFAULT NULL,
`SSN` varchar(10) DEFAULT NULL,
`Address2` varchar(100) DEFAULT NULL,
`City2` varchar(100) DEFAULT NULL,
`State2` varchar(2) DEFAULT NULL,
`Zipcode2` varchar(10) DEFAULT NULL,
`Customerinput` varchar(10) DEFAULT NULL,
`AddrDiscrepancyFlg` varchar(10) DEFAULT NULL,
`Permid` varchar(10) DEFAULT NULL,
`score_date` varchar(10) DEFAULT NULL,
`reason_code_1` varchar(10) DEFAULT NULL,
`reason_code_2` varchar(10) DEFAULT NULL,
`reason_code_3` varchar(10) DEFAULT NULL,
`reason_code_4` varchar(10) DEFAULT NULL,
`CreateDateTime` TIMESTAMP not null DEFAULT CURRENT_TIMESTAMP,
`UpdateDateTime` TIMESTAMP not null DEFAULT CURRENT_TIMESTAMP,
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `SSN` (`SSN`)
) ENGINE=InnoDB AUTO_INCREMENT=876800 DEFAULT CHARSET=utf8;
please help me to solve it.
I am working on same thing these days.
First of all I use timestamp as the datatype as apart from having datetime capabilities, it has MANY usable functions as a string as well as timestamp datatype.
I got the SAME issue as yours, what I did was alter my table was created, I would click on ALTER table using any Db tool (SQLYog in my case) and then Delete the 0000-00-00 00:00:00 value and uncheck the Not NULL check box.
This issue gets resolved EVERYTIME (whenever I create same type of table/columns)after this simple one step.
Hope this helps you too, let me know if anything still bothers you....!!
I have the following simple join query
SELECT
count(*)
FROM
DBx.caseview p2015
INNER JOIN DBy.caseview p2014 ON p2015.casenumber=p2014.casenumber;
For some reason it just leaves MySQL hanging there for a lot of time until I get tired and cancel it. On the contrary, if run exactly the same code on MSSQL with the same data set the query takes a few seconds at most.
Is there a parameter that needs to be changed on MySQL to speed up this type of queries?
Here's my table in MySQL
CREATE TABLE `caseview` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`CASEID` varchar(18) DEFAULT NULL,
`CASENUMBER` int(10) DEFAULT NULL,
`ACCOUNTID` varchar(18) DEFAULT NULL,
`ACCOUNT` varchar(256) DEFAULT NULL,
`ASSETID` varchar(18) DEFAULT NULL,
`SAPPRODUCTGROUP` varchar(10) DEFAULT NULL,
`PRODUCT` varchar(128) DEFAULT NULL,
`FAMILY` varchar(128) DEFAULT NULL,
`CONTACTID` varchar(18) DEFAULT NULL,
`OWNERID` varchar(18) DEFAULT NULL,
`TYPE` varchar(128) DEFAULT NULL,
`PRIORITY` varchar(24) DEFAULT NULL,
`ORIGIN` varchar(24) DEFAULT NULL,
`SUBJECT` varchar(256) DEFAULT NULL,
`STATUS` varchar(24) DEFAULT NULL,
`LASTACTIVITY` varchar(1024) DEFAULT NULL,
`INITALDESCRIPTION` varchar(1024) DEFAULT NULL,
`CLOSEDDATE` datetime DEFAULT NULL,
`CREATEDDATE` datetime DEFAULT NULL,
`LASTMODIFIEDDATE` datetime DEFAULT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `ID_UNIQUE` (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=122393 DEFAULT CHARSET=utf8;
There's two tables with exactly the same configuration as above, just different data
DBx.caseview p2015 has 197647 rows
DBy.caseview p2014 has 122392 rows
Create an Index for CASENUMBER field.
ALTER TABLE `caseview` ADD INDEX ( `CASENUMBER` ) ;
Give it a few seconds to create the index and run the query again.
This surely seems like poor design of the college_major table.
CREATE TABLE `college_majors` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date_time` datetime DEFAULT NULL,
`UNITID` varchar(255) DEFAULT NULL,
`CIPCODE` varchar(255) DEFAULT NULL,
`AWLEVEL` varchar(255) DEFAULT NULL,
`CTOTALT` varchar(255) DEFAULT NULL,
`CTOTALM` varchar(255) DEFAULT NULL,
`CTOTALW` varchar(255) DEFAULT NULL,
`CAIANT` varchar(255) DEFAULT NULL,
`CASIAT` varchar(255) DEFAULT NULL,
`CBKAAT` varchar(255) DEFAULT NULL,
`CHISPT` varchar(255) DEFAULT NULL,
`CNHPIT` varchar(255) DEFAULT NULL,
`CWHITT` varchar(255) DEFAULT NULL,
`C2MORT` varchar(255) DEFAULT NULL,
`CUNKNT` varchar(255) DEFAULT NULL,
`CNRALT` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=270167 DEFAULT CHARSET=utf8;
I can reduce this table to three columns - id, CIPCODE and UNITID. But the problem is even simple queries like *select * FROM college_majors* is taking too long to execute and sometime not even executing.
I increased the query execution to 6000.00 sec, but still the query won't run.
Any suggestion on how to improve the design, create a new table and insert the data from this table (college_majors).
Thanks,
A
If field 'codevalue' in college_majors_mapping is unique you can indexed it and increase join performance.
The following query is timing out after 600 seconds.
update placed p
,Results r
set p.position = r.position
where p.competitor = r.competitor
AND p.date = r.date
AND REPLACE(p.time,":","") = r.time;
The structure is as follows:
'CREATE TABLE `placed` (
`idplaced` varchar(50) DEFAULT NULL,
`date` decimal(8,0) DEFAULT NULL,
`time` varchar(45) DEFAULT NULL,
`field1` varchar(45) DEFAULT NULL,
`competitor` varchar(45) DEFAULT NULL,
`field2` int(2) DEFAULT NULL,
`field3` varchar(45) DEFAULT NULL,
`field4` varchar(45) DEFAULT NULL,
`field5` decimal(6,2) DEFAULT NULL,
`field6` decimal(10,2) DEFAULT NULL,
`field7` decimal(6,2) DEFAULT NULL,
`field8` char(1) DEFAULT NULL,
`field9` varchar(45) DEFAULT NULL,
`position` char(4) DEFAULT NULL,
`field10` decimal(6,2) DEFAULT NULL,
`field11` char(1) DEFAULT NULL,
`field12` char(1) DEFAULT NULL,
`field13` decimal(6,2) DEFAULT NULL,
`field14` decimal(6,2) DEFAULT NULL,
`field15` decimal(6,2) DEFAULT NULL,
`field16` decimal(6,2) DEFAULT NULL,
`field17` decimal(6,2) DEFAULT NULL,
`field18` char(1) DEFAULT NULL,
`field19` char(20) DEFAULT NULL,
`field20` char(1) DEFAULT NULL,
`field21` char(5) DEFAULT NULL,
`field22` char(5) DEFAULT NULL,
`field23` int(11) DEFAULT NULL
PRIMARY KEY (`idplaced`),
UNIQUE KEY `date_time_competitor_field18_combo` (`date`,`time`,`competitor`,`field18`)
) ENGINE=InnoDB AUTO_INCREMENT=100688607 DEFAULT CHARSET=latin1;
CREATE TABLE `results` (
`idresults` int(11) NOT NULL AUTO_INCREMENT,
`date` char(8) DEFAULT NULL,
`time` char(4) DEFAULT NULL,
`field1` varchar(45) DEFAULT NULL,
`competitor` varchar(45) DEFAULT NULL,
`position` char(4) DEFAULT NULL,
`field2` varchar(45) DEFAULT NULL,
`field3` decimal(2,0) DEFAULT NULL,
PRIMARY KEY (`idresults`)
) ENGINE=InnoDB AUTO_INCREMENT=6644 DEFAULT CHARSET=latin1;
The PLACED table has 65,000 records, the RESULTS table has 9,000 records.
I am assuming the solution involves a JOIN statement of some descript, and I have tried taking several suggestions from this site, but am simply not finding the answer I am looking for. Simply put, I would be grateful for suggestions on this. I can put up example tables / create table code if requried.
The index cannot be used efficiently to perform the join because of your REPLACE operation.
I'd suggest creating an index with the columns in the following slightly different order:
(date, competitor, time, position)
It may also help to add this index on both tables.
It would be even better if you could modify the data in the database so that the data in the time column was stored in the same format in both tables.
First of all, you'd better send us your full tables description, using
show create table
Second, you'd better use join syntax :
update placed p
join Results r on r.competitor = p.competitor
set p.position = r.position
where p.date = r.date
AND REPLACE(p.time,":","") = r.time;
Hope this will help.
I have table named as contacts which has nearly 1.2 million records we use
MyIsam engine whenever we query this table mysql hangs down so now we are trying our hands with Innodb engine so that if it slows down, but it will not hang up for others
So we want make fast with Myisam we tried many indexes on this table but it goes down and hangs the system
What should be done to make it more faster and it should not hang up the system
This is the table:
CREATE TABLE `contacts` (
`id` varchar(36) NOT NULL,
`deleted` tinyint(1) NOT NULL default '0',
`date_entered` datetime NOT NULL default '0000-00-00 00:00:00',
`date_modified` datetime NOT NULL default '0000-00-00 00:00:00',
`modified_user_id` varchar(36) default NULL,
`assigned_user_id` varchar(36) default NULL,
`created_by` varchar(36) default NULL,
`team_id` varchar(36) default NULL,
`salutation` varchar(5) default NULL,
`first_name` varchar(100) default '',
`last_name` varchar(100) default '',
`username` varchar(25) default '',
`lead_source` varchar(100) default NULL,
`title` varchar(50) default NULL,
`department` varchar(100) default NULL,
`reports_to_id` varchar(36) default NULL,
`birthdate` date default NULL,
`do_not_call` char(3) default '0',
`phone_home` varchar(25) default NULL,
`phone_mobile` varchar(25) default NULL,
`phone_work` varchar(25) default '',
`phone_other` varchar(25) default NULL,
`phone_fax` varchar(25) default '',
`email1` varchar(100) default '',
`email2` varchar(100) default NULL,
`assistant` varchar(75) default NULL,
`assistant_phone` varchar(25) default NULL,
`email_opt_out` char(3) default 'off',
`primary_address_street` varchar(150) default NULL,
`primary_address_city` varchar(100) default NULL,
`primary_address_state` varchar(100) default NULL,
`primary_address_postalcode` varchar(20) default NULL,
`primary_address_country` varchar(100) default NULL,
`alt_address_street` varchar(150) default NULL,
`alt_address_city` varchar(100) default NULL,
`alt_address_state` varchar(100) default NULL,
`alt_address_postalcode` varchar(20) default NULL,
`alt_address_country` varchar(100) default NULL,
`description` text,
`portal_name` varchar(255) default NULL,
`portal_active` tinyint(1) NOT NULL default '0',
`portal_app` varchar(255) default NULL,
`salesforceid` varchar(36) default NULL,
`phone_direct` varchar(25) default NULL,
`invalid_email` tinyint(1) default '0',
`parent_is_lead` char(3) default 'no',
`advisory_board_member` varchar(25) default NULL,
`direct_marketing` varchar(25) default NULL,
`efx_id` varchar(36) default NULL,
`fax_opt_out` char(3) default 'off',
`ppc_keyword` varchar(50) default NULL,
`status` varchar(25) default NULL,
`web_form` varchar(50) default NULL,
`efx_export_date` datetime default NULL,
`bmtn` varchar(225) default '',
`employee_location` varchar(50) default NULL,
`pronunciation` varchar(250) default NULL,
`duplicate_of` varchar(36) default NULL,
`job_category` varchar(50) default NULL,
`last_ska_upload_key` varchar(50) default NULL,
`persid` varchar(36) default NULL,
`last_web_upload_key` varchar(50) default NULL,
`last_webinar_upload_key` varchar(50) default NULL,
`primary_address_latitude` float default NULL,
`primary_address_longitude` float default NULL,
`first_name_soundex` varchar(30) default NULL,
`last_name_soundex` varchar(30) default NULL,
`primary_address_street_soundex` varchar(30) default NULL,
`campaign_id` varchar(36) default NULL,
`portal_password` varchar(32) default NULL,
`pss_branch` varchar(40) default NULL,
`pss_id` int(12) default NULL,
`source_detail` varchar(100) default NULL,
`source` varchar(100) default NULL,
`pss_region` varchar(30) default NULL,
`source_added` datetime default NULL,
`terminated_user` char(3) default 'off',
`invite_opt_out` char(3) default 'off',
`newsletter_opt_out` char(3) default 'off',
`stream_opt_out` char(3) default 'off',
PRIMARY KEY (`id`),
KEY `idx_contacts_del_last` (`deleted`,`last_name`),
KEY `idx_cont_del_reports` (`deleted`,`reports_to_id`,`last_name`),
KEY `idx_contact_del_team` (`deleted`,`team_id`),
KEY `idx_contact_salesforceid` (`salesforceid`),
KEY `idx_contacts_username` (`username`),
KEY `idx_email_opt_out` (`email_opt_out`),
KEY `idx_primary_address_street` (`primary_address_street`),
KEY `idx_primary_address_city` (`primary_address_city`),
KEY `idx_primary_address_state` (`primary_address_state`),
KEY `idx_primary_address_postalcode` (`primary_address_postalcode`),
KEY `idx_primary_address_country` (`primary_address_country`),
KEY `idx_modified_user_id` (`modified_user_id`),
KEY `idx_assigned_user_id` (`assigned_user_id`),
KEY `idx_created_by` (`created_by`),
KEY `idx_team_id` (`team_id`),
KEY `idx_reports_to_id` (`reports_to_id`),
KEY `idx_contacts_efx_id` (`efx_id`),
KEY `idx_contacts_title1` (`title`,`deleted`),
KEY `idx_contacts_email1` (`email1`),
KEY `idx_contacts_email2` (`email2`),
KEY `idx_contacts_job_category` (`job_category`),
KEY `idx_contacts_first_name_sdx` (`first_name_soundex`),
KEY `idx_contacts_primary_street_sdx` (`primary_address_street_soundex`),
KEY `idx_contacts_last_name_sdx` (`last_name_soundex`),
KEY `idx_contacts_portal_name` (`portal_name`),
KEY `idx_contacts_portal_active` (`portal_active`),
KEY `idx_contacts_del_last_first` (`deleted`,`last_name`,`first_name`),
KEY `idx_contacts_del_first` (`deleted`,`first_name`),
KEY `idx_pss_id` (`pss_id`),
KEY `idx_phone_work_last_name_first_name_deleted` (`phone_work`,`last_name`,`first_name`,`deleted`),
KEY `idx_phone_work_last_name_first_name_deleted_sdx` (`phone_work`,`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_email1_last_name_first_name_deleted` (`email1`,`last_name`,`first_name`,`deleted`),
KEY `idx_email1_last_name_first_name_deleted_sdx` (`email1`,`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_phone_fax_last_name_first_name_deleted` (`phone_fax`,`last_name`,`first_name`,`deleted`),
KEY `idx_phone_fax_last_name_first_name_deleted_sdx` (`phone_fax`,`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_phone_work_last_name_deleted` (`phone_work`,`last_name`,`deleted`),
KEY `idx_phone_work_last_name_deleted_sdx` (`phone_work`,`last_name_soundex`,`deleted`),
KEY `idx_email1_last_name_deleted` (`email1`,`last_name`,`deleted`),
KEY `idx_email1_last_name_deleted_sdx` (`email1`,`last_name_soundex`,`deleted`),
KEY `idx_phone_fax_last_name_deleted` (`phone_fax`,`last_name`,`deleted`),
KEY `idx_phone_fax_last_name_deleted_sdx` (`phone_fax`,`last_name_soundex`,`deleted`),
KEY `idx_email1_first_name_deleted` (`email1`,`first_name`,`deleted`),
KEY `idx_email1_first_name_deleted_sdx` (`email1`,`first_name_soundex`,`deleted`),
KEY `idx_phone_fax_first_name_deleted` (`phone_fax`,`first_name`,`deleted`),
KEY `idx_phone_fax_first_name_deleted_sdx` (`phone_fax`,`first_name_soundex`,`deleted`),
KEY `idx_email1_deleted` (`email1`,`deleted`),
KEY `idx_last_name_first_name_deleted_sdx` (`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_phone_mobile_deleted` (`phone_mobile`,`deleted`,`id`),
KEY `idx_first_name_bmtn` (`first_name`,`bmtn`),
KEY `idx_first_name_bmtn_email1` (`first_name`,`bmtn`,`email1`),
KEY `idx_bmtn_email1` (`bmtn`,`email1`),
KEY `idx_deleted` (`deleted`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
-
SELECT acc.id, acc.name, con_reports_to.first_name, con_reports_to.last_name
from contacts
left join accounts_contacts a_c on a_c.contact_id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e' and a_c.deleted=0
left join accounts acc on a_c.account_id = acc.id and acc.deleted=0
left join contacts con_reports_to on con_reports_to.id = contacts.reports_to_id
where contacts.id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e'
I suspect the assertion "whenever we query this table mysql hangs down" is an overbid -- for example, with MyISAM, SELECT COUNT(*) FROM TheTable should be very fast, essentially "no matter what". Sure, some queries will be slow -- especially if the table is not indexed properly for the queries, or if MySQL's alleged optimizer is picking the wrong strategy (but you could give it hints).
Why don't you show us the CREATE TABLE (including indices), a couple of the queries that take too long, ideally a precise measure of how long they take, and the output of EXPLAIN SELECT (&c) for those couple queries -- I bet we could really be of some help then!
Edit: the CREATE TABLE essentially shows that the table is just too "broad" -- far too many columns -- to expect decent performance (even though no queries were shown). The schema needs a redesign, breaking up chunks of this huge monolithic table (e.g., the address-related information) into other auxiliary tables. Exactly how to best do it depends entirely on the queries that are most important to optimize, so, not knowing the queries in question, I'm not even going to attempt the task.
Edit again: so the query has been posted and uses other tables, accounts and account_contacts, as well as the hugely broad contacts one described; the query as posted (trying to make sense of it by formatting &c) is:
SELECT acc.id, acc.name, con_reports_to.first_name, con_reports_to.last_name
FROM contacts
LEFT JOIN accounts_contacts a_c
ON a_c.contact_id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e' AND
a_c.deleted=0
LEFT JOIN accounts acc
ON a_c.account_id = acc.id AND
acc.deleted=0
LEFT JOIN contacts con_reports_to
ON con_reports_to.id = contacts.reports_to_id
WHERE contacts.id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e'
Why the LEFT JOINs here instead of normal INNER joins? Is it possible in each case that there's no corresponding row on the right-hand-side table? For example, if there's no line in a_c with the given values for contact_id and deleted, then all the fields of a_c in the first LEFT JOIN will be NULL, so there can be no correspondence for acc either: is it important to emit NULL, NULL as the first two columns in this case? Moreover the JOIN conditions for a_c an acc make no reference at all to contacts, so this will be a cartesian product: every line selected from acc, if any, will pair up with every line selected from con_reports_to. So the a_c/acc query could be entirely separated from the one on contacts and con_reports, presumably ligthtening the query considerably (the two logically separate results could of course easily be put together again in the client).
What does EXPLAIN SELECT say for this complex query and what does it say for the two lighter-weight separate ones I'm suggesting? What indices are on the accounts and account_contact tables?
horizontal splitting? though i guess 1.2 million records are not that much to introduce horizontal splitting.. try to locate the bottom neck... also the problem may lie with your hardware as well for example harddisk almost full etc.