bugs in mysql when connecting 2 table - mysql

When I am trying to rum query I find error like
#1267 - Illegal mix of collations (latin1_general_ci,IMPLICIT) and (latin1_swedish_ci,IMPLICIT) for operation '='
My table structure are
CREATE TABLE `empregistration` (
`id` int(100) NOT NULL auto_increment,
`cname` text NOT NULL,
`cpersonname` text NOT NULL,
`roaddress` text NOT NULL,
`txtcity` text NOT NULL,
`txtstate` text NOT NULL,
`txtcountry` text NOT NULL,
`faxno` varchar(255) NOT NULL,
`contactno` varchar(255) NOT NULL,
`url` varchar(255) NOT NULL,
`empCat` varchar(255) NOT NULL,
`empno` varchar(255) NOT NULL,
`ctype` varchar(255) NOT NULL,
`establishyear` varchar(255) NOT NULL,
`txtjobcategory` text NOT NULL,
`cemailid` varchar(255) NOT NULL,
`calteremailid` varchar(255) NOT NULL,
`aboutcompany` text NOT NULL,
`password` varchar(255) NOT NULL,
`conpassword` varchar(255) NOT NULL,
`empLogo` varchar(255) NOT NULL,
`paymenttype` varchar(255) NOT NULL,
`regDate` int(11) default NULL,
`countViewProf` varchar(200) default '0',
`cntDownProf` varchar(200) default '0',
`cntJobPost` varchar(200) default '0',
`cntupdatejob` varchar(200) NOT NULL default '0',
`limitjobpost` varchar(200) NOT NULL default '0',
`limitofupdatejob` varchar(200) NOT NULL default '0',
`limitofviewcv` varchar(200) NOT NULL default '0',
`limitofdowncv` varchar(200) NOT NULL default '0',
`paymentforpostjob` varchar(200) NOT NULL default '0',
`paymentforsearchcv` varchar(200) NOT NULL default '0',
`takenPlan` varchar(255) default '0',
`planDate` date default NULL,
`takenplanforpostjob` varchar(200) NOT NULL default '0',
`postjobplandate` varchar(200) NOT NULL default '0',
`cntJobPost1` varchar(200) default '0',
`cntupdatejob1` varchar(200) default '0',
`status` varchar(255) NOT NULL default 'Active',
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=214 ;
and other table is
CREATE TABLE `blockedcompanies` (
`blockedId` int(11) NOT NULL auto_increment,
`empId` varchar(50) collate latin1_general_ci NOT NULL,
`regId` int(11) NOT NULL,
`jobSeekId` varchar(50) collate latin1_general_ci NOT NULL,
`blockDate` date NOT NULL,
PRIMARY KEY (`blockedId`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=5 ;

You don't show your query, but i expect you're doing a where somehow with an '=' in there. Now your default collation seems to be swedish, and your 'blockedcompanies' table has a collation to latin1_general.
The error says "implicit", so you're not doing something with collation in your query. Now you have to different collation being compared. You've got to change that using COLLATE
check this: http://dev.mysql.com/doc/refman/5.0/en/charset-collate.html

when you create the table empregistration, change the final line from
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=214 ;
to
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=214 ;
and your query should start working.

Related

Migrating an .sql database to WordPress

I have an SQL file for a database from an old custom blog CMS. I am trying to import the articles into WordPress but I am now stuck. So far, I have managed to isolate the table structures for the CMS (see below). I have also identified the columns that i need. These are (Post, Title, Date, Image, Category). Now I have to map these columns to some columns in the Wordpress schema somehow and that is where I am getting stuck.
Here is the SQL structure of the database:
CREATE TABLE `ads_ng_ads` (
`id` int(10) unsigned NOT NULL auto_increment,
`ad_type` enum('media','text','html') character set latin1 collate latin1_bin NOT NULL,
`media_type` enum('image','flash','other') character set latin1 collate latin1_bin default NULL,
`media_width` smallint(6) default NULL,
`media_height` smallint(6) default NULL,
`media_size` int(6) default NULL,
`media_path` varchar(128) character set latin1 collate latin1_bin default NULL,
`uri` varchar(128) character set utf8 collate utf8_bin NOT NULL,
`is_hidden` tinyint(1) NOT NULL,
`created_on` int(11) NOT NULL,
`active_from` int(11) NOT NULL,
`active_to` int(11) NOT NULL,
`room_id` int(11) default NULL,
`geo_target` int(11) default NULL,
`alternative` text character set utf8 NOT NULL,
`zone_target` text character set ascii collate ascii_bin NOT NULL,
`uri_target` text character set utf8 collate utf8_bin NOT NULL,
PRIMARY KEY (`id`),
KEY `zone_target` (`is_hidden`,`active_from`,`active_to`,`geo_target`)
) ENGINE=MyISAM AUTO_INCREMENT=757 DEFAULT CHARSET=latin1;
CREATE TABLE `ads_ng_statistics` (
`ad_id` int(10) unsigned NOT NULL,
`date` int(11) NOT NULL,
`views` mediumint(8) unsigned NOT NULL default '0',
`clicks` mediumint(8) unsigned NOT NULL default '0',
PRIMARY KEY (`ad_id`,`date`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE `countries` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(100) NOT NULL,
`alias` varchar(3) NOT NULL,
`restricted` tinyint(3) NOT NULL default '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=253 DEFAULT CHARSET=utf8;
CREATE TABLE `featured_column` (
`id` int(11) unsigned NOT NULL auto_increment,
`date_modified` timestamp NOT NULL default CURRENT_TIMESTAMP,
`date_created` timestamp NOT NULL default '1000-01-01 00:00:00',
`title` varchar(255) NOT NULL,
`alias` varchar(255) NOT NULL,
`body` text NOT NULL,
`body_html` text NOT NULL,
`user_id` int(11) NOT NULL,
`tags` varchar(255) NOT NULL,
`summary` varchar(255) NOT NULL,
`in_homepage` int(1) NOT NULL default '0',
`img_title` int(1) NOT NULL default '0',
`meta_t` varchar(255) NOT NULL,
`meta_k` varchar(255) NOT NULL,
`meta_d` varchar(255) NOT NULL,
`public` int(1) NOT NULL default '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1381 DEFAULT CHARSET=utf8;
Assuming that you're using the latest WordPress version(4.9.1), you just need to create a script that will import all data to table wp_posts for example if you want to import as WordPress posts. Example: "Title" to "post_title", "post" to "post_content", etc.

multiple joins SQL query optimization

I have the following query :
SELECT
COUNT(DISTINCT a0_.id) AS sclr0
FROM
account a0_
INNER JOIN customer c1_ ON (c1_.account = a0_.id)
LEFT JOIN sf_user_data s2_ ON (s2_.user_id = a0_.id)
LEFT JOIN address a3_ ON (c1_.customer_address = a3_.id)
WHERE
a3_.city IS NOT NULL
resulting in the following output :
sclr0
+--------+
298279
with the following EXPLAIN :
id select_type table partitions type possible_keys key key_len ref rows filtered Extra
+--+---------------+-------+-----------+-------+--------------------------------------------+-----------------------+-----------+--------------------------------+-------+-----------+-------+
1 SIMPLE c1_ NULL ALL UNIQ_81398E097D3656A4,UNIQ_81398E091193CB3F NULL NULL NULL 405508 100.00 NULL
1 SIMPLE a0_ NULL eq_ref PRIMARY PRIMARY 8 evoportail.c1_.account 1 100.00 Using index
1 SIMPLE s2_ NULL eq_ref UNIQ_E904BFD1A76ED395 UNIQ_E904BFD1A76ED395 8 evoportail.c1_.account 1 100.00 Using index
1 SIMPLE a3_ NULL eq_ref PRIMARY PRIMARY 8 evoportail.c1_.customer_address 1 90.00 Using where
approximative number of rows in the tables :
account : 430000
customer: 430000
sf_user_data : 115000
address : 550000
Right now, this query is running in 3 seconds. Is there any way to speed it up ?
the CREATE statements :
CREATE TABLE `account` (
`id` bigint(20) unsigned NOT NULL auto_increment,
`identifier` varchar(255) collate utf8_bin NOT NULL,
`hash` varchar(255) collate utf8_bin default NULL,
`date_create` datetime default NULL,
`group` varchar(50) collate utf8_bin default NULL,
`sub_group` varchar(50) collate utf8_bin NOT NULL default 'NULL',
`date_last_action` datetime default NULL,
`date_last_connection` datetime default NULL,
`connection_counter` int(10) unsigned default NULL,
`connection_since_customer` int(10) unsigned NOT NULL default '0',
`salt` varchar(255) collate utf8_bin default NULL,
`roles` longtext collate utf8_bin COMMENT '(DC2Type:array)',
`is_v3` tinyint(1) NOT NULL default '1',
`password_token` varchar(255) collate utf8_bin default NULL,
`password_token_expired_at` datetime default NULL,
`is_included_in_newsletters` tinyint(1) NOT NULL default '0',
PRIMARY KEY (`id`),
UNIQUE KEY `identifier_UNIQUE` (`identifier`)
) ENGINE=MyISAM AUTO_INCREMENT=434243 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `address` (
`id` bigint(20) unsigned NOT NULL auto_increment,
`city` varchar(64) collate utf8_bin default NULL,
`street` varchar(255) collate utf8_bin default NULL,
`complement` varchar(128) collate utf8_bin default NULL,
`zipcode` varchar(16) collate utf8_bin default NULL,
`country_id` int(11) default NULL,
`cedex` tinyint(1) NOT NULL default '0',
`abroad` tinyint(1) NOT NULL default '0',
PRIMARY KEY (`id`),
KEY `IDX_D4E6F81F92F3E70` (`country_id`)
) ENGINE=MyISAM AUTO_INCREMENT=541873 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `customer` (
`id` bigint(20) unsigned NOT NULL auto_increment,
`account` bigint(20) unsigned NOT NULL,
`source` varchar(250) collate utf8_bin NOT NULL default 'DECLARATION',
`last_source` varchar(250) collate utf8_bin NOT NULL,
`source_domain_name` varchar(255) collate utf8_bin default NULL,
`subscription_offer` varchar(255) collate utf8_bin default NULL,
`formalities_center_address` bigint(20) unsigned default NULL,
`customer_address` bigint(20) unsigned default NULL,
`business_address` bigint(20) unsigned default NULL,
`shipping_address` bigint(20) default NULL,
`activity` text collate utf8_bin,
`state` enum('NONE','CREATE','UPDATE','COMPLETE') collate utf8_bin NOT NULL default 'NONE',
`num_dossier` varchar(255) collate utf8_bin default NULL,
`email` varchar(255) collate utf8_bin NOT NULL,
`lastname` varchar(255) collate utf8_bin NOT NULL,
`firstname` varchar(255) collate utf8_bin NOT NULL,
`sexe` int(1) NOT NULL default '0',
`activityset` int(1) NOT NULL default '0',
`phone` varchar(16) collate utf8_bin NOT NULL,
`business_name` varchar(255) collate utf8_bin default NULL,
`payment_method` enum('NONE','CB_OK','CB_KO','CHEQUE_OK','CHEQUE_KO','WAITING','IMPACTPLUS_OK','PRELEV') collate utf8_bin default 'NONE',
`payment_waiting_comment` text collate utf8_bin,
`sub_sent_recovery` smallint(6) default '0',
`transaction_number` varchar(30) collate utf8_bin default NULL,
`transaction_date` datetime default NULL,
`properties` set('ACCRE','CFE','WANT_WEBSITE','HAVE_WEBSITE','NEWSLETTER','SUBSCRIBE','OLD_CUSTOMER') collate utf8_bin default NULL,
`comments` text collate utf8_bin,
`activity_declaration` varchar(512) collate utf8_bin default NULL COMMENT 'file:///',
`cfe_center` varchar(255) collate utf8_bin default NULL,
`date_create` datetime default NULL,
`date_complete` datetime default NULL,
`date_subscribe` datetime default NULL,
`date_next_payement` datetime default NULL,
`date_ae_subscribe` datetime default NULL,
`siret` varchar(128) collate utf8_bin default NULL,
`current_quotation` bigint(20) unsigned default NULL,
`current_invoice` bigint(20) unsigned default NULL,
`has_create_quotation` tinyint(1) NOT NULL default '0',
`has_create_invoice` tinyint(1) NOT NULL default '0',
`created_by` int(20) NOT NULL default '0',
`updated_by` int(11) NOT NULL default '0',
`updated_date` datetime default NULL,
`abo_running` tinyint(1) NOT NULL default '1',
`show_bn` tinyint(1) NOT NULL default '1',
`taxe_type_activite` enum('NULL','ACHAT','SERVICE','BOTH') collate utf8_bin default NULL,
`taxe_categorie_activite` enum('NULL','COMMERCIALE','ARTISANALE','CIPAV','RSI') collate utf8_bin default NULL,
`taxe_liberatoire` enum('NULL','OUI','NON') collate utf8_bin default NULL,
`taxe_statut_accre` enum('NULL','OUI','NON','DK') collate utf8_bin default NULL,
`know` varchar(50) collate utf8_bin default NULL,
`sms_relance` int(11) default NULL,
`fdae` int(1) NOT NULL default '0',
`display_fdae` tinyint(1) NOT NULL default '0',
`show_dispense_immat` enum('RCS','RM','RSAC') collate utf8_bin default NULL,
`show_dispense_immat_city` varchar(255) collate utf8_bin default NULL,
`subscription_fdae` date default NULL,
`nbsocial` varchar(30) collate utf8_bin default NULL,
`atclic` int(1) NOT NULL default '0',
`merassurance` int(1) NOT NULL default '0',
`dossier_canceled` tinyint(1) NOT NULL default '0',
`dossier_canceled_date` datetime default NULL,
`tva_intra` varchar(20) collate utf8_bin default NULL,
`site_url` varchar(100) collate utf8_bin default NULL,
`freeguide` tinyint(1) NOT NULL default '0',
`hiscox` int(1) NOT NULL default '0',
`assurland` int(1) NOT NULL default '0',
`unpaid_advisor` int(11) default NULL,
`unpaid_date` datetime default NULL,
`ecl_send` tinyint(1) default NULL,
`ecl_date` datetime default NULL,
`birthdateyear` int(11) NOT NULL default '0',
`quaCategorie` varchar(500) collate utf8_bin default NULL,
`quaNature` varchar(500) collate utf8_bin default NULL,
`quaType` varchar(500) collate utf8_bin default NULL,
`april` int(1) NOT NULL default '0',
`date_guide` datetime default NULL,
`no_pub` tinyint(1) NOT NULL default '0',
`campaign_manual` tinyint(1) NOT NULL default '0',
`export_matmut` date default NULL,
`formality_date` datetime default NULL,
`call_count_commerciaux` int(11) default '0',
`call_count_assistance` int(11) default '0',
`call_last` datetime default NULL,
`prospect` tinyint(1) default NULL,
`gender_id` int(11) default NULL,
`birthdate` date default NULL,
`current_customer_source_history_id` int(11) default NULL,
`original_customer_source_history_id` int(11) default NULL,
`bounce` tinyint(1) NOT NULL default '0',
PRIMARY KEY (`id`),
UNIQUE KEY `UNIQ_81398E097D3656A4` (`account`),
UNIQUE KEY `UNIQ_81398E09E7927C74` (`email`),
UNIQUE KEY `UNIQ_81398E091193CB3F` (`customer_address`),
UNIQUE KEY `UNIQ_81398E09507DD4CC` (`business_address`),
UNIQUE KEY `UNIQ_81398E09BA38C653` (`formalities_center_address`),
KEY `num_dossier` (`num_dossier`),
KEY `sub_send_recovery` (`sub_sent_recovery`),
KEY `dossier_canceled` (`dossier_canceled`),
KEY `freeguide` (`freeguide`),
KEY `IDX_81398E09708A0E0` (`gender_id`),
KEY `IDX_81398E0935655550` (`current_customer_source_history_id`),
KEY `IDX_81398E0981A1F986` (`original_customer_source_history_id`)
) ENGINE=MyISAM AUTO_INCREMENT=433026 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `sf_user_data` (
`id` int(11) NOT NULL auto_increment,
`user_id` bigint(20) unsigned NOT NULL,
`register_id` int(11) default NULL,
`insurance_id` int(11) default NULL,
`activity_started_at` date default NULL,
`accre_request_accepted` tinyint(1) default NULL,
`accre_request_accepted_at` date default NULL,
`declaration_frequency_months` smallint(6) default NULL,
`declaration_reminder` tinyint(1) default NULL,
`activities_number` smallint(6) default NULL,
`computed_main_activity_percent_total` double default NULL,
`computed_secondary_activity_percent_total` double default NULL,
`company_address_is_personal_address` tinyint(1) default NULL,
`register_city` varchar(255) collate utf8_unicode_ci default NULL,
`invoice_last_increment` smallint(6) NOT NULL default '0',
`quotation_last_increment` smallint(6) NOT NULL default '0',
`asset_last_increment` smallint(6) NOT NULL default '0',
`payplug_parameters` varchar(255) collate utf8_unicode_ci default NULL,
`displayed_first_connection_dialog` tinyint(1) NOT NULL default '0',
`displayed_first_invoice_display_dialog` tinyint(1) NOT NULL default '0',
`payplug_parameters_created_at` date default NULL,
`payplug_first_payment_at` date default NULL,
`latest_payplug_http_code` int(11) default NULL,
`register_number` varchar(255) collate utf8_unicode_ci default NULL,
`register_code` varchar(255) collate utf8_unicode_ci default NULL,
`register_bis_city` varchar(255) collate utf8_unicode_ci default NULL,
`register_bis_number` varchar(255) collate utf8_unicode_ci default NULL,
`registerBis_id` int(11) default NULL,
`main_activity_type_id` int(11) default NULL,
`secondary_activity_type_id` int(11) default NULL,
`declaration_reminder_popup` tinyint(1) default NULL,
`declaration_reminder_popup_latest_choice` smallint(6) default NULL COMMENT '1 = Me le rappeler demain, 2 = Ne plus afficher cette alerte',
`declaration_reminder_popup_latest_choice_date` date default NULL,
`main_activity_id` int(11) default NULL,
`secondary_activity_id` int(11) default NULL,
`primary_socio_economic_classification_id` int(11) default NULL,
`secondary_socio_economic_classification_id` int(11) default NULL,
`income_bracket_id` int(11) default NULL,
`main_activity_custom` varchar(255) collate utf8_unicode_ci default NULL,
`secondary_activity_custom` varchar(255) collate utf8_unicode_ci default NULL,
`default_further_information` longtext collate utf8_unicode_ci,
`gclid` varchar(255) collate utf8_unicode_ci default NULL,
`main_activity_type_old_id` smallint(6) default NULL,
`main_activity_nature_old_id` smallint(6) default NULL,
`secondary_activity_type_old_id` smallint(6) default NULL,
`secondary_activity_nature_old_id` smallint(6) default NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `UNIQ_E904BFD1A76ED395` (`user_id`),
UNIQUE KEY `UNIQ_E904BFD1D1E63CD1` (`insurance_id`),
KEY `IDX_E904BFD14976CB7E` (`register_id`),
KEY `IDX_E904BFD1DBC024CC` (`registerBis_id`),
KEY `IDX_E904BFD12E864BE8` (`main_activity_type_id`),
KEY `IDX_E904BFD132198C62` (`secondary_activity_type_id`),
KEY `IDX_E904BFD15543A800` (`main_activity_id`),
KEY `IDX_E904BFD1798B8812` (`secondary_activity_id`),
KEY `IDX_E904BFD1D17B29D3` (`primary_socio_economic_classification_id`),
KEY `IDX_E904BFD17758CEBC` (`secondary_socio_economic_classification_id`),
KEY `IDX_E904BFD1BAF920D3` (`income_bracket_id`)
) ENGINE=MyISAM AUTO_INCREMENT=116384 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
It seems like this simple amendment will be faster:
SELECT COUNT(DISTINCT a0_.id) sclr0
FROM account a0_
JOIN customer c1_
ON c1_.account = a0_.id
JOIN address a3_
ON c1_.customer_address = a3_.id
And if data integrity is important to you, consider switching to InnoDB.
Change LEFT JOIN address a3 to plain JOIN. That way, the optimizer could consider starting with address. And add INDEX(city) to address.
Currently, the EXPLAIN is showing hitting 405K rows in each of 3 tables. (1.2M total) With the above change, the optimizer would start with the city index, there by looking at, say 1K rows. After that, it would look up 1K rows in each of the other two tables. (3K total)
LEFT JOIN sf_user_data s2_ ON (s2_.user_id = a0_.id) seems to be totally useless; remove it. (Now down to 2K rows)
Don't blindly use (255), use reasonable numbers. In doing so, if identifier is not very big, make it the PRIMARY KEY and get rid of id.
It is extremely rare to have 6 different columns each being UNIQUE. I think you will run into business-logic trouble because of it.

Laravel where method won't works correctly on my server but work right on my Mac

I have a weird problem, this line wont fetch any record on my server but works all right on my computer, mysql and php version are same, and when i cast id field to string it's work on server (Debian), but it is risky because i use where statement with id very much. this is the code:
$myAnswers=Questioner::where('id',$questioner_id)->first()->getLinkedAnsweres()->where("subject_id",Auth::user()->id);
and when i change it to this, it's work right (cast id to string):
$myAnswers=Questioner::where('id',$questioner_id)->first()->getLinkedAnsweres()->where("subject_id",(string)Auth::user()->id);
subject_id and user id are both unsigned_integer with same size.
How can i fix this problem? Should i overwrite where method? or is there any config which cast unsigned int to string in query?
thanks
PS: Tables structure:
users | CREATE TABLE `users` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`family` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`username` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`email` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`gender` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`age` int(11) DEFAULT NULL,
`password` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`default_password` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`access_level` varchar(255) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'subject',
`phone` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`comments` longtext COLLATE utf8_unicode_ci,
`project_limit` int(10) unsigned NOT NULL,
`questioner_limit` int(10) unsigned NOT NULL,
`created_by` int(10) unsigned DEFAULT NULL,
`remember_token` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `users_username_unique` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci |
answers | CREATE TABLE `answers` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`question_id` int(10) unsigned NOT NULL,
`selected_option` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`subject_id` int(10) unsigned NOT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci |

Search invisible characters in mysql

There are some columns in my mysql table which contain values such as "Sbeed Pharmaceuticals åÊ" . Two questions:
How do I prevent getting these characters from getting into the table in the first place ?
How do I find rows containing these characters ?
Here is the table:
CREATE TABLE `drugs` (
`drug-id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`th-class` varchar(512) DEFAULT NULL,
`division` varchar(512) DEFAULT NULL,
`name` varchar(512) DEFAULT NULL,
`brand-name` varchar(64) DEFAULT NULL,
`pack` varchar(512) DEFAULT NULL,
`rx-type` varchar(16) DEFAULT NULL,
`pack-form` varchar(128) DEFAULT NULL,
`available-in` varchar(512) DEFAULT NULL,
`mrp` double DEFAULT '-1',
`mrp-per-unit` double DEFAULT NULL,
`manufacturer` varchar(512) DEFAULT NULL,
`schemes` text,
`drug-status` varchar(64) DEFAULT 'new',
`created-by` varchar(512) DEFAULT NULL,
`created-at` datetime DEFAULT CURRENT_TIMESTAMP,
`approved-at` datetime DEFAULT NULL,
`approved-by` varchar(512) DEFAULT NULL,
`dosage` varchar(64) DEFAULT NULL,
`manufacturer-id` int(10) DEFAULT NULL,
PRIMARY KEY (`drug-id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

MySQL Can't create table [errno: 150]

Getting the above error on the following script:
DROP TABLE IF EXISTS `incident`;
SET #saved_cs_client = ##character_set_client;
SET character_set_client = utf8;
CREATE TABLE `incident` (
`incident_id` int(11) NOT NULL auto_increment,
`patient_id` int(11) default NULL,
`history_id` int(11) default NULL,
`incident_date` datetime default NULL,
`incidentname_id` int(11) default NULL,
`incident_type_id` int(11) default NULL,
`pop_first_symptom` int(11) default NULL,
`first_symptom_date` datetime default NULL,
`similar_symptom_date` datetime default NULL,
`first_consultation_date` datetime default NULL,
`accident` int(1) default NULL,
`accident_date` datetime default NULL,
`pop_accident_state` int(11) default NULL,
`pop_condition_related_to` int(11) default NULL,
`flag_prothesis` char(1) collate utf8_unicode_ci NOT NULL default '0',
`flag_initial_placement` char(1) collate utf8_unicode_ci NOT NULL default '0',
`flag_orthodontics` char(1) collate utf8_unicode_ci NOT NULL default '0',
`flag_occupational_illness` char(1) collate utf8_unicode_ci NOT NULL default '0',
`box19` varchar(80) collate utf8_unicode_ci NOT NULL default '',
`color` varchar(8) collate utf8_unicode_ci NOT NULL default 'ffffff00',
`referrals_id` int(11) default NULL,
`referral_date` datetime default NULL,
`facility_id` int(11) default NULL,
`attorney_id` int(11) default NULL,
`attorney_notes` text collate utf8_unicode_ci,
`attorney_report_sent` date default NULL,
`attorney_deposition_date` date default NULL,
`attorney_trial_date` date default NULL,
`attorney_notice_filing` char(1) collate utf8_unicode_ci default NULL,
`attorney_final_settlement` date default NULL,
`effective_date` datetime default NULL,
`termination_date` datetime default NULL,
`date` datetime default NULL,
`pop_branch` int(11) default NULL,
`pop_status` int(1) default NULL,
`pop_eligibility` int(1) default NULL,
`partial_begin_date` datetime default NULL,
`partial_end_date` datetime default NULL,
`last_work_date` datetime default NULL,
`partial_return_to_work_date` datetime default NULL,
`total_begin_date` datetime default NULL,
`total_end_date` datetime default NULL,
`last_xray_date` datetime default NULL,
`last_seen_date` date default NULL,
`assumed_date` datetime default NULL,
`relinquished_date` datetime default NULL,
`radiographs` varchar(30) collate utf8_unicode_ci NOT NULL default '',
`oral_images` varchar(30) collate utf8_unicode_ci NOT NULL default '',
`models` varchar(30) collate utf8_unicode_ci NOT NULL default '',
`date_appliance_placed` date default NULL,
`date_prior_placement` date default NULL,
`paymentSourceCode` varchar(10) collate utf8_unicode_ci NOT NULL default '',
`locatorCode` varchar(10) collate utf8_unicode_ci NOT NULL default '',
`SAExceptionCode` varchar(10) collate utf8_unicode_ci NOT NULL default '',
`flag_expanded` int(1) default NULL,
`flag_archived` int(1) default NULL,
`cda_reason_code` int(11) default NULL,
`is_extraction` int(11) default '0',
`extraction_tooth` varchar(10) collate utf8_unicode_ci default NULL,
`initial_placement_date` date default NULL,
`initial_placement_location` int(11) default '0',
`prosthesis_material` int(11) default '0',
PRIMARY KEY (`incident_id`),
KEY `patient_id_key` (`patient_id`),
CONSTRAINT `incident_ibfk_1` FOREIGN KEY (`patient_id`) REFERENCES `patient` (`patient_id`)
) ENGINE=InnoDB AUTO_INCREMENT=9854 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
SET character_set_client = #saved_cs_client;
Here is the patient table giving me the same error:
DROP TABLE IF EXISTS `patient`;
SET #saved_cs_client = ##character_set_client;
SET character_set_client = utf8;
CREATE TABLE `patient` (
`patient_id` int(11) NOT NULL auto_increment,
`history_id` int(11) default NULL,
`person_id` int(11) default NULL,
`account_id` int(11) default NULL,
`display_id` varchar(10) collate utf8_unicode_ci default NULL,
`chart_num` varchar(10) collate utf8_unicode_ci default NULL,
`bridge_num` varchar(10) collate utf8_unicode_ci default NULL,
`provider_id` int(11) default NULL,
`office_id` int(11) default NULL,
`feeschedule_id` int(11) default NULL,
`pop_marital_status` int(11) default NULL,
`pop_employment` int(11) default NULL,
`new_patient_date` date default NULL,
`primary_ins_id` int(11) default NULL,
`primary_ins_type` char(1) collate utf8_unicode_ci default NULL,
`insured_id_number` varchar(30) collate utf8_unicode_ci default NULL,
`flag_special_insurance` char(1) collate utf8_unicode_ci default '0',
`flag_family_planning` char(1) collate utf8_unicode_ci default '0',
`flag_epstd` char(1) collate utf8_unicode_ci default '0',
`pop_relation_to_primary` int(11) default NULL,
`pop_relation_to_secondary` int(11) default NULL,
`pop_hipaa_release` int(11) default NULL,
`hipaa_release_date` date default NULL,
`patientNotes` text collate utf8_unicode_ci,
`autoremind_notification_type_id` int(11) NOT NULL default '0',
`ethnicity_id` int(11) default NULL,
`patient_status` int(1) NOT NULL default '0',
`created_by_module` char(1) collate utf8_unicode_ci NOT NULL default 'P',
`pop_patient_exception_code` int(11) default '0',
`pop_native_language` int(11) default '0',
`school_name` varchar(30) collate utf8_unicode_ci default NULL,
`graduation_date` date default NULL,
`hygienist_id` int(11) default NULL,
`school_address_id` int(11) default NULL,
`language_id` int(11) default NULL,
PRIMARY KEY (`patient_id`),
KEY `account_id_key` (`account_id`),
KEY `person_id_key` (`person_id`),
KEY `patient_status_index` (`patient_status`),
KEY `person_id_index` (`person_id`),
KEY `account_id_index` (`account_id`),
CONSTRAINT `patient_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `account` (`account_id`),
CONSTRAINT `patient_ibfk_2` FOREIGN KEY (`person_id`) REFERENCES `person` (`person_id`)
) ENGINE=InnoDB AUTO_INCREMENT=9855 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
SET character_set_client = #saved_cs_client;
I assume that your foreign key patient_id is not NULL. It must match to your referenced data type.
`patient_id` int(11) default NULL