Is there a way to use MySQL fulltext to search related tables? - mysql

I have a table called persons which contains data about, well, people. It also contains foreign keys to another table. I'd like to make a fulltext index that is able to search the related tables for full text.
Here is some sample data: (see http://sqlfiddle.com/#!9/036fc5/2)
CREATE TABLE IF NOT EXISTS `states` (
`id` char(2) NOT NULL,
`name` varchar(45) NOT NULL,
PRIMARY KEY (`id`)
);
INSERT INTO `states` (`id`, `name`) VALUES
('NY', 'New York'),
('NJ', 'New Jersey'),
('CT', 'Connecticut'),
('PA', 'Pennsylvania');
CREATE TABLE IF NOT EXISTS `persons` (
`id` int auto_increment NOT NULL,
`first_name` varchar(45) NOT NULL,
`last_name` varchar(45) NOT NULL,
`state_id` char(2) not null,
PRIMARY KEY (`id`),
FULLTEXT (first_name, last_name, state_id)
);
INSERT INTO `persons` (`first_name`, `last_name`, `state_id`) VALUES
('Arnold', 'Asher', 'NY'),
('Bert', 'Bertold', 'NJ'),
('Charlie', 'Chan', 'NJ'),
('Darrin', 'Darcy', 'CT');
So, I'd like to be able to search for persons from "Jersey", such as:
SELECT * FROM persons WHERE MATCH(first_name, last_name, state_id) AGAINST('Jersey');
But, of course, the text "Jersey" exists only in the states table and not in the persons table. Does it make sense to make a materialized/generated index? Is there a simpler way?

You need to put a separate full-text index on the states table, and join with that.
CREATE TABLE IF NOT EXISTS `states` (
`id` char(2) NOT NULL,
`name` varchar(45) NOT NULL,
PRIMARY KEY (`id`),
FULLTEXT (name)
);
CREATE TABLE IF NOT EXISTS `persons` (
`id` int auto_increment NOT NULL,
`first_name` varchar(45) NOT NULL,
`last_name` varchar(45) NOT NULL,
`state_id` char(2) not null,
PRIMARY KEY (`id`),
FULLTEXT (first_name, last_name);
SELECT p.*
FROM persons p
JOIN states s ON s.id = p.state_id
WHERE MATCH(s.name) AGAINST ('Jersey')
UNION
SELECT *
FROM persons
WHERE MATCH(first_name, last_name) AGAINST ('Jersey')

In MySQL, no type of index spans multiple tables. Not fulltext indexes, not spatial indexes, not btree indexes, not hash indexes.
Every type of index you can define belongs to exactly one table, and can index only the values in that table.

Related

Unique Key on two table

is it possible to make two tables have a unique key like if my first table has the unique key and the second table cannot have the same text as the first table, it is possible to make that?
--
-- Table structure for table admin
DROP TABLE IF EXISTS admin;
CREATE TABLE IF NOT EXISTS admin (
ID varchar(11) NOT NULL,
Name varchar(100) NOT NULL,
Password varchar(100) NOT NULL,
Email varchar(100) NOT NULL,
PhoneNumber varchar(255) NOT NULL,
UniqueCode varchar(255) NOT NULL,
PRIMARY KEY (ID),
UNIQUE KEY UniqueCode (UniqueCode)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
--
-- Dumping data for table admin
INSERT INTO admin (ID, Name, Password, Email, PhoneNumber, UniqueCode) VALUES
('AA11', 'Admin Low', '827ccb0eea8a706c4c34a16891f84e7b', 'AA11#gmail.com', '6012346778', 'Lmao'),
('AA12', 'Admin Tyler', '827ccb0eea8a706c4c34a16891f84e7b', 'AA11#gmail.com', '6033556778', 'Rofl');
--
-- Table structure for table lecturer
DROP TABLE IF EXISTS lecturer;
CREATE TABLE IF NOT EXISTS lecturer (
ID varchar(11) NOT NULL,
Name varchar(100) NOT NULL,
Password varchar(100) NOT NULL,
Email varchar(100) NOT NULL,
PhoneNumber varchar(255) NOT NULL,
UniqueCode varchar(255) NOT NULL,
PRIMARY KEY (ID),
UNIQUE KEY UniqueCode (UniqueCode)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
--
-- Dumping data for table lecturer
INSERT INTO lecturer (ID, Name, Password, Email, PhoneNumber, UniqueCode) VALUES
('AL11', 'Cat Eat my son', '827ccb0eea8a706c4c34a16891f84e7b', 'AL11#gmail.com', '6012342222', 'Meow'),
('AL12', 'Dog Eat my son', '827ccb0eea8a706c4c34a16891f84e7b', 'AL12#gmail.com', '6033345678', 'Woof');
You can simply check before inserting data by useng "EXIST".
The EXISTS operator is used to test for the existence of any record in a subquery.
The EXISTS operator returns TRUE if the subquery returns one or more records.
SELECT column_name(s)
FROM table_name
WHERE EXISTS
(SELECT column_name FROM table_name WHERE condition);

MySQL - Improve Performance for Multi Country

I have a table company, which has a column called country_id is a reference of country table Primary Key ID.
CREATE TABLE `company` (
`Id` INT(11) NOT NULL,
`Name` VARCHAR(100) NOT NULL,
`Symbol` VARCHAR(50) NOT NULL,
`Industry` VARCHAR(100) NOT NULL,
`Type` VARCHAR(20) NOT NULL,
'country_id' INT(11) NOT NULL,
PRIMARY KEY (`Id`),
CONSTRAINT `company_ibfk_1` FOREIGN KEY (`country_id`) REFERENCES `country` (`Id`)
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB;
In this table has multiple company for multiple country. Like the below table.
My company table has huge data. I want to improve the select query on company table performance based on country. Select query will be like
select * from company where country_id = 2
What will be the best design approach? Do I need to add indexing on country_id column or do I need to do the partitioning based on the country_id column? Please suggest.
if you have big count of countries (more then 100), you should use index.
If few number countries in your DB would be better to use partitioning.
But this option has one big disadvantage. If you will add new country, you should add new partition linked to new one.
InnoDB will automatically generate index on your foreign key country_id, so you do not need to indexing on it.
About partition, InnoDB does not support foreign key and partition together and if you use partition you can just create an index on country_id (not foreign key) and join both tables if you needed.
If you have more than one million records on company and your query is not fast enough, you can use partition, like below:
CREATE TABLE `company` (
`Id` INT(11) NOT NULL,
`Name` VARCHAR(100) NOT NULL,
`Symbol` VARCHAR(50) NOT NULL,
`Industry` VARCHAR(100) NOT NULL,
`Type` VARCHAR(20) NOT NULL,
`country_id` INT(11) NOT NULL,
PRIMARY KEY (`Id`),
INDEX (`country_id`)
)
PARTITION BY RANGE (`country_id`) (
PARTITION P1 VALUES LESS THAN (20),
PARTITION P2 VALUES LESS THAN (50),
PARTITION P3 VALUES LESS THAN (100),
PARTITION P4 VALUES LESS THAN MAXVALUE)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB;

PHP SQL Inner Join or Union

I want to select different fields from two different tables My users table and my address table as shown below:
my users table:
CREATE TABLE IF NOT EXISTS `users` (
`ID` int(3) NOT NULL AUTO_INCREMENT,
`AccountNumber` int(8) NOT NULL,
`FirstName` text NOT NULL,
`LastName` text NOT NULL,
`EmailAddress` varchar(200) NOT NULL,
`Password` varchar(200) NOT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `AccountNumber` (`AccountNumber`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=2 ;
INSERT INTO `users` (`ID`, `AccountNumber`, `FirstName`, `LastName`, `EmailAddress`, `Password`) VALUES
(1, 123456, 'test', 'test', 'test#test.ac.uk', '$2y$10$/j9nTE5ugmyrWuV8VNWFxe5iHInqyaTwxt5wDaxyQwPUZTDDjqNKm');
my address table:
CREATE TABLE IF NOT EXISTS `address` (
`AddID` int(4) NOT NULL AUTO_INCREMENT,
`HouseNumber` int(11) NOT NULL,
`StreetName` varchar(200) NOT NULL,
`City` varchar(200) NOT NULL,
`Postcode` varchar(8) NOT NULL,
`AccountNumber` int(8) NOT NULL,
PRIMARY KEY (`AddID`),
KEY `AccountNumber` (`AccountNumber`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=2 ;
INSERT INTO `address` (`AddID`, `HouseNumber`, `StreetName`, `City`, `Postcode`, `AccountNumber`) VALUES
(1, 123, 'Some Road', 'someCity', 'b66', 123456);
ALTER TABLE `address`
ADD CONSTRAINT `address_ibfk_1` FOREIGN KEY (`AccountNumber`) REFERENCES `users` (`AccountNumber`);
Ive manage to use the following inner join successfully:
SELECT * FROM users INNER JOIN address ON users.AccountNumber=address.AccountNumber
However that also seems to retrieve the auto incremented ID's as well as the password field which i dont want.
I tried union but kept getting the error about different number of columns. How would i change my select statement to only retrieve the FirstName, LastName and EmailAddress from the users table and the HouseNumber, Street, City and Postcode from the address table based on the account number stored in both tables.
Any info will be appreciated
Thanks!
Try this u will get the result
SELECT FirstName,LastName,EmailAddress,HouseNumber,
Street, City, Postcode FROM users
INNER JOIN address ON users.AccountNumber=address.AccountNumber
instead of select *, just list out the columns youd like
SELECT u.FirstName,u.LastName,u.EmailAddress,a.HouseNumber,a.StreetName,a.City,a.Postcode
FROM users u
INNER JOIN address a
ON u.AccountNumber=a.AccountNumber
from there you can add a where statement if you want to limit it down to one AccountNumber:
SELECT u.FirstName,u.LastName,u.EmailAddress,a.HouseNumber,a.StreetName,a.City,a.Postcode
FROM users u
INNER JOIN address a
ON u.AccountNumber=a.AccountNumber
WHERE u.AccountNumber=12345

Can I add a Unique key on table creation in SQL?

I am trying to translate a collection of MySQL functions to SQL, and I'm having issues with a UNIQUE KEY issue:
-- -----------------------------------------------------
-- Table testform
-- -----------------------------------------------------
CREATE TABLE `testform` (
`FormId` INT(11) NOT NULL AUTO_INCREMENT,
`TTId` INT(11) NULL DEFAULT NULL,
`TestName` VARCHAR(100) NULL,
PRIMARY KEY (`FormId`),
UNIQUE KEY `TF_Composite` (`TTId`, `TestName`));
When I try and test this in SQLFiddle, it's giving me the error
Incorrect syntax near the keyword 'KEY'.
I have tried searching for this, but so far all I have come up with is "Unique Constraints". Is there a difference between a "Key" and a "Constraint" in SQL? And if so, how can I add this in the table creation statement?
Your syntax is all messed up. Please look at books on-line (MSDN).
https://msdn.microsoft.com/en-us/library/ms174979.aspx
The sample code below create a table in tempdb. This table automatically gets destroyed when the service is restarted.
-- Just a example, throw away after reboot
USE [tempdb]
GO
-- Create the table
CREATE TABLE DBO.TESTFORM
(
FORM_ID INT IDENTITY(1, 1) NOT NULL ,
TT_ID INT NULL,
TEST_NAME VARCHAR(100) NULL,
CONSTRAINT PK_FORM_ID PRIMARY KEY (FORM_ID),
CONSTRAINT UN_COMPOSIT UNIQUE (TT_ID, TEST_NAME)
);
-- Seventies Band
INSERT INTO TEMPDB.DBO.TESTFORM VALUES (1, 'John');
INSERT INTO TEMPDB.DBO.TESTFORM VALUES (2, 'Paul');
INSERT INTO TEMPDB.DBO.TESTFORM VALUES (3, 'Mary');
GO
-- Show data
SELECT * FROM TEMPDB.DBO.TESTFORM
GO
The image below shows the data in this table.
Try This.
CREATE TABLE testform (
FormId INT(11) NOT NULL AUTO_INCREMENT,
TTId INT(11) NULL DEFAULT NULL,
TestName VARCHAR(100) NULL,
PRIMARY KEY (FormId),
CONSTRAINT TF_Composite UNIQUE (TTId,TestName));
More Details..
For Better Understanding about Primary and Unique you can refer below page.
Primary and Unique Key Creation
For MySQL Database
CREATE TABLE `phone` (
`id` MEDIUMINT(8) UNSIGNED NOT NULL AUTO_INCREMENT,
`country` DECIMAL(5,0) UNSIGNED NOT NULL,
`area` DECIMAL(5,0) UNSIGNED NOT NULL,
`number` DECIMAL(8,0) UNSIGNED NOT NULL,
`extension` DECIMAL(5,0) UNSIGNED DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `ix_phone` (`country`, `area`, `number`, `extension`),
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8;
For alter Table :
ALTER TABLEphone
ADD UNIQUE INDEXix_phone(country,area,number,extension);

MySQL-Query too slow

I am using the following tables in my MySQL-Database:
--
-- Table structure for table `company`
--
CREATE TABLE IF NOT EXISTS `company` (
`numb` varchar(4) NOT NULL,
`cik` varchar(30) NOT NULL,
`sNumber` varchar(30) NOT NULL,
`street1` varchar(255) NOT NULL,
`street2` varchar(255) NOT NULL,
`city` varchar(255) NOT NULL,
`state` varchar(100) NOT NULL,
`zip` varchar(100) NOT NULL,
`phone` varchar(255) NOT NULL,
`name` varchar(255) NOT NULL,
`dateChanged` varchar(30) NOT NULL,
`name2` varchar(255) NOT NULL,
`seriesId` varchar(30) NOT NULL,
`symbol` varchar(10) NOT NULL,
`exchange` varchar(20) NOT NULL,
PRIMARY KEY (`cik`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
INSERT INTO `company` (`numb`, `cik`, `sNumber`, `street1`, `street2`, `city`, `state`, `zip`, `phone`, `name`, `dateChanged`, `name2`, `seriesId`, `symbol`, `exchange`) VALUES
('6798', 'abc', '953551121', '701 AVENUE', '', 'GLENDALE', 'CA', '91201-2349', '818-244-8080', '', '', 'Public Store', '', 'PSA', 'NYSE')
--
-- Table structure for table `data`
--
CREATE TABLE IF NOT EXISTS `data` (
`id` int(100) NOT NULL AUTO_INCREMENT,
`number` varchar(100) NOT NULL,
`elementname` mediumtext NOT NULL,
`date` varchar(100) NOT NULL,
`elementvalue` longtext NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=18439;
INSERT INTO `data` (`id`, `number`, `elementname`, `date`, `elementvalue`) VALUES
(1, '0001393311-10-000004', 'StockholdersEquityIncludingPortionAttributableToNoncontrollingInterest', '2009-12-31', '3399777000')
--
-- Table structure for table `filing`
--
CREATE TABLE IF NOT EXISTS `filing` (
`number` varchar(100) NOT NULL,
`file_number` varchar(100) NOT NULL,
`type` varchar(100) NOT NULL,
`amendment` tinyint(1) NOT NULL,
`date` varchar(100) NOT NULL,
`cik` varchar(30) NOT NULL,
PRIMARY KEY (`accession_number`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
INSERT INTO `filing` (`number`, `file_number`, `type`, `amendment`, `date`, `cik`) VALUES
('0001393311-10-000004', '001-33519', '10-K', 0, '2009-12-31', '0000751653'),
('0000751652-10-000006', '001-08796', '10-K', 0, '2009-12-31', '0000751652')
The data table has around 22.000 entries, filing and company tables have around 400 entries each. I want to operate the database with a lot more entries in the future.
I perform the following query, which selects the newest item with a given type:
SELECT data.elementname, data.elementvalue, company.name2 FROM data
JOIN filing ON data.number = filing.number
JOIN company ON filing.cik = company.cik
WHERE elementname IN ('Elem1', 'Elem2', 'Elem3', 'Elem4', 'Elem5', 'ElemN')
AND number IN (
SELECT number
FROM filing
WHERE filing.cik IN ('cik1', 'cik2', 'cikN')
AND filing.type = '1L'
GROUP BY filing.cik
)
It takes between ~0.28 and 0.4 seconds to complete, which appears to be very slow.
When i perform the query without the following line
WHERE filing.cik IN ('cik1', 'cik2', 'cikN')
it takes only ~0.035 seconds.
Any idea how to speed the query up or to optimize the table structure because the table is growing rapidly and it's already too slow.
First off, the table structure you posted for filing is incorrect, as the primary key you specified doesn't. I'll assume you mean number. Additionally, you didn't specify the table definition for company, which makes trying to provide advice for this somewhat difficult.
However, both of the comments are correct. You need some indexes. Based on the query, you should probably some the following indexes.
ALTER TABLE company ADD INDEX ( cik )
ALTER TABLE data ADD INDEX ( number )
I would also recommend taking a look at whether data.elementname actually needs to be a MEDIUMTEXT, which is a pretty huge column. If the rest of the data looks like the example data you provided, you should probably change it into a varchar. TEXT columns can cause some serious performance penalties due to the way they're stored.
Additionally, your PRIMARY KEY number columns, which are currently strings, look as though they could be reformatted into different columns that are actually of type INT. Keep in mind that VARCHAR PRIMARY KEY columns will not be as efficient as INTs, just because they're so much bigger.
Lastly, 22k rows isn't all that much data. You should a take a look at your my.cnf settings. Your key_buffer value may be too small to fit indexes entirely in memory. Additionally, you may want to consider using INNODB for these tables, combined with an innodb_buffer_pool value that'll keep everything in memory.