Subquerying in mysql with full-text querying - mysql

Question

I am trying to query a database. I already have a file that includes some primary keys out of this whole database. Now I want to filter these primary keys and get only those primary keys which also agree to "other condition". My primary keys are associated to abstracts in the database. The abstracts are full-text indexed. Now I want to consider the abstracts using the given primary keys, look for my "other condition(terms)" in those abstracts and if its present I want to pull their primary keys out(which is going to be same from the file). My "other condition" is another file with a list of terms. I want to get the abstracts that contain those terms within the given primary keys.
My full-text search is something like this:
while(<FILE1>){
$PK = $_;
foreach $foo(#foo){
my $sth = $dbh->prepare(qq{
SELECT value
FROM value_table
WHERE MATCH (column_text_indexed) AGAINST (? IN BOOLEAN MODE)
}) AND primary_key=$PK;
$sth->execute(qq{+"$foo"});
}
}
where $PK is coming from the list of primary keys I already have.
$foo will be the list of the terms (condition 2) I am looking for.
Normally, I can run this query number of $PK times number of $foo. But I learned something about optimization by sub querying where I won't be running my query # $PK times # $foo. That will get rid of inner loop but will still form combination of every $PK with every term in file 2 that is #foo. Something like as follows:
while(<FILE1>){
$PK = $_;
my $sth = $dbh->prepare(qq{
SELECT value
FROM value_table
WHERE MATCH (column_text_indexed) AGAINST (**SUB QUERYING HERE**)
}) AND primary_key=$PK;
$sth->execute(qq{+"$foo"});
}
Just I don't know how to do it. I may be wrong with the syntax. I want to know how to write code for full-text search as well as a subquery. I hope this will be efficient than querying directly the combinations. Any help is greatly appreciated.

Answer 1

I don't think you need to use a subquery. But you can still get rid of the inner loop by combining the match strings.
my $against = join ' ', map {qq/"$_"/} #foo;
while (my $PK = <FILE1>) {
chomp $PK;
my $sth = $dbh->prepare(qq{
SELECT value
FROM value_table
WHERE primary_key = ?
# no '+' so it can match against at least one of the words in the list
AND MATCH (column_text_indexed) AGAINST (? IN BOOLEAN MODE)
});
$sth->execute($PK, $against);
Update
I revised it and have completely removed the query from the loops.
my #primary_keys;
while (my $PK = <FILE1>) {
chomp $PK;
push #primary_keys, $PK;
}
my $PK_list = join ',', map {qq/'$_'/} #primary_keys;
my $against = join ' ', map {qq/"$_"/} #foo;
my $sth = $dbh->prepare(qq{
SELECT value
FROM value_table
# placeholders can only represent single scalar values so $PK_list can't be bound
WHERE primary_key IN ($PK_list)
# no '+' so it can match against at least one of the words in the list
AND MATCH (column_text_indexed) AGAINST (? IN BOOLEAN MODE)
});
$sth->execute($against);
# continue with fetching the rows
...;

Answer 2

Your syntax looks dodgy. I think you meant:
while(<FILE1>){
$PK = $_;
foreach $foo (#foo){
my $sth = $dbh->prepare(qq{
SELECT value
FROM value_table
WHERE MATCH (column_text_indexed)
AGAINST (**SUB QUERYING HERE**)
AND primary_key=$PK }); # '})' after AND clause
$sth->execute(qq{ $foo });
}
}
But why not make $PK an additional argument in this case (and use best practices)? :
while ( my $PK = <FILE1> ) {
chomp $PK; # Remove trailing newline
foreach my $foo ( #foo ) { # Lexical $foo
my $sth = $dbh->prepare( qq{
SELECT value
FROM value_table
WHERE MATCH (column_text_indexed)
AGAINST (**SUB QUERYING HERE**)
AND primary_key=? }); # Extra placeholder
$sth->execute( $foo, $PK );
}
}

Answer 3

If you want efficiency I would recommend to use least database transactions and operations. So in this case I think the best option is to just get the abstract from the database based the primary key and then search for the terms in that abstract by doing simple string search in your pearl or any other standard language code. I am not very sure of the length of your list of terms. But if its possible you can save it in a standard data-structure like array or list. The same operation in database would definitely take lot more time. I am not that good with pearl syntax so I am writing the algorithm.
for all the terms in PK get the abstract as a string variable :
for each term in array/list: find the term in string variable containing abstract. If found add the PK to a new file.
continue with next pk.
if not found then continue next term in array/list.