Map Reduce with Relational Databases - mysql

Question

I have 2 relational tables
Table A (Person 1, Title of Book Read)
Table B (Book Title, Author Name)
I'm creating a map-reduce job which counts the books by author which are read by every person in table 1.
This means that if there were 2 books by the same author and the person read both, then the map-reduce would yield:
(Person1, Author 1, 2);
My map function (at the Meta-level) is:
map {
emit(TableB.BookTitle, 1)
}
and my reduce function is:
reduce function (title,values)
{
while(values.hasNext())
{
if(title == tableA.bookRead)
sum+=values
}
output.collect(tableA.person1, tableB.author, sum)
}
I know there are some holes to be filled between the person reading the books but I'm not quite sure how to approach it? Also would I have to run this query for every person in the Table B?

Answer 1

We can break the given problem into two jobs:
1) In the first part we should create a map reduce job with two mapper. For First Mapper-A Table A is the input and for second Mapper-B table B is the input. And there will be only one reducer.
Mapper A emits "BooK Title" as Key and "Person Name#Table-A".
Mapper B emits "Book Title" as Key and "Author Name#Table-B"
Since in Map-Reduce records for one key goes to same reducer and in this job we just have one reducer so records will reach over there like
{Book Title,
Then you need to implement logic to extract out Person Name and Author Name. At the reducer end and Reducer will emit its output as:
Book Title %Author Name%PersonName
for eg.
while(values.hasNext())
{
String line = values.next().toString();
String[] det_array = line.split("#");
if(det_array[0].equals("person_book"))
{
person_name = det_array[1];
emit_value = emit_value + person_name + ",";
}
else if(det_array[0].equals("auth_book") && !author_seen)
{
author_name = det_array[1];
emit_value = emit_value + "%" + author_name + "%" + ",";
author_seen = true;
}
}
output.collect(new Text(key),new Text(emit_value));
Then Your Final Output File Will Look Like:
Book Title %Author_Name%Person Name
2) In the Second Map Reduce Job: Code Just One Mapper and Reducer. Input for Your Job is of format:
Book Title %Author_Name%Person Name1,PersonName2 etc..
For Your Mapper Output Key is Author_Name+Person and Value is 1.
As at this stage you have Combination of Author_Name and Person in Reducer you just need to count 1 and emit outout as Person Name, Author Name and Total Count.
Please let me know if this is not clear to you or you would like to see actual java code.
THanks !!