Group and sort results by max and count in joined table using QueryBuilder - mysql

I'm making an app for selling books but I'm struggling with this part of it.
When a user wants to buy a book he can send a message to the book's seller. I have a table called conversations and another table called messages, which holds the messages for a conversation.
I have a page where a user can see a list of conversations he's involved. Conversations with the latest messages should be shown first. I solved this already but I also want to show the number of unread messages next to each conversation (messages have a "read" boolean field). I haven't been able to add the count as part of my original query; I know I could make a query for each item in the conversations collection but that would be inefficient as hell.
This is my query so far with the QueryBuilder:
$conversations = DB::table('conversations')
->select([
'books.title as bookTitle', 'conversations.id',
DB::raw('max(messages.created_at) as lastMessage'),
// DB::raw('(select count(id) from messages')
])
->where('conversations.from_user', $this->user->id)
->orWhere('conversations.to_user', $this->user->id)
->join('books', 'conversations.book_id', '=', 'books.id')
->join('messages', 'conversations.id', '=', 'messages.conversation_id')
->groupBy('messages.conversation_id')
->orderBy('lastMessage', 'DESC')
->get();
If i add the part commented out to attempt to get a field with the number of unread messages for that conversation I get a SQL syntax error. I also been thinking if maybe is not possible to add the count field since it might be exclusive to my original query: I sort the conversations taking into account all messages (read or unread) and pick the latest message for each conversation; the count field I want to add should only count unread messages.
Any ideas? Hope I explained myself.
Thanks.

It seems that instead of:
DB::raw('(select count(id) from messages')
you should simply use:
DB::raw('(select count(id) from messages) as messageCount')
You were missing here )

Related

Group Conversation by latest message Laravel

I can't seem to figure this out since mysql is not my strong suit. I've done a bit of research here but can't put two and two together, so I'm turning to the community for some help.
I am building a chat function inside my Laravel app where two users can talk to each other. I can't figure out how to build the inbox (group the conversations together) portion of things. I've figured out how to get last message grouped by sending it.
The following is inside my User Model:
public function lastMessages() {
return $this->hasMany('App\Message', 'recipient_id')
->select('*')
->join(DB::raw('(Select max(id) as id from messages group by author_id) LatestMessage'), function($join) {
$join->on('messages.id', '=', 'LatestMessage.id');
})
->orderBy('created_at', 'desc');
}
My messages table consists of the following: author_id, recipient_id, messaged, viewed.
What I need to do is group messages, whether incoming our outgoing and display the latest message as inbox entry and once I click on that messages, the rest of the conversation pops up on the page.
My current sql shown above only gives me 1 last message, not the whole conversation.
Thanks in advance for all the help.
Firstly, I think you should keep the relationship function separate from logical(query) function.
If I understood your question correctly , you should do something like this :
Message::where('recipient_id',$userid)->orWhere('author_id',$userid)->orderBy('created_at','desc')->get();
This will give you messages send or received by user. Please clarify your question with Author and Message models if this is not what you want.

Querying multiple relationships in Laravel and left join another table

i'm trying to do something perhaps a bit too crazy with Eloquent right now, i have a database where i have the following Tables
Crons - (Has Many) - Campaign - (Has Many) - Leads - (Has Many) - Conversions
I need to get all leads from a Cron, that have no entries in the Conversions table in the last X amount of days
I'm thinking of using a Scope on the Cron model but i'm completely stuck on how to proceed from here.
public function scopeWithValidLeads($query) {
return $query->with(['leads' => function($q) {
}]);
}
So i need to get LEADS where the following is true.
A - The leads belong to a campaign associated with the Cron via a Many-To-Many relationship.
B - They have no record in the conversions table Under this specific campaign or if they do, that the lead is older than X amount of days.
You can get your desired result using doesntHave() method like this:
$x = 10; // last 10 days
$crons = Cron::doesntHave('compaign.leads.conversions')
->whereBetween('created_at', [Carbon::now(), Carbon::now()->addDays($x)])
->get();
Querying Relationship Absence: When accessing the records for a model, you may wish to limit your results based on the absence of a relationship. For example, imagine you want to retrieve all blog posts that don't have any comments. To do so, you may pass the name of the relationship to the doesntHave method
UPDATE
As per the updated question conditions, according to my understanding the leads can be obtained by:
$leads = Lead::whereHas('compaign', function($q) use($compaign) {
$q->has('crons')
->where('id', $compaign->id);
// Use the above line if in case of a compaign is to be filtered out
})->doesntHave('conversions')
->whereBetween('created_at', [Carbon::now(), Carbon::now()->addDays($x)])
->get();
Hope this helps!

Order all queries

I'm creating a new chat system with ColdFusion, but I'm having a lot of trouble with sorting out a little problem.
First of all I'll explain how the system works. When creating a new chat with someone, it creates a conversation row in the MySQL table conversations. After that, users can send each other messages. When sending a message, a row is inserted into the table conversations_messages with the timestamp, the user that sent the message, the message, and of course the id of the message. Sounds logical, but now comes the big problem: organizing all those messages.
Users can view their messages with others on a page, this is the code of that page:
// Check conversation ID based on page url
<cfif IsDefined('URL.chat') and URL.chat neq "">
// Load conversation based on page url
<cfquery name = "getconv" datasource = "#DSN#">
SELECT *
FROM `conversations`
</cfquery>
// Get all messages from this conversation
<cfoutput query="getconv">
<cfquery name = "getmessages" datasource = "#DSN#">
SELECT *
FROM `conversations_messages`
WHERE `conversation` = '#chat#'
ORDER BY `conversations_messages`.`id` DESC
</cfquery>
// Get messages sent by 'user_one'
<cfquery name = "my" datasource = "#DSN#">
SELECT *
FROM `conversations_messages`
WHERE `conversation` LIKE '#chat#'
AND `user` LIKE '#user.id#'
ORDER BY `conversations_messages`.`id` DESC
</cfquery>
// Get messages sent by 'user_two'
<cfquery name = "friend" datasource = "#DSN#">
SELECT *
FROM `conversations_messages`
WHERE `conversation` LIKE '#chat#'
AND `user` LIKE '#getconv.user_two#'
</cfquery>
<div class="content">
// Messages sent by 'user_one' aka. user that created conversation
You: <cfoutput><cfloop query="my">#my.message#<br></cfloop></cfoutput>
// Messages sent by 'user_two'
Friend: <cfoutput><cfloop query="friend">#friend.message#</cfloop></cfoutput>
</cfoutput>
</div>
</cfif>
This is how the results show up right now: (as you can see it's sorted by user)
I want it to be sorted by message ID, so it shows up like this:
I hope it's a bit clearer now!
Help would be appreciated!
(not sure why everyone is answering in comments rather than as "an answer"?)
The order that your results are returned from the DB are purely down to the order you tell them to be returned. So if you want them chronologically, then don't fetch them ordered by name; order them chronologically.
So don't get user1's messages, and then user2's messages, just get the messages for the entire conversation in conversation order. Then when you're outputting them, look at which user made which comment and do the "you" / "friend" stuff.
That said, I think the approach you're taking is less than ideal. What you should perhaps be doing is fetching the messages incrementally, as they happen, rather than waiting to get all of them (if you see what I mean). So on each person's UI get all the messages since the last displayed message, and display 'em (in chronological order), then wait for some polling interval, and then fetch the next lot of messages that have occurred since the last time (etc).
If you're using ColdFusion 10 all this stuff has already been done for you via the Web Sockets technology CF has. So that would be the way to go if that's an option for you.
(as Adam suggested here is an answer)
It seems to me that your query getmessages has what you are after. It looks to be all messages from this conversation = chat ordered by conversations_messages.id.
You really shouldn't query for * either, specify the needed columns only. You will get better performance that way.
Also, always use <cfqueryparam> tags in your cfquery blocks.
Also, you cannot nest <cfoutput> tags without a group attribute. Your code is already in a <cfoutput> so no need for another.
If you are on ColdFusion 10 I would definitely give #Adam's answer a look.

MySQL: showing totals

I am trying to figure out how to have PHP check and print 2 different functions.
Both of these questions are referring to table called "remix".
The first, and more important problem at the minute, is I would like to know how to show how many DIFFERENT values are under "author", as to compile the amount of total authors registered. I need to know not only how to most efficiently use COUNT on returning UNIQUE names under "author", but how to show it inline with the total number of rows, which are currently numbered.
The second question would be asking how I would be able to set up a top 3 artists, based on how many times their name occurs in a list. This also would show on the same page as the above code.
Here is my current code:
require 'remix/archive/connect.php';
mysql_select_db($remix);
$recentsong = mysql_query("SELECT ID,song,author,filename FROM remix ORDER by ID desc limit 1;");
$row = mysql_fetch_array($recentsong);
echo'
<TABLE BORDER=1><TR><TD WIDTH=500>
Currently '.$row['ID'].' Remixes by **(want total artists here)** artists.<BR>
Most recent song: <A HREF=remix/archive/'.$row['filename'].'>'.$row['song'].'</A> by <FONT COLOR=white>'.$row['author'].'</FONT>
So as you can see, I have it currently set up to show the most recent song (not the most efficient way), but want the other things in there, such as at least the top contributor, but don't know if I would be able to put it all in one php block, break it, or be able to do it all within one quarry call, with the right code.
Thanks for any help!
I'm not sure I really understood everything in your question but we'll work this through together :p
I've created an SQLFiddle to work on some test data: http://sqlfiddle.com/#!2/9b613/1/0.
Note the INDEX on the author field, it will assure good performance :)
In order to know how to show how many DIFFERENT values are under "author" you can use:
SELECT COUNT(DISTINCT author) as TOTAL_AUTHORS
FROM remix;
In order to know the total number of rows, which are currently numbered you can use:
SELECT COUNT(*) as TOTAL_SONGS
FROM remix;
And you can combine both in a single query:
SELECT
COUNT(DISTINCT author) as TOTAL_AUTHORS,
COUNT(*) as TOTAL_SONGS
FROM remix;
To the top 3 subject now. This query will give you the 3 authors with the greatest number of songs, first one on top:
SELECT
author,
COUNT(*) as AUTHOR_SONGS
FROM remix
GROUP BY author
ORDER BY AUTHOR_SONGS DESC
LIMIT 3;
Let me know if this answer is incomplete and have fun with SQL !
Edit1: Well, just rewrite your PHP code in:
(...)
$recentsong = mysql_query("SELECT COUNT(DISTINCT author) as TOTAL_AUTHORS, COUNT(*) as TOTAL_SONGS FROM remix;");
$row = mysql_fetch_array($recentsong);
(...)
Currently '.$row['TOTAL_SONGS'].' Remixes by '.$row['TOTAL_AUTHORS'].' artists.<BR>
(...)
For the top3 part, use another mysql_query and create your table on the fly :)

Is this a case for denormalisation?

I have a site with about 30,000 members to which I'm adding a functionality that involves sending a random message from a pool of 40 possible messages. Members can never receive the same message twice.
One table contains the 40 messages and another table maps the many-to-many relationship between messages and members.
A cron script runs daily, selects a member from the 30,000, selects a message from the 40 and then checks to see if this message has been sent to this user before. If not, it sends the message. If yes, it runs the query again until it finds a message that has not yet been received by this member.
What I'm worried about now is that this m-m table will become very big: at 30,000 members and 40 messages we already have 1.2 million rows through which we have to search to find a message that has not yet been sent.
Is this a case for denormalisation? In the members table I could add 40 columns (message_1, message_2 ... message_40) in which a 1 flag is added each time a message is sent. If I'm not mistaken, this would make the queries in the cron script run much faster
?
I know that doesn't answer your original question, but wouldn't it be way faster if you selected all the messages that weren't yet sent to a user and then select one of those randomly?
See this pseudo-mysql here:
SELECT
CONCAT_WS(',', messages.ids) unsent_messages,
user.id user
FROM
messages,
user
WHERE
messages.id NOT IN (
SELECT
id
FROM
sent_messages
WHERE
user.id = sent_messages.user
)
GROUP BY ids
You could also append the id of the sent messages to a varchar-field in the members-table.
Despite of good manners, this would make it easily possible to use one statement to get a message that has not been sent yet for a specific member.
Just like this (if you surround the ids with '-')
SELECT message.id
FROM member, message
WHERE member.id = 2321
AND member.sentmessages NOT LIKE '%-' && id && '-%'
1.2 M rows # 8 bytes (+ overhead) per row is not a lot. It's so small I wouldn't even bet it needs indexing (but of course you should do it).
Normalization reduces redundancy and it is what you'll do if you have large amount of data which seems to be your case. You need not denormalize. Let there be an M-to-M table between members and messages.
You can archive the old data as your M-to-M data increases. I don't even see any conflicts because your cron job runs daily for this task and accounts only for the data for the current day. So you can archive M-to-M table data every week.
I believe there will be maintenance issue if you denormalize by adding additional coloumns to members table. I don't recommend the same. Archiving of old data can save you from trouble.
You could store only available (unsent) messages. This implies extra maintenance when you add or remove members or message types (nothing that can't be automated with foreign keys and triggers) but simplifies delivery: pick a random line from each user, send the message and remove the line. Also, your database will get smaller as messages get sent ;-)
You can achieve the effect of sending random messages by preallocating the random string in your m-m table and a pointer to the offset of the last message sent.
In more detail, create a table MemberMessages with columns
memberId,
messageIdList char(80) or varchar ,
lastMessage int,
primary key is memberId.
Pseudo-code for the cron job then looks like this...
ONE. Select next message for a member. If no row exists in MemberMessages for this member, go to step TWO. The sql to select next message looks like
select substr(messageIdList, 2*lastMessage + 1, 2) as nextMessageId
from MemberMessages
where member_id = ?
send the message identified by nextMessageId
then update lastMessage incrementing by 1, unless you have reached 39 in which case reset it to zero.
update MemberMessages
set lastMessage = MOD(lastMessage + 1, 40)
where member_id = ?
TWO. Create a random list of messageIds as a String of couplets like 2117390740... This is your random list of message IDs as an 80 char String. Insert a row to MemberMessages for your member_id setting message_id_list to your 80 char String and set last_message to 1.
Send the message identified by the first couplet from the list to the member.
You can create a kind of queue / heap.
ReceivedMessages
UserId
MessageId
then:
Pick up a member and select message to send:
SELECT * FROM Messages WHERE MessageId NOT IN (SELECT MessageId FROM ReceivedMessages WHERE UserId = #UserId) LIMIT 1
then insert MessageId and UserId to ReceivedMessages
and do send logic here
I hope that helps.
There are potential easier ways to do this, depending on how random you want "random" to be.
Consider that at the beginning of the day you shuffle an array A, [0..39] which describes the order of the messages to be sent to users today.
Also, consider that you have at most 40 Cron jobs, which are used to send messages to the users. Given the Nth cron job, and ID the selected user ID, numeric, you can choose M, the index of the message to send:
M = (A[N] + ID) % 40.
This way, a given ID would not receive the same message twice in the same day (because A[N] would be different), and two randomly selected users have a 1/40 chance of receiving the same message. If you want more "randomness" you can potentially use multiple arrays.