I'm working on a email project. I would like to display email in threads just like gmail.
What is the best approach to display mails in thread?
I have checked jwz threading algorithm. But looks like that algorithm is written for projects that has no databases.That algorithm focuses on these three header keys. Message-ID, In-Reply-To and References
Can someone tell me what is the proper, efficient and most accurate way to achieve threading using mysql database.?
Do I have to use separate table for threads and references?
If possible give me some sample mysql queries. So I can understand better.
Thank you.
Any message board design eg wordpress etc should also works for email. I also come up with a design:
email.id user_id subject status folder created updated ...other info you'd save
1 123 Hello New Inbox Y-m-d.. Y-m-d.. ...
2 3456 World Replied Inbox ...
reply.id email_id reply_to_id user_id created email_txt ip ...
10 1 0 890 Y-m-d.. Hi ...
20 2 0 5678 ...
30 2 20 3456 ... Replyto 2
55 2 30 5678 .... Replyto 3
So in your email.folder.index page:
SELECT * FROM email WHERE user_id = 12345 ORDER BY updated desc LIMIT 50
And when you click one an email on index page, goto email details page:
SELECT * FROM reply WHERE email_id = 2 ORDER BY created, reply_to_id
The key magic is that the tree is built based on reply_to_id
Related
I have library-like system. In one table bunch of books in different copies and other table for reservations.
Books table (for simplisity only three columns)
id
title
inventory_number
1
Coding is awesome
1111
2
Coding is awesome
2222
3
Coding is awesome
3333
4
Get rich quick
4444
5
Get rich quick
5555
6
YOLO
6666
Reservation table (for simplisity only three columns)
id
book_id
user_id
1
1
1
2
1
2
3
2
1
4
4
1
5
5
2
Although in Books table is 6 books, to end user it means that there is only 3 books - "Coding is awesome", "Get rich quick" and "YOLO" and he don't need to know nothing about other book copies in stock.
When user comes to "Library/Store" and asks for some book than he should get the first book with minimum reservations for current book.
From given data above can be understood that:
"Coding is awesome" (ID:1) is reserved 2 times
"Coding is awesome" (ID:2) is reserved 1 times
"Coding is awesome" (ID:3) is reserved 0 times
So when user asks for "Coding is awesome" book he needs to get book with ID:3
And when he asks for "Get ritch quick" he gets one of two books because all two books is reserved equal number of times.
I need a solution for "end user" that will return only books with different (unique) title and other to current book related data
Although in Books table is 6 books, In this case need to return three books:
id
title
inventory_number
3
Coding is awesome
3333
4
Get rich quick
4444
6
YOLO
6666
If there will be more books with diferent names then they also need to be shown. (for each unique title one record)
If it is possible then I need to return these data from database using Eloquant or Query Builder not filter out with Collections to save resources and use pagging.
Using Laravel 8, Mysql 5.7
Assuming you have a reservations relationship set up in the Book model, using withCount, you can get a reservations_count for all books.
Book::query()
->select('id', 'title', 'inventory_number')
->withCount('reservations')
->get();
This will give you a result set like
id
title
inventory_number
reservations_count
1
Coding is awesome
1111
2
2
Coding is awesome
2222
1
3
Coding is awesome
3333
0
4
Get rich quick
4444
0
5
Get rich quick
5555
0
6
YOLO
6666
0
Let's call this query 1. You could make something like
SELECT *
FROM (
-- query 1
) books
GROUP BY id, title, inventory_number
ORDER BY MAX(reservations_count) DESC
to get only one relevant row.
In theory, this should work.
$query_1 = Book::query()
->select('id', 'title', 'inventory_number')
->withCount('reservations');
$results = Book::query()
->fromSub($query_1, 'books')
->groupByRaw('id, title, inventory_number')
->orderByRaw('reservations_count ASC')
->limit(3)
->get();
I found something similar at this other question
Another possibility is to use Collections to do the hard part.
$results = Book::query()
->withCount('reservations')
->get() // replace with cursor() if you're not using `with()` in the query
->sortBy('reservations_count')
->unique('title')
->take(3); // or ->random(3)
Okay, let me explain.
You know how Facebook Messenger and Discord both have the last messaged friends list? It is a list of all of your friends order by whoever texted you last. Now, if you had over 100 people in that list, it would be better to send the list to the client in chunks of 10. Once the client reaches the bottom, it asks for the next 10. This can be done with the LIMIT offset, amount.
But now, the problem. The user might open the messenger, talk to someone for 10 minutes, and then scroll further down in the last messaged friends list. In this case, the table has changed before the user has retrieved the full list. The list in database is now in a different order because someone messaged them in the meanwhile and is now on top of the list, but the client already has the first chunk of the list, but this doesn't contain the people that texted them.
In case my explanation was not enough, here's a visual demonstration:
Visual demo
'Last messaged friends' list in database (ordered by latest timestamp):
Person 23
Person 77
Person 93
Person 99
Person 67
Person 85
User connects, asks for the first 3 entries.
Client now has (ordered by latest timestamp):
Person 23
Person 77
Person 93
'Person 99' messages that user. 'Person 99' is now on top of the list.
'Last messaged friends' list in database (ordered by latest timestamp):
Person 99
Person 23
Person 77
Person 93
Person 67
Person 85
User scrolls down. Client asks for the next 3 entries.
Client now has (ordered by latest timestamp):
Person 23
Person 77
Person 93
Person 93 (duplicate)
Person 67
Person 85
('Person 99' missing)
Is this something I could fix/implement with a more advanced SQL query?
If I can't, how could I implement this in other ways?
For information, I have a Socket.io (a.k.a more advanced WebSocket) connection between the server and the client, I can send whatever necessary information thru that.
It sounds like you want aggregation to avoid duplicates:
select person_id
from messages
group by person_id
order by max(timestamp) desc;
Thank you, Hector Vido, for the suggestion. (he made a comment right below my question, go upvote)
"Selecting messages by timestamp don't solve this? You keep the last timestamp and then ask by anoter 10 registries >= that timestamp"
Solution
I'll keep the oldest and newest timestamp in the client side.
If the client scrolls down, I will request 10 entries before the oldestTimestamp and then the new oldestTimestamp will be the oldest timestamp of the received entries.
Also after every 10 seconds, I could request for entries after the newestTimestamp and then the new newestTimestamp will be the newest timestamp of the received entries.
I'm building a pretty simple chat app that allows both 1 on 1 message and chat rooms for groups of people. I'm planning to have one Message table to store all chat messages, each message will also keep the sender ID and receiver ID, in the case of messages sent in a chat room, we also keep the ID of that chat room. Below is the table:
Message Table
ID Message Sender Receiver Chatroom Timestamp
1 Hello, David 123 321 1495330074
2 Hi, Linda 321 123 1495930032
3 Hi everyone! 456 999 1495930132
4 What up? 321 123 1495930192
...
Then if I'm user 321, and I want to retrieve my conversation with user 123, I just need to SELECT * FROM Message WHERE Sender=123 or Receiver=123 or Sender=321 or Receiver=321 and Chatroom IS NULL
There is one issue with this design - a user can't delete a message that he doesn't want to see any more.
To solve that, I think I can have a separate table to store what messages a user received or sent, like below:
User Message Table
ID UserID MessageID
1 123 1
2 321 1
3 321 4
...
It seems a little redundant, but this way David can delete a message in his conversation with Linda, while Linda can still see full conversation history.
Is there better design of the tables? And is this good practice to throw all chat messages in one giant table? Should I add some index to make query faster?
You can use this query.
delete from Message
where sender = 123 and user = 321
But this will delete all the chat messages between this user and sender. To delete specific message you can use ID
delete from Message
where ID = 1
QUESTION: Is it okay to have "shortcut" identifiers in a table so that I don't have to do a long string of joins to get the information I need?
To understand what I'm talking about, I'm going to have to lay ouf an example here that looks pretty complicated but I've simplified the problem quite a bit here, and it should be easily understood (I hope).
The basic setup: A "company" can be an "affiliate", a "client" or both. Each "company" can have multiple "contacts", some of which can be "users" with log in privileges.
`Company` table
----------------------------------------------
ID Company_Name Address
-- ----------------------- -----------------
1 Acme, Inc. 101 Sierra Vista
2 Spacely Space Sprockets East Mars Colony
3 Cogswell Cogs West Mars Colony
4 Stark Industries Los Angeles, CA
We have four companies in our database.
`Affiliates` table
---------------------
ID Company_ID Price Sales
-- ---------- ----- -----
1 1 50 456
2 4 50 222
3 1 75 14
Each company can have multiple affiliate id's so that they can represent the products at different pricing levels to different markets.
Two of our companies are affiliates (Acme, Inc. and Stark Industries), and Acme has two affiliate ID's
`Clients` table
--------------------------------------
ID Company_ID Referring_affiliate_id
-- ---------- ----------------------
1 2 1
2 3 1
3 4 3
Each company can only be a client once.
Three of our companies are clients (Spacely Space Sprockets, Cogswell Cogs, and Stark Industries, who is also an affiliate)
In all three cases, they were referred to us by Acme, Inc., using one of their two affiliate ID's
`Contacts` table
-----------------------------------------
ID Name Email
-- -------------- ---------------------
1 Wylie Coyote wcoyote#acme.com
2 Cosmo Spacely boss#spacely.com
3 H. G. Cogswell ceo#cogs.com
4 Tony Stark tony#stark.com
5 Homer Simpson simpson#burnscorp.com
Each company has at least one contact, but in this table, there is no indication of which company each contact works for, and there's also an extra contact (#5). We'll get to that in a moment.
Each of these contacts may or may not have a login account on the system.
`Contacts_type` table
--------------------------------------
contact_id company_id contact_type
---------- ---------- --------------
1 1 Administrative
2 2 Administrative
3 3 Administrative
4 4 Administrative
5 1 Technical
4 2 Technical
Associates a contact with one or more companies.
Each contact is associated with a company, and in addition, contact 5 (Homer Simpson) is a technical contact for Acme, Inc, and contact 4 (Tony Stark) is a both an administrative contact for company 4 (Stark Industries) and a technical contact for company 3 (Cogswell Cogs)
`Users` table
-------------------------------------------------------------------------------------
ID contact_id company_id client_id affiliate_id user_id password access_level
-- ---------- ---------- --------- ------------ -------- -------- ------------
1 1 1 1 1 wylie A03BA951 2
2 2 2 2 NULL cosmo BF16DA77 3
3 3 3 3 NULL cogswell 39F56ACD 3
4 4 4 4 2 ironman DFA9301A 2
The users table is essentially a list of contacts that are allowed to login to the system.
Zero or one user per contact; one contact per user.
Contact 1 (Wylie Coyote) works for company 1 (Acme) and is a customer (1) and also an affiliate (1)
Contact 2 (Cosmo Spacely) works for company 2 (Spacely Space Sprockets) and is a customer (2) but not an affiliate
etc...
NOW finally onto the problem, if there is one...
Do I have a circular reference via the client_id and affiliate_id columns in the Users table? Is this a bad thing? I'm having a hard time wrapping my head around this.
When someone logs in, it checks their credentials against the users table and uses users.contact_id, users.client_id, and users.affiliate_id to do a quick look up rather than having to join together a string of tables to find out the same information. But this causes duplication of data.
Without client_id in the users table, I would have to find the following information out like this:
affiliate_id: join `users`.`contact_id` to `contacts_types`.`company_id` to `affiliates`.`company_id`
client_id: join `users`.`contact_id` to `contacts_types`.`company_id` to `clients`.`company_id`
company_id: join `users`.`contact_id` to `contacts_types`.`company_id` to `company`.`company_id`
user's name: join `users`.`contact_id` to `contacts_types`.`contact_id` to `contacts`.`contact_id` > `name`
In each case, I wouldn't necessarily know if the user even has an entry in the affiliate table or the clients table, because they likely have an entry in only one of those tables and not both.
Is it better to do these kinds of joins and thread through multiple tables to get the information I want, or is it better to have a "shortcut" field to get me the information I want?
I have a feeling that over all, this is overly complicated in some way, but I don't see how.
I'm using MySQL.
it's better to do the joins. you should only be denormalizing your data when you have timed evidence of a slow response.
having said that, there are various ways to reduce the amount of typing:
use "as" to give shorter names to your fields
create views. these are "virtual tables" that already have your standard joins built-in, so that you don't have to repeat that stuff every time.
use "with" in sql. this lets you define something like a view within a single query.
it's possible mysql doesn't support all the above - you'll need to check the docs [update: ok, recent mysql seems to support views, but not "with". so you can add views to do the work of affiliate_id, client_id etc and treat them just like tables in your queries, but keeping the underlying data nicely organised.]
One of my coworkers is working on a SQL query. After several joins (workers to accounts to tasks), she's got some information sort of like this:
Worker Account Date Task_completed
Bob Smith 12345 01/01/2010 Received
Bob Smith 12345 01/01/2010 Received
Bob Smith 12345 01/01/2010 Processed
Sue Jones 23456 01/01/2010 Received
...
Ultimately what she wants is something like this - for each date, for each account, how many tasks did each worker complete for that account?
Worker Account Date Received_count Processed_count
Bob Smith 12345 01/01/2010 2 1
... and there are several other statuses to count.
Getting one of these counts is pretty easy:
SELECT
COUNT(Task_completed)
FROM
(the subselect)
WHERE
Task_completed = 'Received'
GROUP BY
worker, account, date
But I'm not sure the best way to get them all. Essentially we want multiple COUNTs using different GROUP BYs. The best thing I can figure out is to copy and paste the subquery several times, change the WHERE to "Processed", etc, and join all those together, selecting just the count from each one.
Is there a more obvious way to do this?
SELECT worker, account, date,
SUM(task_completed = 'Received') AS received_count,
SUM(task_completed = 'Processed') AS processed_count
FROM mytable
GROUP BY
worker, account, date