I'm trying to use a database as the back-end for a messaging system in my game (sort of like instant messaging). I am using a local database to store received messages and a database on my server to send them. Here are the tables that I am using:
Users:
userName (varchar)
displayName (varchar)
currentGames (varchar)
Messages:
sender (varchar)
receiver (varchar)
message (varchar)
timestamp (int)
My plan is that when a user sends a message, I first store the message in their local database and then send the message off to the server.
When a user checks to see if there are any new messages (polling), he first gets the latest timestamp from his local database and uses this time to query the online database for all messages sent after that time. All messages received are then deleted from the database.
Is there something wrong with the way I'm doing this? I'm trying to prepare for the worst, and I have no idea how this sort of plan will scale. I'm not using a unique id for the "Users" table and I feel that I should. Since my database experience is limited I don't fully understand the significance of the unique auto-increment id or how it would help me here. Any advice/criticism would be appreciated.
Since most gamer tags are transient and you probably want to differentiate between a gamer's ID (private) and their user name (public, at least to friends) then you want a local design like this:
FRIEND -- The local user's own tag should go in here too.
( user_id
, current_gamer_tag
, last_update_timestamp
)
GAME
( game_id
, game_name -- No timestamp here because the name doesn't change?
)
MESSAGE
( message_id -- If you make this a GUID you can use it for synching with the server.
, sending_user_id -- FK to FRIEND
, receiving_user_id -- Also FK to FRIEND
, timestamp
, content
)
This holds both outgoing and incoming messages locally and allows the message display to focus on gamer tags while at the same time being easy to synchronize with the server. By the way, you might also consider changing the receiving_user_id to a sub-table containing a list of recipients if you have three or more-way game play.
Using unique IDs is important for a lot of reasons. The most important is that it allows you to modify your gamer tags and prevents you from having to reveal your players' user IDs in the message displays. There is also a space saving here because an integer, even a bigint is smaller than a gamer tag. This is better for scalability. Using a GUID instead of an increasing integer for the message ID means that you won't have an "insert hot-spot" on your server's message table, which will perform better as long as your server message table has adequate free space built into it. Also, the message IDs can be generated at the client end and you can be pretty confident that there won't be any key collisions when the messages hit the server.
Here's my look on it. This is just how I would do it....
It all depends how your system works. If allow every username to be used once and not allow the user to change it, then logically you won't need to use an id. Personally, I use a auto_incremented id for my users database. If I were you, I would use a ID, it would simplify everything.
Probally on the local database you will need to have something like this:
Friend's database:
USER_ID USERNAME
Game's database:
GAME_ID USER_PLAYING_ID
Messages database:
TO_USER TIMESTAMP GAME_ID
So when submitting to the online database you will have the user's id to send to as well as the game information.
Again, just how I would do it. Other than everything else, it seems like you know what you are doing.
Related
I am working on a little package using PHP and MySQL to handle entries for events. After completing an entry form the user will see all his details on a page called something like website.com/entrycomplete.php?entry_id=15 where the entry_id is a sequential number. Obviously it will be laughably easy for a nosey person to change the entry_id number and look at other people's entries.
Is there a simple way of camouflaging the entry_id? Obviously I'm not looking to secure the Bank of England so something simple and easy will do the job. I thought of using MD5 but that produces quite a long string so perhaps there is something better.
Security through obscurity is no security at all.
Even if the id's are random, that doesn't prevent a user from requesting a few thousand random id's until they find one that matches an entry that exists in your database.
Instead, you need to secure the access privileges of users, and disallow them from viewing data they shouldn't be allowed to view.
Then it won't matter if the id's are sequential.
If the users do have some form of authentication/login, use that to determine if they are allowed to see a particular entry id.
If not, instead of using a url parameter for the id, store it in and read it from a cookie. And be aware that this is still not secure. An additional step you could take (short of requiring user authentication) is to cryptographically sign the cookie.
A better way to implement this is to show only the records that belong to that user. Say the id is the unique identifier for each user. Now store both entry_id and id in your table (say table name is entries).
Now when the user requests for record, add another condition in the mysql query like this
select * from entries where entry_id=5 and id=30;
So if entry_id 5 does not belong to this user, it will not have any result at all.
Coming towards restricting the user to not change his own id, you can implement jwt tokens. You can give a token on login and add it to every call. You can then decrypt the token in the back end and get the user's actual id out of it.
I create a VueJS application with express and sequelize to access a mysql database (currently running with XAMPP). I have a database which consist of 3 tables:
users: [id (primary key), name, email, family_id (foreign key)]
families: [id (primary key), name]
hobbies: [id (primary key), name, user_id (foreign key)]
All of these IDs are auto_increment so the first user registered gets the ID 1 and so on.
Every user within the same family (so with equal family_id) is allowed to see the hobbies of the other family members. I have a SQL query, which gives me all the family members. On my websity I have a simple drop down menu, where I can select the member. With a GET request I then want to retrieve all hobbies of the selected member.
Now I can basically decide if I use the id or the email for the request parameter e.g. /api/hobbies/:id or /api/hobbies/:email. Email reveals more private information while id reveals information about my internal strucutre like "At least (id) number of users exists.". I think it is better to use the id.
Maybe there is also the possibility to assign a random id (not auto increment) in the database? But I dont know how to to this.
Nothing you send as a parameter to a GET request is private. Those parameters are part of the URL you GET, and those URLs can be logged in various proxy servers, etc, all over the internet without your consent or your users' consent.
It seem to me that family members' hobbies can be sensitive data. What if the whole family likes, say, golf? A cybercreep could easily figure out that a good time for burglary would be Saturday afternoons.
And if your app does GET operations with autoincrementing id values, it's child's play for a cybercreep to examine any record they want. Check out the Panera Bread data breach for example. https://krebsonsecurity.com/2018/04/panerabread-com-leaks-millions-of-customer-records/
At a minimum use POST for that kind of data.
Better yet, use a good authentication / session token system on your app, and conceal data from users if they're not members of that family.
And, if you want to use REST style GET parameters, you need to do these things to be safe:
Use randomized id values. It must be very difficult for a cybercreep to guess a second value from knowing a first value. Serial numbers won't do. Neither will email addresses.
Make sure unauthenticated users can see no data.
Make sure authenticated users can only see the subset of data for which they're authorized.
My suggestion to avoid REST-style GET parameters comes from multiple security auditors saying, you must change that.
I am using PHP , MySQL for Handling My site. I have a user table called user_table containing user_id PRIMARY and Now user can chat with other user. For that I am using a table called message_table. this table contains essential fields like msg_id , sender_id , reciver_id , replay_id etc. this works very fine.But I need to handle also guest users[Not registered] with this chat. how can I handle them. I inserted the guset_id as the mixture of entering time and his name to a table called guest_table. Now I can use this guest_table like user_table. Is that a good idea ?
But I need to handle also guest users[Not registered] with this chat.
These guests look to me like a special sort of user. I suggest you handle them as if they were users. You may wish to put a column called something like is_guest in your user table, and set it to 1 when you know the user is a guest.
Keeping a separate table, parallel to the user table, for your guests is NOT a good idea. It will greatly complicate your queries, especially when a person in your user table communicates with a person in your guest table.
Instead, put everybody in the user table, guests and registered users alike.
A nice little benefit: if you convert a guest to a registered user, all you have to do is flip the is_guest flag and let them specify their own username and other data. They get to keep their history.
Assuming I want to have a web application that requires storing user information, images, etc as well as storing status updates or posts/comments would I want to separate tables?
For example if I have a "users" table that contains users information like passwords, emails, and typical social networking info like age, location etc. Would it be a good idea do create a second table("posts") that handles user content such as comments and/or post?
Table one: "users"
UserID
Username
Age
etc.
Table Two: "posts"
PostID
PostContent
PostAuthor
PostDate
etc
Is this a valid organization? Furthermore if I wanted to keep track of media should I do this in ANOTHER table?
Table Three: "media"
ID
Type
Uploader
etc.
Any help is much appreciated. I'm curious to see if I'm on the right track or just completely lost. I am mostly wondering if I should have many tables or if I should have larger less segregated tables.
Also of note thus far I planned on keeping information such as followers(or friends) in the 'users' table but I'm not sure that's a good idea in retrospect.
thanks in advance,
Generally speaking to design a database you create a table for each object you will be dealing with. In you example you have Users, Posts, Comments and Media. From that you can flesh out what it is you want to store for each object. Each item you want to store is a field in the table:
[Users]
ID
Username
PasswordHash
Age
Birthdate
Email
JoinDate
LastLogin
[Posts]
ID
UserID
Title
Content
CreateDate
PostedDate
[Comments]
ID
PostID
UserID
Content
[Media]
ID
Title
Description
FileURI
Taking a look above you can see a basic structure for holding the information for each object. By the field names you can even tell the relationships between the objects. That is a post has a UserID so the post was created by that user. the comments have a PostID and a UserID so you can see that a comment was written by a person for a specific post.
Once you have the general fields identified you can look at some other aspects of the design. For example right now the Email field under the Users table means that a user can have one (1) email address, no more. You can solve this one of two ways... add more email fields (EmailA, EmailB, EmailC) this generally works if you know there are specific types of emails you are dealing with, for example EmailWork or EmailHome. This doesn't work if you do not know how many emails in total there will be. To solve this you can pull the emails out into its own table:
[Users]
ID
Username
PasswordHash
Age
Birthdate
JoinDate
LastLogin
[Emails]
ID
UserID
Email
Now you can have any number of emails for a single user. You can do this for just about any database you are trying to design. Take it in small steps and break your bigger objects into smaller ones as needed.
Update
To deal with friends you should think about the relationship you are dealing with. There is one (1) person with many friends. In relation to the tables above its one User to many Users. This can be done with a special table that hold no information other than the relationship you are looking for.
[Friends]
[UserA]
[UserB]
So if the current user's ID is in A his friend's ID is in B and visa-verse. This sets up the friendship so that if you are my friend, then I am your friend. There is no way for me to be your friend without you being mine. If you want to setup the ability for one way friendships you can setup the table like this:
[Friends]
[UserID]
[FriendID]
So If we are both friends with each other there would have to be 2 records, one for my friendship to you and one for your freindship to me.
You need to use multiple tables.
The amount of tables depends on how complex you want your interactive site to be. Based on what you have posted you would need a table that would store information about the users, a table for comments, and more such as a table to store status types.
For example tbl_Users should store:
1. UserID
2. First Name
3. Last name
4. Email
5. Password (encrypted)
6. Address
7. City
8. State
9. Country
10. Date of Birth
11. UserStatus
12. Etc
This project sounds like it should be using a relational DB that will pull up records, such as comments, by relative userIDs.
This means that you will need a table that stores the following:
1. CommentID (primary key, int, auto-increment)
2. Comment (text)
3. UserID (foreign key, int)
The comment is attached to a user through a foreign key, which is essentially the userId from the tbl_Users table. You would need to combine these tables in an SQL statement with your script to query the information as a single piece of information. See example code
$sql_userWall = "SELECT tbl_Users.*, tbl_Comments.*, tbl_userStatus FROM tbl_Users
INNER JOIN tbl_Comments ON tbl_Users.userID = tbl_Comments.userID
INNER JOIN tbl_UserStatus ON tbl_Users.userID = tbl.UserStatus
WHERE tbl_Users.userID = $userID";
This statement essentially says get the information of the provided user from the users table and also get all the comments with that has the same userID attached to it, and get the userStatus from the table of user status'.
Therefore you would need a table called tbl_userStatus that held unique statusIDs (primary key, int, auto-incrementing) along with a text (varchar) of a determined length that may say for example "online" or "offline". When you started the write the info out from e record using php, asp or a similar language the table will automatically retrieve the information from tbl_userStatus for you just by using a simple line like
<?php echo $_REQUEST['userStatus']; ?>
No extra work necessary. Most of your project time will be spent developing the DB structure and writing SQL statements that correctly retrieve the info you want for each page.
There are many great YouTube video series that describe relational DBS and drawing entity relational diagrams. This is what you should look into for learning more on creating the tye of project you were describing.
One last note, if you wanted comments to be visible for all members of a group this would describe what is known as a many-to-many relationship which would require additional tables to allow for multiple users to 'own' a relationship to a single table. You could store a single groupID that referred to a table of groups.
tbl_groups
1. GroupID
2. GroupName
3. More group info, etc
And a table of users registered for the group
Tbl_groupMembers
1. membershipCountID (primary key, int, auto-increment)
2. GroupID (foriegn key, int)
3. UserID (foriegn key, int)
This allows users to registrar for a group and inner join them to group based comments. These relationships take a little more time to understand, the videos will help greatly.
I hope this helps, I'll come back and post some YouTube links later that I found helpful learning this stuff.
I am just trying to figure out how Facebook's database is structured for tracking notifications.
I won't go much into complexity like Facebook is. If we imagine a simple table structure for notificaitons:
notifications (id, userid, update, time);
We can get the notifications of friends using:
SELECT `userid`, `update`, `time`
FROM `notifications`
WHERE `userid` IN
(... query for getting friends...)
However, what should be the table structure to check out which notifications have been read and which haven't?
I dont know if this is the best way to do this, but since I got no ideas from anyone else, this is what I would be doing. I hope this answer might help others as well.
We have 2 tables
notification
-----------------
id (pk)
userid
notification_type (for complexity like notifications for pictures, videos, apps etc.)
notification
time
notificationsRead
--------------------
id (pk) (i dont think this field is required, anyways)
lasttime_read
userid
The idea is to select notifications from notifications table and join the notificationsRead table and check the last read notification and rows with ID > notificationid. And each time the notifications page is opened update the row from notificationsRead table.
The query for unread notifications I guess would be like this..
SELECT `userid`, `notification`, `time` from `notifications` `notificationsRead`
WHERE
`notifications`.`userid` IN ( ... query to get a list of friends ...)
AND
(`notifications`.`time` > (
SELECT `notificationsRead`.`lasttime_read` FROM `notificationsRead`
WHERE `notificationsRead`.`userid` = ...$userid...
))
The query above is not checked.
Thanks to the idea of db design from #espais
You could add another table...
tblUserNotificationStatus
-------------------------
- id (pk)
- notification_id
- user_id
- read_status (boolean)
If you wanted to keep a history, you could keep the X latest notifications and delete the rest that are older than your last notification in the list....
If, when you give notifications, you give all relevant notifications available at that time, you can make this simpler by attaching timestamps to notifiable events, and keeping track of when each user last received notifications. If you are in a multi-server environment, though, you do have to be careful about synchronization. Note that this approach doesn't require true date-time stamps, just something that increases monotonically.
I see no-one here addresses the fact, that notifications are usually re-occurring, aka. notification of an upcoming transaction is always going to be the same, but with a different transaction ID or Date in it. as so: { You have a new upcoming payment: #paymentID, with a due date of #dueDate }.
Having texts in a different table can also help with
If you want to change the notification text later on
Making the app multilingual is easier, because I can just layer the notifications table with a language code and retrieve the appropriate string
Thus I also made a table for those abstract notifications, which are just linked under the the user with a middle table, where one notification type can be sent to one user at multiple times. I also linked the notifications to the user not by a foreign key ID, but I made notification codes for all notifications and full_text indexed the varchar field of those codes, for faster read speeds. Due to the fact that these notifications need to be sent at specific times, it is also easier for the developer to write
NotificationService::sendNew( Notification::NOTE_NEW_PAYMENT, ['paymentId'] => 123, ['dueDate'] => Carbon::now(), 'userIdToSendTo' );
Now since my messages are going to have custom data in them, that is inserted into the string, as you can see from the second argument beforehand, then I will store them in a database blob. as such
$values = base64_encode(serialize($valuesInTextArray));
This is because I want to decouple the notifications from other tables and as such I dont want to crete unnessecary FK relations from and to the notifications table, so that I can for example say notification 234 is attached to transaction 23 and then join and get that transaction ID. Decoupling this takes away the overhead of managing these relations. The downside is, it is nigh impossible to delete notifications, when for example a transaction is deleted, but in my use case I decided, this is not needed anyway.
I will retrieve and fill the texts on the App side as follows. Ps. I am using someones vksprintf function (https://github.com/washingtonpost/datawrapper/blob/master/lib/utils/vksprintf.php), props to him!
$valuesToFillInString = unserialize(base64_decode($notification->values));
vksprintf( $notificationText->text, $valuesToFillInString )
Notice also which fields I index, because I am going to find or sort by them
My Database design is as follows
==============================
TABLE: Users
id (pk)
==============================
TABLE: Notifications
id (pk)
user_id (fk, indexed)
text_id (fk - NotificationTexts table)
values (blob) [containing the array of values, to input into the text string]
createdDateTime (DateTime)
read (boolean)
[ClusterIndex] => (user_id, createdDateTime)
==============================
TABLE: NotificationTexts
id (pk)
text_id (uniquem indexed)
text (varchar) [{ You have a new upcoming payment: #paymentID, with a due date of #dueDate }]
note (varchar, nullable) [notes for developers, informational column]
I am also trying to figure out how to design a notification system. Regarding notification status (read, unread, deleted, archived, ect) I think that it would be good a good candidate to for ENUM. I think it is possible that there will be more than two different types of status other than READ and UNREAD such as deleted, archived, seen, dismissed, ect.
That will allow you to expand as your needs evolve.
Also I think it may make sense (at least in my case) to have a field to store an action url or a link. Some notifications could require or prompt the user to follow a link.
It also may make sense to have a notification type as well if you want different types. I am thinking there could be system notifications (such as a verify email notification) and user prompted notifications (such as a friend request).
Here is the structure I think would be a minimum to have a decent notification system.
users
-------------
id
username
password
email
notifications
-------------
id
user_id (fk)
notification_type (enum)
notification_status (enum)
notification_action (link)
notification_text
date_created (timestamp)
Table are following
User
userId (Integer)
fullName(VarChar)
Notification
notificationId (Integer)
creationDate (Date)
notificationDetailUrl (VarChar)
isRead (bollean)
description (VarChar)
userId (F.K)