I'm making an Online game where I will host a game server. Players will login to my game server. They will then be taken to a lobby where they can choose a game to join. I will be keeping track of wins and loses and a few other statistics.
My requirements are as follows:
At any time in game, a player should be able to click on another
player and get their latest up-to-date statistics.
A player should also be able to go to my Web Site and get the same
statistics. (Ideally, up to date immediately, but less important than
in game)
I will also have a leader-board that will be generated from data on
the Web Site.
My question is: What type of solution would typically be used for this type of situation?
It is vital that I never lose data. One thing that worries me about using a Web Site database is data loss.
I'm also unsure how the interactions between the Web Site database and the game server would work. Is there a capability with mySQL to do this sort of thing? My other concern with using a Web Site database is how much bandwidth I would consume monthly. I generously estimate that I will have 1000 people online at any given time. A game lasts around 20 minutes.
How are these types of situations typically solved? I've looked all over but I've yet to find a clear answer to my concerns.
Thanks
I would recommend a few things based on your requirements. Your question is very open ended so the answers given are quite general:
Databases are fine to store data as they write to a harddrive and are transactional (meaning they fine to survive web server crashes).
Databases can be backed up using any one of numerous back up tools, such as: https://www.google.com/search?q=sql+backup&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a
For up to date statistics you should probably be pulling active game players information from a cache (otherwise you might find you are pounding the database when most of your data isnt going to change (ie possibly most gamers could be offline and their data will remain static but might want to be viewed.
Investigate what kind of database you want. NOSQL, or SQL. There is no obvious choice here without evaluating the benefits of each.
Investigate N-Tier or MultiTier design. http://en.wikipedia.org/wiki/Multitier_architecture
Consider some sort of cloud like infrastructure such as appfabric, azure, (there are other linux ones too) etc. There are many cloud services which can provide high scalability. It could be a short cut for the previous points.
Related
I read that StackExchange uses two data centers to house all of their servers, both data centers are in the US. I'm in Ireland so I'm sure US servers are fine for me, but how can StackExchange load quickly for users in Australia if all the database servers are in the US?
I'd just like to ask, does this mean for services like MySQL, being geographically close to the server isn't as big of a deal for keeping page load times fast?
I know they use a CDN to speed up their page load time and they probably cache certain pages to speed things up, but even if I go to some really old, unpopular question I can't notice any slow-down.
The location of the database server relative to the viewer is not the significant performance factor. As a site visitor, you aren't talking to the database -- you're talking to a web application server, which is talking to the database.
Far more important, usually, is the location of the database server relative to the application server, because many applications require multiple queries and thus multiple round trips to the database in order to render a single page, and these round trips increase the time it takes for a page to be rendered. When the database is physically proximate to the application tier, that time becomes negligible.
Speaking in general web terms, in a well-managed site like SE, with all the supporting assets in a CDN, the only delay that is relevant to you is the transit time required for that one big HTTP request/response necessary to render the page content. The transit time is not negligible, because the speed of light is still finite, so round trip times to far-flung locales even on the best routes can easily be in the 200-300ms range... but if you only need to traverse it once, you still have a respectable response time.
A site that uses a lot of ajax to fetch additonal data would not fare so well with the web server so far away. If such design were needed, you'd need geographically distributed web servers, with adjacent database replicas, and geo-routing in DNS to send read-only ajax requests to the nearest web server, which could query its local replica, get a quick response, and return a quick answer.
I once moved a MySQL server -- relative to the app server -- from being ~0.5 ms away to being ~25ms away. The page load time on the site (which was already not optimal) increased from 2 sec to 10 sec. The reason? The app had been through many iterations over the years and made a lot of unnecessary requests to the database... if I remember right, even the simplest page required 13 different queries, most of which were fetching data that wasn't actually used (like fetching your score even for pages that didn't actually display your score). This inefficiency went undetected as long as the app and the db were very, very close. But, again, this was about the distance between the web server and the database, not the database and the browser.
Stack Exchange has two data centers but at last check one of them is only a hot standby/failover site. The main site does all the work under normal operations. And, SE uses MSSQL, but that, too, is immaterial, because the fundamental phenomenon at work here is a law of physics.
Perhaps StackExchange uses several copies of databases (DB Slaves) geographically distributed across different regions of the world. That explains high speed of work even with unpopular SQL-requests.
Also between Australia and West Coast of United States, direct communication via an underwater cable is possible, which ensures a high speed of operation.
I wanted to create a (nearly) live dashboard from MySQL databases I tried PowerBI, SSRS and other similar tools but they were not as fast as I wanted. What I have in mind is the data to be updated every 1 minute or even less. Is it possible? and are there any free (or inexpensive) tools for this?
Edit: I want to build a wallboard to show some data on a big TV screen. I need it to be real-time. I tried SSRS autorefresh as well but it has a loading sign and very slow, plus PowerBI uses Azure which is very complex to configure and blocked for my country.
This is a topic which has many more layers than to ask which tool is best for this case.
You have to consider
Velocity
Veracity
Variety
Kind
Use Case
of the data. Sure, this is usually only being recounted if talking about Big Data, but will give you a feeling about the size and complexity of data.
Loading
Is the data being loaded and you "just" use it? Or do you also need to load it realtime or near-realtime (for clarification read this answer here)?
Polling/Pushing
Do you want to poll data every x seconds or minutes? Or do you want to work event based? What are the requirements which will need you to show data this fast?
Use case
Do you want to show financial data? Do you need to show data about error and system logs of servers and applications? Do you want to generate insights as soon as a visitor of a webpage is making a request?
Conclusion
When thinking about those questions, keep in mind this should just be a hint to go into one direction or another. Depending on the data and the use case, you might use an ELK stack (for logs), Power BI (for financial data) or even some scripts (for billing).
I have a node.js application that uses a MySQL database. I wanted to know what would be a good place for storing the sessions?
My application is actually a final project for one of my courses, but it could be a real world application later, as we are re-writing a software that is currently used by the university. I can use MySQL for session store, but I want to make my application using the most reliable or best practice in my situation.
I have read many posts/answers/forums, and the opinion is divided. Using another technology like Memcached/MemcacheDB or Redis, just for session store, would it be a recommended approach? Or should I just stick to MySQL, and later deal with scaling if the server load increases?
Even if the application is later used in real world, it would only be used by the undergraduate university students and faculties, so the users are sort of limited.
As of now, I'm leaning towards MySQL for the session store.
I am replying under the assumption that you are using MySQL throughout the whole application.
If the application will be used in the context of your university possibly it will not have scaling issues. SQL databases are not bad, they are able to handle quite a lot of data efficiently, you just need to be careful in the first place and to create efficient queries. Be careful with the joins because can really kill the server. You need to analyze quite a lot your application. For example, why do you think that you will have scaling/performance issues on the sessions and not in another place of your application? Do a bit of load testing, get some metrics and try to understand if you need it or no.
If you are a student though and you don't have prior experience with redis, I would go with redis because it is good to work with a new technology and gain a bit more of experience :)
All s/w is Windows based, coded in Delphi.
Some guys submit some data, which I send by TCP to a database server running MySql.
Some other guys add a pass/fail to their data and update the database.
And a third group are just looking at reports.
Now, the first group can see a history of what they submitted. When the second group adds pass/fail, I would like to update their history. My options seem to be
blindly refresh the history regularly (in Delphi, I display on a DB grid so I would close then open the query), but this seems inefficient.
ask the database server regularly if anything changed in the last X minutes.
never poll the database server, instead letting it inform the user's app when something changes.
1 seems inefficient. 2 seems better. 3 reduces TCP traffic, but that isn't much. Anyway, just a few bytes for each 2. However, it has the disadvantage that both sides are now both TCP client and server.
Similarly, if a member of the third group is viewing a report and a member of either of the first two groups updates data, I wish to reflect this in the report. What it the best way to do this?
I guess there are two things to consider. Most importantly, reduce network traffic and, less important, make my code simpler.
I am sure this is a very common pattern, but I am new to this kind of thing, so would welcome advice. Thanks in advance.
[Update] Close voters, I have googled & can't find an answer. I am hoping for the beneft of your experience. Can you help me reword this to be acceptable? or maybe give a UTL which will help me? Thanks
Short answer: use notifications (option 3).
Long answer: this is a use case for some middle layer which propagates changes using a message-oriented middleware. This decouples the messaging logic from database metadata (triggers / stored procedures), can use peer-to-peer and publish/subscribe communication patterns, and more.
I have blogged a two-part article about this at
Firebird Database Events and Message-oriented Middleware (part 1)
Firebird Database Events and Message-oriented Middleware (part 2)
The article is about Firebird but the suggested solutions can be applied to any application / database.
In your scenarios, clients can also use the middleware message broker send messages to the system even if the database or the Delphi part is down. The messages will be queued in the broker until the other parts of the system are back online. This is an advantage if there are many clients and update installations or maintenance windows are required.
Similarly, if a member of the third group is viewing a report and a
member of either of the first two groups updates data, I wish to
reflect this in the report. What it the best way to do this?
If this is a real requirement (reports are usually a immutable 'snapshot' of data, but maybe you mean a view which needs to be updated while beeing watched, similar to a stock ticker) but it is easy to implement - a client just needs to 'subscribe' to an information channel which announces relevant data changes. This can be solved very flexible and resource-saving with existing message broker features like message selectors and destination wildcards. (Note that I am the author of some Delphi and Free Pascal client libraries for open source message brokers.)
Related questions:
Client-Server database application: how to notify clients that data was changed?
How to communicate within this system?
Each of your proposed solutions are all viable in certain situations.
I've been writing software for a long time and comments below relate to personal experience which dates way back to 1981. I have no doubt others will have alternative opinions which will also answer your questions.
Please allow me to justify the positives and negatives of each approach, and the parameters around each comment.
"blindly refresh the history regularly (in Delphi, I display on a DB grid so I would close then open the query), but this seems inefficient."
Yes, this is inefficient
Is often the quickest and simplest thing to do.
Seems like the best short-term temporary solution which gives maximum value for minimal effort.
Good for "exploratory coding" helping derive a better software design.
Should be a good basis to refine / explore alternatives.
It's very important for programmers to strive to document and/or share with team members who could be affected by your changes their team when a tech debt-inducing fix has been checked-in.
If not intended as production quality code, this is acceptable.
If usability is poor, then consider more efficient solutions, like what you've described below.
"ask the database server regularly if anything changed in the last X minutes."
You are talking about a "pull" or "polling" model. Consider the following API options for this model:
What's changed since the last time I called you? (client to provide time to avoid service having to store and retrieve seesion state)
If nothing has changed, server can provide a time when the client should poll again. A system under excessive load is then able to back-off clients, i.e if a server application has an awareness of such conditions, then it is therefore better able to control the polling rate of compliant clients, by instructing them to wait for a longer period before retrying.
After considering that, ask "Is the API as simple as it can possibly be?"
"never poll the database server, instead letting it inform the user's app when something changes."
This is the "push" model you're talking about- publishing changes, ready for subscribers to act upon.
Consider what impact this has on clients waiting for a push - timeout scenarios, number of clients, etc, System resource consumption, etc.
Consider that the "pusher" has to become aware of all consuming applications. If using industry standard messaging queueing systems (RabbitMQ, MS MQ, MQ Series, etc, all naturally supporting Publish/Subscribe JMS topics or equivalent then this problem is abstracted away, but also added some complexity to your application)
consider the scenarios where clients suddenly become unavailable, hypothesize failure modes and test the robustness of you system so you have confidence that it is able to recover properly from failure and consistently remain stable.
So, what do you think the right approach is now?
Well I don't know if the title for this question is appropriate or not, but I didn't know how to put this in few words.
I'm currently developing a multiplayer 2D game using NodeJS and Socket.io on the server side and HTML5 on the client side. This game doesn't need to save the players progress unless when they finish it. And I must make sure that all information about the players, such as, scores and helps are always valid. So I decided to centralize this information on the server, this way the clients never send a score to the server, instead based on the information sent the server calculates the scores and send them to all clients. Besides that, I can have different sessions of this game running with 2 players minimum and 4 players maximum.
At first I decided to implement this with the server always maintaing the games sessions data in memory, but now I'm questioning myself if I shouldn't have used a database to store this data instead. I'm using a database to store the sessions data only when they are finished because I have no use for unfinished sessions data. But should I instead maintain the sessions and players data on the database while they are playing? My problem here is that the clients communicate very frequently with the server, with this approach I would have to first request their data from the database, make the necessary changes, store it back into the database, and repeat this process on each client request. Maybe the answer to this is obvious, but should I use this approach instead?
It is the first time I'm developing a game, so I have no idea how things usually work. I just know that I want the server to be fast. I chose to maintain everything on memory mainly because all examples and tutorials I found about multiplayer games development never mentioned a database...
Thank you!
I am also developing a multiplayer game using node.js and socket.io. You should not access the database on each client request because,
1) I/O operations are expensive.
Reading/writing to database is "expensive" (slow). It is much faster to read and write from memory.
2) I/O operations should be asynchronous in Node.js.
function read_or_alter_database(input, function(callback){ update_the_client(); });
This makes the database operation non-blocking: the rest of your application will still run, until the operation is done. Then, the callback function is executed. If the player's client rely on the database access to update the game state, then this becomes a blocking operation (since the game cannot proceed until the database operation is done), which negates the main purpose of Node.js.
3) There will be a high volume of client requests in multiplayer games.
This plus point 1 results in a big efficiency loss.
Sounds like you already have your answer: Since the state is only important once a session is complete, only store the data upon completion of that session. Storing the intermediate values serves no purpose, and only slows things down.
Node.js is a relatively new development platform - it is uncertain if it would be able to support such a system. As far as what is NORMALLY done - as much information as possible is stored client side, first off, and then servers typically hold data in memory, or, at least, I know this is the case for the popular games League of Legends, World of Warcraft, and Starcraft II. Then again, those applications are not HTML/Node.js, are very very demanding on the system, and are by huge development companies - it may be that the DB approach is fine for a smaller game, I've just never heard of it.