In producing a web-based data entry system, is the fact that you are adding an extra server request per page a significant concern when deciding whether or not to use a post/redirect/get design?
The request alone isn't a problem, especially that the alternative gives a pretty bad user experience.
However, when using a site with load balancing and/or database replication, you need to take care to ensure that the GET after POST will see the data that has been posted.
When using load balancing and caching, this is sometimes solved with "sticky sessions" that direct the same user to the same machine, so data stored in a write-through cache on that machine will be current.
When using database replication, GET requests after POST may need to read directly from the "primary" database, instead of a local "secondary" as usual.
If I understand your question (and I'm not entirely sure I do), it is definitely good design to do a redirect after a post, even if you are showing them the same page with the updated info.
By doing the redirect you are breaking the connection between the page being viewed and the POST which caused the change. The user can bookmark and/or refresh the page without any popup asking "Do you want to resend the data?"
Most of the time, posts only happen when data is changed. The most traffic and CPU time on sites is generated by queries (GETS) rather than changes, so I think these extra requests aren't very significant.
I think the usability that this offers outweighs the small performance hit.
Test it out by performing some performance benchmarks and you will be able to see if it is going to be a concern in your particular case. See this article for more information.
Related
Meaning every page in my website queries something from the database.
The query in itself is very small and the time it takes to load is unnoticeable, but I'm wondering if it's okay to do this for every page since I don't really know much about how querying from the database works and whether doing it multiple times, and in my case for every page load, affects anything significantly.
As with all things, the answer is it depends. :-)
Most web sites you visit queries something from a database on every page load. If the queries are crafted well, they look up just the data they need, and avoid scanning through a big database. You might like my presentation How to Design Indexes, Really (video) to help with this.
Another strategy is to use a fast cache in RAM for data that is needed frequently. RAM is thousands of times faster than disk drives. You might like to familiarize yourself with the Numbers Everyone Should Know. Those numbers are just examples, but the intention is to get programmers to think about the fact that moving data around has different cost as you use RAM vs. disk vs.network vs. CPU.
P.S.: Please don't buy into the myth that you're not good at computers because you're a woman. Everyone starts out as a novice, no matter what their gender or background. Only through practice and study do any of us learn this stuff. I recommend seeing Hidden Figures, the story of the women who did pioneering math and programming for NASA.
Another notable woman is Margaret Hamilton, who practically invented the profession of "software engineering."
Yes you are OK to query the database on every page load.
Think about websites like Facebook. When you visit the site it needs to know who you are - it gets that from a database. It needs to know all of the status updates that it's going to show you - it gets that from a database. When you hit the bottom of the news feed and it gets more for you to read - it gets that from a database.
That's normal. Most web applications have to query the database for each page load (usually several times), since most of the page content comes from the database.
If you're concerned about performance, think about this: is the query different for each page? Or is it loading the same data over and over? If it keeps querying the same thing (like the current user's name), you can improve performance by storing the data in the application's session state. But if it's different (like how many unread messages the user has), you'll need to run the query each time.
Imagine visiting a website which has features like 'whois online' or a messaging system, when ever you click on another page the site needs to update the database so that it can keep track of where you are on the site. If you receive a private message it would be accessible on the next page click since the database would have been updated when the message was sent. The trick is to run queries only to perform tasks which is required at that time. For instance if you were looking for a username in the database, if you searched the whole database it will run a lot slower as it needs to search the whole database. If you searched by a particular column it will be faster, it will be even faster if you used things like limits such as LIMIT in the query.
I've been traveling for the last couple weeks and have found a issue with the method that Ajax uses to construct a website. I understand that the webpage requesting only the pieces it needs is the most efficient method for the servers but when working in an environment where signal comes and goes or is being throttled by a provider, most websites running on this model become completely unresponsive and turn every interaction into a several minute wait.
In situations where the bandwidth is limited, the best performance generally comes from websites that have all of their content on one single page that is constructed for the user before it is sent. I understand that this is not the restful way but I was wondering if there was a middle ground to this solution.
Is there a way to batch many different AJAX calls where the user would only be sending one large call to the server which then the server would compile everything that is listed and then returns it in one heap? Or is this something that hasn't been formed into a standard yet and a custom server architecture would end up needing to do?
In a situation where bandwidth is extremely limited, everything you will try to do will be a pain.
Yes, in this scenario, frequently opening connections to the server through multiple requests (which is very typical of ajax single page applications) will make the experience worst than opening one single connection to the server.
However, you need to ask yourself if you want your web application to cater to clients with fast connections or to cater to clients with slow connections and design your web application accordingly. If you make it only to accommodate slow clients then the user experience for those with faster connections will suffer and vice versa.
You could also decide to cater to both audiences by creating a version for each but it's a lot of extra work
I have no idea what your web application does. But if it's to simply "view" data then perhaps you can get away with loading all the data from the start. However, if your web application contains a lot of data manipulation features then you have no choice, stick with Ajax and get a better internet connection.
If you want to batch your requests then your web application needs to be designed that way which would allow you to do everything you need to do on the client side before clicking on a "save" button that will gather all the changes you made and send it all in 1 request.
You should always build your web application according to your client's situation. If you're traveling a lot then that might be strictly your problem and won't ever be your client's problem. In this case, stick with ajax and get a better internet connection.
If the client is yourself then heck you could do whatever you want to ease your pain including loading everything from the get go.
Unfortunately there's no magic solution.
Hope it helps!
All s/w is Windows based, coded in Delphi.
Some guys submit some data, which I send by TCP to a database server running MySql.
Some other guys add a pass/fail to their data and update the database.
And a third group are just looking at reports.
Now, the first group can see a history of what they submitted. When the second group adds pass/fail, I would like to update their history. My options seem to be
blindly refresh the history regularly (in Delphi, I display on a DB grid so I would close then open the query), but this seems inefficient.
ask the database server regularly if anything changed in the last X minutes.
never poll the database server, instead letting it inform the user's app when something changes.
1 seems inefficient. 2 seems better. 3 reduces TCP traffic, but that isn't much. Anyway, just a few bytes for each 2. However, it has the disadvantage that both sides are now both TCP client and server.
Similarly, if a member of the third group is viewing a report and a member of either of the first two groups updates data, I wish to reflect this in the report. What it the best way to do this?
I guess there are two things to consider. Most importantly, reduce network traffic and, less important, make my code simpler.
I am sure this is a very common pattern, but I am new to this kind of thing, so would welcome advice. Thanks in advance.
[Update] Close voters, I have googled & can't find an answer. I am hoping for the beneft of your experience. Can you help me reword this to be acceptable? or maybe give a UTL which will help me? Thanks
Short answer: use notifications (option 3).
Long answer: this is a use case for some middle layer which propagates changes using a message-oriented middleware. This decouples the messaging logic from database metadata (triggers / stored procedures), can use peer-to-peer and publish/subscribe communication patterns, and more.
I have blogged a two-part article about this at
Firebird Database Events and Message-oriented Middleware (part 1)
Firebird Database Events and Message-oriented Middleware (part 2)
The article is about Firebird but the suggested solutions can be applied to any application / database.
In your scenarios, clients can also use the middleware message broker send messages to the system even if the database or the Delphi part is down. The messages will be queued in the broker until the other parts of the system are back online. This is an advantage if there are many clients and update installations or maintenance windows are required.
Similarly, if a member of the third group is viewing a report and a
member of either of the first two groups updates data, I wish to
reflect this in the report. What it the best way to do this?
If this is a real requirement (reports are usually a immutable 'snapshot' of data, but maybe you mean a view which needs to be updated while beeing watched, similar to a stock ticker) but it is easy to implement - a client just needs to 'subscribe' to an information channel which announces relevant data changes. This can be solved very flexible and resource-saving with existing message broker features like message selectors and destination wildcards. (Note that I am the author of some Delphi and Free Pascal client libraries for open source message brokers.)
Related questions:
Client-Server database application: how to notify clients that data was changed?
How to communicate within this system?
Each of your proposed solutions are all viable in certain situations.
I've been writing software for a long time and comments below relate to personal experience which dates way back to 1981. I have no doubt others will have alternative opinions which will also answer your questions.
Please allow me to justify the positives and negatives of each approach, and the parameters around each comment.
"blindly refresh the history regularly (in Delphi, I display on a DB grid so I would close then open the query), but this seems inefficient."
Yes, this is inefficient
Is often the quickest and simplest thing to do.
Seems like the best short-term temporary solution which gives maximum value for minimal effort.
Good for "exploratory coding" helping derive a better software design.
Should be a good basis to refine / explore alternatives.
It's very important for programmers to strive to document and/or share with team members who could be affected by your changes their team when a tech debt-inducing fix has been checked-in.
If not intended as production quality code, this is acceptable.
If usability is poor, then consider more efficient solutions, like what you've described below.
"ask the database server regularly if anything changed in the last X minutes."
You are talking about a "pull" or "polling" model. Consider the following API options for this model:
What's changed since the last time I called you? (client to provide time to avoid service having to store and retrieve seesion state)
If nothing has changed, server can provide a time when the client should poll again. A system under excessive load is then able to back-off clients, i.e if a server application has an awareness of such conditions, then it is therefore better able to control the polling rate of compliant clients, by instructing them to wait for a longer period before retrying.
After considering that, ask "Is the API as simple as it can possibly be?"
"never poll the database server, instead letting it inform the user's app when something changes."
This is the "push" model you're talking about- publishing changes, ready for subscribers to act upon.
Consider what impact this has on clients waiting for a push - timeout scenarios, number of clients, etc, System resource consumption, etc.
Consider that the "pusher" has to become aware of all consuming applications. If using industry standard messaging queueing systems (RabbitMQ, MS MQ, MQ Series, etc, all naturally supporting Publish/Subscribe JMS topics or equivalent then this problem is abstracted away, but also added some complexity to your application)
consider the scenarios where clients suddenly become unavailable, hypothesize failure modes and test the robustness of you system so you have confidence that it is able to recover properly from failure and consistently remain stable.
So, what do you think the right approach is now?
I have a system (develop by someone else) where all registered user can query data (similar to data.stackexchange.com). The system is getting big and more user query the system and during the high traffic time the database is slow and I am afraid of security now.
What can I do to make the system more secure?
What can I do to make the queries faster to execute?
I have a very basic knowledge of mysql and databases and I want to learn. Can you point where I need to look and what can I do? (I would like to build my self, so please no code)
Well, you have two large jobs to do :)
How to make the system more secure? Well, use SSL where you need to. If the data is not important you can get away without it. That said, if you want to ultra-secure your logins, then insist on HTTPS. Above that, ensure that you never compare passwords directly, rather you compare the hashes of the passwords (with the inclusion of a salt). Additionally, if your website allows people to be remembered, use a token-based approach. This allows you to assign a unique cookie ID with the client for a period of time that it is valid. It's not fool-proof, but better than nothing. Paired with your SSL login requirements, it will be pretty good.
Have a look at cache managers. But before you do, have a gander at what is taking the most time. What particular pages are hitting your website the hardest? Once you ascertain that you can come up with a caching strategy which is, unfortunately, completely website-dependant. What works for one site, would be inadmissable for you. You can use some kind of memcache to store the common stuff so that the basic "Front page" and "Portal" queries are cached efficiently. The rest will have to be dealt with in the regular way.
I am building a very simple classified site.
There is a form that puts data in mysql table.
Now how should this data be displayed ? Is it better to build html pages from the data in a table , and then display it to the users OR is it better to, fetch the data from the mysql table each time a user wants to see the data ?
I hope I was clear!
Performance-wise, it's generally better to keep the static versions of the HTML pages.
However, you may have too many dynamic content which can bloat your disk space, and you should apply some extra effort to track cache expiration (which can be even more expensive than generating the content dynamically).
It's a matter of tradeoff, and to make any other advices we would need to know the nature of your data.
If it's a blog with content updated rarely but read often, it's better to cache.
If it's a product search engine with mostly unique queries and changing stock, it's better to always query the database.
Note that MySQL implements query cache: it can cache the resultsets of the queries and if the query is repeated verbatim and no underlying tables were changed since the last query, then it's served out of the cache.
It tracks the cache expiration automatically, saves you of the need to keep the files on the disk and generally combines the benefits of both methods.
You can use Php caching techniques if the data would not change frequently. Keep loading the cached contents for frequent visits.
http://www.developertutorials.com/tutorials/php/php-caching-1370/
Use both, via a caching mechanism. Based on parameters, the page would be re-rendered (has not been viewed in X time or at all) or displayed from cache otherwise.
As stated though, it depends heavily on the amount of and frequency with which the data is accessed. More information would warrant a more detailed response.
It depends on a few things. Ask yourself two easy questions:
1) How often does the content change? Are your classified ads static or are they changing a lot on the page. How much control do you want on that page to have rotating ads, comments from users, reviews etc.
2) Are you going to be VERY high traffic? So much so that you are looking at a bottleneck at the database?
If you can say "Yes, no doubts, tomorrow" to question #2, go static. even it means adding other things in via ajax or non database calls (ie includes) in order to make the page pseudo-dynamic.
Otherwise if you say "Yes" to question #1, go with a dynamic page so you have the freshest content at all times. These days users have gotten very used to instant gratification on posts and such. Gone are the days we would wait for hours for a comment to appear in a thread (I am looking at you Slashdot).
Hope this helps!
Start with the simplest possible solution that satisfies the requirements, and go from there.
If you implement a cache but didn't need one, you have wasted time (and/or money). You could have implemented features instead. Also, now you (might) have to deal with the cache everytime you add features.
If you don't implement a cache and realize you need one, you are now in a very good position to implement a smart one, because now you know exactly what needs to be cached.