Is it considered crazy to store common SQL queries for my web app in a database for use in execution? Or is that common practice? Or is it impossible?
My thinking is, this way, I avoid hard-coding SQL into my application files, and add another level of abstraction.
Is this crazy? Is this what a stored procedure is? Or is that something else?
EDIT: The below answers are useful as a background for 'stored procedures', but didn't answer my core question: Is a 'stored procedure' just when I have a database table that contains queries that can be called? ie, something like this
INDEX | NAME | QUERY
1 | show_names | "SELECT names.first, names.last FROM names;"
2 | show_5_cities | "SELECT cities.city FROM cities LIMIT 0,5;"
etc.
Or is there a more complicated mechanism that encompasses the concept of stored procedures? Is my example an actual example of something people do?
Along with MUG4N's great reasons on why to use stored procedures, here are three more:
Security
You can grant access to your application to execute stored procedures while denying direct table access.
Think defense in depth. If your app is cracked, then they will be limited to executing ONLY the procedures you have defined. This means things like 'drop table' would be explicitly disallowed, unless, of course, you have a procedure to do that.
Conversely, if your app is cracked and you allow the app to have full access to your sql server, then one of two things will happen. Either your data disappears and/or the cracker easily get's a copy.
Unit Testing.
It's much easier to unit test your queries if you can hit them directly without having to go through the application itself.
In Flight Changes:
If you need to modify a query AFTER you have published your site, it's much easier to just make a proc change than redeploy code that may have undergone other changes since the last deployment. For example, let's say you have a page that isn't performing all that well. After evaluation, you determine that just changing the joins on a query will fix this. Modify the proc and go.
Well in my opinion you should definitly use stored procedures. And this is common practice!
Here are just two advantages of using stored procedures:
They will run in all environments, and there is no need to recreate the logic. Since they are on the database server, it makes no difference what application environment is used - the stored procedure remains consistent. If your setup involves different clients, different programming languages - the logic remains in one place. Web developers typically make less use of this feature, since the web server and database server are usually closely linked. However, in complex client-server setups, this is a big advantage. The clients are automatically always in sync with the procedure logic as soon as its been updated.
They can reduce network traffic. Complex, repetitive tasks may require getting results, applying some logic to them, and using this to get more results. If this only has to be done on the database server, there is no need to send result sets and new queries back and forth from application server to database server. Network traffic is a common bottleneck causing performance issues, and stored procedures can help reduce this. More often though, it is the database server itself that is the bottleneck, so this may not be much of an advantage.
The idea certainly has its appeal -- but the problems is, they are nearly impossible to scale.. I have never seen a scalable solution to maintaining stored procs (especially in MySQL) that has not made me shutter.
Since it seems you're heading the PHP/MySQL route, I'll give a few examples of my experience with stored procs in MySQL:
They are generally far less readable and far more difficult to write than PHP.
They make debugging a nightmare
Trying to figure out why changing a value in table_1 triggers a change in table_2 (if you're even lucky enough to recognize that this happens) is much more difficult to determine by looking through dozens of stored procedures than it is to, say, look in the Model that handles changes to table_1.
To my knowledge there is no standardized & automated way to integrate stored procs / triggers / etc into any revision control system
A stored procedure is just one or more SQL statements that are "pre-compiled" and live inside the database. You call them to return one or more rows of data, or to update, insert, or delete data.
If you tell us what web framework and database you are using, we can give you actual examples of how to call a stored procedure, or at least point you to an article or two to get you going.
You could also consider using an ORM framework, such as Hibernate. This will allow you to get away from dealing with SQL code altogether. I am a .Net developer, so I'm not sure what is available to you on the PHP/MySQL platform, but I am sure there is a lot out there to choose from.
You should think about it, when developing a commercial grade tiered application there is always people behind the database making it secure and reliable, other people are behind the application logic and other people behind the web code, so you can get the best of all working together.
Once the application has been designed, everyone start making their implementations, the db people give to the others some kind of API to use hiding the SQL, the developers won't have to think about it and focus on their code, i had worked as db developer and used some COM techniques to overcome the expansion and modification of the application logic or reuse, the database in these kind of products is too important to leave it in the wild so security it's a really serious issue.
But in most cases, web applications are made by web developers and they tend to have no design time, making big changes on the near time so they don't use stored procedures, also they don't even secure execution or try to leave security to the application leaving the database unprotected and prone to attacks.
If you're doing everything and changing your product too often you should avoid them since it will be double work and most of the times will be useless, once you stabilize your logic then you could start migrating your heavier queries to stored procedures.
Related
how much work should we do in the database?
Ok I'm really confused as to exactly how much "work" should be done IN the database, and how much work had to be done instead at the application level?
I mean I'm not talking about obvious stuff like we should convert strings into SHA2 hashes at the application level instead of the database level..
But rather stuff that are more blur, including, but not limited to "should we retrieve the data for 4 column and do a uppercase/concatenation at the application level, or should we do those stuff at the database level and send the calculated result to the application level?
And if you could list any more other examples it would be great.
It really depends on what you need.
I like to do my business logic in the database, other people are religously against that.
You can use triggers and stored procedures/functions in SQL.
Links for MySQL:
http://dev.mysql.com/doc/refman/5.5/en/triggers.html
http://www.mysqltutorial.org/introduction-to-sql-stored-procedures.aspx
http://dev.mysql.com/doc/refman/5.5/en/stored-routines.html
My reasons for doing business logic in triggers and stored proces
Note that I'm not talking about bending the database structure towards the business logic, I'm talking about putting the business logic in triggers and stored procedures.
It centralizes your logic, the database is a central place, everything has to go through it. If you have multiple insert/update/delete points in your app (or you have multiple apps) you'll need to do the checks multiple times, if you do it in the database you only have to do the checks in one place.
It simplifies the application e.g., you can just add a member, the database will figure out if the member is already known and take the appopriate action.
It hides the internals of your database from the application, if you do all your logic in the application you will need intricate knowledge of your database in the application. If you use database code (triggers/procs) to hide that, you don't need to know every database detail in your app.
It makes it easier to restucture your database If you have the logic in your database, you can just change a tablelayout, replace the old table with a blackhole table, put a trigger on that and let the trigger do the updates to the new table, your app does not even need to know the database has changed, this allows legacy apps to keep working unchanged, whilst new apps can use the improved database layout.
Some things are easier in SQL
Some things work faster in SQL
I don't like to use (lots of and/or complicated) SQL code in my application, I like to put SQL code in a stored procedure/function and try to only put simple queries in my application code, that way I can just write code that explains what I mean in my application and let the database layer do the heavy lifting.
Some people disagree strongly with this, but this approach works well for me and has simplified debugging and maintenance of my applications a lot.
Generally, its a good practice to expect only "Data" from the Database. Its upto Application(s), to apply Business/Domain Logic and make sense of the data retrieved. Its highly recommended to do the following things in the Application Layer:
1) Formatting Date
2) Applying Math functions, such as interpolation/extrapolation, etc
3) Dynamic sorting (based on columns)
However, situations sometime warrant few things to be done at the database level.
In my opinion application should use data and database should provide them and that should be clear separation of concerns. So database gives records sorted, ordered and filtered according to requested conditions but it is up to application to apply some business logic to that records and "convert" them into something meaningful to the user.
For example, in my previous company we worked on big application for work time calculations. One of obvious functionalities in this kind of application is tracking vacation days of employees - how many days employee has per year, how many he used, how many left, etc. Basically we could write some triggers and procedures that would update those columns automatically. So when employee had his vacation days approved amount of days he applied for is taken from his "vacation pool" and added to "vacation days used". Pretty easy stuff but we decided to make it explicit on application level and boy, very soon we were happy we did it that way. Application had to be labor law compliant and it quickly turned out that not for all employees vacation days are calculated equally and sometimes vacation day can be not so vacation day at all but that is beside the point. Had we put this "easy" operation in database we had to version our database with every little change to a vacation days related logic and that would lead us straight to hell in customer support field due to a fact that it was possible to update only application without a need to update database (except clear "breakthrough" moments where database structure was changed of course).
In my experience I've found that many applications start with a straight-forward set of tables and then and handful of stored procedures to provide basic functionality. This works very well; it usually yields high performance and is simple to understand, it also mitigates any need for a complex middle-tier.
However, applications grow. It's not unusual to see large data-driven applications with thousands of stored procedures. Throw triggers into the mix and you have an application which, for anybody other than the original developers (if they're still working on it), is very difficult to maintain.
I will put a word in for applications which place most logic in the database - they can work well when you have some good database developers and/or you have a legacy schema which cannot be changed. The reason I say this is that ORMs take much of the pain out of this part of application development when you let them control the schema (if not, you often need to do a lot of fiddling to get it working).
If I was designing a new application then I would usually opt for a schema which is dictated by my application domain (the design of which will be in code). I would normally let an ORM handle the mapping between the objects and the database. I would treat stored procedures as exceptions to the rule when it came to data access (reporting can be much easier in sprocs than trying to coax an ORM into producing a complex output efficiently).
The most important thing to remember though, is that there are no "best practices" when it comes to design. It is up to you the developer to weigh up the pros and cons of each option in the context of your design.
We are at the beginning of a new project, and we are really wondering if we should use stored procedures in MySQL or not.
We would use the stored procedures only to insert and update business model entities. There are several tables which represent a model entity, and we would abstract it in those stored procedures insert/update.
On the other hand, we can call insert and update from the Model layer but not in MySQL but in PHP.
In your experience, Which is the best option? advantages and disadvantages of both approaches. Which is the fastest one in terms of high performance?
PS: It is is a web project with mostly read and high performance is the most important requisite.
Unlike actual programming language code, they:
not portable (every db has its own version of PL/SQL. Sometimes different versions of the same database are incompatible - I've seen it!)
not easily testable - you need a real (dev) database instance to test them and thus unit testing their code as part of a build is virtually impossible
not easily updatable/releasable - you must drop/create them, ie modify the production db to release them
do not have library support (why write code when someone else has)
are not easily integratable with other technologies (try calling a web service from them)
they use a language about as primitive as Fortran and thus are inelegant and laborious to get useful coding done, so it is difficult to express business logic, even though typically that is what their primary purpose is
do not offer debugging/tracing/message-logging etc (some dbs may support this - I haven't seen it though)
lack a decent IDE to help with syntax and linking to other existing procedures (eg like Eclipse does for java)
people skilled in coding them are rarer and more expensive than app coders
their "high performance" is a myth, because they execute on the database server they usually increase the db server load, so using them will usually reduce your maximum transaction throughput
inability to efficiently share constants (normally solved by creating a table and questing it from within your procedure - very inefficient)
etc.
If you have a very database-specific action (eg an in-transaction action to maintain db integrity), or keep your procedures very atomic and simple, perhaps you might consider them.
Caution is advised when specifying "high performance" up front. It often leads to poor choices at the expense of good design and it will bite you much sooner than you think.
Use stored procedures at your own peril (from someone who's been there and never wants to go back). My recommendation is to avoid them like the plague.
Unlike programming code, they:
render SQL injection attacks almost
impossible (unless you are are
constructing and executing dynamic
SQL from within your procedures)
require far less data to be sent over
the IPC as part of the callout
enable the database to far better
cache plans and result sets (this is
admittedly not so effective with
MySQL due to its internal caching
structures)
are easily testable in isolation
(i.e. not as part of JUnit tests)
are portable in the sense that they
allow you to use db-specific
features, abstracted away behind a
procedure name (in code you are stuck
with generic SQL-type stuff)
are almost never slower than SQL
called from code
but, as Bohemian says, there are plenty of cons as well (this is just by way of offering another perspectve). You'll have to perhaps benchmark before you decide what's best for you.
As for performances, they have the potential to be really performant in a future MySQL version (under SQL Server or Oracle, they are a real treat!). Yet, for all the rest... They totally blow up competition. I'll summarize:
Security: You can give your app the EXECUTE right only, everything is fine. Your SP will insert update select ..., with no possible leak of any sort. It means global control over your model, and an enforced data security.
Security 2: I know it's rare, but sometimes php code leaks out from the server (i.e. becomes visible to public). If it includes your queries, possible attackers know your model. This is pretty odd but I wanted to signal it anyway
Task force: yes, creating efficient SQL SPs requires some specific resources, sometimes more expensive. But if you think you don't need these resources just because you're integrating your queries in your client... you're going to have serious problems. I'd mention the analogy of web development: it's good to separate the view from the rest because your designer can work on their own technology while the programmers can focus on programming the business layer.
Encapsulating business layer: using stored procedures totally isolates the business where it belongs: the damn database.
Quickly testable: one command line under your shell and your code is tested.
Independence from the client technology: if tomorrow you'd like to switch from php to something else, no problem. Ok, just storing these SQL in a separate file would do the trick too, that's right. Also, good point in the comments about if you decide to switch sql engines, you'd have a lot of work to do. You have to have a good reason to do that anyway, because for big projects and big companies, that rarely happens (due to the cost and HR management mostly)
Enforcing agile 3+-tier developments: if your database is not on the same server than your client code, you may have different servers but only one for the database. In that case, you don't have to upgrade any of your php servers when you need to change the SQL related code.
Ok, I think that's the most important thing I had to say on the subject. I developed in both spirits (SP vs client) and I really, really love the SP style one. I just wished Mysql had a real IDE for them because right now it's kind of a pain in the ass limited.
Stored procedures are good to use because they keep your queries organized and allow you to perform a batch at once. Stored procedures are normally quick in execution because they are pre-compiled, unlike queries that are compiled on every run. This has significant impact in situations where database is on a remote server; if queries are in a PHP script, there are multiple communication between the application and the database server - the query is send, executed, and result thrown back. However, if using stored procedures, it only need to send a small CALL statement instead of big, complicated queries.
It might take a while to adapt to programming a stored procedure because they have their own language and syntaxes. But once you are used to it, you'll see that your code is really clean.
In terms of performance, it might not be any significant gain if you use stored procedures or not.
I will let know my opinion, despite my toughts possibly are not directly related to the question.:
As in many issues, reply about using Stored Procedures or an application-layer driven solution relies on questions that will drive the overall effort:
What you want to get.
Are you trying to do either batch operations or on-line operations? are they completely transactional? how recurrent are those operations? how heavy is the awaited workload for the database?
What you have in order to get it.
What kind of database technology you have? What kind of infrastucture? Is your team fully trained in the database technology? Is your team better capable of building a database-aegnostic solution?
Time for get it.
No secrets about that.
Architecture.
Is your solution required to be distributed onto several locations? is your solution required to use remote communications? is your solution working on several database servers, or possibly using a cluster-based architecture?
Mainteinance.
How much is the application required to change? do you have personal specifically trained for maintain the solution?
Change Management.
Do you see your database technology will change at a short, middle, long time? do you see will be required to migrate the solution frequently?
Cost
How much will cost to implement that solution using one or another strategy?
The overall of those points will drive the answer. So you have to care each of this points when making a decision about using or not any strategy. There are cases where using of stored procedures are better than application-layer managed queries, and others when, conducting queries and using an application-layer based solution is best.
Using of stored procedures tends to be more addequate when:
Your database technology isn't provided to change at a short time.
Your database technology can handle parallelized operations, table partitions or anything else strategy for divide the workload onto several processors, memory and resources (clustering, grid).
Your database technology is fully integrated with the stored proceduce definition language, that is, support is inside the database engine.
You have a development team who aren't afraid about using a procedural language (3rd. Generation language) for getting a result.
Operations you wanna achieve are built-in or supported inside the database (Exporting to XML data, managing data integrity and coherence appropiately with triggers, scheduled operations, etc).
Portability isn't an important issue and you do not whatch a technology change at a short time into your organization, even, it is not desirable. Generally, portability is seen like a milestone by the application-driven and layered-oriented developers. From my point of view, portability isn't an issue when your application isn't required to be deployed for several platforms, less when there are no reasons for making a technology change, or the effort for migrating all the organizational data is higher than the benefit for making a change. What you can win by using an application-layer driven approach (portability) you can loose in performance and value obtained from your database (Why to spend thousands of dollars for to get a Ferrari that you'll drive no more than 60 milles/hr?).
Performance is an issue. First: In several cases, you can achieve better results by using a single stored procedure call than multiple requests for data from another application. Moreover, some characteristics you need to perform may be built-in your database and its use less expensive in terms of workload. When you use an application-layer driven solution you have to take in account the cost associated to make database connections, making calls to the database, network traffic, data wrapping (i.e., using either Java or .NET, there is an implicit cost when using JDBC/ADO.NET calls as you have to wrap your data into objects that represents the database data, so instantiation has an associated cost in terms of processing, memory, and network when data comes from and goes to outside).
Using of application-layer driven solutions tends to be more addequate when:
Portability is an important issue.
Application will be deployed onto several locations with only one or few database repositories.
Your application will use heavy business-oriented rules, that need to be agnostic of the underlying database technology.
You have in mind to do change technology providers based on market tendencies and budget.
Your database isn't fully integrated with the stored procedure language that calls to the database.
Your database capabilities are limited and your requirement goes beyond what you can achieve with your database technology.
Your application can support the penalty inherent to external calls, is more transactional-based with business-specific rules and has to abstract the database model onto a business model for the users.
Parallelizing database operations isn't important, moreover, your database has not parallelization capabilities.
You have a development team which is not well-trained onto the database technology and is better productive by using an application-driven based technology.
Hope this may help to anyone asking himself/herself what is better to use.
I would recommend you don't use stored procedures:
Their language in MySQL is very crappy
There is no way to send arrays, lists, or other types of data structure into a stored procedure
A stored procedure cannot ever change its interface; MySQL permits neither named nor optional parameters
It makes deploying new versions of your application more complicated - say you have 10x application servers and 2 databases, which do you update first?
Your developers all need to learn and understand the stored procedure language - which is very crap (as I mentioned before)
Instead, I recommend to create a layer / library and put all your queries in there
You can
Update this library and ship it on your app servers with your app
Have rich data types, such as arrays, structures etc passed around
Unit test this library, instead of the stored procedures.
On performance:
Using stored procedures will decrease the performance of your application developers, which is the main thing you care about.
It is extremely difficult to identify performance problems within a complicated stored procedure (it is much easier for plain queries)
You can submit a query batch in a single chunk over the wire (if CLIENT_MULTI_STATEMENTS flag is enabled), which means you don't get any more latency without stored procedures.
Application-side code generally scales better than database-side code
If your database is complex and not a forum type with responses, but true warehousing SP will definitely benefit. You can out all your business logic in there and not a single developer is going to care about it, they just call your SP's. I have been doing this joining over 15 tables is not fun, and you cannot explain this to a new developer.
Developers also don't have access to a DB, great! Leave that up to database designers and maintainers. If you also decide that the table structure is going to get changed, you can hide this behind your interface. n-Tier, remember??
High performance and relational DB's is not something that goes together, not even with MySQL InnoDB is slow, MyISAM should be thrown out of the window by now. If you need performance with a web-app, you need proper cache, memcache or others.
in your case, because you mentioned 'Web' I would not use stored procedures, if it was data warehouse I would definitely consider it (we use SP's for our warehouse).
Tip:
Since you mentioned Web-project, ever though about nosql sort of solution? Also, you need a fast DB, why not use PostgreSQL? (trying to advocate here...)
I used to use MySql and my understanding of sql was poor at best, I spent a fair amount of time using Sql Server, I have a clear separation of a data layer and an application layer, I currently look after a server with 0.5 terabytes.
I have felt frustrated at times not using an ORM as development is really quick with stored procedures it is much slower. I think much of our work could have been sped up by using an ORM.
When your application reaches critical mass, the ORM performance will suffer, a well written stored procedure, will give you your results faster.
As an example of performance I collect 10 different types of data in an application, then convert that to XML, which I process in the stored procedure, I have one call to the database rather than 10.
Sql is really good at dealing with sets of data, one thing that gets me frustrated is when I see someone getting data from sql in a raw form and using application code to loop over the results and format and group them, this really is bad practice.
My advice is to learn and understand sql enough and your applications will really benefit.
Lots of info here to confuse people, software development is a evolutionary. What we did 20 years ago isn't best practice now. Back in the day with classic client server you wouldnt dream of anything but SPs.
It is absolutely horses for courses, if you are a big organisation with you will use multi tier, and probably SPs but you will care little about them because a dedicated team will be sorting them out.
The opposite which is where I find myself trying to quickly knock up a web app solution, that fleshes out business requirements, it was super fast to leave the developer (remote to me) to knock up the pages and SQL queries and I define the DB structure.
However complexity is growing and without an easy way to provide APIs, I am staring to use SPs to contain the business logic. I think it is working well and sensible, I control this because I can build logic and provide a simple result set for my offshore developer to build a front end around.
Should I find my software a phenomenal success, then more separation of concerns will occur and different implementations of n teir will come about but for now SPs are perfect.
You should know all the tool sets available to you and match them is wise to start with. Unless you are building an enterprise system to start with then fast and simple is best.
I would recommend that you stay away from DB specific Stored Procedures.
I've been through a lot of projects where they suddently want to switch DB platform and the code inside a SP is usually not very portable = extra work and possible errors.
Stored Procedure development also requires the developer to have access directly to the SQL-engine, where as a normal connection can be changed by anyone in the project with code-access only.
Regarding your Model/layer/tier idea: yes, stick with that.
Website calls Business layer (BL)
BL calls Data layer (DL)
DL calls whatever storage (SQL, XML, Webservice, Sockets, Textfiles etc.)
This way you can maintain the logic level between tiers. IF and ONLY IF the DL calls seems to be very slow, you can start to fiddle around with Stored Procedures, but maintain the original none-SP code somewhere, if you suddently need to transfer the DB to a whole new platform. With all the Cloud-hosting in the business, you never know whats going to be the next DB platform...
I keep a close eye on Amazon AWS of the very same reason.
I think there is a lot of misinformation floating around about database stored queries.
I would recommend using MySQL Stored Procedures if you're doing many static queries for data manipulation. Especially if you're moving things from one table to another (i.e. moving from a live table to a historical table for whatever reason). There are drawbacks of course in that you'll have to keep a separate log of changes to them (you could in theory make a table that just holds changes to the stored procedures that the DBA's update). If you have many different applications interfacing with the database, especially if say you have a desktop program written in C# and a web program in PHP, it might be more beneficial to have some of your procedures stored in the database as they are platform independent.
This website has some interesting information on it you may find useful.
https://www.sitepoint.com/stored-procedures-mysql-php/
As always, build in a sandbox first, and test.
Try to update 100,000,000 records on a live system from a framework, and let me know how it goes. For small apps, SPs are not a must, but for large serious systems, they are a real asset.
Are they useful for anything outside of a database administrator? If I understand them correctly it's merely queries that can be saved directly into MySQL, so it'd be useless for any web development team to use them.
Stored procedures are code that runs on the database server.
They have a number of uses. Think: If I could run code directly on the database server, what could I use that for?
Among their many uses, stored procedures can be used to shift some of the processing load to the database server, to reduce network traffic, and to improve security.
http://en.wikipedia.org/wiki/Stored_procedure
Here are two good, simple advantages not covered in the other answers:
Security - parameterized stored procedures are safer than contencating strings for SQL (Google SQL Injection for about a million documents on this) However, parameterized queries are also good for this if your language supports them
Simplification of Maintenance - It's a heck of a lot easier to update a stored procedure than to recompile code and re-deploy. In my 15 years of development I've learned this the hard way. If there's a chance the query might change, put it in a stored proc. It's SOOO much easier to work with than having to recompile and redeploy code.
Added
They also reduce network chatter. If you have a lot of complex wueries to run, you can have them all in one stored procedure, and your app only needs to make one call to do all the work.
Also, in most platforms, stored procedures have performance benefits. In SQL Server, for example, the Database Engine optimizes and saves the executio plan to speed things up.
these links also answer your question:
http://blog.sqlauthority.com/2007/04/13/sql-server-stored-procedures-advantages-and-best-advantage/
http://searchsqlserver.techtarget.com/news/1052737/Why-use-stored-procedures
And I can't take credit for this answer, but I think this quote is a good point, even though I consider myself to be pretty skilled on both sides of the equation - there is something to be said for specialized knowledge.
Advantage 4: Stored procedures are
usually written by database
developers/administrators. Persons
holding these roles are usually more
experienced in writing efficient
queries and SQL statements. This frees
the GUI application developers to
utilize their skills on the functional
and graphical presentation pieces of
the application. If you have your
people performing the tasks to which
they are best suited, then you will
ultimately produce a better overall
application.
Think of a stored procedures as a library function. Do you want to rewrite sqrt (for example) every time you have to compute a square root or would you rather call a function to do it?
That's the benefit (in a nutshell) of stored procedures.
Stored procedures have lots of benefits. Among other things they help decouple application code from the database tables, simplify database maintenance and versioning and help take the best advantage of DBMS features such as query parameterisation, optimisation and security.
it'd be useless for any web
development team to use them
Huh? Stored procedures are extremely useful for any developers who need to use a database that supports them.
Stored Procedures can do far more than just query the database. They can contain any T-SQL statement. So you could use them to perform business logic, execute queries, do backups etc.
Many companies have a policy that all database activity is to be done via stored procedures. So, in now way would I say that a web development team would have no use for them. They might make great use of them.
On the other hand, in our company, we're not using them much of all for our next generation manufacturing applications. We're using an ORM (Linq-To-SQL) instead, and have very little use for stored procedures, at this point. I suspect though we'll still use them somewhat, in order to avoid several trips back and forth to the server. Somethings are just more efficient if done in a stored procedure, if you're already doing work on the server anyway.
Back in the 90's, stored procedures were the safest way to prevent anyone from accessing the data tables directly.
At first hand, they were more likely to counter security issues. Secondly, they were meant to be easier to work with the data, as there were no ORM tools as today's.
In the days, they are meant for complex transactions mainly. When I say complex, I mean something that cannot be solved with simple CRUD operations such as NHibernate or Entity Framework can do.
So, when a stored procedure only perform a SELECT, an INSERT, an UPDATE or a DELETE, you may be right that they are now useless somehow, as you can perform these basic repeated operations through an ORM tool. However, when you have to build a report, for instance, that requires desperate information data, and some other calculations performed, etc. and that the result is pretty complex to compute, then you better let the database engine work this out as it is designed to compute such data.
In addition to what others have said about security, encapsulation, performance, etc. of stored procedures, I'd like to add that the usefulness of stored procedures increases with the richness of the stored procedure language.
I don't have much experience with MySQL, but as far as I know the stored procedure language is pretty limited.
T-SQL (in Microsoft SQL Server) is more capable, but has several shortcomings compared to full-featured programming languages. For example, it is not possible to declare a constant value in T-SQL, and until quite recently there was no exception handling, so error handling was a pain. There is no concept of a package, so all your code will be stand-alone procedures with no way to group them together except for a good naming convention. (Although it's true that you can write stored procedures in .NET languages.)
On the other hand, PL/SQL (in Oracle), is a full-featured programming language with complex data types, exception handling, packages for grouping procedures (with separate public and private sections), object types, and lots and lots of built-in packages that deal with everything from file access to compression and generating web pages. All that, plus seamless integration with the database and the SQL language. Entire applications can be built using PL/SQL, without "leaving the database", so to speak. Check out http://apex.oracle.com for an example of a massive (framework) application implemented in pure PL/SQL.
Let's say you insert two rows in different tables, the second insert requires the id from the first.
$sql = "INSERT INTO t1 (f1,f2...) VALUES (v1, v2...)";
mysql_query($sql, $conn);
$id = mysql_insert_id();
$sql2 = "INSERT INTO t2 (f1,f2,id,f3...) VALUES (v1,v2,$id,v3....);
mysql_query($sql2,$conn)l
You went to the database twice, two server request/response. If you can store the process of INSERT, #id=insert id, INSERT all on the server in my_proc, you only have to do so once.
$sql = "CALL my_proc(arguments)";
mysql_query($sql);
Think of them as procedures in any regular program. A way of encapsulating a chunk of logic under a single invocation method.
If you consider functions in programs useless, then this conversation ends here. If you think they're useful, then there's nowhere else to go either. Nothing forces you to use them, but they're available if you should choose to.
If I understand them correctly it's
merely queries that can be saved
directly into MySQL, so it'd be
useless for any web development team
to use them.
Even with that (limited) definition, you're implying that web development teams never have any need of querying a database? Really?
A well written set of stored procs can completely remove queries from your client application and replace all of that with calls to the procedures. Now, I'm not saying that that's the only way of doing things, or even the right way. There's quite the discussion about that going on overall. But it is a very valid way of using them.
So, useless for a webdev team?
In addition to all the answers given here, i would also like to point out that stored procedures are a way to save the execution plan of a query.
you may have a set of SQL statements you just call from your application, but each time you execute the query SQL server has no way of knowing that the query you just invoked is the exact same query that as the one you called a few minutes ago (which would happen very frequently in a web application). So SQL server has to repeat the all the processing again (build the query plan and execute it).
Now if the same query had been encapsulated within a stored procedure, SQL server would have saved the execution plan for that stored procedure so that each time you call the sproc, it doesn't have to recompile the execution plan all the time. (It may even cache the data based on the parameters passed to the sproc, but i dont know exactly how this works)
It is easier to performance tune code in stored procs than what most ORMs create. It is easier to use stored procs when there are multiple applications that access the same database and might need to do the same things. It is far easier to refactor databases when all code is in stored procs because you can easily see where the changes need to be made. It is easier to use stored procs for things that don't normally hook up to ORMs like SSIS or reporting applications. Using stored procs you can limit access to only what the proc does and not allow access directly to the tables or views. This is critical in enforcing internal controls on financial data for instance and helps prevent fraud.
I've written complex procs that were well over a 1000 lines long. Try getting an ORM to write that kind of SQL. Then try to get it so that it will run without timing out.
I'm running a website for a jewellery wholesaler.
The prices on all their products are calculated using the current bullion metal fixes that are updated every night.
Currently, for the website, the calculations are worked out via a php include function, which works fine under current circumstances.
There are around 10,000 products, but the prices are calculated in real-time (ie when the web page is requested). The calculations are simple, but there are lots of them (Around 50+) and i'm worried that an increase in traffic may slow the current script down.
I'm redesigning the site and was wondering whether it would be beneficial to create a procedure in MySQL to do the calculations instead.
Is this likely to be faster that the current php script?
Anyone know any good reading reference on using procedures?
Here's a benchmark with stored procedure vs php.
http://mtocker.livejournal.com/45222.html
The stored procedure was slower by 10x.
You might also want to look at this:
http://www.tonymarston.net/php-mysql/stored-procedures-are-evil.html
If the reason you are thinking about this is due to performance and scalability, then I would recommend continuing the calculation in PHP.
The reason for this is that regardless whether there is a performance penalty in your PHP, when you are scaling your web application it is generally much easier to move to multiple web servers than multiple database servers. It is therefore preferable to do more calculation in PHP and less in MySQL.
Other than the performance aspect, I still generally prefer avoiding stored procedures in favour of having the logic in the application because
It can be less portable. Stored procedure add to the effort required to deploy a new instance of your application.
They are written in a different language than PHP, so a PHP developer may not find them easy to understand.
It can be difficult to have them kept in source control.
These problems can of course all be solved, without too much difficulty, but it all adds to the complexity overhead.
If it is absolutely necessary to update the prices on every page request and you're worried that the site will be getting a lot of traffic I wouldn't recommend stored procedures.
I'd recommend caching the information you use (and it is hard to elaborate without knowing how you're doing this) in memory (perhaps using memcached) and keep reading it from PHP.
I'll admit that I haven't done any benchmarking between stored procedures vs in-memory PHP performance but if the procedure doesn't directly affect your query, I recommend caching.
In short, keep them in php. Easier to maintain.
For the current site, there is unlikely to ever hit a performance problem where the difference between the speed of calc in php vs the speed of the calc in the database is ever noticeable. If you were then there is something fundamentally wrong with the code of the site. (This includes realtime currency conversions if being done).
Saying that, keeping the calc in PHP is usually preferred as it is easier to control and debug. It does require the web coders to know the database somewhat but that is normally not a problem. 90% of the code speed ups happen on 10% of the code and it would be easy enough for a dba to identify the queries causing db load if it ever happened.
I am using MySQL and PHP for a project I am working. I have created separate users for carrying out different functions (one for running select queries, one for running update queries, etc.) to provide an extra layer of security. This way, I figure if someone does manage to carry out an injection attack (which is unlikely since I am using prepared statements), whatever they are able to do will be limited to the type of query that was originally meant to be run. Is this a good practice or is it not worth the trouble?
Aside from the extended logic, you will also have different connections and essential overhead in that area.
IMHO it's wise to not do all your queries in a webapp with the root user and if the data is so hot, then make sure the designated user has no DROP, DELETE etc. priviledges. You could implement soft-delete if it's necessary in your application.
Last but not least, make sure to sanitize all GPC and make sure to properly quote/escape files in your queries. Using prepared statements can be one thing, but in the end it can be as simple as using mysql_real_escape_string() or whatever quoting-methods your DBAL/ORM offer.
I personally don't think it's worth the bother, since it's trickier to code, test and deploy. Make sure your software is immune to SQL injection instead.