I've experimented with a number of techniques for monitoring the health of our SQL Servers, ranging from using the Management Data Warehouse functionality built into SQL Server 2008, through other commercial products such as Confio Ignite 8 and also of course rolling my own solution using perfmon, performance counters and collecting of various information from the dynamic management views and functions.
What I am finding is that whilst each of these approaches has its own associated strengths, they all have associated weaknesses too. I feel that to actually get people within the organisation to take the monitoring of SQL Server performance seriously whatever solution we roll out has to be very simple and quick to use, must provide some form of a dashboard, and the act of monitoring must have minimal impact on the production databases (and perhaps even more importantly, it must be possible to prove that this is the case).
So I'm interested to hear what others are using for this task? Any recommendations?
RedGate's SQL Response is definitely a great tool for the job.
EDIT #1
There is also SQL IO, which tests the I/O of SQL Server. You will have some further information following the link provided.
There are other performance testing tools such as DBMonster which are open source (you need to scroll down).
I can't remember the name of the absolute testing tool I already referenced here on SO. I shall write it here when I found out.
Check out idera and Quest for additional tools/ideas.
[www.idera.com/Content/Home.aspx][1]
[www.quest.com/][2]
Don't forget the 2008 custom reports & Report Server for a roll-your-own.
Related
What's the advantage of using Google's Cloud SQL offering versus installing/maintaining instances yourself?
It seems to me that most of the features they list can be implemented fairly trivially by experienced mysql admins. Am I wrong?
CLARIFICATION: I am not asking which product is better. I am trying to understand what features Cloud SQL adds on top of self-hosted MySQL installations.
You can look at these types of questions from several angles: money, maintenance effort and quality of service.
Money-wise there is probably not that much of a difference, although it would not hurt to make the calculations for your usage scenario just to be sure.
In terms of maintenance-effort you are probably better of with using the Cloud SQL since most mundane tasks (such as creating back-ups and upgrading versions) are already handled for you.
Depending on your context, the quality of service of a 'hand-maintained' instance might be good enough, but if you need to support multiple sites then it quickly becomes a nuisance to create proper replications across the world (which from the looks of it is a standard feature of Cloud SQL).
Overall I would say it is greatly a matter of convenience.
Until now, I have worked primarily with LAMP applications, but have a project in which I need to set up two-way syncing between two SQL 2008 servers (one is web-based, the other is inside a firewall). I am finding a plethora of detailed information, but am having trouble answering some basic questions (in bold below).
As I understand it based on my reading to this point:
Replication is an end-to-end solution that can solve many common problems but is not as flexible as other options.
Change Tracking (SQL >= 2008) can record information about DB transactions and store the information in system tables. This information can be accessed via standard SQL queries and used in conjunction with with other tools to build a complete syncing solution.
Sync Framework is a framework that can be used in conjunction with programming languages (and change tracking) for a complete and flexible syncing solution.
SQL Server Integration Services (SSIS) is typically used when integrating SQL Server data with another data source. The development environment is easy to learn and fast to develop in. I believe this is also an end-to-end solution, but it can also be used in conjunction with other programming languages (creating new tasks, etc.).
Based on those understandings, here are my solutions:
I have some fairly complex logic for resolving syncing conflicts, which rules out replication (merge replication may be able to handle the logic, but it sound like the overhead would be much greater than with other solutions available in SQL 2008).
Being new to SQL Server, I really like what I have read about the ease of using SSIS, however most articles/examples I have read indicate it is primarily for integrating SQL Server with other data sources. Is SSIS an appropriate solution for syncing? I would also like to use the data recorded by 2008's Change Tracking, can I access this data through SSIS?
The other option would be using the Sync Framework and Change Tracking, which would involve a much steeper learning curve. What are the benefits of using this combination rather than SSIS and Change Tracking?
All comments on my understanding of MS syncing technologies and my solutions are appreciated.
Chris
Found a nice summary that explains when to use each technology here
Microsoft's Complex Event Processing (CEP) offering StreamInsight was released in tandem with SQL Server 2008 R2.
I am wondering:
if StreamInsight is in any way tied
to the SQL Server 2008 R2 database and
if it can be run separately.
(From scanning over the documentation
it looks like StreamInsight can be
run separately but clarification would be desirable).
If you have experience using StreamInsight separate from SQL Server 2008 R2 how has the experience been?
Thanks!
Responding to Tom's issues ...
Lacks query code reuse options
How so? First, you have Query Templates, which allow for query code reuse right out of the box. Second, you can create an architecture (as we have) that encapsulates related sets of queries in a provider model and then hook them up to various input and output adapters.
Adapters are constrained to pulling rather than pushing
Again, how so? The input adapters push data into the StreamInsight engine. Whether or not the data gets pushed into the input adapter depends on the source. We've created an OPC-DA input adapter that uses subscriptions and has data pushed from OPC. OSISoft's PI adapter works the same way.
No fail-over mechanism, no persistency guaranteed.
That is true.
Window logic is not as straightforward as other engines
That may be your opinion based on what you've used before and what you are familiar with. I've found StreamInsight's model to be relatively straightforward, flexible and powerful. I also find the query language (LINQ) much easier to use and very natural for a C# developer.
On another note, most of the other CEP engines out there are a very "heavy" install and can have a significant impact on an enterprise architecture - this from someone that's seen several installations of TIBCO do very unpleasant things to a network. StreamInsight is very lightweight and has a very small footprint compared to other engines.
Just my $.02 worth.
"As for your thoughts regarding StreamInsight (SI) and SQL Server 2008 R2; at least at this moment, SI has no dependencies on SQL Server 2008 R2 what so ever. For certain operations you may need a SQL Server Compact (CE) db, which stores metadata – but you can definitely use SI against other databases than SQL if you so wish." Niels of http://www.sqldevelop.com/
StreamInsight have some limitations in the event types, he accept simple (very simple) variable types. In case of an OPC server it is useless ...
my experience with streaminsight has been in a POC we've made here.
this product is not mature enough in my opinion to be a production infrastructure.
Our company needs to synchronize two sql server 2008 databases on two different servers.
The database schemas are about 50% different so transformation is needed.
Synchronization needs to be done in real time.
Synchronization needs to be bi-directional.
What are some good practices used for this purpose?
We have analyzed the following solutions and they didn't work for us
Microsoft Sync Framework. This option doesn't work because of the amount of time required to set up the framework (specifically the metadata tables/triggers/sprocs that the framework uses). It is also a newer framework so documentation/examples are scarce and the product might not be as stable as some other solutions.
SQL Server Integration Services. This solution has a learning curve and possible road blocks. It also seems to be too much for just this purpose.
Any help is greatly appreciated.
SQL Server Replication.
Create views in both databases to emulate the 50% of the other side that is different. These, along with the tables that still match will server as your publication source. For the tables that match, just set up two-way replication.
For the tables that do not match use the emulations views as the publication sources. Add a "SourceID" to their base tables to identify what server they were originally created on, and then setup replication filters to insure that no server ever receives a row that it originally created. Publish these views to the other server(s) one-way only. You may need to make these as indexed views in order for it to work (sorry, I can't remember).
I've used both and I've found MySql to have several frustrating bugs, limited support for: IDE integration, profiling, integration services, reporting, and even lack of a decent manager. Total cost of ownership of MSSQL Server is touted to be less than MySQL too (.net environment), but maintaining an open mind could someone point out any killer features of MySql?
I've used MySQL in the past and I'm using MSSQL lately but I can't remember anything that MySQL has and MSSQL can't do.
I think the most killer feature of MySQL it's the simplicity. For some projects you just don't need all the power you can have with a huge system like MSSQL. I have an UNIX heritage and find the simple configuration file like my.ini a killer feature of MySQL.
Also the security system of MySQL is much less robust but it makes the job right for most of applications. I believe MySQL it's killer itself from this point of view, and should stay that way, letting young users being introduced to RDBMS with a simple view first. If your project gets big enough that you are considering switch to a more robust system, then MSSQL can pop as a possibility.
That's what happened to me.
The only thing I can think of, off hand, is locking. SQLServer has traditionally had poor locking strategy that has tripped many people up.
You should use what you prefer, ultimately. Its not as if MySQL is not good enough to compete with MS SQL, eg. Slashdot uses MySQL, so its hardly got problems with high-scalability performance.
Its killer feature though, is that it is free - you can deploy as many of them without worrying one fig about licensing issues. That's more important for the spread of software than anyone could imagine.
(TCO is a difficult thing to calculate - and is advice only ever given from paid consultants and other vested interests. Ignore that. MSSQL is expensive and MySQL is free.)
About 6 years ago I developed a custom e-commernce website using ASP and MySQL for the database. At the time MySQL was clearly a better choice than MSDE which had built in throttling which concerned me enough to use MySQL. Also the difference in coding between using MySQL and MSDE/SQL was not that different or much of a concern.
Now all these years later I'm trying to get the code converted to .NET and even after purchasing commercial MySQL drivers from CRLab. I found that, as you hinted, the IDE integration is just not up to par.
I will say that MySQL is doing a great job even with our database tables approaching 4GB. So when I switch to MSSQL I have to go ahead and get SQL Workstation or higher ($$$), and not use SQL Express which has a 4gb limit.
All of my experience has changed the way I develop new websites. Now, unless it is expected to have a lot of traffic. I use VistaDB and then upgrade to SQL Server if needed. VistaDB is syntax and datasource compatible with SQL Server. And the best part is it is only a single file for the database and a dll for your bin folder.
That's my two cents based on my personal experience with using MySQL in ASP and now .NET.
I work with MSSQL, MySql and PostGres regularly (using .net, java and PHP). One of my favorite things about about MySQL (esp. compared to MSSQL) is the ease with which you can run and restore full database backups.
MSSQL's model of using .bak files is really ugly and time-consuming (topic for another post.) But if you want to do somethign like automated testing, or automated build processes (that include building a db from scratch), MySQL can be a bit easier to deal with.
A few other points:
The management tools have gotten a lot better since the early days.
If you are interested in transactions, constraints, etc.. be sure you are defining your tables to use the InnoDB storage engine (instead of MyISAM which is designed for speed.)
I do miss MSSQL's schema generating tool, but I think there are equivalent tools out there.
We've used a Linux database server and a window's web server (for .net apps) with great success.
If you are using something like NHibernate or some other non-MS data abstraction layer, the case to look beyond MSSQL is stronger too...
Three points to consider; unfortunately the first two are contradictory:
1) .NET and MySQL were not designed to interact with one another, and there is no official support from either side. You're invariably going to encounter issues trying to use them together.
2) If portability off of Windows may ever be an issue (much .NET code runs quite nicely on other platforms via Mono), you'll want to avoid locking yourself too deeply to MSSQL. That doesn't mean not using it, but being careful that you don't rely on its particular quirks too much.
3) TCO is just a buzzword. It's complete nonsense when it's calculated by anyone other than you. Nobody can make such a calculation and honestly claim that it has any relevance outside their particular environment. There are too many factors, most of which have absolutely nothing to do with things like tool availability.
I've been using the community version of MySQL for alsmost 99% of my project. I like MySQL is that I can deploy via Xcopy and is powerful compare to other "xcopy-able" database server. I also wrote a wrapper to start and stop MySQL & Apache (like LAMP), but with my own implemetation and addon capability
MySQL probably has a lower TCO, since administration and configuration is more simple and straightforward than the Spaghetti GUI that MS SQL makes you do most of the configuration through, having to dig through hundreds of obscure properties dialogs to accomplish even basic administration tasks.
There is one area where MS SQL clearly excels over MySQL in my experience:
Integration with other technologies. MS SQL allows you to replicate back and forth with Oracle and MySQL databases, and provides SSIS for executing scheduled data transformations from other database servers.
There may be others, but I don't have experience with them.