Microsoft's Complex Event Processing (CEP) offering StreamInsight was released in tandem with SQL Server 2008 R2.
I am wondering:
if StreamInsight is in any way tied
to the SQL Server 2008 R2 database and
if it can be run separately.
(From scanning over the documentation
it looks like StreamInsight can be
run separately but clarification would be desirable).
If you have experience using StreamInsight separate from SQL Server 2008 R2 how has the experience been?
Thanks!
Responding to Tom's issues ...
Lacks query code reuse options
How so? First, you have Query Templates, which allow for query code reuse right out of the box. Second, you can create an architecture (as we have) that encapsulates related sets of queries in a provider model and then hook them up to various input and output adapters.
Adapters are constrained to pulling rather than pushing
Again, how so? The input adapters push data into the StreamInsight engine. Whether or not the data gets pushed into the input adapter depends on the source. We've created an OPC-DA input adapter that uses subscriptions and has data pushed from OPC. OSISoft's PI adapter works the same way.
No fail-over mechanism, no persistency guaranteed.
That is true.
Window logic is not as straightforward as other engines
That may be your opinion based on what you've used before and what you are familiar with. I've found StreamInsight's model to be relatively straightforward, flexible and powerful. I also find the query language (LINQ) much easier to use and very natural for a C# developer.
On another note, most of the other CEP engines out there are a very "heavy" install and can have a significant impact on an enterprise architecture - this from someone that's seen several installations of TIBCO do very unpleasant things to a network. StreamInsight is very lightweight and has a very small footprint compared to other engines.
Just my $.02 worth.
"As for your thoughts regarding StreamInsight (SI) and SQL Server 2008 R2; at least at this moment, SI has no dependencies on SQL Server 2008 R2 what so ever. For certain operations you may need a SQL Server Compact (CE) db, which stores metadata – but you can definitely use SI against other databases than SQL if you so wish." Niels of http://www.sqldevelop.com/
StreamInsight have some limitations in the event types, he accept simple (very simple) variable types. In case of an OPC server it is useless ...
my experience with streaminsight has been in a POC we've made here.
this product is not mature enough in my opinion to be a production infrastructure.
Related
I am confused about the database I should choose, since the application I am doing will store and handle large amounts of data, I think SQL Server will be better than MySQL. Any recommendations?
Both platforms SQL Server and MySQL can handle small and large software projects, so you should anticipate similar performance from both, provided the database designer and programmer are familiar with the right way to optimize queries and code.
Structured query language (SQL) is the language of relational databases. Several database platforms use SQL, but a slight variation on it—each tend to have a slightly different syntax. Microsoft SQL and MySQL are two of the most common database platforms on the web. Once you go with one, it can be very difficult to switch to the other. That’s because the database platform you choose will end up being the core of your dynamic content moving forward. It stores, secures, and retrieves all the data for your applications.
It’s an important decision to make, and will likely hinge on a few things. If you’re having a hard time deciding which one is right for your project, here’s a look at some similarities and differences between the two, SQL and MySQL.
THE BASICS
Whether you want to store, retrieve or edit your data—the way dynamic websites and applications perform nearly every request a user makes—SQL is the language of choice for relational databases. On the surface, both Microsoft and MySQL look similar:
They both give you the ability to host several databases on one
server.
They use tables to store data.
They have primary and foreign key constraints.
They use indexes to sort data and speed up performance, and they both support desktop and web applications.
SQL Server is slightly older than MySQL. Microsoft SQL Server was introduced in 1989 and MySQL was introduced in 1995 as an open-source project. Since both of them have been in production for years, they both have a firm foothold in the market. MySQL runs on either Windows or Linux, typically as a part of a LAMP environment. SQL Server runs on Windows, and is usually a part of a Windows environment.
Both platforms handle small and large software projects, so you should anticipate similar performance from both, provided the database designer and programmer are familiar with the right way to optimize queries and code.
Microsoft SQL Server vs. MySQL: Similarities
Both Microsoft and MySQL are relational database platforms, so they have several similarities. Most developers specialize in either one or the other, because although they appear similar, the way they work in the underlying architecture is very different. Here are some similarities, which make it somewhat easy for a database developer to work on both platforms efficiently, even if they specialize in only one.
Scalability: Both platforms allow you to scale as your business grows. You can use both for small projects, however should these projects take off to an enterprise-level, they can still support millions of transactions a day.
High-performance: A database is your application’s backbone. It stores your all of data, so you need a database that can return data in less than a second. Both platforms can handle this type of high-performance speed.
Tables: Both platforms use the standard relational database table model to store data in rows and columns.
Keys: Both platforms use primary and foreign keys to establish relationships between tables.
Syntax: Syntax between the two database platforms are similar, although there are some minor differences across different CRUD (create, read, update, delete) statements.
Web-based popularity: Aside from Oracle, Microsoft SQL Server and MySQL are the most common databases used for web applications. When you sign up for hosting, you typically get a choice between MySQL databases or SQL Server.
Drivers: You can find connection drivers for almost any popular language on the web, so you can easily connect to both platforms without writing complex code.
MICROSOFT SQL SERVER VS. MYSQL: DIFFERENCES
While the two platforms are similar in the interface and basic relational database standards, they are two very different programs and operate differently. Most of the differences are in the way they operate in the background, and these differences are not seen by the average user. As long as the database performs well, it can be used with your project. However, it’s still important to know these differences because they’ll play a huge role in your developer’s choice platform.
Native compatibility: You can use either database with both Windows and Linux projects, but MySQL works natively with PHP and MSSQL is mainly used with .NET. It makes integration simpler if you stick with MySQL for PHP and MSSQL for Windows projects.
MyISAM and InnoDB: Both of these engines are configurations for MySQL and allow the developer to perform very different design and programming. With MSSQL, you create a database and don’t specify different engines.
Cost: SQL Server is generally expensive to run, because you need licenses for the server running the software. MySQL is free and open-source, but you’ll pay for support if you need it.
LINQ: With MSSQL, you can set up your entity framework classes in .NET and get started with LINQ queries. With MySQL and .NET, you need to download third-party provider tools.
IDE tools: Both platforms have IDE tools, but you need the right tool with the right server. MSSQL uses Management Studio and MySQL has Enterprise Manager. These tools let you connect to the server and manage settings and configurations for security, architecture, and table design.
WHICH ONE SHOULD YOU USE?
The database you use usually depends on the hosting environment you choose. Linux hosting providers usually offer MySQL. Since MySQL is open-source and free, you can have as many databases as you need. You can have 10 of them to support 10 different projects if need be.
Because SQL Server costs money for licenses, Windows hosts will give you one MSSQL database and you must pay for additional ones. Overall, this makes SQL Server more costly than MySQL. However, SQL Server works natively with .NET applications, so it’s the choice for software that runs on a Windows server or desktop. The development tools are free, but the production environment is not free.
The best way to determine the right platform is to first post your project in the marketplace and discuss your requirements with a few developers. You will get different opinions and preferences based on the developer’s area of expertise, but most Windows developers work with MSSQL and Linux developers work with MySQL. You should decide which environment you want to target, and then you can get a clearer idea of which platform is right for you.
Source
sql-vs-mysql-which-relational-database-is-right-for-you
Both Mysql and mssql can handle big data. but mysql is better in this ways
security: if You choose Mssql u'll host on windows server which is considered less secure.
NB: SQL Server 2017 available on Linux. but most hosting companys haven't updated it
Cost: hosting Companies will give little free database space on Mssql compared to mysql.
example in godaddy MSSQL 1 database x 200MB(economyPlan) 2 x 200MB(deluxe) Unlimited x 200MB(ultimate).
MySQL databases 10 x 1 GB(economy) 25 x 1 GB(deluxe) Unlimited x 1 GB(ultimate).
If your not sure, then you probably don't have alot of experience. I would suggest you start with MySQL assuming you have Linux available. It is much more forgiving and allows you to quickly re-start the database if you mess something up. SQL Server is almost impossible to stop and restart without rebooting the windows server, which is a long process and could bring your whole system down for 10 minutes or more. A Mysql restart in linux takes 2 seconds or less.
Sql Server is MUCH more expensive but it has many more tools and features.
If you are looking to use this as a backend to get more development work, SQL server DB's generally get more money, and it is used more in corporate world. Microsoft has extensive tools and addins with SQL server, much more than my sql.
It is interesting to note that SQL Server can connect to MySQl to allow you to write queries in SQL server that directly query Mysql (this is called a linked server) , but the reverse is not possible. So in the future, you could split up your databases. AS a general rule of thumb, if you have website traffic coming in AND backend processing, create multiple databases, even if they are on the same server, so you can move one of them off to another server when your the database gets busy.
Until now, I have worked primarily with LAMP applications, but have a project in which I need to set up two-way syncing between two SQL 2008 servers (one is web-based, the other is inside a firewall). I am finding a plethora of detailed information, but am having trouble answering some basic questions (in bold below).
As I understand it based on my reading to this point:
Replication is an end-to-end solution that can solve many common problems but is not as flexible as other options.
Change Tracking (SQL >= 2008) can record information about DB transactions and store the information in system tables. This information can be accessed via standard SQL queries and used in conjunction with with other tools to build a complete syncing solution.
Sync Framework is a framework that can be used in conjunction with programming languages (and change tracking) for a complete and flexible syncing solution.
SQL Server Integration Services (SSIS) is typically used when integrating SQL Server data with another data source. The development environment is easy to learn and fast to develop in. I believe this is also an end-to-end solution, but it can also be used in conjunction with other programming languages (creating new tasks, etc.).
Based on those understandings, here are my solutions:
I have some fairly complex logic for resolving syncing conflicts, which rules out replication (merge replication may be able to handle the logic, but it sound like the overhead would be much greater than with other solutions available in SQL 2008).
Being new to SQL Server, I really like what I have read about the ease of using SSIS, however most articles/examples I have read indicate it is primarily for integrating SQL Server with other data sources. Is SSIS an appropriate solution for syncing? I would also like to use the data recorded by 2008's Change Tracking, can I access this data through SSIS?
The other option would be using the Sync Framework and Change Tracking, which would involve a much steeper learning curve. What are the benefits of using this combination rather than SSIS and Change Tracking?
All comments on my understanding of MS syncing technologies and my solutions are appreciated.
Chris
Found a nice summary that explains when to use each technology here
I've experimented with a number of techniques for monitoring the health of our SQL Servers, ranging from using the Management Data Warehouse functionality built into SQL Server 2008, through other commercial products such as Confio Ignite 8 and also of course rolling my own solution using perfmon, performance counters and collecting of various information from the dynamic management views and functions.
What I am finding is that whilst each of these approaches has its own associated strengths, they all have associated weaknesses too. I feel that to actually get people within the organisation to take the monitoring of SQL Server performance seriously whatever solution we roll out has to be very simple and quick to use, must provide some form of a dashboard, and the act of monitoring must have minimal impact on the production databases (and perhaps even more importantly, it must be possible to prove that this is the case).
So I'm interested to hear what others are using for this task? Any recommendations?
RedGate's SQL Response is definitely a great tool for the job.
EDIT #1
There is also SQL IO, which tests the I/O of SQL Server. You will have some further information following the link provided.
There are other performance testing tools such as DBMonster which are open source (you need to scroll down).
I can't remember the name of the absolute testing tool I already referenced here on SO. I shall write it here when I found out.
Check out idera and Quest for additional tools/ideas.
[www.idera.com/Content/Home.aspx][1]
[www.quest.com/][2]
Don't forget the 2008 custom reports & Report Server for a roll-your-own.
Our company needs to synchronize two sql server 2008 databases on two different servers.
The database schemas are about 50% different so transformation is needed.
Synchronization needs to be done in real time.
Synchronization needs to be bi-directional.
What are some good practices used for this purpose?
We have analyzed the following solutions and they didn't work for us
Microsoft Sync Framework. This option doesn't work because of the amount of time required to set up the framework (specifically the metadata tables/triggers/sprocs that the framework uses). It is also a newer framework so documentation/examples are scarce and the product might not be as stable as some other solutions.
SQL Server Integration Services. This solution has a learning curve and possible road blocks. It also seems to be too much for just this purpose.
Any help is greatly appreciated.
SQL Server Replication.
Create views in both databases to emulate the 50% of the other side that is different. These, along with the tables that still match will server as your publication source. For the tables that match, just set up two-way replication.
For the tables that do not match use the emulations views as the publication sources. Add a "SourceID" to their base tables to identify what server they were originally created on, and then setup replication filters to insure that no server ever receives a row that it originally created. Publish these views to the other server(s) one-way only. You may need to make these as indexed views in order for it to work (sorry, I can't remember).
I've used both and I've found MySql to have several frustrating bugs, limited support for: IDE integration, profiling, integration services, reporting, and even lack of a decent manager. Total cost of ownership of MSSQL Server is touted to be less than MySQL too (.net environment), but maintaining an open mind could someone point out any killer features of MySql?
I've used MySQL in the past and I'm using MSSQL lately but I can't remember anything that MySQL has and MSSQL can't do.
I think the most killer feature of MySQL it's the simplicity. For some projects you just don't need all the power you can have with a huge system like MSSQL. I have an UNIX heritage and find the simple configuration file like my.ini a killer feature of MySQL.
Also the security system of MySQL is much less robust but it makes the job right for most of applications. I believe MySQL it's killer itself from this point of view, and should stay that way, letting young users being introduced to RDBMS with a simple view first. If your project gets big enough that you are considering switch to a more robust system, then MSSQL can pop as a possibility.
That's what happened to me.
The only thing I can think of, off hand, is locking. SQLServer has traditionally had poor locking strategy that has tripped many people up.
You should use what you prefer, ultimately. Its not as if MySQL is not good enough to compete with MS SQL, eg. Slashdot uses MySQL, so its hardly got problems with high-scalability performance.
Its killer feature though, is that it is free - you can deploy as many of them without worrying one fig about licensing issues. That's more important for the spread of software than anyone could imagine.
(TCO is a difficult thing to calculate - and is advice only ever given from paid consultants and other vested interests. Ignore that. MSSQL is expensive and MySQL is free.)
About 6 years ago I developed a custom e-commernce website using ASP and MySQL for the database. At the time MySQL was clearly a better choice than MSDE which had built in throttling which concerned me enough to use MySQL. Also the difference in coding between using MySQL and MSDE/SQL was not that different or much of a concern.
Now all these years later I'm trying to get the code converted to .NET and even after purchasing commercial MySQL drivers from CRLab. I found that, as you hinted, the IDE integration is just not up to par.
I will say that MySQL is doing a great job even with our database tables approaching 4GB. So when I switch to MSSQL I have to go ahead and get SQL Workstation or higher ($$$), and not use SQL Express which has a 4gb limit.
All of my experience has changed the way I develop new websites. Now, unless it is expected to have a lot of traffic. I use VistaDB and then upgrade to SQL Server if needed. VistaDB is syntax and datasource compatible with SQL Server. And the best part is it is only a single file for the database and a dll for your bin folder.
That's my two cents based on my personal experience with using MySQL in ASP and now .NET.
I work with MSSQL, MySql and PostGres regularly (using .net, java and PHP). One of my favorite things about about MySQL (esp. compared to MSSQL) is the ease with which you can run and restore full database backups.
MSSQL's model of using .bak files is really ugly and time-consuming (topic for another post.) But if you want to do somethign like automated testing, or automated build processes (that include building a db from scratch), MySQL can be a bit easier to deal with.
A few other points:
The management tools have gotten a lot better since the early days.
If you are interested in transactions, constraints, etc.. be sure you are defining your tables to use the InnoDB storage engine (instead of MyISAM which is designed for speed.)
I do miss MSSQL's schema generating tool, but I think there are equivalent tools out there.
We've used a Linux database server and a window's web server (for .net apps) with great success.
If you are using something like NHibernate or some other non-MS data abstraction layer, the case to look beyond MSSQL is stronger too...
Three points to consider; unfortunately the first two are contradictory:
1) .NET and MySQL were not designed to interact with one another, and there is no official support from either side. You're invariably going to encounter issues trying to use them together.
2) If portability off of Windows may ever be an issue (much .NET code runs quite nicely on other platforms via Mono), you'll want to avoid locking yourself too deeply to MSSQL. That doesn't mean not using it, but being careful that you don't rely on its particular quirks too much.
3) TCO is just a buzzword. It's complete nonsense when it's calculated by anyone other than you. Nobody can make such a calculation and honestly claim that it has any relevance outside their particular environment. There are too many factors, most of which have absolutely nothing to do with things like tool availability.
I've been using the community version of MySQL for alsmost 99% of my project. I like MySQL is that I can deploy via Xcopy and is powerful compare to other "xcopy-able" database server. I also wrote a wrapper to start and stop MySQL & Apache (like LAMP), but with my own implemetation and addon capability
MySQL probably has a lower TCO, since administration and configuration is more simple and straightforward than the Spaghetti GUI that MS SQL makes you do most of the configuration through, having to dig through hundreds of obscure properties dialogs to accomplish even basic administration tasks.
There is one area where MS SQL clearly excels over MySQL in my experience:
Integration with other technologies. MS SQL allows you to replicate back and forth with Oracle and MySQL databases, and provides SSIS for executing scheduled data transformations from other database servers.
There may be others, but I don't have experience with them.