Rails and CouchDB - Architectural Concerns - mysql

I am working on a project that is going to use CouchDB for flexible storage of documents. The requirements of my system are a neat match for CouchDB for storage.
BUT
My question really boils down to this:
Should I keeop using ActiveRecord and MySQL as well ... there are a raft of handy Plugins that are all readily available for use with ActiveRecord (such as authentication and access control). Just wondering if the advantages of leveraging existing plugins is worth the extra management overhead and possible integration issues (working across disparate datastores).

It is not uncommon to have to deal with several persistent stores in a single application. A very common approach is to use a relational database that stores paths pointing to files that are stored in a file system.
So you might think as CouchDB as a special "file system" for a special part of your data model.
Also, in larger applications, multiple stores and complex physical architectures are quite common, so don't be shy of using more than one persistent store for your models.

You can use both; Some models can still be ActiveRecord, and others can be CouchDB.

Related

Mongo db vs (My Sql OR Sql Server) for a classified application

We have a classified website. I read few things but still not got a solid reason to choose between these.
My requirement is to just create user, listing and manage these. These are paid listings so monetary transactions would be there.
As per my research (https://www.mongodb.com/compare/mongodb-mysql) I found that mongo db has edge over other options except for complex transactions. What are complex transaction in this context? Any specific examples?
What factors should I consider before choosing one. I have experience with My SQL and Entity framework so its more appealing to me now but are these really bad for scaling when compared with mongo?
Feel free to suggest any other good performance option apart from these choices.
There is a vast difference between mongoDB and MySQL or SQL Server.
MongoDB is a NoSQL database meaning it is schemaless and doesnot follow CODD Rules of RDBMS whereas MySQL/SQL Server is relational Database.
Each are meant for different use cases. As per mongoDB doc
Why use MongoDB instead of MySQL?
Organizations of all sizes are adopting MongoDB because it enables them to build applications faster, handle highly diverse data types, and manage applications more efficiently at scale.
Development is simplified as MongoDB documents map naturally to modern, object-oriented programming languages. Using MongoDB removes the complex object-relational mapping (ORM) layer that translates objects in code to relational tables.
MongoDB’s flexible data model also means that your database schema can evolve with business requirements. For example, the ALTER TABLE command required to add a single, new field to Craiglist’s MySQL database would take months to execute. The Craigslist team migrated to MongoDB because it can accommodate changes to the data model without such costly schema migrations.
MongoDB can also be scaled within and across multiple distributed data centers, providing new levels of availability and scalability previously unachievable with relational databases like MySQL. As your deployments grow in terms of data volume and throughput, MongoDB scales easily with no downtime, and without changing your application. In contrast, to achieve scale with MySQL often requires significant, custom engineering work.
What are common use cases for MongoDB?
MongoDB is a general purpose database that is used for a variety of use cases. The most common use cases for MongoDB include Single View, Internet of Things, Mobile, Real-Time Analytics, Personalization, Catalog, and Content Management.
When would MySQL be a better fit?
While most modern applications require a flexible, scalable system like MongoDB, there are use cases for which a relational database like MySQL would be better suited. Applications that require complex, multi-row transactions (e.g., a double-entry bookkeeping system) would be good examples. MongoDB is not a drop-in replacement for legacy applications built around the relational data model and SQL.
A concrete example would be the booking engine behind a travel reservation system, which also typically involves complex transactions. While the core booking engine might run on MySQL, those parts of the app that engage with users – serving up content, integrating with social networks, managing sessions – would be better placed in MongoDB
As there is no concept of join in MongoDB so if you want to store data, either you have to create a hack to store data in two table(although it is expensive cal to join in MongoDB) or you have to store data as a single document,, making your document too complex.In these cases MySQL has upperhand.

Ejabberd Redis + MySql database combination

I am newbie to the Ejabberd so I am still exploring all the possibilities and possible setups for chat server.
From the documentation I have seen that Ejabberd supports Redis database for transient data, user session I suppose...
I was wondering has anyone had any experience using Redis for storing transient data and then MySql for the rest of the data? Will this setup be beneficial comparing to Mnesia + MySql one? Maybe Redis + Riak is even better setup?
Just looking for some general opinions since I am a newcomer in this area...
Full disclosure: I work for Basho, the maintainers of Riak, so I have a clear preference here.
Looking at the source of Ejabberd, I see it's written in Erlang as well, which is optimized as a distributed system. Their architecture diagram specifically shows Riak as a NoSQL backend. Redis is often paired with Riak due to its simple retrieval and key/value design. If scale is a concern on the transient side as well, you could use Riak's in-memory backend alongside the disk-based backend for durable data (more on backends here).
Riak is designed for scaling, so if you anticipate growth beyond a single server's worth of CPU, memory or storage, then it's perfect. If you do not anticipate this growth, then Riak may be overkill. For more on when to use it, read this.

MongoDB for small datasets

Are there any benefits to using MongoDB for a Node.js application rather than a traditional SQL database such as MySQL, if I'm not planning to have large (>1000 item) collections and am already comfortable with SQL?
MongoDB is schema-less document based database. This means you can insert a JSON object with other nested objects. This can make development easier, especially for prototyping.
For a small project, why not? For a larger project you should do more research. Large or small, doesn't hurt to do the research anyway. You want to consider how your application uses the database (reads vs writes) and how MongoDB scales horizontally, and how it handles failures.
There's a thing called the CAP theorem that defines NoSQL databases. MongoDB is CP. This visual guide shows the relationships between different databases. What is most important to you and your application?
Something else to consider is that most NoSQL databases are not ACID compliant. If you're using MySQL with InnoDB, that can be something significant to give up, depending on your application. For example, transactions might be something you might not want to give up.
Lots of pros and cons. Best thing to ask yourself is: What am I gaining? What am I giving up? There are many things, and it really depends on your use-case.
There are lots of reasons to stick with a simple dbms for a small-scale application. One of them is the widespread availability of cheap hosting services providing MySQL. Another is ease of deployment and maintenance.
Of course, if you're trying to learn to use MongoDB, go for it!

NoSQL Database Design

I'm looking at SQL & NoSQL Databases - namely MySQL and DynamoDB (both at AWS).
I'm building a dating/social network and demos I've built have been using a MySQL Database with around 50 tables for logical separation of data and then using SQL queries (often with joins) to extract required chucks of data to send back to browsers.
I'm moving to AWS and are doing a rebuilt of the system and wanted to know if it would be possible to write a site like this 100% in NoSQL. I understand you don't know the specifics of the site but it could be compared to any other dating/social network like facebook (obviously more involved) or Eharmony/Match Maker etc...
Could a Social Site be built 100% on NoSQL? or would a mix of NoSQL & SQL be move realistic?
thx
It's a very difficult question to answer without a deeper understanding of exactly what features you're after, and what language you are going to be writing the site in. There are lots of different types of NoSQL solutions.
NoSQL databases like Dynamo and Cassandra are Key-Value Stores. They offer a very different set of features than Document Databases like MongoDB and RavenDB. There are many other types as well.
Personally, I would be more than comfortable writing a social media site based entirely on RavenDB. But that's because I tend to focus on Domain Driven Design, and like to write in .Net/C#. It has all the features you would need, like querying indexes, map/reduce for big data jobs, full-text search, and spatial distance proximity searches. You could use their http/rest api if you wanted to program from php or javascript, but their C# client is much easier to use.
Your requirements may be different than mine would be though. I would encourage you to try out several different NoSQL technologies before you settle on one. You may still find that you need a SQL (or MySQL) database for certain things that your NoSQL solution doesn't handle. For example, RavenDB isn't recommended for ad-hoc reporting - so many people set up a separate SQL Server database and replicate data from Raven into SQL so they can provide a separate reporting database to their power users.
The biggest thing to remember is that most noSQL engines (like Cassandra) don't support querying, so that has to be a factor in your design (i.e. many things you take for granted in SQL like JOINs are much harder in a noSQL solution). With that being said, you most definitely can build full-featured applications using a noSQL solution. I encourage you to look into resources available from the many providers out there, like Cassandra, MongoDB, Dyanamo, and many others.

What would use for versioned (large) text storage for the web app backend?

Arguing with a friend of mine — I am advocating git/hg, he is advocating CLOBs MySQL. What is your preference?
I believe that totally and completely depends on your requirements. If my main requirement is to simply keep a versioned copy, so there is always a backup, then you can use GIT or CVS or Subversion, whatever you like. However, if I'm storing large documents for business users, that are frequently accessed and need to be searched then storing in a database is probably the best way. I'm not too familiar with MySQL, but in my Oracle experience there is a lot less guessing when I store large business documents in a BLOB column. That combined with the fact, Oracle can search text index those BLOB's for me, made it an attractive solution for the application I wrote for my place of business.
You should follow the golden rule to organize your data storage depending on the intended queries you are going to make. So, it can be just plain files on a NAS, git or mercurial backends, a database or a custom solution. It depends on the way / frequency you will query / update them.
So given your lack of context, there is no way to choose.
Depending on how large large is and how many of them you will have I think you will want to avoid storing them in any database. Version control and file systems are VERY good at storing large digital objects. Relational databases aren't.
Usually, databases are easier to interface with than a tool. You may want to look into using something like CouchDB or Apache Jackrabbit which does document versioning.