I've just been tasked with automating a reporting task at work. Previously, someone would run large, arbitrary SELECTs on a MySQL database using a GUI tool, then use that same tool to export the results to CSV. Now I want to write a Ruby script to do this.
I know about FasterCSV in Ruby, but as far as SQL queries, I've only used ActiveRecord, where you're generally not writing the queries but using models and associations. The last time I wrote out complete SQL in code was when I coded PHP.
What's the most straightforward way to do this sort of thing in Ruby? Should I use ActiveRecord?
If you are writing a complex application, there is absolutely nothing wrong with using ActiveRecord. Especially since you are already comfortable with it.
If you are writing a quick script and don't want to bother with ActiveRecord, you should also check out the mysql and mysql2 gems.
I highly recommend Sequel. It has great documentation, active development, a thriving and helpful community, and it is (IMHO) simpler and better than ActiveRecord, especially for simple use cases like you appear to have.
You might want to start by reading the README, Cheat Sheet and Sequel for SQL Users.
I often find that in reporting engines you end up with complex queries processing large amounts of data and ORMs like ActiveRecord just don't cut it. Unless your reports are really simple, I think you'll want to use raw SQL (with a tool like Sequel) or call stored procedures.
Related
I want to know that is there MYSQL Tool available in which we can modify the existing Stored Procedures by using GUI. I do not want to write code manually in the Stored Procedure.
(My)SQL is a scripting language (well, actually a query language, but for the purpose of this answer that comes pretty close). It's all about writing text to create commands/queries. How can this be done using a GUI (other than providing a text editor)? I have seen visual query builders in the past and they all suffer from the language's complexity. Most of the time it's easier to see the written text than a graphical view of the query. And as soon as you enter the world of non-trivial queries such query builders will be lost.
So, there's a reason that serious visual query builders are pretty uncommon among the popular MySQL GUI tools.
Is there a solution that helps you manage database schema patches, preferably that runs in Node. I am looking at node-db-migrate which has support for MySQL but I want to be able to run plain vanilla SQL on it instead of using wrapper functions offered by this package.
Is there a solution that offers the same functionality but also lets you have plain SQL in patches?
I'm personally a fan of Sequelize, which has a (poorly named) sister project called Umzug designed to handle migrations. You can absolutely run vanilla SQL in a package like this, but be very careful doing so. Running raw SQL can be an easy way to do something like populate a new column with data (perhaps a new summary column that needs to be filled with the results of a more complex query you've been running until now). However, it can also make it very easy to make mistakes if you do this for things like ALTER TABLE. Whenever you can, it's best to use provided constructs for things like that.
I am looking for a tool or a library for Rails to validate/parse queries that could be SQL and/or DDL. Currently, I did not find anything that I could use quickly and easily.
I found Parslet that I can use to define my own SQL/DDL language to validate SQL/DDL statements.
The goal to reach is to have a tool that we can use to validate the SQL/DDL syntax before any run on the database. For example, DDL queries are not transactional with MySQL and therefore, if one statement fails at the middle of a bigger script, we need to restore the database or run the script from the failure point (that is not really userfriendly). If we can, at least, validate the syntax, we will improve our daily work by removing a lot of "stupid" errors.
This post lists a few Ruby SQL parsers you might be interested in taking a look at. This one in particular has a Treetop grammar file you could probably use as a base for your own validations.
We're working on an ASP.NET web application with C# code behind. The database is MySQL 5.1 using InnoDB. The data access layer uses ADO.NET to call stored procedures and then builds various data structures out of the result sets (no object mapping). This works fine, but it is a little verbose.
Not surprisingly, we made some mistakes when designing the first version of our data model, but the experience has made us smarter and we decided to refactor the data model. We don't have to change our data access layer, but we are considering our options for that as well.
It's been difficult for us to ignore the popularity of ORM tools these days; we feel like we are way behind or something for not being familiar with them. Not only that, but we have already designed an object model that nicely describes our data model. The main ORM tools we would consider are NHibernate, ADO.NET Entity Framework, and LINQ to SQL. We would prefer LINQ to SQL because we have read (on S.O.) that it is more light weight than full ORM tools.
We think one drawback to using an ORM tool is the learning curve, but we can already see how using LINQ could reduce the amount of code we will have to write, which could save us time in the long run. However, we are using MySQL, not SQL Server.
So my question is, would DBLinq work well enough for a production system? Or, is LINQ to SQL a compeling enough reason for us to make the move to SQL Server 2008? Incedentally, I'd prefer to use SQL Server over MySQL, but cost is the obvious drawback. After 3 years on BizSpark, we'd be on the hook for $6K. Or, should we consider other ORM tools instead? Or, should we just ignore the hype and not use an ORM tool, but maybe take advantage of LINQ to DataSet?
I searched S.O. for info on DBLinq, but only found 17 questions with the DBLinq tag, so it doesn't appear to be popular.
EDIT
Looks at though dotConnect for MySQL has support for LINQ, so that's another option.
Can anyone speak to how well that driver works for writing LINQ queries?
Check bl-toolkit. It's free, very fast and has great LINQ support. Newest addition are T4 templates for generating your data model from database.
Is there any software for Windows that allows you to setup your tables and then tell the software what you want to SELECT given the conditions and it will tell you the SQL query that you need? Thanks.
Probably you'll find software that will do this for you in a GUI, but eventually you will be required to write the queries yourself, unless you're developing something really trivial.
A GUI tool for SELECT statements is a too-high level of abstraction for most scenarios, and you would have to face the "Law of Leaky Abstractions" eventually.
Visualization tools are quite handy for defining the database schema, and there are some very good tools for that. However queries remain quite tricky to visualize, in my opinion. I think you'll be able to mentally visualize queries, given enough practice. Nevertheless, I'm quite sure you will be able to get started doing SQL queries within a few days, even non-trivial ones.
In addition, you can use Stackoverflow if you require assistance with complicated queries. I'm quite sure that the community here will suggest better SQL than any GUI tool!
These sound like they will do what you want:
SQL Maestro
dbForge Query Builder for MySQL
Having said that, I do agree with the other answers here - learning to manually write SQL will give you a lot more control. I highly recommend the (free) MySQL Workbench for that.
The people at sqlmanager.net build some tools to deal with databases, but I agree with Daniel: if you have very complex queries to work with, it's probably more work telling the tool what to do that creating the queries manually - especially as you'll have to verify what the tool did, so if you can do that, you could have done it by hand in the first place :)
As said before by daniel vassallo, you should learn how to create those sql queries "by hand". where i work we use a program that does exactly this (it's used by accountants only) and it creates horrible sql select statements and we have constantly whining accountants that think that the problem is allways on the server side even when we show them that those queries get them the wrong data... it's a never ending circle :(
Use HeidiSQL to manage your database, and keep an eye on what's going on in the SQL log. You'll soon pick it up.