How can I log all executed SQL statements to the console/stdout when using Ruby MySQL?
Like the log/development.log output from rails.
If you mean Ruby/MySQL, that provides a debug() function which performs the same function as mysql_debug() -- if your client library has been compiled with debugging, you can get DBUG to trace things for you. This might give what you're after (with a bit of clean-up.)
Another approach would be to capture the MySQL packets using tcpdump and decode them with maatkit.
A third approach would be to alias Mysql.query and Mysql.real_query with your own functions that do logging. Something like this (untested! trivial example! doesn't handle blocks!):
class Mysql
alias_method :old_query, :query
def query(sql)
$stderr.puts "SQL: #{sql}"
old_query(sql)
end
end
Rails uses ORMs like ActiveRecord which has it's logger associated to it.
I don't think MySQL gem has a logger...
Related
Currently we're relying on Sphinx's PHP library to manage our faceted search, which depends on the ability to use Sphinx's multi-queries feature.
The latest Sphinx search documentation describes how to perform the same multi-query procedure in SphinxQL, via MySQL. It gives an example using PHP.
http://sphinxsearch.com/docs/manual-2.0.4.html#sphinxql-multi-queries
Do any MySQL gems exist for ruby that support multi-queries in this way?
I'm looking at the mysql2 gem, which seems to be the latest thing, but it doesn't appear to support it. Am I still at a loss when it comes to Sphinx multi-queries in ruby?
I'm going to write a client that supports them in the next few days at work anyway if not, but obviously SphinxQL would make this much easier. I'd also rather not have to make my gem connect to two different protocols for RT indexes (which can only be written to via SphinxQL). It seems like SphinxQL is basically where it's at.
It appears the ruby-mysql gem supports this: https://github.com/tmtm/ruby-mysql/blob/master/lib/mysql.rb#L406-419
I assume it is processing this correctly. It says 'execute', but in reality it appears to simply be fetching the next set of results from a query that was already executed.
# execute next query if multiple queries are specified.
# === Return
# true if next query exists.
def next_result
return false unless more_results
check_connection
#fields = nil
nfields = #protocol.get_result
if nfields
#fields = #protocol.retr_fields nfields
#result_exist = true
end
return true
end
There is reference to it here too: http://zetcode.com/db/mysqlrubytutorial/ (Under 'Multiple Statements')
I ended up writing my own gem, which includes a little wrapper around MySQL. Not a fully-fledged mysql client, but a minimal bridge in order to support SphinxQL.
You can see the gem here: https://github.com/d11wtq/oedipus
And the C extension here: https://github.com/d11wtq/oedipus/blob/master/ext/oedipus/oedipus.c
I am trying to insure that only one instance of a perl script can run at one time. The script performs some kind of db_operation depending on the parameters passed in. The script does not necessarily live in one place or on one machine, and possibly multiple OSs. Though the file system is automounted across the various machines.
My first aproach was to just create a .lock file, and do the following:
use warnings;
use strict;
use Fcntl qw(:DEFAULT :flock);
...
open(FILE,">>",$lockFilePath);
flock(FILE,LOCK_EX) or die("Could not lock ");
do_something();
flock(FILE,LOCK_UN) or die("Could not unlock ");
close(FILE);
but I keep getting the following errors:
Bareword "LOCK_EX" not allowed while "strict subs" in use
Bareword "LOCK_UN" not allowed while "strict subs" in use
So I am looking for another way to approach the problem. Locking the DB itself is also not practical since the db could be used by other scripts(which is acceptable), I am just trying to prevent this script from running. And locking a table for write is not practical, since my script is not aware of what table the operation is taking place, it just launches another perl script supplied as a parameter.
I am thinking of adding a table to the db, with just one value, and to use that as a muetex, but I don't know how practical/reliable that is(a lot of red flags go up in my head). I have a DBI connection to a db that this script useses.
Thanks
The Bareword error you are getting sounds like you've done something in that "..." to confuse Perl with regard to the imported Fcntl constants. There's nothing wrong with using those constants like that. You might try something like LOCK_UN() to see what error that gets you.
If you are using MySQL, you can use the GET_LOCK() and RELEASE_LOCK() mechanism. It works reasonably well for cases like this:
SELECT GET_LOCK("script_lock");
and then when you are finished:
SELECT RELEASE_LOCK("script_lock");
See http://dev.mysql.com/doc/refman/4.1/en/miscellaneous-functions.html for details.
You may want to avoid the file locking; from what I remember it's notoriously unreliable on non-local filesystems. Your better bet is to just use the existence of the file itself to the indicator that the script is already running (similar to a UNIX PID file) Granted, this won't be 100% reliable but should work reasonably reliably with very low overhead, provided the script isn't getting invoked incessantly.
If you need better reliability than that, using the database for the mutex is a good solution.
Is there any MySQL library for Ruby that supports parameterization? The documentation for mysql2 gives this example:
escaped = client.escape("gi'thu\"bbe\0r's")
results = client.query("SELECT * FROM users WHERE group='#{escaped}'")
And that seems kind of clunky and screw-up-able to me.
Sequel does, too. But for MySQL it only simulates them:
The MySQL ruby driver does not support bound variables, so the bound variable methods fall back to string interpolation.
Apparently DBI does
http://ruby-dbi.rubyforge.org/
I'm creating a set of SQL INSERT statements for a database that doesn't exist yet, and I'm saving them to file.
How can I use Perl's powerful DBI module to create those INSERT statements without accessing a specific database. In particular, it looks like using the $dbh->quote() function requires that I instantiate $dbh with a connection to a database.
Unfortunately, the actual quote() behaviour isn't always a portable characteristic, so each driver will do them differently. Unless you connect to a driver, you don't know which quoting format to use in practice. (There is one module that might do this without a connection, DBIx::Abstract, but it is not especially current.).
The quote() method is actually implemented by the corresponding driver class, in the DBD::* namespace. You might attempt to load the driver you need and call the function directly (see http://search.cpan.org/~timb/DBI-1.616/lib/DBI/DBD.pm#Writing_DBD::Driver::db::quote) but this feels grubby.
I'd still make a DBI connection, if only so that you get the right format of quoting. You don't need to actually send it any statements, but then you do know that the quoting format will be correct for the database you will use.
From DBI::quote:
For most database types, at least those that conform to SQL standards, quote would return 'Don''t' (including the outer quotation marks). For others it may return something like 'Don\'t'
That is, the behavior of DBI::quote varies from database to database, and it doesn't make sense to call it in a database-independent way.
Make a trivial connection to a database of the same type you are writing for, or learn your database's quoting conventions and implement a quote method yourself. See the DBI source for a reference implementation.
Usually you would use DBI by specifying a database like so:
my $dbh = DBI->connect("DBI:mysql:database=$db_name;host=127.0.0.1");
However, your database does not yet exist so you cannot connect to it. You can use DBI without specifying a database like so:
my $dbh = DBI->connect("DBI:mysql:;host=127.0.0.1");
You could use DBD::CSV or DBD::AnyData as a dummy database. SQLite is good for this purpose, too.
A hidden advantage of using SQLite here is that it's a semi-real database, and will tend to make you write code in a way that's decoupled from any specific database.
According to perl -MDBI -E 'say join(q{,},DBI->available_drivers);'
in clean Debian chroot with only DBI (package "libdbi-perl") installed the following drivers are available right away:
DBM,ExampleP,File,Gofer,Proxy,Sponge
The minimum DBI connect statement that works for me is
my $dbh=DBI->connect("DBI:DRIVER:"); # DRIVER is one of [DBM,File,ExampleP,Sponge,mysql,SQLite]
That is enough to use $dbh->quote() with no database whatsoever.
DBM and File escape q{'} as q{\'} ("mysql" style);
ExampleP and Sponge escape: q{'} as q{''} ("SQLite" style).
You can also use:
DBD::_::db->quote()
To access the quote function without setting up a database handle. I don't believe it is specific to MySQL though.
I'm interessted, how does Linq2Sql handles a compiled query, that returns IQueryable.
If I call an extension method based on a compiled query like "GetEntitiesCompiled().Count()" or "GetEntitiesCompiled().Take(x)". What does Linq2Sql do in the background? This would be very bad, so in this situation I should write a compiled query like "CountEntitiesCompiled".
Does he load the result (in this case "GetEntitiesCompiled()") into the memory (mapped to the entity class like "ToList()")?
So what situations make sense, when the compiled queries return IQueryable, that query is not able to modify, before request to the Sql-Server.
So in my opinion I can just as good return List.
Thanks for answers!
As I understand it - if it can't use the pre-compiled query exactly (because you have composed it further), it just runs it as it would any regular IQueryable query - so it will indeed still issue a SELECT COUNT(1) FROM ... (it shouldn't iterate the entire table / whatever).
But the real answer is: profile it; you can hook .Log to see the TSQL, for example:
myDataContext.Log = Console.Out; // write TSQL to the console
or just use SQL trace to see what goes up and down the wire.
Linq2Sql is not smart enough in such cases. From my experience it always performs compiled part as is. In case of GetEntitiesCompiled().Count() it will fetch all the records and then perform in-memory Count().