I have a perl application that uses a MySQL backend.
How do I verify that the MySQL and database drivers are accessible?
I'm currently doing the following, but I don't know if it is actually a sufficient test -- I'd hate to find out six months down the line that I'm not doing the right test:
use Test::More;
BEGIN { use_ok('DBI'); }
BEGIN { use_ok('DBD::mysql'); }
done_testing();
This is enough to check that connection will be available from Perl side. If you want to check if mysql server is available, you need to connect to it.
P.S. You can also add version checking for DBD::mysql. For ex. 4.001 fixes serious bug in utf8 support.
Well, it looks fine to me, but if you want to be sure, you can check for the presence of the actual filepaths in your %INC ie:
perl -MDBI -e 'die unless $INC{"DBI.pm"}'
perl -MDBD::mysql -e 'die unless $INC{"DBD/mysql.pm"}'
Related
It's been a decade I have worked with Postgres and Perl.
One of my oldest still-operated applications, an dictionary of government addresses and departmental responsibilities, has issues handling query terms containing accented characters, for example köln. In other words, whenever a query term contains a accented character (mainly umlauts) there are 0 results returned.
I have to mention that this behavior is only happening using this application with Postgres as the database. If I switch to MySQL5 (same data) same queries are working correctly.
Trying to track the cause of this problem I have checked the following:
Postgres database is UTF-8 (using the command show server_encoding;)
Postgres client encoding is also UTF8 (using show client_encoding;)
If I use the Postgres monitor and execute the same SQL query as the application does, using accented characters in the query term, I get correct results
The Perl application itself is handling UTF-8, the HTML-Header is set correctly, contents of the output display correct and not garbled
All Perl code files, scripts, .pm package files and templates are UTF-8 encoded (I verified that with file --mime perl_file_name)
I fiddled with the database connection, setting $self->{dbh}->{pg_enable_utf8} = 1; or/and $self->{dbh}->do("SET CLIENT_ENCODING TO 'UTF8';"); or/and $self->{dbh}->do("SET NAMES 'UTF8';"); with no change
I've updated the DBD::Pg module to version 3.6.2, no change.
So I am pretty much out of ideas what else to check or try to get Postgres fully working. Like mentioned in my intro, same application just using MySQL as database works flawlessly.
2 years ago the application was changed to handle UTF-8 data, I did not do the changes myself, but as far as I can see in the code (compared to the code in my GIT repo) its just the HTML UTF8-Header print "Content-type: text/html; charset=utf-8\n\n"; and a few unrelated template parts. Perhaps that change somewhere is the origin for all the problems but I don't know what esp. to adjust for Postgres.
The current Perl version is 5.22.1, using Apache/2.2.22 (Ubuntu). The vhost configuration is simple:
AddHandler cgi-script .cgi .pl
ScriptAlias /...abs-path-to-app.../cgi-bin/
<Directory "/...abs-path-to-app.../cgi-bin/">
AllowOverride None
Options +Indexes +ExecCGI +MultiViews +SymLinksIfOwnerMatch
<IfVersion < 2.4>
Allow from all
</IfVersion>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
Allow from all
</Directory>
Postgres is version 9.1.24.
Edit:
Collate and Ctype is set to en_US.UTF-8, Encoding is set to UTF-8 for the database in question.
Taking a look into the tables, all character varying columns use pg_catalog."default" collation. Executing show lc_collate; show already mentioned en_US.UTF-8.
Edit2:
Using the DBD::Pg flag pg_enable_utf8 and setting it to 0 seems to work out and I get the expected results. Using a value other than 0, for example '-1or1` does not work. I tried out that flag (once again) right after the database connect. Actually I have to verify this as I still do not really understand what's going on.
since met so many startup errors,I decide to analyze mysql startup shell.while some code fragment I cannot understand clearly.
version:
mysql Ver 14.14 Distrib 5.5.43, for osx10.8 (i386) using readline 5.1
368 #
369 # First, try to find BASEDIR and ledir (where mysqld is)
370 #
372 if echo '/usr/local/mysql/share' | grep '^/usr/local/mysql' > /dev/null
373 then
374 relpkgdata=echo '/usr/local/mysql/share' | sed -e 's,^/usr/local/mysql,,' -e 's,^/,,' -e 's,^,./,'
375 else
376 # pkgdatadir is not relative to prefix
377 relpkgdata='/usr/local/mysql/share'
378 fi
what's the purpose of line 372? a little weird
any help will be appreciated.
At first glance, this is very strange indeed... but here's a solution to this mystery.
372: if echo '/usr/local/mysql/share' | grep '^/usr/local/mysql' > /dev/null
373: then
grep returns true if it matches and false if it doesn't, so this is testing whether the string /usr/local/mysql/share begins with (^) /usr/local/mysql. Output goes /dev/null because we don't need to see it, we just want to compare it.
"Well," you interject, "that's obvious enough. The question is why?" Stick with me.
If it matches:
374: relpkgdata=echo '/usr/local/mysql/share' | sed -e 's,^/usr/local/mysql,,' -e 's,^/,,' -e 's,^,./,'
Beginning with /usr/local/mysql/share, strip off the beginning /usr/local/mysql, then strip off the beginning / then prepend ./.
So /usr/local/mysql/share becomes ./share.
Otherwise, use the string /usr/local/mysql/share.
375: else
376: # pkgdatadir is not relative to prefix
377: relpkgdata='/usr/local/mysql/share'
"That's all fine, too," I hear you say, "but why go through all these gyrations to (apparently) compare and massage two fixed literal strings?? We already know the answer, so what's up with all the tests and substitution?"
It's a fair question.
My first suspicion was that there was some sort of magic bash hackery going on that I didn't recognize, but no, this code is really all too simple to be something along those lines.
My second suspicion, since this is notably absent from MySQL 5.0.96 (which I am not running but keep on hand for reference), was that this was an abandoned attempt to introduce some new magical behavior into mysqld_safe which was never finished and replaced with actual variables, the testing and massaging of which would have made a lot more sense than doing the same thing to literal strings.
But, no. When the only tool you have is a hammer, everything looks like a nail. What this is, is an example of doing something simple... the hard way. At least that's what it looks like to me. There actually is a somewhat rational explanation. To find it answer, you have to look into the source code (not binary) distribution.
MySQL has a lot of "hard-coded" defaults. This turns out to be an example of these.
In the source file scripts/mysqld_safe.sh, the snippet above looks very different:
if echo '#pkgdatadir#' | grep '^#prefix#' > /dev/null
then
relpkgdata=`echo '#pkgdatadir#' | sed -e 's,^#prefix#,,' -e 's,^/,,' -e 's,^,./,'`
else
# pkgdatadir is not relative to prefix
relpkgdata='#pkgdatadir#'
fi
Ah, source munging. Pattern substitution.
When you're compiling MySQL from source, the file scripts/Makefile contains instruction that use sed to replace things like #prefix# and #pkgdatadir# with the literal values. The same thing, of course, happens when Oracle or the Linux disto maintainers compile their binary distribution from source. These paths get hard-coded into many, many, many places in the code, including this script... resulting in the otherwise incomprehensible comparison of two literal strings that somebody should already have known the answer to.
Instead of testing at build time, whether one path is an anchored substring of the other, and the "relpkgdata" value should be expressed relative to the current directory and modifying this script accordingly, that logical test is actually deferred until runtime, comparing two literals that were substituted in for their placeholders at build time.
I've gone to this amount of detail, not because it will help you troubleshoot, because I suspect it won't. It was, however, just bizarre enough to warrant some further investigation.
If you are having difficulty getting MySQL Server running... well, you shouldn't be, because it's a well-established system and it should work. If /bin/sh on your system isn't a symlink to /bin/bash, you might want to change mysqld_safe's shebang line from #!/bin/sh to #!/bin/bash, but beyond that, I suspect you are sniffing down the wrong rabbit hole by looking at mysqld_safe to get to the bottom of your issue. As convoluted as mysqld_safe is, it can't be said that it isn't time-tested. As they say, "the problem is somewhere else."
If I may, I'll suggest that you familiarize yourself with some of our other communities where you're likely to find the answer you need, particularly Ask Ubuntu, Super User, Server Fault, and Database Administrators. Familiarize yourself with each site's community, scope, and the level of existing expertise that each community expects on the part of those who ask questions there, and search the sites for the specific problem you're encountering. It's very likely someone has seen it and we've fixed it on one of them, if not here on SO.
I'm currently trying to run NERVE a vaccine development program that is made up of Perl scripts and have trouble getting it to run properly. I've downloaded and installed all prerequisites but every time I reach the mysql step of the code, it crashes, and an error message says: Can't locate object method "connect" via package "Mysql" (perhaps you forgot to load "Mysql"?) at ./NERVE line 340, line 5. I took a look at the code and changed "use mysql" to "use DBI" and "use DBD:mysql" but then I get the error: Can't use string ("") as a HASH ref while "strict refs" in use at /usr/local/lib/perl/5.14.2/DBI.pm line 604, line 5.
I would appreciate if anyone could look the code over for the mysql section and give me advice on how to fix it so I can run the program properly. I tried emailing the developers but no response so I'm hoping you guys can help me.
Below is the perl code for NERVE for the mysql section.
use mysql;
$db=Mysql->connect("$host","$database","$user","$password");
if(!$db || (!$host || !$database || !$user || !$password)){
print "\nAttention: mysql connection parameters are missing or not correct!\n";
print "I need you to specify: host, database, user, password;\n";
print "You can do it now typing them right in that order and separeted only by comma;\n";
print "For example:localhost,Pathogens,sandro,xvzhs\n";
print "So, your Mysql connection settings are (type q to quit):";
while (!($db && $mysql =~ /,/) & $mysql ne "q"){
chomp($mysql = <STDIN>);
die "Ok,let's quit this work! Call me when your mind is clearer about Mysql parameters! Bye :-)\n" if $mysql eq "q";
($host,$database,$user,$password) = split (',',$mysql);
$db=Mysql->connect("$host","$database","$user","$password");
last if($db && $mysql =~ /,/);
print "\nMysql connection settings still not correct!\n";
print "Remember: host, database, user, password, separeted only by comma.\n";
print "For example:localhost,Pathogens,sandro,xvzhs\n";
print "Please, try again:";
}
print "Ok, Mysql connection to \"$database\" database was successful!\n";
}
The error message is correct. You are attempting to use a module named , but you never loaded it. Change
use mysql;
to
use Mysql;
to use that module. In the comments, you mention that results in
Can't locate Mysql.pm in #INC
Unless you have reason to believe the module is installed, that indicates it needs to be installed[1]
That module used to be part of the DBD-mysql distribution, but it's obsolete. It's so ancient it was removed from DBD some years ago. To obtain it, you will need to downgrade your DBD-mysql distribution to version 3.0008.
That's pretty awful thing to do. The script should have DBI instead.
cjm points
since the Mysql.pm in 3.0008 is just a compatibility layer using DBI under the hood, you should be able to install Mysql.pm & Mysql/Statement.pm from that old dist along with a current DBD-mysql.
So if you extract Mysql.pm from the distro I linked above as /usr/lib/perl5/Mysql.pm and Mysql/Statement.pm as /usr/lib/perl5/Mysql/Statement.pm, you should have an easy pain-free solution.
In newer versions of Perl, the error message has been improved. It now reads as follows:
Can't locate Mysql.pm in #INC (you may need to install the Mysql module)
I cant find anything about this from searching here.
I use mysql on the command line at work and I work with fairly large tables so I set the mysql pager allowing a more readable result if I run a query, that returns 1000's of results. I use the command below to set the pager.
\P less -Sin
This suits my needs but has left me wondering if there are any more pager styles that mysql uses on the command line.
The MySQL client just passes its output to whatever command you specify with \P (for "Pager").
-Sin are commandline switches to the program less. From man less:
-i Causes searches to ignore case
-n Suppresses line numbers
-S Causes lines longer than the screen width to be chopped rather than folded.
For more options of the MySQL client, see reference.
mysql> pager less
PAGER set to 'less'
You might want to try pspg:
Unix pager designed for work with tables. Designed for PostgreSQL, but MySQL is supported too.
Main target
possibility to freeze first few rows, first few columns
possibility to use fancy colors - like mcview or FoxPro
In action:
This post is old, but still very helpful.
You can set the pager to whatever you want, including a script that parses all output before feeding it back to you. The examples there include using an add-on tool that makes EXPLAIN output more readable.
Also note that to turn off this functionality and return to normal stdout the command is nopager.
If you don't like less you can use more :)
\P more
This is weird and I'm not sure who the culprit really is.
I'm doing some scripting, on FreeBSD (6.2)? which makes extensive use of the following ***bash***ism:
do_something <(mysql --skip-column-names -B -e 'select ... from ... where ...;')
... where "do_something is a somewhat crufty utility (in Perl) that won't read from a pipeline. If I use a regular file it works fine. My bash script using things like exec 4< <(...) with these sorts of queries (following by loops of the form while read x y z <&4; do ... never seem to have any issues.
However, Perl (5.8.x) seems to periodically block (apparently forever). I tried changing out the chomp(my $data = <MYDATA>); with a routine that used sysread and I wrote some test cases in Python for comparison. These seem to block far less often than the idiomatic Perl code, but they still do it sometimes. (The Python code using f.read() or os.read(f.fileno()...) seems to behave about equally in this issue).
I've tried reproducing the issue using ... <(cat ...) (where I'm cating the regular file) and that never seems to reproduce that stall.
I've glanced at some ktrace/kdump data ... but I'm far more familiar with Linux strace or even Solaris truss ... so I haven't figured out what's going from there yet, either.
I suppose we can mostly rule out Perl, because I've reproduced the same issue using Python ... I don't see how the bash could be doing anything wrong here (it's just creating a named pipe in /var/tmp/sh-np-xxx and wiring the processes up to that).
What could the mysql shell/utility be doing that might cause this? I don't think I've seen it from anything else (such as cat or dd). I haven't tested this scenario under Linux ... but I've used <(...) (process substitution) for years under Linux and don't recall ever seeing this.
Is it a FreeBSD issue?
Sure I can work around the issue using temporary files ... but I'd sure rather understand why it's doing this (and avoid some of the races and clean-up messiness that temporary files entail).
Any suggestions?
The big difference between operating on the output of mysql and directly on a file is timing. When the perl process is stalled, the big question is: "why is it not making forward progress"? You can use the "l" option to ps to see the wait channel for the perl process; that way you can see if it blocked on a read, or if something else is going on. If it is really blocked on pipe input, I expect the MWCHAN entry for perl to be "piperd".
The same information would be interesting for the mysql process.
What does your Python test code look like?
Another way of writing this while avoiding the bashism is this; that would allow you to rule out bash:
mysql --skip-column-names -B -e 'select ... from ... where ...;' | do_something /dev/stdin
Other interesting questions:
Does the --unbuffered option to mysql change anything?
Does piping the mysql output through dd change anything? (eg. "perlscript <(mysql ... | dd)
Summary: Need more information.