I need to make a query where I will be looking for a specific string through several columns and I need to know which column (name) contains the value that I need.
In the example below I need a query where I can ask which column contains the value 1000000000002101214 and that it returns f1. DB is MySQL and I need the programming to be done in Perl.
+---------------------+---------------------+---------------------+
| f1 | f2 | f3 |
+---------------------+---------------------+---------------------+
| 1000000000002101214 | 1000000000001989129 | 1000000000001881637 |
| 1000000000002080453 | 1000000000001968481 | 1000000000001862284 |
| 1000000000002085919 | 1000000000001973677 | 1000000000001866854 |
| 1000000000002075076 | 1000000000001963189 | 1000000000001857288 |
+---------------------+---------------------+---------------------+
I was able to find an almost-answer to my question from another site where I could get the column names of the fields in the table with the following:
my #cols = #{$sth->{NAME}}; # or NAME_lc if needed
while (#row = $sth->fetchrow_array) {
print "#row\n";
}
$sth->finish;
foreach ( #cols ) {
printf( "Note: col : %s\n", $_ );
}
The problem is partially resolved. In the example table I provided in the original question I needed to know on which column my answer resides, the query contains several OR statemens:
SELECT * FROM table WHERE (f1='1000000000002101214' OR f2='1000000000002101214' OR f3='1000000000002101214')
And I need the result to show that the column name where the number is located is f1. So....
Any thoughts?
I don't even know where to start
Check out Perl's DBI module. Read the documentation. You'll have to do something like below:
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
#Connect to your database, provide id, password
my $dbh = DBI->connect('dbi:mysql:perltest','root','password') or die "Connection Error: $DBI::errstr\n";
#Write your query
my $sql = "select * from database_schema";
my $sth = $dbh->prepare($sql);
#Execute it
$sth->execute or die "SQL Error: $DBI::errstr\n";
#Fetch the value
while (my #row = $sth->fetchrow_array) {
#Do something with your result
print "#row\n";
}
If you are new to Perl then see: http://learn.perl.org/
Edit: Query to find out column name based on the value found in column.
Select 'f1'
from database_schema
where database_schema.f1 = 1000000000002101214
union
Select 'f2'
from database_schema
where database_schema.f2 = 1000000000002101214
union
Select 'f3'
from database_schema
where database_schema.f3 = 1000000000002101214
Related
Definition of task:
Fetch data from two different columns using OR.
Problem:
While its working with the plain (MySQL) query, Perl DBI throws an exception due to the uneven-number of bind variables.
Let's assume the following DB schema:
customer vpn_primary_ip vpn_secondary_ip
1000 1.1.1.1 2.2.2.2
1001 3.3.3.3 NULL
1002 4.4.4.4 5.5.5.5
1003 NULL 6.6.6.6
Side note:
As the column where the ip address is stored is not predictable, I combine the search for the columns vpn_primary_ip AND vpn_secondary_ip using the OR operator. The plain SQL query is as follows:
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' )
OR
vpn_secondary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' );
The query above gives the following (appropriate) result:
+----------+-----------------+------------------+
| customer | vpn_primary_ip | vpn_secondary_ip |
+----------+-----------------+------------------+
| 1000 | 1.1.1.1 | 2.2.2.2 |
| 1002 | 4.4.4.4 | 5.5.5.5 |
| 1003 | NULL | 6.6.6.6 |
+----------+-----------------+------------------+
The same SQL query with Perl DBI:
my #ip_addresses = ('1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6');
my $sth = $dbh->prepare (
"SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( #{[join',', ('?') x #ip_addresses]} )
OR
vpn_secondary_ip IN ( #{[join',', ('?') x #ip_addresses]} )"
);
$sth->execute(#ip_addresses);
Throws the following exception:
DBD::mysql::st execute failed: called with 4 bind variables when 8 are needed at get_vpn_customers line 211, <DATA> line 1.
DBD::mysql::st execute failed: called with 4 bind variables when 8 are needed at get_vpn_customers line 211, <DATA> line 1.
The only idea to make it work, is to pass #ip_addresses to the execute method twice:
$sth->execute(#ip_addresses, #ip_addresses);
Question:
Is this the proper approach or is there another, let's say, best or better practice?
$sth->execute(#ip_addresses, #ip_addresses);
This is the correct approach. All DBI knows is that you have passed it an SQL query containing eight bind points. It, therefore, needs eight matching values passed to the execute() method.
There is no way for Perl, DBI or MySQL to know that the bind values are repeated.
Other possible solution is to massage SQL query to workable state before $sth->execute()
use strict;
use warnings;
use feature 'say';
my #ip_addresses = ('1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6');
my $query = "
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( #{[join',', ('?') x #ip_addresses]} )
OR
vpn_secondary_ip IN ( #{[join',', ('?') x #ip_addresses]} )
";
say $query;
my $ip_addresses;
my $flag = 0;
for (#ip_addresses) {
$ip_addresses .= ', ' if $flag;
$ip_addresses .= "'$_'";
$flag = 1;
}
$query = "
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( $ip_addresses )
OR
vpn_secondary_ip IN ( $ip_addresses )
";
say $query;
Output
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( ?,?,?,? )
OR
vpn_secondary_ip IN ( ?,?,?,? )
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' )
OR
vpn_secondary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' )
Suppose I have a simple database table that doesn't have an ID_KEY but has a name column. I want to display the output like this
+----+---------+
| | name |
+----+---------+
| 1 | dog |
| 2 | cat |
| 3 | penguin |
| 4 | lax |
| 5 | whale |
| 6 | ostrich |
+----+---------+
Then have a <STDIN> for like, say, 3 to select penguin. 3 is just the line number that appears when you do the select call.
Is there any way to do this, or is it only possible with an id key associated and then a subsequent select statement matching that id key?
I misunderstood you at first but I've caught on. But it doesn't make much sense, as when you're entering a number into a Perl program you won't be working with the MySQL command-line tool, and won't be able to see what numbers to enter..
What you need to do is to write your Perl program so that it prints all the name fields from the table together with a line number. Then your program can translate from an input animal number to its name because it knows what it printed.
Something like this would work. Of course you will have to set the name, IP address and credentials correctly so that DBI can connect to the database.
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect(
'DBI:mysql:database=animal_test',
'username',
'password',
{ RaiseError => 1 },
);
my $names = map #$_, $dbh->selectall_arrayref('SELECT name FROM animals');
for my $i ( 0 .. $#$names ) {
printf "%3d: %s\n", $i+1, $names->[$i];
}
print "\n";
print "Enter animal number: ";
my $animal = <>;
chomp $animal;
my $name = $names->[$animal-1];
printf "Animal chosen is %s\n", $name;
Option 1 - You would have put a id field in the DB if you want to find by integer 3 because row 3 will not always be penguin from an SQL query.
Option 2 - Dump the data into and array or hash and use the index of that to find the item from with in the variable and not the DB when 3 is captured from STIN.
Just use query:
my $select = $dbh->prepare('
SET #id:=0;
SELECT name,
#id = #id+1
FROM table
');
I have a table with millions of rows and a single column of text that is exactly 11,159 characters long. It looks like this:
1202012101...(to 11,159 characters)
1202020120...
0121210212...
...
(to millions of rows)
I realize that I can use
SELECT SUBSTR(column,2,4) FROM table;
...if I wanted to pull out characters 2, 3, 4, and 5:
1202012101...
1202020120...
0121210212...
^^^^
But I need to extract noncontiguous characters, e.g. characters 1,5,7:
1202012101...
1202020120...
0121210212...
^ ^ ^
I realize this can be done with a query like:
SELECT CONCAT(SUBSTR(colm,1,1),SUBSTR(colm,5,1),SUBSTR(colm,7,1)) FROM table;
But this query gets very unwieldy to build for thousands of characters that I need to select. So for the first part of the question - how do I build a query that does something like this:
SELECT CHARACTERS(string,1,5,7) FROM table;
Furthermore, the indices of the characters I want to select are from a different table that looks something like this:
char_index keep_or_discard
1 keep
2 discard
3 discard
4 discard
5 keep
7 discard
8 keep
9 discard
10 discard
So for the second part of the question, how could I build a query to select specific characters from the first table based on whether keep_or_discard="keep" for that character's index in the second table?
this function does what you want:
CREATE DEFINER = `root`#`localhost` FUNCTION `test`.`getsubset`(selection mediumtext, longstring mediumtext)
RETURNS varchar(200)
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT 'This function returns a subset of characters.'
BEGIN
SET #res:='';
SET #selection:=selection;
WHILE #selection<>'' DO
set #pos:=CONVERT(#selection, signed);
set #res := concat_ws('',#res,SUBSTRING(longstring,#pos,1));
IF LOCATE(',',#selection)=0 THEN
SET #selection:='';
END IF;
set #selection:=SUBSTRING(#selection,LOCATE(',',#selection)+1);
END WHILE;
RETURN #res;
END
Note: the CONVERT('1,2,3,4',signed) will yield 1, but it will give a warning.
I have it defined to be available in the database test.
The function takes two parameters; a string(!) with a list of positions, and a long string from where you want the characters taken.
An example of using this:
mysql> select * from keepdiscard;
+---------+------------+
| charind | keepordisc |
+---------+------------+
| 1 | keep |
| 2 | discard |
| 3 | keep |
| 4 | discard |
| 5 | keep |
| 6 | keep |
+---------+------------+
6 rows in set (0.00 sec)
mysql> select * from test;
+-------------------+
| longstring |
+-------------------+
| abcdefghijklmnopq |
| 123456789 |
+-------------------+
2 rows in set (0.00 sec)
mysql> select getsubset(group_concat(charind ORDER BY charind),longstring) as result from keepdiscard, test where keepordisc='keep' group by longstring;
+--------+
| result |
+--------+
| 1356 |
| acef |
+--------+
2 rows in set, 6 warnings (0.00 sec)
The warnings stem from the fast conversion to integer that is done in the function. (See comment above)
How about dynamic sql? (You will need to build the select part of the query)
CREATE PROCEDURE example_procedure()
BEGIN
--
--build the concat values here
--
SET #ids := '';
SET #S = 'SELECT #ids := built_concat_of_values FROM table';
PREPARE n_StrSQL FROM #S;
EXECUTE n_StrSQL;
DEALLOCATE PREPARE n_StrSQL;
END
You can write a php script to do this for you:
<?php
//mysql connect
$conn = mysql_connect('localhost', 'mysql_user', 'mysql_password');
if (!$conn) {
echo 'Unable to connect to DB: ' . mysql_error();
exit;
}
//database connect
$db = mysql_select_db('mydb');
if (!$db) {
echo 'Unable to select mydb: ' . mysql_error();
exit;
}
//get the keep numbers you’re going to use.
//and change the number into string so, for example, instead of 5 you get 'SUBSTR(colm,5,1)'
$result = mysql_query("SELECT number FROM number_table WHERE keep_or_discard='keep'");
$numbers = array();
while ($row = mysql_fetch_assoc($result)) {
$row = 'SUBSTR(colm,' . $row . ',1)';
$numbers = $row;
}
//implode the array so you get one long string with all the substrings
//eg. 'SUBSTR(colm,1,1),SUBSTR(colm,5,1),SUBSTR(colm,12,1)'
$numbers = implode(",", $numbers);
//pull the numbers you need and save them to an array.
$result = mysql_query("SELECT " . $numbers . " FROM table");
$concat = array();
while ($row = mysql_fetch_assoc($result)) {
$concat= $row;
}
And there you have an array with the correct numbers.
I'm sorry if you can't/don't want to use PHP for this, I just don't really know how to do this without PHP, Perl, Python or some other similar language. Hopefully this solution will help somehow...
The source of your difficulty is that your schema does not represent the true relationships between the data elements. If you wanted to achieve this with "pure" SQL, you would need a schema more like:
table
ID Index Char
1 0 1
1 1 2
1 2 0
charsToKeep
ID Index Keep
1 0 false
1 1 true
1 2 true
Then, you could perform a query like:
SELECT Char FROM table t JOIN charsToKeep c ON t.ID = c.ID WHERE c.Keep = true
However, you probably have good reasons for structuring your data the way you have (my schema requires much more storage space per character and the processing time is also probably much longer from what I am about to suggest).
Since SQL does not have the tools to understand the schema you have embedded into your table, you will need to add them with a user-defined function. Kevin's example of dynamic SQL may also work, but in my experience this is not as fast as a user-defined function.
I have done this in MS SQL many times, but never in MySql. You basically need a function, written in C or C++, that takes a comma-delimited list of the indexes you want to extract, and the string from which you want to extract them from. Then, the function will return a comma-delimited list of those extracted values. See these links for a good starting point:
http://dev.mysql.com/doc/refman/5.1/en/adding-functions.html
http://dev.mysql.com/doc/refman/5.1/en/adding-udf.html
To build the concatenated list of indexes you want to extract from the char_index table, try the group_concat function:
http://dev.mysql.com/doc/refman/5.0/en/group-by-functions.html#function_group-concat
Hope this helps!
I have 2 tables in mysql database: CUSTOMER and GROUP
The CUSTOMER table:
NAME |PHONE
A |222
B |333
C |777
D |888
E |111
F |555
and so on.
The GROUP table has only 3 value:
GN | NUM
NEW |807
OLD |455
INT |504
I would like to get the following result:
A, NEW, 807
B, OLD, 455
C, INT, 504
D, NEW, 807
E, OLD, 455
F, INT, 504
and so on..
The GROUP table must repeat until the end of CUSTOMER table.
Here is my code:
#!/usr/bin/perl
# PERL MODULES
use DBI;
use strict;
use warnings;
# MYSQL CONFIG VARIABLES
my $dsn = 'DBI:mysql:test:127.0.0.1';
my $tablename = "CUSTOMER";
my $user = "root";
my $pw = "xxxx";
# DEFINE A MySQL QUERY
my $myquery1 = "SELECT NAME FROM $tablename";
# PERL CONNECT()
my $dbh = DBI->connect($dsn, $user, $pw);
# EXECUTE THE QUERY
my $getname = $dbh->prepare($myquery1);
$getnum->execute();
my $getlogin = $dbh->prepare("select * from GROUP");
$getlogin->execute();
my($login, $password);
# FETCHROW ARRAY
while (my $name = $getname->fetchrow_array()) {
while (my #row = $getlogin->fetchrow_array()) {
my ($gn,$num) = #row;
$login=$gn;
$password=$num;
print "$name\t\t $login \t\t $password \n";
}
}
When i execute my code i get:
A NEW 807
B OLD 455
C INT 504
DBD::mysql::st fetchrow_array failed: fetch() without execute() at ./main.pl line 29.
How can i do this?
Any help would be appreciated.
First of all, you have a typo -- you are doing $getnum->execute() not $getname->execute(). Did you really run the exact code you pasted?
You are encountering an error after the third iteration because you only have three rows of data in the GROUP table. You need to either start the loop again with a fresh query (perform the execute() inside the first while loop, just before you start the second), or cache all its data into an array that you can loop over repeatedly:
my $getname = $dbh->prepare($myquery1);
my $getlogin = $dbh->prepare("select * from GROUP");
# FETCHROW ARRAY
$getname->execute();
while (my $name = $getname->fetchrow_array())
{
$getlogin->execute();
while (my #row = $getlogin->fetchrow_array())
{
my ($gn,$num) = #row;
my $login=$gn;
my $password=$num;
print "$name\t\t $login \t\t $password \n";
}
}
It sounds like you just want the rows in the CUSTOMER table to be assigned alternating values, rotating through the GROUP table -- and you don't much care who gets what value (or you would have put ORDERs into your SELECTs).
What I'd do is: add a column to the GROUP table with the values 0, 1 and 2; give the CUSTOMER table an incrementing id; and join them on (CUSTOMER.id % 3 = GROUP.id). That rotates the GROUP values down the CUSTOMER table in what I think is exactly the way you want.
ALTER TABLE `GROUP` ADD COLUMN id INT UNSIGNED NOT NULL, ADD INDEX idx_id (id);
UPDATE GROUP SET id=0 WHERE GN='NEW';
UPDATE GROUP SET id=1 WHERE GN='OLD';
UPDATE GROUP SET id=2 WHERE GN='INT';
ALTER TABLE `CUSTOMER` ADD COLUMN
id INT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT;
Then it's just one SELECT to get the pairings you want:
SELECT NAME, GN, NUM FROM `GROUP`, CUSTOMER WHERE GROUP.id = CUSTOMER.id % 3;
(P.S. I suggest not naming a table an SQL keyword like "GROUP", you'll have to quote it every time you use it.)
The problem is that you're calling $getlogin->fetchrow_array after all elements are processed. This happens when CUSTOMER loop doing second iteration. You should call $getlogin->execute just in the start of CUSTOMER loop. Like this:
while (my $name = $getname->fetchrow_array()) {
## start new query each time we want to loop GROUP table
my $getlogin = $dbh->prepare("select * from GROUP");
$getlogin->execute();
while (my #row = $getlogin->fetchrow_array()) {
But this can kill performance of the script. I suggest you to select all GROUPs before CUSTOMERs loop into array and use it instead of loading data from DB each iteration.
I have a database in the following format:
idA | idB | stringValue
----+-----+------------
xyz | 1 | ANDFGFRDFG
xyz | 2 | DTGDFHFGH
xyz | 3 | DFDFDS
abc | 5 | DGDFHHGH
abc | 6 | GG
...
idA and idB together are unique.
I have a file (list.txt) with a list of id pairs like this:
xyz 2
abc 5
abc 6
...
and I would like to print it with stringValue in each row. Note some pairs of ids from the file might be absent from the DB. In that case, I don't want to print them at all.
In short, I want to filter a table using a file.
SELECT idA, idB, stringValue FROM table WHERE idA = 'xyz' AND idB = 12;
Would give you all columns where idA equals "xyz" and idB equals 12.
The number of OR clauses will be dynamic based on the number of combinations in your file.
You will probably need to make this a dynamically created query on the lines of
SELECT * FROM myTable
WHERE
(idA = 'xyz'AND idB = '2')
OR
(idA = 'abc'AND idB = '5')
OR
(idA = 'abc'AND idB = '6')
......
Some SQL clients do provide the facility that allow you to create queries or data straight from files. However I believe the better approach would be to write a script for this.
Based on other questions you recently gave like this one then perhaps the solution below in Perl may help you:
use 5.012;
use warnings;
use SQL::Abstract;
my $sql = SQL::Abstract->new;
my #cols = qw/ idA idB stringValue /;
my $where = build_sql_where_from_file( 'list.txt', #cols[0,1] );
my ($query, #bind) = $sql->select(
'yourTable',
\#cols,
$where,
);
sub build_sql_where_from_file {
my $file = shift;
my #build;
open my $fh, '<', $file or die $!;
for my $line (<$fh>) {
chomp $line;
my #fields = split / /, $line;
push #build, {
-and => [
map { $_ => shift #fields } #_,
]
};
}
return \#build;
}
If I now do the following using your example list.txt...
say $query;
say join ":", #bind;
then I get:
SELECT idA, idB, stringValue FROM yourTable WHERE ( ( ( idA = ? AND idB = ? ) OR ( idA = ? AND idB = ? ) OR ( idA = ? AND idB = ? ) ) )
xyz:2:abc:5:abc:6
Which is exactly what I need to then plugin straight into a DBI query.
Hope that helps.
/I3az/