I have a database in the following format:
idA | idB | stringValue
----+-----+------------
xyz | 1 | ANDFGFRDFG
xyz | 2 | DTGDFHFGH
xyz | 3 | DFDFDS
abc | 5 | DGDFHHGH
abc | 6 | GG
...
idA and idB together are unique.
I have a file (list.txt) with a list of id pairs like this:
xyz 2
abc 5
abc 6
...
and I would like to print it with stringValue in each row. Note some pairs of ids from the file might be absent from the DB. In that case, I don't want to print them at all.
In short, I want to filter a table using a file.
SELECT idA, idB, stringValue FROM table WHERE idA = 'xyz' AND idB = 12;
Would give you all columns where idA equals "xyz" and idB equals 12.
The number of OR clauses will be dynamic based on the number of combinations in your file.
You will probably need to make this a dynamically created query on the lines of
SELECT * FROM myTable
WHERE
(idA = 'xyz'AND idB = '2')
OR
(idA = 'abc'AND idB = '5')
OR
(idA = 'abc'AND idB = '6')
......
Some SQL clients do provide the facility that allow you to create queries or data straight from files. However I believe the better approach would be to write a script for this.
Based on other questions you recently gave like this one then perhaps the solution below in Perl may help you:
use 5.012;
use warnings;
use SQL::Abstract;
my $sql = SQL::Abstract->new;
my #cols = qw/ idA idB stringValue /;
my $where = build_sql_where_from_file( 'list.txt', #cols[0,1] );
my ($query, #bind) = $sql->select(
'yourTable',
\#cols,
$where,
);
sub build_sql_where_from_file {
my $file = shift;
my #build;
open my $fh, '<', $file or die $!;
for my $line (<$fh>) {
chomp $line;
my #fields = split / /, $line;
push #build, {
-and => [
map { $_ => shift #fields } #_,
]
};
}
return \#build;
}
If I now do the following using your example list.txt...
say $query;
say join ":", #bind;
then I get:
SELECT idA, idB, stringValue FROM yourTable WHERE ( ( ( idA = ? AND idB = ? ) OR ( idA = ? AND idB = ? ) OR ( idA = ? AND idB = ? ) ) )
xyz:2:abc:5:abc:6
Which is exactly what I need to then plugin straight into a DBI query.
Hope that helps.
/I3az/
Related
I have code for my filter. It worked well until I add new product in my database. I found the problem, but dont know what to do with that.
I have parameters "alc_min" and "alc_max" in my filter. I get these from crawling all products. After I send this filter, I fire this code:
$meta_query = array();
$b = "alc_min";
$c = "alc_max";
if (isset ( $data [$b] ) && isset ( $data [$c] )) {
$compare = "BETWEEN";
$a = array (
'key' => "alc",
'value' => array (
$data [$b],
$data [$c]
),
'compare' => $compare
);
array_push ( $meta_query, $a );
}
$items = new WP_Query ( array (
'post_type' => $type,
'posts_per_page' => $posts_per_page,
'order' => $order,
'meta_key' => $orderkey,
'orderby' => $orderby,
'post_status' => 'publish',
'meta_query' => $meta_query,
'paged' => $paged
) );
Until now, it worked well. No I add new product with "alc" <10 and I found, that if I have "alc_min" and "alc_max" <10 or >10, it is ok. But if "alc_min" is <10 and "alc_max" >10 I get no results at all.
Does anyone any idea what to check or fix?
After the clarification, I've suspected that the reason why selecting "alc_min" = 7 and "alc_max" = 13 doesn't yield any result is because of the column datatype. Consider this example:
CREATE TABLE table1 (
alc VARCHAR(50));
INSERT INTO table1 VALUES
('7'),
('9'),
('11'),
('13');
The table above is created with alc column datatype as VARCHAR instead of INTEGER (or numeric datatype). I've tested that running either one of the query below:
SELECT * FROM table1 WHERE alc BETWEEN '7' AND '9';
SELECT * FROM table1 WHERE alc BETWEEN '11' AND '13';
will return the expected result. However, with this query:
SELECT * FROM table1 WHERE alc BETWEEN '7' AND '13';
yields no result. This is because the values are treated as string instead of numbers and when that happens, 1 is always smaller than 7. See below what happen you run select query with order by on the data set above:
SELECT * FROM table1 ORDER BY alc;
+-----+
| alc |
+-----+
| 11 |
| 13 |
| 7 |
| 9 |
+-----+
As you can see, since the data is treated as string (according to the column datatype), then you could imagine this in alphabetical form as the following:
+-----+--------------+
| alc | alphabetical |
+-----+--------------+
| 11 | AA |
| 13 | AC |
| 7 | G |
| 9 | I |
+-----+--------------+
So, the condition of BETWEEN '7' AND '13' becomes BETWEEN 'G' AND 'AC'; which doesn't really make sense. And if you change to BETWEEN '11' AND '9' you'll get the correct result but that made the query even more confusing and not making sense at all.
Now, I've discovered that there are at least 3 workaround/solution for this:
One of the oldest way I can think of is by adding +0 to the column in the query. I didn't find any official docs about this but I assume that doing this will change the data value to numeric in the query:
SELECT * FROM table1
WHERE alc+0 BETWEEN '7' AND '13';
This is probably the same as above is just that I'm not sure if this is version specific or not. It turns out that in my testing, if you didn't wrap the searched value in quotes, you'll get the result as if the data is numeric:
SELECT * FROM table1
WHERE alc BETWEEN 7 AND 13;
This require a change of column datatype but afterwards any of the query with or without quotes on the searched value should work:
ALTER TABLE table1 CHANGE alc alc INT;
I hope that this is true and the issue is really about column datatype. As far as I know, this is the closest thing to what your situation is that I had experience with.
Here's a fiddle for reference
Definition of task:
Fetch data from two different columns using OR.
Problem:
While its working with the plain (MySQL) query, Perl DBI throws an exception due to the uneven-number of bind variables.
Let's assume the following DB schema:
customer vpn_primary_ip vpn_secondary_ip
1000 1.1.1.1 2.2.2.2
1001 3.3.3.3 NULL
1002 4.4.4.4 5.5.5.5
1003 NULL 6.6.6.6
Side note:
As the column where the ip address is stored is not predictable, I combine the search for the columns vpn_primary_ip AND vpn_secondary_ip using the OR operator. The plain SQL query is as follows:
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' )
OR
vpn_secondary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' );
The query above gives the following (appropriate) result:
+----------+-----------------+------------------+
| customer | vpn_primary_ip | vpn_secondary_ip |
+----------+-----------------+------------------+
| 1000 | 1.1.1.1 | 2.2.2.2 |
| 1002 | 4.4.4.4 | 5.5.5.5 |
| 1003 | NULL | 6.6.6.6 |
+----------+-----------------+------------------+
The same SQL query with Perl DBI:
my #ip_addresses = ('1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6');
my $sth = $dbh->prepare (
"SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( #{[join',', ('?') x #ip_addresses]} )
OR
vpn_secondary_ip IN ( #{[join',', ('?') x #ip_addresses]} )"
);
$sth->execute(#ip_addresses);
Throws the following exception:
DBD::mysql::st execute failed: called with 4 bind variables when 8 are needed at get_vpn_customers line 211, <DATA> line 1.
DBD::mysql::st execute failed: called with 4 bind variables when 8 are needed at get_vpn_customers line 211, <DATA> line 1.
The only idea to make it work, is to pass #ip_addresses to the execute method twice:
$sth->execute(#ip_addresses, #ip_addresses);
Question:
Is this the proper approach or is there another, let's say, best or better practice?
$sth->execute(#ip_addresses, #ip_addresses);
This is the correct approach. All DBI knows is that you have passed it an SQL query containing eight bind points. It, therefore, needs eight matching values passed to the execute() method.
There is no way for Perl, DBI or MySQL to know that the bind values are repeated.
Other possible solution is to massage SQL query to workable state before $sth->execute()
use strict;
use warnings;
use feature 'say';
my #ip_addresses = ('1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6');
my $query = "
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( #{[join',', ('?') x #ip_addresses]} )
OR
vpn_secondary_ip IN ( #{[join',', ('?') x #ip_addresses]} )
";
say $query;
my $ip_addresses;
my $flag = 0;
for (#ip_addresses) {
$ip_addresses .= ', ' if $flag;
$ip_addresses .= "'$_'";
$flag = 1;
}
$query = "
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( $ip_addresses )
OR
vpn_secondary_ip IN ( $ip_addresses )
";
say $query;
Output
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( ?,?,?,? )
OR
vpn_secondary_ip IN ( ?,?,?,? )
SELECT
customer,
vpn_primary_ip,
vpn_secondary_ip,
FROM
table
WHERE
vpn_primary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' )
OR
vpn_secondary_ip IN ( '1.1.1.1', '4.4.4.4', '5.5.5.5', '6.6.6.6' )
For a list query I want to show how many prices there are for a specific product on a specific site.
I have found a way to do this with RawSQL, something I had never ever had to do before. Am I overlooking something?
queryset = Price.objects.all()
site_annotations = {}
for site in Site.objects.all():
site_annotations['num_prices_{}'.format(site.name)] = RawSQL(
'SELECT COUNT(id) '
'FROM `product_siteprice` '
'WHERE `product_siteprice`.`price_id` = `product_price`.`id` '
'AND site_id=%s', [site.pk]
)
queryset = queryset.annotate(**site_annotations)
edit
My models look simplified like this:
class Site(Model):
name = models.CharField(_('display name'), max_length=50)
class Price(Model):
name = models.CharField()
class SitePrice(Model):
price = models.ForeignKey(Price)
site = models.ForeignKey(Site)
Now the table I want is:
Product | Nr of prices on Site A | Nr of prices on Site B |
--------|------------------------|------------------------|
Iphone | 6 | 3 |
Xperia | 42 | 66 |
And I want to sort and filter on the number of prices, so thats why I need the annotation.
NB The name Price is a bit misleading if you don't know the rest of context, in this context it can be seen more like a Product
From what I understand, you just need to filter the prices on the related site, then get the count of the number of objects.
Price.objects.filter(site_price__site_id=site.pk).count()
Although you may just want to remove the site price model and replace it with a many to many field
I think the query that you want would look something like this:
qs = SitePrice.objects.values('price_id', 'site_id').annotate(d_count=Count('id'))
And to get the table that you want, you’d format it into a dict with something like this:
from collections import defaultdict
values = defaultdict(dict)
for x in qs:
values[ x['price_id'] ][ x['site_id'] ] = x['d_count']
To print out this table, you would use:
price_ids = list(values.keys()) # you don’t need `list` for python 2
site_ids = set()
for x in values.values():
site_ids |= set(x.keys())
site_ids = list(site_ids)
site_ids.sort()
prices = dict(Price.objects.filter(id__in=price_ids).values_list('id', 'name'))
sites = dict(Site.objects.filter(id__in=site_ids).values_list('id', 'name'))
print(' ', end='') # for python 3
# print ' ', # for python 2
for x in site_ids:
print('%-8s ' % sites[x], end='')
print('')
for x, counts in values.items():
print('%-8s ' % prices[x], end='')
for site_id in site_ids:
d_count = counts.get(site_id, 0)
print('%-8s ' % d_count, end='')
print('')
I need to make a query where I will be looking for a specific string through several columns and I need to know which column (name) contains the value that I need.
In the example below I need a query where I can ask which column contains the value 1000000000002101214 and that it returns f1. DB is MySQL and I need the programming to be done in Perl.
+---------------------+---------------------+---------------------+
| f1 | f2 | f3 |
+---------------------+---------------------+---------------------+
| 1000000000002101214 | 1000000000001989129 | 1000000000001881637 |
| 1000000000002080453 | 1000000000001968481 | 1000000000001862284 |
| 1000000000002085919 | 1000000000001973677 | 1000000000001866854 |
| 1000000000002075076 | 1000000000001963189 | 1000000000001857288 |
+---------------------+---------------------+---------------------+
I was able to find an almost-answer to my question from another site where I could get the column names of the fields in the table with the following:
my #cols = #{$sth->{NAME}}; # or NAME_lc if needed
while (#row = $sth->fetchrow_array) {
print "#row\n";
}
$sth->finish;
foreach ( #cols ) {
printf( "Note: col : %s\n", $_ );
}
The problem is partially resolved. In the example table I provided in the original question I needed to know on which column my answer resides, the query contains several OR statemens:
SELECT * FROM table WHERE (f1='1000000000002101214' OR f2='1000000000002101214' OR f3='1000000000002101214')
And I need the result to show that the column name where the number is located is f1. So....
Any thoughts?
I don't even know where to start
Check out Perl's DBI module. Read the documentation. You'll have to do something like below:
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
#Connect to your database, provide id, password
my $dbh = DBI->connect('dbi:mysql:perltest','root','password') or die "Connection Error: $DBI::errstr\n";
#Write your query
my $sql = "select * from database_schema";
my $sth = $dbh->prepare($sql);
#Execute it
$sth->execute or die "SQL Error: $DBI::errstr\n";
#Fetch the value
while (my #row = $sth->fetchrow_array) {
#Do something with your result
print "#row\n";
}
If you are new to Perl then see: http://learn.perl.org/
Edit: Query to find out column name based on the value found in column.
Select 'f1'
from database_schema
where database_schema.f1 = 1000000000002101214
union
Select 'f2'
from database_schema
where database_schema.f2 = 1000000000002101214
union
Select 'f3'
from database_schema
where database_schema.f3 = 1000000000002101214
I have 2 tables in mysql database: CUSTOMER and GROUP
The CUSTOMER table:
NAME |PHONE
A |222
B |333
C |777
D |888
E |111
F |555
and so on.
The GROUP table has only 3 value:
GN | NUM
NEW |807
OLD |455
INT |504
I would like to get the following result:
A, NEW, 807
B, OLD, 455
C, INT, 504
D, NEW, 807
E, OLD, 455
F, INT, 504
and so on..
The GROUP table must repeat until the end of CUSTOMER table.
Here is my code:
#!/usr/bin/perl
# PERL MODULES
use DBI;
use strict;
use warnings;
# MYSQL CONFIG VARIABLES
my $dsn = 'DBI:mysql:test:127.0.0.1';
my $tablename = "CUSTOMER";
my $user = "root";
my $pw = "xxxx";
# DEFINE A MySQL QUERY
my $myquery1 = "SELECT NAME FROM $tablename";
# PERL CONNECT()
my $dbh = DBI->connect($dsn, $user, $pw);
# EXECUTE THE QUERY
my $getname = $dbh->prepare($myquery1);
$getnum->execute();
my $getlogin = $dbh->prepare("select * from GROUP");
$getlogin->execute();
my($login, $password);
# FETCHROW ARRAY
while (my $name = $getname->fetchrow_array()) {
while (my #row = $getlogin->fetchrow_array()) {
my ($gn,$num) = #row;
$login=$gn;
$password=$num;
print "$name\t\t $login \t\t $password \n";
}
}
When i execute my code i get:
A NEW 807
B OLD 455
C INT 504
DBD::mysql::st fetchrow_array failed: fetch() without execute() at ./main.pl line 29.
How can i do this?
Any help would be appreciated.
First of all, you have a typo -- you are doing $getnum->execute() not $getname->execute(). Did you really run the exact code you pasted?
You are encountering an error after the third iteration because you only have three rows of data in the GROUP table. You need to either start the loop again with a fresh query (perform the execute() inside the first while loop, just before you start the second), or cache all its data into an array that you can loop over repeatedly:
my $getname = $dbh->prepare($myquery1);
my $getlogin = $dbh->prepare("select * from GROUP");
# FETCHROW ARRAY
$getname->execute();
while (my $name = $getname->fetchrow_array())
{
$getlogin->execute();
while (my #row = $getlogin->fetchrow_array())
{
my ($gn,$num) = #row;
my $login=$gn;
my $password=$num;
print "$name\t\t $login \t\t $password \n";
}
}
It sounds like you just want the rows in the CUSTOMER table to be assigned alternating values, rotating through the GROUP table -- and you don't much care who gets what value (or you would have put ORDERs into your SELECTs).
What I'd do is: add a column to the GROUP table with the values 0, 1 and 2; give the CUSTOMER table an incrementing id; and join them on (CUSTOMER.id % 3 = GROUP.id). That rotates the GROUP values down the CUSTOMER table in what I think is exactly the way you want.
ALTER TABLE `GROUP` ADD COLUMN id INT UNSIGNED NOT NULL, ADD INDEX idx_id (id);
UPDATE GROUP SET id=0 WHERE GN='NEW';
UPDATE GROUP SET id=1 WHERE GN='OLD';
UPDATE GROUP SET id=2 WHERE GN='INT';
ALTER TABLE `CUSTOMER` ADD COLUMN
id INT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT;
Then it's just one SELECT to get the pairings you want:
SELECT NAME, GN, NUM FROM `GROUP`, CUSTOMER WHERE GROUP.id = CUSTOMER.id % 3;
(P.S. I suggest not naming a table an SQL keyword like "GROUP", you'll have to quote it every time you use it.)
The problem is that you're calling $getlogin->fetchrow_array after all elements are processed. This happens when CUSTOMER loop doing second iteration. You should call $getlogin->execute just in the start of CUSTOMER loop. Like this:
while (my $name = $getname->fetchrow_array()) {
## start new query each time we want to loop GROUP table
my $getlogin = $dbh->prepare("select * from GROUP");
$getlogin->execute();
while (my #row = $getlogin->fetchrow_array()) {
But this can kill performance of the script. I suggest you to select all GROUPs before CUSTOMERs loop into array and use it instead of loading data from DB each iteration.