Doctrine2 query behavior with groupBy - mysql

I have the following custom method in the repo:
$query = $qb->select('Client', 'Organization')
->from(':Client', 'Client')
->leftJoin('Client.organizations', 'Organization');
--different searh conditions--
Then I use paginator to get the results:
$paginator = new \Doctrine\ORM\Tools\Pagination\Paginator($query->getQuery());
$clients = $paginator->getQuery()
->setMaxResults($length)
->setFirstResult($start);
Here where the magic comes.
I set length to 10 and first result to 0. So basicaly I should get 10 results on the page. However if there are 5 Organizations for the Client (Client mtm Organization), there will be 5 results, if there are 7 Organizations - 3 results.
But if I add
$query->groupBy('Client');
Then all is "ok" with the root level of results: i.e. there are 10 Clients on the page, but there is not all of the Organizations (max 1).
Did anyone experience the same issue? Any thoughts, suggestions?

It probably counts the same item several times.
Try once to add a distinct clause to your query.
$query = $qb->select('Client', 'Organization')
->from(':Client', 'Client')
->leftJoin('Client.organizations', 'Organization');
->distinct();

Related

Sql query doesnt execute correcly (where)

I have a little problem executing sql query.
In my bd I have 3 column (experimentID, applicationID, intent)
I want to get some specific intent with experimentID = 3
my query is :
SELECT experimentID, applicationID, intent
from intents
WHERE experimentID = 3 AND intent='android.intent.action.AIRPLANE_MODE' OR intent='android.intent.action.ALL_APPS' OR intent='android.intent.action.ANSWER' OR intent='android.intent.action.APPLICATION_PREFERENCES' OR intent='android.intent.action.APPLICATION_RESTRICTIONS_CHANGED' OR intent='android.intent.action.ACTION_EXTERNAL_APPLICATIONS_AVAILABLE' OR intent='android.intent.action.APP_ERROR' OR intent='android.intent.action.ASSIST' OR intent='android.intent.action.PHONE_STATE' OR intent='android.intent.action.ATTACH_DATA' OR intent='android.intent.ACTION_SCREEN_OFF' OR intent='android.intent.action.BATTERY_CHANGED' OR intent='android.intent.action.BATTERY_CHANGED_ACTION' OR intent='android.intent.action.BATTERY_LOW' OR intent='android.service.wallpaper.WallpaperService' OR intent='android.intent.action.QUICKBOOT_POWERON' OR intent='android.net.conn.CONNECTIVITY_CHANGE' OR intent='android.intent.action.BATTERY_OKAY' OR intent='android.intent.action.START_SMS_SERVICE' OR intent='android.app.action.ACTION_DEVICE_ADMIN_DISABLE_REQUESTED' OR intent='android.intent.action.BOOT_COMPLETED' OR intent='android.intent.action.BUG_REPORT' OR intent='android.intent.action.CALL' OR intent='android.intent.action.RESPOND_VIA_MESSAGE' OR intent='android.intent.action.CALL_BUTTON' OR intent='android.intent.action.CAMERA_BUTTON' OR intent='android.intent.action.CARRIER_SETUP';
The problem is that I get also others experimentID...(not only experimentID = 3)
I think it's because of the OR in the query but how can I do the query to get only experiementID = 3? The problem is obviously in the WHERE
Thanks for any help!
Parentheses can solve the problem. But a better solution is in:
SELECT experimentID, applicationID, intent
FROM intents
WHERE experimentID = 3 AND
intent IN ('android.intent.action.AIRPLANE_MODE',
. . .
'android.intent.action.CARRIER_SETUP'
);
SELECT experimentID, applicationID, intent
from intents
WHERE experimentID = 3
((AND intent='android.intent.action.AIRPLANE_MODE')( OR intent='android.intent.action.ALL_APPS' OR intent='android.intent.action.ANSWER' OR intent='android.intent.action.APPLICATION_PREFERENCES' OR intent='android.intent.action.APPLICATION_RESTRICTIONS_CHANGED' OR intent='android.intent.action.ACTION_EXTERNAL_APPLICATIONS_AVAILABLE' OR intent='android.intent.action.APP_ERROR' OR intent='android.intent.action.ASSIST' OR intent='android.intent.action.PHONE_STATE' OR intent='android.intent.action.ATTACH_DATA' OR intent='android.intent.ACTION_SCREEN_OFF' OR intent='android.intent.action.BATTERY_CHANGED' OR intent='android.intent.action.BATTERY_CHANGED_ACTION' OR intent='android.intent.action.BATTERY_LOW' OR intent='android.service.wallpaper.WallpaperService' OR intent='android.intent.action.QUICKBOOT_POWERON' OR intent='android.net.conn.CONNECTIVITY_CHANGE' OR intent='android.intent.action.BATTERY_OKAY' OR intent='android.intent.action.START_SMS_SERVICE' OR intent='android.app.action.ACTION_DEVICE_ADMIN_DISABLE_REQUESTED' OR intent='android.intent.action.BOOT_COMPLETED' OR intent='android.intent.action.BUG_REPORT' OR intent='android.intent.action.CALL' OR intent='android.intent.action.RESPOND_VIA_MESSAGE' OR intent='android.intent.action.CALL_BUTTON' OR intent='android.intent.action.CAMERA_BUTTON' OR intent='android.intent.action.CARRIER_SETUP'));

Webmatrix if statement issues. any ideas?

I have written a SQL query, which extracts the lowest weekly price for a property stored in my database:
var rPropertyId = Request.QueryString["PropertyID"];
var cheapestrate = "SELECT TOP 1 * FROM RateInfo WHERE PropertyID=#0 ORDER BY RateWeekly ASC";
var qcheapestrate = db.QuerySingle (cheapestrate, rPropertyId);
I'm pretty confident that this statement is correct. The problem i have, is that not ALL propertys have pricing, so i only want to show this price if they do. I have created the below if statement, but it's telling me i'm missing an ; somewhere?
#if(qcheapestrate=!null){
Rates From qcheapestrate.rateweekly per week
}
So i'm trying to check if the query returns an entry. if it does, i want to show the lowest "rateweekly" value. Hopefully this all makes sense!
Try this...
#if(qcheapestrate!=null){
<text>Rates From</text>
#qcheapestrate.rateweekly
<text>per week</text>
}

Django ORM query field weight?

I'm doing the following query:
People.objects.filter(
Q(name__icontains='carolina'),
Q(state__icontains='carolina'),
Q(address__icontains='carolina'),
)[:9]
I want the first results of the query to be the people who is named "Carolina" (and also matches other fields, but name first). The problem is that I don't think is any way to determine a field "weight" or "priority".
Any idea?
Thanks!
You'll need to do 3 queries for this to work:
names_match = People.objects.filter(name__icontains='carolina')[:9]
states_match = People.objects.filter(state__icontains='carolina')[:9]
addresses_match = People.objects.filter(address__icontains='carolina')[:9]
all_objects = list(names_match) + list(states_match) + list(addresses_match)
all_objects = all_objects[:9]
There are two problems with this approach, which are fairly easily worked round:
It does unnecessary queries (what if names_match contained enough items already).
It allows for duplicates (what if someone in North Carolina is called Carolina?)
This should work:
qs = People.objects.filter(name__icontains='carolina') | People.objects.filter( Q(state__icontains = 'carolina'), Q(address__icontains='carolina')).distinct()
qs = list(qs)[:9]
Or if you want a pure duplicate free list:
qs = list(set(qs))[:9] #for a duplicate free list

MySql: Best way to run high number of search queries on a table

I have two tables, one is static database that i need to search in, the other is dynamic that i will be using to search the first database. Right now i have two separate queries. First on page load, values from second table are passed to first one as search term, and i am "capturing" the search result using cURL. This is very inefficient and probably really wrong way to do it, so i need help in fixing this issue. Currently page (html, front-end) takes 40 seconds to load.
Possible solutions: Turn it into function, but still makes so many calls out. Load table into memory and then run queries and unload cache once done. Use regexp to help speed up query? Possible join? But i am a noob so i can only imagine...
Search script:
require 'mysqlconnect.php';
$id = NULL;
if(isset($_GET['n'])) { $id = mysql_real_escape_string($_GET['n']); }
if(isset($_POST['n'])) { $id = mysql_real_escape_string($_POST['n']); }
if(!empty($id)){
$getdata = "SELECT id, first_name, last_name, published_name,
department, telephone FROM $table WHERE id = '$id' LIMIT 1";
$result = mysql_query($getdata) or die(mysql_error());
$num_rows = mysql_num_rows($result);
while($row = mysql_fetch_array($result, MYSQL_ASSOC))
{
echo <<<PRINTALL
{$row[id]}~~::~~{$row[first_name]}~~::~~{$row[last_name]}~~::~~{$row[p_name]}~~::~~{$row[dept]}~~::~~{$row[ph]}
PRINTALL;
}
}
HTML Page Script:
require 'mysqlconnect.php';
function get_data($url)
{
$ch = curl_init();
$timeout = 5;
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$getdata = "SELECT * FROM $table WHERE $table.mid != '1'ORDER BY $table.$sortbyme $o LIMIT $offset, $rowsPerPage";
$result = mysql_query($getdata) or die(mysql_error());
while($row = mysql_fetch_array($result, MYSQL_ASSOC))
{
$idurl = 'http://mydomain.com/dir/file.php?n='.$row['id'].'';
$p_arr = explode('~~::~~',get_data($idurl));
$p_str = implode(' ',$p_arr);
//Use p_srt and p_arr if exists, otherwise just output rest of the
//html code with second table values
}
As you can see, second table may or may not have valid id, hence no results but second table is quiet large, and all in all, i am reading and outputting 15k+ table cells. And as you can probably see from the code, i have tried paging but that solution doesn't fit my needs. I have to have all of the data on client side in single html page. So please advice.
Thanks!
EDIT
First table:
id_row id first_name last_name dept telephone
1 aaa12345 joe smith ANS 800 555 5555
2 bbb67890 sarah brown ITL 800 848 8848
Second_table:
id_row type model har status id date
1 ATX Hybrion 88-85-5d-id-ss y aaa12345 2011/08/12
2 BTX Savin none n aaa12345 2010/04/05
3 Full Hp 44-55-sd-qw-54 y ashley a 2011/07/25
4 ATX Delin none _ smith bon 2011/04/05
So the second table is the one that gets read and displayed, first is read and info displayed if ID is positive match. ID is only unique in the first one, second one has multi format input so it could or could not be ID as well as could be duplicate ID. Hope this gives better understanding of what i need. Thanks again!
A few things:
Curl is completely unnecessary here.
Order by will slow down your queries considerably.
I'd throw in an if is_numeric check on the ID.
Why are you using while and mysql_num_rows when you're limiting to 1 in the query?
Where are $table and these other things being set?
There is code missing.
If you give us the data structure for the two tables in question we can help you with the queries, but the way you have this set up now, I'm surprised its even working at all.
What you're doing is, for each row in $table where mid!=1 you're executing a curl call to a 2nd page which takes the ID and queries again. This is really really bad, and much more convoluted than it needs to be. Lets see your table structures.
Basically you can do:
select first_name, last_name, published_name, department, telephone FROM $table1, $table2 WHERE $table1.id = $table2.id and $table2.mid != 1;
Get rid of the curl, get rid of the exploding/imploding.

MySQL - Perl: How to get array of zip codes within submitted "x" miles of submitted "zipcode" in Perl example

I have found many calculations here and some php examples and most are just over my head.
I found this example:
SELECT b.zip_code, b.state,
(3956 * (2 * ASIN(SQRT(
POWER(SIN(((a.lat-b.lat)*0.017453293)/2),2) +
COS(a.lat*0.017453293) *
COS(b.lat*0.017453293) *
POWER(SIN(((a.lng-b.lng)*0.017453293)/2),2))))) AS distance
FROM zips a, zips b
WHERE
a.zip_code = '90210' ## I would use the users submitted value
GROUP BY distance
having distance <= 5; ## I would use the users submitted value
But, I am having trouble understanding how to implement the query with my database.
It looks like that query has all I need.
However, I cannot even find/understand what b.zip_code actually is! (whats the b. and zips a, zips b?)
I also do not need the state in the query.
My mySQL db structure is like this:
ZIP | LAT | LONG
33416 | 26.6654 | -80.0929
I wrote this in attempt to return some kind of results (not based on above query) but, it only kicks out one zip code.
## Just for a test BUT, in reality I desire to SELECT a zip code WHERE ZIP = the users submitted zip code
## not by a submitted lat lon. I left off the $connect var, assume it's there.
my $set1 = (26.6654 - 0.20);
my $set2 = (26.6654 + 0.20);
my $set3 = (-80.0929 - 0.143);
my $set4 = (-80.0929 + 0.143);
my $test123 = $connect->prepare(qq{SELECT `ZIP` FROM `POSTAL`
WHERE `LAT` >= ? AND `LAT` <= ?
AND `LONG` >= ? AND `LONG` <= ?}) or die "$DBI::errstr";
$test123->execute("$set1","$set2","$set3","$set4") or die "$DBI::errstr";
my $cntr;
while(#zip = $test123->fetchrow_array()) {
print qq~$zip[$cntr]~;
push(#zips,$zip[$cntr]);
$cntr++;
}
As you can see, I am quite the novice so, I need some hand holding here with verbose explanation.
So, in Perl, how can I push zip codes into an array from a USER SUBMITTED ZIP CODE and user submitted DISTANCE in miles. Can be a square instead of a circle, not really that critical of a feature. Faster is better.
I'll tackle the small but crucial part of the question:
However, I cannot even find/understand what b.zip_code actually is! (whats the "b." and "zips a, zips b"?)
Basically, the query joins two tables. BUT, both tables being joined are in fact the same table - "zips" (in other words, it joins "zips" table to itself"). Of course, since the rest of the query needs to understand when you are referring to the first copy of the "zips" table and when to the second copy of the "zips" table, you are giving a table alias to each copy - to wit, "a" and "b"'.
So, "b.xxx" means "column xxx from table zips, from the SECOND instance of that table being joined".
I don't see what's wrong with your first query. You have latitude and longitude in your database (if I'm understanding, you're comparing a single entry to all others). You don't need to submit or return the state that's just part of the example. Make the first query work like this:
my $query = "SELECT b.zip_code,
(3956 * (2 * ASIN(SQRT(
POWER(SIN(((a.lat-b.lat)*0.017453293)/2),2) +
COS(a.lat*0.017453293) *
COS(b.lat*0.017453293) *
POWER(SIN(((a.lng-b.lng)*0.017453293)/2),2))))) AS distance
FROM zips a, zips b WHERE
a.zip_code = ?
GROUP BY distance having distance <= ?";
my $sth = $dbh->prepare($query);
$sth->execute( $user_submitted_zip, $user_submitted_distance );
while( my ($zip, $distance) = $sth->fetchrow() ) ) {
# do something
}
This won't be that fast, but if you have a small record set ( less than 30k rows ) it should be fine. If you really want to go faster you should look into a search engine such as Sphinx which will do this for you.
fetchrow_array returns a list of list references, essentially a two-dimensional array, where each row represents a different result from the database query and each column represents a field from the query (in your case, there is only one field, or column, per row).
Calling while ($test123->fetchrow_array()) will cause an infinite loop as your program executes the query over and over again. If the query returns results, then the while condition will be satisfied and the loop will repeat. The usual idiom would be to say something more like for my $row ($test123->fetchrow_array()) { ..., which will only execute the query once and then iterate over the results.
Each result is a list reference, and the zip code you are interested in is in the first (and only) column, so you could accumulate the results in an array like this:
my #zips = (); # for final results
for my $row ($test123->fetchrow_array()) {
push #zips, $row->[0];
}
or even more concisely with Perl's map statement:
my #zips = map { $_->[0] } $test123->fetchrow_array()
which does the same thing.