Group By Issue converting MySQL to PostgreSQL - mysql

I am trying to convert an old symfony 1.4 app from MySQL to PostgreSQL.
I keep running into the same issue over and over again.
I have some long joins and it is always complaining about having to have a Group By... and in some cases I add the Group By and it asks for another one.
I am new to postgres and I have absolutely no idea what is going on.
Here are the doctrine queries and error messages.
public function getClassifieds($gs_id, $sort_param, $sorter) {
$q = Doctrine_Query::create()
->select('a.*, u.*, t.*,ug.*,ur.*, avg(ur.rating) as rating ')
->from('Items a')
->leftJoin('a.MonitoredAds ma ON ma.item_id = a.id')
->leftJoin('a.GameCategories gc ON gc.game_id = a.game_id')
->leftJoin('a.User u ON u.id = a.user_id')
->leftJoin('u.Profile ug ON ug.user_id = u.id')
->leftJoin('a.UserRatings ur ON ug.user_id = ur.seller_id')
->where('a.game_server_id = ?', $gs_id)
->andWhere('a.quantity > ?', 0);
switch ($sorter) {
case 'datetime_oldest_to_newist':
$q->orderBy('a.created_at asc');
break;
case 'datetime_newest_to_oldest':
$q->orderBy('a.created_at desc');
break;
case 'price_lowest_to_highest':
$q->orderBy('a.price + 0 ASC');
break;
case 'price_highest_to_lowest':
$q->orderBy('a.price + 0 DESC');
break;
case 'quantity_lowest_to_highest';
$q->orderBy('a.quantity asc');
break;
case 'quantity_highest_to_lowest';
$q->orderBy('a.quantity desc');
break;
case 'quantity_highest_to_lowest';
$q->orderBy('a.quantity desc');
break;
case 'sortby_usernameasc':
$q->orderBy('u.username asc');
break;
case 'sortby_usernamedesc':
$q->orderBy('u.username desc');
break;
case 'sortby_userstatus':
$q->orderBy('ug.user_status asc');
break;
}
return $q;
}
Gives Me this error message
SQLSTATE[42803]: Grouping error: 7 ERROR: column "i.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT i.id AS i__id, i.featured AS i__featured, i.status AS...
So I change the code to ...
->andWhere('a.quantity > ?', 0)
->groupBy('a.id');
Then I get a new error...
SQLSTATE[42803]: Grouping error: 7 ERROR: column "s.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: ... AS i__created_at, i.updated_at AS i__updated_at, s.id AS s_...
I stopped here because this is changing the behaviour of the query in a major way and I really don't know what's going on.
I am not afraid of reading the docs but I don't even know where to start.
If you know what the issue could be please let me know. If you know where I should be looking in the docs please let me know that as well.
Thank you in advance.
Scott

It is not correct for PostgreSQL without GROUP BY all columns besides ur.rating:
->select('a.*, u.*, t.*,ug.*,ur.*, avg(ur.rating) as rating ')
Every selected and ordered column must be in GROUP BY or in agregate function.
For example, rewrite query:
->select('u.id, avg(ur.rating) as rating ')
...
->orderBy('u.id');
->groupBy('u.id');

Related

Codeigniter's Model with QueryBuilder JOIN query

Really new to working with CI4's Model and struggling to adapt my existing MySQL JOIN queries to work with the examples in its User Guide.
I have adapted part of my code like so:
public function brand_name($brand_name_slug)
{
return $this->asArray()
->where('availability', 'in stock')
->where('sku !=', '')
->where('brand_name_slug', $brand_name_slug)
->groupBy('gtin')
->orderBy('brand_name, subbrand_name, product, size, unit')
->findAll();
}
It works fine. I have looked at examples, and figured out I can add the code ->table('shop a') and it still works, but I also need to to add the following JOIN statement:
JOIN (SELECT gtin, MIN(sale_price) AS sale_price FROM shop GROUP BY gtin) AS b ON a.gtin = b.gtin AND a.sale_price = b.sale_price
As soon as I add ->join('shop b', 'a.gtin = b.gtin and a.sale_price = b.sale_price') I get a '404 - File Not Found' error.
When I look at all examples of CI4 joins and adapt my code to fit, my foreach($shop as $row) loop generates a 'Whoops...' error because they end with a getResult() or getResultArray - instead of findAll().
Which is the way forward, and do I need to change my foreach loop.
Full MySQL statement:
SELECT * FROM shop a JOIN (SELECT gtin, MIN(sale_price) AS sale_price FROM shop GROUP BY gtin) AS b ON a.gtin = b.gtin AND a.sale_price = b.sale_price WHERE availability = 'in stock' AND sku != '' AND brand_name_slug = $brand_name_slug GROUP BY gtin ORDER BY brand_name, subbrand_name, product, size
Query builders have their limits. That's why the query method exists. If you have a complex query I'd advise you to just use $this->query();.
It will make you lose less time and effort converting something you know already works. And in the top of that, while converting complex queries you usually end up using the query builder but with big part of your SQL in it.
In your model extending CodeIgniter\Model :
$query = $this->db->query("SELECT * FROM shop a JOIN (SELECT gtin, MIN(sale_price) AS sale_price FROM shop GROUP BY gtin) AS b ON a.gtin = b.gtin AND a.sale_price = b.sale_price WHERE availability = 'in stock' AND sku != '' AND brand_name_slug = \$brand_name_slug GROUP BY gtin ORDER BY brand_name, subbrand_name, product, size");
// your array result
$result_array = $query->getResultArray();
// your object result
$result_object = $query->getResult();
BaseBuilder Class in Codeigniter expects the first join parameter to be the table name. So try passing the table name and join it on the table name itself. I haven't personally used the table aliases so I might also be wrong.
Following are the parameter that the JOIN query expects :
public function join(string $table, string $cond, string $type = '', bool $escape = null)
Here, it expects the first name be a table, so try out by switching aliases for the table's name directly.
For your second part of query, It would be better if you could show the whole error rather than just posting the first of the error.
Managed to figure it out in the end:
public function brand_name($brand_name_slug)
{
return $this
->db
->table('shop a')
->select()
->join('(SELECT sku, MIN(sale_price) AS sale_price FROM shop GROUP BY sku) AS b', 'a.sku = b.sku AND a.sale_price = b.sale_price')
->where('availability', 'in stock')
->where('a.sku !=', '')
->where('brand_name_slug', $brand_name_slug)
->groupBy('a.sku')
->orderBy('brand_name, subbrand_name, product, size, unit')
->get()
->getResult();
}
Thanks for all your pointers!

mysql results ordering incorrectly using set FIELD and case

I've got the following variable and query. For some reason, it is sorting the results in the wrong order.
$fieldlist = 'SMOOTH,EMBOSSED,AAR,EMBOSSED AAR';
$strap_query = $db1q->query("
SELECT id,name,tag,size,break,footage,coils_per_skid,core_size,grade,color,cost
FROM Plastic_Strapping
WHERE subcat=$subcat AND visibility='1'
ORDER BY case when tag in ('$fieldlist') then 0 else 1 end, FIELD(tag,'$fieldlist'), sort_order ASC")
or die ('Unable to execute query. '. mysqli_error($db1q));
The order is SUPPOSED TO BE
SMOOTH
EMBOSSED
AAR
EMBOSSED AAR
But instead, it is sorting them in the following INCORRECT order
SMOOTH
AAR
EMBOSSED
EMBOSSED AAR
Any idea why?
is it because EMBOSSED AAR contains EMBOSSED? so its putting it after AAR? If so, any idea how to fix that?
Edit
Placing 'SMOOTH','EMBOSSED','AAR','EMBOSSED AAR' in the ORDER BY FIELD section works; however, I'd still like to be able to edit this via a variable. The following does NOT work.
$fieldlist = "'SMOOTH','EMBOSSED','AAR','EMBOSSED AAR'";
$strap_query = $db1q->query("SELECT id,name,tag,size,break,footage,coils_per_skid,core_size,grade,color,cost FROM Plastic_Strapping WHERE subcat=$subcat AND visibility='1' ORDER BY case when tag in ('$fieldlist') then 0 else 1 end, FIELD(tag, $fieldlist), sort_order ASC") or die ('Unable to execute query. '. mysqli_error($db1q));
throws this error
Unable to execute query. You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right
syntax to use near 'SMOOTH','EMBOSSED','AAR','EMBOSSED AAR'') then 0
else 1 end, FIELD(tag, 'SMOOTH'' at line 1
Try this:
$fieldlist = 'SMOOTH,EMBOSSED,AAR,EMBOSSED AAR';
$query = "SELECT id,name,tag,size,break,footage,coils_per_skid,core_size,grade,
color,cost FROM Plastic_Strapping WHERE subcat = $subcat AND visibility='1'
ORDER BY FIELD( tag, '".implode("','",explode(',',$fieldlist))."' )";
Update
If your $fieldlist variable is like this:
$fieldlist = "'SMOOTH','EMBOSSED','AAR','EMBOSSED AAR'";
Your query would be like this:
$query = "SELECT id,name,tag,size,break,footage,coils_per_skid,core_size,grade,
color,cost
FROM Plastic_Strapping WHERE subcat = $subcat AND visibility = '1'
ORDER BY case when tag in ($fieldlist) then 0 else 1 end,
FIELD(tag, $fieldlist), sort_order ASC ";

How to select DB Expression as value using knex/Bookshelf

I'm trying to execute the following query using knex.js and MySql
SELECT
m.id,
TIME(date_created) AS `timestamp`,
u.username,
m.`message`
FROM
`messages` AS m
INNER JOIN users AS u ON u.id = m.user_id
WHERE
m.game_id IS NULL
AND m.`date_created` > DATE_SUB(
CURRENT_TIMESTAMP (),
INTERVAL 12 HOUR
)
ORDER BY
m.`date_created` ASC
LIMIT 50
For proper handling expressions in where closure such as DATE_SUB(CURRENT_TIMESTAMP(), INTERVAL 12 HOUR) according to documentation there is whereRow() method in knex.
I tried to use select() method as
select('messages.id', 'TIME(date_created) AS timestamp', 'users.username', 'messages.message')
But knex masks TIME(date_created) expression in a way it should to be a column name. Does anybody know a way to use a custom expressions in selects?
I did not found exact answer on my question but I've found a solution which allows me to move forward. I created separate model which uses standard Bookshelf(knex) export:
var Bookshelf = require('bookshelf')(knex);
module.exports.DB = Bookshelf;
And created separate method in that model where I could use DB.knex.raw() for masking DB expressions in SELECT. So I became able to write the query above in the following way:
var DB = require('./../db').DB;
var Messages = DB.Model.extend({
tableName: 'messages',
hasTimestamps: false,
idAttribute: 'id'
},
{
getMessagesHistory: function (gameId) {
return this.query().select('messages.id', DB.knex.raw('TIME(date_created) AS timestamp'), 'users.username', 'messages.message')
.innerJoin('users', 'messages.user_id', 'users.id')
.whereRaw("messages.game_id " + (gameId == 0 ? "IS NULL" : "= " + gameId))
.andWhereRaw("messages.date_created > DATE_SUB(CURRENT_TIMESTAMP(), INTERVAL 12 HOUR)")
.orderBy('messages.date_created', 'ASC')
.limit(50);
}
}
);
module.exports = Messages;
You can wrap any argument with knex.raw() to tell the framework that it's a raw piece of SQL:
select(
'messages.id',
knex.raw('TIME(date_created) AS timestamp'),
'users.username',
'messages.message',
)
See http://knexjs.org/#Raw-Bindings

how to translate a very long mysql query with select and join to zend framework 1.11 model

I have this mysql query:
SELECT
freeAnswers.*,
(SELECT `districtCode`
FROM `geodatas`
WHERE `zipCode` = clients.zipCode
GROUP BY `zipCode`
LIMIT 0, 1) as districtCode,
clients.zipCode,
clients.gender,
clients.startAge,
clients.endAge,
clients.mail,
clients.facebook,
surveys.customerId,
surveys.activityId,
surveys.name as surveyName,
customers.companyName,
activities.name as activityName
FROM freeAnswers,
clients,
surveys,
customers,
activities
WHERE freeAnswers.surveyId = surveys.id
AND surveys.customerId = customers.id
AND activities.id = surveys.activityId
AND clients.id = freeAnswers.clientId
AND customers.id = 1
ORDER BY activityName asc
LIMIT 0, 10
the query is correct on my mysql server but when I try to use it in Zend Framework 1.11 model
I get this error: Mysqli prepare error: Operand should contain 1 column(s)
Please, could anyone help me to make it run well?
Best Regards,
Elaidon
Here is some code that should work. Zend_Db_Select doesn't really provide a way to select from multiple tables in the FROM clause without using a JOIN so this feels a bit hackish to me in regards to one small part of the query. Your best bet will probably be to rewrite the query using JOINs where appropriate.
$subselect = $db->select()
->from('geodatas', 'districtCode')
->where('zipCode = clients.zipCode')
->group('zipCode')
->limit(1, 0);
$from = $db->quoteIdentifier('freeAnswers') . ', ' .
$db->quoteIdentifier('clients') . ', ' .
$db->quoteIdentifier('surveys') . ', ' .
$db->quoteIdentifier('customers') . ', ' .
$db->quoteIdentifier('activities');
$select = $db->select()
->from(array('activities' => new Zend_Db_Expr($from)),
array('freeanswers.*',
'districtCode' =>
new Zend_Db_Expr('(' . $subselect . ')'),
'clients.zipCode', 'clients.gender', 'clients.startAge',
'clients.endAge', 'clients.mail', 'clients.facebook',
'clients.customerId', 'clients.activityId',
'surveyName' => 'surveys.name', 'customers.companyName',
'activityName' => 'activities.name'))
->where('freeAnswers.surveyId = surveys.id')
->where('surveys.customerId = customers.id')
->where('activities.id = surveys.activityId')
->where('clients.id = freeAnswers.clientId')
->where('customers.id = ?', 1)
->order('activityName ASC')
->limit(10, 0);
The only reason I say it is hackish is because of the line:
->from(array('activities' => new Zend_Db_Expr($from)),
Since from() really only works with one table, I create a Zend_Db_Expr and specify the correlation as the last table name in the expression. If you don't pass a Zend_Db_Expr, it will either quote your comma separated table name incorrectly, or if you pass an array of table names, it just uses the first. When you pass a Zend_Db_Expr with no name, it defaults to use AS t which also doesn't work in your case. That is why I put it as is.
That returns the exact SQL you provided except for the last thing mentioned. Here is actually what it returns:
SELECT
`freeanswers`.*,
(SELECT `geodatas`.`districtCode`
FROM `geodatas`
WHERE (zipCode = clients.zipCode)
GROUP BY `zipCode`
LIMIT 1) AS `districtCode`,
`clients`.`zipCode`,
`clients`.`gender`,
`clients`.`startAge`,
`clients`.`endAge`,
`clients`.`mail`,
`clients`.`facebook`,
`clients`.`customerId`,
`clients`.`activityId`,
`surveys`.`name` AS `surveyName`,
`customers`.`companyName`,
`activities`.`name` AS `activityName`
FROM `freeAnswers`,
`clients`,
`surveys`,
`customers`,
`activities` AS `activities`
WHERE (freeAnswers.surveyId = surveys.id)
AND (surveys.customerId = customers.id)
AND (activities.id = surveys.activityId)
AND (clients.id = freeAnswers.clientId)
AND (customers.id = 1)
ORDER BY `activityName` ASC
LIMIT 10
So that will work but eventually you will want to rewrite it using JOIN instead of specifying most of the WHERE clauses.
When dealing with subqueries and Zend_Db_Select, I find it easy to write each subquery as their own queries before writing the final query, and just insert the subqueries where they need to go and Zend_Db handles the rest.
Hope that helps.

Using comparison as alias in select for Doctrine2

Trying to do this in Doctrine2:
...->createQuery('SELECT m.id, (m.status != 1) as verified...
But that throws an error - if I take parenthesis off I get another error. How do I achieve this m.status comparison?
Thanks
Doctrine 2 doesn't support these comparisons in the SELECT clause (at least not up to 2.3, not sure about 2.4).
You can use a CASE expression as workaround:
SELECT m.id, CASE WHEN m.status != 1 THEN 1 ELSE 0 END AS verified ...
or:
SELECT m.id, CASE WHEN m.status = 1 THEN 0 ELSE 1 END AS verified ...
If you need verified for an ORDER BY clause (or something like that), but don't actually need it in the result, you can use the HIDDEN expression:
SELECT m.id, CASE WHEN m.status = 1 THEN 0 ELSE 1 END AS HIDDEN verified ...
A completely different solution is to write a custom DQL function.
You can use the solution proposed here: Cumulative DQL with Doctrine
When working with entities, keep in mind that adding selects will make the query return an array for each result:
$res = $em->createQueryBuilder()
->from('BlogPost', 'p')
->select('p')
->addSelect('(2+3) AS dummy')
->getQuery()->getResult();
Iterating over $res will return an array:
foreach($res as $mixed){
echo get_class($mixed[0]); //$mixed[0] contains the BlogPost
echo $mixed['dummy']; //displays the dummy result (5)
}
check this out: 13.2.4. Using Expr* classes to create conditionals
using Expression methods you could do something like:
$qb = $this->entityManager->createQueryBuilder();
$query = $qb->select('m.id')
->from('Entities\MyEntity', 'm')
/*neq() is the "not equal" comparison function*/
->where($qb->expr()->neq('m.status', $someValue)),
->getQuery();