Laravel querybuilder distinct function not working in search - mysql

I tried putting distinct() in my query but when i get the results in my frontend and in the api, I still get duplicate records. Does anyone know why distinct is not working in my code?
My code
$result = DB::connection('mysql2')
->table('xp_pn_ura_transactions')
->whereRaw(DB::raw("CONCAT(block, ' ', street,' ',project_name,' ', postal_code,'')LIKE '%$request->projectname%' order by STR_TO_DATE(sale_date, '%d-%M-%Y') asc"))
->limit($request->limit)
->distinct()
->get();
return \Response::json(array(
//'total_count' => $count,
'result' => $result,
));
Front end result
My response, I only get the first two objects that duplicates
{
"id": 228686,
"transtype": "RESI",
"project_name": "WATERFRONT WAVES",
"unitname": "08-06 ",
"block": "760",
"street": "Bedok Reservoir Road ",
"level": "08",
"stack": "06 ",
"no_of_units": "1",
"area": "147",
"type_of_area": "Strata",
"transacted_price": "1300500",
"nettprice": "-",
"unitprice_psm": "8847",
"unitprice_psf": "822",
"sale_date": "20-JAN-2008",
"contract_date": " ",
"property_type": "Condominium",
"tenure": "99 Yrs From 31/10/2007",
"completion_date": "Uncompleted",
"type_of_sale": "New Sale",
"purchaser_address_indicator": "Private",
"postal_district": "16",
"postal_sector": "47",
"postal_code": "479245",
"planning_region": "East Region",
"planning_area": "Bedok",
"update_time": "2019-12-09 17:14:35"
},
{
"id": 224686,
"transtype": "RESI",
"project_name": "WATERFRONT WAVES",
"unitname": "08-06 ",
"block": "760",
"street": "Bedok Reservoir Road ",
"level": "08",
"stack": "06 ",
"no_of_units": "1",
"area": "147",
"type_of_area": "Strata",
"transacted_price": "1300500",
"nettprice": "-",
"unitprice_psm": "8847",
"unitprice_psf": "822",
"sale_date": "20-JAN-2008",
"contract_date": " ",
"property_type": "Condominium",
"tenure": "99 Yrs From 31/10/2007",
"completion_date": "Uncompleted",
"type_of_sale": "New Sale",
"purchaser_address_indicator": "Private",
"postal_district": "16",
"postal_sector": "47",
"postal_code": "479245",
"planning_region": "East Region",
"planning_area": "Bedok",
"update_time": "2019-12-09 17:11:57"
}
They got different id but same records, is there a way to ignore the id and get the other fields?

You need to select the field that you need to distinct, or it will distinct all the fields that you selected:
So according to your post, the id and updated_time are not duplicated, you don't need to select it out.
Try something like this:
$result = DB::connection('mysql2')
->table('xp_pn_ura_transactions')
->whereRaw(DB::raw("CONCAT(block, ' ', street,' ',project_name,' ', postal_code,'')LIKE '%$request->projectname%' order by STR_TO_DATE(sale_date, '%d-%M-%Y') asc"))
->limit($request->limit)
# select the fields which is duplicated.(In your post, select the field without id and updated_time)
->select("transtype",
"project_name",
"unitname",
"block",
"street",
"level",
"stack",
"no_of_units",
"area",
"type_of_area",
"transacted_price",
"nettprice",
"unitprice_psm",
"unitprice_psf",
"sale_date",
"contract_date",
"property_type",
"tenure",
"completion_date",
"type_of_sale",
"purchaser_address_indicator",
"postal_district",
"postal_sector",
"postal_code",
"planning_region",
"planning_area")
->distinct()
->get();
if you need to select the fields not duplicated, you can use groupBy() instead of distinct()

Related

How to join an element which is in an array which is also an part of another array for best performance in couchbase?

Below is the sample document for organization
{
"org": {
"id": "org_2_1084",
"organizationId": 1084,
"organizationName": "ABC",
"organizationRoles": [
{
"addressAssociations": [
{
"activeDate": "2019-08-03T18:52:00.857Z",
"addressAssocTypeId": -2,
"addressId": 100,
"ownershipStatus": 1,
"srvAddressStatus": 1
},
{
"activeDate": "2019-08-03T18:52:00.857Z",
"addressAssocTypeId": -2,
"addressId": 105,
"ownershipStatus": 1,
"srvAddressStatus": 1
}
],
"name": "NLUZ",
"organizationRoleId": 893,
"roleSpecId": -104,
"statusId": 1,
"statusLastChangedDate": "2019-08-04T13:14:44.616Z"
},
{
"addressAssociations": [
{
"activeDate": "2019-08-03T18:52:00.857Z",
"addressAssocTypeId": -2,
"addressId": 582,
"ownershipStatus": 1,
"srvAddressStatus": 1
},
{
"activeDate": "2019-08-03T18:52:00.857Z",
"addressAssocTypeId": -2,
"addressId": 603,
"ownershipStatus": 1,
"srvAddressStatus": 1
}
],
"name": "TXR",
"organizationRoleId": 894,
"partyRoleAssocs": [
{
"partyRoleAssocId": "512"
}
],
"roleSpecId": -103,
"statusId": 1,
"statusLastChangedDate": "2019-08-04T13:14:44.616Z"
},
}
and below is the sample document for address
{
"address": {
"address1": "string",
"address2": "string",
"addressId": "1531",
"changeWho": "string",
"city": "string",
"fxGeocode": "string",
"houseNumber": "string",
"id": "1531",
"isActive": true,
"postalCode": "string",
"state": "string",
"streetName": "string",
"tenantId": "2",
"type": "address",
"zip": "string"
}
}
In an organization there are multiple organizationRoles and in an organizationRole there are multiple addressAssociations.Each addressAssociation contains an addressId and corresponding to this addressId
address is stored in address document.
Now i have to get organizationRole name, organizationRole id, city, zip from the two documents.
What should be the best way to approach this situation for the best performance in couchbase?
I am thinking about using join but not able to come up with an exact query for this scenario.
I have tried the below query but its not working.
select *
from 'contact' As A UNNEST 'contact'.organizationRoles as Roles
UNNEST Roles.addressAssociations address
Join 'contact' As B
on address.addressID=B.addressID
where A.type="organization" and B.type="address";
You are in the right direction.
In addressAssociations the addressId is number, In address addressId is string. string and number not same and no implicit type casting. You must fix data or do explicit type casting using TOSTRING(), TONUMBER() etc...
Also N1QL field names are case-sensitive your query using addressID vs addressId (in the document)
SELECT r.name AS organizationRoleName, r.organizationRoleId, a.city, a.zip
FROM contact AS c
UNNEST c.organizationRoles AS r
UNNEST r.addressAssociations AS aa
jOIN contact AS a
ON aa.addressId = a.addressId
WHERE c.type = "organization" AND a.type = "address";
CREATE INDEX ix1 ON contact(addressId, city, zip) WHERE type = "address";
Check out https://blog.couchbase.com/ansi-join-support-n1ql/

Format JSON Postgresql

I'm using inner join to join 3 tables, Owner, Store and Machine.
I'm trying to view output JSON from multiple tables like this:
SELECT ow.*, st.*, ma.*
FROM owner ow
INNER JOIN st.store ON ow.OwnerId = st.OwnerId
INNER JOIN machine ma ON ma.StoreId = st.StoreId;
I want JSON formatted like this:
{
"OwnerId": "1d2dd",
"Name": "name test",
"Store":[{
"StoreId": "s3ss5",
"Name": "Store1",
"Code": "bla",
"Machine":[{
"MachineId": "axpeo",
"Name": "Machine1",
"Type": "type1"
}]
},
{
"StoreId": "ddf22",
"Name": "Store2",
"Code": "ble",
"Machine":[{
"MachineId": "weds",
"Name": "Machine2",
"Type": "type2"
},
{
"MachineId": "axdso",
"Name": "Machine3",
"Type": "type3"
}]
}]
}
but the return JSON is not formatted like this
I'm using PostgreSQL.
The easiest (and probably only sensible) way to do this is to build JSON sub-documents from individual records at table level and only then hierarchically joining them:
SELECT json_build_object('OwnerId', ownerid,
'Name', name,
'Store', stores)
FROM owner
JOIN (
SELECT ownerid,
json_agg(
json_build_object('StoreId', storeid,
'Name', name,
'Code', code,
'Machine', machines)) AS stores
FROM store
JOIN (
SELECT storeid,
json_agg(
json_build_object('MachineId', machineid,
'Name', name,
'Type', type)) AS machines
FROM machine
GROUP BY storeid) m USING (storeid)
GROUP BY ownerid) s USING (ownerid);
The output is not exactly what i want, but it is better...this is the output
[{
"OwnerId": "1d2dd",
"Name": "name test",
"Store":{
"StoreId": "s3ss5",
"Name": "Store1",
"Code": "bla",
"Machine":{
"MachineId": "axpeo",
"Name": "Machine1",
"Type": "type1"
}
}
},
{
"OwnerId": "1d2dd",
"Name": "name test",
"Store":{
"StoreId": "ddf22",
"Name": "Store2",
"Code": "ble",
"Machine":{
"MachineId": "weds",
"Name": "Machine2",
"Type": "type2"
}
}
},
{
"OwnerId": "1d2dd",
"Name": "name test",
"Store":{
"StoreId": "ddf22",
"Name": "Store2",
"Code": "ble",
"Machine":{
"MachineId": "axdso",
"Name": "Machine3",
"Type": "type3"
}
}
}]
it does not join the machines from the same store yet like an array
For one-to-many relationships formatted to JSON try something like this:
SELECT "owner"."id",
json_agg(DISTINCT "store".*) AS "stores",
json_agg(DISTINCT "machine".*) AS "machines"
FROM "owners"
INNER JOIN "stores"
ON "stores"."ownerId" = "owners"."id"
INNER JOIN "machines"
ON "machines"."storeId" = "stores"."id"
WHERE "owner" = 1
GROUP BY "owner"."id";

JQ Array to new fields

I have a sample json data like:
{
"phone_number": "780-414-2085",
"city": "Edmonton",
"updated": "2015-10-19T00:03:10",
"name": "Sir William Place ",
"url": "http://www.bwalk.com/en-CA/Rent/Details/Alberta/Edmonton/Sir-William-Place",
"last_visited": "2015-10-19T00:03:10",
"rooms": [{
"available": "Available",
"bathrooms": ["1"],
"suite_type": "1 Bedroom",
"square_feet": ["594", "649"],
"deposit": ["$499"],
"price_range": ["$1059", "$1209"]
}, {
"available": "Available",
"bathrooms": ["1"],
"suite_type": "1 Bedroom + Den",
"square_feet": ["771"],
"deposit": ["$499"],
"price_range": ["$1169", "$1249"]
}, {
"available": "Available",
"bathrooms": ["1", "2"],
"suite_type": "2 Bedroom",
"square_feet": ["894", "970"],
"deposit": ["$499"],
"price_range": ["$1344", "$1494"]
}, {
"available": "Available",
"bathrooms": ["2"],
"deal": ["October FREE and $299 Security Deposit on 12 month leases "],
"suite_type": "2 Bedroom Bi-level",
"square_feet": ["894"],
"deposit": ["$499"],
"price_range": ["$1344", "$1394"]
}, {
"available": "Waiting List",
"bathrooms": ["1"],
"suite_type": "Bachelor",
"square_feet": ["540"],
"deposit": ["$499"],
"price_range": ["$1004", "$1054"]
}],
"address": "8830-85 St., Edmonton, Alberta, T6C 3C3",
"zip_code": "T6C 3C3"
}
And I am running a jq expression like:
'{phone_number, city, updated, name, address, zip_code, url, last_visited} + (.rooms[] | {suite_type, price_range_start: .price_range[0], price_range_end: .price_range[1]} + {available, square_foot_start:.square_feet[0], square_foot_end:.square_feet[1], deposit:.deposit[0], bathrooms:.bathrooms[0]})'
This gives me an ok output but repeats the same names because I just list the rooms array. I want to be able to set each item in the rooms array to something like room1, room2, room3 etc. But also to keep it in one entry, so for example with the sample here it ends up 5 entries because there is 5 rooms and name for instance gets repeated 5 times because the way I have it set now. I think I need to map the rooms to something but not sure how to do that.
Can someone advise on how to do this?
You can update the elements in the array whilst keeping the other elements as is like this:
'.rooms[] |= {suite_type, price_range_start: .price_range[0],
price_range_end: .price_range[1]} + {available,
square_foot_start:.square_feet[0], square_foot_end:.square_feet[1],
deposit:.deposit[0], bathrooms:.bathrooms[0]}'
Here is a solution which uses functions.
def common_columns:
"phone_number", "city", "updated", "name", "address", "zip_code", "url", "last_visited"
;
def common:
.phone_number, .city, .updated, .name, .address, .zip_code, .url, .last_visited
;
def room_columns(n):
range(n)
| (
"available_\(.)", "bathrooms_\(.)", "suite_type_\(.)",
"square_feet_start_\(.)", "square_feet_end_\(.)", "deposit_\(.)",
"price_range_start_\(.)", "price_range_end_\(.)"
)
;
def rooms(n):
. as $r
| range(n)
| $r.rooms[.]
| (
.available, .bathrooms[0], .suite_type,
.square_feet[0,1], .deposit[0], .price_range[0,1]
)
;
[ common_columns, room_columns(6) ]
, [ common, rooms(6) ]
| #csv
You can change the 6 to however many sets of room columns you need.

Get Count Of Nested Entities using Laravel Eloquent

I have got 3 database tables clients, coupons and categories
clients table
id,
name,
website,
description,
logo,
slug
categories table
id,
name,
slug
coupons table
id,
client_id,
category_id,
type,
coupon,
title,
description,
link,
views,
slug,
expiry
The relationship is
1) many coupons belongs to client ( many to one relationship)
2) many coupons belongs to category ( many to one relationship)
I am using laravel 5.1.
How can i get the unique count of clients with the clients details , number of coupons a client has and the count of total categories an individual client has.
simplified : i need to get the client details and display that xxx number of coupons are available in the xxx number of categories for a particular client.
so far i can get the unique client details and the number of the coupons count.
public function getAvailableClientsWithItemCountList($page = 1)
{
return Client::join('coupons', 'clients.id', '=', 'coupons.client_id')
->join('categories', 'coupons.category_id', '=', 'categories.id')
->where('coupons.expiry', '>', Carbon::today())
->groupBy('clients.id')
->skip(STORES_PER_REQUEST*($page-1))
->take(STORES_PER_REQUEST)
->get(['clients.id', 'clients.name', 'clients.slug', 'clients.logo', DB::raw('count(clients.id) as dealsCount'), DB::raw('count(categories.id) as categoriesCount')]);
}
STORES_PER_REQUEST = 9 (constant) for paginating.
thanks in advance.
If you have your relationships set up you could do something like:
/**
* Mock relationship for eager loading coupon count
*
* #return mixed
*/
public function couponCount()
{
return $this->hasOne(Coupon::class)
->selectRaw('client_id, count(*) as aggregate')
->groupBy('client_id');
}
public function getCouponCountAttribute()
{
// if relation is not loaded already, let's do it first
if (!$this->relationLoaded('couponCount')) {
$this->load('couponCount');
}
$related = $this->getRelation('couponCount');
// then return the count directly
return ($related) ? (int) $related->aggregate : 0;
}
The above can be used in your Client model, and then you can just alter the couponCount relationship method for your Category model (if you wanted to).
Then add the following for your Category count:
/**
* Mock relationship for eager loading category count
*
* #return mixed
*/
public function categoryCount()
{
return $this->hasOne(Coupon::class)
->selectRaw('category_id, count(*) as aggregate')
->groupBy('client_id, category_id');
}
public function getCategoryCountAttribute()
{
// if relation is not loaded already, let's do it first
if (!$this->relationLoaded('categoryCount')) {
$this->load('categoryCount');
}
$related = $this->getRelation('categoryCount');
// then return the count directly
return ($related) ? (int) $related->aggregate : 0;
}
You can then add a query scope in your Coupon model for getting coupons that haven't expired by something like:
public function scopeActive($query)
{
$query->where('expiry', '>', Carbon::today());
}
If you're only ever going to be getting the count for coupons that haven't expired you can add you can add this straight on to the relationship e.g.
groupBy('client)id')->active()
Now you should be able to eager load the relationship like so:
$clients = Client::with('couponCount', 'clientCount')
->skip(STORES_PER_REQUEST * ($page - 1))
->take(STORES_PER_REQUEST)
->get();
Or you could attach the query scope to the eager load i.e.
$clients = Client::with(['couponCount' => function ($q) {$q->active()}, 'clientCount' => function ($q) {$q->active()}]) ...
Hope this helps!
Okay i figured out myself with additional info as coupon type and the categories available.
The key i did was just added the case in count and removed the join of the categories table.
the final code looked like this
return App\Client::join('coupons', 'clients.id', '=', 'coupons.client_id')
->where('coupons.expiry', '>', \Carbon\Carbon::today())
->orderBy('clients.position', 'desc')
->groupBy('clients.id')
->skip(STORES_PER_REQUEST*(1-1))
->take(STORES_PER_REQUEST)
->get(['clients.id', 'clients.name', 'clients.slug', 'clients.logo', DB::raw('count(clients.id) as total'), DB::raw('count(CASE WHEN coupons.type=\'Coupon\' THEN 1 ELSE NULL END) as couponsCount'), DB::raw('count(CASE WHEN coupons.type=\'Deals\' THEN 1 ELSE NULL END) as dealsCount'), DB::raw('count(Distinct category_id) as categoriesCount')]);
The result was
[{
"id": "8",
"name": "Archies Online",
"slug": "archies-online",
"logo": "Archiesonline.jpg",
"total": "22",
"couponsCount": "20",
"dealsCount": "2",
"categoriesCount": "9"
}, {
"id": "5",
"name": "Shop Clues",
"slug": "shop-clues",
"logo": "Shopclues.jpg",
"total": "24",
"couponsCount": "24",
"dealsCount": "0",
"categoriesCount": "9"
}, {
"id": "6",
"name": "Lens Kart",
"slug": "lens-kart",
"logo": "Lenskart.jpg",
"total": "25",
"couponsCount": "25",
"dealsCount": "0",
"categoriesCount": "8"
}, {
"id": "7",
"name": "Amazer",
"slug": "amazer",
"logo": "Amzer.jpg",
"total": "21",
"couponsCount": "21",
"dealsCount": "0",
"categoriesCount": "8"
}, {
"id": "1",
"name": "Flipkart",
"slug": "flipkart",
"logo": "Flipkart.jpg",
"total": "17",
"couponsCount": "17",
"dealsCount": "0",
"categoriesCount": "9"
}, {
"id": "2",
"name": "Make My Trip",
"slug": "make-my-trip",
"logo": "Makemytrip.jpg",
"total": "11",
"couponsCount": "11",
"dealsCount": "0",
"categoriesCount": "8"
}]
This did the trick for now :);

Translation SQL to JPQL

I'm having a hard time figuring out, how to translate the following SQL to JPQL
SELECT * FROM consorder
WHERE
consorder.redistid LIKE '123%'
HAVING
MAX(consorder.redistid)
(which works like a charm under MySQL).
#Query("select c from ConsignmentOrder c" +
" where c.redistributionId like ?1" +
" having max(c.redistributionId)")
I get the following exception:
Caused by: <openjpa-2.2.2-r422266:1468616 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Encountered "max ( c . redistributionId ) <EOF>" at character 79, but expected: ["(", ")", "*", "+", "-", ".", "/", ":", "<", "<=", "<>", "=", ">", ">=", "?", "ABS", "ALL", "AND", "ANY", "AS", "ASC", "AVG", "BETWEEN", "BOTH", "BY", "CASE", "COALESCE", "CONCAT", "COUNT", "CURRENT_DATE", "CURRENT_TIME", "CURRENT_TIMESTAMP", "DELETE", "DESC", "DISTINCT", "EMPTY", "ESCAPE", "EXISTS", "FETCH", "FROM", "GROUP", "HAVING", "IN", "INDEX", "INNER", "IS", "JOIN", "KEY", "LEADING", "LEFT", "LENGTH", "LIKE", "LOCATE", "LOWER", "MAX", "MEMBER", "MIN", "MOD", "NEW", "NOT", "NULL", "NULLIF", "OBJECT", "OF", "OR", "ORDER", "OUTER", "SELECT", "SET", "SIZE", "SOME", "SQRT", "SUBSTRING", "SUM", "TRAILING", "TRIM", "TYPE", "UPDATE", "UPPER", "VALUE", "WHERE", <BOOLEAN_LITERAL>, <DATE_LITERAL>, < DECIMAL_LITERAL>, <IDENTIFIER>, <INTEGER_LITERAL>, <STRING_LITERAL2>, <STRING_LITERAL>, <TIMESTAMP_LITERAL>, <TIME_LITERAL>].
at org.apache.openjpa.kernel.jpql.JPQL.generateParseException(JPQL.java:13162)
at org.apache.openjpa.kernel.jpql.JPQL.jj_consume_token(JPQL.java:13036)
at org.apache.openjpa.kernel.jpql.JPQL.conditional_primary(JPQL.java:1980)
at org.apache.openjpa.kernel.jpql.JPQL.conditional_factor(JPQL.java:1958)
at org.apache.openjpa.kernel.jpql.JPQL.conditional_term(JPQL.java:1807)
at org.apache.openjpa.kernel.jpql.JPQL.conditional_expression(JPQL.java:1769)
at org.apache.openjpa.kernel.jpql.JPQL.having_clause(JPQL.java:1701)
at org.apache.openjpa.kernel.jpql.JPQL.select_statement(JPQL.java:107)
at org.apache.openjpa.kernel.jpql.JPQL.parseQuery(JPQL.java:63)
at org.apache.openjpa.kernel.jpql.JPQLExpressionBuilder$ParsedJPQL.parse(JPQLExpressionBuilder.java:2401)
at org.apache.openjpa.kernel.jpql.JPQLExpressionBuilder$ParsedJPQL.<init>(JPQLExpressionBuilder.java:2388)
at org.apache.openjpa.kernel.jpql.JPQLParser.parse(JPQLParser.java:49)
... 96 moreEncountered "max ( c . redistributionId ) <EOF>" at character 79, but expected: ["(", ")", "*",
Any help will be appreciated.
Edit:
I restructured the query a little bit:
#Query("select c from ConsignmentOrder c where c.redistributionId = (select max(co.redistributionId) from (select co ConsignmentOrder co where co.redistributionId like ?1) as tmp )")
Now the exception is similar:
Caused by: <openjpa-2.2.2-r422266:1468616 nonfatal user error> org.apache.openjpa.persistence.ArgumentException: Encountered "c . redistributionId = ( select max ( co . redistributionId ) from (" at character 40, but expected: ["(", ")", "*", "+", ",", "-", ".", "/", ":", "<", "<=", "<>", "=", ">", ">=", "?", "ABS", "ALL", "AND", "ANY", "AS", "ASC", "AVG", "BETWEEN", "BOTH", "BY", "CASE", "CLASS", "COALESCE", "CONCAT", "COUNT", "CURRENT_DATE", "CURRENT_TIME", "CURRENT_TIMESTAMP", "DELETE", "DESC", "DISTINCT", "ELSE", "EMPTY", "END", "ENTRY", "ESCAPE", "EXISTS", "FETCH", "FROM", "GROUP", "HAVING", "IN", "INDEX", "INNER", "IS", "JOIN", "KEY", "LEADING", "LEFT", "LENGTH", "LIKE", "LOCATE", "LOWER", "MAX", "MEMBER", "MIN", "MOD", "NEW", "NOT", "NULL", "NULLIF", "OBJECT", "OF", "OR", "ORDER", "OUTER", "SELECT", "SET", "SIZE", "SOME", "SQRT", "SUBSTRING", "SUM", "THEN", "TRAILING", "TRIM", "TYPE", "UPDATE", "UPPER", "VALUE", "WHEN", "WHERE", <BOOLEAN_LITERAL>, <DATE_LITERAL>, <DECIMAL_LITERAL>, <IDENTIFIER>, <INTEGER_LITERAL>, <STRING_LITERAL2>, <STRING_LITERAL>, <TIMESTAMP_LITERAL>, <TIME_LITERAL>].
Without the misleading EOF.
Is it important to convert it to JPQL? You can easily run native SQLs in JPA
With createNativeQuery() method that exists in your Entity Manager you can create executable Queries with SQL syntax