Laravel Query Builder get sum with clause and results - json

I have this table :
created_at | name | source
this is my builder query:
$data = Contact::select(
DB::raw('DATE_FORMAT(created_at, "%m-%d-%Y")'),
DB::raw('SUM(source = "IG") as instagram'),
DB::raw('SUM(source = "FBS") as facebook')
)
->orderBy('data', 'DESC')
->groupBy(DB::raw("DATE_FORMAT(created_at, '%d-%m-%Y')"))
->get();
return response()->json(['contacts' => $data], 200);
I get a correct Json with sum, groupBy and ordered by data:
{
"contacts": [
{
"created_at": "01-17-2022",
"www": "0",
"FBS": "42"
},
{
"created_at": "01-16-2022",
"www": "3",
"FBS": "68"
}
]
}
But in this way I would be forced to make another json to explode all the data, instead I'd like to return results with details for each rows, like this:
{
"contacts": [
{
"created_at": "01-17-2022",
"www": [
{
counts : 0
}
],
"FBS": [
{
counts : 42
},
{
name : "John Doe",
source: "FBS"
},
...
]
},
{
"created_at": "01-16-2022",
"www": [
{
counts : 3
},
{
name : "John Doe",
source: "FBS"
},
{
name : "Jane Doe",
source: "FBS"
},
...
],
"FBS": [
{
counts : 68,
},
{
name : "John Doe",
source: "FBS"
},
{
name : "Jane Doe",
source: "FBS"
},
...
]
}
]
}
Should I completely change my query? What is the best way for get it?

Related

Sequelize SUM function return as whole rather than grouping

I was trying to SUM an Review Score on association (relational) table between Post & UserReviews using MySQL, I had SQL like this.
sql
SELECT SUM(`score`) / COUNT(`scoreableId`) FROM `UserReviews` GROUP BY `scoreableId`;
I want convert it into sequelize query builder, I had try like this.
sequelize
db.Post.findAndCountAll({
attributes: [
'id',
],
include: [{
model: db.PostAttribute,
as: 'attributes',
attributes: [
'title',
'description',
],
order: [
['updatedAt', sort.includes('date') ? 'ASC' : 'DESC'],],
include: [{
model: db.PostPhoto,
attributes: ['url'],
as: 'photos'
}, {
model: db.UserReview,
as: 'reviews',
attributes: [
'scoreableId',
[db.sequelize.fn('SUM', db.sequelize.col('rating')), 'rating']
],
group: ['scoreableId']
}]
}],
})
But resulting output from those query was whole SUM of RATING (all users rating as single) inside single row.
{
"data": [
{
"id": 1,
"attributes": {
"title": "Tasty Frozen Salad",
"description": "description goes here",
"photos": [
{
"url": "https://loremflickr.com/500/650/fashion?47655"
}
],
"reviews": [
{
"scoreableId": 1,
"rating": "392"
}
]
}
}
]
}
I was expecting it result were rather like this.
{
"data": [
{
"id": 1,
"attributes": {
"title": "Tasty Frozen Salad",
"description": "description goes here",
"photos": [
{
"url": "https://loremflickr.com/500/650/fashion?47655"
}
],
"reviews": [
{
"scoreableId": 1,
"rating": "50"
}
]
}
},
{
"id": 2,
"attributes": {
"title": "Another Tasty Frozen Salad",
"description": "description goes here",
"photos": [
{
"url": "https://loremflickr.com/500/650/fashion?47655"
}
],
"reviews": [
{
"scoreableId": 2,
"rating": "40"
}
]
}
}
]
}

Sequelize - Nested Query Condition to Top Level

I want to query a database of promotions that don't have phone number in the blacklist of promotion.
my project use sequelize.org for manage database.
Mock Database http://sqlfiddle.com/#!9/438587/1
Association
Promotion Model
this.belongsToMany(models.CustomerList, {
through: models.BlacklistPromotion,
as: 'Blacklist',
});
this.belongsToMany(models.CustomerList, {
through: models.WhitelistPromotion,
as: 'Whitelist',
});
CustomerList Model
this.belongsToMany(models.CustomerPhoneNumber, {through: models.CustomerListPhoneNumber,});
CustomerPhoneNumber Model
this.belongsToMany(models.CustomerList, { through: models.CustomerListPhoneNumber});
Code sequelize findAndCountAll Promotion
await Promotion.findAndCountAll({
where : {}, <-------- i want query here or somewhere.
include :[
{
model: CustomerList,
as: 'Blacklist',
include :[{
model: CustomerPhoneNumber,
as: 'CustomerPhoneNumbers',
attributes: ['phone_number'],
through: {
attributes: [],
}
}],
through: {
attributes: [],
},
},
{
model: CustomerList,
as: 'Whitelist',
include :[{
model: CustomerPhoneNumber,
as: 'CustomerPhoneNumbers',
attributes: ['phone_number'],
through: {
attributes: [],
}
}],
through: {
attributes: [],
},
}
]
})
when run query
{
"Promotion": [
{
"id": 1,
"title": "Promotion1"
},
{
"id": 2,
"title": "Promotion2"
},
{
"id": 3,
"title": "Promotion3",
"Blacklist": [
{
"id": 1,
"name": "Group5",
"CustomerPhoneNumbers": [
{
"id": 5,
"phone_number": "0877777777"
},
{
"id": 6,
"phone_number": "0888888888"
},
{
"id": 7,
"phone_number": "0899999999"
}
]
}
]
}
]
}
try query
where : {
'$Blacklist.CustomerPhoneNumbers.phone_number$': {
[Op.ne]: '0877777777',
}
}
// output
{
"Promotion": [
{
"id": 1,
"title": "Promotion1"
},
{
"id": 2,
"title": "Promotion2"
},
{
"id": 3,
"title": "Promotion3",
"Blacklist": [
{
"id": 1,
"name": "Group5",
"CustomerPhoneNumbers": [
{
"id": 6,
"phone_number": "0888888888"
},
{
"id": 7,
"phone_number": "0899999999"
}
]
}
]
}
]
}
but i want to remove promotion3
What do you expect to happen?
Find solution query a database of promotions that don't have phone number in the blacklist of promotion.
I want to get:
{
"Promotion": [
{
"id": 1,
"title": "Promotion1"
},
{
"id": 2,
"title": "Promotion2"
},
]
}

advanced JSON query language

I've explored couple of existing JSON query language such JMESPath, JsonPath and JSONiq. Unfortunately, none of them seem to be able to support my use case in a generic way.
Basically, I'm receiving different type of responses from different web services. I need to give the ability to the user to remap the response in a 2 dimensional array in other to leverage our visualization tool. Based on the new format, the user can decide how to display his data between existing widgets. Pretty much like a customisable dashboard entirely managed on the UI.
Anyway my input looks like:
{
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
}
expected output:
[
{
"name": "medium",
"count": 10,
"category": "1"
},
{
"name": "high",
"count": 20,
"category": "1"
},
{
"name": "medium",
"count": 30,
"category": "2"
},
{
"name": "high",
"count": 40,
"category": "2"
}
]
The closer I went is with JMESPath but my query isn't dynamic at all. The user needs to be aware of possible category of grouping.
The query looks like: [ category_1[].{name: name, count: count, category: '1'}, category_2[].{name: name, count: count, category: '2'} ] | []
In other words, I need an enough powerful JSON query language to perform this JavaScript code:
const output = flatMap(input, (value, key) => {
return value.map(x => {
return { ...x, category: key };
});
});
Any thoughts?
This is indeed not currently possible in JMESPath (0.15.x). There are other spec compliant JMESPath packages that (with a bit of extra effort) will do what you require. Using NPM package #metrichor/jmespath (a typescript implementation) you could extend it with the functions you require as follows:
import {
registerFunction,
search,
TYPE_ARRAY,
TYPE_OBJECT
} from '#metrichor/jmespath';
registerFunction(
'flatMapValues',
([inputObject]) => {
return Object.entries(inputObject).reduce((flattened, entry) => {
const [key, value]: [string, any] = entry;
if (Array.isArray(value)) {
return [...flattened, ...value.map(v => [key, v])];
}
return [...flattened, [key, value]];
}, [] as any[]);
},
[{ types: [TYPE_OBJECT, TYPE_ARRAY] }],
);
With these extended functions a JMESPath expression would now look like this to remap the key into every value:
search("flatMapValues(#)[*].merge([1], {category: [0]})", {
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
});
// OUTPUTS:
[
{
category: 'category_1',
count: 10,
name: 'medium',
},
{
category: 'category_1',
count: 20,
name: 'high',
},
{
category: 'category_2',
count: 30,
name: 'medium',
},
{
category: 'category_2',
count: 40,
name: 'high',
},
]
That said you could just register the function you wrote above and use it
Finally, managed a way with JSONiq using Zorba implementation. Definitively the way to go if you need powerful JSON queries. Apparently this has been integrated in Apache Spark with Rumble
Anyway, here's my solution:
jsoniq version "1.0";
let $categories :=
{
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
}
for $key in keys($categories), $row in flatten($categories.$key)
return {"count": $row.count, "name": $row.name, "category": $key}
output:
{ "count" : 10, "name" : "medium", "category" : "category_1" }{ "count" : 20, "name" : "high", "category" : "category_1" }{ "count" : 30, "name" : "medium", "category" : "category_2" }{ "count" : 40, "name" : "high", "category" : "category_2" }
You can try Zorba here.
This is an alternative possibility in JSONiq that does not explicitly list the keys in each row, with the merge constructor {| |}:
jsoniq version "1.0";
let $categories :=
{
"category_1": [
{
"name": "medium",
"count": 10
},
{
"name": "high",
"count": 20
}
],
"category_2": [
{
"name": "medium",
"count": 30
},
{
"name": "high",
"count": 40
}
]
}
for $key in keys($categories),
$row in members($categories.$key)
return {|
$row,
{ "category": $key }
|}
For the sake of completeness, this is the reverse query that would turn the output back into the original input (which uses a group by clause):
jsoniq version "1.0";
let $output :=
(
{ "count" : 10, "name" : "medium", "category" : "category_1" },
{ "count" : 20, "name" : "high", "category" : "category_1" },
{ "count" : 30, "name" : "medium", "category" : "category_2" },
{ "count" : 40, "name" : "high", "category" : "category_2" }
)
return
{|
for $row in $output
group by $category := $row.category
return { $category : [ $row ] }
|}
This is simple with ~Q (disclaimer: I'm the developer).
{
"results:{}:[]": [{
"{}:":".",
"category":"$key"
}]
}
Output:
{
"results": [
{
"name": "medium",
"count": 10,
"category": "category_1"
},
{
"name": "high",
"count": 20,
"category": "category_1"
},
{
"name": "medium",
"count": 30,
"category": "category_2"
},
{
"name": "high",
"count": 40,
"category": "category_2"
}
]
}
Edit: some more info to explain the syntax:
"results:{}:[]"
The :{} part means "iterate over all keys in the object", :[] means "iterate over all array elements".
"{}:":"."
This copies each field in the current object to the output.
"category":"$key"
Add a field called "category", with the current traversed key as value.
If we wanted to get the numbers (i.e. 1,2,... instead of category_1, category_2, etc), we can use substr:
"category": "$key substr(9)"
You actually don't need any additional libs for that. Here is a small function which does the trick. You only need to split the key.
const transform = (obj) => {
const ret = [];
for (let key in obj) {
const tmp = key.split('_');
for (let item of obj[key]) {
ret.push({
...item,
[tmp[0]]: tmp[1],
});
}
}
return ret;
};
const result = transform(obj);

Re-arrange JSON file (using adjacency matrix)

I have a json file that looks like this:
[
{
"id": 1,
"country": "Greece",
"names": [
"Alex",
"Betty",
"Catherine",
"Dave",
"Edward",
"Frank",
"George",
"Helen",
"Irene"
]
},
{
"id": 2,
"country": "US",
"names": [
"John",
"Alex",
"Edward",
"Kate",
"Robert",
"Irene",
"Tim",
"Sam"
]
},
{
"id": 3,
"country": "France",
"names": [
"Helen",
"Kate",
"Louise",
"Tim",
"Catherine",
"Arthur",
"Frank",
"Natalie",
"Dave"
]
},
{
"id": 4,
"country": "India",
"names": [
"Ritesh",
"Alex",
"Betty",
"Robert"
]
},
{
"id": 5,
"country": "India",
"names": [
"Nafeez",
"Tom",
"Natalie",
"Gunar",
"Louise",
"Arthur"
]
}
]
I want it to be "name centered" and look like this:
{
"groups": [
{
"gr_id":1
"name":"Alex",
"country":"Greece"
},
.........
{
"gr_id":1
"name":"Irene",
"country":"Greece"
},
{
"gr_id":2
"name":"John",
"country":"US"
..........
{
"gr_id":2
"name":"Sam",
"country":"US"
},
{
"gr_id":3
"name":"Helen",
"country":"France"
},
.........
{
"gr_id":3
"name":"Dave",
"country":"France"
},
{
"gr_id":4
"name":"Ritesh",
"country":"India"
},
........
{
"gr_id":4
"name":"Robert",
"country":"India"
},
{
"gr_id":5
"name":"Nafeez",
"country":"India"
},
...........
{
"gr_id":5
"name":"Arthur",
"country":"India"
}
],
"links": [
{
"source":"Alex"
"target":"Irene",
"count":1
"country":"Greece"
},
...
{
"source":"Alex"
"target":"Arthur",
"count":0
"country":"India"
},
...
]
}
For count in Links I have an adjacency matrix for each country/name (csv format) like this :screenshot of csv file (ad. matrix for India)
This json is just an example. I have much bigger file (I need it for D3 graph visualization)
Reduce() and map() work perfectly for this. This basically takes each item and then maps over the names, appending the results of map() to an array:
let obj = {}
obj.groups = json.reduce(
(acc, curr) => acc.concat(curr.names.map(
item => ({gr_id: curr.id, country: curr.country, name: item})
)), [])
console.log(obj)
// { groups:
// [ { gr_id: 1, country: 'Greece', name: 'Alex' },
// { gr_id: 1, country: 'Greece', name: 'Betty' },
// ...etc
// ]
// }

How to iterate on json fields and insert new values using json4s?

I have a simple json file:
val myJson = {
"field1": [
{
"name": "john",
"lname": "knight"
},
{
"name": "jack",
"lname": "samuel"
},
{
"name": "elinor",
"lname": "cooper"
}
],
"field2": [
{
...
},
{
...
},
{
...
}
],
"field3": [
{
...
},
{
...
},
{
...
}
]
}
and what i want is to be able to iterate on "field1" and for each name to call a method that returns some value and insert this value to the json under "fiel1".
// this returns a list of strings
val kids = getKids("john")
// this is will me the returned value
kids = List("amy", "tom")
now I want to insert it:
{
"field1": [
{
"name": "john",
"lname": "knight"
"kids": ["amy", "tom"]
},
{
"name": "jack",
"lname": "samuel"
"kids": ["edi", "keren"]
},
{
"name": "elinor",
"lname": "cooper"
"kids": ["lilly", "mag"]
}
]
...
but I want to iterate on all the names and do this for each one...how can I accomplish this with json4s?
so lets say i have the parsed json:
val myParsedJson = JsonMethods.parse(myJson)
how do I go from here?
thanks!