Server side pagination in Graphql Scala - mysql

I need to do server side pagination in graphql in scala. I have seven reports and have used one graphql query with different operations to get the data. Now I need to add server side pagination to it and I am not able to do that. Any help would be appreciated. Below are my code:
Schema:
val PanelNominationStatusReportDataType = ObjectType(
"PanelNominationStatus",
"Dashboard Reports",
fields[Unit, PanelRewardSatusDetail](
Field("awardName", OptionType(StringType), resolve = _.value.awardName),
Field("nominatorEmail", OptionType(StringType), resolve = _.value.nominatorEmail),
Field("nomineeEmail", OptionType(StringType), resolve = _.value.nomineeEmail),
Field("nomineeLocation", OptionType(StringType), resolve = _.value.nomineeLocation),
Field("approverEmail", OptionType(StringType), resolve = _.value.approverEmail),
Field("citation", OptionType(StringType), resolve = _.value.citation),
Field("businessJustification", OptionType(StringType), resolve = _.value.businessJustification),
Field("rating", OptionType(IntType), resolve = _.value.rating),
Field("votingStatus", OptionType(StringType), resolve = _.value.votingStatus),
Field("nominatedDate", OptionType(StringType), resolve = _.value.nominatedDate),
Field("approvedDate", OptionType(StringType), resolve = _.value.approvedDate)
)
)
Graphql Query:
Field(
"PanelNominationStatus",
ListType(PanelNominationStatusReportDataType),
description = Some("Returns the Employee Reward Report Data"),
arguments = companyIdType :: rewardNamesType :: startDateType :: endDateType :: Nil,
resolve = { c =>
c.ctx.getPanelStatusReport(c.arg(companyIdType), c.arg(startDateType), c.arg(endDateType), c.arg(rewardNamesType))
})
DataRepo:
def getPanelStatusReport(companyId: Int, startDate: Long, endDate: Long, rewardNames: Seq[String]): List[PanelRewardSatusDetail] = {
val redemptionReport = Await.result(reportsModel.findPanelRewardStatusDetailsReport(companyId, rewardNames, startDate, endDate), 20.seconds).toList
redemptionReport
}
And at last the model:
def findPanelRewardStatusDetailsReport(companyId: Int, rewardNames: Seq[String], startDate: Long, endDate: Long): Future[Seq[PanelRewardSatusDetail]] = {
val df = new SimpleDateFormat("dd-MM-yyyy")
val start = stringToDateTime(df.format(startDate * 1000L), None).toDateTime()
val end = stringToDateTime(df.format(endDate * 1000L), None).toDateTime()
val rewardFilter = if(rewardNames.nonEmpty) "AND vrrc.reward_name IN (" + rewardNames.map(a => "'" + a + "'").mkString(",") + ")" else ""
implicit val getOrdersResult = GetResult(r => PanelRewardSatusDetail(r.<<, r.<<, r.<<, r.<<, r.<<, r.<<, r.<<, r.<<, r.<<, r.<<, r.<<))
val q = sql"""select vrrc.reward_name, (select login from users where id=vrur.sender_id), (select login from users where id=vrur.receiver_ids), (select city_name from cities where id=(select city_id from users where id=vrur.receiver_ids)), (select login from users where id=approver_user_id), vrur.comment, vrur.business_justification, vrpa.rating, CASE WHEN vrpa.is_approved = 1 THEN 'VOTED' ELSE 'NOT VOTED' END, date(vrpa.created_at), date(vrpa.approved_at) from vr_panel_approval vrpa inner join vr_user_reward vrur on vrur.id=vrpa.user_reward_id inner join vr_reward_config vrrc on vrrc.id=vrur.reward_config_id where vrrc.company_id = $companyId and date(vrpa.created_at) between date($start) and date($end) #$rewardFilter"""
db.run(q.as[PanelRewardSatusDetail])
}
My request query:
{
"query": "query PanelNominationStatus($startDate: Long!, $endDate: Long!, $companyId: Int!, $rewardNames: [String!]!) { PanelNominationStatus( startDate: $startDate, endDate: $endDate, companyId: $companyId, rewardNames: $rewardNames ) { awardName nominatorEmail nomineeEmail nomineeLocation approverEmail citation businessJustification rating votingStatus nominatedDate approvedDate }}",
"operationName": "PanelNominationStatus",
"variables": {
"startDate": 1285891200,
"endDate": 1576108800,
"companyId": 355,
"rewardNames": ["PANEL AWARD FEBRUARY", "PANEL AWARD MARCH"]
}
}
I can make it possible using limit in the query and accepting it in the query but I need to send the total count of rows in the response as well. But I am not able to figure out where to make change in schema.

You just need to define another GraphQL type for your result which also contains the count,
val PanelNominationStatusResultType = ObjectType(
"PanelNominationStatusResult",
"Dashboard Reports",
fields[Unit, List[PanelRewardSatusDetail]](
Field(
"count",
IntType,
description = Some("count of nomination-status-reports in this response"),
resolve = _.value.length
),
Field(
"panelNominationStatusList",
ListType(PanelNominationStatusReportDataType),
description = Some("list of nomination-status-reports"),
resolve = _.value
)
)
)
Now, as for your query,
Field(
"panelNominationStatusResult",
PanelNominationStatusResultType,
description = Some("Returns the details of ............"),
arguments = companyIdType :: rewardNamesType :: startDateType :: endDateType :: Nil,
resolve = { c =>
c.ctx.getPanelStatusReport(c.arg(companyIdType), c.arg(startDateType), c.arg(endDateType), c.arg(rewardNamesType))
}
)
If you want the totalCount, then first of all you need to change your getPanelStatusReport method to also return the totalCount
def findPanelRewardStatusDetailsReport(
companyId: Int,
rewardNames: Seq[String],
startDate: Long,
endDate: Long
): Future[(Int, Seq[PanelRewardSatusDetail])] = ???
// updated result type
val PanelNominationStatusResultType = ObjectType(
"PanelNominationStatusResult",
"Dashboard Reports",
fields[Unit, (Int, List[PanelRewardSatusDetail])](
Field(
"totalCount",
IntType,
description = Some("total count of nomination-status-reports"),
resolve = _.value._1
),
Field(
"count",
IntType,
description = Some("count of nomination-status-reports in this response"),
resolve = _.value._2.length
),
Field(
"panelNominationStatusList",
ListType(PanelNominationStatusReportDataType),
description = Some("list of nomination-status-reports"),
resolve = _.value._2
)
)
)

Related

JSON to Excel PowerQuery import - how to get a row per nested field

I'm looking to use the Excel Power Query to import some json that looks like the following (but much bigger, more fields etc.):
example-records.json
{
"records": {
"record_id_1": {
"file_no": "5792C",
"loads": {
"load_id_1": {
"docket_no": "3116115"
},
"load_id_2": {
"docket_no": "3116118"
},
"load_id_3": {
"docket_no": "3208776"
}
}
},
"record_id_2": {
"file_no": "5645C",
"loads": {
"load_id_4": {
"docket_no": "2000527155"
},
"load_id_5": {
"docket_no": "2000527156"
},
"load_id_6": {
"docket_no": "2000527146"
}
}
}
}
}
I want to get a table like the following at the load_id / docket level. A row per load_id
What I've tried
Clicking buttons in power query UI I get the following.
The problem is I can't include a file_no column and this doesn't work when there are lots of load ids.
let
Source = Json.Document(File.Contents("H:\Software\Site Apps\example-records.json")),
records = Source[records],
#"Converted to Table" = Record.ToTable(records),
#"Expanded Value" = Table.ExpandRecordColumn(#"Converted to Table", "Value", {"file_no", "loads"}, {"Value.file_no", "Value.loads"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Value",{"Value.file_no"}),
#"Expanded Value.loads" = Table.ExpandRecordColumn(#"Removed Columns", "Value.loads", {"load_id_1", "load_id_2", "load_id_3", "load_id_4", "load_id_5", "load_id_6"}, {"Value.loads.load_id_1", "Value.loads.load_id_2", "Value.loads.load_id_3", "Value.loads.load_id_4", "Value.loads.load_id_5", "Value.loads.load_id_6"}),
#"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Expanded Value.loads", {"Name"}, "Attribute", "Value"),
#"Expanded Value1" = Table.ExpandRecordColumn(#"Unpivoted Columns", "Value", {"docket_no"}, {"Value.docket_no"})
in
#"Expanded Value1"
You can use
let Source = JSON(Json.Document(File.Contents("c:\temp\example.json"))),
#"Removed Other Columns" = Table.SelectColumns(Source,{"Name.1", "Name.3", "Value"}),
#"Added Custom" = Table.AddColumn(#"Removed Other Columns", "Custom", each if [Name.3]=null then [Value] else null),
#"Filled Down" = Table.FillDown(#"Added Custom",{"Custom"}),
#"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Name.3] <> null))
in #"Filtered Rows"
based on this function I named JSON which comes from Imke https://www.thebiccountant.com/2018/06/17/automatically-expand-all-fields-from-a-json-document-in-power-bi-and-power-query/ which is reproduced below
let
func = (JSON) =>
let
Source = JSON,
ParseJSON = try Json.Document(Source) otherwise Source,
TransformForTable =
if Value.Is(ParseJSON, type record) then
Record.ToTable(ParseJSON)
else
#table(
{"Name", "Value"},
List.Zip({List.Repeat({0}, List.Count(ParseJSON)), ParseJSON})
),
AddSort = Table.Buffer(Table.AddColumn(TransformForTable, "Sort", each 0)),
LG = List.Skip(
List.Generate(
() => [Next = AddSort, Counter = 1, AddIndex = #table({"Sort"}, {{""}})],
each [AddIndex]{0}[Sort] <> "End",
each [
AddIndex = Table.AddIndexColumn([Next], "Index", 0, 1),
MergeSort = Table.CombineColumns(
Table.TransformColumnTypes(
AddIndex,
{{"Sort", type text}, {"Index", type text}},
"en-GB"
),
{"Sort", "Index"},
Combiner.CombineTextByDelimiter(".", QuoteStyle.None),
"Sort"
),
PJson = Table.TransformColumns(
MergeSort,
{{"Value", each try Json.Document(_) otherwise _}}
),
AddType = Table.AddColumn(
PJson,
"Type",
each
if Value.Is([Value], type record) then
"Record"
else if Value.Is([Value], type list) then
"List"
else if Value.Is([Value], type table) then
"Table"
else
"other"
),
AddStatus = Table.AddColumn(
AddType,
"Status",
each if [Type] = "other" then "Finished" else "Unfinished"
),
Finished = Table.SelectRows(AddStatus, each ([Status] = "Finished")),
Unfinished = Table.SelectRows(AddStatus, each ([Status] = "Unfinished")),
AddNext = Table.AddColumn(
Unfinished,
"Next",
each if [Type] = "Record" then {[Value]} else [Value]
),
RemoveCols = Table.RemoveColumns(AddNext, {"Value", "Type", "Status"}),
ExpandNext = Table.ExpandListColumn(RemoveCols, "Next"),
AddIndex2 = Table.AddIndexColumn(ExpandNext, "Index", 0, 1),
MergeSort2 = Table.CombineColumns(
Table.TransformColumnTypes(
AddIndex2,
{{"Sort", type text}, {"Index", type text}},
"en-GB"
),
{"Sort", "Index"},
Combiner.CombineTextByDelimiter(".", QuoteStyle.None),
"Sort"
),
TransformRecord = Table.TransformColumns(
MergeSort2,
{
{
"Next",
each try
Record.ToTable(_)
otherwise
try
if Value.Is(Text.From(_), type text) then
#table({"Value"}, {{_}})
else
_
otherwise
_
}
}
),
FilterOutNulls = Table.SelectRows(TransformRecord, each [Next] <> null),
Next =
if Table.IsEmpty(FilterOutNulls) then
#table({"Sort"}, {{"End"}})
else if Value.Is(FilterOutNulls[Next]{0}, type table) = true then
Table.ExpandTableColumn(
FilterOutNulls,
"Next",
{"Name", "Value"},
{"Name." & Text.From([Counter]), "Value"}
)
else
Table.RenameColumns(FilterOutNulls, {{"Next", "Value"}}),
Counter = [Counter] + 1
],
each Table.AddColumn([Finished], "Level", (x) => _[Counter] - 2)
)
),
Check = LG{2},
Combine = Table.Combine(LG),
Clean = Table.RemoveColumns(Combine, {"Status", "Type"}),
Trim = Table.TransformColumns(Clean, {{"Sort", each Text.Trim(_, "."), type text}}),
// Dynamic Padding for the sort-column so that it sorts by number in text strings
SelectSort = Table.SelectColumns(Trim, {"Sort"}),
SplitSort = Table.AddColumn(
SelectSort,
"Custom",
each List.Transform(try Text.Split([Sort], ".") otherwise {}, Number.From)
),
ToTable = Table.AddColumn(
SplitSort,
"Splitted",
each Table.AddIndexColumn(Table.FromColumns({[Custom]}), "Pos", 1, 1)
),
ExpandTable = Table.ExpandTableColumn(ToTable, "Splitted", {"Column1", "Pos"}),
GroupPos = Table.Group(
ExpandTable,
{"Pos"},
{{"All", each _, type table}, {"Max", each List.Max([Column1]), type text}}
),
Digits = Table.AddColumn(GroupPos, "Digits", each Text.Length(Text.From([Max]))),
FilteredDigits = List.Buffer(Table.SelectRows(Digits, each ([Digits] <> null))[Digits]),
SortNew = Table.AddColumn(
Trim,
"SortBy",
each Text.Combine(
List.Transform(
List.Zip({Text.Split([Sort], "."), List.Positions(Text.Split([Sort], "."))}),
each Text.PadStart(_{0}, FilteredDigits{_{1}}, "0")
),
"."
)
),
FilterNotNull = Table.SelectRows(SortNew, each ([Value] <> null)),
Reorder = Table.ReorderColumns(
FilterNotNull,
{"Value", "Level", "Sort", "SortBy"}
& List.Difference(
Table.ColumnNames(FilterNotNull),
{"Value", "Level", "Sort", "SortBy"}
)
),
Dots = Table.AddColumn(
#"Reorder",
"Dots",
each List.Select(Table.ColumnNames(#"Reorder"), (l) => Text.StartsWith(l, "Name"))
),
// This sort is just to view in the query editor. When loaded to the data model it will not be kept. Use "Sort by column" in the data model instead.
Sort = Table.Sort(Dots, {{"SortBy", Order.Ascending}})
in
Sort,
documentation = [
Documentation.Name = " Table.JsonExpandAll ",
Documentation.Description
= " Dynamically expands the <Json> Record and returns values in one column and additional columns to navigate. ",
Documentation.LongDescription
= " Dynamically expands the <Json> Record and returns values in one column and additional columns to navigate. Input can be JSON in binary format or the already parsed JSON. ",
Documentation.Category = " Table ",
Documentation.Version = " 1.2: Added column [Dots] (22/02/2019)",
Documentation.Author = " Imke Feldmann: www.TheBIccountant.com . ",
Documentation.Examples = {[Description = " ", Code = " ", Result = " "]}
]
in
Value.ReplaceType(func, Value.ReplaceMetadata(Value.Type(func), documentation))
Managed to use an added custom column, the action that enables the expansion to one load id per row.
#"Added Custom" = Table.AddColumn(#"Expanded Value", "Custom", each Record.ToTable([Value.loads]))
let
Source = Json.Document(File.Contents("H:\Software\Site Apps\example-records.json")),
records = Source[records],
#"Converted to Table" = Record.ToTable(records),
#"Expanded Value" = Table.ExpandRecordColumn(#"Converted to Table", "Value", {"file_no", "loads"}, {"Value.file_no", "Value.loads"}),
#"Added Custom" = Table.AddColumn(#"Expanded Value", "Custom", each Record.ToTable([Value.loads])),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Value.loads"}),
#"Expanded Custom" = Table.ExpandTableColumn(#"Removed Columns", "Custom", {"Name", "Value"}, {"Custom.Name", "Custom.Value"}),
#"Expanded Custom.Value" = Table.ExpandRecordColumn(#"Expanded Custom", "Custom.Value", {"docket_no"}, {"Custom.Value.docket_no"}),
#"Renamed Columns" = Table.RenameColumns(#"Expanded Custom.Value",{{"Name", "record_id"}, {"Value.file_no", "file_no"}, {"Custom.Name", "load_id"}, {"Custom.Value.docket_no", "docket_no"}})
in
#"Renamed Columns"

Laravel How Make DB raw can inside funtion

I have problem on laravel. I want to showing percentage on with eloquent, but return null value
This Is My Controller
$group_categories = Proyek::with(['modul' => function($query){
$query->select(DB::raw('((select count(*) from modul where status = 0 and deleted_at IS NULL) /
(select count(*) from modul where deleted_at IS NULL)) *
100 as count' ));
}])->get();
This Is Json return
{
id: "10",
proyek: "JuZka",
created_at: "2018-08-12 01:54:04",
updated_at: "2018-09-23 05:49:13",
modul: [ ]
},
You can use code like this:
$all_models = (new Proyek)->count();
$models = (new Proyek)->where('status', 0)->count();
$percentage = $models / $all_models * 100;
$group_categories = Proyek::wheere('modul', $percentage);

Scala : How to do GroupBy sum for String values?

I have RDD[Row] :
|---itemId----|----Country-------|---Type----------|
| 11 | US | Movie |
| 11 | US | TV |
| 101 | France | Movie |
How to do GroupBy itemId so that I can save the result as List of json where each row is separate json object(each row in RDD) :
{"itemId" : 11,
"Country": {"US" :2 },"Type": {"Movie" :1 , "TV" : 1} },
{"itemId" : 101,
"Country": {"France" :1 },"Type": {"Movie" :1} }
RDD :
I tried :
import com.mapping.data.model.MappingUtils
import com.mapping.data.model.CountryInfo
val mappingPath = "s3://.../"
val input = sc.textFile(mappingPath)
The input is list of jsons where each line is json which I am mapping to the POJO class CountryInfo using MappingUtils which takes care of JSON parsing and conversion:
val MappingsList = input.map(x=> {
val countryInfo = MappingUtils.getCountryInfoString(x);
(countryInfo.getItemId(), countryInfo)
}).collectAsMap
MappingsList: scala.collection.Map[String,com.mapping.data.model.CountryInfo]
def showCountryInfo(x: Option[CountryInfo]) = x match {
case Some(s) => s
}
val events = sqlContext.sql( "select itemId EventList")
val itemList = events.map(row => {
val itemId = row.getAs[String](1);
val çountryInfo = showTitleInfo(MappingsList.get(itemId));
val country = if (countryInfo.getCountry() == 'unknown)' "US" else countryInfo.getCountry()
val type = countryInfo.getType()
Row(itemId, country, type)
})
Can some one let me know how can I achieve this ?
Thank You!
I can't afford the extra time to complete this, but can give you a start.
The idea is that you aggregate the RDD[Row] down into a single Map that represents your JSON structure. Aggregation is a fold that requires two function parameters:
seqOp How to fold a collection of elements into the target type
combOp How to merge two of the target types.
The tricky part comes in combOp while merging, as you need to accumulate the counts of values seen in the seqOp. I have left this as an exercise, as I have a plane to catch! Hopefully someone else can fill in the gaps if you have trouble.
case class Row(id: Int, country: String, tpe: String)
def foo: Unit = {
val rows: RDD[Row] = ???
def seqOp(acc: Map[Int, (Map[String, Int], Map[String, Int])], r: Row) = {
acc.get(r.id) match {
case None => acc.updated(r.id, (Map(r.country, 1), Map(r.tpe, 1)))
case Some((countries, types)) =>
val countries_ = countries.updated(r.country, countries.getOrElse(r.country, 0) + 1)
val types_ = types.updated(r.tpe, types.getOrElse(r.tpe, 0) + 1)
acc.updated(r.id, (countries_, types_))
}
}
val z = Map.empty[Int, (Map[String, Int], Map[String, Int])]
def combOp(l: Map[Int, (Map[String, Int], Map[String, Int])], r: Map[Int, (Map[String, Int], Map[String, Int])]) = {
l.foldLeft(z) { case (acc, (id, (countries, types))) =>
r.get(id) match {
case None => acc.updated(id, (countries, types))
case Some(otherCountries, otherTypes) =>
// todo - continue by merging countries with otherCountries
// and types with otherTypes, then update acc
}
}
}
val summaryMap = rows.aggregate(z) { seqOp, combOp }

How to match a top level array in json with specs2

In specs2 you can match an array for elements like this:
val json = """{"products":[{"name":"shirt","price":10, "ids":["1", "2", "3"]},{"name":"shoe","price":5}]}"""
def aProductWith(name: Matcher[JsonType], price: Matcher[JsonType]): Matcher[String] =
/("name").andHave(name) and /("price").andHave(price)
def haveProducts(products: Matcher[String]*): Matcher[String] =
/("products").andHave(allOf(products:_*))
json must haveProducts(
aProductWith(name = "shirt", price = 10) and /("ids").andHave(exactly("1", "2", "3")),
aProductWith(name = "shoe", price = 5)
)
(Example taken from here: http://etorreborre.github.io/specs2/guide/SPECS2-3.0/org.specs2.guide.Matchers.html)
How do I do the same thing i.e. match the contents of products if products is a root element in the json? What should haveProducts look like?
val json = """[{"name":"shirt","price":10, "ids":["1", "2", "3"]},{"name":"shoe","price":5}]"""
You can replace /("products").andHave(allOf(products:_*)) with have(allOf(products:_*)) like this:
val json = """[{"name":"shirt","price":10, "ids":["1", "2", "3"]},{"name":"shoe","price":5}]"""
def aProductWith(name: Matcher[JsonType], price: Matcher[JsonType]): Matcher[String] =
/("name").andHave(name) and /("price").andHave(price)
def haveProducts(products: Matcher[String]*): Matcher[String] = have(allOf(products:_*))
json must haveProducts(
aProductWith(name = "shirt", price = 10) and /("ids").andHave(exactly("1", "2", "3")),
aProductWith(name = "shoe", price = 5)
)

Does linq to sql have a with ties option?

I'm trying to move the following query to Linq-to-sql, is it possible?
select * from (
Select top (#Percent) percent with ties *
from(
Select distinct
LoanNumber as LoanNo
From CHE
Left Join RecordingInfo as Rec
On CHE.LoanNumber = Rec.LoanNo
Where Channel = 'LINX'
and CHE.Doc in ('MTG','MOD')
and Rec.LoanNo is null
and LoanNumber >= '#LoanNo'
) A
order by LoanNo #Order
) B
order by LoanNo
I have not seen anyway to do with ties in linq.
I think something like this will work for you.
public static IQueryable<T> TopPercentWithTies<T, TKey>(this IOrderedQueryable<T> query, Expression<Func<T, TKey>> groupByExpression, double percent)
{
var groupedQuery = query.GroupBy(groupByExpression);
int numberToTake = groupedQuery.Count() * percent / 100;
return groupedQuery.Take(numberToTake).SelectMany(t => t);
}
I only tested it with IEnumerable, so I don't know for sure that it'll work properly with IQueryable. I also sorted the list before calling TopPercentWithTies().
Here's the code I used to test it.
int percent = 50;
var people = new []
{
new { Age = 99, Name = "Adam" },
new { Age = 99, Name = "Andrew" },
new { Age = 89, Name = "Bob" },
new { Age = 50, Name = "Cecil" },
new { Age = 50, Name = "Doug" },
new { Age = 50, Name = "Everett" },
new { Age = 35, Name = "Frank" },
new { Age = 25, Name = "Greg" },
new { Age = 15, Name = "Hank" }
};
var sortedPeople = people.AsQueryable().OrderByDescending(person => person.Age);
var results = sortedPeople.TopPercentWithTies(person => person.Age, percent);
foreach (var person in results)
Console.WriteLine(person);
Hope it helps or at least gets you in the right direction. You may want to tweak the logic for calculating numberToTake.