How do I create a cumulative sum graph in Palantir Foundry Workshop? - palantir-foundry

I have some objects that represent maintenance jobs, each with a time column like this:
Due On
2021-12-01
2022-06-17
2022-07-05
2022-07-05
2022-08-01
2023-09-02
How can I generate a cumulative graph in Palantir Foundry Workshop, plotting values like this?

You can do this using a Foundry Function. Create a TypeScript functions repository and use the following code (see comments inline for an explanation):
import { Function, Double, ThreeDimensionalAggregation, IRange, IRangeable, Timestamp, BucketKey, BucketValue } from "#foundry/functions-api";
// Replace MaintenanceJob with your object
// To make your object available, add it in the Settings > Ontology tab
import { ObjectSet, MaintenanceJob } from "#foundry/ontology-api";
export class MyFunctions {
// You will find this function in Workshop after it's published
// Replace MaintenanceJob with your object
#Function()
public async cumulativeJobsByMonth(jobs: ObjectSet<MaintenanceJob>): Promise<TwoDimensionalAggregation<IRange<Timestamp>, Double>> {
const bucketedJobs = await jobs
.groupBy(j => j.dueOn.byMonth())
.count();
const sortedBucketedJobs = sortBuckets(bucketedJobs);
const cumulativeSortedBucketedJobs = cumulativeSum2D(sortedBucketedJobs);
return cumulativeSortedBucketedJobs
}
}
/**
* Sort buckets of a 2D or 3D aggregation by the first axis in ascending order
*
* Example input 1:
* { buckets: [
* { key: { min: "2022-01-01", max: "2022-12-31" }, value: 456 },
* { key: { min: "2021-01-01", max: "2021-12-31" }, value: 123 },
* { key: { min: "2023-01-01", max: "2023-12-31" }, value: 789 },
* ]}
*
* Example output 1:
* { buckets: [
* { key: { min: "2021-01-01", max: "2021-12-31" }, value: 123 },
* { key: { min: "2022-01-01", max: "2022-12-31" }, value: 456 },
* { key: { min: "2023-01-01", max: "2023-12-31" }, value: 789 },
* ]}
*
* Example input 2:
* { buckets: [
* { key: 22, value: 456 },
* { key: 21, value: 123 },
* { key: 23, value: 789 },
* ]}
*
* Example output 2:
* { buckets: [
* { key: 21, value: 123 },
* { key: 22, value: 456 },
* { key: 23, value: 789 },
* ]}
*
* Example input 3:
* { buckets: [
* { key: { min: "2022-01-01", max: "2022-12-31" }, value: [{ key: "open", value: 789 }, { key: "closed", value: 910 }] },
* { key: { min: "2021-01-01", max: "2021-12-31" }, value: [{ key: "open", value: 123 }, { key: "closed", value: 456 }] },
* { key: { min: "2023-01-01", max: "2023-12-31" }, value: [{ key: "open", value: 314 }, { key: "closed", value: 42 }] },
* ]}
*
* Example output 3:
* { buckets: [
* { key: { min: "2021-01-01", max: "2021-12-31" }, value: [{ key: "open", value: 123 }, { key: "closed", value: 456 }] },
* { key: { min: "2022-01-01", max: "2022-12-31" }, value: [{ key: "open", value: 789 }, { key: "closed", value: 910 }] },
* { key: { min: "2023-01-01", max: "2023-12-31" }, value: [{ key: "open", value: 314 }, { key: "closed", value: 42 }] },
* ]}
*/
function sortBuckets<K1 extends BucketKey, K2 extends BucketKey, V extends BucketValue>(buckets: ThreeDimensionalAggregation<K1, K2, V>): ThreeDimensionalAggregation<K1, K2, V>;
function sortBuckets<K extends BucketKey, V extends BucketValue>(buckets: TwoDimensionalAggregation<K, V>): TwoDimensionalAggregation<K, V>;
function sortBuckets<K extends BucketKey, V>(buckets: { buckets: { key: K, value: V}[] }): { buckets: { key: K, value: V}[] } {
return {
// See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
buckets: buckets.buckets.sort(({ key: k1 }, { key: k2 }) => {
if (typeof k1 !== typeof k2) throw new Error("Inconsistent bucket key types")
// If not objects, these must be either numbers or booleans which can be compared like this
if (typeof k1 !== "object" || typeof k2 !== "object") return Number(k1) - Number(k2);
// If a bucket doesn't have a minimum, it suggests that it is the global unbounded minimum bucket, so must be lower
if (!(k1 as IRange<IRangeable>).min) return -1;
if (!(k2 as IRange<IRangeable>).min) return 1;
// Otherwise, compare both buckets' minimums
return (k1 as IRange<IRangeable>).min!.valueOf() - (k2 as IRange<IRangeable>).min!.valueOf();
}),
};
}
/**
* Calculates a cumulative sum for a TwoDimensionalAggregation over numbers, along the first axis
* The order of the buckets into the function matters for how the values are aggregated
*
* Example input 1:
* { buckets: [
* { key: { min: "2021-01-01", max: "2021-12-31" }, value: 123 },
* { key: { min: "2022-01-01", max: "2022-12-31" }, value: 789 },
* { key: { min: "2023-01-01", max: "2023-12-31" }, value: 314 },
* ]}
*
* Example output 1:
* { buckets: [
* { key: { min: "2021-01-01", max: "2021-12-31" }, value: 123 },
* { key: { min: "2022-01-01", max: "2022-12-31" }, value: 912 },
* { key: { min: "2023-01-01", max: "2023-12-31" }, value: 1226 },
* ]}
*
* Example input 2:
* { buckets: [
* { key: 23, value: 314 },
* { key: 22, value: 789 },
* { key: 21, value: 123 },
* ]}
*
* Example output 2:
* { buckets: [
* { key: 23, value: 314 },
* { key: 22, value: 1103 },
* { key: 21, value: 1226 },
* ]}
*/
const cumulativeSum2D = <K extends BucketKey>(buckets: TwoDimensionalAggregation<K, number>): TwoDimensionalAggregation<K, number> => {
// This holds the running total
let cumulativeValue = 0;
return {
buckets: buckets.buckets.map(b => ({
key: b.key,
// This line adds the current value to the running total, and uses the result of that
value: cumulativeValue += b.value,
}))
}
}
Commit your changes and publish a function version. There is a step-by-step guide in the Palantir Foundry documentation on how to create a repository and publish functions.
In your Workshop, you can create a 'Chart: XY' widget. As the data source, select the function you created and pass in the relevant object set. There's also Palantir Foundry documentation on using a derived aggregation in Workshop.

Related

how to query mongodb nested query three level?

I have three collections (docs example struct see below :)
my mongodb version is 4.4
section
{
_id: ObjectId('62b131211331e0e9ba284187'),
name: 'name1',
},
{
_id: ObjectId('62b131211331e0e9ba284187'),
name: 'name2',
}
...
2.problems
{
_id: ObjectId('62b13d5f1331e0e9ba2841e6'),
title: 'title1',
score: 10,
section_id: ObjectId('62b131211331e0e9ba284188')
},
{
_id: ObjectId('62b13d5f1331e0e9ba2841e6'),
title: 'title2',
score: 20,
section_id: ObjectId('62b131211331e0e9ba284188')
},
{
_id: ObjectId('62b13d5f1331e0e9ba2841e6'),
title: 'title3',
score: 30,
section_id: ObjectId('62b131211331e0e9ba284188')
}
...
3.choices
{
_id: ObjectId('62b164ae1331e0e9ba284236'),
text: 'text1',
value: 0,
checked: false,
problem_id: ObjectId('62b13d5f1331e0e9ba2841ed')
},
{
_id: ObjectId('62b164ae1331e0e9ba284236'),
text: 'text2',
value: 0,
checked: false,
problem_id: ObjectId('62b13d5f1331e0e9ba2841ed')
},
{
_id: ObjectId('62b164ae1331e0e9ba284236'),
text: 'text3',
value: 0,
checked: false,
problem_id: ObjectId('62b13d5f1331e0e9ba2841ed')
},
...
the relationship of those collections
sections has many problems
problems has many choices
My question is :
Now I have an array of sections's id like below:
[
ObjectId('62b131211331e0e9ba284188'),
ObjectId('62b131211331e0e9ba28418a'),
ObjectId('62b131211331e0e9ba28418c')
]
I want to perform one query get the result like below:
[
{
_id: ObjectId('62b131211331e0e9ba284187'),
name: 'name1',
problems: [
{
_id: ObjectId('62b13d5f1331e0e9ba2841e6'),
title: 'title1',
score: 10,
section_id: ObjectId('62b131211331e0e9ba284188'),
choices: [
{}, // choice doc
]
},
{
_id: ObjectId('62b13d5f1331e0e9ba2841e6'),
title: 'title2',
score: 10,
section_id: ObjectId('62b131211331e0e9ba284188'),
choices: [
{}, // choice doc
]
},
...
]
},
{
_id: ObjectId('62b131211331e0e9ba284187'),
name: 'name1',
problems: [
{},
]
},
{
_id: ObjectId('62b131211331e0e9ba284187'),
name: 'name1',
problems: [
{},
]
}
]
I have try run this query in my mongoexpress but it didn't work:
// match stage (ps: start with sections collection)
{
"_id": {
$in: [
ObjectId('62b131211331e0e9ba284188'),
ObjectId('62b131211331e0e9ba28418a'),
ObjectId('62b131211331e0e9ba28418c')
]
}
// lookup stage
{
from: 'problems',
let: {"sid": "$_id"},
pipeline: [
{
"$match": {
"$expr": {
"$eq": ["$section_id", "$$sid"]
},
},
},
{
"$lookup": {
from: "choices",
let: {"chid": "$_id"},
pipeline: [
{
"$match": {
"$expr": {},
},
},
],
as: "choices"
},
},
],
as: "problems"
}

Schema/Resolve for nested objects graphql/mongoose

I am using graphql with mongoose and I am trying to access a nested object array in a json of this form:
"Plans": [
{
"id": ...
"name": ...
"frequency": ...
"lastExecuted": ...
"Steps": {
"Step": [
{
"id": ...
"shortDescription": ...
"description": ...
...
},
{...],
}
I created a mongoose model:
const PlanModel = Mongoose.model("Plan", {
name: String,
frequency: GraphQLString,
lastExecuted: String,
Steps: []
})
Intuitively I would insert my Stepmodel in the array, but this is giving me an error.
So I tried populating the array with the resolver:
Plans: {
type: GraphQLList(PlanType),
args: getGraphQLQueryArgs(PlanType),
resolve: (root, args, context, info) => {
return PlanModel
.find()
.populate("Steps")
.populate("Steps.Step")
.exec();
}
},
This is my PlanType:
const PlanType = new GraphQLObjectType({
name: 'Plan',
fields: () => ({
id: {
type: GraphQLID
},
name: {
type: GraphQLString
},
frequency: {
type: GraphQLString
},
lastExecuted: {
type: GraphQLString
},
maintenanceSteps: {
type: GraphQLList(StepType)
},
})
})
My GraphQL query returns an empty array in this case. I know this is a common problem, but I couldn't find any solution for my problem
The solution to my problem was adding another type:
const StepsType = new GraphQLObjectType({
name: 'Steps',
fields: () => ({
Step: {
type: GraphQLList(StepType)
}
})
})
const PlanType = new GraphQLObjectType({
name: 'Plan',
fields: () => ({
_id: {
type: GraphQLID
},
id: {
type: GraphQLString
},
name: {
type: GraphQLString
},
frequency: {
type: GraphQLString
},
lastExecuted: {
type: GraphQLString
},
status: {
type: GraphQLString
},
Steps: {
type: StepsType
},
})
})

Highcharts - how change JSON to csv

I have a small problem...
$(function () {
$.get('../../../abc-test.csv', function(data) {
// split the data set into ohlc and volume
var ohlc = [],
volume = [],
dataLength = data.length,
// set the allowed units for data grouping
groupingUnits = [[
'week', // unit name
[1] // allowed multiples
], [
'month',
[1, 2, 3, 4, 6]
]],
i = 0;
for (i; i < dataLength; i += 1) {
ohlc.push([
data[i][0], // the date
data[i][1], // open
data[i][2], // high
data[i][3], // low
data[i][4] // close
]);
volume.push([
data[i][0], // the date
data[i][5] // the volume
]);
}
$('#test').highcharts({
rangeSelector: {
selected: 1
},
title: {
text: 'AAPL Historical'
},
yAxis: [{
labels: {
align: 'right',
x: -3
},
title: {
text: 'OHLC'
},
height: '60%',
lineWidth: 2
}, {
labels: {
align: 'right',
x: -3
},
title: {
text: 'Volume'
},
top: '65%',
height: '35%',
offset: 0,
lineWidth: 2
}],
data: {
csv: data
//csv: document.getElementById('csv').innerHTML
},
series: [{
type: 'candlestick',
name: 'AAPL',
data: ohlc,
dataGrouping: {
units: groupingUnits
}
}, {
type: 'column',
name: 'Volume',
data: volume,
yAxis: 1,
dataGrouping: {
units: groupingUnits
}
}]
});
});
});
abc-test.csv:
Date,Open,High,Low,Close,Volume
2013-12-20,9371.08,9413.09,9352.98,9400.18,161686900
2013-12-19,9279.68,9351.9,9257.24,9335.74,98276500
2013-12-18,9145.35,9190.73,9122.05,9181.75,82342700
2013-12-17,9142.75,9161.8,9085.12,9085.12,72207500
2013-12-16,9004.62,9187.78,8997.75,9163.56,99105600
2013-12-13,9016.78,9046.63,8990.58,9006.46,67761700
2013-12-12,9032.67,9060.54,8984.28,9017,75120200
2013-12-11,9093.26,9153.14,9065.51,9077.11,64845800
2013-12-10,9180.29,9223.73,9091.97,9114.44,74363400
Why this code dont't work ?
Don't work:
- Two panes, candlestick and volume
- OHLC
- 1.7 million points with async loading...
The issue is in the "date" in csv or something else ... ?

Grails JSON.registerObjectMarshaller

I need to change the format for a json response. I have a simple domain class, GeoCounter, mapped against a database table that looks like:
import grails.rest.Resource
import groovy.transform.ToString
#Resource(uri="/fieldvaluecounters", formats=['json'])
#ToString
class GeoCounter {
String id
int value
static transients = ['geoCountry']
static mapping = {
table 'geo_counter'
id name: 'geoCountry', column: 'geo_country', generator: 'assigned'
value column: 'numOfHits'
version false
}
static constraints = {
id(unique:true, blank: false)
}
void setGeoCountry(String geoCountry) {
id = geoCountry
}
String getGeoCountry() {
return id
}
}
and I need this returned:
`data = [{key: "twn", "value": 3 },
{key: "gbr", "value": 1 },
{key: "en", "value": 4}];`
My spring resources.groovy has:
omnitureMarshallerRegistrar(MarshallerRegistrar)
and the Marshaller has:
class MarshallerRegistrar {
#PostConstruct
void registerMarshallers() {
println "Registering customer marshallers"
JSON.registerObjectMarshaller(GeoCounter) { GeoCounter gc ->
return [
key: gc.getGeoCountry(),
value: gc.getValue()
]
}
}
}
but my method
/poc/api/fieldvaluecounters
still returns id & value instead of key, value. What am I missing?
[
{
id: "twn",
value: 3
},
{
id: "gbr",
value: 1
},
{
id: "en;q=0.8",
value: 1
},
{
id: "en;q=0.5",
value: 8
},
{
id: "jpn",
value: 36
},
{
id: "idn",
value: 1
},
{
id: "chn",
value: 2
},
{
id: "fra",
value: 11
},
{
id: "en-AU;q=0.8",
value: 1
},
{
id: "usa",
value: 2
}
]

Saving JSON from API into Mongo using Mongoose

I am working on a node.js app with Mongo (using the mongoose solution) to create a custom reporting solution with data from the Quickbooks Online API.
Here is the workflow:
I authenticate my app with the QBO API
I make the API call, and get JSON
This is where I would like to parse the JSON to save it to my DB.
The JSON response
{ Header:
{ Time: '2014-09-09T10:55:01-07:00',
ReportName: 'VendorBalanceDetail',
StartPeriod: '2014-10-01',
EndPeriod: '2014-10-09',
Currency: 'USD',
Option: [ { Name: 'report_date', Value: '2014-10-09' } ] },
Columns:
{ Column:
[ { ColTitle: 'Date', ColType: 'tx_date' },
{ ColTitle: 'Transaction Type', ColType: 'txn_type' },
{ ColTitle: 'Num', ColType: 'doc_num' },
{ ColTitle: 'Due Date', ColType: 'due_date' },
{ ColTitle: 'Amount', ColType: 'subt_neg_amount' },
{ ColTitle: 'Open Balance', ColType: 'subt_neg_open_bal' },
{ ColTitle: 'Balance', ColType: 'rbal_neg_open_bal' } ] },
Rows:
{ Row:
[ { Header:
{ ColData:
[ { value: 'GS & CO' },
{ value: '' },
{ value: '' },
{ value: '' },
{ value: '' },
{ value: '' },
{ value: '' } ] },
Rows:
{ Row:
[ { ColData:
[ { value: '01/31/2014' },
{ value: 'Bill' },
{ value: 'FY/2013-01/2014' },
{ value: '01/31/2014' },
{ value: '9963.14' },
{ value: '9963.14' },
{ value: '9963.14' } ],
type: 'Data' },
{ ColData:
[ { value: '02/28/2014' },
{ value: 'Bill' },
{ value: '02/2014' },
{ value: '02/28/2014' },
{ value: '6378.14' },
{ value: '6378.14' },
{ value: '16341.28' } ],
type: 'Data' },
{ ColData:
[ { value: '03/31/2014' },
{ value: 'Bill' },
{ value: '03/2014' },
{ value: '03/31/2014' },
{ value: '2556.0' },
{ value: '2556.0' },
{ value: '18897.28' } ],
type: 'Data' },
{ ColData:
[ { value: '04/30/2014' },
{ value: 'Bill' },
{ value: '04/2014' },
{ value: '04/30/2014' },
{ value: '5221.0' },
{ value: '5221.0' },
{ value: '24118.28' } ],
type: 'Data' },
{ ColData:
[ { value: '05/31/2014' },
{ value: 'Bill' },
{ value: '05/2014' },
{ value: '05/31/2014' },
{ value: '2735.96' },
{ value: '2735.96' },
{ value: '26854.24' } ],
type: 'Data' },
{ ColData:
[ { value: '06/30/2014' },
{ value: 'Bill' },
{ value: '06/2014' },
{ value: '06/30/2014' },
{ value: '658.0' },
{ value: '658.0' },
{ value: '27512.24' } ],
type: 'Data' },
{ ColData:
[ { value: '07/31/2014' },
{ value: 'Bill' },
{ value: '6-17 to 7-31' },
{ value: '07/31/2014' },
{ value: '162.32' },
{ value: '162.32' },
{ value: '27674.56' } ],
type: 'Data' } ] },
Summary:
{ ColData:
[ { value: 'Total for GS & CO' },
{ value: '' },
{ value: '' },
{ value: '' },
{ value: '27674.56' },
{ value: '27674.56' },
{ value: '' } ] },
type: 'Section' }
My model:
module.exports = mongoose.model('vbDetail', {
company_name: String,
row:{
date: Date,
transaction_type: String,
transaction_num: String,
due_date: Date,
amount: Number,
open_balance: Number,
balance: Number,
identifier: String,
processing_date: Date,
processing_amount :Date,
notes: String
}
})
The last three fields are custom fields that I want to store in addition to the JSON response.
My question is how to I take the JSON from the response and parse it so it "adheres" to my model?