Related
I have a JSON file and I am trying to calculate the JSON file key based on the value and reformating it. My JSON file looks like below:
data=[
{
pet:'Cat',
fruit:'Apple',
fish:'Hilsha'
},
{
pet:'Dog',
fish:'Carp'
},
{
pet:'Cat',
fruit:'Orange',
fish:'Lobster'
}
];
I do like to calculate and formate it like below:
data=[
{
label:'Pet',
total:3,
list:[
{
name:'Cat',
value: 2,
},
{
name:'Dog',
value: 1,
}
]
},
{
label:'Fruit',
total:2,
list:[
{
name:'Apple',
value: 1,
},
{
name:'Orange',
value: 1,
}
]
},
{
label:'Fish',
total:3,
list:[
{
name:'Hilsha',
value: 1,
},
{
name:'Carp',
value: 1,
},
{
name:'Lobster',
value: 1,
}
]
},
];
If anybody can help me, it will be very help for me and will save a day.
I have fixed this task myself. If I have any wrong, you can put your comment fill-free :)
``
ngOnInit(): void {
this.dataService.$data.subscribe(data => {
// Create new object and calculation according to category
let petObj: any = {}
let fruitObj: any = {}
let fishObj: any = {}
data.forEach((el: any) => {
if (el.pet != undefined) {
petObj[el.pet] = (petObj[el.pet] || 0) + 1;
}
if (el.fruit != undefined) {
fruitObj[el.fruit] = (fruitObj[el.fruit] || 0) + 1;
}
if (el.fish != undefined) {
fishObj[el.fish] = (fishObj[el.fish] || 0) + 1;
}
});
// Create list according to category
let pet_list: any = [];
let fruit_list: any = [];
let fish_list: any = [];
for (var key in petObj) {
let pet = {
label: key,
value: petObj[key]
}
pet_list.push(pet)
}
for (var key in fruitObj) {
let fruit = {
label: key,
value: fruitObj[key]
}
fruit_list.push(fruit)
}
for (var key in fishObj) {
let fish = {
label: key,
value: fishObj[key]
}
fish_list.push(fish)
}
// Calculate total sum according to category
var totalPet = pet_list.map((res: any) => res.value).reduce((a: any, b: any) => a + b);
var totalFruit = fruit_list.map((res: any) => res.value).reduce((a: any, b: any) => a + b);
var totalFish = fish_list.map((res: any) => res.value).reduce((a: any, b: any) => a + b);
// Rearrange the JSON
this.rearrangeData = [
{
label: 'Pet',
total: totalPet,
list: pet_list
},
{
label: 'Fruit',
total: totalFruit,
list: fruit_list
},
{
label: 'Fish',
total: totalFish,
list: fish_list
}
]
console.log(this.rearrangeData)
// End rearrange the JSON
});
}
``
You can simplify your function. Take a look this one
group(oldData) {
const data = []; //declare an empty array
oldData.forEach((x) => {
//x will be {pet: 'Cat',fruit: 'Apple',fish: 'Hilsha'},
// {pet: 'Dog',fish: 'Carp'}
// ...
Object.keys(x).forEach((key) => {
//key will be 'pet','fruit',...
const item = data.find((d) => d.label == key); //search in the "data array"
if (item) { //if find it
item.total++; //add 1 to the property total of the element find it
// and search in the item.list the 'Cat'
const list = item.list.find((l) => l.name == x[key]);
//if find it add 1 to the property value of the list
if (list)
list.value++;
else
//if not, add to the list
//an object with property "name" and "value" equal 1
item.list.push({ name: x[key], value: 1 });
} else
//if the element is not in the "array data"
//add an object with properties label, total and list
//see that list is an array with an unique element
data.push({
label: key,
total: 1,
list: [{ name: x[key], value: 1 }],
});
});
});
return data;
}
You can use like
this.dataService.$data.subscribe(data => {
this.rearrangeData=this.group(data)
}
NOTE: this function the labels are 'pet','fruit' and 'fish' not 'Pet', 'Fruit' and 'Fish'
Did you try reading the text leading up to this exercise? That'd be my first approach. After that, I'd use reduce. You can do pretty much anything with reduce.
I am trying to import a custom plugin into my chart.
Got this plugin from my previous question: Question
Its a plugin so that I can use Grace in my version of Chart.js.
The version of Chart.js I am using is V2.9.4.
I am using vue-chartjs in Nuxt!.
Couldn't really find an answer anywhere else.
This is how it looks now
This is how I want it to look
Thanks in advance. :)
I made a Component called 'BarChart' in my Components folder.
I made a normal .vue file in my pages directory. In the <template> tag I added my <Barchart/> component.
In that same .vue file I added a script in the <script> tag.
The plugin code is included in the codes below, I didn't include it anywhere yet.
Barchart.vue (Component)
<script>
import {Bar} from "vue-chartjs";
export default {
extends: Bar,
props: {
data: {
type: String,
default: () => {},
},
options: {
type: Object,
default: () => {},
},
},
computed: {
Chart() {
return['data', 'options'];
},
},
mounted() {
this.renderChart(this.data, this.options);
},
};
</script>
.vue file (Include component)
<div class="chart">
<BarChart :data="barChartData" :options="barChartOptions" :height="200"/>
</div>
.vue file (script tags)
<script>
import BarChart from "~/components/plugins/BarChart";
export default {
components: {
BarChart,
},
data() {
return {
barChartData: {
labels: ["Verzonden", "Ontvangen", "Geopend", "Kliks"],
datasets: [
{
data: [25, 20, 20, 18],
backgroundColor: [
'#7782FF',
'#403DD3',
'#FFB930',
'#00E437',
],
barThickness : 50,
},
],
},
barChartOptions: {
responsive: true,
legend: {
display: false,
},
scales: {
xAxes: [
{
gridLines: {
display: false,
},
ticks: {
fontColor: "black",
fontSize: 14,
},
},
],
yAxes: [
{
ticks: {
beginAtZero: true,
min: 0,
stepSize: 5,
fontColor: '#ABACB3',
},
gridLines: {
display: true,
borderDash: [4, 4],
color: '#EEEDFB',
drawBorder: false,
},
},
],
},
},
};
},
};
</script>
Plugin code (where do I put this and how do I make it work?)
const plugin = {
id: "customScale",
beforeLayout: (chart, options, c) => {
let max = Number.MIN_VALUE;
let min = Number.MAX_VALUE
let grace = options.grace || 0
chart.data.datasets.forEach((dataset) => {
max = Math.max(max, Math.max(...dataset.data));
min = Math.min(min, Math.min(...dataset.data))
})
if (typeof grace === 'string' && grace.includes('%')) {
grace = Number(grace.replace('%', '')) / 100
chart.options.scales.yAxes[0].ticks.suggestedMax = max + (max * grace)
chart.options.scales.yAxes[0].ticks.suggestedMin = min - (min * grace)
} else if (typeof grace === 'number') {
chart.options.scales.yAxes[0].ticks.suggestedMax = max + grace
chart.options.scales.yAxes[0].ticks.suggestedMin = min - grace
}
}
}
According to the vue-chartjs documentation you can do this in 2 ways.
If you want the plugin to be available for all your charts you can use the global registration like so:
import Chart from 'chart.js'
Chart.pluginService.register({
id: "customScale",
beforeLayout: (chart, options, c) => {
let max = Number.MIN_VALUE;
let min = Number.MAX_VALUE
let grace = options.grace || 0
chart.data.datasets.forEach((dataset) => {
max = Math.max(max, Math.max(...dataset.data));
min = Math.min(min, Math.min(...dataset.data))
})
if (typeof grace === 'string' && grace.includes('%')) {
grace = Number(grace.replace('%', '')) / 100
chart.options.scales.yAxes[0].ticks.suggestedMax = max + (max * grace)
chart.options.scales.yAxes[0].ticks.suggestedMin = min - (min * grace)
} else if (typeof grace === 'number') {
chart.options.scales.yAxes[0].ticks.suggestedMax = max + grace
chart.options.scales.yAxes[0].ticks.suggestedMin = min - grace
}
}
});
This way of importing and registering should work from anywhere in your app.
The second way is an inline plugin. This needs to be done in your BarChart.vue and goes like this:
mounted() {
this.addPlugin(
Chart.pluginService.register({
id: "customScale",
beforeLayout: (chart, options, c) => {
let max = Number.MIN_VALUE;
let min = Number.MAX_VALUE
let grace = options.grace || 0
chart.data.datasets.forEach((dataset) => {
max = Math.max(max, Math.max(...dataset.data));
min = Math.min(min, Math.min(...dataset.data))
})
if (typeof grace === 'string' && grace.includes('%')) {
grace = Number(grace.replace('%', '')) / 100
chart.options.scales.yAxes[0].ticks.suggestedMax = max + (max * grace)
chart.options.scales.yAxes[0].ticks.suggestedMin = min - (min * grace)
} else if (typeof grace === 'number') {
chart.options.scales.yAxes[0].ticks.suggestedMax = max + grace
chart.options.scales.yAxes[0].ticks.suggestedMin = min - grace
}
}
});
)
}
My data that I'm given is formulated like the following. I'm struggling to get any usable data out of it using reduce or map
const data = [
{
id: 25,
status: 1,
description: "No Description",
length: 4,
data: [
{
id: 43,
comment: "Comment1",
eventTimestamp: 1541027189000,
intensity: 29
},
{
comment: "Comment2",
eventTimestamp: 1541027191000,
intensity: 33
},
{
id: 45,
comment: "Comment3",
eventTimestamp: 1541027193000,
intensity: 30
}
],
tTypes: [
{
id: 3,
label: "Johnny",
certainty: "TEST",
comment: "Test Purposes Only",
icon: "bottle",
number: 0
}
]
}
];
I've tried flatting, I've tried iterating the JSON twice and I just seem to end up with either "NaN" or Undefined. I'd like to be able to order them in time order (using time stamp), get the mix/max/ave from the intensity values and more. I have that figured out for the length which is a level higher, but just can't seem to figure out the rest. Can someone point me in the right direction?
export default function App() {
let tTypesArray = data.map((a) => a.tTypes);
let Walker = tTypesArray.reduce((a, tTypes) => tTypes.label === "Johnny" ? ++a : a, 0);
console.log(Walker);
console.log(tTypesArray[0].label);
console.log([].concat(...data)
.map(data => data.tTypes.number)
.reduce((a, b) => a + b))
console.log([].concat(...data).reduce((a, { tTypes: { id }}) => id, 0))
return <div className="App">ARG!</div>;
}
Are some of the examples I've tried.
https://codesandbox.io/s/purple-cache-ivz1y?file=/src/App.js
Is the link to the sandbox.
What I understood from you question is that you need to loop data and for each element in data you want to extract values and do some calculations.
First you need to loop your data input. I will use Array.forEach:
data.forEach(element => { ... })
Now that we have a loop we can access each element property and extract the information we want. For instance lets say you want to sort the comments by timestamp in ascending order:
const sortedComments = element.data.sort((a, b) => a.eventTimestamp - b.eventTimestamp);
console.log(sortedComments)
Now let's say you want the min, max, and average intensity from the comments. There are several ways to get it. Here is an algorithm for that:
let min = Infinity;
let max = -Infinity;
let sum = 0;
for(comment of sortedComments) {
if(comment.intensity < min) {
min = comment.intensity;
}
if(comment.intensity > max) {
max = comment.intensity;
}
sum += comment.intensity;
}
const avg = sum / sortedComments.length;
console.log({min, max, avg})
Putting it all together:
const data = [
{
id: 25,
confirmationStatus: 1,
description: "No Description",
length: 4,
data: [
{
id: 43,
comment: "Comment1",
eventTimestamp: 1541027189000,
intensity: 29
},
{
comment: "Comment2",
eventTimestamp: 1541027191000,
intensity: 33
},
{
id: 45,
comment: "Comment3",
eventTimestamp: 1541027193000,
intensity: 30
}
],
tTypes: [
{
id: 3,
label: "Johnny",
certainty: "TEST",
comment: "Test Purposes Only",
icon: "bottle",
number: 0
}
]
}
];
data.forEach(element => {
const sortedComments = element.data.sort((a, b) => a.eventTimestamp - b.eventTimestamp);
console.log(sortedComments);
let min = Infinity;
let max = -Infinity;
let sum = 0;
for(comment of sortedComments) {
if(comment.intensity < min) {
min = comment.intensity;
}
if(comment.intensity > max) {
max = comment.intensity;
}
sum += comment.intensity;
}
const avg = sum / sortedComments.length;
console.log({min, max, avg});
let walker = element.tTypes.reduce(
(a, tType) => (tType.label === "Johnny" ? ++a : a), 0
);
console.log(walker)
});
I hope it puts you in the right direction.
Currently we are trying to input uber data that includes time of day and ride fare into our TensorFlow.js model. We noticed that when we ran the model on the browser, the points are showing up on our scatterplot but during the training the loss and mean squared error values are not showing up and most importantly our model is not displaying a prediction line.
var userData = [
{
"City": "San Francisco",
"Product_Type": "UberEATS Marketplace",
"Trip_or_Order_Status": "COMPLETED",
"Request_Time": "2019-06-16 04:10:44 +0000 UTC",
"Begin_Trip_Time": "2019-06-16 04:44:40 +0000 UTC",
"Begin_Trip_Lat": "37.7352602",
"Begin_Trip_Lng": "-122.4203465",
"Begin_Trip_Address": "",
"Dropoff_Time": "2019-06-16 04:44:40 +0000 UTC",
"Dropoff_Lat": "37.7352602",
"Dropoff_Lng": "-122.4203465",
"Dropoff_Address": "",
"Distance_miles": "2.04",
"Fare_Amount": "32.34",
"Fare_Currency": "USD"
}...]
async function getData() {
const carsData = await userData;
// Here we map out the values for each car and filter out the list item that do not have an day or a pay value
const cleaned = carsData.map(car => ({
timeInMinutes: calculateMins(car.Request_Time),
pay_rate: normalizeUberPrice(car.Distance_miles, car.Fare_Amount),
}))
.filter(car => (car.day != null && car.pay != null));
return cleaned;
}
async function run() {
const data = await getData();
const values = data.map(d => ({
x: d.day,
y: d.pay,
}));
tfvis.render.scatterplot(
{ name: 'Horsepower v MPG' },
{ values },
{
xAxisDomain: [0, 1600],
yAxisDomain: [0,10],
xLabel: 'Day',
yLabel: 'Pay',
height: 300
}
);
const model = createModel();
tfvis.show.modelSummary({ name: 'Model Summary' }, model);
// Convert the data to a form we can use for training.
const tensorData = convertToTensor(data);
console.log(tensorData)
const { inputs, labels } = tensorData;
// Train the model
await trainModel(model, inputs, labels);
console.log('Done Training');
testModel(model, data, tensorData);
}
function createModel() {
const model = tf.sequential();
model.add(tf.layers.dense({ inputShape: [1], units: 25, useBias: true }));
model.add(tf.layers.dense({ units: 50, activation: "sigmoid" }));
model.add(tf.layers.dense({ units: 1, useBias: true }));
return model;
}
function convertToTensor(data) {
return tf.tidy(() => {
tf.util.shuffle(data);
const inputs = data.map(d => d.pay)
const labels = data.map(d => d.day);
const inputTensor = tf.tensor2d(inputs, [inputs.length, 1]);
const labelTensor = tf.tensor2d(labels, [labels.length, 1]);
//Step 3. Normalize the data to the range 0 - 1 using min-max scaling
const inputMax = inputTensor.max();
const inputMin = inputTensor.min();
const labelMax = labelTensor.max();
const labelMin = labelTensor.min();
const normalizedInputs = inputTensor.sub(inputMin).div(inputMax.sub(inputMin));
const normalizedLabels = labelTensor.sub(labelMin).div(labelMax.sub(labelMin));
return {
inputs: normalizedInputs,
labels: normalizedLabels,
// Return the min/max bounds so we can use them later.
inputMax,
inputMin,
labelMax,
labelMin,
}
});
}
async function trainModel(model, inputs, labels) {
model.compile({
optimizer: tf.train.adam(),
loss: tf.losses.meanSquaredError,
metrics: ['mse'],
});
const batchSize = 32;
const epochs = 30;
callbacks: tfvis.show.fitCallbacks(
{ name: 'Training Performance' },
['loss', 'mse'],
{
xAxisDomain: [0, 100],
yAxisDomain: [0,1],
height: 200,
callbacks: ['onEpochEnd'] }
// ',onBatchEnd'
),
history: tfvis.show.history({
name: 'History'},
history,
["loss","mse"])
});
}
function testModel(model, inputData, normalizationData) {
const { inputMax, inputMin, labelMin, labelMax } = normalizationData;
const [xs, preds] = tf.tidy(() => {
const xs = tf.linspace(0, 1, 100);
const preds = model.predict(xs.reshape([100, 1]));
const unNormXs = xs
.mul(inputMax.sub(inputMin))
.add(inputMin);
const unNormPreds = preds
.mul(labelMax.sub(labelMin))
.add(labelMin);
return [unNormXs.dataSync(), unNormPreds.dataSync()];
});
const predictedPoints = Array.from(xs).map((val, i) => {
return { x: val, y: preds[i] }
});
const originalPoints = inputData.map(d => ({
x: d.pay, y: d.day,
}));
console.log("ORIGINAL POINTS:")
console.log(originalPoints)
tfvis.render.scatterplot(
{ name: 'Model Predictions vs Original Data' },
{ values: [originalPoints, predictedPoints], series: ['original', 'predicted'] },
{
xAxisDomain: [0,10],
yAxisDomain: [0,1600],
xLabel: 'Horsepower',
yLabel: 'MPG',
height: 1000
}
);
}
document.addEventListener('DOMContentLoaded', run);
Basically we want to see a predicted line for our data but were not getting anything back.
It worked when we used data like this:
var userData = [{
day: 1
pay: 20
},...]
The data processing is not well performed. Thus, the values used for prediction contain NaN and Infinity. As a result, the error computed by model.fit is NaN and could therefore not be displayed on the chart of tfjs-vis.
The filtering
.filter(car => (car.day != null && car.pay != null));
is not removing NaN and Infinity. Instead, this condition can be used:
.filter(car => isFinite(car.pay + car.day) && !isNaN(car.pay + car.day));
Though, the NaN and Infinity are found within the values of car.day, here a general filtering is made over car.pay and car.day - thus the additional operation - to make sure that these values will not appear anywhere in the cleaned data.
here you can see how to display the loss.
I have a simple line graph with data in the format:
[
{
label: "lebel1",
x: 0,
y: 128
},
{
label: "lebel1",
x: 1,
y: 128
},
....
{
label: "lebel2",
x: 25,
y: 128
},
....
{
label: "lebel8",
x: 285,
y: 128
},
....
}
and I pass this into my nvd3 object:
nv.addGraph(function()
{
var chart = nv.models.lineChart();
chart.xAxis
.axisLabel("My X-Axis")
.ticks(36)
.tickFormat(function(d) { return d; });
chart.yAxis
.axisLabel('Voltage (v)')
.tickFormat(d3.format('.02f'));
d3.select('div svg')
.datum(myData)
.transition().duration(500)
.call(chart);
nv.utils.windowResize(function() { d3.select(gridSvgId).call(chart) });
return chart;
});
How can I have my x-axis ticks to show:
* eight labels: label1 - label8
Rather than have the grids broken up into a variable number of lines?
Try something like this
chart.xAxis.tickValues(['Label 1','Label 2','Label 3','Label 4','Label 5','Label 6','Label 7','Label 8']);
or if you want to get it from the dataset, you could try something like this,
chart.xAxis.tickValues(function(d) {
// do all you stuff and return an array
var dataset = ['Build Array from data'];
return dataset;
};)
Hope it helps