dc.js: show all domain x values in heatmap - heatmap

Requirement
I'm trying to create a heatmap indicating the frequency of an event through the hours of the week. On the Y axis I have the days of the week and on the X axis I have the hours of the day. I want to see all the hours of the day on the X axis, even those that do not have any corresponding frequency.
Problem
Unfortunately I'm not able to find out a way to display all the hours of the day. The Crossfilter includes only the hours of the day with a corresponding frequency during the week.
Current graphic result
As you can see not all hours of the day are displayed.
Code
let name = [
'0', // S
'6', // M
'5', // T
'4', // W
'3', // T
'2', // F
'1' // S
];
this.heatmapDimension= this.data_CrossfilterObject.dimension((d) => {
return [+d.hour, d.day + '.' + name[d.day]];
});
this.heatmapGroup= this.heatmapDimension.group().reduceSum((d) => d));
this.giornoOraDashboardItem.chart
.dimension(this.heatmapDimension)
.group(this.heatmapGroup)

The heatmap allows you to specify the x and y domains using .cols() and .rows() respectively.
It doesn't work for filtering out data but it does work for making sure that rows/columns exist.
In your case, since you are generating a string value for the y/row dimension, it would look like:
.rows(d3.range(7).map(i => i + '.' + name[i]))
.cols(d3.range(24))
Here is a demo fiddle with random data.
Using day names
Update: since your intention is to show day names, here is one way to do that.
Day names:
let name = [
'Su',
'M',
'Tu',
'W',
'Th',
'F',
'Sa'
];
Use the name directly in the dimension key accessor (no numeric prefix needed):
const heatmapDimension= cf.dimension((d) => {
return [d.hour, name[d.day]];
});
Specify the names, in reverse order, as the rows, and disable sorting of rows:
.rows(name.reverse()).rowOrdering(null)
(Reverse order is needed because the chart's zero is at the bottom.)
New fiddle.

Related

logic in setFormat to not roundup the values

I created a range using offset and setting the format for the range
I did that using below code
a1_range = Sheet1.getRange("D43");
var a4_range = a1_range.offset(0,6);
a4_range = a4_range.offset(0,1,15,6);
a4_range.setBackground("#e6e6e6").setBorder(true, true, true, true, true, true);
a4_range.setNumberFormat("$#");
The setNumberFormat is setting is write as I want , but the problem is when user enters the data, it is rounding up the value as shown in the picture.
However, to not round the values, I did this
a4_range.setNumberFormat("$#, ##0.000000");
This works but has extra trailing zeroes which I do not want as below,
What I want is when user comes and enters 56.768 in that range, the value should be exactly $56.768 and NOT 56.768000 and NOT have it round off to any decimal places.
Similarly, if the user enters 34.3 then it should be $34.3 and if user enters 8.98945 it should be $8.98945
How do I achieve this?
Try this:
a4_range.setNumberFormat("$0.#######");

Using 2 different outputs of 'return' of a function in separate elements of a plot

I am drawing a plot of voltage per time. For the voltage values, I want the values to be evaluated by a 'scaling' function which converts the values from volts to kilovolts if the biggest element is higher than 1000 volts (11000 volts to 11 KILOvolts).
This function is supposed to return 2 separate outputs; one for (new) values of voltage and one for the unit. The values are fed into the y axis values of the plot and the unit is given to the labeling line of that axis. For example:
import numpy as np
time = np.array([0, 1, 2, 3])
system_voltage1 = np.array([110, 120, 130, 150])
system_voltage2 = np.array([11000, 12000, 13000, 15000])
scaling_function(input)
if np.amax(input) < 1000:
output = input/1
Voltage_label = 'Voltage in Volts'
if np.amax(input) > 1000:
output = input/1000
Voltage_label = 'Voltage in KILOVolts'
return(output, Voltage_label)
fig14 = plt.figure(figsize=(16,9))
ax1 = fig14.add_subplot(111)
l1, = ax1.plot(time, scaling_function(system_voltage), color='r')
ax1.set_xlabel("time in second", color='k')
ax1.set_ylabel(Voltage_label, color='k')
Now, I am having trouble, calling this function properly. I need the function to only receive the output for scaling_function(system_voltage), and receive Voltage_label in ax1.set_ylabel(Voltage_label, color='k'). Now:
A) My problem: I don't know how to write the code so only the first output is received and used for scaling_function(system_voltage) , and the second element for the labeling line.
B) Something I tried but didn't work:Voltage_label does not recognize the value of voltage_label from scaling_function, as it is located in an outer loop than the function. I mean, I cannot access voltage_label as its value is not globally assigned.
Can anyone help me with this?
y,l = scaling_function(system_voltage)
l1, = ax1.plot(time, y, color='r')
ax1.set_xlabel("time in second", color='k')
ax1.set_ylabel(l, color='k')

Mean_squared_error output in function includes dtype and '0'

I want to calculate test statistics for a fb prophet forecast in a function because I want to average the test stats over different forecasts and cutoff points after using the fb-prophet cross_validation to get df_cv. I created a function that I apply to the dataframe after grouping by the cutoff points, in order to receive a measure per cutoff point. Then I calculate the mean over all these values.
The problem is that my function returns not only the value I am looking for but also a 0 as well as an information of the dtype. I can still do calculations with the returned value but when I want to plot etc. later it is very inconvenient. How can I strip these unnecessary values from the output?
def compute_avg_stats(df_cv,perf_measure):
measures = {'mse':mean_squared_error,'mae':mean_absolute_error,'mape':mean_absolute_percentage_error,'rmse':mean_squared_error}
performance_stats = {}
if perf_measure == 'rmse':
measure = np.sqrt(measures[perf_measure](y_true=df_cv['y'],y_pred=df_cv['yhat']))
else:
measure = measures[perf_measure](y_true=df_cv['yu'],y_pred=df_cv['yhat'])
return measure
df_cv.groupby('cutoff').apply(compute_avg_stats,perf_measure='rmse').to_frame().mean()
I think .mean() returns a Series. Try with .mean()[0]

How to process csv data (datetime) month, week, day, hour in highstock highcharts

I have a CSV file with following format:
<pre id="csv" style="display:none">
DATES,WHOLESALE,ECOMMERCE,RETAIL,LOANS,BONDISSUER
01/10/2018 00:00,25,16,13,1,0
01/10/2018 01:00,24,5,9,3,2
01/10/2018 02:00,28,6,17,0,6
The data range is 01/10/2018 00:00 - 31/10/2018 00:00
Interval is every hour.
I am using highstock stacked column with 5 categories: WHOLESALE,ECOMMERCE,RETAIL,LOANS,BONDISSUER.
My problem is, that the highstock navigator displays the data incorrectly. I think I have to customise property in range selector or navigator, but I can't find any documentation online. I tried inputDateParser, but it didn't work. Here is the jsfiddle
inputDateParser: function (value) {
value = value.split(/[:\.]/);
return Date.UTC(
1970,
0,
1,
parseInt(value[0], 10),
parseInt(value[1], 10),
parseInt(value[2], 10),
parseInt(value[3], 10)
);
}
How do I get the data range to be correct: month of October 2018 according to the dates in CSV?
I should not see a whole year in the navigator, when I only have data for October.
Thanks much appreciated
You would need to format the dates correctly, it can be done using the beforeParse callback function, like this:
data: {
csv: document.getElementById('csv').innerHTML,
beforeParse: function(e) {
let csv = e.split('\n'); //split by newline
let processedTable = []
processedTable.push(csv[0].split(','))
for (let i = 1; i < csv.length; i++) {
let row = csv[i].split(',');
if (row.length != 6) //skip empty rows or rows with more/less columns
continue;
let date = row[0].split(' ')[0].split('/')
let time = row[0].split(' ')[1].split(':')
processedTable.push(
[(new Date(date[2], date[1] - 1, date[0], time[0], time[1], 0)).getTime(), //get the timestamp for the date
parseInt(row[1]),
parseInt(row[2]),
parseInt(row[3]),
parseInt(row[4]),
parseInt(row[5])
].join(',')
)
}
return processedTable.join('\n') //join the array into a string again
},
},
Every row is parsed, by splitting it apart, the date is found, and milliseconds since 1970 is returned by getTime(). Then we join the cells into strings, and lastly the rows into a long string. The reason we convert this back into a string, is because highcharts is going to read it in from a string.
Working JSFiddle example: https://jsfiddle.net/ewolden/spmtgv3a/
API on beforeParse: https://api.highcharts.com/highcharts/data.beforeParse

Highcharts with external CSV $.get - No xAxis date

I'm trying to create a spline chart using this CSV:
slave_id,date,time,rtc_temp,temp1,temp2,temp3
1,2017/12/26,16:42:59,21,11.50,13.13,5.88
2,2017/12/26,16:43:29,21,14.13,20.63,99.99
1,2017/12/26,16:44:00,21,11.50,13.13,5.88
2,2017/12/26,16:44:30,21,14.13,20.63,99.99
1,2017/12/26,16:45:01,21,11.50,13.13,5.88
2,2017/12/26,16:45:31,21,14.13,20.63,99.99
1,2017/12/26,16:46:02,21,11.50,13.13,5.88
2,2017/12/26,16:46:32,21,14.13,20.63,99.99
As you can see here [IMAGE], the graph is showing the date and time, but the x Axis is not accepting the date / time.
Ive tried using date.UTC, but that did not work either. Can someone point me in the right direction?
https://jsfiddle.net/asvoy6b9/ [not working due to CSV missing]
Full code [Hastebin]
I see that date variable in your code is a string:
// all data lines start with a double quote
line = line.split(',');
date = line[1] + " " + line[2];
(...)
RTC.push([
date,
parseInt(line[3], 10)
]);
If you choose to construct the point's options as an array of two values and the first value is a string then it's treated as its name property (not x).
Explanation: https://www.highcharts.com/docs/chart-concepts/series
In that case Highcharts assigns subsequent integers as x values for all points (that's why there're values like 00:00:00.000 (1 Jan 1970), 00:00:00.001 etc.).
You need to parse your date to timestamp. You can use Date.UTC() (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/UTC) or some other function for this.
I've managed to get it working with Date.UTC using the following code:
var yyyymmdd = line[2].split("-"); //Split the date: 2017 12 16
var hhmmss = line[3].split(":"); //Split the time: 16 11 14
var date = Date.UTC(yyyymmdd[0], yyyymmdd[1] - 1, yyyymmdd[2], hhmmss[0], hhmmss[1], hhmmss[2]); //Stitch 'em together using Date.UTC