Following up on a previous question... I've got my minimal horizon chart example much more minimaler than before ( minimal cubism.js horizon chart example (TypeError: callback is not a function) )
<body>
<div class="mag"></div>
<script type="text/javascript">
var myContext = cubism.context();
var myMetr = myContext.metric(function(start, stop, step, callback) {
d3.json("../json/600s.json.php?t0=" + start/1000 + "&t1=" + stop/1000 + "&ss=" + step/1000, function(er, dt) {
if (!dt) return callback(new Error("unable to load data, or has NaNs"));
callback(null, dt.val);
});
});
var myHoriz = myContext.horizon()
.metric(myMetr);
d3.select(".mag")
.call(myHoriz);
</script>
</body>
The d3.json() bit calls a server side .php that I've written that returns a .json version of my measurements. The .php takes the start, stop, step (which cubism's context.metric() uses) as the t0, t1, and ss items in its http query string and sends back a .json file. The divides by 1000 are because I made my .php expect parameters in s, not ms. And the dt.val is because the actual array of my measurements is in the "val" member of the json output, e.g.
{
"other":"unused members...",
"n":5,
"val":[
22292.078125,
22292.03515625,
22292.005859375,
22292.02734375,
22292.021484375
]
}
The problem is, now that I've got it pared down to (I think) the bare minimum, AND I actually understand all of it instead of just pasting from other examples and hoping for the best (in which scenario, most things I try to change just break things instead of improving them), I need to start adding parameters and functions back to make it visually more useful.
Two problems first of all are, this measurement hovers all day around 22,300, and only varies +/- 10 maybe all day, so the graph is just a solid green rectangle, AND the label just says constantly "22k".
I've fixed the label with .format(d3.format(".3f")) (versus the default .2s which uses SI metric prefixes, thus the "22k" above).
What I can't figure out is how to use either axis, scale, extent, or what, so that this only shows a range of numbers that are relevant to the viewer. I don't actually care about the positive-green and negative-blue and darkening colours aspects of the horizon chart. I just used it as proof-of-concept to get the constantly-shifting window of measurements from my .json data source, but the part I really need to keep is the serverDelay, step, size, and such features of cubism.js that intelligently grab the initial window of data, and incrementally grab more via the .json requests.
So how do I keep the cubism bits I need, but usefully change my all-22300s graph to show the important +/- 10 units?
update re Scott Cameron's suggestion of horizon.extent([22315, 22320])... yes I had tried that and it had zero effect. Other things I've changed so far from "minimal" above...
var myHoriz = myContext.horizon()
.metric(myMetr)
.format(d3.format(".2f"))
.height(100)
.title("base1 (m): ")
.colors(["#08519c", "#006d2c"])
// .extent([22315, 22320]) // no effect with or without this line
;
I was able to improve the graph by using metric.subtract inserting it above the myHoriz line like so: (but it made the numerical label useless now):
var myMetr2 = myMetr.subtract(22315);
var myHoriz = myContext.horizon()
.metric(myMetr2)
.format...(continue as above)
All the examples seem so concise and expressive and work fine verbatim but so many of the tweaks I try to make to them seem to backfire, I'm not sure why that is. And similarly when I refer to the API wiki... maybe 4 out of 5 things I use from the API work immediately, but then I always seem to hit one that seems to have no effect, or breaks the chart completely. I'm not sure I've wrapped my head around how so many of the parameters being passed around are actually functions, for one thing.
Next hurdles after this scale/extent question, will be getting the horizontal time axis back (after having chopped it out to make things more minimal and easier to understand), and switching this from an area-looking graph to more of a line graph.
Anyway, all direction and suggestion appreciated.
Here's the one with the better vertical scale, but now the numerical label isn't what I want:
Have you tried horizon.extent? It lets you specify the [min, max] value for the horizon chart. By default, a linear scale will be created to map values within the extent to the pixels within the chart's height (specified with `horizon.height or default to 30 pixels).
Related
//#version=5
tspy = 60
blue = color.new(color.blue,tspy)
now when this following code executes many lines below/later:
tspy := 0
plot(rsi, title='bullish', color=blue)
As would be expected this does not change the transparency of variable blue by itself, obviously I can put blue := color.new(color.blue,0), but that is not ideal.
Is there a type of variable that is dynamic and will update itself whenever a variable contained within it changes, rather than having to manually re-execute the variables expression?
I have also tried setting the colours into a "function" call, but this seems to get complicated and causes more problems and code than just reentering/duplicating the original code. This may seem trivial, but it has a much wider importance and use case in more complex coding than this particular example.
Why not use a function?
//#version=5
indicator("My Script")
get_color(transp) => color.new(color.blue, transp)
plot(series=1, color=get_color(100))
plot(series=2, color=get_color(80))
plot(series=3, color=get_color(60))
plot(series=4, color=get_color(40))
plot(series=5, color=get_color(20))
I want to compare two smoothing methods for a bigram model:
Add-one smoothing
Interpolated Absolute Discounting
For the first method, I found some codes.
def calculate_bigram_probabilty(self, previous_word, word):
bigram_word_probability_numerator = self.bigram_frequencies.get((previous_word, word), 0)
bigram_word_probability_denominator = self.unigram_frequencies.get(previous_word, 0)
if self.smoothing:
bigram_word_probability_numerator += 1
bigram_word_probability_denominator += self.unique__bigram_words
return 0.0 if bigram_word_probability_numerator == 0 or bigram_word_probability_denominator == 0 else float(
bigram_word_probability_numerator) / float(bigram_word_probability_denominator)
However, I found nothing for the second method except for some references for 'KneserNeyProbDist'. However, this is for trigrams!
How can I change my code above to calculate it? The parameters of this method must be estimated from a development-set.
In this answer I just clear up a few things that I just found about your problem, but I can't provide a coded solution.
with KneserNeyProbDist you seem to refer to a python implementation of that problem: https://kite.com/python/docs/nltk.probability.KneserNeyProbDist
There exists an article about Kneser–Ney smoothing on wikipedia: https://en.wikipedia.org/wiki/Kneser%E2%80%93Ney_smoothing
The article above links this tutorial: https://nlp.stanford.edu/~wcmac/papers/20050421-smoothing-tutorial.pdf but this has a small fault on the most important page 29, the clear text is this:
Modified Kneser-Ney
Chen and Goodman introduced modified Kneser-Ney:
Interpolation is used instead of backoff. Uses a separate discount for one- and two-counts instead of a single discount for all counts. Estimates discounts on held-out data instead of using a formula
based on training counts.
Experiments show all three modifications improve performance.
Modified Kneser-Ney consistently had best performance.
Regrettable the modified Version is not explained in that document.
The original documentation by Chen & Goodman luckily is available, the Modified Kneser–Ney smoothing is explained on page 370 of this document: http://u.cs.biu.ac.il/~yogo/courses/mt2013/papers/chen-goodman-99.pdf.
I copy the most important text and formula here as screenshot:
So the Modified Kneser–Ney smoothing now is known and seems being the best solution, just translating the description beside formula in running code is still one step to do.
It might be helpful that below the shown text (above in screenshot) in the original linked document is still some explanation that might help to understand the raw description.
I have an SSRS Line chart which plots supply points with square feet on the X axis and Price on the Y axis. Right now I don't really care about making it pretty just getting the lines to show up correctly. I am plotting the points and grouping by Subdivision/Builder.
So for example Subdivision A has builders Y and Z. I want to show different colors and lines for Subdivision A builder Y verses Subdivision A Builder Z.
The problem is that the lines are not connecting when a point for another subdivision builder combination breaks up that line.
The grey line and points below are not all connected as the yellow point is between the grey points so the grey line is not connected to all grey points.
How can I make the points of the same color (same Subdivision/Builder) connected via a line?
As I found out the hard way recently, this problem is often caused by null values in the data not being properly handled by SSRS. Without seeing your data, I can't be certain that's the cause, but nulls were the culprit I encountered the same behavior.
The solutions usually involve assigning values to the color of the EmptyPoint property on the Series, sometimes in conjunction with setting the EmptyPointValue to specify null handling. I've found many references to this problem on the web, but I'll only post links to the best two, both of which are on StackExchange:
The thread SSRS Line Chart NULL VALUE - Horizontal Line contains a thorough discussion of this issue. The usual workaround given is to hard-code a color expression for each line using an IIf, but sometimes this isn't an option, especially if the field you're grouping on has dynamic, unpredictable values, as my dataset did.
The picture posted there depicts clear examples of the same type of line breaks. The user named trubs posted a code sample which illustrates how to set the EmptyPoint, in case where an Iif will work:
=iif(isNothing(Fields!SelectedValue.Value),'No Color',"LightBlue")
The first reply in SSRS Line Chart Not Connecting Data Points details a workaround for cases when the EmptyPoint value & nulls are the root cause and simple hard-coded IIfs won't do the trick. Although I have yet to get my line colors to match the point markers the way I'd like, I can verify that this solution at least gives you your lines back and allows you to assign a variety of colors to them. It's fairly simple and involves merely pasting in some VB code for a couple color properties.
I was asked in the comments section to provide the details of the solutions, but don't want to plagiarize, so I'll simply do a long direct quote of JohnBob's answer:
Firstly, in order to get the lines to join up, you need to set the
EmptyPoint colour for the series.
Select your series in your chart In the properties tab (not the
dialog) drill down into the EmptyPoint property and set the colour to
be Black
This will get them joining up - yay! But part of the line is colour
and the other part is black, right? That's a bit silly, especially
considering if you leave the colour to Automatic on the EmptyPoint
that it will be transparent.
So, then we need to get the series and the EmptyPoint's colours in
sync. Using code from here. I added some code to the code of the
report.
1). Right click on an empty space on the report and select "Report
Properties" 2). In the code tab, paste the following:
Private colorPalette As String() = {"#418CF0", "#FCB441", "#E0400A", "#05642E", "#1A3B69", "#BFBFBF", "#E0400A", "#FCB441", "DarkBlue", "Tomato", "Orange", "CornflowerBlue", "Gold", "Red", "Green", "LightBlue", "Lime", "Maroon", "LightSteelBlue", "Tan", "Silver"}
Private count As Integer = 0
Private mapping As New System.Collections.Hashtable()
Public Function GetColor(ByVal groupingValue As String) As String
If mapping.ContainsKey(groupingValue) Then
Return mapping(groupingValue)
End If
Dim c As String = colorPalette(count Mod colorPalette.Length)
count = count + 1
mapping.Add(groupingValue, c)
Return c
End Function
Then we need to call this code when setting the colour of the series
and of the EmptyPoint.
Select your series
In the properties tab paste something the following (replace WhateverTheGroupIsForYourSeries with your series group name):
=Code.GetColor(Fields!*WhateverTheGroupIsForYourSeries*.Value)
Drill down to the color element of the EmptyPoint Series property
Paste the same text as from point two [e.g. =Code.GetColor(Fields!*WhateverTheGroupIsForYourSeries*.Value)]
And voila! You're done! I can't believe how unnecessarily difficult
this is :D
I hope this helps.
Just put your Fields!(YourSeriesGroup).Value in Series Groups to above of
Fields!(YourCategoryGroup).Value in Category Groups, your series group should be in both Series Groups and Category Groups (should be above of your initial category group).
And after that right click horizontal axis and select Horizontal Axis Properties. Set Axis Type to Scalar and click OK.
I am currently working an application that is using a Cesium Viewer. I need to be able to display a collection of shapes that will be updated dynamically. I am having trouble understanding the best way to do this.
I currently am using Entities and using CallbackProperties to allow for the updating of shapes.
You can through this into a sandcastle to get an idea of how I am doing this. There is a polygon object that is being used as the basis for the cesiumCallback, and it is getting edited by another piece of code. (simulated with the setTimeout)
var viewer = new Cesium.Viewer('cesiumContainer', {});
var polygon = {};
polygon.coordinates = [
{longitude: 0, latitude: 0, altitude: 0},
{longitude: 10, latitude: 10, altitude: 0},
{longitude: 10, latitude: 0, altitude: 0}
];
// converts generic style options to cesium one (aka color -> material)
var polOpts = {};
// function for getting location
polOpts.hierarchy = new Cesium.CallbackProperty(function() {
var hierarchy = [];
for (var i = 0; i < polygon.coordinates.length; i++) {
var coordinate = polygon.coordinates[i];
hierarchy.push(Cesium.Cartesian3.fromDegrees(coordinate.longitude, coordinate.latitude, coordinate.altitude));
}
return hierarchy;
}, false);
viewer.entities.add({polygon: polOpts});
setInterval(function(polygon){
polygon.coordinates[0].longitude--;
}.bind(this, polygon), 1000);
The polygon being passed in is a class that generically describes a polygon, so it has an array of coordinates and style options, as well as a render method that calls this method renderPolygon passing in itself.
This method of rendering shapes works for everything I need it to, but it is not very performant. There are two cases for shapes updating, one type of shape will be updated over a long period of time, as a slow rate like once every few seconds. The other is shapes that will will get updated many times, like thousands, in a few seconds, then not change again for a long time, if ever.
I had two ideas for how to fix this.
Idea 1:
Have two methods, a renderDynamicPolygon and a renderStaticPolygon.
The renderDynamicPolygon method would do the above functionality, using the cesiumCallbackProperties. This would be used for shapes that are getting updated many times during the short time they are being updated.
The renderStaticPolygon method would replace the entities properties that are using callbackProperties with constant values, once the updating is done.
This creates a lot of other work to make sure shapes are in the right state, and doesn't help the shapes that are being updated slowly over a long period of time.
Idea 2:
Similarly to how the primitives work, I tried removing the old entity and adding it again with its updated properties each time its need to be updated, but this resulted in flickering, and unlike primitives, i could not find a async property for entities.
I also tried using primitives. It worked great for polylines, I would simply remove the old one and add a new one with the updated properties. I was also using the async = false to ensure there was no flickering. This issue I ran into here was not all shapes can be created using primitives. (Is this true?)
The other thing I tried was using the geometry instance using the geometry and appearance. After going through the tutorial on the cesium website I was able to render a few shapes, and could update the appearance, but found it close to impossible to figure out how to update the shapes correctly, and also have a very hard time getting them to look correct. Shapes need to have the right shape, a fill color and opacity and a stroke color, opacity and weight. I tried to use the polygonOutlineGeometry, but had not luck.
What would be the best way to implement this? Are one of these options headed the right way or is there some other method of doing this I have not uncovered yet?
[Edit] I added an answer of where I have gotten, but still not complete and looking for answers.
I have came up with a pretty good solution to this, but it still has one small issue.
I made too ways of showing entities. I am calling one render and one paint. Render uses the the Cesium.CallbackProperty with the isConstant property true, and paint with the isConstantProperty false.
Then I created a function to change the an entity from render to paint and vice vera. It goes through the entities callback properties an uses the setCallback property to overwrite the property with a the correct function and isConstant value.
Example:
I create a ellipse based on a circle object I have defined.
// isConst is True if it is being "painted" and false if it is being "rendered"
ellipse: lenz.util.extend(this._getStyleOptions(circle), {
semiMinorAxis: new Cesium.CallbackProperty(
this._getRadius.bind(this, circle),
isConst
),
semiMajorAxis: new Cesium.CallbackProperty(
this._getRadius.bind(this, circle),
isConst
),
})
So when the shape is being updated (while the user is drawing a shape) the shape is rendered with the isConstant being false.
Then when the drawing is complete it is converted to the painted version using some code like this:
existingEntity.ellipse.semiMinorAxis.setCallback(
this._getRadius.bind(this, circle),
isConst
);
existingEntity.ellipse.semiMajorAxis.setCallback(
this._getRadius.bind(this, circle, 1),
isConst
);
This works great performance wise. I am able to draw hundreds of shapes without the frame dropping much at all. I have attached a screen shot of the cesium map with 612 entities before and after my changes, the frame rate is in the upper right using the chrome render tool.
Before: Locked up at fps 0.9
Note: I redacted the rest of the ui, witch makes the globe look cut off, sorry
And after the changes: The fps remains at 59.9, almost perfect!
Whenever the entity is 'converted' from using constant to not constant callback properties, it and all other entities of the same type flash off then on again. I cannot find a better way to do this conversion. I feel as thought there must still be some thing I am missing.
You could try using a PositionPropertyArray as the polygon's hierarchy with SampledPositionProperty for any dynamic positions and ConstantPositionProperty for any static positions. I'm not sure if it would perform any better than your solution, but it might be worth testing. Here is an example of how it might work that you can paste into the Cesium Sandcastle:
var viewer = new Cesium.Viewer('cesiumContainer', {});
// required if you want no interpolation of position between times
var noInterpolation = {
type: 'No Interpolation',
getRequiredDataPoints: function (degree) {
return 2;
},
interpolateOrderZero: function (x, xTable, yTable, yStride, result) {
if (!Cesium.defined(result)) {
result = new Array(yStride);
}
for (var i = 0; i < yStride; i++) {
result[i] = yTable[i];
}
return result;
}
};
var start = viewer.clock.currentTime;
// set up the sampled position property
var sampledPositionProperty = new Cesium.SampledPositionProperty();
sampledPositionProperty.forwardExtrapolationType = Cesium.ExtrapolationType.HOLD;
sampledPositionProperty.addSample(start, new Cesium.Cartesian3.fromDegrees(0, 0)); // initial position
sampledPositionProperty.setInterpolationOptions({
interpolationAlgorithm: noInterpolation
});
// set up the sampled position property array
var positions = [
sampledPositionProperty,
new Cesium.ConstantPositionProperty(new Cesium.Cartesian3.fromDegrees(10, 10)),
new Cesium.ConstantPositionProperty(new Cesium.Cartesian3.fromDegrees(10, 0))
];
// add the polygon to Cesium viewer
var polygonEntity = new Cesium.Entity({
polygon: {
hierarchy: new Cesium.PositionPropertyArray(positions)
}
});
viewer.zoomTo(viewer.entities.add(polygonEntity));
// add a sample every second
var counter = 1;
setInterval(function(positionArray) {
var time = new Cesium.JulianDate.addSeconds(start, counter, new Cesium.JulianDate());
var position = new Cesium.Cartesian3.fromDegrees(-counter, 0);
positionArray[0].addSample(time, position);
counter++;
}.bind(this, positions), 1000);
One nice thing about this is you can set the timeline start/end time to a reasonable range and use it to see your polygon at any time within the sample range so you can see the history of your polygons through time (See here for how to change the timeline start/end time). Additionally, you don't need to use timers to set the positions, the time is built in to the SampledPositionProperty (although you can still add samples asynchronously).
However, this also means that the position depends on the current time in the timeline instead of a real-time array value. And you might need to keep track of a time somewhere if you aren't adding all the samples at once.
I've also never done this using ellipses before, but the semiMinorAxis and semiMajorAxis are properties, so you might still be able to use a SampledProperty.
Of course, this doesn't really matter if there are still performance issues. Hopefully it will improve as you don't need to recreate the array from scratch each callback and, depending on how you're getting the data to update the polygons, you might be able to add multiple samples at once. This is just speculation, but it's something to consider.
EDIT
Cesium can handle quite a bit of samples added to a sampled position, for example in the above code if you add a million samples to the position it takes a few seconds to load them all, but renders the polygon at any time without any performance issues. To test this, instead of adding samples using a timer, just add them all directly to the property.
for (var i = 0; i < 1000000; i++) {
var time = new Cesium.JulianDate.addSeconds(start, i, new Cesium.JulianDate());
var position = new Cesium.Cartesian3.fromDegrees(-(i % 2), 0);
positions[0].addSample(time, position);
}
However, if you run into memory problems currently there is no way to remove samples from a position property without accessing private variables. A work around would be to periodically create a new array containing new position properties and use the previous position property array's setValue() method to clear previous values or perhaps to use a TimeIntervalCollectionProperty as in this answer and remove time intervals with the removeInterval method.
I'm here with a new question.
I'm making a custom algorithm that need precomputed data for the graph edges. I use the AllEdgesIterator like this :
AllEdgesIterator it = graph.getAllEdges();
int nbEdges = it.getCount();
int count = 0;
int[] myData = new int[nbEdges];
while (it.next())
{
count++;
...
}
The first weird thing is that nbEdges is equal to 15565 edges but count is only equal to 14417. How is it possible ?
The second weird thing is when I run my custom A* : I simply browse nodes using the outEdgeExplorer but I get an IndexOutOfBound at index 15569 on myData array. I thought that the edge indexes were included in [0 ; N-1] where N is the number of edges, is it really the case ?
What could be happening here ? By the way, I have disabled graph contraction hierarchies.
Thank you for answering so fast every time !
The first weird thing is that nbEdges is equal to 15565 edges
but count is only equal to 14417. How is it possible ?
This is because of the 'compaction' where unreachable subnetworks are removed, but currently only nodes are removed from the graph the edges are just disconnected and stay in the edges-'array' marked as deleted. So iter.getCount is just an upper limit but the AllEdgeIterator excludes such unused edges correctly when iterating and has the correct count. But using iter.getCount to allocate your custom data array is the correct thing to do.
Regarding the second question: that is probably because the QueryGraph introduces new virtual edges with a bigger edgeId as iter.getCount. Depending on the exact scenario there are different solutions like just excluding or using the original edge instead etc