Is crossfilter manipulating my data?
Background
I have performed all the processing I need server side and just want to graph exactly what comes down the json pipe. So far I've get the graph working exactly how I want it to except for it seems my data is being manipulated.
Here's my crossfilter code:
ndx = crossfilter(rData);
runDimension = ndx.dimension(function (d) { return [+d.series, +d.norm_1]; });
runGroup = runDimension.group();
runGroup.reduceSum(function (d) { return d.value;});
Note: norm_1 is unique
Issues
Basically I'm noticing two issues:
I know for a fact that all my data will be between -1 and 1 (I've run several checks to test this), BUT when graphing it I see it dips down to -1.4 in some places.
My server sends exactly 1000 rows of data, but by breakpointing some of the dc.js code I can see it's only graphing 752 rows.
More Evidence
On my chart I've set the valueAccessor and added some checks to test the values going out of bounds, and I can see very clearly it goes out:
.valueAccessor(function (d) {
if (d.value > 1 || d.value < -1) {
console.log(d);
}
return d.value;
})
The data from the server requires a small amount formatting before going into crossfilter (it comes down as a table and needs to be split into series objects). I used this as an opportunity to test whether the data goes out of bounds, and I can see clearly it stays within bounds:
for (var i = 0; i < $scope.remoteData.rows.length; i++) {
for (var j = 0; j < $scope.remoteData.labels.length; j++) {
var label = $scope.remoteData.labels[j];
var value = $scope.remoteData.rows[i][label];
if (value > 1 || value < -1) {
console.log({
label: label,
i: i,
series: j,
norm_1: $scope.remoteData.rows[i].norm_1,
value: value,
});
}
rData.push({
series: j,
norm_1: $scope.remoteData.rows[i].norm_1,
value: value
})
}
}
Discussion
I suspect my problems have something to do with:
runGroup.reduceSum(function (d) { return d.value;});
Is this function adding together certain data points?
Sounds like you have some rows for which [+d.series, +d.norm_1] is not unique. And yes any rows with the same key will be added with reduceSum.
I'd suggest making your dimension key be something that's really unique.
If you don't have a unique key, with a little more work you could use the array indices themselves as the dimension key. It will mean you have to use both key and value accessors everywhere to look back in the original array.
Something like:
ndx = crossfilter(d3.range(0, rData.length));
runDimension = ndx.dimension(function(d) { return d; })
runGroup = runDimension.group().reduceSum(function(d) {
return rData[d].value;
})
chart.keyAccessor(function(kv) { return rData[kv.key].x; })
.valueAccessor(function(kv) { return rData[kv.key].y; })
Related
I have a huge component that can receive a start and end date, plus some floors numbers, and based on these numbers, it generates a table, something like the below image.
Example of this table:
The problem is, how to make this table generate faster?
To generate the floors I use a for a loop.
JavaScript code:
for (let i = 0; i < section.floorQuantity; i++) {
floors.push(i + 1)
}
and for each day, I generate a column with the number of floors as cells (divs).
you can use "useMemo" to cache the same computation and it's result change when input data changes
const computeExpensiveValue=(inputs)=>{
//do something
return result
}
const memoizedValue = useMemo(() => computeExpensiveValue(a, b), [a, b]);
Good day,
I am trying to get data into a table, with the tour_id and every single media_id (the station_id i am getting from somewhere else), the ordernumber is what is giving me a headache:
I am trying to get every station one number for every media i am posting.
For example:
station 1 has 2 medias
and station 2 has 3
then the odernumbers should be like this: 0, 0, 1, 1, 1
I am using the following Code at this moment:
for(var i = 0; i < this.currentStations.length; i++){
this.http.get("http://localhost:3000/mediasforstation/" + this.currentStations[i].id).subscribe((res) => {
medias = res;
for (var j = 0; j < medias.length; j++){
this.http.post("http://localhost:3000/posttourstations",
{"tour_id": id, "media_id": medias[j].id, "ordernumber": i}).subscribe(function(res) {
console.log(res);
}.bind(this));
}
});
}
Everything but the ordernumber works, however, the ordernumber always takes the number of stations involved, in our example above it would be 2.
How do I fix this?
Thank you very much for your help.
As I understand, you need to keep the index value. The type of variable i is var which is function scoped. Within outer loop, you are calling an API that returns some response, meanwhile the value of i is updated and for next index/counter, the API call has been sent. When you get response from API calls, you get the value of i where the outer loop has been called of.
In other words, you need to understand the difference between var and let. Your problem can be solved by replacing
for(var i=0;...)
with
for(let i=0;...)
Here's providing you the sample code.
//block scoped - retains value of i
for (let i=0;i<10;i++){
this.http.get('https://jsonplaceholder.typicode.com/users').subscribe(res=>{
for(var j=0;j<5;j++){
console.log(`i=>${i}`)
}
})
}
//function scoped - gets updated value of i
for (var i=0;i<10;i++){
http.get('https://jsonplaceholder.typicode.com/users').subscribe(res=>{
for(var j=0;j<5;j++){
console.log(`i=>${i}`)
}
})
}
Below is the code snippet for a barchart with colored bars:
var Dim2 = ndx.dimension(function(d){return [d.SNo, d.something ]});
var Group2 = Dim2.group().reduceSum(function(d){ return d.someId; });
var someColors = d3.scale.ordinal().domain(["a1","a2","a3","a4","a5","a6","a7","a8"])
.range(["#2980B9","#00FFFF","#008000","#FFC300","#FF5733","#D1AEF1","#C0C0C0","#000000"]);
barChart2
.height(250)
.width(1000)
.brushOn(false)
.mouseZoomable(true)
.x(d3.scale.linear().domain([600,800]))
.elasticY(false)
.dimension(Dim2)
.group(Group2)
.keyAccessor(function(d){ return d.key[0]; })
.valueAccessor(function(d){return d.value; })
.colors(someColors)
.colorAccessor(function(d){return d.key[1]; });
How do I add a legend to this chart?
Using composite keys in crossfilter is really tricky, and I don't recommend it unless you really need it.
Crossfilter only understands scalars, so even though you can produce dimension and group keys which are arrays, and retrieve them correctly, crossfilter is going to coerce those arrays to strings, and that can cause trouble.
Here, what is happening is that Group2.all() iterates over your data in string order, so you get keys in the order
[1, "a1"], [10, "a3"], [11, "a4"], [12, "a5"], [2, "a3"], ...
Without changing the shape of your data, one way around this is to sort the data in your legendables function:
barChart2.legendables = function() {
return Group2.all().sort((a,b) => a.key[0] - b.key[0])
.map(function(kv) {
return {
chart: barChart2,
name: kv.key[1],
color: barChart2.colors()(kv.key[1]) }; }) };
An unrelated problem is that dc.js takes the X domain very literally, so even though [1,12] contains all the values, the last bar was not shown because the right side ends right at 12 and the bar is drawn between 12 and 13.
So:
.x(d3.scale.linear().domain([1,13]))
Now the legend matches the data!
Fork of your fiddle (also with dc.css).
EDIT: Of course, you want the legend items unique, too. You can define uniq like this:
function uniq(a, kf) {
var seen = [];
return a.filter(x => seen[kf(x)] ? false : (seen[kf(x)] = true));
}
Adding a step to legendables:
barChart2.legendables = function() {
var vals = uniq(Group2.all(), kv => kv.key[1]),
sorted = vals.sort((a,b) => a.key[1] > b.key[1] ? 1 : -1);
// or in X order: sorted = vals.sort((a,b) => a.key[0] - b.key[0]);
return sorted.map(function(kv) {
return {
chart: barChart2,
name: kv.key[1],
color: barChart2.colors()(kv.key[1]) }; }) };
Note that we're sorting by the string value of d.something which lands in key[1]. As shown in the comment, sorting by x order (d.SNo, key[0]) is possible too. I wouldn't recommend sorting by y since that's a reduceSum.
Result, sorted and uniq'd:
New fiddle.
I'm working on a match-3 style puzzle game using Flixel, and so I'm working on checking each row and column to see if there is a match at any given time. However, I have 6 different pieces (as of right now) that are active, and each piece is identified by an integer. Given that, I can check, for each and every single piece, by doing something like this:
public function matchingCheck():void
{
if (piecesArray[0][1] == 1 && piecesArray[1][1] == 1 && piecesArray[2][1] == 1) {
FlxG.log("Yay!");
}
}
However, this is rather unwieldy and would basically cause way too much repetition for my liking.
At the very least, I would like to be able to check if the values in these arrays are equal to one another, without having to specify which value it is. At the very best, I'd love to be able to check an entire row for three (or more) adjacent pieces, but I will settle for doing that part manually.
Thanks for your help!
EDIT: Nevermind, my edit didn't work. It was just checking if piecesArray[2][1] == 1, which makes me a sad panda.
EDIT 2: I've selected the correct answer below - it's not exactly what I used, but it definitely got me started. Thanks Apocalyptic0n3!
You could cut down on that code a little bit by using another function
private function checkValid( arrayOfItemsToCheck:Array, value:* ):Boolean {
for ( var i:Number = 0; i < arrayOfItemsToCheck.length; i++ ) {
if ( arrayOfItemsToCheck[i] != value ) {
return false;
}
}
return true;
}
Then you just do this in your if statement:
if ( checkValid( [ piecesArray[0][1], piecesArray[1][1], piecesArray[2][1] ], 1 ) ) {
FlxG.log("Yay!");
}
That does assume all items need to be equal to 1, though. It's still a lot of code, but it cuts out one set of "= 1 &&" for each check.
How about something like this which would tell you both if a match existed and what match it was:
public function checkForMatch():void{
var rows:int = piecesArray.length;
for(var i:int=0; i<rows; i++){
var match:int = checkRow(piecesArray[i]);
if(match > -1) {
FlxG.log("Yay you matched " + match);
}
}
}
private function ckeckRow(row:Array):int{
if(row[0] == row[1] == row[2]){
return row[0];
}
return -1;
}
So I have a list of words (the entire English dictionary).
For a word matching game, when a player moves a piece I need to check the entire dictionary to see if the the word that the player made exists in the dictionary. I need to do this as quickly as possible. simply iterating through the dictionary is way too slow.
What is the quickest algorithm in AS3 to search a long list like this for a match, and what datatype should I use? (ie array, object, Dictionary etc)
I would first go with an Object, which is a hash table (at least, storage-wise).
So, for every word in your list, make an entry in your dictionary Object and store true as its value.
Then, you just have to check if a given word is a key into your dictionary to know whether the word the user has choosen is valid or not.
This works really fast in this simple test (with 10,000,000 entries):
var dict:Object = {};
for(var i:int = 0; i < 10000000; i++) {
dict[i] = true;
}
var btn:Sprite = new Sprite();
btn.graphics.beginFill(0xff0000);
btn.graphics.drawRect(0,0,50,50);
btn.graphics.endFill();
addChild(btn);
btn.addEventListener(MouseEvent.CLICK,checkWord);
var findIt:Boolean = true;
function checkWord(e:MouseEvent):void {
var word:String;
if(findIt) {
word = "3752132";
} else {
word = "9123012456";
}
if(dict[word]) {
trace(word + " found");
} else {
trace(word + " not found");
}
findIt = !findIt;
}
It takes a little longer to build the dictionary, but lookup is almost instantaneous.
The only caveat is that you will have to consider certain keys that will pass the check and not necessarily be part of your words list. Words such as toString, prototype, etc. There are just a few of them, but keep that in mind.
I would try something like this with your real data set. If it works fine, then you have a really easy solution. Go have a beer (or whatever you prefer).
Now, if the above doesn't really work after testing it with real data (notice I've build the list with numbers cast as strings for simplicity), then a couple of options, off the top of my head:
1) Partition the first dict into a set of dictionaries. So, instead of having all the words in dict, have a dictionary for words that begin with 'a', another for 'b', etc. Then, before looking up a word, check the first char to know where to look it up.
Something like:
var word:String = "hello";
var dictKey:String = word.charAt(0);
// actual check
if(dict[dictKey][word]) {
trace("found");
} else {
trace("not found");
}
You can eventually repartition if necessary. I.e, make dict['a'] point to another set of dictionaries indexed by the first two characters. So, you'll have dict['a']['b'][wordToSearch]. There are a number of possible variations on this idea (you'd also have to come up with some strategy to cope with words of two letters, such as "be", for instance).
2) Try a binary search. The problem with it is that you'll first have to sort the list, upfront. You have to do it just once, as it doesn't make sense to remove words from your dict. But with millions of words, it might be rarther intensive.
3) Try some fancy data structures from open source libraries such as:
http://sibirjak.com/blog/index.php/collections/as3commons-collections/
http://lab.polygonal.de/ds/
But again, as I said above, I'd first try the easiest and simpler solution and check if it works against the real data set.
Added
A simple way to deal with keywords used for Object's built-in properties:
var dict:Object = {};
var keywordsInDict:Array = [];
function buildDictionary():void {
// let's assume this is your original list, retrieved
// from XML or other external means
// it contains "constructor", which should be dealt with
// separately, as it's a built-in prop of Object
var sourceList:Array = ["hello","world","foo","bar","constructor"];
var len:int = sourceList.length;
var word:String;
// just a dummy vanilla object, to test if a word in the list
// is already in use internally by Object
var dummy:Object = {};
for(var i:int = 0; i < len; i++) {
// also, lower-casing is a good idea
// do that when you check words as well
word = sourceList[i].toLowerCase();
if(!dummy[word]) {
dict[i] = true;
} else {
// it's a keyword, so store it separately
keywordsInDict.push(word);
}
}
}
Now, just add an extra check for built-in props in the checkWords function:
function checkWord(e:MouseEvent):void {
var word:String;
if(findIt) {
word = "Constructor";
} else {
word = "asdfds";
}
word = word.toLowerCase();
var dummy:Object = {};
// check first if the word is a built-in prop
if(dummy[word]) {
// if it is, check if that word was in the original list
// if it was present, we've stored it in keywordsInDict
if(keywordsInDict.indexOf(word) != -1) {
trace(word + " found");
} else {
trace(word + " not found");
}
// not a built-in prop, so just check if it's present in dict
} else {
if(dict[word]) {
trace(word + " found");
} else {
trace(word + " not found");
}
}
findIt = !findIt;
}
This isn't specific to ActionScript, but a Trie is a suitable data structure for storing words.