I have a JSON source that look like this... ( I checked it at jsonlint and it appears valid )
[{
"day": "06/19/2016",
"region": "Ohio",
"daily_er": "98.61"
}, {
"day": "06/19/2016",
"region": "Western NE",
"daily_ef": "98.63"
}, {.........
I'm trying to load the day as the y Axis with the region as X Axis
In my experimentation with loading from a local CSV, I DID manage to get the below to work....
(data.csv)
Categories,06/19/16,06/20/16,06/21/16,06/22/16,06/23/16,06/24/16,06/25/16
Ohio,98.61,97.75,97.19,97.21,97.97,93.66,98.3
Western NE,98.63,98.42,98.25,98.27,98.29,98.35,98.24
$.get('data.csv', function(data) {
// Split the lines
var lines = data.split('\n');
// Iterate over the lines and add categories or series
$.each(lines, function(lineNo, line) {
var items = line.split(',');
// header line containes categories
if (lineNo == 0) {
$.each(items, function(itemNo, item) {
if (itemNo > 0) options.xAxis.categories.push(item);
});
}
else {
var series = {
data: []
};
$.each(items, function(itemNo, item) {
if (itemNo == 0) {
series.name = item;
} else {
series.data.push(parseFloat(item));
}
});
options.series.push(series);
}
});
// Create the chart
var chart = new Highcharts.Chart(options);
});
I want to apologize up front because while in my mind I get how CSV gets parsed, I so rarely work with JSON that I am stumped as to how I parse the JSON above. In my brain I'm getting hung up on dupes for the day values and how that's handled. Has anyone had an experience with this they'd like to share, maybe get me pointed in the right direction?
Many thanks!
JW
Related
Using the latest Flare build originally built by prefuse, I am trying to get an indent field like the one in Layouts seen here. I am working with a list of objects that I pull from Google Firebase. While I can put them on a graph just fine and compare one and other values I can't find instructions on the different layouts. I am about to study the docs but I wanted to know if there was anything else out there I could reference.
Edit:
http://flare.prefuse.org/api/flare/vis/operator/layout/Layout.html I have found the general layouts here. However I only am able to show 1 or two circles unless I do AxisLayout.
For reference, my data pulled from firebase is something like this.
An array of objects.
Each object has properties name, sales, date, active and such.
I want it to act like the example above and show the item name in each circle. Then when the user clicks the circle he is able to show the properties of the item.
Edit: I was able to find an article on it, but after hours of constructing my data around his set format, I am not able to mock it entirely.
http://simon.oconnorlamb.com/ria/2012/03/visualising-data-with-flare/
Edit: To go into detail:
When I pull from my json list from Google Firebase I parse it so that it puts the items in referencable categories. All of which go into an array to mock the tutorial I linked above. I am trying to mock the structure as close as possible.
private function handleDataRead(e:DatabaseEvent):void
{
var trueDataArray:Array = new Array();
//Extract manufacturers.
var manufacturers:Vector.<String> = new Vector.<String>();
for each (var item:Object in e.data)
{
if (manufacturers.indexOf(item.Manufacturer) == -1)
{
manufacturers.push(item.Manufacturer);
//Example: {type:'Manufacturer',id:'0',name:'Company A'}
trueDataArray.push( {type:'manufacturer',
id:manufacturers.indexOf(item.Manufacturer).toString(),
name:item.Manufacturer});
}
}
//Extract Item Name
var itemNames:Vector.<String> = new Vector.<String>();
for each (var item:Object in e.data)
{
if (itemNames.indexOf(item.ItemName) == -1)
{
itemNames.push(item.ItemName);
var idValue:String = new String(itemNames.indexOf(item.ItemName) + (manufacturers.length - 1) +1);
trueDataArray.push( {type:'item',
id:idValue,
name:item.ItemName,
manufacturer:manufacturers.indexOf(item.Manufacturer).toString()} );
}
}
//Extract property 1
var mlCount:Vector.<int> = new Vector.<int>();
for each (var item:Object in e.data)
{
if (item.hasOwnProperty("ML"))
{
if (mlCount.indexOf(item.ML) == -1)
{
mlCount.push(item.ML);
var idValue:String = new String(mlCount.indexOf(item.ML) + (itemNames.length - 1) + (manufacturers.length - 1) +1);
trueDataArray.push({type:'mL',
id:idValue,
name:(item.ML as int).toString(),
item:itemNames.indexOf(item.ItemName).toString()});
}
}
}
//Extract another property
var mgCount:Vector.<int> = new Vector.<int>();
for each (var item:Object in e.data)
{
if (item.hasOwnProperty("MG"))
{
if (mgCount.indexOf(item.MG) == -1)
{
mgCount.push(item.MG);
var idValue:String = new String(mgCount.indexOf(item.MG) + mlCount.indexOf(item.ML) + (itemNames.length - 1) + (manufacturers.length - 1) +1);
trueDataArray.push({type:'mG',
id:idValue,
name:(item.MG as int).toString(),
mL:mlCount.indexOf(item.ML).toString()});
}
}
}
The result looks like this.
[
{
"name":"Company A",
"type":"manufacturer",
"id":"0"
},
{
"name":"Company B",
"type":"manufacturer",
"id":"1"
},
{
"name":"Company C",
"type":"manufacturer",
"id":"2"
},
{
"name":"Company D",
"type":"manufacturer",
"id":"3"
},
{
"name":"Company E",
"type":"manufacturer",
"id":"4"
},
{
"type":"manufacturer",
"id":"5"
},
... //So on
{
"manufacturer":"0",
"name":"Item Name 1",
"type":"item",
"id":"18"
},
{
"manufacturer":"0",
"name":"Item Name 2",
"type":"item",
"id":"19"
},
{
"manufacturer":"0",
"name":"Item Name 3",
"type":"item",
"id":"20"
},
{
"manufacturer":"0",
"name":"Item Name 4",
"type":"item",
"id":"21"
...//So on
{
"name":"60",
"item":"0",
"type":"mL",
"id":"195"
},
{
"name":"100",
"item":"5",
"type":"mL",
"id":"196"
},
{
"name":"120",
"item":"36",
"type":"mL",
"id":"197"
},
{
"name":"30",
"item":"100",
"type":"mL",
"id":"198"
}
...//and so forth
]
When I pass this to my function to create the nodes and edges (that I again based off the blog in the link above)
public function buildTree(arr:Array):Data
{
var d:Data = new Data(true);
//Keyed lookup for easy edge addition in step 2
var nodeLookup:Object = {};
var row:Object;
var ns:NodeSprite;
//Step 1: Add all rows of data;
for each(row in arr){
ns = d.addNode(row);
nodeLookup[row.id] = ns;
}
//Step 2: Add edges
for each(ns in d.nodes){
if(ns.data.hasOwnProperty('manufacturer')){
d.addEdgeFor(nodeLookup[ns.data.manufacturer],ns);
}
else if(ns.data.hasOwnProperty('item')){
d.addEdgeFor(nodeLookup[ns.data.item],ns);
}
else if(ns.data.hasOwnProperty('mL')){
d.addEdgeFor(nodeLookup[ns.data.mL],ns);
}
}
return d;
}
and construct it
data = buildTree(trueDataArray);
sourceTree = new ItemVisualisation(data);
sourceTree.bounds = new Rectangle(10, 10, 550, 550);
sourceTree.x = 20;
sourceTree.y = 20;
addChild(sourceTree);
sourceTree.operators.add(new IndentedTreeLayout());
sourceTree.operators.add(new ShapeEncoder("data.type"));
sourceTree.operators.add(new ColorEncoder("data.type", Data.NODES, "lineColor", ScaleType.CATEGORIES));
sourceTree.data.nodes.setProperties({fillColor:0, lineWidth:2});
sourceTree.update();
I get the following..
I almost have this down but I don't know what I am doing wrong. Everything seems to be as it should in relation.
Edit: It seems that the nodes may be linking properly with each other however this is not the layout I desire. I cannot get any other layouts to work either.
trueDataArray.push({type:'root', id:0, name:'rootname'});
I was able to solve this problem by binding everything to 1 node. I was having some trouble listing anything beyond two steps but that is beyond the requirement of my project.
I have to parse a JSON file that has many objects but no structure to the file. It looks like this:
{"obj1": "john"}
{"obj2": "sally"}
{"obj3": "veronica"}
Each object is on on it's own there is no container. So when I open the file and try to iterate through it I get the error Unexpected token { in JSON
Aside from wrapping the objects in an array and then manually going through the whole file to add commas, how can I parse this?
If it's really one-object-per-line, it's fairly straightforward to take the string, break it into lines, and JSON.parse each line:
const str =
'{"obj1": "john"}\n' +
'{"obj2": "sally"}\n' +
'{"obj3": "veronica"}';
const array = str.split(/[\r\n]+/)
.map(entry => JSON.parse(entry));
console.log(array);
...but that's assuming it really is one object per line.
If you're reading the file, you don't have to start out with all in one string as above; just read line by line as Kevin B points out.
(Since you're using Node, I've happily used ES2015+ features above...)
If you assume each line of the input file is complete, self-standing JSON, then a split-into-lines-then-parse-each strategy works well.
But even if the data isn't limited to a single line, not all is lost. You can heuristically parse the file. It isn't hyper-efficient, but except for very large files you'll probably never know the difference:
function incrementallyParseJSON(filepath) {
var lines = fs.readFileSync(filepath)
.toString()
.split(/\n/g);
var result = [];
var [start, stop] = [0, 1];
while (stop <= lines.length) {
try {
var part = JSON.parse(lines.slice(start, stop).join('\n'));
result.push(part);
[start, stop] = [stop, stop+1];
} catch (e) {
stop += 1;
}
}
return result;
}
So if your file is:
{"obj1": "john"}
{"obj2": "sally",
"more": "other"}
{"obj3": "veronica"}
"something"
12
The result will be:
[ { obj1: 'john' },
{ obj2: 'sally', more: 'other' },
{ obj3: 'veronica' },
'something',
12 ]
Example:
function incrementallyParseJSON(str) {
var lines = str.split(/\n/g);
var result = [];
var [start, stop] = [0, 1];
while (stop <= lines.length) {
try {
var part = JSON.parse(lines.slice(start, stop).join('\n'));
result.push(part);
[start, stop] = [stop, stop+1];
} catch (e) {
stop += 1;
}
}
return result;
}
var str =
'{"obj1": "john"}\n' +
'{"obj2": "sally",\n' +
' "more": "other"}\n' +
'{"obj3": "veronica"}\n' +
'"something"\n' +
'12';
console.log(incrementallyParseJSON(str));
I currently have a highchart that takes values from a csv file and displays them in a line chart, which works fine.
However, I'm looking to add drill down data for the points in my chart. The drill down values need to be taken from a file ( preferably the same csv file but can be another one as well). I'm unsure how to format my csv file and change my code to do this. Any help would be greatly appreciated.
Here is what I have so far https://jsfiddle.net/g8mdLLLp/
$.get('http://localhost/Pre/myfile.csv', function(data) {
// Split the lines
var lines = data.split('\n');
$.each(lines, function(lineNo, line) {
var items = line.split(',');
// header line containes categories
if (lineNo == 0) {
$.each(items, function(itemNo, item) {
if (itemNo > 0) options.xAxis.categories.push(item);
});
}
// the rest of the lines contain data with their name in the first position
else {
var series = {
data: []
};
$.each(items, function(itemNo, item) {
if (itemNo == 0) {
series.name = item;
} else {
series.data.push(parseFloat(item));
}
});
options.series.push(series);
}
});
var chart = new Highcharts.Chart(options);
});
My csv file looks like:
Results
ValueOne,11,22,16,10,11,33,03,19,13,02,15
ValueTwo,23,34,32,23,32,16,30,21,34,21,31
ValueThree,86,76,79,77,91,74,81,81,75,63,64
ValueFour,3,16,13,15,27,11,24,21,19,13,26
ValueFive 1,77,61,64,71,71,75,63,63,74,64,71
Thanks in advance
I would have said that it worked before. Now, I have updated meanwhile to latest Highcharts version. And, no idea why, it suddenly wants absolutely to display the first "column" of my CSV file. It looks like this:
,Switzerland,Europe,Global
1980,0.02854,0.01931,0.00547
1981,0.02898,0.01931,0.00549
Highcharts ("my code") wants to display this ""-column. And if I change it to "Years", or "Categories", it's all the same. I don't have three, but four entries in the legend.
The Highcharts-code looks like this:
// Split the lines
var lines = data.split('\n');
$.each(lines, function(lineNo, line)
{
var items = line.split(',');
// header line containes series name
if (lineNo === 0)
{
$.each(items, function(itemNo, item)
{
if (item!='')
{
if(item == 'Switzerland')
{
options.series.push(
{
name:item,
lineWidth: 5,
data:[]
});
}
else
{
options.series.push(
{
name:item,
data:[]
});
}
}
});
}
....
I tried to change the line
if (item!='')
to something like
if ((item!='') && (item!=' '))
or
if ((item!='') && (item!='Years'))
(when I added "Years" in the first place of the CSV file), but I only get than error messages...
Thanks for any hints!
I found similar example from the past and the problem is only with index, which is used during push points to series.
Parser:
var options = {
chart: {
renderTo: 'container',
zoomType:'xy'
},
xAxis:{
categories:[],
tickInterval:10,
tickmarkPlacement:'on'
},
title: {
text: 'Oviedo hoy'
},
series: []
};
$.get('data.csv', function(data) {
// Split the lines
var lines = data.split('\n');
$.each(lines, function(lineNo, line) {
var items = line.split(',');
// header line containes series name
if (lineNo === 0) {
$.each(items, function(itemNo, item) {
if(item!='')
{
options.series.push({
name:item,
data:[]
});
}
});
}
// the rest of the lines contain data with their name in the first position
else {
console.log(options.series);
$.each(items, function(itemNo, item) {
if(itemNo == 0)
options.xAxis.categories.push(item)
else
options.series[itemNo-1].data.push(parseFloat(item));
});
}
});
var chart = new Highcharts.Chart(options);
});
After some back and forth, here is how it works now:
$.get('data.csv', function(data)
{
// Split the lines
var lines = data.split('\n');
$.each(lines, function(lineNo, line)
{
var items = line.split(',');
// header line containes series name
if (lineNo === 0)
{
$.each(items, function(itemNo, item)
{
if (itemNo > 0)
{
if(item == 'Switzerland')
{
options.series.push(
{
name:item,
lineWidth: 5,
data:[]
});
}
else
{
options.series.push(
{
name:item,
data:[]
});
}
}
});
}
// the rest of the lines contain data with their name in the first position
else
{
$.each(items, function(itemNo, item)
{
if(itemNo == 0)
{
options.xAxis.categories.push(item);
}
else if (item == "null")
{
options.series[itemNo-1].data.push(null);
}
else
{
options.series[itemNo-1].data.push(parseFloat(item));
}
});
}
});
//console.log(options.series);
var chart = new Highcharts.Chart(options);
});
});
[
{
"tree_id": 6,
"fields" : ["id","lft", "rgt"], // tree_id is stripped if requested via fields because redundant
"values" :
[1,1,4,[
[2,2,3,[]]
]]
}
// more could follow ...
]
above is the json code that Bobab uses to export/import nested sets.
Baobab nested set json export/import format
How can i parse a nested html list to yield json like above?
I am trying to manipulate nested lists using drag and drop
Nestable list
It has 2 functions that kind of do what i want to achieve, but my head keeps twisting around it.
toHierarchy: function(options) {
var o = $.extend({}, this.options, options),
sDepth = o.startDepthCount || 0,
ret = [];
$(this.element).children(o.items).each(function () {
var level = _recursiveItems(this);
ret.push(level);
});
//console.log(JSON.stringify(ret));
return ret;
function _recursiveItems(item) {
var id = ($(item).attr(o.attribute || 'id') || '').match(o.expression || (/(.+)[-=_](.+)/));
if (id) {
var currentItem = {"id" : id[2]};
if ($(item).children(o.listType).children(o.items).length > 0) {
currentItem.children = [];
$(item).children(o.listType).children(o.items).each(function() {
var level = _recursiveItems(this);
currentItem.children.push(level);
});
}
return currentItem;
}
}
},
toArray: function(options) {
var o = $.extend({}, this.options, options),
sDepth = o.startDepthCount || 0,
ret = [],
left = 2;
ret.push({
"item_id": o.rootID,
"parent_id": 'none',
"depth": sDepth,
"left": '1',
"right": ($(o.items, this.element).length + 1) * 2
});
$(this.element).children(o.items).each(function () {
left = _recursiveArray(this, sDepth + 1, left);
});
ret = ret.sort(function(a,b){ return (a.left - b.left); });
//console.log(JSON.stringify(ret));
return ret;
function _recursiveArray(item, depth, left) {
var right = left + 1,
id,
pid;
if ($(item).children(o.listType).children(o.items).length > 0) {
depth ++;
$(item).children(o.listType).children(o.items).each(function () {
right = _recursiveArray($(this), depth, right);
});
depth --;
}
id = ($(item).attr(o.attribute || 'id')).match(o.expression || (/(.+)[-=_](.+)/));
if (depth === sDepth + 1) {
pid = o.rootID;
} else {
var parentItem = ($(item).parent(o.listType)
.parent(o.items)
.attr(o.attribute || 'id'))
.match(o.expression || (/(.+)[-=_](.+)/));
pid = parentItem[2];
}
if (id) {
ret.push({"item_id": id[2], "parent_id": pid, "depth": depth, "left": left, "right": right});
}
left = right + 1;
return left;
}
},
If your goal is to insert that data on the database using the Baobab library then you do not need to create the JSON code with the left/right indexes yourself, which can be fairly complicated to do.
Just send to the server tree structured data and on server side iterate over it adding the objects to the database.
You could create a generic tree structure with something like this (using jQuery to have a shorter example):
function genTree(domNode){
var parentObj = {
data : { /* filled with data found in domNode, e.g. the baobab node id */ },
children: []
};
$(domNode).find('> li, > ul > li').each(function(){
parentObj.children.push(genTree(this));
});
return parentObj;
}
Then when you'll travel the structure you will use the Baobab API to add the nodes to your database (at that point you can export it to JSON if you really need it)