How to parse unstructured JSON file in Node? - json

Question

I have to parse a JSON file that has many objects but no structure to the file. It looks like this:
{"obj1": "john"}
{"obj2": "sally"}
{"obj3": "veronica"}
Each object is on on it's own there is no container. So when I open the file and try to iterate through it I get the error Unexpected token { in JSON
Aside from wrapping the objects in an array and then manually going through the whole file to add commas, how can I parse this?

Answer 1

If it's really one-object-per-line, it's fairly straightforward to take the string, break it into lines, and JSON.parse each line:
const str =
'{"obj1": "john"}\n' +
'{"obj2": "sally"}\n' +
'{"obj3": "veronica"}';
const array = str.split(/[\r\n]+/)
.map(entry => JSON.parse(entry));
console.log(array);
...but that's assuming it really is one object per line.
If you're reading the file, you don't have to start out with all in one string as above; just read line by line as Kevin B points out.
(Since you're using Node, I've happily used ES2015+ features above...)

Answer 2

If you assume each line of the input file is complete, self-standing JSON, then a split-into-lines-then-parse-each strategy works well.
But even if the data isn't limited to a single line, not all is lost. You can heuristically parse the file. It isn't hyper-efficient, but except for very large files you'll probably never know the difference:
function incrementallyParseJSON(filepath) {
var lines = fs.readFileSync(filepath)
.toString()
.split(/\n/g);
var result = [];
var [start, stop] = [0, 1];
while (stop <= lines.length) {
try {
var part = JSON.parse(lines.slice(start, stop).join('\n'));
result.push(part);
[start, stop] = [stop, stop+1];
} catch (e) {
stop += 1;
}
}
return result;
}
So if your file is:
{"obj1": "john"}
{"obj2": "sally",
"more": "other"}
{"obj3": "veronica"}
"something"
12
The result will be:
[ { obj1: 'john' },
{ obj2: 'sally', more: 'other' },
{ obj3: 'veronica' },
'something',
12 ]
Example:
function incrementallyParseJSON(str) {
var lines = str.split(/\n/g);
var result = [];
var [start, stop] = [0, 1];
while (stop <= lines.length) {
try {
var part = JSON.parse(lines.slice(start, stop).join('\n'));
result.push(part);
[start, stop] = [stop, stop+1];
} catch (e) {
stop += 1;
}
}
return result;
}
var str =
'{"obj1": "john"}\n' +
'{"obj2": "sally",\n' +
' "more": "other"}\n' +
'{"obj3": "veronica"}\n' +
'"something"\n' +
'12';
console.log(incrementallyParseJSON(str));