How to parse JSON string containing "NaN" in Node.js - json

Have a node.js app that is receiving JSON data strings that contain the literal NaN, like
"[1, 2, 3, NaN, 5, 6]"
This crashes JSON.parse(...) in Node.js. I'd like to parse it, if i can into an object.
I know NaN is not part of JSON spec. Most SO links (sending NaN in json) suggest to fix the output.
Here, though the data is produced in a server I don't control, it's by a commercial Java library where I can see the source code. And it's produced by Google's Gson library:
private Gson gson = (new GsonBuilder().serializeSpecialFloatingPointValues().create());
...
gson.toJson(data[i], Vector.class, jsonOut)
So that seems like a legitimate source. And according to the Gson API Javadoc it says I should be able to parse it:
Section 2.4 of JSON specification disallows special double values
(NaN, Infinity, -Infinity). However, Javascript specification (see
section 4.3.20, 4.3.22, 4.3.23) allows these values as valid
Javascript values. Moreover, most JavaScript engines will accept these
special values in JSON without problem. So, at a practical level, it
makes sense to accept these values as valid JSON even though JSON
specification disallows them.
Despite that, this fails in both Node.js and Chrome: JSON.parse('[1,2,3,NaN,"5"]')
Is there a flag to set in JSON.parse()? Or an alternative parser that accepts NaN as a literal?
I've been Googling for a while but can't seem to find a doc on this issue.
PHP: How to encode infinity or NaN numbers to JSON?

Have a node.js app that is receiving JSON data strings that contain the literal NaN, like
Then your NodeJS app isn't receiving JSON, it's receiving text that's vaguely JSON-like. NaN is not a valid JSON token.
Three options:
1. Get the source to correctly produce JSON
This is obviously the preferred course. The data is not JSON, that should be fixed, which would fix your problem.
2. Tolerate the NaN in a simple-minded way:
You could replace it with null before parsing it, e.g.:
var result = JSON.parse(yourString.replace(/\bNaN\b/g, "null"));
...and then handle nulls in the result. But that's very simple-minded, it doesn't allow for the possibility that the characters NaN might appear in a string somewhere.
Alternately, spinning Matt Ball's reviver idea (now deleted), you could change it to a special string (like "***NaN***") and then use a reviver to replace that with the real NaN:
var result = JSON.parse(yourString.replace(/\bNaN\b/g, '"***NaN***"'), function(key, value) {
return value === "***NaN***" ? NaN : value;
});
...but that has the same issue of being a bit simple-minded, assuming the characters NaN never appear in an appropriate place.
3. Use (shudder!) eval
If you know and trust the source of this data and there's NO possibility of it being tampered with in transit, then you could use eval to parse it instead of JSON.parse. Since eval allows full JavaScript syntax, including NaN, that works. Hopefully I made the caveat bold enough for people to understand that I would only recommend this in a very, very, very tiny percentage of situations. But again, remember eval allows arbitrary execution of code, so if there's any possibility of the string having been tampered with, don't use it.

When you deal with about anything mathematical or with industry data, NaN is terribly convenient (and often infinities too are). And it's an industry standard since IEEE754.
That's obviously why some libraries, notably GSON, let you include them in the JSON they produce, losing standard purity and gaining sanity.
Revival and regex solutions aren't reliably usable in a real project when you exchange complex dynamic objects.
And eval has problems too, one of them being the fact it's prone to crash on IE when the JSON string is big, another one being security risks.
That's why I wrote a specific parser (used in production) : JSON.parseMore

You can use JSON5 library. A quote from the project page:
The JSON5 Data Interchange Format (JSON5) is a superset of JSON that aims to alleviate some of the limitations of JSON by expanding its syntax to include some productions from ECMAScript 5.1.
This JavaScript library is the official reference implementation for JSON5 parsing and serialization libraries.
As you would expect, among other things it does support parsing NaNs (compatible with how Python and the like serialize them):
JSON5.parse("[1, 2, 3, NaN, 5, 6]")
> (6) [1, 2, 3, NaN, 5, 6]

The correct solution is to recompile the parser, and contribute an "allowNan" boolean flag to the source base. This is the solution other libraries have (python's comes to mind).
Good JSON libraries will permissively parse just about anything vaguely resembling JSON with the right flags set (perl's JSON.pm is notably flexible)... but when writing a message they produce standard JSON.
IE: leave the room cleaner than you found it.

Just a minor addition to TJ Crowder's already comprehensive enough reply, I'd rather use
var result = JSON.parse(yourString.replace(/\bNaN\b/g, '"NaN"'));
because I actually need to know if its a NaN value.
Also I'd do this inside a fetch or axios GET request, only if the default JSON parsing failed and the data came as a string.
const StringConstructor = "".constructor;
if (data.constructor === StringConstructor) {
data = JSON.parse(tableData.data.replace(/\bNaN\b/g, '"NaN"'))
}

Related

Regex for replacing unnecessary quotation marks within a JSON object containing an array

I am currently trying to format a JSON object using LabVIEW and have ran into the issue where it adds additional quotation marks invalidating my JSON formatting. I have not found a way around this so I thought just formatting the string manually would be enough.
Here is the JSON object that I have:
{
"contentType":"application/json",
"content":{
"msgType":2,
"objects":"["cat","dog","bird"]",
"count":3
}
}
Here is the JSON object I want with the quotation marks removed.
{
"contentType":"application/json",
"content":{
"msgType":2,
"objects":["cat","dog","bird"],
"count":3
}
}
I am still not an expert with regex and using a regex tester I was only able to grab the "objects" and "count" fields but I would still feel I would have to utilize substrings to remove the quotation marks.
Example I am using (would use a "count" to find the start of the next field and work backwards from there)
"([objects]*)"
Additionally, all the other Regex I have been looking at removes all instances of quotation marks whereas I only need a specific area trimmed. Thus, I feel that a specific regex replace would be a much more elegant solution.
If there is a better way to go about this I am happy to hear any suggestions!
Your question suggests that the built-in LabVIEW JSON tools are insufficient for your use case.
The built-in library converts LabVIEW clusters to JSON in a one-shot approach. Bundle all your data into a cluster and then convert it to JSON.
When it comes to parsing JSON, you use the path input terminal and the default type terminals to control what data is parsed from a JSON string.
If you need to handle JSON in a manner similar to say JavaScript, I would recommend something like the JSONText Toolkit which is free to use (and distribute) under the BSD licence. This allows more complex and iterative building of JSON strings from LabVIEW types and has text-path style element access along with many more features.
The Output controls from both my examples are identical - although JSONText provides a handy Pretty Print vi.
After using a regex from one of the comments, I ended up with this regex which allowed me to match the array itself.
(\[(?:"[^"]*"|[^"])+\])
I was able to split the the JSON string into before match, match and after match and removed the quotation marks from the end of 'before match' and start of 'after match' and concatenated the strings again to form a new output.

Generic JSON to XML transformation by templating

I'm trying to devise a service to convert a generic JSON data representation into XML data representation.
The first idea that came to my mind (and that I found on the internet) takes advantage of the Go templating utilities.
If I have a JSON data representation like the following:
{
"user": {
"name": "Luca",
"surname": "Rossi"
}
}
I could devise a template like the following:
<xml>
<user name="{{.user.name}}" surname="{{.user.surname}}" />
</xml>
to generate:
<xml>
<user name="Luca" surname="Rossi" />
</xml>
The problem is: Go requires the definition of a struct which declares how to marshal and unmarshal a JSON data representation; at the same time, however, I'd like to provide the template to generate the XML as a service configuration available at runtime.
The question is: "is it possible"?
I know (thanks tothis question) that I can take do something like this:
var anyJson map[string]interface{}
json.Unmarshal(bytes, &anyJson)
The problem comes when I have to access values: I'd be asked to do a type assertion, like
anyJson["id"].(string)
Now, I might be able to know the type of anyJson["id"] by means of JSON schema, for example, but I don't know if I can do a parametric type assertion, something like
anyJson["id"].(typeForIDFromJSONSchema)
When you unmarshal into map[string]interface{}, every nested JSON object will also be map[string]interface{}. So type assertions of the contained elements to string may typically work, but not to any struct type - the unmarshaller would always be unaware of your structs.
So the two options I suggest are
to do it 'the hard way' using type switches and type assertions - this is workable and fast, but not all that nice; or
to use a different tool such as jsoniter or gjson - these might be a little slower but they do allow you to traverse arbitrary JSON graphs
I have used both GJson and Jsoniter. GJson works by reading byte by byte through the input, using buffering to keep its speed up, providing an API that allows inspection of the current element and assertions to convert the values.
Jsoniter looks to me like a cleaner implementation along the lines of successful parsers in Java, but I haven't used it for parsing in this manner yet. (It can also be used simply as a fast drop-in replacement for the standard Go encoding/json.) I suggest you focus on using its Iterator and its WhatIsNext method.

Specifiy type when converting from XML to JSON in MarkLogic

Using MarkLogic 8, I'm using a custom XML to JSON conversion for json:transform-to-json, and I've got it working just about right except the conversion is outputting numbers as strings.
Is there a way to specify that the value of a particular element should be a number value, not a string?
I don't see anything in the doc for json:config, but just in case there's something I've missed, or if you have a neat post-processing trick, I'd love to hear about how to solve this problem.
You can do that by defining an XML Schema for the non-string type elements. Just make sure it is available in the context (by loading it into xdmp:schemas-database()), and that it is recognized (your XML needs to have a namespace that matches the XML Schema, and you might wanna use import schema)..
HTH!

OData Library serializes Edm.Single as number

According to the specification (and the odata.org summary) Edm.Single (and Edm.Double) are represented 'as a JSON string' (i.e. surrounded by double quotes). However the OData Library serializes Edm.Single (I haven't verified that Edm.Double has the same behavior) as number (i.e. unquoted). Is this a problem with the library - or the spec?
This looks like a bug in the spec (both the MS-ODATA as well as the odata.org). I verified that the product (both OData Library and WCF DS) read and write these values as un-quoted, since double can fit into JSON's number just fine. (Note that one exception of NaN and Inf which are quoted). (I'll see what we can do about fixing the spec :-))
It is likely a problem with the odata.org summary.
If in doubt you should check out the more canonical documentation for the [format][1]: http://msdn.microsoft.com/en-us/library/dd541188(v=PROT.10).aspx

Is it valid to define functions in JSON results?

Part of a website's JSON response had this (... added for context):
{..., now:function(){return(new Date).getTime()}, ...}
Is adding anonymous functions to JSON valid? I would expect each time you access 'time' to return a different value.
No.
JSON is purely meant to be a data description language. As noted on http://www.json.org, it is a "lightweight data-interchange format." - not a programming language.
Per http://en.wikipedia.org/wiki/JSON, the "basic types" supported are:
Number (integer, real, or floating
point)
String (double-quoted Unicode
with backslash escaping)
Boolean
(true and false)
Array (an ordered
sequence of values, comma-separated
and enclosed in square brackets)
Object (collection of key:value
pairs, comma-separated and enclosed
in curly braces)
null
The problem is that JSON as a data definition language evolved out of JSON as a JavaScript Object Notation. Since Javascript supports eval on JSON, it is legitimate to put JSON code inside JSON (in that use-case). If you're using JSON to pass data remotely, then I would say it is bad practice to put methods in the JSON because you may not have modeled your client-server interaction well. And, further, when wishing to use JSON as a data description language I would say you could get yourself into trouble by embedding methods because some JSON parsers were written with only data description in mind and may not support method definitions in the structure.
Wikipedia JSON entry makes a good case for not including methods in JSON, citing security concerns:
Unless you absolutely trust the source of the text, and you have a need to parse and accept text that is not strictly JSON compliant, you should avoid eval() and use JSON.parse() or another JSON specific parser instead. A JSON parser will recognize only JSON text and will reject other text, which could contain malevolent JavaScript. In browsers that provide native JSON support, JSON parsers are also much faster than eval. It is expected that native JSON support will be included in the next ECMAScript standard.
Let's quote one of the spec's - https://www.rfc-editor.org/rfc/rfc7159#section-12
The The JavaScript Object Notation (JSON) Data Interchange Format Specification states:
JSON is a subset of JavaScript but excludes assignment and invocation.
Since JSON's syntax is borrowed from JavaScript, it is possible to
use that language's "eval()" function to parse JSON texts. This
generally constitutes an unacceptable security risk, since the text
could contain executable code along with data declarations. The same
consideration applies to the use of eval()-like functions in any
other programming language in which JSON texts conform to that
language's syntax.
So all answers which state, that functions are not part of the JSON standard are correct.
The official answer is: No, it is not valid to define functions in JSON results!
The answer could be yes, because "code is data" and "data is code".
Even if JSON is used as a language independent data serialization format, a tunneling of "code" through other types will work.
A JSON string might be used to pass a JS function to the client-side browser for execution.
[{"data":[["1","2"],["3","4"]],"aFunction":"function(){return \"foo bar\";}"}]
This leads to question's like: How to "https://stackoverflow.com/questions/939326/execute-javascript-code-stored-as-a-string".
Be prepared, to raise your "eval() is evil" flag and stick your "do not tunnel functions through JSON" flag next to it.
It is not standard as far as I know. A quick look at http://json.org/ confirms this.
Nope, definitely not.
If you use a decent JSON serializer, it won't let you serialize a function like that. It's a valid OBJECT, but not valid JSON. Whatever that website's intent, it's not sending valid JSON.
JSON explicitly excludes functions because it isn't meant to be a JavaScript-only data
structure (despite the JS in the name).
A short answer is NO...
JSON is a text format that is completely language independent but uses
conventions that are familiar to programmers of the C-family of
languages, including C, C++, C#, Java, JavaScript, Perl, Python, and
many others. These properties make JSON an ideal data-interchange
language.
Look at the reason why:
When exchanging data between a browser and a server, the data can only
be text.
JSON is text, and we can convert any JavaScript object into JSON, and
send JSON to the server.
We can also convert any JSON received from the server into JavaScript
objects.
This way we can work with the data as JavaScript objects, with no
complicated parsing and translations.
But wait...
There is still ways to store your function, it's widely not recommended to that, but still possible:
We said, you can save a string... how about converting your function to a string then?
const data = {func: '()=>"a FUNC"'};
Then you can stringify data using JSON.stringify(data) and then using JSON.parse to parse it (if this step needed)...
And eval to execute a string function (before doing that, just let you know using eval widely not recommended):
eval(data.func)(); //return "a FUNC"
Via using NodeJS (commonJS syntax) I was able to get this type of functionality working, I originally had just a JSON structure inside some external JS file, but I wanted that structure to be more of a Class, with methods that could be decided at run time.
The declaration of 'Executor' in myJSON is not required.
var myJSON = {
"Hello": "World",
"Executor": ""
}
module.exports = {
init: () => { return { ...myJSON, "Executor": (first, last) => { return first + last } } }
}
Function expressions in the JSON are completely possible, just do not forget to wrap it in double quotes. Here is an example taken from noSQL database design:
{
"_id": "_design/testdb",
"views": {
"byName": {
"map": "function(doc){if(doc.name){emit(doc.name,doc.code)}}"
}
}
}
although eval is not recommended, this works:
<!DOCTYPE html>
<html>
<body>
<h2>Convert a string written in JSON format, into a JavaScript function.</h2>
<p id="demo"></p>
<script>
function test(val){return val + " it's OK;}
var someVar = "yup";
var myObj = { "func": "test(someVar);" };
document.getElementById("demo").innerHTML = eval(myObj.func);
</script>
</body>
</html>
Leave the quotes off...
var a = {"b":function(){alert('hello world');} };
a.b();