I'm trying to write a JSON column to MySQL using JOOQ and Jackson but I'm not sure why it isn't serialized as JSON but as a toString representation.
Here is a table for which I generate JOOQ definitions:
create table JsonPayload
(
name varchar(127) primary key,
rules JSON not null,
defaultValue tinyint(1) default 0 not null
);
This are the classes I'd like to bind my model.
data class RuleTest(val name: String, val test: Boolean)
data class Rule(val name: String, val test: Boolean, val rule: RuleTest)
data class JsonPayload(val name: String, val rules: List<Rule>, val defaultValue: Boolean)
Insertion code:
dsl.insertInto(JSONPAYLOAD)
.set(dsl.newRecord(
JSONPAYLOAD,
JsonPayload(
"Test",
listOf(Rule("rule1", false, RuleTest("rule1", false)),
Rule("rule2", true, RuleTest("rule1", false))),
true
)
))
.execute()
It serializes and deserializes fine, however it doesn't write correct JSON to MySQL:
mysql> select * from JsonPayload;
+------+-----------------------------------------------------------------------------------------------------------------------------------------------+--------------+
| name | rules | defaultValue |
+------+-----------------------------------------------------------------------------------------------------------------------------------------------+--------------+
| Test | ["Rule(name=rule1, test=false, rule=RuleTest(name=rule1, test=false))", "Rule(name=rule2, test=true, rule=RuleTest(name=rule1, test=false))"] | 1 |
+------+-----------------------------------------------------------------------------------------------------------------------------------------------+--------------+
This is the demo project I created to demonstrate this behaviour https://github.com/v1ctor/jooq-json-demo
Can you please help me to understand how to write correct Json to MySQL?
That's an interesting feature idea, which isn't supported yet by the DefaultRecordUnmapper implementation in jOOQ. I've created feature requests for this:
#13604 The possibility to make reflective use of Jackson from the DefaultRecordUnmapper
#13605 The possibility to use out of the box Jackson converters in the code generator
Currently, Jackson can only be used for mapping JSON and JSONB to your own data structures when reading from the database. Not when writing to the database. But there isn't any reason why the inverse logic shouldn't be available as well.
In the meantime, you have to implement a data type Converter<JSON, List<Rule>> (or a Binding, if you need more power) and attach that to your generated code, see:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-advanced/codegen-config-database/codegen-database-forced-types/
The benefit of using a Converter is that you now get type safety whenever you read/write to this column.
I am using lua in asterisk pbx. I encounter following problem while processing json string.
json "null" value converted to function type in lua. why?
how to handle this scenario? because i am expecting nil because no value means null in json and nil means nothing in lua.
local json = require( "json" )
local inspect = require("inspect")
local myjson_str='{"Sms":{"key":"xxxxxxxxxxxxxxxxxxxxx","to":"{caller}","senderid":null,"type":"Simple","content":"Your request has been accepted in Previous Miss call. We get back to you very soon."}}'
local myjson_table = json.decode(myjson_str)
print(type(myjson_table["Sms"]["senderid"]))
print(myjson_table)
print(inspect(myjson_table))
print(json.encode(myjson_table))
out put for above is
function
table: 0xf5e770
{
Sms = {
content = "Your request has been accepted in Previous Miss call. We get back to you very soon.",
key = "xxxxxxxxxxxxxxxxxxxxx",
senderid = <function 1>,
to = "{caller}",
type = "Simple"
}
}
{"Sms":{"type":"Simple","key":"xxxxxxxxxxxxxxxxxxxxx","senderid":null,"content":"Your request has been accepted in Previous Miss call. We get back to you very soon.","to":"{caller}"}}
It is up to specific library to decide how to represent null value.
Using nil has its own problem because its not possible find either
original JSON has key with null value or there no such key at all.
So some libraries just return some unique value. Some provide
a way to pass this value like json.deconde(str, NULL_VALUE).
So answer is just read the doc/source of library you use.
Most likely it provide something like json.null value to check
either value is null. But function is really strange choice because
they have some undetermined rules of uniqueness.
Or try another library.
First of all, #moteus is right:
It is up to specific library to decide how to represent null value
If you're using the JSON library by Jeffrey Friedl the solution is to use a placeholder instead of null and serializing the table structure to a json string using designated encode options:
-- define a placeholder
NullPlaceholder = "\0"
-- use it in an internal table
tableStructure = {}
tableStructure['someNullValue'] = NullPlaceholder
-- pass the placeholder to the encode methode
encode_options = { null = NullPlaceholder }
jsonString = JSON:encode(tableStructure, nil, encode_options)
which leads to
{"someNullValue": null}
In Swift, NSJSONSerialization.JSONObjectWithData uses Optional(<null>) for null value in JSON. Is that possible to tell the parse to use nil instead (i.e., drop fields with null values because Swift Dictionary doesn't allow nil value)?
Optional() is an NSNull value. You can avoid error like this:
if !(myField is NSNull) {
//Whatever you want to do with myField when not null
} else {
//Whatever you want to do when myField is null
}
NSJSONSerialization uses NSNull objects, and that's what you're seeing; you can simply check for them by comparing to NSNull().
I would say this is a problem with your JSON schema. null and string are different types of things in JSON. The fact that your key can have a value that is sometimes null and sometimes a string seems to me that you guys are not following a rigorous schema.
A good design would have the key either not be there, or if it is there, its value is guaranteed to be a string. Accessing the dictionary key in Swift produces an optional, which is nil if the key is not there, or a value if the key is there (in which case you know it's a string), which matches what you want.
I am trying to map a complex json object to my objects
my json contains a property that is a true or false value:
object {
rootProp2 = 1; <== fails to get mapped (this is a BOOL object)
} to object result: (null), (null),(null),(null) with object mapping (null)
how do i tell restkit to map rootProp2 to a BOOL value
it stays as 0, even though server returns "true" for this value
why arent my BOOL values mapped correctly ?
This is very sad i had to debug this for a few hours
but the reason for the problem that my json came with property name rootProop2
and my mapping had rootProp2 -> 1 missing letter and all goes kaput!
What is the preferred method for returning null values in JSON? Is there a different preference for primitives?
For example, if my object on the server has an Integer called "myCount" with no value, the most correct JSON for that value would be:
{}
or
{
"myCount": null
}
or
{
"myCount": 0
}
Same question for Strings - if I have a null string "myString" on the server, is the best JSON:
{}
or
{
"myString": null
}
or
{
"myString": ""
}
or (lord help me)
{
"myString": "null"
}
I like the convention for collections to be represented in the JSON as an empty collection http://jtechies.blogspot.nl/2012/07/item-43-return-empty-arrays-or.html
An empty Array would be represented:
{
"myArray": []
}
EDIT Summary
The 'personal preference' argument seems realistic, but short sighted in that, as a community we will be consuming an ever greater number of disparate services/sources. Conventions for JSON structure would help normalize consumption and reuse of said services. As far as establishing a standard, I would suggest adopting most of the Jackson conventions with a few exceptions:
Objects are preferred over primitives.
Empty collections are preferred over null.
Objects with no value are represented as null.
Primitives return their value.
If you are returning a JSON object with mostly null values, you may have a candidate for refactoring into multiple services.
{
"value1": null,
"value2": null,
"text1": null,
"text2": "hello",
"intValue": 0, //use primitive only if you are absolutely sure the answer is 0
"myList": [],
"myEmptyList": null, //NOT BEST PRACTICE - return [] instead
"boolean1": null, //use primitive only if you are absolutely sure the answer is true/false
"littleboolean": false
}
The above JSON was generated from the following Java class.
package jackson;
import java.util.ArrayList;
import java.util.List;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JacksonApp {
public static class Data {
public Integer value1;
public Integer value2;
public String text1;
public String text2 = "hello";
public int intValue;
public List<Object> myList = new ArrayList<Object>();
public List<Object> myEmptyList;
public Boolean boolean1;
public boolean littleboolean;
}
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
System.out.println(mapper.writeValueAsString(new Data()));
}
}
Maven dependency:
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.3.0</version>
</dependency>
Let's evaluate the parsing of each:
http://jsfiddle.net/brandonscript/Y2dGv/
var json1 = '{}';
var json2 = '{"myCount": null}';
var json3 = '{"myCount": 0}';
var json4 = '{"myString": ""}';
var json5 = '{"myString": "null"}';
var json6 = '{"myArray": []}';
console.log(JSON.parse(json1)); // {}
console.log(JSON.parse(json2)); // {myCount: null}
console.log(JSON.parse(json3)); // {myCount: 0}
console.log(JSON.parse(json4)); // {myString: ""}
console.log(JSON.parse(json5)); // {myString: "null"}
console.log(JSON.parse(json6)); // {myArray: []}
The tl;dr here:
The fragment in the json2 variable is the way the JSON spec indicates null should be represented. But as always, it depends on what you're doing -- sometimes the "right" way to do it doesn't always work for your situation. Use your judgement and make an informed decision.
JSON1 {}
This returns an empty object. There is no data there, and it's only going to tell you that whatever key you're looking for (be it myCount or something else) is of type undefined.
JSON2 {"myCount": null}
In this case, myCount is actually defined, albeit its value is null. This is not the same as both "not undefined and not null", and if you were testing for one condition or the other, this might succeed whereas JSON1 would fail.
This is the definitive way to represent null per the JSON spec.
JSON3 {"myCount": 0}
In this case, myCount is 0. That's not the same as null, and it's not the same as false. If your conditional statement evaluates myCount > 0, then this might be worthwhile to have. Moreover, if you're running calculations based on the value here, 0 could be useful. If you're trying to test for null however, this is actually not going to work at all.
JSON4 {"myString": ""}
In this case, you're getting an empty string. Again, as with JSON2, it's defined, but it's empty. You could test for if (obj.myString == "") but you could not test for null or undefined.
JSON5 {"myString": "null"}
This is probably going to get you in trouble, because you're setting the string value to null; in this case, obj.myString == "null" however it is not == null.
JSON6 {"myArray": []}
This will tell you that your array myArray exists, but it's empty. This is useful if you're trying to perform a count or evaluation on myArray. For instance, say you wanted to evaluate the number of photos a user posted - you could do myArray.length and it would return 0: defined, but no photos posted.
null is not zero. It is not a value, per se: it is a value outside the domain of the variable indicating missing or unknown data.
There is only one way to represent null in JSON. Per the specs (RFC 4627 and json.org):
2.1. Values
A JSON value MUST be an object, array, number, or string, or one of
the following three literal names:
false null true
There is only one way to represent null; that is with null.
console.log(null === null); // true
console.log(null === true); // false
console.log(null === false); // false
console.log(null === 'null'); // false
console.log(null === "null"); // false
console.log(null === ""); // false
console.log(null === []); // false
console.log(null === 0); // false
That is to say; if any of the clients that consume your JSON representation use the === operator; it could be a problem for them.
no value
If you want to convey that you have an object whose attribute myCount has no value:
{ "myCount": null }
no attribute / missing attribute
What if you to convey that you have an object with no attributes:
{}
Client code will try to access myCount and get undefined; it's not there.
empty collection
What if you to convey that you have an object with an attribute myCount that is an empty list:
{ "myCount": [] }
I would use null to show that there is no value for that particular key. For example, use null to represent that "number of devices in your household connects to internet" is unknown.
On the other hand, use {} if that particular key is not applicable. For example, you should not show a count, even if null, to the question "number of cars that has active internet connection" is asked to someone who does not own any cars.
I would avoid defaulting any value unless that default makes sense. While you may decide to use null to represent no value, certainly never use "null" to do so.
I would pick "default" for data type of variable (null for strings/objects, 0 for numbers), but indeed check what code that will consume the object expects. Don't forget there there is sometimes distinction between null/default vs. "not present".
Check out null object pattern - sometimes it is better to pass some special object instead of null (i.e. [] array instead of null for arrays or "" for strings).
According to the JSON spec, the outermost container does not have to be a dictionary (or 'object') as implied in most of the comments above. It can also be a list or a bare value (i.e. string, number, boolean or null). If you want to represent a null value in JSON, the entire JSON string (excluding the quotes containing the JSON string) is simply null. No braces, no brackets, no quotes. You could specify a dictionary containing a key with a null value ({"key1":null}), or a list with a null value ([null]), but these are not null values themselves - they are proper dictionaries and lists. Similarly, an empty dictionary ({}) or an empty list ([]) are perfectly fine, but aren't null either.
In Python:
>>> print json.loads('{"key1":null}')
{u'key1': None}
>>> print json.loads('[null]')
[None]
>>> print json.loads('[]')
[]
>>> print json.loads('{}')
{}
>>> print json.loads('null')
None
This is a personal and situational choice. The important thing to remember is that the empty string and the number zero are conceptually distinct from null.
In the case of a count you probably always want some valid number (unless the count is unknown or undefined), but in the case of strings, who knows? The empty string could mean something in your application. Or maybe it doesn't. That's up to you to decide.
'null' is best for practical use
FWIW, using PHP as an example, PHP interprets empty sets as entries made by PHP...
// This loop will iterate one item with the value 'the car'
$empty_json = '["the car"]';
$some_json_array = json_decode($empty_json);
foreach ($some_json_array as $i) {
echo "PHP sees one item";
}
output: PHP sees the car
// This loop will iterate one item, but with no values
$empty_json = '[""]';
$some_json_array = json_decode($empty_json);
foreach ($some_json_array as $i) {
echo "PHP sees: $i";
}
output: PHP sees
// This loop will iterate nothing because PHP's `json_decode()` function knew it was `null`
$empty_json = 'null';
$some_json_array = json_decode($empty_json);
foreach ($some_json_array as $i) {
echo "PHP sees one item";
}
output: (nothing, foreach will not loop)