How to get JSON when decoder fails to unmarshal? - json

Question

I am using a json.Decoder to decode JSON delivered over a network stream. It works fine, but whenever someone sends data that doesn't fit the schema (e.g. sending a negative integer when the struct's field type is unsigned) it returns an error with a vague message that doesn't pinpoint the offending property. This makes debugging more difficult.
Is there any way to extract the JSON that the decoder was trying to unmarshal when it errored? Here's a small reproducable snippet:
package main
import (
"bytes"
"fmt"
"encoding/json"
)
func main() {
buff := bytes.NewBufferString("{\"bar\": -123}")
decoder := json.NewDecoder(buff)
var foo struct{
Bar uint32
}
if err := decoder.Decode(&foo); err != nil {
fmt.Println(err)
fmt.Println("TODO: how to get JSON that caused this error?")
} else {
fmt.Println(foo.Bar)
}
}
Or on playground: https://play.golang.org/p/-WdYBkYEzJ

Answer 1

Some information is in the error, which is of type *json.UnamrshalTypeError
type UnmarshalTypeError struct {
Value string // description of JSON value - "bool", "array", "number -5"
Type reflect.Type // type of Go value it could not be assigned to
Offset int64 // error occurred after reading Offset bytes
}
You can get the offset in the json string, the reflect.Type of the field being unmarshaled into, and the json description of the Value. This can still pose a problem for types that implement their own unmarshaling logic, which is referenced by issue 11858

Answer 2

As of Go 1.8 this is now possible. The UnmarshalTypeError type now contains Struct and Field values which provide the name of the struct and field which caused a type mismatch.
package main
import (
"bytes"
"fmt"
"encoding/json"
)
func main() {
buff := bytes.NewBufferString("{\"bar\": -123}")
decoder := json.NewDecoder(buff)
var foo struct{
Bar uint32
}
if err := decoder.Decode(&foo); err != nil {
if terr, ok := err.(*json.UnmarshalTypeError); ok {
fmt.Printf("Failed to unmarshal field %s \n", terr.Field)
} else {
fmt.Println(err)
}
} else {
fmt.Println(foo.Bar)
}
}
The error message string also was changed to contain this new information.
Go 1.8:
json: cannot unmarshal number -123 into Go struct field .Bar of type uint32
Go 1.7 and earlier:
json: cannot unmarshal number -123 into Go value of type uint32

Answer 3

You can get at each element, key, value, and even delimiters using decode.Token() such as this in the playground, and below (modified from your example):
package main
import (
"bytes"
"encoding/json"
"fmt"
)
func main() {
buff := bytes.NewBufferString(`{"foo": 123, "bar": -123, "baz": "123"}`)
decoder := json.NewDecoder(buff)
for {
t, err := decoder.Token()
if _, ok := t.(json.Delim); ok {
continue
}
fmt.Printf("type:%11T | value:%5v //", t, t)
switch t.(type) {
case uint32:
fmt.Println("you don't see any uints")
case int:
fmt.Println("you don't see any ints")
case string:
fmt.Println("handle strings as you will")
case float64:
fmt.Println("handle numbers as you will")
}
if !decoder.More() {
break
}
if err != nil {
fmt.Println(err)
}
}
}
This will output
type: string | value: foo //handle strings as you will
type: float64 | value: 123 //handle numbers as you will
type: string | value: bar //handle strings as you will
type: float64 | value: -123 //handle numbers as you will
type: string | value: baz //handle strings as you will
type: string | value: 123 //handle strings as you will
You can switch on the type and handle each one as you wish. I have shown a simple example of that as well, each of the "//comments" in the result are conditional based on the type.
You'll also notice that each numbers' type is float64, although they would fit into an int, or even uint in the case of the "foo" value, and I check for those types in the switch but they never get used. You would have to provide your own logic in order to convert the float64 values to the types you wanted if they could be and handle types that wouldn't convert as special cases, or errors or whatever you wanted.