If I have some JSON data like this:
{
nullableID: ""
}
How can I get unmarshalling this struct:
help := struct {
ID *primitive.ObjectID `json:"nullableID",omitempty`
}{}
To decode into help such that help.ID == nil
Have ObjectID implement the Unmarshaler interface and check for an empty string:
func (o *ObjectID) UnmarshalJSON(data []byte) error {
if err := json.Unmarshal(data, o); err != nil {
return err
}
if string(*o) == "" {
o = nil
}
return nil
}
If the ObjectID type is imported from another package, you can create a new type that wraps that type:
// objID is a copy of primitive.ObjectID but with it's own json unmarshalling.
type objID struct {
*primitive.ObjectID
}
func (o *objID) UnmarshalJSON(data []byte) error {
// Same implementation as above
}
This article explains it in a lot more detail.
Related
I have a json response like this
{
"foo" : "bar",
"object" : {
"type" : "action",
"data" : "somedata"
}
}
Here the object could be one of multiple types. I define the types and have them implement a common interface.
type IObject interface {
GetType() string
}
type Action struct {
Type string `json:"type"`
Data string `json:"data"`
}
func (a Action) GetType() string {
return "action"
}
type Activity struct {
Type string `json:"type"`
Duration int `json:"duration"`
}
func (a Activity) GetType() string {
return "activity"
}
And a response struct
type Response struct {
Foo string `json:"foo"`
Object IObject `json:"object"`
}
As the type information of a struct that implements IObject is contained within the struct, there is no way to learn in without unmarshaling. I also cannot change the structure of the json response. Currently I am dealing with this problem using a custom unmarshaller:
func UnmarshalObject(m map[string]interface{}, object *IObject) error {
if m["type"] == "action" {
b, err := json.Marshal(m)
if err != nil {
return err
}
action := Action{}
if err = json.Unmarshal(b, &action); err != nil {
return err
}
*object = action
return nil
}
if m["type"] == "activity" {
b, err := json.Marshal(m)
if err != nil {
return err
}
activity := Activity{}
if err = json.Unmarshal(b, &activity); err != nil {
return err
}
*object = activity
return nil
}
return errors.New("unknown actor type")
}
func (r *Response) UnmarshalJSON(data []byte) error {
raw := struct {
Foo string `json:"foo"`
Object interface{} `json:"object"`
}{}
err := json.Unmarshal(data, &raw)
if err != nil {
return err
}
r.Foo = raw.Foo
if err = UnmarshalObject(raw.Object.(map[string]interface{}), &r.Object); err != nil
{
return err
}
return nil
}
So what I do is basically
Unmarshall the object into an interface{}
Typecast to map[string]interface{}
Read the "type" value to determine the type
Create a new instance of the determined type
Marshal back to json
Unmarshal again to the new instance of the determined type
Assign the instance to the field
This feels off and I am not comfortable with it. Especially the marshaling/unmarshaling back and forth. Is there a more elegant way to solve this problem?
You can use json.RawMessage.
func (r *Response) UnmarshalJSON(data []byte) error {
var raw struct {
Foo string `json:"foo"`
Object json.RawMessage `json:"object"`
}
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
r.Foo = raw.Foo
var obj struct {
Type string `json:"type"`
}
if err := json.Unmarshal(raw.Object, &obj); err != nil {
return err
}
switch obj.Type {
case "action":
r.Object = new(Action)
case "activity":
r.Object = new(Activity)
}
return json.Unmarshal(raw.Object, r.Object)
}
https://go.dev/play/p/6dqiybS4zNp
I get a response from an external API that has a field which can have 2 values:
{"field": []}
or
{"field": {"key1": "value", "key2": "value"}}
I set the struct to be
type Object Struct {
Field map[string]string `json:"field,omitempty"`
}
And then call my own implemented function to decode the response
func decode(response *http.Response) (*Object, error) {
var response Object
err := json.NewDecoder(response.Body).Decode(&response)
if err != nil {
return nil, err
}
return &response, nil
}
But this works only for the second response ( when field not empty is). For the first response I get an error.
you can do a custom marshaler type for the Field. Example:
type keys struct {
Key1 string
Key2 string
}
type mytype struct {
EmptySlice bool
Keys *keys
}
func (m *mytype) UnmarshalJSON(b []byte) error {
if bytes.Equal(b, []byte("[]")) {
m.EmptySlice = true
return nil
}
m.Keys = &keys{}
return json.Unmarshal(b, &m.Keys)
}
type Object struct {
Field mytype `json:"field"`
}
func main() {
input := []string{
`{"field": []}`,
`{"field": {"key1": "value", "key2": "value"}}`,
}
for i, s := range input {
var o Object
err := json.Unmarshal([]byte(s), &o)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%d: %+v\n", i+1, o)
}
}
https://go.dev/play/p/OqSKfUXHFyb
You can use a custom type and implement the UnmarshalJSON interface method on that type.
For example:
type Field struct {
arr []string
m map[string]string
}
func (f *Field) UnmarshalJSON(b []byte) error {
var m map[string]string
err := json.Unmarshal(b, &m)
if err == nil {
f.m = m
return nil
}
var arr []string
err = json.Unmarshal(b, &arr)
if err == nil {
f.arr = arr
return nil
}
return fmt.Errorf("type of property not array or map")
}
https://go.dev/play/p/JuFE--hWAjw
I read lots of similar questions but not able to find the right solution yet. Hope someone can give me better idea.
My error response could be one of these format:
var errProxy = `{"errors":{"id": "1", "message": "failed to resolve the ip", "status": "failed"}}`
var errServer = `{"errors": "failed to ping the dns server."}`
I am trying to solve this by using two structs and trying to unmarshall one after another until err is nil and return:
// ErrorServer and ErrorProxy is my two struct to represent my above two error response.
type ErrorServer struct {
Error string `json:"errors"`
}
type ErrorResponse struct {
ID string `json:"id"`
Message string `json:"message"`
Status string `json:"status"`
}
type ErrorProxy struct {
ErrorResponse
}
This is the function I used to parse it:
func parseErrorResponse(body io.ReadCloser) (string, error) {
var errServer ErrorServer
var errProxy ErrorProxy
err := json.NewDecoder(body).Decode(&errServer)
if err != nil {
err = json.NewDecoder(body).Decode(&errProxy)
if err != nil {
return "", err
}
return errProxy.Error, nil
}
return errServer.Errors, nil
}
I think a custom unmarshal function is good for this case
type ErrorServer struct {
Error string
}
type ErrorResponse struct {
ID string
Message string
Status string
}
type ErrorProxy struct {
Err ErrorResponse
}
func parseErrorResponse(body io.Reader) (interface{}, error) {
data := make(map[string]interface{})
if err := json.NewDecoder(body).Decode(&data); err != nil {
return nil, err
}
// Check type of errors field
switch errors := data["errors"].(type) {
case string:
// Unmarshal for ErrorServer
errServer := &ErrorServer{
Error: errors,
}
return errServer, nil
case map[string]interface{}:
// Unmarshal for ErrorProxy
errProxy := &ErrorProxy{
Err: ErrorResponse{
ID: errors["id"].(string),
Message: errors["message"].(string),
Status: errors["status"].(string),
},
}
return errProxy, nil
default:
return nil, fmt.Errorf(`failed to parse "errors" field`)
}
}
func main() {
body := bytes.NewReader([]byte(`{"errors": "failed to ping the dns server."}`))
//body := bytes.NewReader([]byte(`{"errors":{"id": "1", "message": "failed to resolve the ip", "status": "failed"}}`))
parsedErr, _ := parseErrorResponse(body)
switch err := parsedErr.(type) {
case *ErrorServer:
fmt.Printf("err server: %+v \n", err)
case *ErrorProxy:
fmt.Printf("err response: %+v \n", err)
}
}
Another way to do that is using type assertion. You can unmarshal that json string into map[string]interface{} and check whether that value interface{} is a string or map[string]interface{}. Depends on its type, you know which kind of error it is and construct a struct from that.
Funny enough, I just gave the answer to this problem here: Use a customized UnmarshalJSON function. How do you modify this struct in Golang to accept two different results?
Applied to your situation:
package main
import (
"encoding/json"
"fmt"
"log"
)
type Answer struct {
Errors Error `json:"errors"`
}
type ErrorResponse struct {
ID string `json:"id"`
Message string `json:"message"`
Status string `json:"status"`
}
type Error struct {
Error string
ErrorResponse ErrorResponse
}
func (s *Error) UnmarshalJSON(b []byte) error {
if len(b) == 0 {
// no data, nothing to do
return nil
}
if b[0] == '{' {
// is object
return json.Unmarshal(b, &s.ErrorResponse)
}
return json.Unmarshal(b, &s.Error)
}
func main() {
var errProxy = []byte(`{"errors":{"id": "1", "message": "failed to resolve the ip", "status": "failed"}}`)
var errServer = []byte(`{"errors": "failed to ping the dns server."}`)
var answ Answer
if err := json.Unmarshal(errProxy, &answ); err != nil {
log.Fatal(err)
}
fmt.Println(answ)
answ = Answer{}
if err := json.Unmarshal(errServer, &answ); err != nil {
log.Fatal(err)
}
fmt.Println(answ)
}
Go PlayGround
Key in the above example is the type that contains both variations. We can also simplify that as both errors could be contained within the ErrorResponse type:
type Answer struct {
Errors ErrorResponse `json:"errors"`
}
type ErrorResponse struct {
ID string
Message string
Status string
}
func (s *ErrorResponse) UnmarshalJSON(b []byte) error {
if len(b) == 0 {
// no data, nothing to do
return nil
}
if b[0] == '{' {
// is object
var tmp struct {
ID string `json:"id"`
Message string `json:"message"`
Status string `json:"status"`
}
if err := json.Unmarshal(b, &tmp); err != nil {
return err
}
s.ID = tmp.ID
s.Message = tmp.Message
s.Status = tmp.Status
return nil
}
return json.Unmarshal(b, &s.Message)
}
The Error struct is not needed here.
Is it possible to generate an error if a field was not found while parsing a JSON input using Go?
I could not find it in documentation.
Is there any tag that specifies the field as required?
There is no tag in the encoding/json package that sets a field to "required". You will either have to write your own MarshalJSON() method, or do a post check for missing fields.
To check for missing fields, you will have to use pointers in order to distinguish between missing/null and zero values:
type JsonStruct struct {
String *string
Number *float64
}
Full working example:
package main
import (
"fmt"
"encoding/json"
)
type JsonStruct struct {
String *string
Number *float64
}
var rawJson = []byte(`{
"string":"We do not provide a number"
}`)
func main() {
var s *JsonStruct
err := json.Unmarshal(rawJson, &s)
if err != nil {
panic(err)
}
if s.String == nil {
panic("String is missing or null!")
}
if s.Number == nil {
panic("Number is missing or null!")
}
fmt.Printf("String: %s Number: %f\n", *s.String, *s.Number)
}
Playground
You can also override the unmarshalling for a specific type (so a required field buried in a few json layers) without having to make the field a pointer. UnmarshalJSON is defined by the Unmarshaler interface.
type EnumItem struct {
Named
Value string
}
func (item *EnumItem) UnmarshalJSON(data []byte) (err error) {
required := struct {
Value *string `json:"value"`
}{}
all := struct {
Named
Value string `json:"value"`
}{}
err = json.Unmarshal(data, &required)
if err != nil {
return
} else if required.Value == nil {
err = fmt.Errorf("Required field for EnumItem missing")
} else {
err = json.Unmarshal(data, &all)
item.Named = all.Named
item.Value = all.Value
}
return
}
Here is another way by checking your customized tag
you can create a tag for your struct like:
type Profile struct {
Name string `yourprojectname:"required"`
Age int
}
Use reflect to check if the tag is assigned required value
func (p *Profile) Unmarshal(data []byte) error {
err := json.Unmarshal(data, p)
if err != nil {
return err
}
fields := reflect.ValueOf(p).Elem()
for i := 0; i < fields.NumField(); i++ {
yourpojectTags := fields.Type().Field(i).Tag.Get("yourprojectname")
if strings.Contains(yourpojectTags, "required") && fields.Field(i).IsZero() {
return errors.New("required field is missing")
}
}
return nil
}
And test cases are like:
func main() {
profile1 := `{"Name":"foo", "Age":20}`
profile2 := `{"Name":"", "Age":21}`
var profile Profile
err := profile.Unmarshal([]byte(profile1))
if err != nil {
log.Printf("profile1 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile1 unmarshal: %v\n", profile)
err = profile.Unmarshal([]byte(profile2))
if err != nil {
log.Printf("profile2 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile2 unmarshal: %v\n", profile)
}
Result:
profile1 unmarshal: {foo 20}
2009/11/10 23:00:00 profile2 unmarshal error: required field is missing
You can go to Playground to have a look at the completed code
You can just implement the Unmarshaler interface to customize how your JSON gets unmarshalled.
you can also make use of JSON schema validation.
package main
import (
"encoding/json"
"fmt"
"github.com/alecthomas/jsonschema"
"github.com/xeipuuv/gojsonschema"
)
type Bird struct {
Species string `json:"birdType"`
Description string `json:"what it does" jsonschema:"required"`
}
func main() {
var bird Bird
sc := jsonschema.Reflect(&bird)
b, _ := json.Marshal(sc)
fmt.Println(string(b))
loader := gojsonschema.NewStringLoader(string(b))
documentLoader := gojsonschema.NewStringLoader(`{"birdType": "pigeon"}`)
schema, err := gojsonschema.NewSchema(loader)
if err != nil {
panic("nop")
}
result, err := schema.Validate(documentLoader)
if err != nil {
panic("nop")
}
if result.Valid() {
fmt.Printf("The document is valid\n")
} else {
fmt.Printf("The document is not valid. see errors :\n")
for _, err := range result.Errors() {
// Err implements the ResultError interface
fmt.Printf("- %s\n", err)
}
}
}
Outputs
{"$schema":"http://json-schema.org/draft-04/schema#","$ref":"#/definitions/Bird","definitions":{"Bird":{"required":["birdType","what it does"],"properties":{"birdType":{"type":"string"},"what it does":{"type":"string"}},"additionalProperties":false,"type":"object"}}}
The document is not valid. see errors :
- (root): what it does is required
code example taken from Strict JSON parsing
Is it possible to generate an error if a field was not found while parsing a JSON input using Go?
I could not find it in documentation.
Is there any tag that specifies the field as required?
There is no tag in the encoding/json package that sets a field to "required". You will either have to write your own MarshalJSON() method, or do a post check for missing fields.
To check for missing fields, you will have to use pointers in order to distinguish between missing/null and zero values:
type JsonStruct struct {
String *string
Number *float64
}
Full working example:
package main
import (
"fmt"
"encoding/json"
)
type JsonStruct struct {
String *string
Number *float64
}
var rawJson = []byte(`{
"string":"We do not provide a number"
}`)
func main() {
var s *JsonStruct
err := json.Unmarshal(rawJson, &s)
if err != nil {
panic(err)
}
if s.String == nil {
panic("String is missing or null!")
}
if s.Number == nil {
panic("Number is missing or null!")
}
fmt.Printf("String: %s Number: %f\n", *s.String, *s.Number)
}
Playground
You can also override the unmarshalling for a specific type (so a required field buried in a few json layers) without having to make the field a pointer. UnmarshalJSON is defined by the Unmarshaler interface.
type EnumItem struct {
Named
Value string
}
func (item *EnumItem) UnmarshalJSON(data []byte) (err error) {
required := struct {
Value *string `json:"value"`
}{}
all := struct {
Named
Value string `json:"value"`
}{}
err = json.Unmarshal(data, &required)
if err != nil {
return
} else if required.Value == nil {
err = fmt.Errorf("Required field for EnumItem missing")
} else {
err = json.Unmarshal(data, &all)
item.Named = all.Named
item.Value = all.Value
}
return
}
Here is another way by checking your customized tag
you can create a tag for your struct like:
type Profile struct {
Name string `yourprojectname:"required"`
Age int
}
Use reflect to check if the tag is assigned required value
func (p *Profile) Unmarshal(data []byte) error {
err := json.Unmarshal(data, p)
if err != nil {
return err
}
fields := reflect.ValueOf(p).Elem()
for i := 0; i < fields.NumField(); i++ {
yourpojectTags := fields.Type().Field(i).Tag.Get("yourprojectname")
if strings.Contains(yourpojectTags, "required") && fields.Field(i).IsZero() {
return errors.New("required field is missing")
}
}
return nil
}
And test cases are like:
func main() {
profile1 := `{"Name":"foo", "Age":20}`
profile2 := `{"Name":"", "Age":21}`
var profile Profile
err := profile.Unmarshal([]byte(profile1))
if err != nil {
log.Printf("profile1 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile1 unmarshal: %v\n", profile)
err = profile.Unmarshal([]byte(profile2))
if err != nil {
log.Printf("profile2 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile2 unmarshal: %v\n", profile)
}
Result:
profile1 unmarshal: {foo 20}
2009/11/10 23:00:00 profile2 unmarshal error: required field is missing
You can go to Playground to have a look at the completed code
You can just implement the Unmarshaler interface to customize how your JSON gets unmarshalled.
you can also make use of JSON schema validation.
package main
import (
"encoding/json"
"fmt"
"github.com/alecthomas/jsonschema"
"github.com/xeipuuv/gojsonschema"
)
type Bird struct {
Species string `json:"birdType"`
Description string `json:"what it does" jsonschema:"required"`
}
func main() {
var bird Bird
sc := jsonschema.Reflect(&bird)
b, _ := json.Marshal(sc)
fmt.Println(string(b))
loader := gojsonschema.NewStringLoader(string(b))
documentLoader := gojsonschema.NewStringLoader(`{"birdType": "pigeon"}`)
schema, err := gojsonschema.NewSchema(loader)
if err != nil {
panic("nop")
}
result, err := schema.Validate(documentLoader)
if err != nil {
panic("nop")
}
if result.Valid() {
fmt.Printf("The document is valid\n")
} else {
fmt.Printf("The document is not valid. see errors :\n")
for _, err := range result.Errors() {
// Err implements the ResultError interface
fmt.Printf("- %s\n", err)
}
}
}
Outputs
{"$schema":"http://json-schema.org/draft-04/schema#","$ref":"#/definitions/Bird","definitions":{"Bird":{"required":["birdType","what it does"],"properties":{"birdType":{"type":"string"},"what it does":{"type":"string"}},"additionalProperties":false,"type":"object"}}}
The document is not valid. see errors :
- (root): what it does is required
code example taken from Strict JSON parsing