I do have this struct for my request
type RequestBody struct {
ForkliftID string `json:"forklift_id"`
WarehouseID string `json:"warehouse_id,omitempty"`
TaskID string `json:"task_id"`
}
If I don't send the "forklift_id" in the json request body, the unmarshalling assigns "" without returning error, I wonder if there is something like
`json: "forklift_id,notempty"`
so the unmarshalling returns an error if the field is not present or empty.
Thanks in advance
I assume what you did (treating empty value from payload as an error) is for validation purposes. If so, I think you can go with #colminator answer, or try to use 3rd party library that designed to solve this particular problem. One example library is https://github.com/go-playground/validator.
type RequestBody struct {
ForkliftID string `json:"forklift_id" validate:"required"`
WarehouseID string `json:"warehouse_id" validate:"required"`
TaskID string `json:"task_id"`
}
// ...
var payload RequestBody
// ...
validate := validator.New()
err := validate.Struct(payload)
if err != nil {
// handle the error
}
The field with tag validate:"required" will be validated when validate.Struct() called.
There are also a lot of useful validation rules available other than required.
For a more detailed example, take a look at the example source code
Another alternative solution would be, by performing explicit checks across those struct's fields. Example:
// ...
if payload.ForkliftID == "" {
err = fmt.Errorf("Forklift ID cannot be empty")
}
if payload.WarehouseID == "" {
err = fmt.Errorf("Warehouse ID cannot be empty")
}
Change the field type to *String. After unmarshaling, if the value is nil, then you know the JSON field was not provided.
See here for a more thorough example of unmarshaling error-checking.
Related
I am working on a general JSON based message passing protocol in Go. What I would like to do is have a BaseMessage that has general information like Type, timestamp, etc. But at the same time I want to be able to define more specific message structures for certain types of data.
For example:
type Message struct {
Type string `json:type`
Timestamp string `json:timestamp`
}
type EventMessage struct {
Message
EventType string
EventCreator string
EventData interface{}
}
I have a set of handlers and to determine which handler should process the message I decode the JSON to the general Message type first to check the Type field. For this example I would get the handler associated with an "Event" message type.
I run into problems when I then want to assert the EventMessage type onto the structure.
The following code is very rough, but hopefully it displays my general idea of how I am trying to handle the messages.
type Handler func(msg Message) Message
handlers := make(map[string]Handler)
var msg Message
decoder.Decode(&msg)
handler := handlers[msg.Type]
handler(msg)
I have tried to use an interface{} but then the JSON decoder just creates a map which I then cant assert either type on. I have figured out workarounds that make it possible, but its very ugly, probably not efficient, and most likely error prone. I would like to keep things simple and straightforward so this code can be easily maintained.
Is there a method of handling generic JSON objects in Go so that the decoded JSON could be one of many struct formats?
I have also played with the idea of having more specific info in a Data interface{} in the main Message struct, but then I run into the same problem of not being able to assert any types onto the interface. There has to be a better way to handle JSON formats that I am just missing.
One way to handle this is to define a struct for the fixed part of the message with a json.RawMessage field to capture the variant part of the message. Decode the json.RawMessage to a type specific to the variant:
type Message struct {
Type string `json:"type"`
Timestamp string `json:"timestamp"`
Data json.RawMessage
}
type Event struct {
Type string `json:"type"`
Creator string `json:"creator"`
}
var m Message
if err := json.Unmarshal(data, &m); err != nil {
log.Fatal(err)
}
switch m.Type {
case "event":
var e Event
if err := json.Unmarshal([]byte(m.Data), &e); err != nil {
log.Fatal(err)
}
fmt.Println(m.Type, e.Type, e.Creator)
default:
log.Fatal("bad message type")
}
playground example
I am trying to retrieve information using Reddit's API. Here is some documentation on their json response, however, I got most of my information by just viewing the link in the browser and pretty-printing the response here.
The following code behaves as intended when the "Replies" field is commented out, but fails when it's not.
[edit] getData() is a function I wrote that uses Go's http Client to get a site response in bytes.
type redditThing struct {
Data struct {
Children []struct {
Data struct {
Permalink string
Subreddit string
Title string
Body string
Replies redditThing
}
}
}
}
func visitLink(link string) {
println("visiting:", link)
var comments []redditThing
if err := json.Unmarshal(getData(link+".json?raw_json=1"), &comments); err != nil {
logError.Println(err)
return
}
}
This throws the following error
json: cannot unmarshal string into Go struct field .Data.Children.Data.Replies.Data.Children.Data.Replies.Data.Children.Data.Replies of type main.redditThing
Any help would be greatly appreciated. Thank you all in advance!
[edit] here a link to some data causing the program to fail
The replies field can be the empty string or a redditThing. Fix by adding an Unmarshal function to handle the empty string:
func (rt *redditThing) UnmarshalJSON(data []byte) error {
// Do nothing if data is the empty string.
if bytes.Equal(data, []byte(`""`)) {
return nil
}
// Prevent recursion by declaring type x with
// same underlying type as redditThing, but
// with no methods.
type x redditThing
return json.Unmarshal(data, (*x)(rt))
}
The x type is used to prevent indefinite recursion. If the final line of the method is json.Unmarshal(data, rt), then json.Unmarshal function will call redditThing.UnmarshalJSON method which calls json.Unmarshal function and so on. Boom!
The statement type x redditThing declares a new type named x with the same underlying type as redditThing. The underlying type is a anonymous struct type. The underlying type has no methods, and crucially, the underlying type does not have the UnmarshalJSON method. This prevents recursion.
I have a struct VideoInfo that has a key in it called embedCode. The API I am querying returns the embed code as embed_code. During unmarshalling the response how do I ensure embed_code goes into embedCode?
Also is there an easy way to take a large json string and automatically turn it into a struct, or can one only use a map?
With respect to remapping the field names use the corresponding annotation in the structure declaration:
type VideoInfo struct {
EmbedCode string `json:"embed_code"`
}
The marshaller/un-marshaller will only process public field, so you need to capitalise the field name.
With respect to converting the whole structure, yes it is easy. Declare an instance to un-marshal into and pass a reference to the json.Unmarshal method (from a test):
data, _ := json.Marshal(request)
var resp response.VideoInfo
if err := json.Unmarshal(data, &resp); err != nil {
t.Errorf("unexpected error, %v", err)
}
At first, struct's field must be start from capital letter to be public. So you need something like that:
type VideoInfo struct {
EmbedCode string `json:"embed_code"`
}
And look at documentation for more info.
I am working on a general JSON based message passing protocol in Go. What I would like to do is have a BaseMessage that has general information like Type, timestamp, etc. But at the same time I want to be able to define more specific message structures for certain types of data.
For example:
type Message struct {
Type string `json:type`
Timestamp string `json:timestamp`
}
type EventMessage struct {
Message
EventType string
EventCreator string
EventData interface{}
}
I have a set of handlers and to determine which handler should process the message I decode the JSON to the general Message type first to check the Type field. For this example I would get the handler associated with an "Event" message type.
I run into problems when I then want to assert the EventMessage type onto the structure.
The following code is very rough, but hopefully it displays my general idea of how I am trying to handle the messages.
type Handler func(msg Message) Message
handlers := make(map[string]Handler)
var msg Message
decoder.Decode(&msg)
handler := handlers[msg.Type]
handler(msg)
I have tried to use an interface{} but then the JSON decoder just creates a map which I then cant assert either type on. I have figured out workarounds that make it possible, but its very ugly, probably not efficient, and most likely error prone. I would like to keep things simple and straightforward so this code can be easily maintained.
Is there a method of handling generic JSON objects in Go so that the decoded JSON could be one of many struct formats?
I have also played with the idea of having more specific info in a Data interface{} in the main Message struct, but then I run into the same problem of not being able to assert any types onto the interface. There has to be a better way to handle JSON formats that I am just missing.
One way to handle this is to define a struct for the fixed part of the message with a json.RawMessage field to capture the variant part of the message. Decode the json.RawMessage to a type specific to the variant:
type Message struct {
Type string `json:"type"`
Timestamp string `json:"timestamp"`
Data json.RawMessage
}
type Event struct {
Type string `json:"type"`
Creator string `json:"creator"`
}
var m Message
if err := json.Unmarshal(data, &m); err != nil {
log.Fatal(err)
}
switch m.Type {
case "event":
var e Event
if err := json.Unmarshal([]byte(m.Data), &e); err != nil {
log.Fatal(err)
}
fmt.Println(m.Type, e.Type, e.Creator)
default:
log.Fatal("bad message type")
}
playground example
I have a struct like this:
type Result struct {
Data MyStruct `json:"data,omitempty"`
Status string `json:"status,omitempty"`
Reason string `json:"reason,omitempty"`
}
But even if the instance of MyStruct is entirely empty (meaning, all values are default), it's being serialized as:
"data":{}
I know that the encoding/json docs specify that "empty" fields are:
false, 0, any nil pointer or interface value, and any array,
slice, map, or string of length zero
but with no consideration for a struct with all empty/default values. All of its fields are also tagged with omitempty, but this has no effect.
How can I get the JSON package to not marshal my field that is an empty struct?
As the docs say, "any nil pointer." -- make the struct a pointer. Pointers have obvious "empty" values: nil.
Fix - define the type with a struct pointer field:
type Result struct {
Data *MyStruct `json:"data,omitempty"`
Status string `json:"status,omitempty"`
Reason string `json:"reason,omitempty"`
}
Then a value like this:
result := Result{}
Will marshal as:
{}
Explanation: Notice the *MyStruct in our type definition. JSON serialization doesn't care whether it is a pointer or not -- that's a runtime detail. So making struct fields into pointers only has implications for compiling and runtime).
Just note that if you do change the field type from MyStruct to *MyStruct, you will need pointers to struct values to populate it, like so:
Data: &MyStruct{ /* values */ }
As #chakrit mentioned in a comment, you can't get this to work by implementing json.Marshaler on MyStruct, and implementing a custom JSON marshalling function on every struct that uses it can be a lot more work. It really depends on your use case as to whether it's worth the extra work or whether you're prepared to live with empty structs in your JSON, but here's the pattern I use applied to Result:
type Result struct {
Data MyStruct
Status string
Reason string
}
func (r Result) MarshalJSON() ([]byte, error) {
return json.Marshal(struct {
Data *MyStruct `json:"data,omitempty"`
Status string `json:"status,omitempty"`
Reason string `json:"reason,omitempty"`
}{
Data: &r.Data,
Status: r.Status,
Reason: r.Reason,
})
}
func (r *Result) UnmarshalJSON(b []byte) error {
decoded := new(struct {
Data *MyStruct `json:"data,omitempty"`
Status string `json:"status,omitempty"`
Reason string `json:"reason,omitempty"`
})
err := json.Unmarshal(b, decoded)
if err == nil {
r.Data = decoded.Data
r.Status = decoded.Status
r.Reason = decoded.Reason
}
return err
}
If you have huge structs with many fields this can become tedious, especially changing a struct's implementation later, but short of rewriting the whole json package to suit your needs (not a good idea), this is pretty much the only way I can think of getting this done while still keeping a non-pointer MyStruct in there.
Also, you don't have to use inline structs, you can create named ones. I use LiteIDE with code completion though, so I prefer inline to avoid clutter.
Data is an initialized struct, so it isn't considered empty because encoding/json only looks at the immediate value, not the fields inside the struct.
Unfortunately, returning nil from json.Marshaler doesn't currently work:
func (_ MyStruct) MarshalJSON() ([]byte, error) {
if empty {
return nil, nil // unexpected end of JSON input
}
// ...
}
You could give Result a marshaler as well, but it's not worth the effort.
The only option, as Matt suggests, is to make Data a pointer and set the value to nil.
There is an outstanding Golang proposal for this feature which has been active for over 4 years, so at this point, it is safe to assume that it will not make it into the standard library anytime soon. As #Matt pointed out, the traditional approach is to convert the structs to pointers-to-structs. If this approach is infeasible (or impractical), then an alternative is to use an alternate json encoder which does support omitting zero value structs.
I created a mirror of the Golang json library (clarketm/json) with added support for omitting zero value structs when the omitempty tag is applied. This library detects zeroness in a similar manner to the popular YAML encoder go-yaml by recursively checking the public struct fields.
e.g.
$ go get -u "github.com/clarketm/json"
import (
"fmt"
"github.com/clarketm/json" // drop-in replacement for `encoding/json`
)
type Result struct {
Data MyStruct `json:"data,omitempty"`
Status string `json:"status,omitempty"`
Reason string `json:"reason,omitempty"`
}
j, _ := json.Marshal(&Result{
Status: "204",
Reason: "No Content",
})
fmt.Println(string(j))
// Note: `data` is omitted from the resultant json.
{
"status": "204"
"reason": "No Content"
}