I have different structures that share a field and I need to decode a JSON file into its corresponding structure in Go.
Example:
type Dog struct {
AnimalType string //will always be "dog"
BarkLoudnessLevel int
}
type Cat struct {
AnimalType string //will always be "cat"
SleepsAtNight bool
}
If I am receiving one of these structures as a JSON string, what would be the most elegant way of parsing it into its proper structure?
So, there are a couple ways of doing this, but the easiest is probably deserializing the payload twice and having conditional branches based off of the "AnimalType" attribute in your payload. Here's a simple example using an intermediate deserialization model:
package main
import (
"fmt"
"encoding/json"
)
type Dog struct {
AnimalType string //will always be "dog"
BarkLoudnessLevel int
}
type Cat struct {
AnimalType string //will always be "cat"
SleepsAtNight bool
}
var (
payloadOne = `{"AnimalType":"dog","BarkLoudnessLevel":1}`
payloadTwo = `{"AnimalType":"cat","SleepsAtNight":false}`
)
func main() {
parseAnimal(payloadOne)
parseAnimal(payloadTwo)
}
func parseAnimal(payload string) {
animal := struct{
AnimalType string
}{}
if err := json.Unmarshal([]byte(payload), &animal); err != nil {
panic(err)
}
switch animal.AnimalType {
case "dog":
dog := Dog{}
if err := json.Unmarshal([]byte(payload), &dog); err != nil {
panic(err)
}
fmt.Printf("Got a dog: %v\n", dog)
case "cat":
cat := Cat{}
if err := json.Unmarshal([]byte(payload), &cat); err != nil {
panic(err)
}
fmt.Printf("Got a cat: %v\n", cat)
default:
fmt.Println("Unknown animal")
}
}
See it in action here.
IMO a better way of approaching this is moving the "metadata" for the payload into a parent structure, though this requires modifying the expected json payload. So, for example, if you were working with payloads that looked like:
{"AnimalType":"dog", "Animal":{"BarkLoudnessLevel": 1}}
Then you could use something like json.RawMessage to partially parse the structure and then conditionally parse the rest as needed (rather than parsing everything twice)--also results in a nicer separation of structure attributes. Here's an example of how you'd do that:
package main
import (
"encoding/json"
"fmt"
)
type Animal struct {
AnimalType string
Animal json.RawMessage
}
type Dog struct {
BarkLoudnessLevel int
}
type Cat struct {
SleepsAtNight bool
}
var (
payloadOne = `{"AnimalType":"dog", "Animal":{"BarkLoudnessLevel": 1}}`
payloadTwo = `{"AnimalType":"cat", "Animal":{"SleepsAtNight": false}}`
)
func main() {
parseAnimal(payloadOne)
parseAnimal(payloadTwo)
}
func parseAnimal(payload string) {
animal := &Animal{}
if err := json.Unmarshal([]byte(payload), &animal); err != nil {
panic(err)
}
switch animal.AnimalType {
case "dog":
dog := Dog{}
if err := json.Unmarshal(animal.Animal, &dog); err != nil {
panic(err)
}
fmt.Printf("Got a dog: %v\n", dog)
case "cat":
cat := Cat{}
if err := json.Unmarshal(animal.Animal, &cat); err != nil {
panic(err)
}
fmt.Printf("Got a cat: %v\n", cat)
default:
fmt.Println("Unknown animal")
}
}
And in action here.
Related
I'm using an API that formats its responses in this way:
{
"err": 0,
"data": **Other json structure**
}
The way I'm getting a response right now is I'm putting the response in an struct like this:
type Response struct {
Err int `json:"err"`
Data interface{} `json:"data"`
}
and then I'm doing this after getting a response
jsonbytes, _ := json.Marshal(resp.Data)
json.Unmarshal(jsonBytes, &dataStruct)
I'm only ignoring errors for this example.
It seems kinda weird to me that I'm marshaling and unmarshaling when I know what the data is supposed to look like and what type it's supposed to be.
Is there a more simple solution that I'm not seeing or is this a normal thing to do?
Edit: I should probably mention that the Data attribute in the response object can vary depending on what API call I'm doing.
The JSON unmarshaller uses reflection to look at the type it is unmarshalling to. Given an uninitialised interface{} as the destination for the unmarshalled data, a JSON object gets unmarshalled into a map[string]interface{} (example in playground).
Here are some ideas.
Option A
If you know the datatype, you can define a new response struct for each type. Example:
type FooResponse struct {
Err int `json:"err"`
Data Foo `json:"data"`
}
type Foo struct {
FooField string `json:"foofield"`
}
type BarResponse struct {
Err int `json:"err"`
Data Bar `json:"data"`
}
type Bar struct {
BarField string `json:"barfield"`
}
Option B
If you prefer to have a single Response struct instead of one per type, you can tell the JSON unmarshaller to avoid unmarshalling the data field until a later time by using the json.RawMessage data type:
package main
import (
"encoding/json"
"fmt"
"log"
)
type Response struct {
Err int `json:"err"`
Data json.RawMessage `json:"data"`
}
type Foo struct {
FooField string `json:"foofield"`
}
type Bar struct {
BarField string `json:"barfield"`
}
func main() {
fooRespJSON := []byte(`{"data":{"foofield":"foo value"}}`)
barRespJSON := []byte(`{"data":{"barfield":"bar value"}}`)
var (
resp Response
foo Foo
bar Bar
)
// Foo
if err := json.Unmarshal(fooRespJSON, &resp); err != nil {
log.Fatal(err)
}
if err := json.Unmarshal(resp.Data, &foo); err != nil {
log.Fatal(err)
}
fmt.Println("foo:", foo)
// Bar
if err := json.Unmarshal(barRespJSON, &resp); err != nil {
log.Fatal(err)
}
if err := json.Unmarshal(resp.Data, &bar); err != nil {
log.Fatal(err)
}
fmt.Println("bar:", bar)
}
Output:
foo: {foo value}
bar: {bar value}
https://play.golang.org/p/Y7D4uhaC4a8
Option C
A third option, as pointed out by #mkopriva in a comment on the question, is to use interface{} as an intermediary datatype and pre-initialise this to a known datatype.
Emphasis lies on the word intermediary -- of course passing around an interface{} is best avoided (Rob Pike's Go Proverbs). The use-case here is to allow any datatype to be used without the need for multiple different Response types. On way to avoid exposing the interface{} is to wrap the response completely, exposing only the data and the error:
package main
import (
"encoding/json"
"fmt"
"log"
)
type Foo struct {
FooField string `json:"foofield"`
}
type Bar struct {
BarField string `json:"barfield"`
}
type Error struct {
Code int
}
func (e *Error) Error() string {
return fmt.Sprintf("error code %d", e.Code)
}
func unmarshalResponse(data []byte, v interface{}) error {
resp := struct {
Err int `json:"err"`
Data interface{} `json:"data"`
}{Data: v}
if err := json.Unmarshal(data, &resp); err != nil {
return err
}
if resp.Err != 0 {
return &Error{Code: resp.Err}
}
return nil
}
func main() {
fooRespJSON := []byte(`{"data":{"foofield":"foo value"}}`)
barRespJSON := []byte(`{"data":{"barfield":"bar value"}}`)
errRespJSON := []byte(`{"err": 123}`)
// Foo
var foo Foo
if err := unmarshalResponse(fooRespJSON, &foo); err != nil {
log.Fatal(err)
}
fmt.Println("foo:", foo)
// Bar
var bar Bar
if err := unmarshalResponse(barRespJSON, &bar); err != nil {
log.Fatal(err)
}
fmt.Println("bar:", bar)
// Error response
var v interface{}
if err := unmarshalResponse(errRespJSON, &v); err != nil {
log.Fatal(err)
}
}
Output:
foo: {foo value}
bar: {bar value}
2009/11/10 23:00:00 error code 123
https://play.golang.org/p/5SVfQGwS-Wy
Let's say I have this JSON structure:
{
"name":"repo",
"tags":["1.0","2.0","3.0"]
}
And I would like to map it to this Go struct:
type Repository struct {
Name string `json:"name"`
Tags []struct {
Tag string `json:"??"`
Sha256 string
}
}
How can I link the "tags" array JSON value to a struct field?
EDIT: The idea will be to access the tags array value like this
repository.Tags[0].Tag.
Implement json.Unmarshaler on a Tag type:
package main
import (
"encoding/json"
"log"
)
type Repository struct {
Name string
Tags []Tag
}
type Tag struct {
Tag string
Sha256 string
}
func (t *Tag) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
t.Tag = s
return nil
}
func main() {
b := []byte(`{ "name":"repo", "tags":["1.0","2.0","3.0"] }`)
var r Repository
err := json.Unmarshal(b, &r)
if err != nil {
log.Fatal(err)
}
log.Printf("%+v\n", r)
}
Try it on the playground: https://play.golang.org/p/ExwWhis0w0V
Marshaling back to JSON is left as an exercise for the reader.
Is it possible to generate an error if a field was not found while parsing a JSON input using Go?
I could not find it in documentation.
Is there any tag that specifies the field as required?
There is no tag in the encoding/json package that sets a field to "required". You will either have to write your own MarshalJSON() method, or do a post check for missing fields.
To check for missing fields, you will have to use pointers in order to distinguish between missing/null and zero values:
type JsonStruct struct {
String *string
Number *float64
}
Full working example:
package main
import (
"fmt"
"encoding/json"
)
type JsonStruct struct {
String *string
Number *float64
}
var rawJson = []byte(`{
"string":"We do not provide a number"
}`)
func main() {
var s *JsonStruct
err := json.Unmarshal(rawJson, &s)
if err != nil {
panic(err)
}
if s.String == nil {
panic("String is missing or null!")
}
if s.Number == nil {
panic("Number is missing or null!")
}
fmt.Printf("String: %s Number: %f\n", *s.String, *s.Number)
}
Playground
You can also override the unmarshalling for a specific type (so a required field buried in a few json layers) without having to make the field a pointer. UnmarshalJSON is defined by the Unmarshaler interface.
type EnumItem struct {
Named
Value string
}
func (item *EnumItem) UnmarshalJSON(data []byte) (err error) {
required := struct {
Value *string `json:"value"`
}{}
all := struct {
Named
Value string `json:"value"`
}{}
err = json.Unmarshal(data, &required)
if err != nil {
return
} else if required.Value == nil {
err = fmt.Errorf("Required field for EnumItem missing")
} else {
err = json.Unmarshal(data, &all)
item.Named = all.Named
item.Value = all.Value
}
return
}
Here is another way by checking your customized tag
you can create a tag for your struct like:
type Profile struct {
Name string `yourprojectname:"required"`
Age int
}
Use reflect to check if the tag is assigned required value
func (p *Profile) Unmarshal(data []byte) error {
err := json.Unmarshal(data, p)
if err != nil {
return err
}
fields := reflect.ValueOf(p).Elem()
for i := 0; i < fields.NumField(); i++ {
yourpojectTags := fields.Type().Field(i).Tag.Get("yourprojectname")
if strings.Contains(yourpojectTags, "required") && fields.Field(i).IsZero() {
return errors.New("required field is missing")
}
}
return nil
}
And test cases are like:
func main() {
profile1 := `{"Name":"foo", "Age":20}`
profile2 := `{"Name":"", "Age":21}`
var profile Profile
err := profile.Unmarshal([]byte(profile1))
if err != nil {
log.Printf("profile1 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile1 unmarshal: %v\n", profile)
err = profile.Unmarshal([]byte(profile2))
if err != nil {
log.Printf("profile2 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile2 unmarshal: %v\n", profile)
}
Result:
profile1 unmarshal: {foo 20}
2009/11/10 23:00:00 profile2 unmarshal error: required field is missing
You can go to Playground to have a look at the completed code
You can just implement the Unmarshaler interface to customize how your JSON gets unmarshalled.
you can also make use of JSON schema validation.
package main
import (
"encoding/json"
"fmt"
"github.com/alecthomas/jsonschema"
"github.com/xeipuuv/gojsonschema"
)
type Bird struct {
Species string `json:"birdType"`
Description string `json:"what it does" jsonschema:"required"`
}
func main() {
var bird Bird
sc := jsonschema.Reflect(&bird)
b, _ := json.Marshal(sc)
fmt.Println(string(b))
loader := gojsonschema.NewStringLoader(string(b))
documentLoader := gojsonschema.NewStringLoader(`{"birdType": "pigeon"}`)
schema, err := gojsonschema.NewSchema(loader)
if err != nil {
panic("nop")
}
result, err := schema.Validate(documentLoader)
if err != nil {
panic("nop")
}
if result.Valid() {
fmt.Printf("The document is valid\n")
} else {
fmt.Printf("The document is not valid. see errors :\n")
for _, err := range result.Errors() {
// Err implements the ResultError interface
fmt.Printf("- %s\n", err)
}
}
}
Outputs
{"$schema":"http://json-schema.org/draft-04/schema#","$ref":"#/definitions/Bird","definitions":{"Bird":{"required":["birdType","what it does"],"properties":{"birdType":{"type":"string"},"what it does":{"type":"string"}},"additionalProperties":false,"type":"object"}}}
The document is not valid. see errors :
- (root): what it does is required
code example taken from Strict JSON parsing
When unmarshalling a string of json, using Golang's json.Unmarshal() function, I always unmarshal the string into a map[string]interface{} - I'm not sure weather there is a more optimal way of decoding json.
To get to the point:
Sometimes the json's unmarshalled type is nil, not string (or int etc.). This always throws a panic, saying:
interface conversion: interface is nil, not int
How can I avoid this panic or check if the interface's type is nil or not?
Example:
Here is an example of my problem in action: https://play.golang.org/p/0ATzXBbdoS
Check if the key exist instead of letting it panic.
func keyExists(decoded map[string]interface{}, key string) {
val, ok := decoded[key]
return ok && val != nil
}
func main() {
jsonText := `{
"name": "Jimmy",
"age": 23
}`
var decoded map[string]interface{}
if err := json.Unmarshal([]byte(jsonText), &decoded); err != nil {
fmt.Println(err)
os.Exit(0)
}
if keyExists(decoded, "name") {
fmt.Println(decoded["name"].(string))
}
if keyExists(decoded, "age") {
fmt.Println(decoded["age"].(float64))
}
if keyExists(decoded, "gender") {
fmt.Println(decoded["gender"].(int))
}
}
Also, this is far from being optimal if you know what your json will look like. As specified in the documentation, you can unmarshal it directly into a struct:
type Human struct {
Name string
Age int
Gender int
}
func main() {
jsonText := `{
"name": "Jimmy",
"age": 23
}`
decoded := Human{}
if err := json.Unmarshal([]byte(jsonText), &decoded); err != nil {
fmt.Println(err)
os.Exit(0)
}
fmt.Println(decoded.Name)
fmt.Println(decoded.Age)
fmt.Println(decoded.Gender)
}
Is it possible to generate an error if a field was not found while parsing a JSON input using Go?
I could not find it in documentation.
Is there any tag that specifies the field as required?
There is no tag in the encoding/json package that sets a field to "required". You will either have to write your own MarshalJSON() method, or do a post check for missing fields.
To check for missing fields, you will have to use pointers in order to distinguish between missing/null and zero values:
type JsonStruct struct {
String *string
Number *float64
}
Full working example:
package main
import (
"fmt"
"encoding/json"
)
type JsonStruct struct {
String *string
Number *float64
}
var rawJson = []byte(`{
"string":"We do not provide a number"
}`)
func main() {
var s *JsonStruct
err := json.Unmarshal(rawJson, &s)
if err != nil {
panic(err)
}
if s.String == nil {
panic("String is missing or null!")
}
if s.Number == nil {
panic("Number is missing or null!")
}
fmt.Printf("String: %s Number: %f\n", *s.String, *s.Number)
}
Playground
You can also override the unmarshalling for a specific type (so a required field buried in a few json layers) without having to make the field a pointer. UnmarshalJSON is defined by the Unmarshaler interface.
type EnumItem struct {
Named
Value string
}
func (item *EnumItem) UnmarshalJSON(data []byte) (err error) {
required := struct {
Value *string `json:"value"`
}{}
all := struct {
Named
Value string `json:"value"`
}{}
err = json.Unmarshal(data, &required)
if err != nil {
return
} else if required.Value == nil {
err = fmt.Errorf("Required field for EnumItem missing")
} else {
err = json.Unmarshal(data, &all)
item.Named = all.Named
item.Value = all.Value
}
return
}
Here is another way by checking your customized tag
you can create a tag for your struct like:
type Profile struct {
Name string `yourprojectname:"required"`
Age int
}
Use reflect to check if the tag is assigned required value
func (p *Profile) Unmarshal(data []byte) error {
err := json.Unmarshal(data, p)
if err != nil {
return err
}
fields := reflect.ValueOf(p).Elem()
for i := 0; i < fields.NumField(); i++ {
yourpojectTags := fields.Type().Field(i).Tag.Get("yourprojectname")
if strings.Contains(yourpojectTags, "required") && fields.Field(i).IsZero() {
return errors.New("required field is missing")
}
}
return nil
}
And test cases are like:
func main() {
profile1 := `{"Name":"foo", "Age":20}`
profile2 := `{"Name":"", "Age":21}`
var profile Profile
err := profile.Unmarshal([]byte(profile1))
if err != nil {
log.Printf("profile1 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile1 unmarshal: %v\n", profile)
err = profile.Unmarshal([]byte(profile2))
if err != nil {
log.Printf("profile2 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile2 unmarshal: %v\n", profile)
}
Result:
profile1 unmarshal: {foo 20}
2009/11/10 23:00:00 profile2 unmarshal error: required field is missing
You can go to Playground to have a look at the completed code
You can just implement the Unmarshaler interface to customize how your JSON gets unmarshalled.
you can also make use of JSON schema validation.
package main
import (
"encoding/json"
"fmt"
"github.com/alecthomas/jsonschema"
"github.com/xeipuuv/gojsonschema"
)
type Bird struct {
Species string `json:"birdType"`
Description string `json:"what it does" jsonschema:"required"`
}
func main() {
var bird Bird
sc := jsonschema.Reflect(&bird)
b, _ := json.Marshal(sc)
fmt.Println(string(b))
loader := gojsonschema.NewStringLoader(string(b))
documentLoader := gojsonschema.NewStringLoader(`{"birdType": "pigeon"}`)
schema, err := gojsonschema.NewSchema(loader)
if err != nil {
panic("nop")
}
result, err := schema.Validate(documentLoader)
if err != nil {
panic("nop")
}
if result.Valid() {
fmt.Printf("The document is valid\n")
} else {
fmt.Printf("The document is not valid. see errors :\n")
for _, err := range result.Errors() {
// Err implements the ResultError interface
fmt.Printf("- %s\n", err)
}
}
}
Outputs
{"$schema":"http://json-schema.org/draft-04/schema#","$ref":"#/definitions/Bird","definitions":{"Bird":{"required":["birdType","what it does"],"properties":{"birdType":{"type":"string"},"what it does":{"type":"string"}},"additionalProperties":false,"type":"object"}}}
The document is not valid. see errors :
- (root): what it does is required
code example taken from Strict JSON parsing