Go yaml how to add new entry to yaml file - json

I need to add entry to a yaml file from struct value config.Sif["snk_prod"] (I need to fill it on runtime) and try the following , I tried the following but I got an error when filling the struct, any idea?
package main
import "gopkg.in/yaml.v3"
const document = `
spec:
mec:
customConfig:
sif:
prom_exporter:
type: prometheus_exporter
snk_dev:
type: sk_hc_logs
inputs:
- tesslt
ep: ${NT}
dken: ${SN}
`
type YamlObject map[string]any
type CustomConfig struct {
Sif map[string]interface{} `yaml:"sif,omitempty"`
}
type ecfg struct {
Type string `yaml:"type,omitempty"`
Inputs []string `yaml:"inputs,omitempty"`
Ep string `yaml:"ep,omitempty"`
Dken string `yaml:"dken,omitempty"`
}
func main() {
t := CustomConfig{}
config := &t
// -------I need to add it as struct values as I got the values on runtime dynamically
config.Sif["snk_prod"] = ecfg{
Type: "sk_hc_ls",
Inputs: []string{"tesslt"},
Ep: "${NT}",
}
yamlBytes, err := yaml.Marshal(t)
doc := make(YamlObject)
if err := yaml.Unmarshal([]byte(document), &doc); err != nil {
panic(err)
}
addon := make(YamlObject)
if err := yaml.Unmarshal(yamlBytes, &addon); err != nil {
panic(err)
}
node := findChild(doc, "spec", "mec", "customConfig", "sif")
if node == nil {
panic("Must not happen")
}
for key, val := range addon {
(*node)[key] = val
}
outDoc, err := yaml.Marshal(doc)
if err != nil {
panic(err)
}
println(string(outDoc))
}
func findChild(obj YamlObject, path ...string) *YamlObject {
if len(path) == 0 {
return &obj
}
key := path[0]
child, ok := obj[key]
if !ok {
return nil
}
obj, ok = child.(YamlObject)
if !ok {
return nil
}
return findChild(obj, path[1:]...)
}
https://go.dev/play/p/6CHsOJPXqpw
After searching I found this answer https://stackoverflow.com/a/74089724/6340176
which is quite similar to mine, the change is the that I need to add it as struct value
at the end I need to add new entry under sif
At the end The output should be like following
spec:
mec:
customConfig:
sif:
prom_exporter:
type: prometheus_exporter
snk_dev:
type: sk_hc_logs
inputs:
- tesslt
ep: ${NT}
dken: ${SN}
snk_prod:
type: sk_hc_ls
inputs:
- tesslt
ep: ${NT}

Replace the creation of the yamlBytes as follows:
t := make(map[string]interface{})
t["snk_prod"] = ecfg{
Type: "sk_hc_ls",
Inputs: []string{"tesslt"},
Ep: "${NT}",
}
yamlBytes, err := yaml.Marshal(t)
Then you will get the expected result.
BTW: The panic is triggered because you are trying to insert a value into an uninitialized map (config.Sif is nil). You could simply create an empty map using, for example, the following line of code before assigning values:
config.Sif = make(map[string]interface{})
but there would be an additional unwanted sif node in the code, e.g. something like this:
...
sif:
prom_exporter:
type: prometheus_exporter
sif:
snk_prod:
...
Therefore, the yaml snippet to be added dynamically should be generated as shown at the beginning.

Related

Golang Interface to struct conversion giving error

I have an json as a string of following format:
{"add": [{"var": ["100"]}, "200"]}
Here the key 'add' is not a constant value for all the jsons. In some cases, it can be 'minus', 'multiply' etc.
The value of that key is an array. In this case [{"var": ["100"]}, "200"]. This means that the 100 should be added to existing value 200.
I am trying to parse this expression. Since the main key(in this case 'add') is not a constant, I cannot convert into a struct. So, I converted it to a json object by following way:
type mathExpVar struct {
valueVar []string `json:"var"`
}
var mathExpJson map[string][]interface{}
var input = "{\"add\": [{\"var\": [\"100\"]}, \"200\"]}"
err := json.Unmarshal([]byte(input), &mathExpJson)
for operator, values := range mathExpJson{
vals, ok := values[0].(mathExpVar) // here values[0] will be {"var": ["100"]}
if !ok{
return nil
}
}
Here 'ok' is always returning false. I am not sure why. There is no additional error message for me to check why this is failing. Could someone help me in resolving this?
Link to go playground for the same: https://go.dev/play/p/POfQmEoPbjD
A Working example: https://go.dev/play/p/02YzI5cv8vV
The whole reason the original response from Burak Serdar was not valid in my opinion is that it does not take into account the fact that you would need to handle the rest of the params as well. If you look closely enough, then you see that the expression is not an array of strings, its of varying type. This implementation handles the custom Unmarshalling and stores all the extra parameters in the Extra field.
Also code:
package main
import (
"encoding/json"
"log"
"reflect"
)
const jsonPayload = "{\"add\": [{\"var\": [\"100\"]}, \"200\"]}"
func main() {
data := MathExpressions{}
err := json.Unmarshal([]byte(jsonPayload), &data)
if err != nil {
log.Println("Failed to unmarshal json, error:", err)
return
}
log.Println(data)
for operation, expression := range data {
log.Print("Op:", operation, "Exp:", expression)
}
log.Println("Finished..")
}
/**
The sub definition for a specific expression in the object
*/
type ExpressionDefinition struct {
Vars []string `json:"var"`
Extra []string
}
func (e *ExpressionDefinition) UnmarshalJSON(data []byte) error {
tokens := make([]interface{}, 0)
err := json.Unmarshal(data, &tokens)
if err != nil {
return err
}
for _, token := range tokens {
log.Println("Processing token:", token, "type:", reflect.TypeOf(token))
switch token.(type) {
case map[string]interface{}:
for _, v := range token.(map[string]interface{})["var"].([]interface{}) {
e.Vars = append(e.Vars, v.(string))
}
case string:
e.Extra = append(e.Extra, token.(string))
}
}
log.Println(tokens)
return nil
}
/**
The main expressions object which contains all the sub-expressions.
*/
type MathExpressions map[string]ExpressionDefinition
Here the entire structure of the parsed json value will be stored in nested map[string]interface{}(json object) or []interface{}(json array) types.
In the line:
vals, ok := values[0].(mathExpVar)
values[0] would be of type map[string]interface{}, which cannot be asserted to mathExpVar, which is a struct, an entirely different datatype.
You need to type assert to map[string]interface{} first, then do this in each nested level as you go forward:
package main
import (
"encoding/json"
"fmt"
)
func main() {
type mathExpVar struct {
valueVar []string `json:"var"`
}
var mathExpJson map[string][]interface{}
var input = "{\"add\": [{\"var\": [\"100\"]}, \"200\"]}"
err := json.Unmarshal([]byte(input), &mathExpJson)
if err != nil {
fmt.Println("Error in unmarshalling")
}
for _, values := range mathExpJson {
var vals mathExpVar
valMap, ok := values[0].(map[string]interface{})
if ok {
varSlice, ok := valMap["var"].([]interface{})
if ok {
for _, v := range varSlice {
nv, ok := v.(string)
if ok {
vals.valueVar = append(vals.valueVar, nv)
} else {
fmt.Printf("%T\n", v)
}
}
} else {
fmt.Printf("%T\n", valMap["var"])
}
} else {
fmt.Printf("%T\n", values[0])
}
fmt.Printf("%+v\n", vals)
}
}
See: https://go.dev/play/p/Ot_9IZr4pwM
For more on interfaces and go reflection, check out: https://go.dev/blog/laws-of-reflection

How do you modify this struct in Golang to accept two different results?

For the following JSON response {"table_contents":[{"id":100,"description":"text100"},{"id":101,"description":"text101"},{"id":1,"description":"text1"}]}
All you have to do is to produce the following code to execute it properly and be able to reads fields from the struct, such as:
package main
import (
"fmt"
"encoding/json"
)
type MyStruct1 struct {
TableContents []struct {
ID int
Description string
} `json:"table_contents"`
}
func main() {
result:= []byte(`{"table_contents":[{"id":100,"description":"text100"},{"id":101,"description":"text101"},{"id":1,"description":"text1"}]}`)
var container MyStruct1
err := json.Unmarshal(result, &container)
if err != nil {
fmt.Println(" [0] Error message: " + err.Error())
return
}
for i := range container.TableContents {
fmt.Println(container.TableContents[i].Description)
}
}
But how do you deal with the following JSON response? {"table_contents":[[{"id":100,"description":"text100"},{"id":101,"description":"text101"}],{"id":1,"description":"text1"}]} You can either get this response or the one above, it is important to modify the struct to accept both.
I did something like this, with the help of internet:
package main
import (
"fmt"
"encoding/json"
)
type MyStruct1 struct {
TableContents []TableContentUnion `json:"table_contents"`
}
type TableContentClass struct {
ID int
Description string
}
type TableContentUnion struct {
TableContentClass *TableContentClass
TableContentClassArray []TableContentClass
}
func main() {
result:= []byte(`{"table_contents":[[{"id":100,"description":"text100"},{"id":101,"description":"text101"}],{"id":1,"description":"text1"}]}`)
var container MyStruct1
err := json.Unmarshal(result, &container)
if err != nil {
fmt.Println(" [0] Error message: " + err.Error())
return
}
for i := range container.TableContents {
fmt.Println(container.TableContents[i])
}
}
but it does not go past the error message :(
[0] Error message: json: cannot unmarshal array into Go struct field MyStruct1.table_contents of type main.TableContentUnion*
Been struggling to come up with a solution for hours. If someone could help I would be happy. Thank you for reading. Let me know if you have questions
Inside table_contents you have two type options (json object or list of json objects). What you can do is to unmarshall into an interface and then run type-check on it when using it:
type MyStruct1 struct {
TableContents []interface{} `json:"table_contents"`
}
...
for i := range container.TableContents {
switch container.TableContents[i].(type){
case map[string]interface{}:
fmt.Println("json object")
case []interface{}:
fmt.Println("list")
}
}
From there you can use some library (e.g. https://github.com/mitchellh/mapstructure) to map unmarshalled struct to your TableContentClass type. See PoC playground here: https://play.golang.org/p/NhVUhQayeL_C
Custom UnmarshalJSON function
You can also create a custom UnmarshalJSON function on the object that has the 2 possibilities. In you case that would be TableContentUnion.
In the custom unmarshaller you can then decide how to unmarshal the content.
func (s *TableContentUnion) UnmarshalJSON(b []byte) error {
// Note that we get `b` as bytes, so we can also manually check to see
// if it is an array (starts with `[`) or an object (starts with `{`)
var jsonObj interface{}
if err := json.Unmarshal(b, &jsonObj); err != nil {
return err
}
switch jsonObj.(type) {
case map[string]interface{}:
// Note: instead of using json.Unmarshal again, we could also cast the interface
// and build the values as in the example above
var tableContentClass TableContentClass
if err := json.Unmarshal(b, &tableContentClass); err != nil {
return err
}
s.TableContentClass = &tableContentClass
case []interface{}:
// Note: instead of using json.Unmarshal again, we could also cast the interface
// and build the values as in the example above
if err := json.Unmarshal(b, &s.TableContentClassArray); err != nil {
return err
}
default:
return errors.New("TableContentUnion.UnmarshalJSON: unknown content type")
}
return nil
}
The rest then works like in your test code that was failing before. Here the working Go Playground
Unmarshal to map and manually build struct
You can always unmarshal a json (with an object at the root) into a map[string]interface{}. Then you can iterate things and further unmarshal them after checking what type they are.
Working example:
func main() {
result := []byte(`{"table_contents":[[{"id":100,"description":"text100"},{"id":101,"description":"text101"}],{"id":1,"description":"text1"}]}`)
var jsonMap map[string]interface{}
err := json.Unmarshal(result, &jsonMap)
if err != nil {
fmt.Println(" [0] Error message: " + err.Error())
return
}
cts, ok := jsonMap["table_contents"].([]interface{})
if !ok {
// Note: nil or missing 'table_contents" will also lead to this path.
fmt.Println("table_contents is not a slice")
return
}
var unions []TableContentUnion
for _, content := range cts {
var union TableContentUnion
if contents, ok := content.([]interface{}); ok {
for _, content := range contents {
contCls := parseContentClass(content)
if contCls == nil {
continue
}
union.TableContentClassArray = append(union.TableContentClassArray, *contCls)
}
} else {
contCls := parseContentClass(content)
union.TableContentClass = contCls
}
unions = append(unions, union)
}
container := MyStruct1{
TableContents: unions,
}
for i := range container.TableContents {
fmt.Println(container.TableContents[i])
}
}
func parseContentClass(value interface{}) *TableContentClass {
m, ok := value.(map[string]interface{})
if !ok {
return nil
}
return &TableContentClass{
ID: int(m["id"].(float64)),
Description: m["description"].(string),
}
}
This is most useful if the json has too many variations. For cases like this it might also make sense sometimes to switch to a json package that works differently like https://github.com/tidwall/gjson which gets values based on their path.
Use json.RawMessage to capture the varying parts of the JSON document. Unmarshal each raw message as appropriate.
func (ms *MyStruct1) UnmarshalJSON(data []byte) error {
// Declare new type with same base type as MyStruct1.
// This breaks recursion in call to json.Unmarshal below.
type x MyStruct1
v := struct {
*x
// Override TableContents field with raw message.
TableContents []json.RawMessage `json:"table_contents"`
}{
// Unmarshal all but TableContents directly to the
// receiver.
x: (*x)(ms),
}
err := json.Unmarshal(data, &v)
if err != nil {
return err
}
// Unmarahal raw elements as appropriate.
for _, tcData := range v.TableContents {
if bytes.HasPrefix(tcData, []byte{'{'}) {
var v TableContentClass
if err := json.Unmarshal(tcData, &v); err != nil {
return err
}
ms.TableContents = append(ms.TableContents, v)
} else {
var v []TableContentClass
if err := json.Unmarshal(tcData, &v); err != nil {
return err
}
ms.TableContents = append(ms.TableContents, v)
}
}
return nil
}
Use it like this:
var container MyStruct1
err := json.Unmarshal(result, &container)
if err != nil {
// handle error
}
Run it on the Go playground.
This approach does not add any outside dependencies. The function code does not need to modified when fields are added or removed from MyStruct1 or TableContentClass.

JSON Unmarshal does not work with dynamic struct created by relect

I'm trying to parse JSON files by using dynamically created structs, but apparently I'm doing something wrong. Can somebody please tell we what am I doing wrong here:
structured := make(map[string][]reflect.StructField)
structured["Amqp1"] = []reflect.StructField{
reflect.StructField{
Name: "Test",
Type: reflect.TypeOf(""),
Tag: reflect.StructTag(`json:"test"`),
},
reflect.StructField{
Name: "Float",
Type: reflect.TypeOf(5.5),
Tag: reflect.StructTag(`json:"float"`),
},
reflect.StructField{
Name: "Connections",
Type: reflect.TypeOf([]Connection{}),
Tag: reflect.StructTag(`json:"connections"`),
},
}
sections := []reflect.StructField{}
for sect, params := range structured {
sections = append(sections,
reflect.StructField{
Name: sect,
Type: reflect.StructOf(params),
},
)
}
parsed := reflect.New(reflect.StructOf(sections)).Elem()
if err := json.Unmarshal([]byte(JSONConfigContent), &parsed); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
https://play.golang.org/p/C2I4Pduduyg
Thanks in advance.
You want to use .Interface() to return the actual, underlying value, which should be a pointer to the concrete anonymous struct.
Note that the reflect.New function returns a reflect.Value representing a pointer to a new zero value for the specified type. The Interface method, in this case, returns that pointer as interface{} which is all you need for json.Unmarshal.
If, after unmarshaling, you need a non-pointer of the struct you can turn to reflect again and use reflect.ValueOf(parsed).Elem().Interface() to effectively dereference the pointer.
parsed := reflect.New(reflect.StructOf(sections)).Interface()
if err := json.Unmarshal([]byte(JSONConfigContent), parsed); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
https://play.golang.org/p/Bzu1hUyKjvM
reflect.New returns a value representing a pointer.
Change line 69:
fmt.Printf(">>> %v", &parsed)
Result:
>>> <struct { Amqp1 struct { Test string "json:\"test\""; Float float64 "json:\"float\""; Connections []main.Connection "json:\"connections\"" } } Value>
I would recommend something very different. I generally avoid reflection where possible. You can accomplish what you are trying to do by simply providing structs for each type of config you expect and then do an initial "pre-unmarshalling" to determine what type of config you should actually use by name (which, in this case is a key of your JSON object):
package main
import (
"encoding/json"
"fmt"
"os"
)
//Amqp1 config struct
type Amqp1 struct {
Config struct {
Test string `json:"test"`
Float float64 `json:"float"`
Connections []Connection `json:"connections"`
} `json:"Amqp1"`
}
//Connection struct
type Connection struct {
Type string `json:"type"`
URL string `json:"url"`
}
//JSONConfigContent json
const JSONConfigContent = `{
"Amqp1": {
"test": "woobalooba",
"float": 5.5,
"connections": [
{"type": "test1", "url": "booyaka"},
{"type": "test2", "url": "foobar"}
]
}
}`
func main() {
configMap := make(map[string]interface{})
if err := json.Unmarshal([]byte(JSONConfigContent), &configMap); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
//get config name
var configName string
for cfg := range configMap {
configName = cfg
break
}
//unmarshal appropriately
switch configName {
case "Amqp1":
var amqp1 Amqp1
if err := json.Unmarshal([]byte(JSONConfigContent), &amqp1); err != nil {
fmt.Printf("unable to parse data from provided configuration file: %s\n", err)
os.Exit(1)
}
fmt.Printf("%s >>\n", configName)
fmt.Printf("Test: %s\n", amqp1.Config.Test)
fmt.Printf("Float: %v\n", amqp1.Config.Float)
fmt.Printf("Connections: %#v\n", amqp1.Config.Connections)
default:
fmt.Printf("unknown config encountered: %s\n", configName)
os.Exit(1)
}
}
The initial "pre-unmarshalling" happens on the 2nd line of main(), to a plain map where the key is a string and the value is interface{} because you don't care. You're just doing that to get the actual type of config that it is, which is the key of the first nested object.
You can run this on playground as well.

Golang - Json decoding, check field exist or not automatically [duplicate]

Is it possible to generate an error if a field was not found while parsing a JSON input using Go?
I could not find it in documentation.
Is there any tag that specifies the field as required?
There is no tag in the encoding/json package that sets a field to "required". You will either have to write your own MarshalJSON() method, or do a post check for missing fields.
To check for missing fields, you will have to use pointers in order to distinguish between missing/null and zero values:
type JsonStruct struct {
String *string
Number *float64
}
Full working example:
package main
import (
"fmt"
"encoding/json"
)
type JsonStruct struct {
String *string
Number *float64
}
var rawJson = []byte(`{
"string":"We do not provide a number"
}`)
func main() {
var s *JsonStruct
err := json.Unmarshal(rawJson, &s)
if err != nil {
panic(err)
}
if s.String == nil {
panic("String is missing or null!")
}
if s.Number == nil {
panic("Number is missing or null!")
}
fmt.Printf("String: %s Number: %f\n", *s.String, *s.Number)
}
Playground
You can also override the unmarshalling for a specific type (so a required field buried in a few json layers) without having to make the field a pointer. UnmarshalJSON is defined by the Unmarshaler interface.
type EnumItem struct {
Named
Value string
}
func (item *EnumItem) UnmarshalJSON(data []byte) (err error) {
required := struct {
Value *string `json:"value"`
}{}
all := struct {
Named
Value string `json:"value"`
}{}
err = json.Unmarshal(data, &required)
if err != nil {
return
} else if required.Value == nil {
err = fmt.Errorf("Required field for EnumItem missing")
} else {
err = json.Unmarshal(data, &all)
item.Named = all.Named
item.Value = all.Value
}
return
}
Here is another way by checking your customized tag
you can create a tag for your struct like:
type Profile struct {
Name string `yourprojectname:"required"`
Age int
}
Use reflect to check if the tag is assigned required value
func (p *Profile) Unmarshal(data []byte) error {
err := json.Unmarshal(data, p)
if err != nil {
return err
}
fields := reflect.ValueOf(p).Elem()
for i := 0; i < fields.NumField(); i++ {
yourpojectTags := fields.Type().Field(i).Tag.Get("yourprojectname")
if strings.Contains(yourpojectTags, "required") && fields.Field(i).IsZero() {
return errors.New("required field is missing")
}
}
return nil
}
And test cases are like:
func main() {
profile1 := `{"Name":"foo", "Age":20}`
profile2 := `{"Name":"", "Age":21}`
var profile Profile
err := profile.Unmarshal([]byte(profile1))
if err != nil {
log.Printf("profile1 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile1 unmarshal: %v\n", profile)
err = profile.Unmarshal([]byte(profile2))
if err != nil {
log.Printf("profile2 unmarshal error: %s\n", err.Error())
return
}
fmt.Printf("profile2 unmarshal: %v\n", profile)
}
Result:
profile1 unmarshal: {foo 20}
2009/11/10 23:00:00 profile2 unmarshal error: required field is missing
You can go to Playground to have a look at the completed code
You can just implement the Unmarshaler interface to customize how your JSON gets unmarshalled.
you can also make use of JSON schema validation.
package main
import (
"encoding/json"
"fmt"
"github.com/alecthomas/jsonschema"
"github.com/xeipuuv/gojsonschema"
)
type Bird struct {
Species string `json:"birdType"`
Description string `json:"what it does" jsonschema:"required"`
}
func main() {
var bird Bird
sc := jsonschema.Reflect(&bird)
b, _ := json.Marshal(sc)
fmt.Println(string(b))
loader := gojsonschema.NewStringLoader(string(b))
documentLoader := gojsonschema.NewStringLoader(`{"birdType": "pigeon"}`)
schema, err := gojsonschema.NewSchema(loader)
if err != nil {
panic("nop")
}
result, err := schema.Validate(documentLoader)
if err != nil {
panic("nop")
}
if result.Valid() {
fmt.Printf("The document is valid\n")
} else {
fmt.Printf("The document is not valid. see errors :\n")
for _, err := range result.Errors() {
// Err implements the ResultError interface
fmt.Printf("- %s\n", err)
}
}
}
Outputs
{"$schema":"http://json-schema.org/draft-04/schema#","$ref":"#/definitions/Bird","definitions":{"Bird":{"required":["birdType","what it does"],"properties":{"birdType":{"type":"string"},"what it does":{"type":"string"}},"additionalProperties":false,"type":"object"}}}
The document is not valid. see errors :
- (root): what it does is required
code example taken from Strict JSON parsing

What input will cause golang's json.Marshal to return an error?

From the docs:
JSON cannot represent cyclic data structures and Marshal does not handle them. Passing cyclic structures to Marshal will result in an infinite recursion.
I've experienced this situation, which results in a runtime panic.
What I'm wondering is if anyone can provide a working program that demonstrates a non-panic situation where json.Marshal returns a non-nil error. The best answers would clearly include the inputs used.
Just to complement Jonathan's answer, the json.Marshal function can return two types of errors: UnsupportedTypeError or UnsupportedValueError
The first one can be caused, as Jonathan said by trying to Marshal an invalid type:
_, err := json.Marshal(make(chan int))
_, ok := err.(*json.UnsupportedTypeError) // ok == true
On the other hand you can also have the Marshal function return an error by passing an invalid value:
_, err := json.Marshal(math.Inf(1))
_, ok := err.(*json.UnsupportedValueError) // ok == true
Update: now using a channel instead of a map[int]int to elicit the error
Go-specific structures,e.g. func or chan refuse to serialize:
package main
import (
"encoding/json"
"fmt"
)
func main() {
value := make(chan int)
_, err := json.Marshal(value)
fmt.Println(err)
}
Read the source code you can found such a function to judge a encoder if not exist will return marshal error: https://github.com/golang/go/blob/master/src/encoding/json/encode.go
func newTypeEncoder(t reflect.Type, allowAddr bool) encoderFunc {
// ignored
switch t.Kind() {
case reflect.Bool:
return boolEncoder
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return intEncoder
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return uintEncoder
case reflect.Float32:
return float32Encoder
case reflect.Float64:
return float64Encoder
case reflect.String:
return stringEncoder
case reflect.Interface:
return interfaceEncoder
case reflect.Struct:
return newStructEncoder(t)
case reflect.Map:
return newMapEncoder(t)
case reflect.Slice:
return newSliceEncoder(t)
case reflect.Array:
return newArrayEncoder(t)
case reflect.Ptr:
return newPtrEncoder(t)
default:
return unsupportedTypeEncoder
}
}
We can find all kinds enum
at https://github.com/golang/go/blob/master/src/reflect/type.go
So it's not hard to see that kinds not in above function are unable to marshal:
UnsafePointer,Complex64,Complex128,Chan,Func
Examples:
json.Marshal(unsafe.Pointer(nil)) // UnsafePointer
json.Marshal(complex64(1)) // Complex64
json.Marshal(complex128(1)) // Complex128
json.Marshal(make(chan struct{})) // Chan
json.Marshal(func() {}) // Func
A while ago I was solving a problem of serializing/deserializing cyclic references in golang, and all the links go to this question. However, it's slightly misleading as the question is broader.
If you got into the same situation like me, and can't find a solution on how to deal with cyclic references, you can now use tahwil - a new library that I published on github. To my knowledge it's now the only library that facilitates serialization/deserialization of cyclic data structures in a generic way.
Readme gives the information on how to use the library, so I will only duplicate the examples here.
Encoding:
package main
import (
"encoding/json"
"fmt"
"github.com/go-extras/tahwil"
)
type Person struct {
Name string
Parent *Person
Children []*Person
}
func main() {
parent := &Person{
Name: "Arthur",
Children: []*Person{
{
Name: "Ford",
},
{
Name: "Trillian",
},
},
}
parent.Children[0].Parent = parent
parent.Children[1].Parent = parent
v, err := tahwil.ToValue(parent)
if err != nil {
panic(err)
}
res, err := json.Marshal(v)
if err != nil {
panic(err)
}
fmt.Println(string(res))
}
Decoding:
package main
import (
"encoding/json"
"fmt"
"github.com/go-extras/tahwil"
)
type Person struct {
Name string `json:"name"`
Parent *Person `json:"parent"`
Children []*Person `json:"children"`
}
func prepareData() []byte {
parent := &Person{
Name: "Arthur",
Children: []*Person{
{
Name: "Ford",
},
{
Name: "Trillian",
},
},
}
parent.Children[0].Parent = parent
parent.Children[1].Parent = parent
v, err := tahwil.ToValue(parent)
if err != nil {
panic(err)
}
res, err := json.Marshal(v)
if err != nil {
panic(err)
}
return res
}
func main() {
data := &tahwil.Value{}
res := prepareData()
err := json.Unmarshal(res, data)
if err != nil {
panic(err)
}
person := &Person{}
err = tahwil.FromValue(data, person)
if err != nil {
panic(err)
}
fmt.Printf(`Name: %s
Children:
- %s
-- parent name: %s
- %s
-- parent name: %s
`, person.Name,
person.Children[0].Name,
person.Children[0].Parent.Name,
person.Children[1].Name,
person.Children[1].Parent.Name)
}
The main idea is to transform the original data to tahwil.Value{}, which essentially adds refid's to all of your fields. Whenever tahwil encounters a cyclic reference, it replaces the actual object with a reference. And after that the graph is technically not cyclic anymore and thus can be marshalled to json.
Restoring the data means a reverse operation, i.e. any reference will be replaced by a pointer to an object.
P.S. Why tahwil? I tried to find some uncommon word for the name, and found an Arabic word (تحويل) that means conversion.