Create instance of struct via reflection and set values - json

what I try to do
I try to pass an instance of a struct - including json tags to a func, create a new instance, and set value on field
after this i try to serialize (JSON), but the values are empty
NOTICE: i looked up loads of articles on SO about setting values via reflection, but it seems i missed a little detail
struct definition
this part defines the struct with json and xml tags
type Person struct {
Name string `json:"Name" xml:"Person>FullName"`
Age int `json:"Age" xml:"Person>Age"`
}
create instance (+wrapping into empty interface)
afterwards I create an instance and store it in an interface{} - why? because in my production code this stuff will be done in a func which accepts a interface{}
var iFace interface{} = Person{
Name: "Test",
Age: 666,
}
creating a new instance of the struct and setting values via reflection
iFaceType := reflect.TypeOf(iFace)
item := reflect.New(iFaceType)
s := item.Elem()
if s.Kind() == reflect.Struct {
fName := s.FieldByName("Name")
if fName.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fName.CanSet() {
// change value of N
switch fName.Kind() {
case reflect.String:
fName.SetString("reflectedNameValue")
fmt.Println("Name was set to reflectedNameValue")
}
}
}
fAge := s.FieldByName("Age")
if fAge.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fAge.CanSet() {
// change value of N
switch fAge.Kind() {
case reflect.Int:
x := int64(42)
if !fAge.OverflowInt(x) {
fAge.SetInt(x)
fmt.Println("Age was set to", x)
}
}
}
}
}
Question
what am I doing wrong?
in production code I fill multiple copies with data and add it to a slice...
but this only makes sense if the json tags are kept in place and the stuff serializes just the same way.
code sample for play
package main
import (
"encoding/json"
"fmt"
"reflect"
)
func main() {
type Person struct {
Name string `json:"Name" xml:"Person>FullName"`
Age int `json:"Age" xml:"Person>Age"`
}
var iFace interface{} = Person{
Name: "Test",
Age: 666,
}
fmt.Println("normal: \n" + JSONify(iFace))
iFaceType := reflect.TypeOf(iFace)
item := reflect.New(iFaceType)
s := item.Elem()
if s.Kind() == reflect.Struct {
fName := s.FieldByName("Name")
if fName.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fName.CanSet() {
// change value of N
switch fName.Kind() {
case reflect.String:
fName.SetString("reflectedNameValue")
fmt.Println("Name was set to reflectedNameValue")
}
}
}
fAge := s.FieldByName("Age")
if fAge.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fAge.CanSet() {
// change value of N
switch fAge.Kind() {
case reflect.Int:
x := int64(42)
if !fAge.OverflowInt(x) {
fAge.SetInt(x)
fmt.Println("Age was set to", x)
}
}
}
}
}
fmt.Println("reflected: \n" + JSONify(item))
}
func JSONify(v interface{}) string {
var bytes []byte
bytes, _ = json.MarshalIndent(v, "", "\t")
return string(bytes)
}

Your item is of type reflect.Value. You have to call Value.Interface() to obtain the value wrapped in it:
fmt.Println("reflected: \n" + JSONify(item.Interface()))
With this change, output will be (try it on the Go Playground):
normal:
{
"Name": "Test",
"Age": 666
}
Name was set to reflectedNameValue
Age was set to 42
reflected:
{
"Name": "reflectedNameValue",
"Age": 42
}
reflect.Value itself is also a struct, but obviously trying to marshal it will not be identical to marshaling a Person struct value. reflect.Value does not implement marshaling the wrapped data to JSON.

Related

access to struct field from struct method field

How I can access struct field Name from struct method field PrintName
example:
type Data struct {
Name string
PrintName func()
}
func main() {
data := Data{
Name: "Name",
PrintName: func() {
fmt.Println(Name)
},
}
data.PrintName()
}
The function value you assign to Data.PrintName has no connection to the enclosing struct whatsoever, so in the general case it can't access the Name field.
If you want to access the name field, you have to pass it explicitly, or use a closure that can access the data struct like this:
data := Data{
Name: "Name",
}
data.PrintName = func() {
fmt.Println(data.Name)
}
data.PrintName()
This will output (try it on the Go Playground):
Name
In this example the function value we assign to data.PrintName field is a closure because it uses a variable from the enclosing block.
You can also choose to pass the Data value explicitly to the function value, but then its signature must be modified:
type Data struct {
Name string
PrintName func(d Data)
}
func main() {
data := Data{
Name: "Name",
}
data.PrintName = func(d Data) {
fmt.Println(d.Name)
}
data.PrintName(data)
}
This outputs the same, try this one on the Go Playground.
Is there any particular reason you want PrintName as a struct field instead of just a method of Data?
type Data struct {
Name string
}
func (data *Data) PrintName(){
fmt.Println(data.Name)
}

How can I skip a field for json.Marshal & not for json.Unmarshal in go?

type Alpha struct {
Name string `json:"name"`
SkipWhenMarshal string `json:"skipWhenMarshal"`
}
func MarshalJSON(out interface{}){
json.Marshal(out)
}
Is it possible to ignore the SkipWhenMarshal field when I do json.Marshal but not when I do
json.Unmarshal.
It should work for any type who calls MarshalJSON
Field tag modifiers like "omitempty" and "-" apply to both marshaling and unmarshaling, so there's no automatic way.
You can implement a MarshalJSON for your type that ignores whatever fields you need. There's no need to implement a custom unmarshaler, because the default works for you.
E.g. something like this:
type Alpha struct {
Id int32
Name string
SkipWhenMarshal string
}
func (a Alpha) MarshalJSON() ([]byte, error) {
m := map[string]string{
"id": fmt.Sprintf("%d", a.Id),
"name": a.Name,
// SkipWhenMarshal *not* marshaled here
}
return json.Marshal(m)
}
You can also make it simpler by using an alias type:
func (a Alpha) MarshalJSON() ([]byte, error) {
type AA Alpha
aa := AA(a)
aa.SkipWhenMarshal = ""
return json.Marshal(aa)
}
Here SkipWhenMarshal will be output, but its value is zeroed out. The advantage in this approach is that if Alpha has many fields, you don't have to repeat them.
What you want simply cannot be done with encoding/json.
But you can have two types
type AlphaIn struct {
Name string `json:"name"`
Bar string `json:"skipWhenMarshal"`
}
type AlphaOut struct {
Name string `json:"name"`
Bar string `json:"-"`
}
Use AlphaIn to deserialise JSON with encoding/json.Unmarshal and use AlphaOut to serialise a struct with encoding/json.Marshal to JSON.
Now this alone would be absolute painful to work with but: struct tags do not play a role in convertibility between types which lets you convert from AlphaIn to AlphaOut with a simple type conversion:
var a AlphaIn = ...
var b AlphaOut = AlphaOut(a)
(A saner naming scheme would be Alpha and AlphaToJSON or soemthing like this.)
I would write a custom marshal function like so: playground
package main
import (
"encoding/json"
"fmt"
"reflect"
"strings"
)
type Alpha struct {
Name string `json:"name"`
SkipWhenMarshal string `json:"SkipWhenMarshal,skip"`
}
func main() {
var a Alpha
a.Name = "John"
a.SkipWhenMarshal = "Snow"
out, _ := marshal(a)
fmt.Println(string(out))
var b Alpha
json.Unmarshal([]byte(`{"Name":"Samwell","SkipWhenMarshal":"Tarly"}`), &b)
fmt.Println(b)
}
// custom marshaling function for json that accepts an additional tag named skip to ignore the field
func marshal(v Alpha) ([]byte, error) {
m := map[string]interface{}{}
ut := reflect.TypeOf(v)
uv := reflect.ValueOf(v)
for i := 0; i < ut.NumField(); i++ {
field := ut.Field(i)
js, ok := field.Tag.Lookup("json")
if !ok || !strings.Contains(js, "skip") {
intf := uv.Field(i).Interface()
switch val := intf.(type) {
case int, int8, uint8:
m[field.Name] = val
case string:
m[field.Name] = val
}
}
}
return json.Marshal(m)
}
It basically rebuilds the struct as a map without the skipped fields and passes it to the original Marshal function.
Now just add the SKIP tag to any field and it should work.
You just need to improve this part:
switch val := intf.(type) {
Since this only assumes you only have strings or ints as fields. Handling more types would be more ideal.
You can try this i.e. breaking the structure into components -> composition
while unmarshalling use Beta type. It will unmarshall only fields that are defined inside Beta struct
type Beta struct {
Name string `json:"name"`
}
type Alpha struct {
*Beta
SkipWhenMarshal string `json:"skipWhenMarshal"`
}

How to show empty object instead of empty struct or nil in Go json marshal

I need to show json's empty object {} when do json.Marshal() for a struct pointer. I can only output either null value or empty struct value.
If the person key is filled with &Person{} or new(Person), it will show empty struct like below:
{
"data": {
"person": {
"name": "",
"age": 0
},
"created_date": "2009-11-10T23:00:00Z"
}
}
And if we don't initialize it at all, it will show null.
{
"data": {
"person": null,
"created_date": "2009-11-10T23:00:00Z"
}
}
I want to show "person": {}. Is it possible?
Go Playground for the complete code: https://play.golang.org/p/tT15G2ESPVc
Option A, use the omitempty tag option on all of the Person's fields and make sure the response's field is allocated before marshaling.
type Person struct {
Name string `json:"name,omitempty"`
Age int `json:"age,omitempty"`
}
// ...
resp.Person = new(Person)
https://play.golang.org/p/o3jWdru_8bC
Option B, use a non-pointer wrapper type that embeds the Person pointer type.
type PersonJSON struct {
*Person
}
type Response struct {
Person PersonJSON `json:"person"`
CreatedDate time.Time `json:"created_date"`
}
https://play.golang.org/p/EKQc7uf1_Vk
Option C, have the Reponse type implement the json.Marshaler interface.
func (r *Response) MarshalJSON() ([]byte, error) {
type tmp Response
resp := (*tmp)(r)
var data struct {
Wrapper struct {
*Person
} `json:"person"`
*tmp
}
data.Wrapper.Person = resp.Person
data.tmp = resp
return json.Marshal(data)
}
https://play.golang.org/p/1qkSCWZ225j
There may be other options...
In Go, an empty struct by definition assigns zero values to field elements. Eg: for int 0, "" for string, etc.
For your case, simply comparing to null would work out. Or, you could define an emptyPerson as:
var BAD_AGE = -1
emptyPerson := &Person{"", BAD_AGE} // BAD_AGE indicates no person
if person[age] == BAD_AGE {
// handle case for emptyPerson}

Is there a convenient way to convert a JSON-like map[string]interface{} to struct , and vice versa, with tags in Go? [duplicate]

I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}

Optional parameters with defaults in Go struct constructors

I've found myself using the following pattern as a way to get optional parameters with defaults in Go struct constructors:
package main
import (
"fmt"
)
type Object struct {
Type int
Name string
}
func NewObject(obj *Object) *Object {
if obj == nil {
obj = &Object{}
}
// Type has a default of 1
if obj.Type == 0 {
obj.Type = 1
}
return obj
}
func main() {
// create object with Name="foo" and Type=1
obj1 := NewObject(&Object{Name: "foo"})
fmt.Println(obj1)
// create object with Name="" and Type=1
obj2 := NewObject(nil)
fmt.Println(obj2)
// create object with Name="bar" and Type=2
obj3 := NewObject(&Object{Type: 2, Name: "foo"})
fmt.Println(obj3)
}
Is there a better way of allowing for optional parameters with defaults?
Dave Cheney offered a nice solution to this where you have functional options to overwrite defaults:
https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
So your code would become:
package main
import (
"fmt"
)
type Object struct {
Type int
Name string
}
func NewObject(options ...func(*Object)) *Object {
// Setup object with defaults
obj := &Object{Type: 1}
// Apply options if there are any
for _, option := range options {
option(obj)
}
return obj
}
func WithName(name string) func(*Object) {
return func(obj *Object) {
obj.Name = name
}
}
func WithType(newType int) func(*Object) {
return func(obj *Object) {
obj.Type = newType
}
}
func main() {
// create object with Name="foo" and Type=1
obj1 := NewObject(WithName("foo"))
fmt.Println(obj1)
// create object with Name="" and Type=1
obj2 := NewObject()
fmt.Println(obj2)
// create object with Name="bar" and Type=2
obj3 := NewObject(WithType(2), WithName("foo"))
fmt.Println(obj3)
}
https://play.golang.org/p/pGi90d1eI52
The approach seems reasonable to me. However, you have a bug. If I explicitly set Type to 0, it will get switched to 1.
My suggested fix: Use a struct literal for the default value: http://play.golang.org/p/KDNUauy6Ie
Or perhaps extract it out: http://play.golang.org/p/QpY2Ymze3b
Take a look at "Allocation with new" in Effective Go. They explain about making zero-value structs a useful default.
If you can make Object.Type (and your other fields) have a default of zero, then Go struct literals already give you exactly the feature you're requesting.
From the section on composite literals:
The fields of a composite literal are laid out in order and must all be present. However, by labeling the elements explicitly as field:value pairs, the initializers can appear in any order, with the missing ones left as their respective zero values.
That means you can replace this:
obj1 := NewObject(&Object{Name: "foo"})
obj2 := NewObject(nil)
obj3 := NewObject(&Object{Type: 2, Name: "foo"})
with this:
obj1 := &Object{Name: "foo"}
obj2 := &Object{}
obj3 := &Object{Type: 2, Name: "foo"}
If it is not possible to make the zero value the default for all of your fields, the recommended approach is a constructor function. For example:
func NewObject(typ int, name string) *Object {
return &Object{Type: typ, Name: name}
}
If you want Type to have a nonzero default, you can add another constructor function. Suppose Foo objects are the default and have Type 1.
func NewFooObject(name string) *Object {
return &Object{Type: 1, Name: name}
}
You only need to make one constructor function for each set of nonzero defaults you use. You can always reduce that set by changing the semantics of some fields to have zero defaults.
Also, note that adding a new field to Object with a zero default value doesn't require any code changes above, because all struct literals use labeled initialization. That comes in handy down the line.
https://play.golang.org/p/SABkY9dbCOD
Here's an alternative that uses a method of the object to set defaults. I've found it useful a few times, although it's not much different than what you have. This might allow better usage if it's part of a package. I don't claim to be a Go expert, maybe you'll have some extra input.
package main
import (
"fmt"
)
type defaultObj struct {
Name string
Zipcode int
Longitude float64
}
func (obj *defaultObj) populateObjDefaults() {
if obj.Name == "" {
obj.Name = "Named Default"
}
if obj.Zipcode == 0 {
obj.Zipcode = 12345
}
if obj.Longitude == 0 {
obj.Longitude = 987654321
}
}
func main() {
testdef := defaultObj{Name: "Mr. Fred"}
testdef.populateObjDefaults()
fmt.Println(testdef)
testdef2 := defaultObj{Zipcode: 90210}
testdef2.populateObjDefaults()
fmt.Println(testdef2)
testdef2.Name = "Mrs. Fred"
fmt.Println(testdef2)
testdef3 := defaultObj{}
fmt.Println(testdef3)
testdef3.populateObjDefaults()
fmt.Println(testdef3)
}
Output:
{Mr. Fred 12345 9.87654321e+08}
{Named Default 90210 9.87654321e+08}
{Mrs. Fred 90210 9.87654321e+08}
{ 0 0}
{Named Default 12345 9.87654321e+08}
You could use the ... operator.
instead of writing ToCall(a=b) like in python you write, ToCall("a",b)
See the Go Play Example
func GetKwds(kwds []interface{}) map[string]interface{} {
result := make(map[string]interface{})
for i := 0; i < len(kwds); i += 2 {
result[kwds[i].(string)] = kwds[i+1]
}
return result
}
func ToCall(kwds ...interface{}) {
args := GetKwds(kwds)
if value, ok := args["key"]; ok {
fmt.Printf("key: %#v\n", value)
}
if value, ok := args["other"]; ok {
fmt.Printf("other: %#v\n", value)
}
}
func main() {
ToCall()
ToCall("other", &map[string]string{})
ToCall("key", "Test", "other", &Object{})
}