Optional parameters with defaults in Go struct constructors - constructor

I've found myself using the following pattern as a way to get optional parameters with defaults in Go struct constructors:
package main
import (
"fmt"
)
type Object struct {
Type int
Name string
}
func NewObject(obj *Object) *Object {
if obj == nil {
obj = &Object{}
}
// Type has a default of 1
if obj.Type == 0 {
obj.Type = 1
}
return obj
}
func main() {
// create object with Name="foo" and Type=1
obj1 := NewObject(&Object{Name: "foo"})
fmt.Println(obj1)
// create object with Name="" and Type=1
obj2 := NewObject(nil)
fmt.Println(obj2)
// create object with Name="bar" and Type=2
obj3 := NewObject(&Object{Type: 2, Name: "foo"})
fmt.Println(obj3)
}
Is there a better way of allowing for optional parameters with defaults?

Dave Cheney offered a nice solution to this where you have functional options to overwrite defaults:
https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
So your code would become:
package main
import (
"fmt"
)
type Object struct {
Type int
Name string
}
func NewObject(options ...func(*Object)) *Object {
// Setup object with defaults
obj := &Object{Type: 1}
// Apply options if there are any
for _, option := range options {
option(obj)
}
return obj
}
func WithName(name string) func(*Object) {
return func(obj *Object) {
obj.Name = name
}
}
func WithType(newType int) func(*Object) {
return func(obj *Object) {
obj.Type = newType
}
}
func main() {
// create object with Name="foo" and Type=1
obj1 := NewObject(WithName("foo"))
fmt.Println(obj1)
// create object with Name="" and Type=1
obj2 := NewObject()
fmt.Println(obj2)
// create object with Name="bar" and Type=2
obj3 := NewObject(WithType(2), WithName("foo"))
fmt.Println(obj3)
}
https://play.golang.org/p/pGi90d1eI52

The approach seems reasonable to me. However, you have a bug. If I explicitly set Type to 0, it will get switched to 1.
My suggested fix: Use a struct literal for the default value: http://play.golang.org/p/KDNUauy6Ie
Or perhaps extract it out: http://play.golang.org/p/QpY2Ymze3b

Take a look at "Allocation with new" in Effective Go. They explain about making zero-value structs a useful default.
If you can make Object.Type (and your other fields) have a default of zero, then Go struct literals already give you exactly the feature you're requesting.
From the section on composite literals:
The fields of a composite literal are laid out in order and must all be present. However, by labeling the elements explicitly as field:value pairs, the initializers can appear in any order, with the missing ones left as their respective zero values.
That means you can replace this:
obj1 := NewObject(&Object{Name: "foo"})
obj2 := NewObject(nil)
obj3 := NewObject(&Object{Type: 2, Name: "foo"})
with this:
obj1 := &Object{Name: "foo"}
obj2 := &Object{}
obj3 := &Object{Type: 2, Name: "foo"}
If it is not possible to make the zero value the default for all of your fields, the recommended approach is a constructor function. For example:
func NewObject(typ int, name string) *Object {
return &Object{Type: typ, Name: name}
}
If you want Type to have a nonzero default, you can add another constructor function. Suppose Foo objects are the default and have Type 1.
func NewFooObject(name string) *Object {
return &Object{Type: 1, Name: name}
}
You only need to make one constructor function for each set of nonzero defaults you use. You can always reduce that set by changing the semantics of some fields to have zero defaults.
Also, note that adding a new field to Object with a zero default value doesn't require any code changes above, because all struct literals use labeled initialization. That comes in handy down the line.

https://play.golang.org/p/SABkY9dbCOD
Here's an alternative that uses a method of the object to set defaults. I've found it useful a few times, although it's not much different than what you have. This might allow better usage if it's part of a package. I don't claim to be a Go expert, maybe you'll have some extra input.
package main
import (
"fmt"
)
type defaultObj struct {
Name string
Zipcode int
Longitude float64
}
func (obj *defaultObj) populateObjDefaults() {
if obj.Name == "" {
obj.Name = "Named Default"
}
if obj.Zipcode == 0 {
obj.Zipcode = 12345
}
if obj.Longitude == 0 {
obj.Longitude = 987654321
}
}
func main() {
testdef := defaultObj{Name: "Mr. Fred"}
testdef.populateObjDefaults()
fmt.Println(testdef)
testdef2 := defaultObj{Zipcode: 90210}
testdef2.populateObjDefaults()
fmt.Println(testdef2)
testdef2.Name = "Mrs. Fred"
fmt.Println(testdef2)
testdef3 := defaultObj{}
fmt.Println(testdef3)
testdef3.populateObjDefaults()
fmt.Println(testdef3)
}
Output:
{Mr. Fred 12345 9.87654321e+08}
{Named Default 90210 9.87654321e+08}
{Mrs. Fred 90210 9.87654321e+08}
{ 0 0}
{Named Default 12345 9.87654321e+08}

You could use the ... operator.
instead of writing ToCall(a=b) like in python you write, ToCall("a",b)
See the Go Play Example
func GetKwds(kwds []interface{}) map[string]interface{} {
result := make(map[string]interface{})
for i := 0; i < len(kwds); i += 2 {
result[kwds[i].(string)] = kwds[i+1]
}
return result
}
func ToCall(kwds ...interface{}) {
args := GetKwds(kwds)
if value, ok := args["key"]; ok {
fmt.Printf("key: %#v\n", value)
}
if value, ok := args["other"]; ok {
fmt.Printf("other: %#v\n", value)
}
}
func main() {
ToCall()
ToCall("other", &map[string]string{})
ToCall("key", "Test", "other", &Object{})
}

Related

Create instance of struct via reflection and set values

what I try to do
I try to pass an instance of a struct - including json tags to a func, create a new instance, and set value on field
after this i try to serialize (JSON), but the values are empty
NOTICE: i looked up loads of articles on SO about setting values via reflection, but it seems i missed a little detail
struct definition
this part defines the struct with json and xml tags
type Person struct {
Name string `json:"Name" xml:"Person>FullName"`
Age int `json:"Age" xml:"Person>Age"`
}
create instance (+wrapping into empty interface)
afterwards I create an instance and store it in an interface{} - why? because in my production code this stuff will be done in a func which accepts a interface{}
var iFace interface{} = Person{
Name: "Test",
Age: 666,
}
creating a new instance of the struct and setting values via reflection
iFaceType := reflect.TypeOf(iFace)
item := reflect.New(iFaceType)
s := item.Elem()
if s.Kind() == reflect.Struct {
fName := s.FieldByName("Name")
if fName.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fName.CanSet() {
// change value of N
switch fName.Kind() {
case reflect.String:
fName.SetString("reflectedNameValue")
fmt.Println("Name was set to reflectedNameValue")
}
}
}
fAge := s.FieldByName("Age")
if fAge.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fAge.CanSet() {
// change value of N
switch fAge.Kind() {
case reflect.Int:
x := int64(42)
if !fAge.OverflowInt(x) {
fAge.SetInt(x)
fmt.Println("Age was set to", x)
}
}
}
}
}
Question
what am I doing wrong?
in production code I fill multiple copies with data and add it to a slice...
but this only makes sense if the json tags are kept in place and the stuff serializes just the same way.
code sample for play
package main
import (
"encoding/json"
"fmt"
"reflect"
)
func main() {
type Person struct {
Name string `json:"Name" xml:"Person>FullName"`
Age int `json:"Age" xml:"Person>Age"`
}
var iFace interface{} = Person{
Name: "Test",
Age: 666,
}
fmt.Println("normal: \n" + JSONify(iFace))
iFaceType := reflect.TypeOf(iFace)
item := reflect.New(iFaceType)
s := item.Elem()
if s.Kind() == reflect.Struct {
fName := s.FieldByName("Name")
if fName.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fName.CanSet() {
// change value of N
switch fName.Kind() {
case reflect.String:
fName.SetString("reflectedNameValue")
fmt.Println("Name was set to reflectedNameValue")
}
}
}
fAge := s.FieldByName("Age")
if fAge.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if fAge.CanSet() {
// change value of N
switch fAge.Kind() {
case reflect.Int:
x := int64(42)
if !fAge.OverflowInt(x) {
fAge.SetInt(x)
fmt.Println("Age was set to", x)
}
}
}
}
}
fmt.Println("reflected: \n" + JSONify(item))
}
func JSONify(v interface{}) string {
var bytes []byte
bytes, _ = json.MarshalIndent(v, "", "\t")
return string(bytes)
}
Your item is of type reflect.Value. You have to call Value.Interface() to obtain the value wrapped in it:
fmt.Println("reflected: \n" + JSONify(item.Interface()))
With this change, output will be (try it on the Go Playground):
normal:
{
"Name": "Test",
"Age": 666
}
Name was set to reflectedNameValue
Age was set to 42
reflected:
{
"Name": "reflectedNameValue",
"Age": 42
}
reflect.Value itself is also a struct, but obviously trying to marshal it will not be identical to marshaling a Person struct value. reflect.Value does not implement marshaling the wrapped data to JSON.

How do I access a method expression (struct function) from within another method expression in Go / Golang?

I am trying to clean up the code in a Go / Golang project of mine. I thought that it might be idiomatic to create a model in an object-oriented way such that I can do, for example:
db.Users.GetID("john") (to do a thing in the "users" table)
db.Purchases.GetTotal() (to do a thing in the "purchasaes" table)
Etc. However, one problem with this is this is that the database functions can't call themselves if they need to.
Here's a tiny, contrived example of what I am talking about:
package main
import "fmt"
// A model that contains all of the structs for our database tables
type Model struct {
Users
Purchases
}
// A struct for functions that have to do with the "users" table
type Users struct {}
func (self *Users) Exists(id int) bool {
// Check to see if the user exists
// SELECT id FROM users WHERE id = ?
return true // (omitted)
}
// A struct for functions that have to do with the "purchases" table
type Purchases struct {}
func (self *Purchases) Count(id int) int {
// First validate that the user exists
if db.Users.Exists(id) == false { // This fails <--------------
return 0
} else {
// Get the count of that user's purchases
// SELECT COUNT(user_id) FROM purchases WHERE user_id = ?
return 50 // (omitted)
}
}
func main() {
db := Model{}
numPurchases := db.Purchases.Count(123)
if numPurchases != 0 {
fmt.Println("User #123 has", numPurchases, "purchases!")
} else {
fmt.Println("User #123 does not exist!")
}
}
This results in an error:
undefined: db in db.Users
If I changed it to just Users.Exists instead of db.Users.Exists:
./test.go:22: invalid method expression Users.Exists (needs pointer receiver: (*Users).Exists)
./test.go:22: Users.Exists undefined (type Users has no method Exists)
Note that, in this contrived example, it is pointless to be validating that the user exists. However, the point is that a DB function should have the ability to call some other DB function in case there is something actually important to validate.
How can I accomplish / work around this?
(Edit - Modified the code snippet for clarity.)
What you are trying to do is not possible in Go. Users.Exists is a Method Expression. With these, you can take a method and convert it into a simple function type.
userExists := Users.Exists
user := User{}
// now userExists can be called as a regular function with
// first argument of a Users type
exists := userExists(&user, id)
As a result you cannot structure your model exactly like you have mentioned above.
Go is not a completely Object Oriented language and you should not try to replicate strictly OOP structures and idioms in it. In your case above, if you need an Exists function, you can always move Users into a separate package and define Exists as a function:
package user
type Users struct {}
// other methods on Users type
func Exists(id int) bool {
// check to see if user exists
}
You can now make the following calls:
import "user"
user.Exists(id)
After you initialize the struct, you can fill it with self references. Since I am new to Go, I'm not sure if it is considered idiomatic, but it seems to work just just fine.
The fixed code snippet:
package main
import "fmt"
// A model that contains all of the structs for our database tables
type Model struct {
Users
Purchases
}
// A struct for functions that have to do with the "users" table
type Users struct {
db *Model
}
func (self *Users) Exists(id int) bool {
// Check to see if the user exists
// SELECT id FROM users WHERE id = ?
return true // (omitted)
}
// A struct for functions that have to do with the "purchases" table
type Purchases struct {
db *Model
}
func (self *Purchases) Count(id int) int {
// First validate that the user exists
if self.db.Users.Exists(id) == false { // This succeeds now <--------------
return 0
} else {
// Get the count of that user's purchases
// SELECT COUNT(user_id) FROM purchases WHERE user_id = ?
return 50 // (omitted)
}
}
func main() {
// Initialize the model and fill it with helpful self-references
db := Model{}
db.Users.db = &db
db.Purchases.db = &db
// In practice, you will want to do this in some sort of custom constructor, like:
// var db Model*
// db := Model.NewModel(db)
numPurchases := db.Purchases.Count(123)
if numPurchases != 0 {
fmt.Println("User #123 has", numPurchases, "purchases!")
} else {
fmt.Println("User #123 does not exist!")
}
}
if you need field inside Users struct, you may use like db.Exists(2) or even Users.Exists(db, 2) both are valid. like this working sample code:
package main
import "fmt"
type Users struct {
Ids []int
}
func (t Users) Exists(id int) bool {
for _, v := range t.Ids {
if v == id {
return true
}
}
return false
}
func main() {
db := Users{[]int{1, 2}}
r1 := db.Exists(2)
r2 := Users.Exists(db, 2)
fmt.Println(r1, r2) // true true
}
you may use it like db.Exists(2) or even Users.Exists(db, 2)
both are valid. you may use Users struct just for grouping methods, like this working sample code:
package main
import "fmt"
type Users struct{}
var Ids = []int{1, 2}
func (t Users) Exists(id int) bool {
for _, v := range Ids {
if v == id {
return true
}
}
return false
}
func main() {
db := Users{}
r1 := db.Exists(2)
r2 := Users.Exists(db, 2)
fmt.Println(r1, r2) // true true
}
you may use UserExists(2) when you do not need grouping, like this working sample code:
package main
import "fmt"
type Users struct{}
var Ids = []int{1, 2}
func UserExists(id int) bool {
for _, v := range Ids {
if v == id {
return true
}
}
return false
}
func main() {
r1 := UserExists(2)
fmt.Println(r1) // true
}
so in your case this works fine:
package main
import "fmt"
type Model struct {
Users
Purchases
}
type Users struct{}
var users Users
func (*Users) Exists(id int) bool {
return true
}
type Purchases struct{}
func (*Purchases) Count(id int) int {
if users.Exists(id) == false {
return 0
} else {
return 50
}
}
func main() {
db := Model{}
numPurchases := db.Purchases.Count(123)
if numPurchases != 0 {
fmt.Println("User #123 has", numPurchases, "purchases!")
}
}
output:
User #123 has 50 purchases!

Is there a convenient way to convert a JSON-like map[string]interface{} to struct , and vice versa, with tags in Go? [duplicate]

I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}

Removing fields from struct or hiding them in JSON Response

I've created an API in Go that, upon being called, performs a query, creates an instance of a struct, and then encodes that struct as JSON before sending back to the caller. I'd now like to allow the caller to be able to select the specific fields they would like returned by passing in a "fields" GET parameter.
This means depending on the fields value(s), my struct would change. Is there any way to remove fields from a struct? Or at least hide them in the JSON response dynamically? (Note: Sometimes I have empty values so the JSON omitEmpty tag will not work here) If neither of these are possible, is there a suggestion on a better way to handle this?
A smaller version of the structs I'm using are below:
type SearchResult struct {
Date string `json:"date"`
IdCompany int `json:"idCompany"`
Company string `json:"company"`
IdIndustry interface{} `json:"idIndustry"`
Industry string `json:"industry"`
IdContinent interface{} `json:"idContinent"`
Continent string `json:"continent"`
IdCountry interface{} `json:"idCountry"`
Country string `json:"country"`
IdState interface{} `json:"idState"`
State string `json:"state"`
IdCity interface{} `json:"idCity"`
City string `json:"city"`
} //SearchResult
type SearchResults struct {
NumberResults int `json:"numberResults"`
Results []SearchResult `json:"results"`
} //type SearchResults
I then encode and output the response like so:
err := json.NewEncoder(c.ResponseWriter).Encode(&msg)
The question is asking for fields to be dynamically selected based on the caller-provided list of fields. This isn't possible to be done with the statically-defined json struct tag.
If what you want is to always skip a field to json-encode, then of course use json:"-" to ignore the field. (Note also that this is not required if your field is unexported; those fields are always ignored by the json encoder.) This isn't what the question asks.
To quote the comment on the json:"-" answer:
This [the json:"-" answer] is the answer most people ending up here from searching would want, but it's not the answer to the question.
I'd use a map[string]interface{} instead of a struct in this case. You can easily remove fields by calling the delete built-in on the map for the fields to remove.
That is, if you can't query only for the requested fields in the first place.
use `json:"-"`
// Field is ignored by this package.
Field int `json:"-"`
// Field appears in JSON as key "myName".
Field int `json:"myName"`
// Field appears in JSON as key "myName" and
// the field is omitted from the object if its value is empty,
// as defined above.
Field int `json:"myName,omitempty"`
// Field appears in JSON as key "Field" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Field int `json:",omitempty"`
doc : http://golang.org/pkg/encoding/json/#Marshal
Another way to do this is to have a struct of pointers with the ,omitempty tag. If the pointers are nil, the fields won't be Marshalled.
This method will not require additional reflection or inefficient use of maps.
Same example as jorelli using this method: http://play.golang.org/p/JJNa0m2_nw
You can use the reflect package to select the fields that you want by reflecting on the field tags and selecting the json tag values. Define a method on your SearchResults type that selects the fields you want and returns them as a map[string]interface{}, and then marshal that instead of the SearchResults struct itself. Here's an example of how you might define that method:
func fieldSet(fields ...string) map[string]bool {
set := make(map[string]bool, len(fields))
for _, s := range fields {
set[s] = true
}
return set
}
func (s *SearchResult) SelectFields(fields ...string) map[string]interface{} {
fs := fieldSet(fields...)
rt, rv := reflect.TypeOf(*s), reflect.ValueOf(*s)
out := make(map[string]interface{}, rt.NumField())
for i := 0; i < rt.NumField(); i++ {
field := rt.Field(i)
jsonKey := field.Tag.Get("json")
if fs[jsonKey] {
out[jsonKey] = rv.Field(i).Interface()
}
}
return out
}
and here's a runnable solution that shows how you would call this method and marshal your selection: http://play.golang.org/p/1K9xjQRnO8
I just published sheriff, which transforms structs to a map based on tags annotated on the struct fields. You can then marshal (JSON or others) the generated map. It probably doesn't allow you to only serialize the set of fields the caller requested, but I imagine using a set of groups would allow you to cover most cases. Using groups instead of the fields directly would most likely also increase cache-ability.
Example:
package main
import (
"encoding/json"
"fmt"
"log"
"github.com/hashicorp/go-version"
"github.com/liip/sheriff"
)
type User struct {
Username string `json:"username" groups:"api"`
Email string `json:"email" groups:"personal"`
Name string `json:"name" groups:"api"`
Roles []string `json:"roles" groups:"api" since:"2"`
}
func main() {
user := User{
Username: "alice",
Email: "alice#example.org",
Name: "Alice",
Roles: []string{"user", "admin"},
}
v2, err := version.NewVersion("2.0.0")
if err != nil {
log.Panic(err)
}
o := &sheriff.Options{
Groups: []string{"api"},
ApiVersion: v2,
}
data, err := sheriff.Marshal(o, user)
if err != nil {
log.Panic(err)
}
output, err := json.MarshalIndent(data, "", " ")
if err != nil {
log.Panic(err)
}
fmt.Printf("%s", output)
}
Take three ingredients:
The reflect package to loop over all the fields of a struct.
An if statement to pick up the fields you want to Marshal, and
The encoding/json package to Marshal the fields of your liking.
Preparation:
Blend them in a good proportion. Use reflect.TypeOf(your_struct).Field(i).Name() to get a name of the ith field of your_struct.
Use reflect.ValueOf(your_struct).Field(i) to get a type Value representation of an ith field of your_struct.
Use fieldValue.Interface() to retrieve the actual value (upcasted to type interface{}) of the fieldValue of type Value (note the bracket use - the Interface() method produces interface{}
If you luckily manage not to burn any transistors or circuit-breakers in the process you should get something like this:
func MarshalOnlyFields(structa interface{},
includeFields map[string]bool) (jsona []byte, status error) {
value := reflect.ValueOf(structa)
typa := reflect.TypeOf(structa)
size := value.NumField()
jsona = append(jsona, '{')
for i := 0; i < size; i++ {
structValue := value.Field(i)
var fieldName string = typa.Field(i).Name
if marshalledField, marshalStatus := json.Marshal((structValue).Interface()); marshalStatus != nil {
return []byte{}, marshalStatus
} else {
if includeFields[fieldName] {
jsona = append(jsona, '"')
jsona = append(jsona, []byte(fieldName)...)
jsona = append(jsona, '"')
jsona = append(jsona, ':')
jsona = append(jsona, (marshalledField)...)
if i+1 != len(includeFields) {
jsona = append(jsona, ',')
}
}
}
}
jsona = append(jsona, '}')
return
}
Serving:
serve with an arbitrary struct and a map[string]bool of fields you want to include, for example
type magic struct {
Magic1 int
Magic2 string
Magic3 [2]int
}
func main() {
var magic = magic{0, "tusia", [2]int{0, 1}}
if json, status := MarshalOnlyFields(magic, map[string]bool{"Magic1": true}); status != nil {
println("error")
} else {
fmt.Println(string(json))
}
}
Bon Appetit!
I created this function to convert struct to JSON string by ignoring some fields. Hope it will help.
func GetJSONString(obj interface{}, ignoreFields ...string) (string, error) {
toJson, err := json.Marshal(obj)
if err != nil {
return "", err
}
if len(ignoreFields) == 0 {
return string(toJson), nil
}
toMap := map[string]interface{}{}
json.Unmarshal([]byte(string(toJson)), &toMap)
for _, field := range ignoreFields {
delete(toMap, field)
}
toJson, err = json.Marshal(toMap)
if err != nil {
return "", err
}
return string(toJson), nil
}
Example: https://play.golang.org/p/nmq7MFF47Gp
You can use tagging attribute "omitifempty" or make optional fields pointers and leave those you want skipped uninitialized.
Here is how I defined my structure.
type User struct {
Username string `json:"username" bson:"username"`
Email string `json:"email" bson:"email"`
Password *string `json:"password,omitempty" bson:"password"`
FullName string `json:"fullname" bson:"fullname"`
}
And inside my function set user.Password = nil for not to be Marshalled.
I didn't have the same problem but similar. Below code solves your problem too, of course if you don't mind performance issue. Before implement that kind of solution to your system I recommend you to redesign your structure if you can. Sending variable structure response is over-engineering. I believe a response structure represents a contract between a request and resource and it should't be depend requests.(you can make un-wanted fields null, I do). In some cases we have to implement this design, if you believe you are in that cases here is the play link and code I use.
type User2 struct {
ID int `groups:"id" json:"id,omitempty"`
Username string `groups:"username" json:"username,omitempty"`
Nickname string `groups:"nickname" json:"nickname,omitempty"`
}
type User struct {
ID int `groups:"private,public" json:"id,omitempty"`
Username string `groups:"private" json:"username,omitempty"`
Nickname string `groups:"public" json:"nickname,omitempty"`
}
var (
tagName = "groups"
)
//OmitFields sets fields nil by checking their tag group value and access control tags(acTags)
func OmitFields(obj interface{}, acTags []string) {
//nilV := reflect.Value{}
sv := reflect.ValueOf(obj).Elem()
st := sv.Type()
if sv.Kind() == reflect.Struct {
for i := 0; i < st.NumField(); i++ {
fieldVal := sv.Field(i)
if fieldVal.CanSet() {
tagStr := st.Field(i).Tag.Get(tagName)
if len(tagStr) == 0 {
continue
}
tagList := strings.Split(strings.Replace(tagStr, " ", "", -1), ",")
//fmt.Println(tagList)
// ContainsCommonItem checks whether there is at least one common item in arrays
if !ContainsCommonItem(tagList, acTags) {
fieldVal.Set(reflect.Zero(fieldVal.Type()))
}
}
}
}
}
//ContainsCommonItem checks if arrays have at least one equal item
func ContainsCommonItem(arr1 []string, arr2 []string) bool {
for i := 0; i < len(arr1); i++ {
for j := 0; j < len(arr2); j++ {
if arr1[i] == arr2[j] {
return true
}
}
}
return false
}
func main() {
u := User{ID: 1, Username: "very secret", Nickname: "hinzir"}
//assume authenticated user doesn't has permission to access private fields
OmitFields(&u, []string{"public"})
bytes, _ := json.Marshal(&u)
fmt.Println(string(bytes))
u2 := User2{ID: 1, Username: "very secret", Nickname: "hinzir"}
//you want to filter fields by field names
OmitFields(&u2, []string{"id", "nickname"})
bytes, _ = json.Marshal(&u2)
fmt.Println(string(bytes))
}
I also faced this problem, at first I just wanted to specialize the responses in my http handler. My first approach was creating a package that copies the information of a struct to another struct and then marshal that second struct. I did that package using reflection, so, never liked that approach and also I wasn't dynamically.
So I decided to modify the encoding/json package to do this. The functions Marshal, MarshalIndent and (Encoder) Encode additionally receives a
type F map[string]F
I wanted to simulate a JSON of the fields that are needed to marshal, so it only marshals the fields that are in the map.
https://github.com/jtorz/jsont
package main
import (
"fmt"
"log"
"net/http"
"github.com/jtorz/jsont/v2"
)
type SearchResult struct {
Date string `json:"date"`
IdCompany int `json:"idCompany"`
Company string `json:"company"`
IdIndustry interface{} `json:"idIndustry"`
Industry string `json:"industry"`
IdContinent interface{} `json:"idContinent"`
Continent string `json:"continent"`
IdCountry interface{} `json:"idCountry"`
Country string `json:"country"`
IdState interface{} `json:"idState"`
State string `json:"state"`
IdCity interface{} `json:"idCity"`
City string `json:"city"`
} //SearchResult
type SearchResults struct {
NumberResults int `json:"numberResults"`
Results []SearchResult `json:"results"`
} //type SearchResults
func main() {
msg := SearchResults{
NumberResults: 2,
Results: []SearchResult{
{
Date: "12-12-12",
IdCompany: 1,
Company: "alfa",
IdIndustry: 1,
Industry: "IT",
IdContinent: 1,
Continent: "america",
IdCountry: 1,
Country: "México",
IdState: 1,
State: "CDMX",
IdCity: 1,
City: "Atz",
},
{
Date: "12-12-12",
IdCompany: 2,
Company: "beta",
IdIndustry: 1,
Industry: "IT",
IdContinent: 1,
Continent: "america",
IdCountry: 2,
Country: "USA",
IdState: 2,
State: "TX",
IdCity: 2,
City: "XYZ",
},
},
}
fmt.Println(msg)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
//{"numberResults":2,"results":[{"date":"12-12-12","idCompany":1,"idIndustry":1,"country":"México"},{"date":"12-12-12","idCompany":2,"idIndustry":1,"country":"USA"}]}
err := jsont.NewEncoder(w).Encode(msg, jsont.F{
"numberResults": nil,
"results": jsont.F{
"date": nil,
"idCompany": nil,
"idIndustry": nil,
"country": nil,
},
})
if err != nil {
log.Fatal(err)
}
})
http.ListenAndServe(":3009", nil)
}
The question is now a bit old, but I came across the same issue a little while ago, and as I found no easy way to do this, I built a library fulfilling this purpose.
It allows to easily generate a map[string]interface{} from a static struct.
https://github.com/tuvistavie/structomap
To extend chhaileng answer, here is the version that remove all occurrences of a field with recursion
// GetJSONWithOutFields - Description: return a string representation of an interface with specified fields removed
func GetJSONWithOutFields(obj interface{}, ignoreFields ...string) (string, error) {
toJson, err := json.Marshal(obj)
if err != nil {
return "", err
}
if len(ignoreFields) == 0 {
return string(toJson), nil
}
toMap := map[string]interface{}{}
err = json.Unmarshal(toJson, &toMap)
if err != nil {
return "", err
}
for _, field := range ignoreFields {
DeleteField(toMap, field)
}
toJson, err = json.Marshal(toMap)
if err != nil {
return "", err
}
return string(toJson), nil
}
// DeleteField - Description: recursively delete field
func DeleteField(toMap map[string]interface{}, field string) {
delete(toMap, field)
for _, v := range toMap {
if m, isMap := v.(map[string]interface{}); isMap {
DeleteField(m, field)
}
}
}

Dynamically create variables of certain type based on string in Go

Simple version
How can you create a variable of a certain type based upon the value of a string?
type ta struct { a int }
type tb struct { b float }
type tc struct { c string }
t := "tb"
v := MagicVarFunc(t) // Returns a new allocated var of type interface{}
v.(tb).b = 8.3
The true example
In my, surprisingly enough, working example below, I am dynamically creating variables based on a string. This is done by registering each struct type in a map with the string being the key and a nil-pointer of the type being the value.
Each type implements an interface with the method New() which returns a new variable of that specific type.
The example below is very close to what I wish to do, where each action has a set of JSON encoded data which will populate the corresponding struct. The way I've structured it is also because I wish to be able to create new stand alone actions that I register to the map.
I am not sure if am abusing the language now.
May anyone give me any pointers if I am completely out of my mind? Is there an obviously easier way?
package main
import (
"fmt"
"encoding/json"
)
// All I require of an action is that it may be executed
type ActionHandler interface {
Exec()
New() ActionHandler
}
// My list of actions
var mActions = make(map[string]ActionHandler)
// Action Exit (leaving the program)
type aExit struct {}
func (s *aExit) Exec() { fmt.Println("Good bye") }
func (s *aExit) New() ActionHandler { return new(aExit) }
func init() {
var a *aExit
mActions[`exit`] = a
}
// Action Say (say a message to someone)
type aSay struct {
To string
Msg string
}
func (s *aSay) Exec() { fmt.Println(`You say, "` + s.Msg + `" to ` + s.To) }
func (s *aSay) New() ActionHandler { return new(aSay) }
func init() {
var a *aSay
mActions[`say`] = a
}
func inHandler(action string, data []byte) {
a := mActions[action].New()
json.Unmarshal(data, &a)
a.Exec()
}
func main(){
inHandler(`say`, []byte(`{"to":"Sonia","msg":"Please help me!"}`))
inHandler(`exit`, []byte(`{}`))
}
You can use reflection to get the zero value of, or to allocate a new value (like new) of a type using reflection, if you can get the Type value at runtime. However, I don't think there is a way to get the Type from a string. You would need to have a value of that type to get the type itself.
I adopted your idea, of using a map. I map the string to the type itself, which you can get using reflect.TypeOf, which gets the type out of an interface value. Then I used reflect.Zero to get the zero value of that type (a convenient value that exists for every type). Then I got the value out as an interface.
package main
import "reflect"
type ta struct { a int }
type tb struct { b float64 }
type tc struct { c string }
var mActions map[string]reflect.Type = make(map[string]reflect.Type)
func init() {
var a ta
mActions[`ta`] = reflect.TypeOf(a)
var b tb
mActions[`tb`] = reflect.TypeOf(b)
var c ta
mActions[`tc`] = reflect.TypeOf(c)
}
func MagicVarFunc(action string) interface{} {
return reflect.Zero(mActions[action]).Interface()
}
func main() {
t := "tb"
v := MagicVarFunc(t) // Returns a new allocated var of type interface{}
x := v.(tb)
x.b = 8.3
}
jorelli's answer is very good. I'm just going to show a few options. Your "true example" looks essentially like command dispatch, with command parameters specified with JSON. To start with simple code that does this,
package main
import (
"encoding/json"
"fmt"
)
func inHandler(action string, data []byte) {
arg := make(map[string]interface{})
json.Unmarshal(data, &arg)
switch action {
case "say":
fmt.Printf("You say, %q to %s\n", arg["msg"], arg["to"])
case "exit":
fmt.Println("Good bye")
}
}
func main() {
inHandler(`say`, []byte(`{"to":"Sonia","msg":"Please help me!"}`))
inHandler(`exit`, []byte(`{}`))
}
Your register new commands by adding cases to the switch statement. Yeah, didn't think you'd like that. So, adding your map and init() idea,
package main
import (
"encoding/json"
"fmt"
)
type jmap map[string]interface{}
var mActions = map[string]func(jmap){}
func init() {
mActions["say"] = func(arg jmap) {
fmt.Printf("You say, %q to %s\n", arg["msg"], arg["to"])
}
}
func init() {
mActions["exit"] = func(jmap) { fmt.Println("Good bye") }
}
func inHandler(action string, data []byte) {
args := make(jmap)
json.Unmarshal(data, &args)
mActions[action](args)
}
func main() {
inHandler(`say`, []byte(`{"to":"Sonia","msg":"Please help me!"}`))
inHandler(`exit`, []byte(`{}`))
}
Now if you wanted, you could put each of those init functions in a separate source file and new commands could be registered by creating a new source file with a new init function.
The rest of the program is simplified with some assumptions that the commands have flat argument lists that the JSON will always encode as an object. This allows you to dispense with separate Go struct definitions for each command. inHandler just creates the same type of object (a map) for all commands, unmarshals into it, and passes it to the command. If you wanted to handle a little more arbitrary JSON, you could unmarshal into an empty interface, and the functions would have to do some extra work to dig out the arguments. If that was too much work and you really wanted to unmarshal directly into a struct, then you arrive near jorelli's solution of making each command function unmarshal its own JSON.
start by defining a function type that does the thing you want:
type Producer func([]byte) interface{}
make a few of them:
func FooProducer(raw []byte) interface{} {
foo := new(Foo)
... // do something to foo
return foo
}
func BarProducter(raw []byte) interface{} {
bar := new(Bar)
... // do something to bar
return bar
}
stick them in a map:
likeThis := map[string]Producer{
"foo": FooProducer,
"bar": BarProducer,
}
and then just do one of these:
myVal := likeThis[someString](raw)
but you probably want to define some interface and make your producer something more like:
type Producer func([]byte) MyAwesomeInterface
since there's probably some common stuff you want to do with those things you're decoding. You also probably want to handle the case of a bad string input, like-a-this:
f, ok := likeThis[someString]
if !ok {
// return, break, panic... something, just get the hell away from here.
}
myVal := f(raw)
The whole notion of inspecting types is kinda cumbersome in Go. It's generally less work to just add new types than it is to try to do reflection gymnastics with the type system.