I'm trying to customize the output of MarshalJSON, using the interface:
func (m *RawMessage) MarshalJSON() ([]byte, error)
I followed that tutorial: http://choly.ca/post/go-json-marshalling/
My purpose is removing replace one of the fields with true/false (if set or not), so I ended up writing that function:
func (u *Edition) MarshalJSON() ([]byte, error) {
var vaultValue bool
vaultValue = true
var onlineValue bool
vaultValue = false
fmt.Println("here")
if u.Vault == nil {
vaultValue = false
}
if u.Online == nil {
onlineValue = false
}
type AliasEdition Edition
return json.Marshal(&struct {
Vault bool `json:"vault,omitempty"`
Online bool `json:"online,omitempty"`
*AliasEdition
}{
Vault: vaultValue,
Online: onlineValue,
AliasEdition: (*Alias)(u),
})
}
The JSON is created from a map with the following instruction:
json.NewEncoder(w).Encode(EditionsMap)
Obviously EditionsMap is a Map of Editions structures:
var EditionsMap map[string]datamodel.Edition
The problem is that the MarshalJSON function apparently is never called.
Probably I'm doing something wrong, but I cannot understand what is the problem, my understanding is that I just need to implement that function in order to get it called.
This is because you declared the Edition.MarshalJSON() method with pointer receiver:
func (u *Edition) MarshalJSON() ([]byte, error)
And you try to marshal non-pointer values (your map contains datamodel.Edition values):
var EditionsMap map[string]datamodel.Edition
// ...
json.NewEncoder(w).Encode(EditionsMap)
Methods with pointer receiver are not part of the method set of the corresponding non-pointer type. The method set of type datamodel.Edition does not contain the method MarshalJSON().
Spec: Method sets:
A type may have a method set associated with it. The method set of an interface type is its interface. The method set of any other type T consists of all methods declared with receiver type T. The method set of the corresponding pointer type *T is the set of all methods declared with receiver *T or T (that is, it also contains the method set of T).
Try to marshal pointer values, define your map to contain pointers:
var EditionsMap map[string]*datamodel.Edition
// ...
if err := json.NewEncoder(w).Encode(EditionsMap); err != nil {
panic(err) // HANDLE error somehow, do not omit it like in your example!
}
Values of the pointer type *Edition does have a method MarshalJSON() which will be called properly by the json package. Try a working example of this on the Go Playground.
Another option would be to define the Edition.MarshalJSON() method with value receiver:
func (u Edition) MarshalJSON() ([]byte, error)
And this way it would work no matter if you marshal pointer or non-pointer values, as the methods with value receiver are part of the method set of both the Edition type and the corresponding *Edition pointer type. Try a working example of this variant on the Go Playground.
Related
I was reading this following article:
https://www.ribice.ba/golang-enums/
There is a function defined in one of the code samples:
func (lt *LeaveType) UnmarshalJSON(b []byte) error {
// Define a secondary type to avoid ending up with a recursive call to json.Unmarshal
type LT LeaveType;
var r *LT = (*LT)(lt);
err := json.Unmarshal(b, &r)
if err != nil{
panic(err)
}
switch *lt {
case AnnualLeave, Sick, BankHoliday, Other:
return nil
}
return errors.New("Inalid leave type")
}
What is the syntax var r *LT = (*LT)(lt); doing in this example?
Go technically does not have casts but rather conversions. The syntax for an explicit conversion is T(x) where T is some type and x is some value that is convertible to that type. See Conversions in the Go specification for details.
As you can see from the function's declaration:
func (lt *LeaveType) UnmarshalJSON(b []byte) error {
lt itself has type pointer to LeaveType and UnmarshalJSON is a receiver function for type *LeaveType. The encoding/json package will call such a function to decode input JSON when the variable that the package would like to set has type LeaveType (or *LeaveType—the package will create the LeaveType variable itself in this case).
As the comment in the code says, the author of the code would now like to have the encoding/json code unmarshal the JSON as if there weren't a function UnmarshalJSON. But there is a function UnmarshalJSON, so if we just invoke the encoding/json code without a little bit of trickery, encoding/json will just call this function again, leading to infinite recursion.
By defining a new type LT whose contents are exactly the same as the existing type LeaveType, we end up with a new type that does not have a receiver function. Invoking the encoding/json on an instance of this type (or of a pointer to this type) won't call the *LeaveType receiver, because LT is a different type, even though its contents match up exactly.
We could do this:
func (lt *LeaveType) UnmarshalJSON(b []byte) error {
type LT LeaveType
var r LT
err := json.Unmarshal(b, &r)
if err != nil {
panic(err)
}
// ...
}
This would fill in r, which has the same size and shape as any LeaveType variable. Then we could use the filled-in r to set *lt:
*lt = LeaveType(r) // an ordinary conversion
after which we could keep going as before, using *lt as the value. But this means that UnmarshalJSON had to set a temporary variable r, which we then had to copy to its final destination. Why not, instead, set up something so that UnmarshalJSON fills in the target variable, but using the type we chose?
That's what the syntax here is for. It's not the shortest version: as Cerise Limón noted, there is a shorter way to spell it (and that shorter spelling is generally preferred). The first set of parentheses in (*LT)(lt) is required to bind the *—the pointer to part—to the LT, as *LT(lt) has the wrong binding: it means the same thing as *(LT(lt)) which is not what we want.
The expression (*LT)(lt) is a conversion to type *LT.
The statement var r *LT = (*LT)(lt); declares variable r as type *LT with initial value (*LT)(lt). The statement can be written more simply as r := (*LT)(lt). There's no need to mention the type twice or to end the line with a semicolon.
The function declares type LT with empty method set to avoid a recursive call to UnMarshalJSON.
json.Unmarshal() unmarshals some JSON text into a Go value. If the value to unmarshal into implements the json.Unmarshaler interface, its UnmarshalJSON() method is called which allows to implement custom unmarshaling logic.
Quoting from json.Unmarshal():
To unmarshal JSON into a value implementing the Unmarshaler interface, Unmarshal calls that value's UnmarshalJSON method, including when the input is a JSON null.
The json.Unmarshaler interface:
type Unmarshaler interface {
UnmarshalJSON([]byte) error
}
LeaveType (or more specifically *LeaveType) has an UnmarshalJSON() method which we can see in the question, so it implements json.Unmarshaler.
And the LeaveType.UnmarshalJSON() method wishes to use the default unmarshaling logic which does the "hard" part, and just wants to make some final adjustments. So it calls json.Unmarshal():
err := json.Unmarshal(b, &r)
If we would pass lt to unmarshal into, –since lt implements json.Unmashaler–LeaveType.UnmarshalJSON() would be called by the json package, effectively causing an infinite "recursion".
Of course, this is not what we want. In order to avoid the infinite recursion, we have to pass a value that does not implement json.Unmarshaler, a value whose type does not have an UnmarshalJSON() method.
This is where creating a new type comes into the picture:
type LT LeaveType
The type keyword creates a new type called LT which is distinct from LeaveType. It does not "inherit" any of LeaveType's methods, so LT does not implement json.Unmarshaler. So if we pass a value of LT or *LT to json.Unmarshal(), it will not result in LeaveType.UnmarshalJSON() to be called (by the json package).
var r *LT = (*LT)(lt)
This declares a variable named r, whose type is *LT. And it assigns the value lt converted to *LT. The conversion is needed because lt is of type *LeaveType, so it cannot be assigned to a variable of type *LT, but since LT has LeaveType as its underlying type, *LeaveType is convertible to *LT.
So r is a pointer, it points to the same value as lt, it has the same memory layout. So if we use the default unmarshaling logic and "populate" the struct pointed by r, then the "same" struct pointed by lt will be populated.
See related / similar question: Call json.Unmarshal inside UnmarshalJSON function without causing stack overflow
It's casting lt, a LeaveType pointer, to an LT pointer.
LT is defined just above by type LT LeaveType; to be equivalent to LeaveType.
It's doing this for the reasons explained in the comment.
// Define a secondary type to avoid ending up with a recursive call to json.Unmarshal
Whether this is effective or necessary, I don't know.
You can see the same effect in play with a simply Stringer interface example, where the fmt.Println function will try to marshal data into string format. If a given value's type has a String() method, it will be used in preference to reflection.
This implementation fails (and go vet issues a warning) as it causes infinite recursion:
type mystring string
func (ms mystring) String() string {
return fmt.Sprintf("mystring: %s", ms)
}
This version is essential what the original code is doing:
type mystring2 string
func (ms mystring2) String() string {
type mystring2 string // <- local type mystring2 overrides global type
v := mystring2(ms)
return fmt.Sprintf("mystring2: %s", v)
}
Remove the type mystring2 string line and see what happens.
I have a parent struct :
type BigPoly struct{
Value []*ring.Poly
}
And two child structs :
type Plaintext BigPoly
type Ciphertext BigPoly
I would like to have functions accepting both Plaintext and Ciphertext. My solution is to use a function of the form :
func Add(a *Ciphertext, b interface{}) (*Ciphertext)
and use a switch-case to decide what to do, but I find it troublesome and it can lead to very complicated cases if the number of inputs grows.
However since Plaintext and Ciphertext have exactly the same structure and internal variables and only differ in their name, is it possible to create a function accepting both Plaintext and Ciphertext in a cleaner way ? I.e. it doesn't care if it is a type Plaintext or Ciphertext, as long as it is a type BigPoly.
Use a non-empty interface:
type Poly interface {
Value() []*ring.Poly
}
Then define your struct as:
type BigPoly struct{
value []*ring.Poly
}
func (p *BigPoly) Value() []*ring.Poly {
return p.value
}
And your consumer as:
func Add(a, b Poly) Poly {
aValue := a.Value()
bValue := b.Value()
// ... do something with aValue and bValue
}
In Golang, I have a struct whose member is a custom int type with constant values. Basically, the custom type is a logical enum.
type Flavor int
const (
Vanilla Flavor = iota
Chocolate
Strawberry
)
func (f *Flavor) MarshalJSON() ([]byte, error) {
return []byte(strconv.Quote(f.String())), nil
}
The custom type has defined MarshalJSON and UnmarshalJSON functions so when I serialize the custom type to JSON, I expect to get the string of the value in the serialized output, not the int value.
My issue is that if I have a pointer to a containing type, then the containing type marshals using the custom function but if try to marshal with just a struct value, the custom MarshalJSON is not invoked by the JSON package
type Dessert struct {
Flavor Flavor `json:"flavor"`
Count int
}
....
d := Dessert{Strawberry, 13}
b, err = json.Marshal(d) // !! does not invoke members Marshal !!
b, err = json.Marshal(&d) // works as expected
....
produces
{"flavor":2,"Count":13}
{"flavor":"Strawberry","Count":13}
I expected the second output in both case.
Why does passing a struct value not invoke MarshalJSON on the member but it does encode otherwise correct JSON?
see https://play.golang.org/p/mOl1GHhgynf
for full working code
In your code Flavor does not have a method MarshalJSON as you defined the method for *Flavor only.
If you want type Flavor to have the MarshalJSON method you must define it on Flavor not *Flavor.
oh huh. I think you had it Volker and Leon. I had assumed that I needed a pointer receiver for MarshalJSON since UnmarshalJSON definitely needs a pointer receiver. But
func (f Flavor) MarshalJSON() ([]byte, error) {
...
func (f *Flavor) UnmarshalJSON(b []byte) error {
...
and mixing the receivers causes the expected output for both json.Marshal(d) and json.Marshal(&d)
An object of any type can be assigned to an empty interface. For example, we have the following function
func Println(i interface{} ) {
fmt.Println(i)
}
We can call it by
Println(3)
Println(1.5)
Println("Hello")
But I can't achieve the same thing for function type
func Map(fn func( interface{} )) {
......
}
I cannot call this with
Map( func( i int) {......} )
because the type func(int) is different from the type func( interface{} ).
But when I define func( interface{} ), I really mean any type of the input parameters. How can I achieve this in Go?
It fails because the signatures don't match.
When you call Println(3), the function isn't taking an integer as its first argument. Rather the integer gets packed inside an interface{} variable (an automatic conversion, since integers conform to the interface), and that variable is passed to the function. This conversion happens on the calling side, so the process of calling the function is different to calling a function matching func(i int).
If you want to write a function that accepts arbitrary unary functions, you will need to declare it to take an interface{} variable as its argument and then check the value using the reflect package. The reflect package can also help you call arbitrary functions where you don't know the signature at compile time.
For example:
func Map(f, v interface{}) interface{} {
fn := reflect.ValueOf(f)
fnType := fn.Type()
if fnType.Kind() != reflect.Func || fnType.NumIn() != 1 || fnType.NumOut() != 1 {
panic("Expected a unary function returning a single value")
}
res := fn.Call([]reflect.Value{reflect.ValueOf(v)})
return res[0].Interface()
}
This will call the given function f with the argument v and return the result. Provided v is assignable to f's first argument the call will succeed without a panic. You can experiment with this example here: http://play.golang.org/p/kkBu56JYb8
I do realise its an old discussion, but came across the post and wanted to play around with the concept of having arbitrary function func (interface{}) within another function, instead of interface{}.
I could write a simple implementation, by providing an inline implementation of a function which would accept interface{}. And we can call this function from within another function
varForGenFunc := func(in interface{}) int {
fmt.Println("type of this object: ",reflect.TypeOf(in))
return 1}
TakeGenericFunc(varForGenFunc, variableForGen)
Going by this, we can write any implementations of func(interface{}) and pass it as parameter to TakeGenericFunc
You can play around with it here:
https://play.golang.org/p/f5UUhyhEx7u
Is there a technical reason why unexported fields are not included by encoding/json? If not and it is an arbitrary decision could there be an additional back door option (say '+') to include even though unexported?
Requiring client code to export to get this functionality feels unfortunate, especially if lower case is providing encapsulation or the decision to marshal structures comes much later than design of them.
How are people dealing with this? Just export everything?
Also, doesn't exporting field names make it difficult to follow suggested idioms. I think if a struct X has field Y, you can not have an accessor method Y(). If you want to provide interface access to Y you have to come up with a new name for the getter and no matter what you'll get something un-idiomatic according to http://golang.org/doc/effective_go.html#Getters
There is a technical reason. The json library does not have the power to view fields using reflect unless they are exported. A package can only view the unexported fields of types within its own package
In order to deal with your problem, what you can do is make an unexported type with exported fields. Json will unmarshal into an unexported type if passed to it without a problem but it would not show up in the API docs. You can then make an exported type that embeds the unexported type. This exported type would then need methods to implement the json.Marshaler and json.Unmarshaler interfaces.
Note: all code is untested and may not even compile.
type jsonData struct {
Field1 string
Field2 string
}
type JsonData struct {
jsonData
}
// Implement json.Unmarshaller
func (d *JsonData) UnmarshalJSON(b []byte) error {
return json.Unmarshal(b, &d.jsonData)
}
// Getter
func (d *JsonData) Field1() string {
return d.jsonData.Field1
}
Stephen's answer is complete. As an aside, if all you really want is lowercase keys in your json, you can manually specify the key name as follows:
type Whatever struct {
SomeField int `json:"some_field"`
}
In that way, marshaling a Whatever produces the key "some_field" for the field SomeField (instead of having "SomeField" in your json).
If you're dead-set on keeping unexported fields, you can also implement the json.Marshaler interface by defining a method with the signature MarshalJSON() ([]byte, error). One way to do this is to use a struct literal that simply has exported versions of the unexported fields, like this:
type Whatever struct {
someField int
}
func (w Whatever) MarshalJSON() ([]byte, error) {
return json.Marshal(struct{
SomeField int `json:"some_field"`
}{
SomeField: w.someField,
})
}
That can be a bit cumbersome, so you can also use a map[string]interface{} if you prefer:
func (w Whatever) MarshalJSON() ([]byte, error) {
return json.Marshal(map[string]interface{}{
"some_field": w.SomeField,
})
}
However it should be noted that marshaling interface{} has some caveats and can do things like marshal uint64 to a float, causing a loss of precision. (all code untested)