Custom MarshalJSON() never gets called in Go - json

I've written custom versions of MarshalJSON and UnmarshalJSON. My UnmarshalJSON gets called the way I want it to, but I can't get it to work with MarshalJSON. Here's code that summarizes my problem:
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
"os"
)
type myStruct struct {
Data string `json:"data"`
}
func (s *myStruct) MarshalJSON() ([]byte, error) {
return []byte(`{"data":"charlie"}`), nil
}
func (s *myStruct) UnmarshalJSON(b []byte) error {
// Insert the string directly into the Data member
return json.Unmarshal(b, &s.Data)
}
func main() {
// Create a struct with initial content "alpha"
ms := myStruct{"alpha"}
// Replace content with "bravo" using custom UnmarshalJSON() (SUCCESSFUL)
if err := json.NewDecoder(bytes.NewBufferString(`"bravo"`)).Decode(&ms); err != nil {
log.Fatal(err)
}
// Use custom MarshalJSON() to get "charlie" back (UNSUCCESSFUL)
if err := json.NewEncoder(os.Stdout).Encode(ms); err != nil {
log.Fatal(err)
}
// Trying another method (UNSUCCESSFUL)
if ret, err := json.Marshal(ms); err != nil {
log.Fatal(err)
} else {
fmt.Println(string(ret))
}
// Verify that the Marshaler interface is correctly implemented
var marsh json.Marshaler
marsh = &ms
ret, _ := marsh.MarshalJSON()
fmt.Println(string(ret)) // Prints "charlie"
}
In short, the program encodes the struct "automatically" in two ways, and then finally calls MarshalJSON manually. The response I want is "charlie". Running the code generates the following output:
{"data":"bravo"}
{"data":"bravo"}
{"data":"charlie"}
Try it at Go Playground: http://play.golang.org/p/SJ05S8rAYN

In this part of the code, ms gets copied into an interface{} variable:
// Trying another method (UNSUCCESSFUL)
if ret, err := json.Marshal(ms); err != nil {
The problem is that this variable does not implement the json.Marshaler interface, since MarshalJSON is not in the method set for myStruct (only for *myStruct).
The fix is to either (a) make your MarshalJSON method take a non-pointer receiver (which will mean it gets a copy of the struct: possibly costly if it is large), or (b) marshal a pointer to the struct (as Kavu mentioned in a comment).
The reason for this behaviour is that Go doesn't let you take a pointer to the value stored inside an interface variable, instead requiring you to make a copy of the value whenever you want to access it. While the language has syntactic sugar to convert ms.MarshalJSON() into (&ms).MarshalJSON() as a way to access the method with a pointer receiver, this can not be done for a value stored in an interface variable. For this reason, the method is not considered to be in its method set.

Related

json.Unmarshal interface pointer with later type assertion

Because I often unmarshal http.Response.Body, I thought I could write a function which handles all the hassle of reading, closing and unmarshaling into various different structs. That's why I introduced a function func unmarhalInterface(closer *io.ReadCloser, v *interface{}) error and can then assert the return value with t:=i.(T).
According to this answer I already wrapped it into a value of type *interface{}, but because the overlying type is interface{}and not myStruct, the json package implementation chooses map[string]interface{}. After that a type assertion fails (of course). Is there anything i am missing or requires this implementation a type assertion "by hand", that means look for all fields in the map and assign those that I want into my struct.
Code below has minimal example with notation in comments. If my explanation is not sufficient, please ask away.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
)
type myStruct struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
jsonBlob := []byte(`{"a":"test","b":"test2"}`)
var foo = interface{}(myStruct{})
closer := ioutil.NopCloser(bytes.NewReader(jsonBlob))
err := unmarshalCloser(&closer, &foo)
if err != nil {
log.Fatal(err)
}
fmt.Println(fmt.Sprintf("%v", foo))
// That´s what i want:
foo2 := foo.(myStruct)
fmt.Println(foo2.A)
}
func unmarshalCloser(closer *io.ReadCloser, v *interface{}) error {
defer func() { _ = (*closer).Close() }()
data, err := ioutil.ReadAll(*closer)
if err != nil {
return err
}
err = json.Unmarshal(data, v)
if err != nil {
return err
}
return nil
}
Golang Playground
An empty interface isn't an actual type, it's basically something that matches anything. As stated in the comments, a pointer to an empty interface doesn't really make sense as a pointer already matches an empty interface since everything matches an empty interface. To make your code work, you should remove the interface wrapper around your struct, since that's just messing up the json type checking, and the whole point of an empty interface is that you can pass anything to it.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
)
type myStruct struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
jsonBlob := []byte(`{"a":"test","b":"test2"}`)
var foo = &myStruct{} // This need to be a pointer so its attributes can be assigned
closer := ioutil.NopCloser(bytes.NewReader(jsonBlob))
err := unmarshalCloser(closer, foo)
if err != nil {
log.Fatal(err)
}
fmt.Println(fmt.Sprintf("%v", foo))
// That´s what i want:
fmt.Println(foo.A)
}
// You don't need to declare either of these arguments as pointers since they're both interfaces
func unmarshalCloser(closer io.ReadCloser, v interface{}) error {
defer closer.Close()
// v NEEDS to be a pointer or the json stuff will barf
// Simplified with the decoder
return json.NewDecoder(closer).Decode(v)
}

Golang JSON decoder does not recognize type

Still a Golang beginner, I am trying to code a generic function to serve ReST requests. I pass a function to create a new resource (struct) with an interface implemented on it, because I would also invoke methods on the struct. When decoding the JSON, logging the type shows the correct (struct) type, but the JSON decoder seems to only recognize the interface, which it cannot decode to.
package main
import (
"encoding/json"
"github.com/julienschmidt/httprouter"
"log"
"net/http"
"strings"
)
// general resource interface
type resource interface {
// check semantics and return an array of errors or nil if no error found
check() []string
// update the resource in backend
update() error
}
// specific resource named "anchor"
type anchor struct {
ID string `json:"id"`
Name string `json:"name"`
}
func newAnchor() resource {
return anchor{}
}
func (a anchor) check() []string {
return nil
}
func (a anchor) update() error {
return nil
}
// generic function to create (POST) a new resource
func restCreate(newResource func() resource) httprouter.Handle {
return func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
const F = "restCreate"
var checkErrs []string
res := newResource()
log.Printf("%s res type %T\n", F, res)
dcdr := json.NewDecoder(r.Body)
err := dcdr.Decode(&res)
log.Printf("%s Unmarshalled into %T: %+v\n", F, res, res)
if err == nil {
checkErrs = res.check()
}
switch {
case err != nil:
w.WriteHeader(http.StatusInternalServerError)
log.Printf("[ERR] %s: %v\n", F, err)
case checkErrs != nil:
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte(strings.Join(checkErrs, "\n")))
log.Printf("%s: %v\n", F, err)
default:
res.update()
bs, _ := json.Marshal(res)
w.Write(bs)
}
}
}
func main() {
r := httprouter.New()
r.POST("/anchors", restCreate(newAnchor))
http.ListenAndServe(":8080", r)
}
The execution log shows:
restCreate res type main.anchor
restCreate Unmarshalled into main.anchor: {ID: Name:}
[ERR] restCreate: json: cannot unmarshal object into Go value of type main.resource
Why does Printf show the struct type and json.Decoder the interface?
I'd appreciate any indicator on what's going wrong and how to solve this in a generic way...
It is because you try to use a pointer to the interface to unmarshal into. You need return a pointer in a function
func newAnchor() resource {
return &anchor{}
}
And you don't need to get address in this line:
err := dcdr.Decode(&res)
Here is small working example: https://play.golang.org/p/3E0RmGTURO
You cannot Unmarshal into an interface unless variable is holding a pointer to the concrete type you desire, as json.Decode won't know which concrete type to use. You have two workarounds at your disposal:
Having newResource return a concrete type under the hood:
func newResource() resource {
return &anchor{}
}
This way json.Decode knows to unmarshal your JSON into an anchor.
Use newAnchor instead of newResource: this will be more readable in your restCreate function, and is more idiomatic[1].
[1] http://idiomaticgo.com/post/best-practice/accept-interfaces-return-structs/

Can someone explain this interface example in Go?

From http://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go there an example illustrating a possible use of interfaces in Go. Code as below:
package main
import (
"encoding/json"
"fmt"
"reflect"
"time"
)
// start with a string representation of our JSON data
var input = `
{
"created_at": "Thu May 31 00:00:01 +0000 2012"
}
`
type Timestamp time.Time
func (t *Timestamp) UnmarshalJSON(b []byte) error {
v, err := time.Parse(time.RubyDate, string(b[1:len(b)-1]))
if err != nil {
return err
}
*t = Timestamp(v)
return nil
}
func main() {
// our target will be of type map[string]interface{}, which is a pretty generic type
// that will give us a hashtable whose keys are strings, and whose values are of
// type interface{}
var val map[string]Timestamp
if err := json.Unmarshal([]byte(input), &val); err != nil {
panic(err)
}
fmt.Println(val)
for k, v := range val {
fmt.Println(k, reflect.TypeOf(v))
}
fmt.Println(time.Time(val["created_at"]))
}
with a result like this:
map[created_at:{63474019201 0 0x59f680}]
created_at main.Timestamp
2012-05-31 00:00:01 +0000 UTC
I am struggling to understand how the function call
json.Unmarshal([]byte(input), &val){...}
relates to the method defined earlier
func (t *Timestamp) UnmarshalJSON(b []byte) error{...}
Reading the doc at http://golang.org/pkg/encoding/json/#Unmarshal is confusing me even more.
I am obviously missing something here, but I can't figure it out.
In Go an interface is implemented just by implementing its methods. It is so much different from the most other popular languages (Java, C#, C++) in which the class interfaces should be explicitly mentioned in the class declaration.
The detailed explanation of this concept you can find in the Go documentation: https://golang.org/doc/effective_go.html#interfaces
So the func (t *Timestamp) UnmarshalJSON(...) defines a method and in a same time implements the interface. The json.Unmarshal then type asserts the elements of val to the Unmarshaler interface (http://golang.org/pkg/encoding/json/#Unmarshaler) and call the UnmarshalJSON method to construct them from the byte slice.

What's the proper way to convert a json.RawMessage to a struct?

I have this struct
type SyncInfo struct {
Target string
}
Now I query some json data from ElasticSearch. Source is of type json.RawMessage.
All I want is to map source to my SyncInfo which I created the variable mySyncInfo for.
I even figured out how to do that...but it seems weird. I first call MarshalJSON() to get a []byte and then feed that to json.Unmarshal() which takes an []byte and a pointer to my struct.
This works fine but it feels as if I'm doing an extra hop. Am I missing something or is that the intended way to get from a json.RawMessage to a struct?
var mySyncInfo SyncInfo
jsonStr, _ := out.Hits.Hits[0].Source.MarshalJSON()
json.Unmarshal(jsonStr, &mySyncInfo)
fmt.Print(mySyncInfo.Target)
As said, the underlying type of json.RawMessage is []byte, so you can use a json.RawMessage as the data parameter to json.Unmarshal.
However, your problem is that you have a pointer (*json.RawMessage) and not a value. All you have to do is to dereference it:
err := json.Unmarshal(*out.Hits.Hits[0].Source, &mySyncInfo)
Working example:
package main
import (
"encoding/json"
"fmt"
)
type SyncInfo struct {
Target string
}
func main() {
data := []byte(`{"target": "localhost"}`)
Source := (*json.RawMessage)(&data)
var mySyncInfo SyncInfo
// Notice the dereferencing asterisk *
err := json.Unmarshal(*Source, &mySyncInfo)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", mySyncInfo)
}
Output:
{Target:localhost}
Playground: http://play.golang.org/p/J8R3Qrjrzx
json.RawMessage is really just a slice of bytes. You should be able to feed it directly into json.Unmarshal directly, like so:
json.Unmarshal(out.Hits.Hits[0].Source, &mySyncInfo)
Also, somewhat unrelated, but json.Unmarshal can return an error and you want to handle that.
err := json.Unmarshal(*out.Hits.Hits[0].Source, &mySyncInfo)
if err != nil {
// Handle
}

Unmarshal on reflected value

Here is the code
package main
import (
"fmt"
"encoding/json"
"reflect"
)
var (
datajson []byte
//ref mapp
)
type mapp map[string]reflect.Type
type User struct {
Name string
//Type map[string]reflect.Type
}
func MustJSONEncode(i interface{}) []byte {
result, err := json.Marshal(i)
if err != nil {
panic(err)
}
return result
}
func MustJSONDecode(b []byte, i interface{}) {
err := json.Unmarshal(b, i)
if err != nil {
panic(err)
}
}
func Store(a interface{}) {
datajson = MustJSONEncode(a)
//fmt.Println(datajson)
}
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType)
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("%s", obj)
}
func main() {
dummy := &User{}
david := User{Name: "DavidMahon"}
Store(david)
Get(datajson, dummy)
}
In the Get function
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType)
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("%s", obj)
}
I am unable to unmarshal the json into the underlying object type.
Whats wrong here? I am so stuck here. Something very simple yet so difficult to figure out.
Thanks
UPDATE::Goal of this problem is to retreive a fully formed object of type passed in Get function.
The approach mentioned by Nick on the comment below doesnot get me the actual object which I already tried before. I can anyways retrieve the data (even when the object has recursive objects underneath) in a map like this
func Get(a []byte) {
var f interface{}
//buf := bytes.NewBuffer(a)
//v := buf.String()
//usr := &User{}
MustJSONDecode(a, &f)
fmt.Printf("\n %v \n", f)
}
However I need the actual object back not just the data. Something like user := &User{"SomeName"} where I need user object back from Unmarshall. The trick is somewhere in reflection but dont know how.
I'm confused as to why you want to do this, but here is how to fix it
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType).Interface()
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("obj = %#v\n", obj)
}
Note the call to Interface().
Playground link
It seems to me that you are going to a lot of trouble to make an empty &User when you already have one in b, eg
func Get(a []byte, b interface{}) {
MustJSONDecode(a, &b)
fmt.Printf("obj = %#v\n", b)
}
But I'm guessing there is some more to this plan which isn't apparent here!
reflect.New(objType) returns a reflect.Value Which is not the thing as the interface you passed. According to the docs for Value It is a struct with only unexported fields. the json package can't work with unexported fields. Since it's not the same object as you passed in and it's not even json encodable/decodable the json package will fail.
You will probably find the Laws of Reflection article useful while trying to use the reflect package.