Can someone explain this interface example in Go? - json

From http://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go there an example illustrating a possible use of interfaces in Go. Code as below:
package main
import (
"encoding/json"
"fmt"
"reflect"
"time"
)
// start with a string representation of our JSON data
var input = `
{
"created_at": "Thu May 31 00:00:01 +0000 2012"
}
`
type Timestamp time.Time
func (t *Timestamp) UnmarshalJSON(b []byte) error {
v, err := time.Parse(time.RubyDate, string(b[1:len(b)-1]))
if err != nil {
return err
}
*t = Timestamp(v)
return nil
}
func main() {
// our target will be of type map[string]interface{}, which is a pretty generic type
// that will give us a hashtable whose keys are strings, and whose values are of
// type interface{}
var val map[string]Timestamp
if err := json.Unmarshal([]byte(input), &val); err != nil {
panic(err)
}
fmt.Println(val)
for k, v := range val {
fmt.Println(k, reflect.TypeOf(v))
}
fmt.Println(time.Time(val["created_at"]))
}
with a result like this:
map[created_at:{63474019201 0 0x59f680}]
created_at main.Timestamp
2012-05-31 00:00:01 +0000 UTC
I am struggling to understand how the function call
json.Unmarshal([]byte(input), &val){...}
relates to the method defined earlier
func (t *Timestamp) UnmarshalJSON(b []byte) error{...}
Reading the doc at http://golang.org/pkg/encoding/json/#Unmarshal is confusing me even more.
I am obviously missing something here, but I can't figure it out.

In Go an interface is implemented just by implementing its methods. It is so much different from the most other popular languages (Java, C#, C++) in which the class interfaces should be explicitly mentioned in the class declaration.
The detailed explanation of this concept you can find in the Go documentation: https://golang.org/doc/effective_go.html#interfaces
So the func (t *Timestamp) UnmarshalJSON(...) defines a method and in a same time implements the interface. The json.Unmarshal then type asserts the elements of val to the Unmarshaler interface (http://golang.org/pkg/encoding/json/#Unmarshaler) and call the UnmarshalJSON method to construct them from the byte slice.

Related

json.Unmarshal interface pointer with later type assertion

Because I often unmarshal http.Response.Body, I thought I could write a function which handles all the hassle of reading, closing and unmarshaling into various different structs. That's why I introduced a function func unmarhalInterface(closer *io.ReadCloser, v *interface{}) error and can then assert the return value with t:=i.(T).
According to this answer I already wrapped it into a value of type *interface{}, but because the overlying type is interface{}and not myStruct, the json package implementation chooses map[string]interface{}. After that a type assertion fails (of course). Is there anything i am missing or requires this implementation a type assertion "by hand", that means look for all fields in the map and assign those that I want into my struct.
Code below has minimal example with notation in comments. If my explanation is not sufficient, please ask away.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
)
type myStruct struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
jsonBlob := []byte(`{"a":"test","b":"test2"}`)
var foo = interface{}(myStruct{})
closer := ioutil.NopCloser(bytes.NewReader(jsonBlob))
err := unmarshalCloser(&closer, &foo)
if err != nil {
log.Fatal(err)
}
fmt.Println(fmt.Sprintf("%v", foo))
// That´s what i want:
foo2 := foo.(myStruct)
fmt.Println(foo2.A)
}
func unmarshalCloser(closer *io.ReadCloser, v *interface{}) error {
defer func() { _ = (*closer).Close() }()
data, err := ioutil.ReadAll(*closer)
if err != nil {
return err
}
err = json.Unmarshal(data, v)
if err != nil {
return err
}
return nil
}
Golang Playground
An empty interface isn't an actual type, it's basically something that matches anything. As stated in the comments, a pointer to an empty interface doesn't really make sense as a pointer already matches an empty interface since everything matches an empty interface. To make your code work, you should remove the interface wrapper around your struct, since that's just messing up the json type checking, and the whole point of an empty interface is that you can pass anything to it.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
)
type myStruct struct {
A string `json:"a"`
B string `json:"b"`
}
func main() {
jsonBlob := []byte(`{"a":"test","b":"test2"}`)
var foo = &myStruct{} // This need to be a pointer so its attributes can be assigned
closer := ioutil.NopCloser(bytes.NewReader(jsonBlob))
err := unmarshalCloser(closer, foo)
if err != nil {
log.Fatal(err)
}
fmt.Println(fmt.Sprintf("%v", foo))
// That´s what i want:
fmt.Println(foo.A)
}
// You don't need to declare either of these arguments as pointers since they're both interfaces
func unmarshalCloser(closer io.ReadCloser, v interface{}) error {
defer closer.Close()
// v NEEDS to be a pointer or the json stuff will barf
// Simplified with the decoder
return json.NewDecoder(closer).Decode(v)
}

Go: Converting JSON string to map[string]interface{}

I'm trying to create a JSON representation within Go using a map[string]interface{} type. I'm dealing with JSON strings and I'm having a hard time figuring out how to avoid the JSON unmarshaler to automatically deal with numbers as float64s. As a result the following error occurs.
Ex.
"{ 'a' : 9223372036854775807}" should be map[string]interface{} = [a 9223372036854775807 but in reality it is map[string]interface{} = [a 9.2233720368547758088E18]
I searched how structs can be used to avoid this by using json.Number but I'd really prefer using the map type designated above.
The go json.Unmarshal(...) function automatically uses float64 for JSON numbers. If you want to unmarshal numbers into a different type then you'll have to use a custom type with a custom unmarshaler. There is no way to force the unmarshaler to deserialize custom values into a generic map.
For example, here's how you could parse values of the "a" property as a big.Int.
package main
import (
"encoding/json"
"fmt"
"math/big"
)
type MyDoc struct {
A BigA `json:"a"`
}
type BigA struct{ *big.Int }
func (a BigA) UnmarshalJSON(bs []byte) error {
_, ok := a.SetString(string(bs), 10)
if !ok {
return fmt.Errorf("invalid integer %s", bs)
}
return nil
}
func main() {
jsonstr := `{"a":9223372036854775807}`
mydoc := MyDoc{A: BigA{new(big.Int)}}
err := json.Unmarshal([]byte(jsonstr), &mydoc)
if err != nil {
panic(err)
}
fmt.Printf("OK: mydoc=%#v\n", mydoc)
// OK: mydoc=main.MyDoc{A:9223372036854775807}
}
func jsonToMap(jsonStr string) map[string]interface{} {
result := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &result)
return result
}
Example - https://goplay.space/#ra7Gv8A5Heh
Related questions - create a JSON data as map[string]interface with the given data

Golang JSON decoder does not recognize type

Still a Golang beginner, I am trying to code a generic function to serve ReST requests. I pass a function to create a new resource (struct) with an interface implemented on it, because I would also invoke methods on the struct. When decoding the JSON, logging the type shows the correct (struct) type, but the JSON decoder seems to only recognize the interface, which it cannot decode to.
package main
import (
"encoding/json"
"github.com/julienschmidt/httprouter"
"log"
"net/http"
"strings"
)
// general resource interface
type resource interface {
// check semantics and return an array of errors or nil if no error found
check() []string
// update the resource in backend
update() error
}
// specific resource named "anchor"
type anchor struct {
ID string `json:"id"`
Name string `json:"name"`
}
func newAnchor() resource {
return anchor{}
}
func (a anchor) check() []string {
return nil
}
func (a anchor) update() error {
return nil
}
// generic function to create (POST) a new resource
func restCreate(newResource func() resource) httprouter.Handle {
return func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
const F = "restCreate"
var checkErrs []string
res := newResource()
log.Printf("%s res type %T\n", F, res)
dcdr := json.NewDecoder(r.Body)
err := dcdr.Decode(&res)
log.Printf("%s Unmarshalled into %T: %+v\n", F, res, res)
if err == nil {
checkErrs = res.check()
}
switch {
case err != nil:
w.WriteHeader(http.StatusInternalServerError)
log.Printf("[ERR] %s: %v\n", F, err)
case checkErrs != nil:
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte(strings.Join(checkErrs, "\n")))
log.Printf("%s: %v\n", F, err)
default:
res.update()
bs, _ := json.Marshal(res)
w.Write(bs)
}
}
}
func main() {
r := httprouter.New()
r.POST("/anchors", restCreate(newAnchor))
http.ListenAndServe(":8080", r)
}
The execution log shows:
restCreate res type main.anchor
restCreate Unmarshalled into main.anchor: {ID: Name:}
[ERR] restCreate: json: cannot unmarshal object into Go value of type main.resource
Why does Printf show the struct type and json.Decoder the interface?
I'd appreciate any indicator on what's going wrong and how to solve this in a generic way...
It is because you try to use a pointer to the interface to unmarshal into. You need return a pointer in a function
func newAnchor() resource {
return &anchor{}
}
And you don't need to get address in this line:
err := dcdr.Decode(&res)
Here is small working example: https://play.golang.org/p/3E0RmGTURO
You cannot Unmarshal into an interface unless variable is holding a pointer to the concrete type you desire, as json.Decode won't know which concrete type to use. You have two workarounds at your disposal:
Having newResource return a concrete type under the hood:
func newResource() resource {
return &anchor{}
}
This way json.Decode knows to unmarshal your JSON into an anchor.
Use newAnchor instead of newResource: this will be more readable in your restCreate function, and is more idiomatic[1].
[1] http://idiomaticgo.com/post/best-practice/accept-interfaces-return-structs/

Custom MarshalJSON() never gets called in Go

I've written custom versions of MarshalJSON and UnmarshalJSON. My UnmarshalJSON gets called the way I want it to, but I can't get it to work with MarshalJSON. Here's code that summarizes my problem:
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
"os"
)
type myStruct struct {
Data string `json:"data"`
}
func (s *myStruct) MarshalJSON() ([]byte, error) {
return []byte(`{"data":"charlie"}`), nil
}
func (s *myStruct) UnmarshalJSON(b []byte) error {
// Insert the string directly into the Data member
return json.Unmarshal(b, &s.Data)
}
func main() {
// Create a struct with initial content "alpha"
ms := myStruct{"alpha"}
// Replace content with "bravo" using custom UnmarshalJSON() (SUCCESSFUL)
if err := json.NewDecoder(bytes.NewBufferString(`"bravo"`)).Decode(&ms); err != nil {
log.Fatal(err)
}
// Use custom MarshalJSON() to get "charlie" back (UNSUCCESSFUL)
if err := json.NewEncoder(os.Stdout).Encode(ms); err != nil {
log.Fatal(err)
}
// Trying another method (UNSUCCESSFUL)
if ret, err := json.Marshal(ms); err != nil {
log.Fatal(err)
} else {
fmt.Println(string(ret))
}
// Verify that the Marshaler interface is correctly implemented
var marsh json.Marshaler
marsh = &ms
ret, _ := marsh.MarshalJSON()
fmt.Println(string(ret)) // Prints "charlie"
}
In short, the program encodes the struct "automatically" in two ways, and then finally calls MarshalJSON manually. The response I want is "charlie". Running the code generates the following output:
{"data":"bravo"}
{"data":"bravo"}
{"data":"charlie"}
Try it at Go Playground: http://play.golang.org/p/SJ05S8rAYN
In this part of the code, ms gets copied into an interface{} variable:
// Trying another method (UNSUCCESSFUL)
if ret, err := json.Marshal(ms); err != nil {
The problem is that this variable does not implement the json.Marshaler interface, since MarshalJSON is not in the method set for myStruct (only for *myStruct).
The fix is to either (a) make your MarshalJSON method take a non-pointer receiver (which will mean it gets a copy of the struct: possibly costly if it is large), or (b) marshal a pointer to the struct (as Kavu mentioned in a comment).
The reason for this behaviour is that Go doesn't let you take a pointer to the value stored inside an interface variable, instead requiring you to make a copy of the value whenever you want to access it. While the language has syntactic sugar to convert ms.MarshalJSON() into (&ms).MarshalJSON() as a way to access the method with a pointer receiver, this can not be done for a value stored in an interface variable. For this reason, the method is not considered to be in its method set.

Override the layout used by json.Marshal to format time.Time

In Golang, is there a way to make the generic encoding/json Marshal to use a different layout when Marshaling the time.Time fields?
Basically I have this struct:
s := {"starttime":time.Now(), "name":"ali"}
and I want to encoding to json using encdoding/json's Marshal function, but I want to use my custom layout, I imagine somewhere time.Format(layout) is being called, I want to control that layout,
As inspired by zeebo's answer and hashed out in the comments to that answer:
http://play.golang.org/p/pUCBUgrjZC
package main
import "fmt"
import "time"
import "encoding/json"
type jsonTime struct {
time.Time
f string
}
func (j jsonTime) format() string {
return j.Time.Format(j.f)
}
func (j jsonTime) MarshalText() ([]byte, error) {
return []byte(j.format()), nil
}
func (j jsonTime) MarshalJSON() ([]byte, error) {
return []byte(`"` + j.format() + `"`), nil
}
func main() {
jt := jsonTime{time.Now(), time.Kitchen}
if jt.Before(time.Now().AddDate(0, 0, 1)) { // 1
x := map[string]interface{}{
"foo": jt,
"bar": "baz",
}
data, err := json.Marshal(x)
if err != nil {
panic(err)
}
fmt.Printf("%s", data)
}
}
This solution embeds the time.Time into the jsonTime struct. Embedding promotes all of time.Time's methods to the jsonTime struct, allowing their use without explicit type conversion (see // 1).
Embedding a time.Time has the downside of also promoting the MarshalJSON method, which the encoding/json marshaling code prioritizes higher than the MarshalText method for backwards compatibility reasons (MarshalText was added in Go 1.2, MarshalJSON predates that). As a result the default time.Time format is used instead of a custom format provided by MarshalText.
To overcome this problem we override MarshalJSON for the jsonTime struct.
Maybe something like this will work for you?
package main
import "fmt"
import "time"
import "encoding/json"
type jsonTime struct {
t time.Time
f string
}
func (j jsonTime) MarshalText() ([]byte, error) {
return []byte(j.t.Format(j.f)), nil
}
func main() {
x := map[string]interface{}{
"foo": jsonTime{t: time.Now(), f: time.Kitchen},
"bar": "baz",
}
data, err := json.Marshal(x)
if err != nil {
panic(err)
}
fmt.Printf("%s", data)
}
also available here: http://play.golang.org/p/D1kq5KrXQZ
Just make a custom type that implements MarshalText the way you want it to show up.
First, I highly recommend against using a time format other than the default RFC3339. It's a good time format, and can be parsed by any number of languages, so unless you are needing a different format because somebody else's API requires it, it's probably best to use the default.
But, I've had to solve this problem in consuming other people's APIs, so here is one solution that shifts the bulk of the work to the Marshal/Unmarshal step, and leaves you with an ideal structure: http://play.golang.org/p/DKaTbV2Zvl