Override the layout used by json.Marshal to format time.Time - json

In Golang, is there a way to make the generic encoding/json Marshal to use a different layout when Marshaling the time.Time fields?
Basically I have this struct:
s := {"starttime":time.Now(), "name":"ali"}
and I want to encoding to json using encdoding/json's Marshal function, but I want to use my custom layout, I imagine somewhere time.Format(layout) is being called, I want to control that layout,

As inspired by zeebo's answer and hashed out in the comments to that answer:
http://play.golang.org/p/pUCBUgrjZC
package main
import "fmt"
import "time"
import "encoding/json"
type jsonTime struct {
time.Time
f string
}
func (j jsonTime) format() string {
return j.Time.Format(j.f)
}
func (j jsonTime) MarshalText() ([]byte, error) {
return []byte(j.format()), nil
}
func (j jsonTime) MarshalJSON() ([]byte, error) {
return []byte(`"` + j.format() + `"`), nil
}
func main() {
jt := jsonTime{time.Now(), time.Kitchen}
if jt.Before(time.Now().AddDate(0, 0, 1)) { // 1
x := map[string]interface{}{
"foo": jt,
"bar": "baz",
}
data, err := json.Marshal(x)
if err != nil {
panic(err)
}
fmt.Printf("%s", data)
}
}
This solution embeds the time.Time into the jsonTime struct. Embedding promotes all of time.Time's methods to the jsonTime struct, allowing their use without explicit type conversion (see // 1).
Embedding a time.Time has the downside of also promoting the MarshalJSON method, which the encoding/json marshaling code prioritizes higher than the MarshalText method for backwards compatibility reasons (MarshalText was added in Go 1.2, MarshalJSON predates that). As a result the default time.Time format is used instead of a custom format provided by MarshalText.
To overcome this problem we override MarshalJSON for the jsonTime struct.

Maybe something like this will work for you?
package main
import "fmt"
import "time"
import "encoding/json"
type jsonTime struct {
t time.Time
f string
}
func (j jsonTime) MarshalText() ([]byte, error) {
return []byte(j.t.Format(j.f)), nil
}
func main() {
x := map[string]interface{}{
"foo": jsonTime{t: time.Now(), f: time.Kitchen},
"bar": "baz",
}
data, err := json.Marshal(x)
if err != nil {
panic(err)
}
fmt.Printf("%s", data)
}
also available here: http://play.golang.org/p/D1kq5KrXQZ
Just make a custom type that implements MarshalText the way you want it to show up.

First, I highly recommend against using a time format other than the default RFC3339. It's a good time format, and can be parsed by any number of languages, so unless you are needing a different format because somebody else's API requires it, it's probably best to use the default.
But, I've had to solve this problem in consuming other people's APIs, so here is one solution that shifts the bulk of the work to the Marshal/Unmarshal step, and leaves you with an ideal structure: http://play.golang.org/p/DKaTbV2Zvl

Related

Go: Converting JSON string to map[string]interface{}

I'm trying to create a JSON representation within Go using a map[string]interface{} type. I'm dealing with JSON strings and I'm having a hard time figuring out how to avoid the JSON unmarshaler to automatically deal with numbers as float64s. As a result the following error occurs.
Ex.
"{ 'a' : 9223372036854775807}" should be map[string]interface{} = [a 9223372036854775807 but in reality it is map[string]interface{} = [a 9.2233720368547758088E18]
I searched how structs can be used to avoid this by using json.Number but I'd really prefer using the map type designated above.
The go json.Unmarshal(...) function automatically uses float64 for JSON numbers. If you want to unmarshal numbers into a different type then you'll have to use a custom type with a custom unmarshaler. There is no way to force the unmarshaler to deserialize custom values into a generic map.
For example, here's how you could parse values of the "a" property as a big.Int.
package main
import (
"encoding/json"
"fmt"
"math/big"
)
type MyDoc struct {
A BigA `json:"a"`
}
type BigA struct{ *big.Int }
func (a BigA) UnmarshalJSON(bs []byte) error {
_, ok := a.SetString(string(bs), 10)
if !ok {
return fmt.Errorf("invalid integer %s", bs)
}
return nil
}
func main() {
jsonstr := `{"a":9223372036854775807}`
mydoc := MyDoc{A: BigA{new(big.Int)}}
err := json.Unmarshal([]byte(jsonstr), &mydoc)
if err != nil {
panic(err)
}
fmt.Printf("OK: mydoc=%#v\n", mydoc)
// OK: mydoc=main.MyDoc{A:9223372036854775807}
}
func jsonToMap(jsonStr string) map[string]interface{} {
result := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &result)
return result
}
Example - https://goplay.space/#ra7Gv8A5Heh
Related questions - create a JSON data as map[string]interface with the given data

Aggregating JSON objects in Go

I'm making a Go service that gathers JSON objects from different sources and aggregates them in a single JSON object.
I was wondering if there was any way to aggregate the child objects without having to unmarshal and re-marshal them again or having to manually build a JSON string.
I was thinking of using a struct containing the already marshalled parts, such as this:
type Event struct {
Place string `json:"place"`
Attendees string `json:"attendees"`
}
Where Place and Attendees are JSON strings themselves. I'd like to somehow mark them as "already marshalled" so they don't end up as escaped JSON strings but get used as is instead.
Is there any way to achieve this?
You can use json.RawMessage
RawMessage is a raw encoded JSON object. It implements Marshaler and Unmarshaler and can be used to delay JSON decoding or precompute a JSON encoding.
Also, json.RawMessage is an alias to []byte so you can values to it this way:
v := json.RawMessage(`{"foo":"bar"}`)
Example:
package main
import (
"encoding/json"
"fmt"
)
type Event struct {
Place json.RawMessage `json:"place"`
Attendees json.RawMessage `json:"attendees"`
}
func main() {
e := Event{
Place: json.RawMessage(`{"address":"somewhere"}`),
Attendees: json.RawMessage(`{"key":"value"}`),
}
c, err := json.Marshal(&e)
if err != nil {
panic(err)
}
fmt.Println(string(c))
// {"place":{"address":"somewhere"},"attendees":{"key":"value"}}
}
Yes, you can use a custom type that implements Marshaler interface.
https://play.golang.org/p/YB_eKlfOND
package main
import (
"fmt"
"encoding/json"
)
type Event struct {
Place RawString `json:"place"`
Attendees RawString `json:"attendees,omitempty"`
}
type RawString string
func (s RawString) MarshalJSON() ([]byte, error) {
return []byte(s), nil
}
func main() {
event := Event{
Place: RawString(`{"name":"Paris"}`),
Attendees: RawString(`[{"name":"John"}, {"name":"Juli"}]`),
}
s, err := json.Marshal(event)
fmt.Println(fmt.Sprintf("event: %v; err: %v", string(s), err))
}

Can someone explain this interface example in Go?

From http://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go there an example illustrating a possible use of interfaces in Go. Code as below:
package main
import (
"encoding/json"
"fmt"
"reflect"
"time"
)
// start with a string representation of our JSON data
var input = `
{
"created_at": "Thu May 31 00:00:01 +0000 2012"
}
`
type Timestamp time.Time
func (t *Timestamp) UnmarshalJSON(b []byte) error {
v, err := time.Parse(time.RubyDate, string(b[1:len(b)-1]))
if err != nil {
return err
}
*t = Timestamp(v)
return nil
}
func main() {
// our target will be of type map[string]interface{}, which is a pretty generic type
// that will give us a hashtable whose keys are strings, and whose values are of
// type interface{}
var val map[string]Timestamp
if err := json.Unmarshal([]byte(input), &val); err != nil {
panic(err)
}
fmt.Println(val)
for k, v := range val {
fmt.Println(k, reflect.TypeOf(v))
}
fmt.Println(time.Time(val["created_at"]))
}
with a result like this:
map[created_at:{63474019201 0 0x59f680}]
created_at main.Timestamp
2012-05-31 00:00:01 +0000 UTC
I am struggling to understand how the function call
json.Unmarshal([]byte(input), &val){...}
relates to the method defined earlier
func (t *Timestamp) UnmarshalJSON(b []byte) error{...}
Reading the doc at http://golang.org/pkg/encoding/json/#Unmarshal is confusing me even more.
I am obviously missing something here, but I can't figure it out.
In Go an interface is implemented just by implementing its methods. It is so much different from the most other popular languages (Java, C#, C++) in which the class interfaces should be explicitly mentioned in the class declaration.
The detailed explanation of this concept you can find in the Go documentation: https://golang.org/doc/effective_go.html#interfaces
So the func (t *Timestamp) UnmarshalJSON(...) defines a method and in a same time implements the interface. The json.Unmarshal then type asserts the elements of val to the Unmarshaler interface (http://golang.org/pkg/encoding/json/#Unmarshaler) and call the UnmarshalJSON method to construct them from the byte slice.

Custom MarshalJSON() never gets called in Go

I've written custom versions of MarshalJSON and UnmarshalJSON. My UnmarshalJSON gets called the way I want it to, but I can't get it to work with MarshalJSON. Here's code that summarizes my problem:
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
"os"
)
type myStruct struct {
Data string `json:"data"`
}
func (s *myStruct) MarshalJSON() ([]byte, error) {
return []byte(`{"data":"charlie"}`), nil
}
func (s *myStruct) UnmarshalJSON(b []byte) error {
// Insert the string directly into the Data member
return json.Unmarshal(b, &s.Data)
}
func main() {
// Create a struct with initial content "alpha"
ms := myStruct{"alpha"}
// Replace content with "bravo" using custom UnmarshalJSON() (SUCCESSFUL)
if err := json.NewDecoder(bytes.NewBufferString(`"bravo"`)).Decode(&ms); err != nil {
log.Fatal(err)
}
// Use custom MarshalJSON() to get "charlie" back (UNSUCCESSFUL)
if err := json.NewEncoder(os.Stdout).Encode(ms); err != nil {
log.Fatal(err)
}
// Trying another method (UNSUCCESSFUL)
if ret, err := json.Marshal(ms); err != nil {
log.Fatal(err)
} else {
fmt.Println(string(ret))
}
// Verify that the Marshaler interface is correctly implemented
var marsh json.Marshaler
marsh = &ms
ret, _ := marsh.MarshalJSON()
fmt.Println(string(ret)) // Prints "charlie"
}
In short, the program encodes the struct "automatically" in two ways, and then finally calls MarshalJSON manually. The response I want is "charlie". Running the code generates the following output:
{"data":"bravo"}
{"data":"bravo"}
{"data":"charlie"}
Try it at Go Playground: http://play.golang.org/p/SJ05S8rAYN
In this part of the code, ms gets copied into an interface{} variable:
// Trying another method (UNSUCCESSFUL)
if ret, err := json.Marshal(ms); err != nil {
The problem is that this variable does not implement the json.Marshaler interface, since MarshalJSON is not in the method set for myStruct (only for *myStruct).
The fix is to either (a) make your MarshalJSON method take a non-pointer receiver (which will mean it gets a copy of the struct: possibly costly if it is large), or (b) marshal a pointer to the struct (as Kavu mentioned in a comment).
The reason for this behaviour is that Go doesn't let you take a pointer to the value stored inside an interface variable, instead requiring you to make a copy of the value whenever you want to access it. While the language has syntactic sugar to convert ms.MarshalJSON() into (&ms).MarshalJSON() as a way to access the method with a pointer receiver, this can not be done for a value stored in an interface variable. For this reason, the method is not considered to be in its method set.

How to Unmarshal JSON into an interface in Go

I am trying to simultaneously unmarshal and strip fields from a number of different JSON responses into appropriate Go structs. To do this, I created a Wrappable interface that defines the Unwrap method (which strips the appropriate fields) and pass that interface to the code that unmarshals and unwraps. It looks like the following example (also at http://play.golang.org/p/fUGveHwiz9):
package main
import (
"encoding/json"
"fmt"
)
type Data struct {
A string `json:"a"`
B string `json:"b"`
}
type DataWrapper struct {
Elements []Data `json:"elems"`
}
type Wrapper interface {
Unwrap() []interface{}
}
func (dw DataWrapper) Unwrap() []interface{} {
result := make([]interface{}, len(dw.Elements))
for i := range dw.Elements {
result[i] = dw.Elements[i]
}
return result
}
func unmarshalAndUnwrap(data []byte, wrapper Wrapper) []interface{} {
err := json.Unmarshal(data, &wrapper)
if err != nil {
panic(err)
}
return wrapper.Unwrap()
}
func main() {
data := `{"elems": [{"a": "data", "b": "data"}, {"a": "data", "b": "data"}]}`
res := unmarshalAndUnwrap([]byte(data), DataWrapper{})
fmt.Println(res)
}
However, when I run the code, Go panics with the following error:
panic: json: cannot unmarshal object into Go value of type main.Wrapper
It seems the unmarshaller doesn't want to be passed a pointer to an interface. I am somewhat surprised by this given that I can get at the underlying type and fields using the reflect package within the unmarshalAndUnwrap method. Can anyone provide insight into this problem and how I might work around it?
As you stated, passing a non-pointer fails. Why are you trying to do this anyway?
Replace
res := unmarshalAndUnwrap([]byte(data), DataWrapper{})
by
res := unmarshalAndUnwrap([]byte(data), &DataWrapper{})
It should do the trick and it avoid unnecessary copy.
This error should help you understand: http://play.golang.org/p/jXxCxPQDOw