Time always 0 in a go program - json

I am trying here to get trades from Bitfinex API and print them on the screen. Everything works fine, except for times.
I wonder why they always prints as 0001-01-01 00:00:00 in this example...
In UnmarshalJSON, "t" contains the time I need. It just doesn't seem to stick to the Trade structure.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strconv"
"time"
)
type Trade struct {
Tid int
Timestamp myTime
}
type myTime time.Time
func (t myTime) String() string {
when := time.Time(t)
return when.Format("2006-01-02 15:04:05")
}
func (t myTime) UnmarshalJSON(b []byte) error {
ts, err := strconv.Atoi(string(b))
if err != nil {
return err
}
t = myTime(time.Unix(int64(ts), 0))
return nil
}
func main() {
trades, _ := GetTrades()
for _, trade := range trades {
fmt.Printf("%d - %s\n", trade.Tid, trade.Timestamp)
}
}
func GetTrades() (trades []Trade, err error) {
r, err := http.Get("https://api.bitfinex.com/v1/trades/BTCUSD")
defer r.Body.Close()
body, _ := ioutil.ReadAll(r.Body)
json.Unmarshal(body, &trades)
return
}

You can unmarshal on a pointer receiver .. otherwise you're just copying unrelated data around:
func (t *myTime) UnmarshalJSON(b []byte) error {
// ^^^^^^^ this
ts, err := strconv.Atoi(string(b))
if err != nil {
return err
}
*t = myTime(time.Unix(int64(ts), 0))
// ^^ this too
return nil
}
I'm not sure how "safe"/"idiomatic" this is to re-assign data in a pointer receiver.. can someone clarify?
(this works btw.. not sure if its "best practice")

You can't assign in UnmarshalJSON to a value, you have to use a pointer or a struct, one approach is:
type MyTime struct {
time.Time
}
func (t MyTime) String() string {
when := time.Time(t.Time)
return when.Format("2006-01-02 15:04:05")
}
func (t *MyTime) UnmarshalJSON(b []byte) error {
ts, err := strconv.Atoi(string(b))
if err != nil {
return err
}
t.Time = time.Unix(int64(ts), 0)
return nil
}

Related

Can't generate scalar JSON in gqlgen

I have a GQL scheme:
extend type MyType #key(fields: "id") {
id: ID! #external
properties: JSON #external
myField: String! #requires(fields: "properties")
}
scalar JSON
In graph/model/model.go:
package model
import (
"encoding/json"
"fmt"
"io"
"strconv"
"strings"
)
type JSON map[string]interface{}
// UnmarshalGQL implements the graphql.Unmarshaler interface
func (b *JSON) UnmarshalGQL(v interface{}) error {
*b = make(map[string]interface{})
byteData, err := json.Marshal(v)
if err != nil {
panic("FAIL WHILE MARSHAL SCHEME")
}
tmp := make(map[string]interface{})
err = json.Unmarshal(byteData, &tmp)
if err != nil {
panic("FAIL WHILE UNMARSHAL SCHEME")
//return fmt.Errorf("%v", err)
}
*b = tmp
return nil
}
// MarshalGQL implements the graphql.Marshaler interface
func (b JSON) MarshalGQL(w io.Writer) {
byteData, err := json.Marshal(b)
if err != nil {
panic("FAIL WHILE MARSHAL SCHEME")
}
_, _ = w.Write(byteData)
}
But when I run go run github.com/99designs/gqlgen generate
error:
generating core failed: type.gotpl: template: type.gotpl:52:28: executing "type.gotpl" at <$type.Elem.GO>: nil pointer evaluating *config.TypeReference.
GOexit status 1
I just need to get map[string]interface{} which called JSON. I knew there's scalar Map, but for apollo federation that field must be called JSON.
it's should to replace MarshalGQL to MarshalJSON like:
type JSON map[string]interface{}
func MarshalJSON(b JSON) graphql.Marshaler {
return graphql.WriterFunc(func(w io.Writer) {
byteData, err := json.Marshal(b)
if err != nil {
log.Printf("FAIL WHILE MARSHAL JSON %v\n", string(byteData))
}
_, err = w.Write(byteData)
if err != nil {
log.Printf("FAIL WHILE WRITE DATA %v\n", string(byteData))
}
})
}
func UnmarshalJSON(v interface{}) (JSON, error) {
byteData, err := json.Marshal(v)
if err != nil {
return JSON{}, fmt.Errorf("FAIL WHILE MARSHAL SCHEME")
}
tmp := make(map[string]interface{})
err = json.Unmarshal(byteData, &tmp)
if err != nil {
return JSON{}, fmt.Errorf("FAIL WHILE UNMARSHAL SCHEME")
}
return tmp, nil
}

What am I doing wrong with converting this map to and from JSON?

I'm trying to store a map with struct key in a file, so I'm converting it to and from JSON. But the map that I end up with has only a single entry with a zero key, and the last value from the original map. (My sample code here doesn't do the file I/O.)
I've looked at various similar examples that seem to work, and I can't see where I've gone wrong.
import (
"encoding/json"
"errors"
"fmt"
)
type XY struct {
X int
Y int
}
func (xy XY) String() string {
return fmt.Sprintf("{X:%v, Y:%v}", xy.X, xy.Y)
}
func (xy XY) MarshalText() ([]byte, error) {
text := []byte(fmt.Sprintf("%d,%d", xy.X, xy.Y))
fmt.Printf("Marshalled %v to %s\n", xy, text)
return text, nil
}
func (xy XY) UnmarshalText(p []byte) error {
n, err := fmt.Sscanf(string(p), "%d,%d", &xy.X, &xy.Y)
if err != nil {
return err
} else if n != 2 {
return errors.New("Cannot parse as XY: '" + string(p) + "'")
}
fmt.Printf("Unmarshalled %s to %v\n", p, xy)
return nil
}
type XYMap = map[XY]string
func loadXY(data []byte, xymap XYMap) error {
err := json.Unmarshal(data, &xymap)
if err != nil {
fmt.Println("unmarshal error:", err)
}
return err
}
func main() {
xymap := make(XYMap)
xymap[XY{1,2}] = "one two"
xymap[XY{3,4}] = "three four"
bytes, err := json.Marshal(xymap)
fmt.Printf("xymap=%v \nbytes=%s err=%v\n", xymap, bytes, err)
newxymap := make(XYMap)
loadXY(bytes, newxymap)
fmt.Printf("newxymap=%v\n", newxymap)
}
Output:
Marshalled {X:1, Y:2} to 1,2
Marshalled {X:3, Y:4} to 3,4
xymap=map[{X:1, Y:2}:one two {X:3, Y:4}:three four]
bytes={"1,2":"one two","3,4":"three four"} err=<nil>
Unmarshalled 1,2 to {X:1, Y:2}
Unmarshalled 3,4 to {X:3, Y:4}
newxymap=map[{X:0, Y:0}:three four]
You have to use a pointer receiver for unmarshal text.
func (xy *XY) UnmarshalText(p []byte) error {
...
}
With your current version, UnmarshalText does not modify the receiver, and the map ends up getting empty keys.

Unmarshall only approved fields

I'm trying to give permissions to users to edit certain fields of a struct.
This fields will vary depending on the action, the context, and the role the current user has.
At the moment I'm doing this in an imperative way but it's rather tedious and not scalable.
I thought having a list of approved_fields could be a nice and more scalable solution but I have no idea how to go through with this. I think reflection is the way to go but I don't know enough yet.
Any inspiration would help.
type Foo struct {
Bar1 int
Bar2 int
Bar3 int
}
foo := Foo{}
approved_fields := []string{"Bar1", "Bar2"}
decode(json_data, &foo, approved_fields)
foo2 := Foo{}
approved_fields := []string{"Bar1", "Bar3"}
decode(json_data, &foo2, approved_fields)
Here's how I would think about solving it. There may be something more efficient since I am unmarshalling the whole item before selectively choosing fields.
import (
"encoding/json"
"log"
"reflect"
"github.com/pkg/errors"
)
func fieldByName(s string, v interface{}) (reflect.Value, error) {
el := reflect.ValueOf(v).Elem()
fbn := el.FieldByName(s)
if !fbn.IsValid() {
return fbn, errors.Errorf("does not have field named %s", s)
}
return fbn, nil
}
func decode(data []byte, v interface{}, approvedFields []string) error {
typeOf := reflect.TypeOf(v)
if typeOf.Kind() == reflect.Ptr {
typeOf = typeOf.Elem()
}
if typeOf.Kind() == reflect.Slice {
return errors.New("does not support slices")
}
newItem := reflect.New(typeOf)
newItemInterface := newItem.Interface()
if err := json.Unmarshal(data, &newItemInterface); err != nil {
return errors.Wrap(err, "json unmarshall")
}
for _, approvedField := range approvedFields {
fbn, err := fieldByName(approvedField, v)
if err != nil {
return errors.Wrap(err, "field by name")
}
val, _ := fieldByName(approvedField, newItemInterface)
fbn.Set(val)
}
return nil
}
a test:
func TestBar1Bar2(t *testing.T) {
var json_data []byte
{
f := &Foo{
Bar1: 1,
Bar2: 2,
Bar3: 3,
}
json_data, _ = json.Marshal(f)
}
approved_fields := []string{"Bar1", "Bar2"}
f := &Foo{}
err := decode(json_data, f, approved_fields)
if err != nil {
t.Fatal(err)
}
assert.Equal(t, f.Bar1, 1)
assert.Equal(t, f.Bar2, 2)
assert.Equal(t, f.Bar3, 0)
}

How to unmarshal JSON into durations?

What is the idiomatic way to unmarshal into time.Duration in Go? How can I make use of time.ParseDuration?
The lack of JSON marshaling and unmarshaling methods on time.Duration was an unfortunate oversight. This should hopefully be resolved in Go2 (see issue #10275).
You can, however, define your own type around time.Duration that supports marshaling to the string representation of the duration and unmarshaling from either the numeric or string representations. Here is an example of such an implementation:
package main
import (
"encoding/json"
"errors"
"fmt"
"time"
)
type Duration struct {
time.Duration
}
func (d Duration) MarshalJSON() ([]byte, error) {
return json.Marshal(d.String())
}
func (d *Duration) UnmarshalJSON(b []byte) error {
var v interface{}
if err := json.Unmarshal(b, &v); err != nil {
return err
}
switch value := v.(type) {
case float64:
d.Duration = time.Duration(value)
return nil
case string:
var err error
d.Duration, err = time.ParseDuration(value)
if err != nil {
return err
}
return nil
default:
return errors.New("invalid duration")
}
}
type Message struct {
Elapsed Duration `json:"elapsed"`
}
func main() {
msgEnc, err := json.Marshal(&Message{
Elapsed: Duration{time.Second * 5},
})
if err != nil {
panic(err)
}
fmt.Printf("%s\n", msgEnc)
var msg Message
if err := json.Unmarshal([]byte(`{"elapsed": "1h"}`), &msg); err != nil {
panic(err)
}
fmt.Printf("%#v\n", msg)
}
https://play.golang.org/p/Zm6hpNR-ZJ2
Just to extend the previous answer. There is another way (very close to Tim's)
type Duration time.Duration
func (d Duration) MarshalJSON() ([]byte, error) {
return json.Marshal(time.Duration(d).String())
}
func (d *Duration) UnmarshalJSON(b []byte) error {
var v interface{}
if err := json.Unmarshal(b, &v); err != nil {
return err
}
switch value := v.(type) {
case float64:
*d = Duration(time.Duration(value))
return nil
case string:
tmp, err := time.ParseDuration(value)
if err != nil {
return err
}
*d = Duration(tmp)
return nil
default:
return errors.New("invalid duration")
}
}

Marshal and unmarshal JSON bodies without touching all fields

I'm looking for a way to unmarshal a JSON body without having to specify targets for all fields. And then be a able to "remarshal" the body with implicit fields untouched.
Something like this would be good, but doesn't work as a expected: (https://play.golang.org/p/fnVOKrmiFj)
package main
import (
"encoding/json"
"fmt"
)
type Transaction struct {
Field1 string `json:"field1"`
X map[string]interface{} `json:"-"`
}
func main() {
body := []byte(`{"field1": "value1", "field2": "value2"}`)
fmt.Printf("%+v\n", string(body))
var unmarshalledTransaction Transaction
json.Unmarshal(body, &unmarshalledTransaction)
fmt.Printf("%+v\n", unmarshalledTransaction)
remarshalledTransaction, _ := json.Marshal(&unmarshalledTransaction)
fmt.Printf("%+v\n", string(remarshalledTransaction))
}
Gives the output
{"field1": "value1", "field2": "value2"}
{Field1:value1 X:map[]}
{"field1":"value1"}
My expected result would be that unmarshalledTransaction contains the "leftover" fields in the X fields. And they are then restored when Marshalling again.
Can this be done?
You would need to implement the MarshalJSON and UnmarshalJSON interfaces, and write your own logic to remap the fields to the appropriate spots:
func (t *Transaction) MarshalJSON() ([]byte, error) {
data := t.X
data["field1"] = t.Field1
return json.Marshal(data)
}
func (t *Transaction) UnmarshalJSON(data []byte) error {
m := make(map[string]interface{})
json.Unmarshal(data, &m)
t.Field1 = m["field1"].(string)
delete(m, "field1")
t.X = m
return nil
}
https://play.golang.org/p/KBGAsXB0xA
If you want a generic solution (that would work with any struct without knowing the fields in advance), you can implement a function that would un-marshal the body into a struct and also return the "leftover" fields.
For that you'd also need to implement a function that would convert any given struct to a map (to be used to then manipulate maps in a generic way instead of known-in-advance structs).
Like so:
func structToMap(object interface{}) (map[string]interface{}, error) {
tempJson, err := json.Marshal(object)
if err != nil {
return nil, err
}
var theMap map[string]interface{}
err = json.Unmarshal(tempJson, &theMap)
if err != nil {
return nil, err
}
return theMap, nil
}
And then:
func unmarshalWithLeftovers(jsonBody []byte, target interface{}) (map[string]interface{}, error) {
err := json.Unmarshal(jsonBody, target)
if err != nil {
return nil, err
}
structMap, err := structToMap(target)
if err != nil {
return nil, err
}
var leftOvers map[string]interface{}
err = json.Unmarshal(jsonBody, &leftOvers)
if err != nil {
return nil, err
}
for k, _ := range structMap {
delete(leftOvers, k)
}
return leftOvers, nil
}
You can then combine the struct and the leftovers map in a similar fashion to re-marshal everything.
See here a working example with the same type and json string that you used in your question:
https://play.golang.org/p/Fot6YVurHH