Parsing JSON in Go Lang: Attribute without name - json

Question

I'm having some trouble parsing a JSON file from an API to Go, this is the JSON I want to parse:
{"method":"stats.provider.ex",
"result":{
"addr":"17a212wdrvEXWuipCV5gcfxdALfMdhMoqh",
"current":[{
"algo":3, // algorithm number (3 = X11)
"name":"X11", // algorithm name
"suffix":"MH", // speed suffix (kH, MH, GH, TH,...)
"profitability":"0.00045845", // current profitability in BTC/suffix/Day
"data":[{ // speed object can contain following fields:
// a (accepted), rt (rejected target), rs (rejected stale),
// rd (rejected duplicate) and ro (rejected other)
// if fields are not present, speed is 0
"a":"23.09", // accepted speed (in MH/s for X11)
"rs":"0.54", // rejected speed - stale
},
"0.0001234" // balance (unpaid)
]},
... // other algorithms here
],
"past":[{
"algo":3,
"data":[
[4863234, // timestamp; multiply with 300 to get UNIX timestamp
{"a":"28.6"}, // speed object
"0" // balance (unpaid)
],[4863235,{"a":"27.4"},"0.00000345"],
... // next entries with inc. timestamps
]},
... // other algorithms here
],
"payments":[{
"amount":"0.00431400",
"fee":"0.00023000",
"TXID":"txidhere",
"time":1453538732, // UNIX timestamp
"type":0 // payment type (0 for standard NiceHash payment)
},
... // other payments here
]
}
}
You can find more info about the API in this link: https://www.nicehash.com/doc-api
The problem I'm experiencing is in the data attribute:
"data":[{ // speed object can contain following fields:
// a (accepted), rt (rejected target), rs (rejected stale),
// rd (rejected duplicate) and ro (rejected other)
// if fields are not present, speed is 0
"a":"23.09", // accepted speed (in MH/s for X11)
"rs":"0.54", // rejected speed - stale
},
"0.0001234" // balance (unpaid)
]},
Because of the balance (unpaid) line, since it doesn't have a name I don't know how to do the struct in go.

Answer 1

It seems that this "data" object can be described by the following struct types (assuming its shape doesn't vary from your examples):
type Data struct {
Timestamp *int64
Speed *Speed
Balance *float64
}
type Speed struct {
Accepted *float64 `json:"a,string,omitempty"`
RejectedTarget *float64 `json:"rt,string,omitempty"`
RejectedStale *float64 `json:"rs,string,omitempty"`
RejectedDuplicate *float64 `json:"rd,string,omitempty"`
RejectedOther *float64 `json:"ro,string,omitempty"`
}
The "Speed" struct has JSON tags since that object is well-suited for the default JSON un/marshaler.
The "Data" struct, however, should implement a custom json.UnmarshalJSON so that it can handle the odd choice of a JSON array with varying types to serialize its fields. Note that my sample implementation below uses the json.RawMessage type to simplify things a bit by allowing the JSON unmarshaler to ensure proper JSON array syntax and store the bytes of each element separately so we can unmarshal them according to their respective types and shapes:
// Parse valid JSON arrays as "Data" by assuming one of the following shapes:
// 1: [int64, Speed, string(float64)]
// 2: [Speed, string(float64)]
func (d *Data) UnmarshalJSON(bs []byte) error {
// Ensure that the bytes contains a valid JSON array.
msgs := []json.RawMessage{}
err := json.Unmarshal(bs, &msgs)
if err != nil {
return err
}
// Parse the initial message as "Timestamp" int64, if necessary.
idx := 0
if len(msgs) == 3 {
ts, err := strconv.ParseInt(string(msgs[idx]), 10, 64)
if err != nil {
return err
}
d.Timestamp = &ts
idx++
}
// Parse the mandatory "Speed" struct per usual.
d.Speed = &Speed{}
err = json.Unmarshal(msgs[idx], &d.Speed)
idx++
if err != nil {
return err
}
// Parse the mandatory "Balance" item after trimming quotes.
balance, err := strconv.ParseFloat(string(msgs[idx][1:len(msgs[idx])-1]), 64)
if err != nil {
return err
}
d.Balance = &balance
return nil
}
As such, you can parse valid, properly shaped JSON arrays as "Data" objects like so:
jsonstr := `[
[4863234, {"a":"28.6"}, "0" ],
[{"a":"23.09","rs":"0.54"},"0.0001234"]
]`
datas := []Data{}
err := json.Unmarshal([]byte(jsonstr), &datas)
if err != nil {
panic(err)
}
// datas[0] = Data{Timestamp:4863234,Speed{Accepted:28.6},Balance:0}
// datas[1] = Data{Speed{Accepted:23.09,RejectedStale:0.54},Balance:0.0001234}
Of course, you would also need to implement json.MarshalJSON if you want to serialize "Data" objects into JSON.

Answer 2

The data field in your JSON object has an array […] as its value, and
in your example that array has two elements: an object and a string apparently containing a floating-point number.
As you can see, this is an array of geterogenous types,
hence in Go, you have two options:
Create a custom type for the elements of that array, and have an
that type implement the encoding/json.Unmarshaler interface.
Then, in that method, you can go creative about interpreting what
kind of data you're about to unmarshal, and act accordingly.
Basically, you'd peek into the input data using Decoder.Token and then
unmarshal the whole input byte slice into a value of an appropriate type
Have the value for that data field to be unmarshaled into a
slice of type []interface{} and then inspect the individual elements
by a type switch
or a series of "comma ok" type asserts.
In this case, an object will be unmarshaled into a map of type
map[string]interface{}, and that string will be unmarshaled
to a value of type string.
Basically these two approaches can be classified as "detect type as you go"
vs "unmarshal everything into data structures of the most generic types
and deal with the real typing afterwards".
Here's also a third approach.
First, it may well turn out that the types of objects in the array
which is the value of that data field are implicit from their positions
in the array. You may act accordingly by unmarshaling the value of data
into an object of your custom type implementing json.Unmarshaler, which
knows which is the real type of each data element it processes.
Second, from that
{
// speed object can contain following fields:
// a (accepted), rt (rejected target), rs (rejected stale),
// rd (rejected duplicate) and ro (rejected other)
// if fields are not present, speed is 0
"a":"23.09", // accepted speed (in MH/s for X11)
"rs":"0.54", // rejected speed - stale
}
I'd say that this "object" really can have different combinations of fields,
so to me, this looks like a candidate to be unmarshaled into
into map[string]string or map[string]float,
and not into some struct-typed object.