How do I Unmarshal JSON? - json

I am trying to unmarshal JSON into a struct. However, the struct has a field with a tag. Using reflection, and I try to see if the tag has the string "json" in it. If it does, then the json to unmarshal should simply be unmarshaled into the field as a string.
Example:
const data = `{"I":3, "S":{"phone": {"sales": "2223334444"}}}`
type A struct {
I int64
S string `sql:"type:json"`
}
Problem is simple - unmarshal "S" in the json as a string into the struct A.
This is how far I have come. But I am stuck here.
http://play.golang.org/p/YzrhjuXxGN

This is the go way of doing it - no reflection requred. Create a new type RawString and create MarshalJSON and UnmarshalJSON methods for it. (playground)
// RawString is a raw encoded JSON object.
// It implements Marshaler and Unmarshaler and can
// be used to delay JSON decoding or precompute a JSON encoding.
type RawString string
// MarshalJSON returns *m as the JSON encoding of m.
func (m *RawString) MarshalJSON() ([]byte, error) {
return []byte(*m), nil
}
// UnmarshalJSON sets *m to a copy of data.
func (m *RawString) UnmarshalJSON(data []byte) error {
if m == nil {
return errors.New("RawString: UnmarshalJSON on nil pointer")
}
*m += RawString(data)
return nil
}
const data = `{"i":3, "S":{"phone": {"sales": "2223334444"}}}`
type A struct {
I int64
S RawString `sql:"type:json"`
}
func main() {
a := A{}
err := json.Unmarshal([]byte(data), &a)
if err != nil {
log.Fatal("Unmarshal failed", err)
}
fmt.Println("Done", a)
}
I modified the implementation of RawMessage to create the above.

The problem here is that you are using one encoding format for two protocols and probably there is something wrong in your model.
That is a valid Json payload and should be handled as such. One trick that you can use is to create
another field to handle the "string" json and one to handle the "json struct".
See this modified example: I first unmarshal json and then marshal back to json to create the final string to upload to the database. One field is used for unmarshaling and the other to communicate with the DB.
package main
import(
"fmt"
"encoding/json"
)
const data = `{"i":3, "S":{"phone": {"sales": "2223334444"}}}`
type A struct {
I int64
Sjson struct {
Phone struct {
Sales string `json:"sales"`
} `json:"phone"`
} `json:"S", sql:"-"`
S string `sql:"type:json",json:"-"`
}
func main() {
msg := A{}
_ = json.Unmarshal([]byte(data), &msg)
data, _ := json.Marshal(msg.Sjson)
msg.S = string(data)
fmt.Println("Done", msg)
}

Looks like the problem is that s interface{} in your code was not addressable. For Value.SetString the Value has to be addressable and with Kind String. you can check the documentation for it - http://golang.org/pkg/reflect/#Value.SetString
How i understand it SetString would not change the value in a, since you are only working with interface s. in Laws of Reflection you can find "reflect.ValueOf is a copy of x, not x itself"(3rd Law).
To make your code work I made some type assertions, and I used reflect.ValueOf on a pointer to asserted struct.
To check if Value is settable or addressable you can use Value.CanSet ad Value.CanAddr
working code: http://play.golang.org/p/DTriENkzA8
No idea whether its correct way to do this

Related

How to convert json to string in golang and echo framework?

I have a json that I receive by post
{"endpoint": "assistance"}
I receive this like this
json_map: = make (map[string]interface{})
Now I need to assign it to a variable as a string but I don't know how to do it.
endpoint: = c.String (json_map ["endpoint"])
A type safe way to do this would be creating a struct that represents your request object and unmarshalling it.
This gives you a panic on unexpected requests.
package main
import (
"encoding/json"
"fmt"
)
type response struct {
Endpoint string
}
func main() {
jsonBody := []byte(`{"endpoint": "assistance"}`)
data := response{}
if err := json.Unmarshal(jsonBody, &data); err != nil {
panic(err)
}
fmt.Println(data.Endpoint)
}
// assistance
This program as an example safely decodes the JSON into a struct and prints the value.
What you are trying to achieve is not to convert a JSON to a string but an empty interface interface{} to a string You can achieve this by doing a type assertion:
endpoint, ok := json_map["endpoint"].(string)
if !ok {
// handle the error if the underlying type was not a string
}
Also as #Lex mentionned, it would probably be much safer to have a Go struct defining your JSON data. This way all your fields will be typed and you will no longer need this kind of type assertion.

How do I only unmarshal into a field if the json bytes field matches a certain type?

What I want to do is commented:
type foo struct {
Message string `json:"message"`
}
func bar() {
//The "message" field contains a bool type which does not
//match the equivalent "message" field in foo, a string type
jsonData := byte[]("{
\"message\": true
}")
var baz foo
//Because the type of "message" in the json bytes is
//bool and not a string, put "" inside baz.Message
json.Unmarshal(jsonData, &baz)
}
How do I Unmarshal the json byte array, then only populate the particular field if the field matches the type in the json byte array field? And if the field does type does not match the json byte array field type, put in a placeholder value of nil, "" etc. ?
There are many ways to handle this, once you fix your question to something that can be answered. The one I like is to unmarshal to a variable of type interface{}, which you can then inspect:
package main
import (
"encoding/json"
"fmt"
)
type foo struct {
Message interface{} `json:"message"`
}
func bar() {
//The "message" field contains a bool type which does not
//match the equivalent "message" field in foo, a string type
jsonData := []byte(`{
"message": true
}`)
var baz foo
//Because the type of "message" in the json bytes is
//bool and not a string, put "" inside baz.Message
err := json.Unmarshal(jsonData, &baz)
fmt.Printf("unmarhal error is: %v\n", err)
if err == nil {
fmt.Printf("baz.Message is now: %T = %v\n", baz.Message, baz.Message)
}
}
func main() {
bar()
}
(Go Playground link)
It should now be fairly obvious how to switch on the type (after decoding) and see if what you got is what you want. If so, use it. If not, use your default. If necessary, decode the incoming json to more-generic Go types first, then fill in the specific type you really want to handle.
The Unmarshaler interface in the encoding/json package allows you to have full control over how the data is decoded into your types.
You only need to implement one method on your type - UnmarshalJSON(data []byte).
In the example below, I've declared a temporary anonymous type in my UnmarshalJSON method and decoded the json into an empty interface.
I can then type assert the interface into a string, and on success I set the Message field on my original Foo type. We don't need to handle the failure case, since strings in Go have a default value of "".
Here's a working example.
type Foo struct {
Message string
}
func (this *Foo) UnmarshalJSON(data []byte) error {
raw := struct {
Message interface{} `json:"message"`
}{}
err := json.Unmarshal(data, &raw)
if err != nil {
return err
}
if msgString, ok := raw.Message.(string); ok {
this.Message = msgString
}
return nil
}

Why would adding a func in an interface cause json.Unmarshal to fail?

Why does this fail with the error :
json: cannot unmarshal object into Go struct field Person.spouse of type main.Spouse
type Spouse interface {
Name() // Adding this causes an error
}
type Address interface {
}
type Person struct {
Name string `json:"name"`
A *Address `json:"address"`
S *Spouse `json:"spouse"`
}
func main() {
b := []byte(`{
"name":"sarah",
"address":{
"street":"101 main"
},
"spouse":{
"name":"joe"
}
}
`)
p := &Person{}
if err := json.Unmarshal(b, p); err != nil {
fmt.Printf("%s", err)
}
}
Looking at the docs, I don't see why adding a function in an interface would cause an error. I was expecting json.Unmarshal to simply ignore the Name() function since it's not part of the list processed by json.Unmarshal.
Here is the code in go playground.
go version go1.10 darwin/amd64
From the fine manual:
func Unmarshal
[...]
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
You're trying to unmarshal a JSON object into an interface so the value behind that interface is going to be a map[string]interface{} to represent the key/value pairs. But map[string]interface{} doesn't have a Name() method so it doesn't implement your Spouse interface.
If we simplify your example a little and get rid of the Name() method we can see what's going on:
type Spouse interface {
}
type Person struct {
S *Spouse `json:"spouse"`
}
func main() {
b := []byte(`{"spouse":{ "name":"joe" } }`)
p := &Person{}
if err := json.Unmarshal(b, p); err != nil {
fmt.Printf("%s", err)
}
spouse := (*p.S).(map[string]interface{})
fmt.Printf("%+v\n", spouse)
}
spouse is a map[string]interface{} as documented.
The problem is that you are trying to map the value of spouse in the json with a particular data type that doesn't match. As reported in this article, json uses empty interface to map generic data types to the right one used in the file.
The json package uses map[string]interface{} and []interface{} values
to store arbitrary JSON objects and arrays; it will happily unmarshal
any valid JSON blob into a plain interface{} value. The default
concrete Go types are:
bool for JSON booleans,
float64 for JSON numbers,
string for JSON strings, and
nil for JSON null.
If you want to see it from another perspective, it means that you can't use anything other than these data types to map data in JSON. And your not empty interface isn't a valid type.

Unmarshall json value into []byte where the string could sometimes be escaped json

I have a json response that looks like this
{
"eventId":"fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e4",
"eventType":"event-type",
"eventNumber":0,
"data":"{\n \"a\": \"1\"\n}",
"metaData":"{\n \"yes\": \"no\"\n}",
"streamId":"test",
"isJson":true,
"isMetaData":true,
"isLinkMetaData":false,
"positionEventNumber":0,
"positionStreamId":"test",
"title":"0#test",
"id":"http://localhost:2113/streams/test/0",
"updated":"2017-12-14T05:09:58.816079Z"
}
the key value pairs of data, and metaData might sometimes be encoded json or it might not.
I want to decode those values into a byte array like this.
// Event represent an event to be stored.
type Event struct {
Data []byte `json:"data"`
Metadata []byte `json:"metaData"`
}
but when I try to unmarshal the json object I get the following error:
illegal base64 data at input byte 0
What could I be doing wrong here?
It works fine if I decode the data and metaData into a string, but I don't want to use a string.
You are looking for the json.RawMessage type
It is just a specialized []byte that you can then use as need be.
type Event struct {
Data json.RawMessage `json:"data"`
Metadata json.RawMessage `json:"metaData"`
}
Then you could treat it as a literal []byte via []byte(e.Data)
Here's an example of use, on play:
package main
import (
"encoding/json"
"fmt"
)
var RAW = []byte(`
{
"eventId":"fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e4",
"eventType":"event-type",
"eventNumber":0,
"data":"{\n \"a\": \"1\"\n}",
"metaData":"{\n \"yes\": \"no\"\n}",
"streamId":"test",
"isJson":true,
"isMetaData":true,
"isLinkMetaData":false,
"positionEventNumber":0,
"positionStreamId":"test",
"title":"0#test",
"id":"http://localhost:2113/streams/test/0",
"updated":"2017-12-14T05:09:58.816079Z"
}
`)
type Event struct {
Data json.RawMessage `json:"data"`
Metadata json.RawMessage `json:"metaData"`
}
func main() {
var e Event
err := json.Unmarshal(RAW, &e)
fmt.Printf("%v -- %+v\n", err, e)
b, err := json.Marshal(e)
fmt.Printf("%v -- %s\n", err, b)
}
I created a type that implements the TextUnmarshaler and TextMarshaler interfaces. The json decoder looks for this if the type doesn't implement MarshalJSON and UnmarshalJSON methods.
type RawData []byte
func (r RawData) MarshalText() (text []byte, err error) {
return r[:], nil
}
func (r *RawData) UnmarshalText(text []byte) error {
*r = text[:]
return nil
}
// Event represent an event to be stored.
type Event struct {
Data RawData `json:"data,omitempty"`
Metadata RawData `json:"metaData,omitempty"`
}
I needed this because sometimes the Data or Metadata would not be json encoded in a string, but could also be other formats like protocol buffers.

Unmarshalling a JSON that may or may not return an array?

I'm retrieving JSON from a third party website (home electricity usage), and depending on what I've requested from the site, the JSON returned may or may not be an array. For example, if I request a list of my smart meters, I get this (results truncated, due to large size):
{"gwrcmds":{"gwrcmd":{"gcmd":"SPA_UserGetSmartMeterList","gdata":{"gip":{"version":"1"...
Where gwrcmd is a single element.
But if I request electricity usage for the last half hour, I get this:
{"gwrcmds":{"gwrcmd":[{"gcmd":"DeviceGetChart","gdata":{"gip":{"version":"1" ...
See how gwrcmd is now an array?
Within my Go app, I have a struct that looks like this (again, truncated, as it goes on for a while. There's more sub-structs and properties beneath "Version":
type Response struct {
Gwrcmds struct {
Gwrcmd struct {
Gcmd string
Gdata struct {
Gip struct {
Version string
If gwrcmd is an array, Gwrcmd needs to be a []struct { }, but if it's not, it's just a regular old struct { }
The problem is that json.Unmarshal just returns an error if the JSON has an array and the struct does not have a slice (or vice versa).
Would I need to create a second struct that duplicates the first one (except with a []struct { } instead), or is there a better way to do it? I thought of something with interfaces, but I haven't really touched those yet, so I'm not 100% sure on them.
Usually, whenever you have a JSON value of unknown type, you will use json.RawMessage to get it, peek into it, and unmarshal it correctly into the corresponding type. A simplified example:
// The A here can be either an object, or a JSON array of objects.
type Response struct {
RawAWrapper struct {
RawA json.RawMessage `json:"a"`
}
A A `json:"-"`
As []A `json:"-"`
}
type A struct {
B string
}
func (r *Response) UnmarshalJSON(b []byte) error {
if err := json.Unmarshal(b, &r.RawAWrapper); err != nil {
return err
}
if r.RawAWrapper.RawA[0] == '[' {
return json.Unmarshal(r.RawAWrapper.RawA, &r.As)
}
return json.Unmarshal(r.RawAWrapper.RawA, &r.A)
}
Playground: http://play.golang.org/p/2d_OrGltDu.
Guessing the content based on the first byte doesn't seem too robust to me though. Usually you'll have some sort of a clue in your JSON (like a length or type field on the same level as the dynamic one) that tells you whether you have an object or an array.
See also:
How can you decode multiple message types with golang websockets?
Partly JSON unmarshal into a map in Go
You can try to make custom json unmarshal method, like
func (a *GwrcmCustom) UnmarshalJSON(b []byte) (err error) {
g, ga := Gwrcmd{}, []Gwrcmd{}
if err = json.Unmarshal(b, &g); err == nil {
*a = make([]Gwrcmd, 1)
[]Gwrcmd(*a)[0] = Gwrcmd(g)
return
}
if err = json.Unmarshal(b, &ga); err == nil {
*a = GwrcmCustom(ga)
return
}
return
}
GwrcmCustom is a custom type, slice of Gwrcm
type GwrcmCustom []Gwrcmd
So we will get slice always
Try this on Go playground
I hope this will help