Unmarshall into an interface type - json

I expected below code to print an object of type struct J, however it prints a map object of type map[string]interface{}. I can feel why it acts like that, however when I run, reflect.ValueOf(i).Kind(), it returns Struct, so it kinda gives me the impression that Unmarshal method should return type J instead of a map. Could anyone enlighten me ?
type J struct {
Text string
}
func main() {
j := J{}
var i interface{} = j
js := "{\"Text\": \"lala\"}"
json.Unmarshal([]byte(js), &i)
fmt.Printf("%#v", i)
}

The type you're passing into Unmarshal is not *J, you're passing in an *interface{}.
When the json package reflects what the type is of the pointer it received, it sees interface{}, so it then uses the default types of the package to unmarshal into, which are
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
There is almost never a reason to use a pointer to an interface. If you find yourself using a pointer to an interface, and you don't know exactly why, then it's probably a mistake. If you want to unmarshal into J, then pass that in directly. If you need to assign that to an intermediary interface, make sure you use a pointer to the original value, not a pointer to its interface.
http://play.golang.org/p/uJDFKfSIxN
j := J{}
var i interface{} = &j
js := "{\"Text\": \"lala\"}"
json.Unmarshal([]byte(js), i)
fmt.Printf("%#v", i)

This is expected behavior: instead of giving json.Unmarshal a pointer to a properly typed place in memory you give it a pointer to a place in memory with type interface{}. It can essentially store anything in there under the type the JSON defines, so it does just that.
See it like this:
Unmarshal gets a place to store the data v with type interface{}
Unmarshal detects a map encoded as JSON
Unmarshal sees that the target type is of type interface{}, creates a go map from it and stores it in v
If you would have given it a different type than interface{} the process would have looked like this:
Unmarshal gets a place to store the data v with type struct main.J
Unmarshal detects a map encoded as JSON
Unmarshal sees that the target type is struct main.J and begins to recursively fit the data to the type
The main point is, that the initial assignment
var i interface{} = j
is completely ignored by json.Unmarshal.

Related

Dynamic Type in Unmarshal

I currently having the following problem:
I get a []byte / string via websocket which looks like
eventname {"JSON": "data", "In": "different formats"}
I split the string by the whitespace between the eventname and the JSON data and, depending on the eventname, I want to json.Unmarshal() the JSON data into a specific type or a var of a specific type, to make sure its all type safe.
So I would probably have a map which holds all the possible eventnames and the corresponding type for the JSON data, but im not sure how I would save a type, maybe by reference or by the stringified name?
type EventTypeList map[string]*interface{}
or
type EventTypeList map[string]string
So I can lookup if the event is in the EventTypeList and then let the Unmarshal func parse the data into the type from the map.
So basicly instead of the specific type "SpecificData":
type SpecificData struct {
JSON string
In string
}
socketData := SpecificData{}
err := json.Unmarshal(jsonData, &socketData)
I want to dynamically create the socketData var by the type from the EventTypeList
socketData := [dynamically determine this Type from EventTypeList]{}
After I parsed the data into that type, I want to call all listeners for that event and give them the socketData with the right type, so they can work with the expected data.
But im really not sure how I would accomplish this, if this is even possible or if this is even the right way...
Thanks for any help!
Ok, after reasearching, I think what I wanted to achive is not possible, because the type needs to be know at compile time, as it has been mentioned here:
Golang: cast an interface to a typed variable dynamically
But I think I can just pass the raw JSON string to the listener callback and Unmarshal the data in that function where the type of the data is known.
Thanks for all contributions!

Preserve json.RawMessage through multiple marshallings

Background
I'm working with JSON data that must be non-repudiable.
The API that grants me this data also has a service to verify that the data originally came from them.
As best as I can tell, they do this by requiring that the complete JSON they originally sent needs to be supplied to them inside another JSON request, with no byte changes.
Issue
I can't seem to preserve the original JSON!
Because I cannot modify the original JSON, I have carefully preserved it as a json.RawMessage when unmarshalling:
// struct I unmarshal my original data into
type SignedResult struct {
Raw json.RawMessage `json:"random"`
Signature string `json:"signature"`
...
}
// struct I marshal my data back into
type VerifiedSignatureReq {
Raw json.RawMessage `json:"random"`
Signature string `json:"signature"`
}
// ... getData is placeholder for function that gets my data
response := SignedResult{}
x, _ := json.Unmarshal(getData(), &response)
// do some post-processing with SignedResult that does not alter `Raw` or `Signature`
// trouble begins here - x.Raw started off as json.RawMessage...
y := json.Marshal(VerifiedSignatureReq{Raw: x.Raw, Signature: x.Signature}
// but now y.Raw is base64-encoded.
The problem is that []bytes / RawMessages are base64-encoded when marshaled. So I can't use this method, because it completely alters the string.
I'm unsure how to ensure this string is correctly preserved. I had assumed that the json.RawMessage specification in my struct would survive the perils of marshaling an already marshaled instance because it implements the Marshaler interface, but I appear mistaken.
Things I've Tried
My next attempt was to try:
// struct I unmarshal my original data into
type SignedResult struct {
Raw json.RawMessage `json:"random"`
Signature string `json:"signature"`
...
}
// struct I marshal my data back into
type VerifiedSignatureReq {
Raw map[string]interface{} `json:"random"`
Signature string `json:"signature"`
}
// ... getData is placeholder for function that gets my data
response := SignedResult{}
x, _ := json.Unmarshal(getData(), &response)
// do some post-processing with SignedResult that does not alter `Raw` or `Signature`
var object map[string]interface{}
json.Unmarshal(x.Raw, &object)
// now correctly generates the JSON structure.
y := json.Marshal(VerifiedSignatureReq{Raw: object, Signature: x.Signature}
// but now this is not the same JSON string as received!
The issue with this approach is that there are minor byte-wise differences in the spacing between the data. It no longer looks exactly the same when catted to a file.
I cannot use string(x.Raw) either because it escapes certain characters when marshaled with \.
You will need a custom type with its own marshaler, in place of json.RawMessage for your VerifiedSignatureReq struct to use. Example:
type VerifiedSignatureReq {
Raw RawMessage `json:"random"`
Signature string `json:"signature"`
}
type RawMessage []byte
func (m RawMessage) MarshalJSON() ([]byte, error) {
return []byte(m), nil
}

mgo error when unmarshal map[string]interface{}

I want to store an arbitrary json object in a struct:
type C struct {
Name string `json:"name" bson:"name"`
Config map[string]interface{} `json:"config" bson:"config"`
}
This works fine when I store any deeply nested json object, but when I go retrieve it and mgo tries to Unmarshal it, I get:
Unmarshal can't deal with struct values. Use a pointer.
I'm not sure what's supposed to be a pointer. I get the same errors if I change it to:
Config *map[string]interface{}
The error occurs here: https://github.com/MG-RAST/golib/blob/master/mgo/bson/bson.go#L493
I don't know what it's reflecting on though.
So when you are unmarshaling the input argument takes a pointer to the struct, and you need to define a type in order to use a pointer to a struct.
type myMap map[string]interface{}
Then you can make a pointer to that type notice the ampersand to indicate pointer to your struct for type myMap, with json you could do something like so:
json := []Byte{`{"name": "value"}`}
c := &myMap{"value": "name"}
json.Unmarshal(c, json)
So you need *myMap to the struct not a pointer to the type. In order to explain the specific solution to this problem you need to add the context of how mongodb is unmarshaling your json.

Parsing an unfamiliar YAML/JSON file in Go

How do you parse YAML data, in Go, without knowing its structure in advance? All of the examples I've seen assume you want to decode a marshaled map whose keys you already know. What if you don't know the keys? What if it's not a map but a marshaled list, a scalar, or some other common type?
Though I'm principally concerned with YAML, here, it seems like the technique might be generally useful for JSON, etc.. since there's a general pattern for parsing structured data (tagged structs, obviously).
For JSON, unmarshal the data to an interface{} value. Use type assertions to determine what's in the value.
var v interface{}
err := json.Unmarshal(data, &v)
if err != nil {
// handle error
}
switch v := v.(type) {
case string:
fmt.Println("string:", v)
case float64:
fmt.Println("number:", v)
case []interface{}:
fmt.Println("array:", v)
case map[string]interface{}:
fmt.Println("object:", v)
case nil:
fmt.Println("nil")
}
In the case of JSON, the standard library json.Unmarshal function will unmarshal arbitrary JSON if you like, if you pass it a pointer to an uninitialized empty interface. (See this example.)
The official docs even say as much:
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
Edit: Though not documented, the same is true of the yaml package; I tested on my workstation, and passing in a pointer to an uninitialized empty interface results in the initialization of correct arrays, maps, and primitives.

Unmarshalling JSON into Go interface{}

I have this struct with a field of type interface{}. In the process of caching it using memcached (https://github.com/bradfitz/gomemcache), the struct is marshalled into a JSON, which is then unmarshalled back into the struct when retrieved from the cache. The resulting interface{} field inevitably points to an object of type map[string]interface{} (as in, the interface{} field can only type asserted as map[string]interface{}), the marshalling and unmarshalling process not having preserved the type information. Is there any way to save this information in the marshalling process, in such a way that it can be unmarshalled properly? Or do I have to use some other codec or something?
type A struct {
value interface{}
}
type B struct {
name string
id string
}
func main() {
a := A{value: B{name: "hi", id: "12345"}}
cache.Set("a", a) // Marshals 'a' into JSON and stores in cache
result = cache.Get("a") // Retrieves 'a' from cache and unmarshals
fmt.Printf("%s", result.value.(B).name) // Generates error saying that
// map[string]interface{} cannot be type asserted as a 'B' struct
fmt.Printf("%s", result.value.(map[string]interface{})["name"].(string)) // Correctly prints "12345"
}
Short version, no you can't do that, you have few options though.
Change A.Value to use B instead of interface{}.
Add a function to A that converts A.Value from a map to B if it isn't already B.
Use encoding/gob and store the bytes in memcache then convert it back with a function like NewA(b []byte) *A.
For using gob, you have to register each struct with it first before encoding / decoding, example:
func init() {
//where you should register your types, just once
gob.Register(A{})
gob.Register(B{})
}
func main() {
var (
buf bytes.Buffer
enc = gob.NewEncoder(&buf)
dec = gob.NewDecoder(&buf)
val = A{B{"name", "id"}}
r A
)
fmt.Println(enc.Encode(&val))
fmt.Println(dec.Decode(&r))
fmt.Printf("%#v", r)
}
JSON isn't able to encode the depth of type information that you can in Go, so you'll always get back the following basic types when unmarshalling:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
From go docs: http://golang.org/pkg/encoding/json/#Unmarshal
If you have that knowledge about the types you need, you can write some methods to construct the right variables on unmarshalling perhaps?