Unmarshal on reflected value - json

Here is the code
package main
import (
"fmt"
"encoding/json"
"reflect"
)
var (
datajson []byte
//ref mapp
)
type mapp map[string]reflect.Type
type User struct {
Name string
//Type map[string]reflect.Type
}
func MustJSONEncode(i interface{}) []byte {
result, err := json.Marshal(i)
if err != nil {
panic(err)
}
return result
}
func MustJSONDecode(b []byte, i interface{}) {
err := json.Unmarshal(b, i)
if err != nil {
panic(err)
}
}
func Store(a interface{}) {
datajson = MustJSONEncode(a)
//fmt.Println(datajson)
}
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType)
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("%s", obj)
}
func main() {
dummy := &User{}
david := User{Name: "DavidMahon"}
Store(david)
Get(datajson, dummy)
}
In the Get function
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType)
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("%s", obj)
}
I am unable to unmarshal the json into the underlying object type.
Whats wrong here? I am so stuck here. Something very simple yet so difficult to figure out.
Thanks
UPDATE::Goal of this problem is to retreive a fully formed object of type passed in Get function.
The approach mentioned by Nick on the comment below doesnot get me the actual object which I already tried before. I can anyways retrieve the data (even when the object has recursive objects underneath) in a map like this
func Get(a []byte) {
var f interface{}
//buf := bytes.NewBuffer(a)
//v := buf.String()
//usr := &User{}
MustJSONDecode(a, &f)
fmt.Printf("\n %v \n", f)
}
However I need the actual object back not just the data. Something like user := &User{"SomeName"} where I need user object back from Unmarshall. The trick is somewhere in reflection but dont know how.

I'm confused as to why you want to do this, but here is how to fix it
func Get(a []byte, b interface{}) {
objType := reflect.TypeOf(b).Elem()
obj := reflect.New(objType).Interface()
//fmt.Println(obj)
MustJSONDecode(a, &obj)
fmt.Printf("obj = %#v\n", obj)
}
Note the call to Interface().
Playground link
It seems to me that you are going to a lot of trouble to make an empty &User when you already have one in b, eg
func Get(a []byte, b interface{}) {
MustJSONDecode(a, &b)
fmt.Printf("obj = %#v\n", b)
}
But I'm guessing there is some more to this plan which isn't apparent here!

reflect.New(objType) returns a reflect.Value Which is not the thing as the interface you passed. According to the docs for Value It is a struct with only unexported fields. the json package can't work with unexported fields. Since it's not the same object as you passed in and it's not even json encodable/decodable the json package will fail.
You will probably find the Laws of Reflection article useful while trying to use the reflect package.

Related

Read extern JSON file

I am trying to read the following JSON file:
{
"a":1,
"b":2,
"c":3
}
I have tried this but I found that I had to write each field of the JSON file into a struct but I really don't want to have all my JSON file in my Go code.
import (
"fmt"
"encoding/json"
"io/ioutil"
)
type Data struct {
A string `json:"a"`
B string `json:"b"`
C string `json:"c"`
}
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
data := Data{}
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for _, letter := range data.Letter {
fmt.Println(letter)
}
}
Is there a way to bypass this thing with something like json.load(file) in Python?
If you only want to support integer values, you could unmarshal your data into a map[string]int. Note that the order of a map is not defined, so the below program's output is non-deterministic for the input.
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
)
func main() {
file, _ := ioutil.ReadFile("/path/to/file.json")
var data map[string]int
if err := json.Unmarshal(file ,&data); err != nil {
panic(err)
}
for letter := range data {
fmt.Println(letter)
}
}
You can unmarshal any JSON data in this way:
var data interface{}
if err := json.Unmarshal(..., &data); err != nil {
// handle error
}
Though, in this way you should handle all the reflection-related stuffs
since you don't know what type the root data is, and its fields.
Even worse, your data might not be map at all.
It can be any valid JSON data type like array, string, integer, etc.
Here's a playground link: https://play.golang.org/p/DiceOv4sATO
It's impossible to do anything as simple as in Python, because Go is strictly typed, so it's necessary to pass your target into the unmarshal function.
What you've written could otherwise be shortened, slightly, to something like this:
func UnmarshalJSONFile(path string, i interface{}) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
return json.NewDecoder(f).Decode(i)
}
But then to use it, you would do this:
func main() {
data := Data{}
if err := UnmarshalJSONFile("/path/to/file.json", &data); err != nil {
panic(err)
}
}
But you can see that the UnmarshalJSONFile is so simple, it hardly warrants a standard library function.

How to decode json strings to sync.Map instead of normal map in Go1.9?

I can decode json strings to a map with go language like this:
func main(){
date := []byte(`{"127.1":{"host":"host1","list":["list123","list456"]},"127.2":{"host":"host2","list":["list223","list256"]}}`)
var x interface{}
json.Unmarshal(date, &x)
t := x.(map[string]interface{})
var aa []interface{}
aa = (t["127.2"].(map[string]interface{})["list"])
for _, v := range aa {
fmt.Println(v.(string))
}
}
but I wonder how to decode it to a sync.Map in Go1.9. I have tried many ways but failed, can anyone help me?
I tried like this:
func main(){
date := []byte(`{"127.1":{"host":"host1","list":["list123","list456"]},"127.2":{"host":"host2","list":["list223","list256"]}}`)
var x interface{}
json.Unmarshal(date, &x)
t := x.((sync.Map)[string]interface{}) //compile error
}
Also I tried like this:
func main(){
date := []byte(`{"127.1":{"host":"host1","list":["list123","list456"]},"127.2":{"host":"host2","list":["list223","list256"]}}`)
var x sync.Map
json.Unmarshal(date, &x)
fmt.Println(x) // but the map has nothing
}
In case you need a snippet to do it another way around
func (f Foo) MarshalJSON() ([]byte, error) {
tmpMap := make(map[YourTypeOfKey]YourTypeOfValue)
f.Range(func(k, v interface{}) bool {
tmpMap[k.(YourTypeOfKey)] = v.(YourTypeOfValue)
return true
})
return json.Marshal(tmpMap)
}
You cannot directly unmarshal into a sync.Map, because sync.Map has no exported fields (so the Unmarshaler doesn't have any way to store data in it), and it doesn't implement the json.Unmarshaler interface.
So you'll have to handle this yourself, probably by including a sync.Map in a type of your own, and implementing json.Unmarshaler on that type:
type Foo struct {
sync.Map
}
func (f *Foo) UnmarshalJSON(data []byte) error {
var tmpMap map[string]interface{}
if err := json.Unmarshal(data, &tmpMap); err != nil {
return err
}
for key, value := range tmpMap {
f.Store(key, value)
}
return nil
}
Just adding to #jakub great answer about MarshalJSON, you can also have a generic function that doesn't check the type (since json doesn't care either)
func MarshalJSON(m *sync.Map) ([]byte, error) {
tmpMap := make(map[interface{}]interface{})
m.Range(func(k, v interface{}) bool {
tmpMap[k] = v
return true
})
return json.Marshal(tmpMap)
}
This could take any kind of sync.Map and just Marshal it.
To add to the UnMarshal function from #Flimzy's answer, you can play a bit with the types by omitting the string part to make it more generics (pun intended, cry in corner):
func UnmarshalJSON(data []byte) (*sync.Map, error) {
var tmpMap map[interface{}]interface{}
m := &sync.Map{}
if err := json.Unmarshal(data, &tmpMap); err != nil {
return m, err
}
for key, value := range tmpMap {
m.Store(key, value)
}
return m, nil
}
In general it is good to play with the same Types when doing this sort of manipulation, because we are lazy and we don't want to rewrite functions ;)

Go Unmarshal reflect.Type got map[string]interface{}

I want to convert json string to some struct ,use as func param
I use reflect to get Param Type,this work fine ,
but if I use json.Unmarshal I always get map[string]interface{}
this is a mini Run sample,File name is json_test.go
package testJson
import (
"reflect"
"encoding/json"
"testing"
"log"
)
type LoginForm struct {
Name string
}
func Login(login LoginForm) {
}
func TestRun(t *testing.T) {
_func := Login
f := reflect.ValueOf(_func)
funcType := f.Type()
paramType := funcType.In(0)
log.Print(paramType.Name())
v := reflect.New(paramType).Elem().Interface()
jsonstr := `{"Name":"123"}`
log.Print(jsonstr)
log.Print(reflect.TypeOf(v).Name())
log.Print("%+v", v)
err := json.Unmarshal([]byte(jsonstr), &v)
if err != nil {
panic(err)
}
log.Print(reflect.TypeOf(v).Name())
log.Print( v)
var in []reflect.Value
in = make([]reflect.Value, funcType.NumIn())
in[0] = reflect.ValueOf(v)
f.Call(in)
}
Thanks for your help!
Don't reflect.New a pointer type because the result of that is a pointer to the pointer, instead reflect.New the Elem of the pointer type.
Change this:
v := reflect.New(paramType).Elem().Interface()
To this:
v := reflect.New(paramType.Elem()).Interface()
At this point v is already a pointer to LoginForm so you don't need to get its address with & when passing to Unmarshal.
https://play.golang.org/p/JM4fRXb7fo
Also the reason you got map[string]interface{} is because the return type of the Interface() method is unsurprisingly enough interface{}, so while v's underlying type could be anything its "topmost" type is interface{}. So then, doing &v will get you *interface{} as opposed to the inteded *LoginForm. And in the docs you can find that trying to unmarshal a JSON object into an interface{} will result in a value of type map[string]interface{}.

Ignore JSON tags when marshalling

I am getting JSON data from an external source. The field names in this JSON are not something I want to carry with me, so I am converting them to names that make sense to me using the json:"originalname" tags.
When I marshal such an object back to JSON, I naturally get the ugly (original) names again.
Is there a way to ignore tags when marshalling? Or a way to specify a different name for marshall and unmarshall?
To clarify, I have prepared an example in the playground and pasted the same code below.
Thanks in advance.
package main
import (
"encoding/json"
"fmt"
)
type Band struct {
Name string `json:"bandname"`
Albums int `json:"albumcount"`
}
func main() {
// JSON -> Object
data := []byte(`{"bandname": "AC/DC","albumcount": 10}`)
band := &Band{}
json.Unmarshal(data, band)
// Object -> JSON
str, _ := json.Marshal(band)
fmt.Println("Actual Result: ", string(str))
fmt.Println("Desired Result:", `{"Name": "AC/DC","Albums": 10}`)
// Output:
// Actual Result: {"bandname":"AC/DC","albumcount":10}
// Desired Result: {"Name": "AC/DC","Albums": 10}
}
You could implement
type Marshaler interface {
MarshalJSON() ([]byte, error)
}
from the standard library's encoding/json package. Example:
type Band struct {
Name string `json:"bandname"`
Albums int `json:"albumcount"`
}
func (b Band) MarshalJSON() ([]byte, error) {
n, _ := json.Marshal(b.Name)
a, _ := json.Marshal(b.Albums)
return []byte(`{"Name":` + string(n) + `,"Albums":` + string(a) + `}`)
}
It's admittedly not a very nice solution, though.
As a generic solution, you could use reflection to create a new type that removes the json tags and then marshall that.
func getVariantStructValue(v reflect.Value, t reflect.Type) reflect.Value {
sf := make([]reflect.StructField, 0)
for i := 0; i < t.NumField(); i++ {
sf = append(sf, t.Field(i))
if t.Field(i).Tag.Get("json") != "" {
sf[i].Tag = ``
}
}
newType := reflect.StructOf(sf)
return v.Convert(newType)
}
func MarshalIgnoreTags(obj interface{}) ([]byte, error) {
value := reflect.ValueOf(obj)
t := value.Type()
newValue := getVariantStructValue(value, t)
return json.Marshal(newValue.Interface())
}
And you would just call it using:
str, _ := MarshalIgnoreTags(band)
Doing the opposite is a little trickier (ignore tags when unmarshalling JSON), but possible with mapstructure:
func UnmarshalIgnoreTags(data []byte, obj interface{}) error {
rv := reflect.ValueOf(obj)
if rv.Kind() != reflect.Ptr || rv.IsNil() {
return errors.New("unmarshal destination obj must be a non-nil pointer")
}
value := reflect.Indirect(rv)
t := value.Type()
newValue := getVariantStructValue(value, t)
i := newValue.Interface()
err := json.Unmarshal(data, &i)
if err == nil {
// We use mapstructure because i is of type map[string]interface{} and it's the easiest way to convert back to struct type
// See: https://stackoverflow.com/a/38939459/2516916
mapstructure.Decode(i, obj)
}
return err
}
See playground here: https://play.golang.org/p/XVYGigM71Cf
One could also use a library as mentioned in this thread: https://stackoverflow.com/a/50966527/5649638
This will give you a structs like this:
type TestJson struct {
Name string `json:"name" newtag:"newname"`
Age int `json:"age" newtag:"newage"`
}

strict json parser in golang

In Go, I have some JSON from third-party API and then I'm trying to parse it:
b := []byte(`{"age":21,"married":true}`)
var response_hash map[string]string
_ = json.Unmarshal(b, &response_hash)
But it fails (response_hash becomes empty) and Unmarshal can handle only JSON with quotes:
b := []byte(`{"age":"21","married":"true"}`)
var response_hash map[string]string
_ = json.Unmarshal(b, &response_hash)
Is there any way to read the first version of JSON from third-party API?
Go is a strongly typed language. You need to specify what types the JSON encoder is to expect. You're creating a map of string values but your two json values are an integer and a boolean value.
Here's a working example with a struct.
http://play.golang.org/p/oI1JD1UUhu
package main
import (
"fmt"
"encoding/json"
)
type user struct {
Age int
Married bool
}
func main() {
src_json := []byte(`{"age":21,"married":true}`)
u := user{}
err := json.Unmarshal(src_json, &u)
if err != nil {
panic(err)
}
fmt.Printf("Age: %d\n", u.Age)
fmt.Printf("Married: %v\n", u.Married)
}
If you want to do it in a more dynamic manner you can use a map of interface{} values. You just have to do type assertions when you use the values.
http://play.golang.org/p/zmT3sPimZC
package main
import (
"fmt"
"encoding/json"
)
func main() {
src_json := []byte(`{"age":21,"married":true}`)
// Map of interfaces can receive any value types
u := map[string]interface{}{}
err := json.Unmarshal(src_json, &u)
if err != nil {
panic(err)
}
// Type assert values
// Unmarshal stores "age" as a float even though it's an int.
fmt.Printf("Age: %1.0f\n", u["age"].(float64))
fmt.Printf("Married: %v\n", u["married"].(bool))
}