Golang Function Call in Map - json

I am a newbie at GO Programming. Here is scenario :-
There exists a JSON file that looks like this :-
{
"template": "linuxbase1",
"checkname": ["check_disk"],
"checkmethod": ["check_disk"]
}
I am Unmarshalling this data into a structure :-
package func1
import (
"io/ioutil"
"os"
"encoding/json"
"fmt"
)
type pluginfunc func() string
type Plugindata struct {
Template string `json:"template"`
Checkname []string `json:"checkname"`
Checkmethod []pluginfunc `json:"checkmethod"`
}
var (
Templatepath = "json_sample1.json"
Templateitems Plugindata
)
func Gettemplatedata() {
tdata, err := ioutil.ReadFile(Templatepath)
if err != nil {
fmt.Printf("Unable to read file %s. Error - %v\n",Templatepath, err.Error())
os.Exit(3)
}
json.Unmarshal(tdata, &Templateitems)
}
The "check_disk" function is here :-
package func1
func check_disk() string {
return "Called check_disk"
}
This is the program with main() :-
package main
import (
"fmt"
"checksexpt/func1"
)
func main() {
func1.Gettemplatedata()
fmt.Printf("Templateitems in Main() => %v\n",func1.Templateitems)
for index,funcname := range func1.Templateitems.Checkmethod {
fmt.Printf("%d = %s\n",index,funcname())
}
}
As expected, when I run main(); I see the error :-
Templateitems in Main() => {linuxbase1 [check_cpu check_disk] [<nil> <nil>]}
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x40115e]
goroutine 1 [running]:
panic(0x50e980, 0xc82000a100)
/opt/go/src/runtime/panic.go:481 +0x3e6
So, I am trying to grab a string from the JSON file and treat it as a function call. That obviously fails ! But, the primary constraint here is that the function names have to be picked from the JSON file. How can I do this ? I know that I can create a static map as follows :-
type checkfunc func() string
var (
Templateitems = map[string]map[string]checkfunc {
"linuxbase1": {
"check_disk": check_disk,
},
}
)
So, A call like - Templateitems["linuxbase1"]["check_disk"]() would work just fine. But, I dont want to create any such static map as the elements in that map needs to keep growing. Any ideas on this?

There is no direct way to parse a function directly from a JSON value. Also, you cannot use string values to refer to variables. So a string check_cpu would not be able to refer to the function with the same name directly.
What you can do instead is parse the json string as is and have a global map for functions. That way, you can call your functions like so:
var funcMap = map[string]pluginfunc{
"check_disk": check_disk,
"check_cpu": check_cpu
}
In your main loop:
for index, funcname := range func1.Templateitems.Checkmethod {
fmt.Printf("%d = %s\n", index, funcMap[funcname]())
}
If however, you really need to put the value in your structure, you can try implementing UnmarshalJSON from the json.Unmarshaler interface. A simple example would be:
type pf map[string]pluginfunc
type Plugindata struct {
Template string `json:"template"`
Checkname []string `json:"checkname"`
Checkmethod pf `json:"checkmethod"`
}
func (p *pf) UnmarshalJSON(data []byte) error {
d := []string{}
if err := json.Unmarshal(data, &d); err != nil {
return err
}
*p = make(pf)
for _, s := range d {
(*p)[s] = funcMap[s]
}
return nil
}
var funcMap = pf{
"check_disk": check_disk,
"check_cpu": check_cpu
}
func main() {
json.Unmarshal(tdata, &Templateitems)
for k, f := range Templateitems.Checkmethod {
fmt.Printf("%s -- %s\n", k, f())
}
}
Working code
Note that this way is not as readable or simple as the first method and it still relies on a function map.
You can read more about json.Unmarshaler here.

Related

Golang JSON decoder does not recognize type

Still a Golang beginner, I am trying to code a generic function to serve ReST requests. I pass a function to create a new resource (struct) with an interface implemented on it, because I would also invoke methods on the struct. When decoding the JSON, logging the type shows the correct (struct) type, but the JSON decoder seems to only recognize the interface, which it cannot decode to.
package main
import (
"encoding/json"
"github.com/julienschmidt/httprouter"
"log"
"net/http"
"strings"
)
// general resource interface
type resource interface {
// check semantics and return an array of errors or nil if no error found
check() []string
// update the resource in backend
update() error
}
// specific resource named "anchor"
type anchor struct {
ID string `json:"id"`
Name string `json:"name"`
}
func newAnchor() resource {
return anchor{}
}
func (a anchor) check() []string {
return nil
}
func (a anchor) update() error {
return nil
}
// generic function to create (POST) a new resource
func restCreate(newResource func() resource) httprouter.Handle {
return func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
const F = "restCreate"
var checkErrs []string
res := newResource()
log.Printf("%s res type %T\n", F, res)
dcdr := json.NewDecoder(r.Body)
err := dcdr.Decode(&res)
log.Printf("%s Unmarshalled into %T: %+v\n", F, res, res)
if err == nil {
checkErrs = res.check()
}
switch {
case err != nil:
w.WriteHeader(http.StatusInternalServerError)
log.Printf("[ERR] %s: %v\n", F, err)
case checkErrs != nil:
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte(strings.Join(checkErrs, "\n")))
log.Printf("%s: %v\n", F, err)
default:
res.update()
bs, _ := json.Marshal(res)
w.Write(bs)
}
}
}
func main() {
r := httprouter.New()
r.POST("/anchors", restCreate(newAnchor))
http.ListenAndServe(":8080", r)
}
The execution log shows:
restCreate res type main.anchor
restCreate Unmarshalled into main.anchor: {ID: Name:}
[ERR] restCreate: json: cannot unmarshal object into Go value of type main.resource
Why does Printf show the struct type and json.Decoder the interface?
I'd appreciate any indicator on what's going wrong and how to solve this in a generic way...
It is because you try to use a pointer to the interface to unmarshal into. You need return a pointer in a function
func newAnchor() resource {
return &anchor{}
}
And you don't need to get address in this line:
err := dcdr.Decode(&res)
Here is small working example: https://play.golang.org/p/3E0RmGTURO
You cannot Unmarshal into an interface unless variable is holding a pointer to the concrete type you desire, as json.Decode won't know which concrete type to use. You have two workarounds at your disposal:
Having newResource return a concrete type under the hood:
func newResource() resource {
return &anchor{}
}
This way json.Decode knows to unmarshal your JSON into an anchor.
Use newAnchor instead of newResource: this will be more readable in your restCreate function, and is more idiomatic[1].
[1] http://idiomaticgo.com/post/best-practice/accept-interfaces-return-structs/

Is there a convenient way to convert a JSON-like map[string]interface{} to struct , and vice versa, with tags in Go? [duplicate]

I want to convert a struct to map in Golang. It would also be nice if I could use the JSON tags as keys in the created map (otherwise defaulting to field name).
Edit Dec 14, 2020
Since structs repo was archived, you can use mapstructure instead.
Edit TL;DR version, Jun 15, 2015
If you want the fast solution for converting a structure to map, see the accepted answer, upvote it and use that package.
Happy coding! :)
Original Post
So far I have this function, I am using the reflect package but I don't understand well how to use the package, please bear with me.
func ConvertToMap(model interface{}) bson.M {
ret := bson.M{}
modelReflect := reflect.ValueOf(model)
if modelReflect.Kind() == reflect.Ptr {
modelReflect = modelReflect.Elem()
}
modelRefType := modelReflect.Type()
fieldsCount := modelReflect.NumField()
var fieldData interface{}
for i := 0; i < fieldsCount; i++ {
field := modelReflect.Field(i)
switch field.Kind() {
case reflect.Struct:
fallthrough
case reflect.Ptr:
fieldData = ConvertToMap(field.Interface())
default:
fieldData = field.Interface()
}
ret[modelRefType.Field(i).Name] = fieldData
}
return ret
}
Also I looked at JSON package source code, because it should contain my needed implementation (or parts of it) but don't understand too much.
I also had need for something like this. I was using an internal package which was converting a struct to a map. I decided to open source it with other struct based high level functions. Have a look:
https://github.com/fatih/structs
It has support for:
Convert struct to a map
Extract the fields of a struct to a []string
Extract the values of a struct to a []values
Check if a struct is initialized or not
Check if a passed interface is a struct or a pointer to struct
You can see some examples here: http://godoc.org/github.com/fatih/structs#pkg-examples
For example converting a struct to a map is a simple:
type Server struct {
Name string
ID int32
Enabled bool
}
s := &Server{
Name: "gopher",
ID: 123456,
Enabled: true,
}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(s)
The structs package has support for anonymous (embedded) fields and nested structs. The package provides to filter certain fields via field tags.
From struct to map[string]interface{}
package main
import (
"fmt"
"encoding/json"
)
type MyData struct {
One int
Two string
Three int
}
func main() {
in := &MyData{One: 1, Two: "second"}
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
// iterate through inrecs
for field, val := range inInterface {
fmt.Println("KV Pair: ", field, val)
}
}
go playground here
Here is a function I've written in the past to convert a struct to a map, using tags as keys
// ToMap converts a struct to a map using the struct's tags.
//
// ToMap uses tags on struct fields to decide which fields to add to the
// returned map.
func ToMap(in interface{}, tag string) (map[string]interface{}, error){
out := make(map[string]interface{})
v := reflect.ValueOf(in)
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
// we only accept structs
if v.Kind() != reflect.Struct {
return nil, fmt.Errorf("ToMap only accepts structs; got %T", v)
}
typ := v.Type()
for i := 0; i < v.NumField(); i++ {
// gets us a StructField
fi := typ.Field(i)
if tagv := fi.Tag.Get(tag); tagv != "" {
// set key of map to value in struct field
out[tagv] = v.Field(i).Interface()
}
}
return out, nil
}
Runnable example here.
Note, if you have multiple fields with the same tag value, then you will obviously not be able to store them all within a map. It might be prudent to return an error if that happens.
I like the importable package for the accepted answer, but it does not translate my json aliases. Most of my projects have a helper function/class that I import.
Here is a function that solves my specific problem.
// Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj) // Convert to a json string
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
And in the main, this is how it would be called...
package main
import (
"fmt"
"encoding/json"
"github.com/fatih/structs"
)
type MyStructObject struct {
Email string `json:"email_address"`
}
func main() {
obj := &MyStructObject{Email: "test#test.com"}
// My solution
fmt.Println(StructToMap(obj)) // prints {"email_address": "test#test.com"}
// The currently accepted solution
fmt.Println(structs.Map(obj)) // prints {"Email": "test#test.com"}
}
package main
import (
"fmt"
"reflect"
)
type bill struct {
N1 int
N2 string
n3 string
}
func main() {
a := bill{4, "dhfthf", "fdgdf"}
v := reflect.ValueOf(a)
values := make(map[string]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
if v.Field(i).CanInterface() {
values[v.Type().Field(i).Name] = v.Field(i).Interface()
} else {
fmt.Printf("sorry you have a unexported field (lower case) value you are trying to sneak past. I will not allow it: %v\n", v.Type().Field(i).Name)
}
}
fmt.Println(values)
passObject(&values)
}
func passObject(v1 *map[string]interface{}) {
fmt.Println("yoyo")
}
I'm a bit late but I needed this kind of feature so I wrote this. Can resolve nested structs. By default, uses field names but can also use custom tags. A side effect is that if you set the tagTitle const to json, you could use the json tags you already have.
package main
import (
"fmt"
"reflect"
)
func StructToMap(val interface{}) map[string]interface{} {
//The name of the tag you will use for fields of struct
const tagTitle = "kelvin"
var data map[string]interface{} = make(map[string]interface{})
varType := reflect.TypeOf(val)
if varType.Kind() != reflect.Struct {
// Provided value is not an interface, do what you will with that here
fmt.Println("Not a struct")
return nil
}
value := reflect.ValueOf(val)
for i := 0; i < varType.NumField(); i++ {
if !value.Field(i).CanInterface() {
//Skip unexported fields
continue
}
tag, ok := varType.Field(i).Tag.Lookup(tagTitle)
var fieldName string
if ok && len(tag) > 0 {
fieldName = tag
} else {
fieldName = varType.Field(i).Name
}
if varType.Field(i).Type.Kind() != reflect.Struct {
data[fieldName] = value.Field(i).Interface()
} else {
data[fieldName] = StructToMap(value.Field(i).Interface())
}
}
return data
}
map := Structpb.AsMap()
// map is the map[string]interface{}

Q: golang pointer to map[string]interface{}

I (golang newbie) am trying to create a map[string]interfaces{} in a function. The code compiles and runs but the map is empty.
package main
import (
"fmt"
"encoding/json"
)
func main() {
var f interface{}
var sJson string // JSON string from VT
var err error // errors
var b []byte // bytearray of JSON string
var rootMap map[string]interface{}
rootMap = make(map[string]interface{})
sJson=`{"key": "foo"}`
fmt.Println(sJson)
err = json2map(&b, &sJson, f, rootMap)
if err != nil { return }
switch v := rootMap["key"].(type) {
case float64:
fmt.Printf("Value: %d",v)
case string:
fmt.Printf("Value: %s", v)
case nil:
fmt.Println("key is nil")
default:
fmt.Println("type is unknown")
}
}
func json2map(b *[]byte, sJson *string, f interface{}, myMap map[string]interface{}) error {
var err error
*b = []byte(*sJson)
err = json.Unmarshal(*b,&f)
myMap = f.(map[string]interface{})
return err
}
The output is:
{"key": "foo"}
key is nil
I found this article which describes how to use map[string]string. This code works as expected:
package main
import (
"fmt"
)
type MyType struct {
Value1 int
Value2 string
}
func main() {
myMap := make(map[string]string)
myMap["key"] = "foo"
ChangeMyMap(myMap)
fmt.Printf("Value : %s\n", myMap["key"])
}
func ChangeMyMap(TheMap map[string]string) {
TheMap["key"] = "bar"
}
So I think my problem has to do with the map being of type interface instead of string but I cannot tell why the first code doesn't work, while the 2nd does.
There's a number of things causing confusion here:
You don't need b at all. You're passing a pointer to a byte slice that you're reassigning inside the function. Just use the string directly.
There's no need to us a pointer to the sJson string. Strings are immutable, and you're not trying to reassign the sJson variable.
You're unmarshaling into an empty interface, and then trying to reassign the myMap variable to the contents of f. Since myMap isn't a pointer, that assignment is only scoped to within the function. Just unmarshal directly into myMap
If you change those things, you'll find the json2map function ends up being one line, and can be dropped altogether:
func json2map(sJson string, myMap map[string]interface{}) error {
return json.Unmarshal([]byte(sJson), &myMap)
}

Partly JSON unmarshal into a map in Go

My websocket server will receive and unmarshal JSON data. This data will always be wrapped in an object with key/value pairs. The key-string will act as value identifier, telling the Go server what kind of value it is. By knowing what type of value, I can then proceed to JSON unmarshal the value into the correct type of struct.
Each json-object might contain multiple key/value pairs.
Example JSON:
{
"sendMsg":{"user":"ANisus","msg":"Trying to send a message"},
"say":"Hello"
}
Is there any easy way using the "encoding/json" package to do this?
package main
import (
"encoding/json"
"fmt"
)
// the struct for the value of a "sendMsg"-command
type sendMsg struct {
user string
msg string
}
// The type for the value of a "say"-command
type say string
func main(){
data := []byte(`{"sendMsg":{"user":"ANisus","msg":"Trying to send a message"},"say":"Hello"}`)
// This won't work because json.MapObject([]byte) doesn't exist
objmap, err := json.MapObject(data)
// This is what I wish the objmap to contain
//var objmap = map[string][]byte {
// "sendMsg": []byte(`{"user":"ANisus","msg":"Trying to send a message"}`),
// "say": []byte(`"hello"`),
//}
fmt.Printf("%v", objmap)
}
Thanks for any kind of suggestion/help!
This can be accomplished by Unmarshaling into a map[string]json.RawMessage.
var objmap map[string]json.RawMessage
err := json.Unmarshal(data, &objmap)
To further parse sendMsg, you could then do something like:
var s sendMsg
err = json.Unmarshal(objmap["sendMsg"], &s)
For say, you can do the same thing and unmarshal into a string:
var str string
err = json.Unmarshal(objmap["say"], &str)
EDIT: Keep in mind you will also need to export the variables in your sendMsg struct to unmarshal correctly. So your struct definition would be:
type sendMsg struct {
User string
Msg string
}
Example: https://play.golang.org/p/OrIjvqIsi4-
Here is an elegant way to do similar thing. But why do partly JSON unmarshal? That doesn't make sense.
Create your structs for the Chat.
Decode json to the Struct.
Now you can access everything in Struct/Object easily.
Look below at the working code. Copy and paste it.
import (
"bytes"
"encoding/json" // Encoding and Decoding Package
"fmt"
)
var messeging = `{
"say":"Hello",
"sendMsg":{
"user":"ANisus",
"msg":"Trying to send a message"
}
}`
type SendMsg struct {
User string `json:"user"`
Msg string `json:"msg"`
}
type Chat struct {
Say string `json:"say"`
SendMsg *SendMsg `json:"sendMsg"`
}
func main() {
/** Clean way to solve Json Decoding in Go */
/** Excellent solution */
var chat Chat
r := bytes.NewReader([]byte(messeging))
chatErr := json.NewDecoder(r).Decode(&chat)
errHandler(chatErr)
fmt.Println(chat.Say)
fmt.Println(chat.SendMsg.User)
fmt.Println(chat.SendMsg.Msg)
}
func errHandler(err error) {
if err != nil {
fmt.Println(err)
return
}
}
Go playground
Further to Stephen Weinberg's answer, I have since implemented a handy tool called iojson, which helps to populate data to an existing object easily as well as encoding the existing object to a JSON string. A iojson middleware is also provided to work with other middlewares. More examples can be found at https://github.com/junhsieh/iojson
Example:
func main() {
jsonStr := `{"Status":true,"ErrArr":[],"ObjArr":[{"Name":"My luxury car","ItemArr":[{"Name":"Bag"},{"Name":"Pen"}]}],"ObjMap":{}}`
car := NewCar()
i := iojson.NewIOJSON()
if err := i.Decode(strings.NewReader(jsonStr)); err != nil {
fmt.Printf("err: %s\n", err.Error())
}
// populating data to a live car object.
if v, err := i.GetObjFromArr(0, car); err != nil {
fmt.Printf("err: %s\n", err.Error())
} else {
fmt.Printf("car (original): %s\n", car.GetName())
fmt.Printf("car (returned): %s\n", v.(*Car).GetName())
for k, item := range car.ItemArr {
fmt.Printf("ItemArr[%d] of car (original): %s\n", k, item.GetName())
}
for k, item := range v.(*Car).ItemArr {
fmt.Printf("ItemArr[%d] of car (returned): %s\n", k, item.GetName())
}
}
}
Sample output:
car (original): My luxury car
car (returned): My luxury car
ItemArr[0] of car (original): Bag
ItemArr[1] of car (original): Pen
ItemArr[0] of car (returned): Bag
ItemArr[1] of car (returned): Pen

Converting Go struct to JSON

I am trying to convert a Go struct to JSON using the json package but all I get is {}. I am certain it is something totally obvious but I don't see it.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Then when I try to run it I get this:
$ 6g test.go && 6l -o test test.6 && ./test
{}
You need to export the User.name field so that the json package can see it. Rename the name field to Name.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string
}
func main() {
user := &User{Name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Output:
{"Name":"Frank"}
Related issue:
I was having trouble converting struct to JSON, sending it as response from Golang, then, later catch the same in JavaScript via Ajax.
Wasted a lot of time, so posting solution here.
In Go:
// web server
type Foo struct {
Number int `json:"number"`
Title string `json:"title"`
}
foo_marshalled, err := json.Marshal(Foo{Number: 1, Title: "test"})
fmt.Fprint(w, string(foo_marshalled)) // write response to ResponseWriter (w)
In JavaScript:
// web call & receive in "data", thru Ajax/ other
var Foo = JSON.parse(data);
console.log("number: " + Foo.number);
console.log("title: " + Foo.title);
This is an interesting question, it is very easy using the new go versions. You should do this:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string `json:"name"`
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Change this name to Name.
You can define your own custom MarshalJSON and UnmarshalJSON methods and intentionally control what should be included, ex:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func (u *User) MarshalJSON() ([]byte, error) {
return json.Marshal(&struct {
Name string `json:"name"`
}{
Name: "customized" + u.name,
})
}
func main() {
user := &User{name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Struct values encode as JSON objects. Each exported struct field becomes
a member of the object unless:
the field's tag is "-", or
the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero. The object's default key string is the struct field name but can be specified in the struct field's tag value. The "json" key in the struct field's tag value is the key name, followed by an optional comma and options.