Partly JSON unmarshal into a map in Go - json

My websocket server will receive and unmarshal JSON data. This data will always be wrapped in an object with key/value pairs. The key-string will act as value identifier, telling the Go server what kind of value it is. By knowing what type of value, I can then proceed to JSON unmarshal the value into the correct type of struct.
Each json-object might contain multiple key/value pairs.
Example JSON:
{
"sendMsg":{"user":"ANisus","msg":"Trying to send a message"},
"say":"Hello"
}
Is there any easy way using the "encoding/json" package to do this?
package main
import (
"encoding/json"
"fmt"
)
// the struct for the value of a "sendMsg"-command
type sendMsg struct {
user string
msg string
}
// The type for the value of a "say"-command
type say string
func main(){
data := []byte(`{"sendMsg":{"user":"ANisus","msg":"Trying to send a message"},"say":"Hello"}`)
// This won't work because json.MapObject([]byte) doesn't exist
objmap, err := json.MapObject(data)
// This is what I wish the objmap to contain
//var objmap = map[string][]byte {
// "sendMsg": []byte(`{"user":"ANisus","msg":"Trying to send a message"}`),
// "say": []byte(`"hello"`),
//}
fmt.Printf("%v", objmap)
}
Thanks for any kind of suggestion/help!

This can be accomplished by Unmarshaling into a map[string]json.RawMessage.
var objmap map[string]json.RawMessage
err := json.Unmarshal(data, &objmap)
To further parse sendMsg, you could then do something like:
var s sendMsg
err = json.Unmarshal(objmap["sendMsg"], &s)
For say, you can do the same thing and unmarshal into a string:
var str string
err = json.Unmarshal(objmap["say"], &str)
EDIT: Keep in mind you will also need to export the variables in your sendMsg struct to unmarshal correctly. So your struct definition would be:
type sendMsg struct {
User string
Msg string
}
Example: https://play.golang.org/p/OrIjvqIsi4-

Here is an elegant way to do similar thing. But why do partly JSON unmarshal? That doesn't make sense.
Create your structs for the Chat.
Decode json to the Struct.
Now you can access everything in Struct/Object easily.
Look below at the working code. Copy and paste it.
import (
"bytes"
"encoding/json" // Encoding and Decoding Package
"fmt"
)
var messeging = `{
"say":"Hello",
"sendMsg":{
"user":"ANisus",
"msg":"Trying to send a message"
}
}`
type SendMsg struct {
User string `json:"user"`
Msg string `json:"msg"`
}
type Chat struct {
Say string `json:"say"`
SendMsg *SendMsg `json:"sendMsg"`
}
func main() {
/** Clean way to solve Json Decoding in Go */
/** Excellent solution */
var chat Chat
r := bytes.NewReader([]byte(messeging))
chatErr := json.NewDecoder(r).Decode(&chat)
errHandler(chatErr)
fmt.Println(chat.Say)
fmt.Println(chat.SendMsg.User)
fmt.Println(chat.SendMsg.Msg)
}
func errHandler(err error) {
if err != nil {
fmt.Println(err)
return
}
}
Go playground

Further to Stephen Weinberg's answer, I have since implemented a handy tool called iojson, which helps to populate data to an existing object easily as well as encoding the existing object to a JSON string. A iojson middleware is also provided to work with other middlewares. More examples can be found at https://github.com/junhsieh/iojson
Example:
func main() {
jsonStr := `{"Status":true,"ErrArr":[],"ObjArr":[{"Name":"My luxury car","ItemArr":[{"Name":"Bag"},{"Name":"Pen"}]}],"ObjMap":{}}`
car := NewCar()
i := iojson.NewIOJSON()
if err := i.Decode(strings.NewReader(jsonStr)); err != nil {
fmt.Printf("err: %s\n", err.Error())
}
// populating data to a live car object.
if v, err := i.GetObjFromArr(0, car); err != nil {
fmt.Printf("err: %s\n", err.Error())
} else {
fmt.Printf("car (original): %s\n", car.GetName())
fmt.Printf("car (returned): %s\n", v.(*Car).GetName())
for k, item := range car.ItemArr {
fmt.Printf("ItemArr[%d] of car (original): %s\n", k, item.GetName())
}
for k, item := range v.(*Car).ItemArr {
fmt.Printf("ItemArr[%d] of car (returned): %s\n", k, item.GetName())
}
}
}
Sample output:
car (original): My luxury car
car (returned): My luxury car
ItemArr[0] of car (original): Bag
ItemArr[1] of car (original): Pen
ItemArr[0] of car (returned): Bag
ItemArr[1] of car (returned): Pen

Related

Parse Error from Converting Json String to Struct

i cannot parse the json value i am sending a playground link
Any idea about that? here is the link and codes
https://play.golang.org/p/qhZpS_-618s
package main
import (
"encoding/json"
"fmt"
//mapstructure "github.com/mitchellh/mapstructure"
)
type presence struct{
id string
m_type string
deny string
}
type jsonHandler struct {
name string
dat map[string]interface{}
}
func main() {
s := `["Presence",{"id":"905356870666#c.us","type":"unavailable","deny":true}]`
data := jsonHandler{}
json.Unmarshal([]byte(s), &data)
fmt.Printf("Operation: %s", data.name)
}
Output :
Operation:
Program exited.
Try with this one: https://play.golang.com/p/UICf_uNNFdC
I've commented a lot in order to enhance code readability. Be sure to handle error properly and remove debug print.
package main
import (
"encoding/json"
"log"
"strings"
)
type Presence struct {
Presence string
ID string `json:"id"`
Type string `json:"type"`
Deny bool `json:"deny"`
}
type JsonHandler struct {
Name string `json:"name"`
Dat Presence `json:"dat"`
}
func main() {
var (
// Used for unmarshal a given json
packedData []json.RawMessage
err error
// Data that does not have a related json key
name []byte
// Used for extract the raw data that will be unmarshalled into the Presence struct
temp []byte
// Nested json
jsonPresence Presence
handler JsonHandler
)
s := `["Presence",{"id":"905356870666#c.us","type":"unavailable","deny":true}]`
log.Println("Dealing with -> " + s)
// Unmarshall into a raw json message
err = json.Unmarshal([]byte(s), &packedData)
if err != nil {
panic(err)
}
// Extract the presence
log.Println("Presence: ", string(packedData[0]))
// Extract the nested json
log.Println("Packed: ", string(packedData[1]))
// NOTE: 0 refers to the first value of the JSON
name, err = packedData[0].MarshalJSON()
if err != nil {
panic(err)
}
log.Println("Value that does not have a key: " + string(name))
handler.Name = strings.Replace(string(name), "\"", "", -1)
// NOTE: 1 refers to the second value of the JSON, the entire JSON
// Unmarshal the nested Json into byte
temp, err = packedData[1].MarshalJSON()
if err != nil {
panic(err)
}
// Unmarshal the raw byte into the struct
err = json.Unmarshal(temp, &jsonPresence)
if err != nil {
panic(err)
}
log.Println("ID:", jsonPresence.ID)
log.Println("Type:", jsonPresence.Type)
log.Println("Deny:", jsonPresence.Deny)
handler.Dat = jsonPresence
log.Println("Data unmarshalled: ", handler)
}
Go Playground Link: https://play.golang.org/p/qe0jyFVNTH1
Few Problem are present in this:
1. Json Package can't refer the Unexported Structure Elements.So please use Deny instead of deny in the following snippet.This is applicable to all variables declared inside the structure
2. The json fields tag are incorrect. eg.mapstructure:"id" should be json:"id"
3. The json to be parsed contains two distinct elements i.e string "Presence" and nested json object.It can't be parsed as a single element.It is better to declare "Presence" as a key and nested json as the value.
4. The deny variable should be bool rather than string
Wow,solved problem by adding only these codes
Here Go Lang Link : https://play.golang.org/p/doHNWK58Cae
func (n *JsonHandler) UnmarshalJSON(buf []byte) error {
tmp := []interface{}{&n.Name, &n.Dat}
wantLen := len(tmp)
if err := json.Unmarshal(buf, &tmp); err != nil {
return err
}
if g, e := len(tmp), wantLen; g != e {
return fmt.Errorf("wrong number of fields in Notification: %d != %d", g, e)
}
return nil
}

Go: Converting JSON string to map[string]interface{}

I'm trying to create a JSON representation within Go using a map[string]interface{} type. I'm dealing with JSON strings and I'm having a hard time figuring out how to avoid the JSON unmarshaler to automatically deal with numbers as float64s. As a result the following error occurs.
Ex.
"{ 'a' : 9223372036854775807}" should be map[string]interface{} = [a 9223372036854775807 but in reality it is map[string]interface{} = [a 9.2233720368547758088E18]
I searched how structs can be used to avoid this by using json.Number but I'd really prefer using the map type designated above.
The go json.Unmarshal(...) function automatically uses float64 for JSON numbers. If you want to unmarshal numbers into a different type then you'll have to use a custom type with a custom unmarshaler. There is no way to force the unmarshaler to deserialize custom values into a generic map.
For example, here's how you could parse values of the "a" property as a big.Int.
package main
import (
"encoding/json"
"fmt"
"math/big"
)
type MyDoc struct {
A BigA `json:"a"`
}
type BigA struct{ *big.Int }
func (a BigA) UnmarshalJSON(bs []byte) error {
_, ok := a.SetString(string(bs), 10)
if !ok {
return fmt.Errorf("invalid integer %s", bs)
}
return nil
}
func main() {
jsonstr := `{"a":9223372036854775807}`
mydoc := MyDoc{A: BigA{new(big.Int)}}
err := json.Unmarshal([]byte(jsonstr), &mydoc)
if err != nil {
panic(err)
}
fmt.Printf("OK: mydoc=%#v\n", mydoc)
// OK: mydoc=main.MyDoc{A:9223372036854775807}
}
func jsonToMap(jsonStr string) map[string]interface{} {
result := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &result)
return result
}
Example - https://goplay.space/#ra7Gv8A5Heh
Related questions - create a JSON data as map[string]interface with the given data

Golang JSON decoder does not recognize type

Still a Golang beginner, I am trying to code a generic function to serve ReST requests. I pass a function to create a new resource (struct) with an interface implemented on it, because I would also invoke methods on the struct. When decoding the JSON, logging the type shows the correct (struct) type, but the JSON decoder seems to only recognize the interface, which it cannot decode to.
package main
import (
"encoding/json"
"github.com/julienschmidt/httprouter"
"log"
"net/http"
"strings"
)
// general resource interface
type resource interface {
// check semantics and return an array of errors or nil if no error found
check() []string
// update the resource in backend
update() error
}
// specific resource named "anchor"
type anchor struct {
ID string `json:"id"`
Name string `json:"name"`
}
func newAnchor() resource {
return anchor{}
}
func (a anchor) check() []string {
return nil
}
func (a anchor) update() error {
return nil
}
// generic function to create (POST) a new resource
func restCreate(newResource func() resource) httprouter.Handle {
return func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
const F = "restCreate"
var checkErrs []string
res := newResource()
log.Printf("%s res type %T\n", F, res)
dcdr := json.NewDecoder(r.Body)
err := dcdr.Decode(&res)
log.Printf("%s Unmarshalled into %T: %+v\n", F, res, res)
if err == nil {
checkErrs = res.check()
}
switch {
case err != nil:
w.WriteHeader(http.StatusInternalServerError)
log.Printf("[ERR] %s: %v\n", F, err)
case checkErrs != nil:
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte(strings.Join(checkErrs, "\n")))
log.Printf("%s: %v\n", F, err)
default:
res.update()
bs, _ := json.Marshal(res)
w.Write(bs)
}
}
}
func main() {
r := httprouter.New()
r.POST("/anchors", restCreate(newAnchor))
http.ListenAndServe(":8080", r)
}
The execution log shows:
restCreate res type main.anchor
restCreate Unmarshalled into main.anchor: {ID: Name:}
[ERR] restCreate: json: cannot unmarshal object into Go value of type main.resource
Why does Printf show the struct type and json.Decoder the interface?
I'd appreciate any indicator on what's going wrong and how to solve this in a generic way...
It is because you try to use a pointer to the interface to unmarshal into. You need return a pointer in a function
func newAnchor() resource {
return &anchor{}
}
And you don't need to get address in this line:
err := dcdr.Decode(&res)
Here is small working example: https://play.golang.org/p/3E0RmGTURO
You cannot Unmarshal into an interface unless variable is holding a pointer to the concrete type you desire, as json.Decode won't know which concrete type to use. You have two workarounds at your disposal:
Having newResource return a concrete type under the hood:
func newResource() resource {
return &anchor{}
}
This way json.Decode knows to unmarshal your JSON into an anchor.
Use newAnchor instead of newResource: this will be more readable in your restCreate function, and is more idiomatic[1].
[1] http://idiomaticgo.com/post/best-practice/accept-interfaces-return-structs/

How do I read one value from a JSON file using Go

I an new to programming in Go so apologies if this is something obvious. I have a JSON file named foo.json:
{"type":"fruit","name":"apple","color":"red"}
and I am writing a Go program that has to do something when the "name" value in the JSON file is "apple". It needs no other information from that JSON file as that file is used for a completely different purpose in another area of the code.
I have read documentation on Decode() and Unmarshal() and abut 30 different web pages describing how to read the whole file into structures, etc. but it all seems extremely complicated for what I want to do which is just write the correct code to implement the first 2 lines of this pseudo-code:
file, _ := os.Open("foo.json")
name = file["name"]
if (name == "apple") {
do stuff
}
such that I end up with a Go variable named name that contains the string value apple. What is the right way to do this in Go?
The easiest method to do what you want is to decode into a struct.
Provided the format remains similar to {"type":"fruit","name":"apple","color":"red"}
type Name struct {
Name string `json:"name"`
}
var data []byte
data, _ = ioutil.ReadFile("foo.json")
var str Name
_ = json.Unmarshal(data, &str)
if str.Name == "apple" {
// Do Stuff
}
Your other option is to use third party libraries such as gabs or jason.
Gabs :
jsonParsed, err := gabs.ParseJSON(data)
name, ok := jsonParsed.Path("name").Data().(string)
Jason :
v, _ := jason.NewObjectFromBytes(data)
name, _ := v.GetString("name")
Update :
The structure
type Name struct {
Name string `json:"name"`
}
is the json equivalent of {"name":"foo"}.
So unmarshaling won't work for the following json with different formats.
[{"name":"foo"}]
{"bar":{"name":"foo"}}
PS : As mentioned by W.K.S. In your case an anonymous struct would suffice since you're not using this structure for anything else.
One thing is reading a file and other one is decoding a JSON document. I leave you a full example doing both. To run it you have to have a file called file.json in the same directory of your code or binary executable:
package main
import (
"encoding/json"
"io/ioutil"
"log"
"os"
)
func main() {
f, err := os.Open("file.json") // file.json has the json content
if err != nil {
log.Fatal(err)
}
bb, err := ioutil.ReadAll(f)
if err != nil {
log.Fatal(err)
}
doc := make(map[string]interface{})
if err := json.Unmarshal(bb, &doc); err != nil {
log.Fatal(err)
}
if name, contains := doc["name"]; contains {
log.Printf("Happy end we have a name: %q\n", name)
} else {
log.Println("Json document doesn't have a name field.")
}
log.Printf("full json: %s", string(bb))
}
https://play.golang.org/p/0u04MwwGfn
I have also tried to find a simple solution such as $d = json_decode($json, true) in PHP and came to the conclusion that there is no such simple way in Golang. The following is the simplest solution I could make (the checks are skipped for clarity):
var f interface{}
err = json.Unmarshal(file, &f)
m := f.(map[string]interface{})
if (m["name"] == "apple") {
// Do something
}
where
file is an array of bytes of JSON string,
f interface serves as a generic container for unknown JSON structure,
m is a map returned by the type assertion.
We can assert that f is a map of strings, because Unmarshal() builds a variable of that type for any JSON input. At least, I couldn't make it return something different. It is possible to detect the type of a variable by means of run-time reflection:
fmt.Printf("Type of f = %s\n", reflect.TypeOf(f))
For the f variable above, the code will print Type of f = map[string]interface {}.
Example
And this is the full code with necessary checks:
package main
import (
"fmt"
"os"
"io/ioutil"
"encoding/json"
)
func main() {
// Read entire file into an array of bytes
file, err := ioutil.ReadFile("foo.json")
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed read file: %s\n", err)
os.Exit(1)
}
var f interface{}
err = json.Unmarshal(file, &f)
if (err != nil) {
fmt.Fprintf(os.Stderr, "Failed to parse JSON: %s\n", err)
os.Exit(1)
}
// Type-cast `f` to a map by means of type assertion.
m := f.(map[string]interface{})
fmt.Printf("Parsed data: %v\n", m)
// Now we can check if the parsed data contains 'name' key
if (m["name"] == "apple") {
fmt.Print("Apple found\n")
}
}
Output
Parsed data: map[type:fruit name:apple color:red]
Apple found
The proper Go way of doing this would be to decode into an instance of an anonymous struct containing only the field you need.
func main() {
myStruct := struct{ Name string }{}
json.Unmarshal([]byte(`{"type":"fruit","name":"apple","color":"red"}`), &myStruct)
fmt.Print(myStruct.Name)
}
Playground Link
Alternatively, You could use Jeffails/gabs JSON Parser:
jsonParsed,_ := gabs.ParseJSON([]byte(`{"type":"fruit","name":"apple","color":"red"}`));
value, ok = jsonParsed.Path("name").Data().(string)

Converting Go struct to JSON

I am trying to convert a Go struct to JSON using the json package but all I get is {}. I am certain it is something totally obvious but I don't see it.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Then when I try to run it I get this:
$ 6g test.go && 6l -o test test.6 && ./test
{}
You need to export the User.name field so that the json package can see it. Rename the name field to Name.
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string
}
func main() {
user := &User{Name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Output:
{"Name":"Frank"}
Related issue:
I was having trouble converting struct to JSON, sending it as response from Golang, then, later catch the same in JavaScript via Ajax.
Wasted a lot of time, so posting solution here.
In Go:
// web server
type Foo struct {
Number int `json:"number"`
Title string `json:"title"`
}
foo_marshalled, err := json.Marshal(Foo{Number: 1, Title: "test"})
fmt.Fprint(w, string(foo_marshalled)) // write response to ResponseWriter (w)
In JavaScript:
// web call & receive in "data", thru Ajax/ other
var Foo = JSON.parse(data);
console.log("number: " + Foo.number);
console.log("title: " + Foo.title);
This is an interesting question, it is very easy using the new go versions. You should do this:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Name string `json:"name"`
}
func main() {
user := &User{name:"Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Printf("Error: %s", err)
return;
}
fmt.Println(string(b))
}
Change this name to Name.
You can define your own custom MarshalJSON and UnmarshalJSON methods and intentionally control what should be included, ex:
package main
import (
"fmt"
"encoding/json"
)
type User struct {
name string
}
func (u *User) MarshalJSON() ([]byte, error) {
return json.Marshal(&struct {
Name string `json:"name"`
}{
Name: "customized" + u.name,
})
}
func main() {
user := &User{name: "Frank"}
b, err := json.Marshal(user)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(b))
}
Struct values encode as JSON objects. Each exported struct field becomes
a member of the object unless:
the field's tag is "-", or
the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero. The object's default key string is the struct field name but can be specified in the struct field's tag value. The "json" key in the struct field's tag value is the key name, followed by an optional comma and options.