Marshal interface{} into json - json

I want to generate a json that's something like this:
{
"A": 1,
"B": "bVal",
"C": "cVal"
}
But I want to keep my code generic enough that I can replace the key-pair "C" with any other type of json object that I want. I tried doing something like what I have below:
type JsonMessage struct {
A int `json:"A"`
B string `json:"B,omitempty"`
genericObj interface{}
}
type JsonObj struct {
C string `json:"C"`
}
func main() {
myJsonObj := JsonObj{C: "cVal"}
myJson := JsonMessage{A: 1, B: "bValue", genericObj: myJsonObj}
body, _ := json.Marshal(myJson)
fmt.Print(body)
}
But the output is just this:
{
"A": 1,
"B": "bVal"
}
Is there a different way to approach this?

This is precisely why json.RawMessage exists.
Here is an example straight from the Go docs:
package main
import (
"encoding/json"
"fmt"
"os"
)
func main() {
h := json.RawMessage(`{"precomputed": true}`)
c := struct {
Header *json.RawMessage `json:"header"`
Body string `json:"body"`
}{Header: &h, Body: "Hello Gophers!"}
b, err := json.MarshalIndent(&c, "", "\t")
if err != nil {
fmt.Println("error:", err)
}
os.Stdout.Write(b)
}
Here is the output:
{
"header": {
"precomputed": true
},
"body": "Hello Gophers!"
}
Go Playground: https://play.golang.org/p/skYJT1WyM1C
Of course you can marshal a value before time to get the raw bytes in your case.

package main
import (
"encoding/json"
"fmt"
)
type JsonMessage struct {
A int `json:"A"`
B string `json:"B,omitempty"`
// simply make this field capitalized,and it will be included in the
// json string
GenericObj interface{}
}
type JsonObj struct {
C string `json:"C"`
}
func main() {
myJsonObj := JsonObj{C: "cVal"}
myJson := JsonMessage{A: 1, B: "bValue", GenericObj: myJsonObj}
body, _ := json.Marshal(myJson)
fmt.Println(string(body))
}
{"A":1,"B":"bValue","GenericObj":{"C":"cVal"}}

Related

identify the type of an object dynamically while reading json data

Quite new to go, sorry if this question sounds obvious.
I would like to use reflection in order to identify the type of an object while reading a json file.
The use case (please see the code below) is the following: I have two structs BoyGift and GirlGift that contain different fields. I have also a Boolean indicator IsBoy that is true if the recipient of the giftis a boy, false otherwise.
The type that encapsulates this behavior is the type Gift:
//Gift type
type Gift struct {
IsBoy bool `json:"isBoy"`
Payload ??? `json:"payload"`
}
that holds the data. How can I define that type in order the json unmarshal to convert dynamically to the correct type? The "json schema" in this case defines that a Gift should be either a BoyGift or a GirlGift. Is it possible to do this via reflection? How?
Doing unmarshal twice would be great, if the Boolean info is known
package main
import (
"encoding/json"
"fmt"
)
//BoyGift type
type BoyGift struct {
Cars uint32 `json:"cars"`
Balls uint32 `json:"balls"`
}
//GirlGift type
type GirlGift struct {
Dolls uint32 `json:"dolls"`
Lego uint32 `json:"lego"`
}
//Gift type
type Gift struct {
IsBoy bool `json:"isBoy"`
Payload GirlGift `json:"payload"`
}
func main() {
b := []byte(`{
"isBoy": true,
"payload": {
"cars": 1,
"balls": 2
}
}`)
var g Gift
err := json.Unmarshal(b, &g)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(g)
}
You should use json.RawMessage to dynamically unmarshal your data.
You can define Gift's Payliad as json.RawMessage and then delay unmarshaling until you know the value of IsBoy. Below you can find a basic example of how to do it.
package main
import (
"encoding/json"
"fmt"
)
//BoyGift type
type BoyGift struct {
Cars uint32 `json:"cars"`
Balls uint32 `json:"balls"`
}
//GirlGift type
type GirlGift struct {
Dolls uint32 `json:"dolls"`
Lego uint32 `json:"lego"`
}
//Gift type
type Gift struct {
IsBoy bool `json:"isBoy"`
Payload json.RawMessage `json:"payload"`
}
func main() {
b1 := []byte(`{
"isBoy": true,
"payload": {
"cars": 1,
"balls": 2
}
}`)
b2 := []byte(`{
"isBoy": false,
"payload": {
"dolls": 3,
"lego": 4
}
}`)
for _, b := range [][]byte{b1, b2} {
var g Gift
err := json.Unmarshal(b, &g)
if err != nil {
fmt.Println(err)
return
}
if g.IsBoy {
var boyGift BoyGift
err := json.Unmarshal(g.Payload, &boyGift)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(boyGift)
} else {
var girlGift GirlGift
err := json.Unmarshal(g.Payload, &girlGift)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(girlGift)
}
}
}
If you want to use Payload as an interface{} (which can be BoyGift or GirlGift), you could create additional helper struct for unmarshalling. Check extended example in Go Playground: https://play.golang.org/p/q1Hn45bgjsc

Converting map to string in Go

I am trying to find the best way to convert map[string]string to type string.
I tried converting to JSON with marshalling to keep the format and then converting back to a string, but this was not successful.
More specifically, I am trying to convert a map containing keys and values to a string to accommodate Environment Variables and structs.go.
For example, the final string should be like
LOG_LEVEL="x"
API_KEY="y"
The map
m := map[string]string{
"LOG_LEVEL": "x",
"API_KEY": "y",
}
You need some key=value pair on each line representing one map entry, and you need quotes around the values:
package main
import (
"bytes"
"fmt"
)
func createKeyValuePairs(m map[string]string) string {
b := new(bytes.Buffer)
for key, value := range m {
fmt.Fprintf(b, "%s=\"%s\"\n", key, value)
}
return b.String()
}
func main() {
m := map[string]string{
"LOG_LEVEL": "DEBUG",
"API_KEY": "12345678-1234-1234-1234-1234-123456789abc",
}
println(createKeyValuePairs(m))
}
Here is a working example on Go Playground.
You can use fmt.Sprint to convert the map to string:
import (
"fmt"
)
func main() {
m := map[string]string{
"a": "b",
"c": "d",
}
log.Println("Map: " + fmt.Sprint(m))
}
Or fmt.Sprintf:
import (
"fmt"
)
func main() {
m := map[string]string{
"a": "b",
"c": "d",
}
log.Println(fmt.Sprintf("Map: %v", m))
}
I would do this very simple and pragmatic:
package main
import (
"fmt"
)
func main() {
m := map[string]string {
"LOG_LEVEL": "x",
"API_KEY": "y",
}
var s string
for key, val := range m {
// Convert each key/value pair in m to a string
s = fmt.Sprintf("%s=\"%s\"", key, val)
// Do whatever you want to do with the string;
// in this example I just print out each of them.
fmt.Println(s)
}
}
You can see this in action in The Go Playground.
This could work:
// Marshal the map into a JSON string.
mJson, err := json.Marshal(m)
if err != nil {
fmt.Println(err.Error())
return
}
jsonStr := string(mJson)
fmt.Println("The JSON data is: ")
fmt.Println(jsonStr)
We could convert map to single line using sf.MapToStr function from github.com/wissance/stringFormatter v1.0.1 (https://github.com/Wissance/stringFormatter):
// ...
import (
sf "github.com/wissance/stringFormatter"
)
// ...
func MyFunc() {
options := map[string]interface{}{
"connectTimeout": 1000,
"useSsl": true,
"login": "sa",
"password": "sa",
}
str := sf.MapToString(&options, sf.KeyValueWithSemicolonSepFormat, ", ")
fmt.Println(str)
}
jsonString, err := json.Marshal(datas)
fmt.Println(err)

JSON: Nesting a populated struct into a new struct

I have a struct like so:
type my_struct struct {
First string `json:"first"`
Second string `json:"second"`
Number int `json:"number"`
}
When I marshal that into JSON, it outputs very simple JSON as you'd expect:
var output_json []byte
output_json, _ = json.Marshal(output)
fmt.Println(string(output_json))
Result:
{"first":"my_string","second":"another_string","number":2}
All fine so far!
What I'd like to do, before marshalling that struct into JSON, is nest it inside another struct. The resulting output would be JSON that looks like this:
{
"fields": {
"first": "my_string",
"number": 2,
"second": "another_string"
},
"meta": "data"
}
How can I do that?
I think you could statically declare another struct to use:
type WrappedOutput struct {
fields my_struct
meta string
}
Then you could embed your struct before marshalling
o := WrappedOutput{fields: output}
Brand new to go so not sure if this is the easiest way to do it
The clean way to do this would be to declare 2 structs (I've done it globally below) and nest My_struct inside the higher level struct.
Then you can initialize the higher level struct before Mashalling it:
package main
import (
"encoding/json"
"fmt"
)
type My_struct struct {
First string `json:"first"`
Second string `json:"second"`
Number int `json:"number"`
}
type Big_struct struct {
Fields My_struct
Meta string
}
func main() {
output := Big_struct{
Fields: My_struct{
First: "my_string",
Second: "another_string",
Number: 2,
},
Meta: "data",
}
output_json, err := json.Marshal(output)
if err != nil {
fmt.Println(err)
}
fmt.Println(string(output_json))
}
if you don't want Big_struct you can declare an anonymous struct within your code as you need it and nest My_struct inside:
package main
import (
"encoding/json"
"fmt"
)
type My_struct struct {
First string `json:"first"`
Second string `json: "second"`
Number int `json:"number"`
}
func main() {
output := struct {
Fields My_struct
Meta string
}{
Fields: My_struct{
First: "my_string",
Second: "another_string",
Number: 2,
},
Meta: "data",
}
output_json, err := json.Marshal(output)
if err != nil {
fmt.Println(err)
}
fmt.Println(string(output_json))
}
If you don't want to use a new structure, you can do:
data := my_struct{First: "first", Second: "2", Number: 123}
result, _ := json.Marshal(&map[string]interface{}{"fields":data,"meta":"data"})
fmt.Println(string(result))
it's not clean, but it does the work.

Golang modify json without struct

type Struct struct {
Value string `json:"value"`
Value1 string `json:"value_one"`
Nest Nested `json:"nest"`
}
type Nested struct {
Something string `json:"something"`
}
I want to add elements which are not in the structs definitions without creating another struct type. For example
Struct.Extra1 = Nested{"yy"}
Struct.Nested.Extra2 = "zz"
Which will result
{
"Value": "xx",
"Value1": "xx",
"Extra1": {
"Something", "yy"
},
"Nest": {
"Something": "xx",
"Extra2": "zz"
}
}
SOLUTION1:
I thought of adding omitempty to achieve this but it makes the structs complex.
type Struct struct {
Value string
Value1 string
Nest Nested
Extra1 Nested `json:"omitempty"`
}
type Nested struct {
Something string
Extra2 string `json:"omitempty"`
}
SOLUTION2:
myextras := make(map[string]interface{})
// get Struct.Nested in map[string]interface{} format
myextras = Struct.Nest
myextras["Extra2"] = "zz"
// get Struct in map[string]interface{} format
struct["Nest"] = myextras
struct["Extra1"] = Nested{"yy"}
// solves the problem with lots of type casting but doesn't support json tag naming
Is there a better solution to add nested elements which are not represented in struct datatype with json-tag support and could be used to output to user.
If someone is not happy with the solution provided:
Try tidwall/sjson. It provides functions for quick JSON editing without having to define any structure. It saved me a bunch of time yesterday :D
Example usage:
value, _ := sjson.Set(`{"name":{"last":"Anderson"}}`, "name.last", "Smith")
println(value)
// Output:
// {"name":{"last":"Smith"}}
Based on this answer: Can I use MarshalJSON to add arbitrary fields to a json encoding in golang?
You could do something like (demo: http://play.golang.org/p/dDiTwxhoNn):
package main
import (
"encoding/json"
"fmt"
"log"
)
type Book struct {
Title string
Author string
// extra is used for additional dynamic element marshalling
extra func() interface{}
}
type FakeBook Book
func (b *Book) SetExtra(fn func() interface{}) {
b.extra = fn
}
func (b *Book) MarshalJSON() ([]byte, error) {
if b.extra == nil {
b.extra = func() interface{} { return *b }
}
return json.Marshal(b.extra())
}
func main() {
ms := &Book{
Title: "Catch-22",
Author: "Joseph Heller",
}
ms.SetExtra(func() interface{} {
return struct {
FakeBook
Extra1 struct {
Something string `json:"something"`
} `json:"extra1"`
}{
FakeBook: FakeBook(*ms),
Extra1: struct {
Something string `json:"something"`
}{
Something: "yy",
},
}
})
out, err := json.MarshalIndent(ms, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
mb := &Book{
Title: "Vim-go",
Author: "Fatih Arslan",
}
mb.SetExtra(func() interface{} {
return struct {
FakeBook
Something string `json:"something"`
}{
FakeBook: FakeBook(*mb),
Something: "xx",
}
})
out, err = json.MarshalIndent(mb, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
mc := &Book{
Title: "Another-Title",
Author: "Fatih Arslan",
}
out, err = json.MarshalIndent(mc, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
}
yes. there is a type json.Raw which not a struct but []byte. you can manage it out of struct, in any marshal/unmarshal way.
UPDATE
use any way to edit json string on the fly
package main
import (
"fmt"
"encoding/json"
"strings"
)
type Struct struct {
Value string `json:"value"`
Value1 string `json:"value_one"`
Nest json.RawMessage`json:"nest"`
}
func main() {
s := Struct{Value1: "struct string"}
buf, _ := json.Marshal(s)
fmt.Println(string(buf))
s2 := strings.ReplaceAll(string(buf), "null", `{"extra2":{"anykey":3.1415926535}}`)
fmt.Println(s2)
}
https://play.golang.org/p/lOCxJBs5iRJ
The map approach is the only sane way to do it, everything else (like json.RawMessage fields would require an extra marshalling step anyway.
if you don't want install extra package, you can change or add new values with this code :
package main
import "fmt"
import "strings"
import "encoding/json"
func main() {
s := []byte(`{ "level1a":"aaa", "level1b":{"level2a":"bbb", "level2b":{"level3":"aaa"} } }`)
var j interface{}
json.Unmarshal(s, &j)
SetValueInJSON(j, "level1a", "new value 1a")
SetValueInJSON(j, "level1b.level2a", "new value 2a")
SetValueInJSON(j, "level1b.level2b.level3", "new value 3")
SetValueInJSON(j, "level1b.level2c", "new key")
s,_ = json.Marshal(j)
fmt.Println(string(s))
// result: {"level1a":"new value 1a","level1b":{"level2a":"new value 2a","level2b":{"level3":"new value 3"},"level2c":"new key"}}
}
func SetValueInJSON(iface interface{}, path string, value interface{}) interface{} {
m := iface.(map[string]interface{})
split := strings.Split(path, ".")
for k, v := range m {
if strings.EqualFold(k, split[0]) {
if len(split) == 1 {
m[k] = value
return m
}
switch v.(type) {
case map[string]interface{}:
return SetValueInJSON(v, strings.Join(split[1:], "."), value)
default:
return m
}
}
}
// path not found -> create
if len(split) == 1 {
m[split[0]] = value
} else {
newMap := make(map[string]interface{})
newMap[split[len(split)-1]] = value
for i := len(split) - 2; i > 0; i-- {
mTmp := make(map[string]interface{})
mTmp[split[i]] = newMap
newMap = mTmp
}
m[split[0]] = newMap
}
return m
}

Is there an easier way to add a layer over a JSON object using Golang JSON Encoding?

The out of the box JSON encoding in Go is really nice, but I need to get the output to match a certain format by adding a layer. I've figured out a way, but was hoping that there would be an easier way than the way I'm doing it.
Below is an example of how I'm doing it.
import (
"bytes"
"encoding/json"
"encoding/xml"
"fmt"
)
type Query struct {
XMLName xml.Name `xml:"http://marklogic.com/appservices/search query" json:"-"`
Format int `xml:"-" json:"-"`
Queries []interface{} `xml:",any" json:"queries"`
}
type TermQuery struct {
XMLName xml.Name `xml:"http://marklogic.com/appservices/search term-query" json:"-"`
Terms []string `xml:"http://marklogic.com/appservices/search text" json:"text"`
Weight float64 `xml:"http://marklogic.com/appservices/search weight,omitempty" json:"weight,omitempty"`
}
// use fakeQuery to avoid an infinite loop
type fakeQuery Query
//MarshalJSON for Query struct in a special way to add wraping {"query":...}
func (q Query) MarshalJSON() ([]byte, error) {
return wrapJSON(`query`, fakeQuery(q))
}
// use fakeTermQuery to avoid an infinite loop
type fakeTermQuery TermQuery
//MarshalJSON for TermQuery struct in a special way to add wraping {"term-query":...}
func (q TermQuery) MarshalJSON() ([]byte, error) {
return wrapJSON(`term-query`, fakeTermQuery(q))
}
func wrapJSON(name string, item interface{}) ([]byte, error) {
var buffer bytes.Buffer
b, err := json.Marshal(item)
buffer.Write([]byte(`{"`))
buffer.Write([]byte(name))
buffer.Write([]byte(`":`))
buffer.Write(b)
buffer.Write([]byte(`}`))
return buffer.Bytes(), err
}
I have a lot of defined structures that I would need to do this to, so I'm hoping for a better solution that won't leave me with with 100+ lines of code to just add a wrapper around the JSON object. Ideally I would like something that could peak at the XML element name defined for the XML encoder and use that to wrap the JSON.
In my case I'm using the MarshalJSON functions because these structures can be nested. If it helps I always know that the Query structure is the root structure.
When I started to use Go & Json I had the same problem. I resolved it by that
func wrapJSON(name string, item interface{}) ([]byte, error) {
wrapped := map[string]interface{}{
name: item,
}
converted, err := json.Marshal(wrapped)
return converted
}
Ideally, rename your method wrapJSON to wrap that return an interface and after convert this interface to JSON or XML
Perhaps I am missing something, but is this what you are looking for?
I started off with the same idea as #Manawasp (using a map[string]interface{}) but decided to try to get the name from the struct tag like you asked about... here's what I came up with (*note: there may be unhandled error cases, and this may overcomplicate something that can be handled pretty easily with the other solution)
http://play.golang.org/p/qO6tDZjtXA
package main
import (
"fmt"
"reflect"
"strings"
)
import (
"encoding/json"
"encoding/xml"
"errors"
)
type Query struct {
XMLName xml.Name `xml:"http://marklogic.com/appservices/search query" json:"-"`
Field1 string
Field2 int64
}
type TermQuery struct {
XMLName xml.Name `xml:"http://marklogic.com/appservices/search term-query" json:"-"`
Field3 string
Field4 int64
}
func getXmlName(d interface{}, label string) (string, bool) {
switch reflect.TypeOf(d).Kind() {
case reflect.Struct:
v, _ := reflect.TypeOf(d).FieldByName(label)
parts := strings.Split(v.Tag.Get("xml"), " ")
return parts[1], true
}
return "", false
}
func wrapJson(item interface{}) ([]byte, error) {
if n, ok := getXmlName(item, "XMLName"); ok {
b, err := json.Marshal(map[string]interface{}{n: item})
if err != nil {
return nil, err
}
return b, nil
}
return nil, errors.New("You failed")
}
func main() {
// create a Query and encode it as {"query": {struct}}
q := Query{Field1: "hello", Field2: 42}
wrappedQ, err := wrapJson(q)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(wrappedQ))
// create a TermQuery and encode it as {"term-query": {struct}}
tq := TermQuery{Field3: "world", Field4: 99}
wrappedTQ, err := wrapJson(tq)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(wrappedTQ))
}
OUTPUT
{"query":{"Field1":"hello","Field2":42}}
{"term-query":{"Field3":"world","Field4":99}}
EDIT
Ok, here is an update now that I can see what your issue is. It might be ugly, and it might not be bullet-proof (error handling, etc)... but for my test it seems to do what you want.
http://play.golang.org/p/8MloLP3X4H
package main
import (
"fmt"
"reflect"
"strings"
)
import (
//"encoding/json"
"encoding/json"
"encoding/xml"
"errors"
)
type Query struct {
XMLName xml.Name `xml:"http://marklogic.com/appservices/search query" json:"-"`
Field1 string
Field2 int64
Queries []interface{} `xml:",any" json:"queries"`
}
type TermQuery struct {
XMLName xml.Name `xml:"http://marklogic.com/appservices/search term-query" json:"-"`
Field3 string
Field4 int64
}
func getXmlName(d interface{}, label string) (string, bool) {
switch reflect.TypeOf(d).Kind() {
case reflect.Struct:
v, _ := reflect.TypeOf(d).FieldByName(label)
parts := strings.Split(v.Tag.Get("xml"), " ")
return parts[1], true
default:
fmt.Println(reflect.TypeOf(d).Kind())
}
return "", false
}
func wrapJson(item interface{}) (map[string]interface{}, error) {
if n, ok := getXmlName(item, "XMLName"); ok {
if k := reflect.ValueOf(item).FieldByName("Queries"); k.IsValid() {
for i := 0; i < k.Len(); i++ {
b, err1 := wrapJson(k.Index(i).Interface())
if err1 != nil {
continue
}
k.Index(i).Set(reflect.ValueOf(b))
}
}
return map[string]interface{}{n: item}, nil
}
return nil, errors.New("You failed")
}
func asJson(i interface{}) []byte {
b, err := json.Marshal(i)
if err != nil {
return []byte(`{"error": "too bad"}`)
}
return b
}
func main() {
// create a TermQuery and encode it as {"term-query": {struct}}
tq := TermQuery{Field3: "world", Field4: 99}
wrappedTQ, err := wrapJson(tq)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(asJson(wrappedTQ)))
// create a Query and encode it as {"query": {struct}}
q := Query{
Field1: "hello",
Field2: 42,
Queries: []interface{}{
TermQuery{Field3: "world", Field4: 99},
TermQuery{Field3: "yay, it works!", Field4: 666},
Query{
Field1: "Hi",
Field2: 21,
Queries: []interface{}{
TermQuery{
Field3: "omg",
Field4: 1,
},
},
},
},
}
wrappedQ, err := wrapJson(q)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(asJson(wrappedQ)))
}
PRETTY-PRINTED OUTOUT
{
"query": {
"Field1": "hello",
"Field2": 42,
"queries": [
{
"term-query": {
"Field3": "world",
"Field4": 99
}
},
{
"term-query": {
"Field3": "yay, it works!",
"Field4": 666
}
},
{
"query": {
"Field1": "Hi",
"Field2": 21,
"queries": [
{
"term-query": {
"Field3": "omg",
"Field4": 1
}
}
]
}
}
]
}
}
Inside your JSON Marshaler you can define one more more (anonymous) structs with the various JSON tags and add the desired wrapping.
Since you have already defined the JSON Marshaler any tags defined in the original struct might be overriden by your JSON Marshaler implementation.
See Golang Playground.
type Book struct {
Title string
Author string
Language string
Publisher string
}
func (b Book) MarshalJSON() ([]byte, error) {
type BookDetailJSON struct {
Name string `json:"Title"`
Author string
Language string `json:",omitempty"`
Publisher string `json:"-"`
}
type BookJSON struct {
Book BookDetailJSON `json:"Novel"`
}
return json.Marshal(BookJSON{BookDetailJSON{
Name: b.Title,
Author: b.Author,
Language: b.Language,
Publisher: b.Publisher,
}})
}
type MultiMatch struct {
Query string `json:"query"`
Fields []string `json:"fields"`
}
func (m *MultiMatch) MarshalJSON() ([]byte, error) {
w := map[string]interface{}{}
w["multi_match"] = *m
return json.Marshal(w)
}