I have a JSON object whose details can contain different types of JSON objects, the rest of the JSON remains the same, in such a case how can I have a single struct in Golang to handle both types of JSON
JSON 1:
{
"field1":"",
"field2":"",
"field3":"",
"field4":"",
"field5":"",
"field6":"",
"field7":"",
"details":{
"detail1":"",
"detail2":[
{
"arr1":"",
"arr2":{
"id":"",
"name":""
},
"list":[
{
"id":"",
"version":1,
"name":""
}
]
}
]
},
"user":{
"id":"",
"name":""
}
}
JSON 2:
{
"field1":"",
"field2":"",
"field3":"",
"field4":"",
"field5":"",
"field6":"",
"field7":"",
"details":{
"anotherdetail1":"",
"anotherdetail2":[
{
"arr7":"",
"arr8":{
"id":"",
"name":""
},
"arr10":{
}
}
]
},
"user":{
"id":"",
"name":""
}
}
My goal is to use a single struct for both these JSON objects. In a language like Java I would create a Parent Class which resembles details in a generic way and have 2 child classes to resemble the type of details that vary and during runtime I would create an object of a child type and assign it to the Parent. I am unsure how this is done in Go.
I am not sure you can have a single struct unless you are ok with a string interface map, but You can prevent the details from being decoded by setting them as a json.RawMessage type int he struct. You can then decode the unknown-typed json data by attempting to decode it into one type, if that returns an error then you try with the next type.
Here is some code, that should give you a better idea to what I am talking about.
https://play.golang.org/p/06owmiJXNaO
package main
import (
"encoding/json"
"fmt"
)
const json1 = `{"name": "foo", "details":[1, 2, 3]}`
const json2 = `{"name": "foo", "details":{"a": [1, 2, 3]}}`
type data struct {
Name string `json:"name"`
Details json.RawMessage `json:"details"`
}
type detailsone []int
type detailstwo struct {
A []int `json:"a"`
}
func main() {
var d1, d2 data
json.Unmarshal([]byte(json1), &d1)
json.Unmarshal([]byte(json2), &d2)
fmt.Printf("%+v\n", d1)
fmt.Printf("%+v\n", d2)
var err error
var b1 detailsone
var b2 detailstwo
// json1
err = json.Unmarshal([]byte(d1.Details), &b1)
if err == nil {
fmt.Printf("d1 is an []int: %+v\n", b1)
}
err = json.Unmarshal([]byte(d1.Details), &b2)
if err == nil {
fmt.Printf("d1 is an detailstwo struct: %+v\n", b2)
}
// json2
err = json.Unmarshal([]byte(d2.Details), &b1)
if err == nil {
fmt.Printf("d2 is an []int: %+v\n", b1)
}
err = json.Unmarshal([]byte(d2.Details), &b2)
if err == nil {
fmt.Printf("d2 is an detailstwo struct: %+v\n", b2)
}
}
type Base struct {
Data map[string]interface{}
Details struct {
*D1
*D2
} `json:"details"
}
type D1 struct {
Detail1 string
Detail2 string
}
type D2 struct {
AnotherDetail1 string
AnotherDetail2 string
}
you can find filled struct by comparing them with nil
Related
My apologies for the basic question. I am new to Golang and I have the json to parse as below
{
"config1":{
"Parameters":{
"Pm1":"value",
"Pm2":"value",
"Pm3":"value"
},
"dynamic_key1":{
"Parameters":{
"a":"value",
"b":"value",
"c":"value",
"d":"value"
},
"Epoch":"value"
},
"Epoch":"value"
}
}
I am trying to write a struct to parse this json and wrote the struct in the following way.
type Parameters struct {
Pm1 string `json:"Pm1"`
Pm2 string `json:"Pm2"`
Pm3 string `json:"Pm3"`
}
type dynamicParametes struct {
a string `json:"a"`
b string `json:"b"`
c string `json:"c"`
d string `json:"d"`
}
type dynamic struct {
Parameters dynamicParametes `json:"Parameters"`
Epoch string `json:"Epoch"`
}
type config1 struct {
Parameters Parameters `json:"Parameters"`
Dynamic_keys map[string]dynamic `json:"-"`
Epoch string `json:"Epoch"`
}
type config struct {
config1 config1 `json:"config1"`
}
I was hoping that the map will match all the matching keys with dynamic structs and show them in the map. But, I see it created an empty map in the response.
Implemented custom unmarshler for config type.
Note
If you don't need Parameters and dynamicParametes as struct types, you can simply unmarshal them into map[string]string
you have to expose all fields in your structs to do json unmarshaling
validate your json string
type config struct {
Config1 config1 `json:"config1"`
}
type _config config
func (b *config) UnmarshalJSON(data []byte) error {
var v = struct {
Config1 map[string]interface{} `json:"config1"`
}{}
if err := json.Unmarshal(data, &v); err != nil {
return err
}
c := _config{}
err := json.Unmarshal(data, &c)
if err != nil {
return err
}
b.Config1.Parameters = c.Config1.Parameters
b.Config1.Epoch = c.Config1.Epoch
if b.Config1.Dynamic_keys == nil {
b.Config1.Dynamic_keys = map[string]dynamic{}
}
for key, config := range v.Config1 {
if key == `Parameters` || key == `Epoch` {
continue
}
data, err := json.Marshal(config)
if err != nil {
return err
}
d := dynamic{}
err = json.Unmarshal(data, &d)
if err != nil {
return err
}
b.Config1.Dynamic_keys[key] = d
}
return nil
}
you can see full code here
All you need is understand how base data types looks in json.
Field Parameters in your json is simple map[string]string and you can unmarshall it with standart json.Unmasrhall without any aditional implementation of interface json.Unmarshaler.
Link for Go Playground with code below
package main
import (
"encoding/json"
"fmt"
)
const jsonStr = `{
"config1":{
"Parameters":{
"Pm1":"value_1",
"Pm2":"value_2",
"Pm3":"value_3"
},
"dynamic_key1":{
"Parameters":{
"a":"value_1",
"b":"value_2",
"c":"value_3",
"d":"value_4"
},
"Epoch":"value"
},
"Epoch":"value"
}
}`
type Data struct {
Config1 struct {
Parameters map[string]string `json:"Parameters"`
Dynamic struct {
Parameters map[string]string `json:"Parameters"`
Epoch string `json:"Epoch"`
} `json:"dynamic_key1"`
Epoch string `json:"Epoch"`
} `json:"config1"`
}
func main() {
var data Data
_ = json.Unmarshal([]byte(jsonStr), &data)
fmt.Printf("%+v\n", data)
}
Output:
{Config1:{Parameters:map[Pm1:value_1 Pm2:value_2 Pm3:value_3] Dynamic:{Parameters:map[a:value_1 b:value_2 c:value_3 d:value_4] Epoch:value} Epoch:value}}
I am writing a simple post api request. I am able to parse the JSON into golang structs upto the peername json object. I do not know the correct syntax to populate a golang slice of a struct by passing values through the JSON body of the api.
I am trying to parse JSON body sent through an api. This is the sample body request -
{
"type":"string",
"name":"string",
"organization":{
"orgID":"1",
"orgName":"string",
"peer":{
"peerID":"1",
"peerName":"string"
},
"attributes":[
["slide0001.html", "Looking Ahead"],
["slide0008.html", "Forecast"],
["slide0021.html", "Summary"]
]
}
} "peerName":"string"
},
"attributes":["name":"string":"value":true]
}
}
And this is my sample golang structs.
//Identity ...
type Identity struct {
Type string `json:"type,omitempty"`
Name string `json:"name,omitempty"`
Organization *Organization `json:"organization,omitempty"`
}
//Organization ....
type Organization struct {
OrgID string `json:"orgID,omitempty"`
OrgName string `json:"orgName,omitempty"`
Peer *Peer `json:"peer,omitempty"`
Attributes *Attributes `json:"attributes"`
}
//Peer ...
type Peer struct {
PeerID string `json:"peerID,omitempty"`
PeerName string `json:"peerName,omitempty"`
}
//Attributes ...
type Attributes []struct {
Name string `json:"name"`
Value bool `json:"value"`
}
Finally figured out the correct syntax. We have to pass an array of structs through JSON.
{
"type":"string",
"name":"string",
"organization":
{
"orgID":"1",
"orgName":"string",
"peer":
{
"peerID":"1",
"peerName":"string"
},
"attributes":
[
{"slide0001.html": "Looking Ahead"},
{"slide0008.html": "Forecast"},
{"slide0021.html": "Summary"}
]
}
}
you can do whatever you want in a UnmarshalJSON Function.
i made an example in playground. https://play.golang.org/p/WY6OCR8K3Co
you can get output: {A:[{Name:slide0001.html Value:Looking Ahead} {Name:slide0008.html Value:Forecast} {Name:slide0021.html Value:Summary}]}
var (
jso = []byte(`
{
"attributes":
[
{"slide0001.html": "Looking Ahead"},
{"slide0008.html": "Forecast"},
{"slide0021.html": "Summary"}
]
}`)
)
type B struct {
A As `json:"attributes"`
}
type As []A
type A struct {
Name string
Value string
}
func (as *As) UnmarshalJSON(data []byte) error {
var attr []interface{}
if err := json.Unmarshal(data, &attr); err != nil {
return err
}
if len(attr) > 0 {
newAs := make([]A, len(attr))
// i := 0
for i, val := range attr {
if kv, ok := val.(map[string]interface{}); ok && len(kv) > 0 {
for k, v := range kv {
a := A{
Name: k,
Value: v.(string),
}
newAs[i] = a
i++
break
}
}
}
*as = newAs
}
return nil
}
Quite new to go, sorry if this question sounds obvious.
I would like to use reflection in order to identify the type of an object while reading a json file.
The use case (please see the code below) is the following: I have two structs BoyGift and GirlGift that contain different fields. I have also a Boolean indicator IsBoy that is true if the recipient of the giftis a boy, false otherwise.
The type that encapsulates this behavior is the type Gift:
//Gift type
type Gift struct {
IsBoy bool `json:"isBoy"`
Payload ??? `json:"payload"`
}
that holds the data. How can I define that type in order the json unmarshal to convert dynamically to the correct type? The "json schema" in this case defines that a Gift should be either a BoyGift or a GirlGift. Is it possible to do this via reflection? How?
Doing unmarshal twice would be great, if the Boolean info is known
package main
import (
"encoding/json"
"fmt"
)
//BoyGift type
type BoyGift struct {
Cars uint32 `json:"cars"`
Balls uint32 `json:"balls"`
}
//GirlGift type
type GirlGift struct {
Dolls uint32 `json:"dolls"`
Lego uint32 `json:"lego"`
}
//Gift type
type Gift struct {
IsBoy bool `json:"isBoy"`
Payload GirlGift `json:"payload"`
}
func main() {
b := []byte(`{
"isBoy": true,
"payload": {
"cars": 1,
"balls": 2
}
}`)
var g Gift
err := json.Unmarshal(b, &g)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(g)
}
You should use json.RawMessage to dynamically unmarshal your data.
You can define Gift's Payliad as json.RawMessage and then delay unmarshaling until you know the value of IsBoy. Below you can find a basic example of how to do it.
package main
import (
"encoding/json"
"fmt"
)
//BoyGift type
type BoyGift struct {
Cars uint32 `json:"cars"`
Balls uint32 `json:"balls"`
}
//GirlGift type
type GirlGift struct {
Dolls uint32 `json:"dolls"`
Lego uint32 `json:"lego"`
}
//Gift type
type Gift struct {
IsBoy bool `json:"isBoy"`
Payload json.RawMessage `json:"payload"`
}
func main() {
b1 := []byte(`{
"isBoy": true,
"payload": {
"cars": 1,
"balls": 2
}
}`)
b2 := []byte(`{
"isBoy": false,
"payload": {
"dolls": 3,
"lego": 4
}
}`)
for _, b := range [][]byte{b1, b2} {
var g Gift
err := json.Unmarshal(b, &g)
if err != nil {
fmt.Println(err)
return
}
if g.IsBoy {
var boyGift BoyGift
err := json.Unmarshal(g.Payload, &boyGift)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(boyGift)
} else {
var girlGift GirlGift
err := json.Unmarshal(g.Payload, &girlGift)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(girlGift)
}
}
}
If you want to use Payload as an interface{} (which can be BoyGift or GirlGift), you could create additional helper struct for unmarshalling. Check extended example in Go Playground: https://play.golang.org/p/q1Hn45bgjsc
I'm calling a REST API from my Go program which takes n number of hotel ids in the request and returns their data as a JSON. The response is like the following when say I pass 2 ids in the request, 1018089108070373346 and 2017089208070373346 :
{
"data": {
"1018089108070373346": {
"name": "A Nice Hotel",
"success": true
},
"2017089208070373346": {
"name": "Another Nice Hotel",
"success": true
}
}
}
Since I'm new to Golang I using a JSON Go tool available at http://mholt.github.io/json-to-go/ to get the struct representation for the above response. What I get is:
type Autogenerated struct {
Data struct {
Num1017089108070373346 struct {
Name string `json:"name"`
Success bool `json:"success"`
} `json:"1017089108070373346"`
Num2017089208070373346 struct {
Name string `json:"name"`
Success bool `json:"success"`
} `json:"2017089208070373346"`
} `json:"data"`
}
I cannot use the above struct because the actual id values and the number of ids I pass can be different each time, the JSON returned will have different keys. How can this situation be mapped to a struct ?
Thanks
Use a map:
type Item struct {
Name string `json:"name"`
Success bool `json:"success"`
}
type Response struct {
Data map[string]Item `json:"data"`
}
Run it on the playground
Here is some sample code that utilizes Mellow Marmots answer and shows how to iterate over the items in the response.
test.json
{
"data": {
"1018089108070373346": {
"name": "A Nice Hotel",
"success": true
},
"2017089208070373346": {
"name": "Another Nice Hotel",
"success": true
}
}
}
test.go
package main
import (
"encoding/json"
"fmt"
"os"
)
// Item struct
type Item struct {
Name string `json:"name"`
Success bool `json:"success"`
}
// Response struct
type Response struct {
Data map[string]Item `json:"data"`
}
func main() {
jsonFile, err := os.Open("test.json")
if err != nil {
fmt.Println("Error opening test file\n", err.Error())
return
}
jsonParser := json.NewDecoder(jsonFile)
var filedata Response
if err = jsonParser.Decode(&filedata); err != nil {
fmt.Println("Error while reading test file.\n", err.Error())
return
}
for key, value := range filedata.Data {
fmt.Println(key, value.Name, value.Success)
}
}
Which outputs:
1018089108070373346 A Nice Hotel true
2017089208070373346 Another Nice Hotel true
type Struct struct {
Value string `json:"value"`
Value1 string `json:"value_one"`
Nest Nested `json:"nest"`
}
type Nested struct {
Something string `json:"something"`
}
I want to add elements which are not in the structs definitions without creating another struct type. For example
Struct.Extra1 = Nested{"yy"}
Struct.Nested.Extra2 = "zz"
Which will result
{
"Value": "xx",
"Value1": "xx",
"Extra1": {
"Something", "yy"
},
"Nest": {
"Something": "xx",
"Extra2": "zz"
}
}
SOLUTION1:
I thought of adding omitempty to achieve this but it makes the structs complex.
type Struct struct {
Value string
Value1 string
Nest Nested
Extra1 Nested `json:"omitempty"`
}
type Nested struct {
Something string
Extra2 string `json:"omitempty"`
}
SOLUTION2:
myextras := make(map[string]interface{})
// get Struct.Nested in map[string]interface{} format
myextras = Struct.Nest
myextras["Extra2"] = "zz"
// get Struct in map[string]interface{} format
struct["Nest"] = myextras
struct["Extra1"] = Nested{"yy"}
// solves the problem with lots of type casting but doesn't support json tag naming
Is there a better solution to add nested elements which are not represented in struct datatype with json-tag support and could be used to output to user.
If someone is not happy with the solution provided:
Try tidwall/sjson. It provides functions for quick JSON editing without having to define any structure. It saved me a bunch of time yesterday :D
Example usage:
value, _ := sjson.Set(`{"name":{"last":"Anderson"}}`, "name.last", "Smith")
println(value)
// Output:
// {"name":{"last":"Smith"}}
Based on this answer: Can I use MarshalJSON to add arbitrary fields to a json encoding in golang?
You could do something like (demo: http://play.golang.org/p/dDiTwxhoNn):
package main
import (
"encoding/json"
"fmt"
"log"
)
type Book struct {
Title string
Author string
// extra is used for additional dynamic element marshalling
extra func() interface{}
}
type FakeBook Book
func (b *Book) SetExtra(fn func() interface{}) {
b.extra = fn
}
func (b *Book) MarshalJSON() ([]byte, error) {
if b.extra == nil {
b.extra = func() interface{} { return *b }
}
return json.Marshal(b.extra())
}
func main() {
ms := &Book{
Title: "Catch-22",
Author: "Joseph Heller",
}
ms.SetExtra(func() interface{} {
return struct {
FakeBook
Extra1 struct {
Something string `json:"something"`
} `json:"extra1"`
}{
FakeBook: FakeBook(*ms),
Extra1: struct {
Something string `json:"something"`
}{
Something: "yy",
},
}
})
out, err := json.MarshalIndent(ms, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
mb := &Book{
Title: "Vim-go",
Author: "Fatih Arslan",
}
mb.SetExtra(func() interface{} {
return struct {
FakeBook
Something string `json:"something"`
}{
FakeBook: FakeBook(*mb),
Something: "xx",
}
})
out, err = json.MarshalIndent(mb, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
mc := &Book{
Title: "Another-Title",
Author: "Fatih Arslan",
}
out, err = json.MarshalIndent(mc, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
}
yes. there is a type json.Raw which not a struct but []byte. you can manage it out of struct, in any marshal/unmarshal way.
UPDATE
use any way to edit json string on the fly
package main
import (
"fmt"
"encoding/json"
"strings"
)
type Struct struct {
Value string `json:"value"`
Value1 string `json:"value_one"`
Nest json.RawMessage`json:"nest"`
}
func main() {
s := Struct{Value1: "struct string"}
buf, _ := json.Marshal(s)
fmt.Println(string(buf))
s2 := strings.ReplaceAll(string(buf), "null", `{"extra2":{"anykey":3.1415926535}}`)
fmt.Println(s2)
}
https://play.golang.org/p/lOCxJBs5iRJ
The map approach is the only sane way to do it, everything else (like json.RawMessage fields would require an extra marshalling step anyway.
if you don't want install extra package, you can change or add new values with this code :
package main
import "fmt"
import "strings"
import "encoding/json"
func main() {
s := []byte(`{ "level1a":"aaa", "level1b":{"level2a":"bbb", "level2b":{"level3":"aaa"} } }`)
var j interface{}
json.Unmarshal(s, &j)
SetValueInJSON(j, "level1a", "new value 1a")
SetValueInJSON(j, "level1b.level2a", "new value 2a")
SetValueInJSON(j, "level1b.level2b.level3", "new value 3")
SetValueInJSON(j, "level1b.level2c", "new key")
s,_ = json.Marshal(j)
fmt.Println(string(s))
// result: {"level1a":"new value 1a","level1b":{"level2a":"new value 2a","level2b":{"level3":"new value 3"},"level2c":"new key"}}
}
func SetValueInJSON(iface interface{}, path string, value interface{}) interface{} {
m := iface.(map[string]interface{})
split := strings.Split(path, ".")
for k, v := range m {
if strings.EqualFold(k, split[0]) {
if len(split) == 1 {
m[k] = value
return m
}
switch v.(type) {
case map[string]interface{}:
return SetValueInJSON(v, strings.Join(split[1:], "."), value)
default:
return m
}
}
}
// path not found -> create
if len(split) == 1 {
m[split[0]] = value
} else {
newMap := make(map[string]interface{})
newMap[split[len(split)-1]] = value
for i := len(split) - 2; i > 0; i-- {
mTmp := make(map[string]interface{})
mTmp[split[i]] = newMap
newMap = mTmp
}
m[split[0]] = newMap
}
return m
}