Custom Json Marshaling - json

I have a third-party json api to work with in go.
It has some endpoints which return data as key-value.
For example here is json for statuses:
{
"result": {
"0": "done",
"1": "incomplete",
"2": "completed",
....
}
}
So as you see it is not an array it is an object.
Is it possible to marshal this kind of json to array of objects like
type Status struct {
Id int
Status string
}
without using additional struct like
type StatusReposne struct {
Result map[string]string `json:"result"`
}
and code to extract values?

As #mkopriva stated in the comment, it's not possible without some extra work. This code does provide means to Marshal/Unmarshal data to/from a slice of Statuss:
func main() {
sr := new(StatusReposne)
json.Unmarshal([]byte(input), sr)
fmt.Printf("%+v\n", sr)
js, _ := json.Marshal(sr)
fmt.Printf("%s\n", js)
}
type StatusReposne struct {
Result []Status `json:"result"`
}
type Status struct {
Id int
Status string
}
func (x *StatusReposne) MarshalJSON() ([]byte, error) {
var buffer struct {
Result map[string]string `json:"result"`
}
buffer.Result = make(map[string]string)
for _, v := range x.Result {
buffer.Result[strconv.Itoa(v.Id)] = v.Status
}
return json.Marshal(&buffer)
}
func (x *StatusReposne) UnmarshalJSON(b []byte) error {
var buffer struct {
Result map[string]string `json:"result"`
}
buffer.Result = make(map[string]string)
json.Unmarshal(b, &buffer)
for k, v := range buffer.Result {
k, _ := strconv.Atoi(k)
x.Result = append(x.Result, Status{Id: k, Status: v})
}
return nil
}
var input = `{
"result": {
"0": "done",
"1": "incomplete",
"2": "completed"
}
}`

Related

How to unmarshal escaped JSON string with UnmarshalJSON

I'm trying to unmarshal the following JSON string
token = `{
"id": 1,
"token": {
"id": 2248637,
"metadata": {
"name": "Name #1",
"formats": "[{\"mimeType\": \"model/gltf-binary\", \"uri\": \"uri1\", {\"mimeType\": \"image/gif\", \"uri\": \"uri2\"}]"
}
}`
I can unmarshal it with 2 phases like this. However, I would like to use custom unmarshalJSON but I fail. I got error
My code as follow:
type FileFormat struct {
MIMEType string
URI string
}
func (format *FileFormat) UnmarshalJSON(data []byte) error {
var aux []interface{}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
format.MIMEType = aux[0].(string)
format.URI = aux[1].(string)
return nil
}
type TokenMetadata struct {
Name string `json:"name"`
Formats []FileFormat `json:"formats"`
}
type Token struct {
ID TokenID `json:"tokenId"`
Metadata TokenMetadata `json:"metadata"`
}
func main() {
var tokenRes OwnedToken
if err := json.Unmarshal([]byte(token), &tokenRes); err != nil {
fmt.Println(err)
}
}
And the error is
json: cannot unmarshal string into Go struct field TokenMetadata.token.metadata.formats of type []main.FileFormat
How can I fix this problem? Many thanks!
The JSON array of file formats is double encoded. Declare a Go type corresponding to the array. Double decode in the UnmarshalJSON method for that type.
type FileFormats []FileFormat
func (ff *FileFormats) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
return json.Unmarshal(
[]byte(s),
(*[]FileFormat)(ff))
}
type TokenMetadata struct {
Name string `json:"name"`
Formats FileFormats `json:"formats"`
}
Note: The conversion from from *FileFormats to *[]FileFormat is required to prevent recursion.

Is it possible to have a structure for dynamic keys along with static keys for json in Golang

My apologies for the basic question. I am new to Golang and I have the json to parse as below
{
"config1":{
"Parameters":{
"Pm1":"value",
"Pm2":"value",
"Pm3":"value"
},
"dynamic_key1":{
"Parameters":{
"a":"value",
"b":"value",
"c":"value",
"d":"value"
},
"Epoch":"value"
},
"Epoch":"value"
}
}
I am trying to write a struct to parse this json and wrote the struct in the following way.
type Parameters struct {
Pm1 string `json:"Pm1"`
Pm2 string `json:"Pm2"`
Pm3 string `json:"Pm3"`
}
type dynamicParametes struct {
a string `json:"a"`
b string `json:"b"`
c string `json:"c"`
d string `json:"d"`
}
type dynamic struct {
Parameters dynamicParametes `json:"Parameters"`
Epoch string `json:"Epoch"`
}
type config1 struct {
Parameters Parameters `json:"Parameters"`
Dynamic_keys map[string]dynamic `json:"-"`
Epoch string `json:"Epoch"`
}
type config struct {
config1 config1 `json:"config1"`
}
I was hoping that the map will match all the matching keys with dynamic structs and show them in the map. But, I see it created an empty map in the response.
Implemented custom unmarshler for config type.
Note
If you don't need Parameters and dynamicParametes as struct types, you can simply unmarshal them into map[string]string
you have to expose all fields in your structs to do json unmarshaling
validate your json string
type config struct {
Config1 config1 `json:"config1"`
}
type _config config
func (b *config) UnmarshalJSON(data []byte) error {
var v = struct {
Config1 map[string]interface{} `json:"config1"`
}{}
if err := json.Unmarshal(data, &v); err != nil {
return err
}
c := _config{}
err := json.Unmarshal(data, &c)
if err != nil {
return err
}
b.Config1.Parameters = c.Config1.Parameters
b.Config1.Epoch = c.Config1.Epoch
if b.Config1.Dynamic_keys == nil {
b.Config1.Dynamic_keys = map[string]dynamic{}
}
for key, config := range v.Config1 {
if key == `Parameters` || key == `Epoch` {
continue
}
data, err := json.Marshal(config)
if err != nil {
return err
}
d := dynamic{}
err = json.Unmarshal(data, &d)
if err != nil {
return err
}
b.Config1.Dynamic_keys[key] = d
}
return nil
}
you can see full code here
All you need is understand how base data types looks in json.
Field Parameters in your json is simple map[string]string and you can unmarshall it with standart json.Unmasrhall without any aditional implementation of interface json.Unmarshaler.
Link for Go Playground with code below
package main
import (
"encoding/json"
"fmt"
)
const jsonStr = `{
"config1":{
"Parameters":{
"Pm1":"value_1",
"Pm2":"value_2",
"Pm3":"value_3"
},
"dynamic_key1":{
"Parameters":{
"a":"value_1",
"b":"value_2",
"c":"value_3",
"d":"value_4"
},
"Epoch":"value"
},
"Epoch":"value"
}
}`
type Data struct {
Config1 struct {
Parameters map[string]string `json:"Parameters"`
Dynamic struct {
Parameters map[string]string `json:"Parameters"`
Epoch string `json:"Epoch"`
} `json:"dynamic_key1"`
Epoch string `json:"Epoch"`
} `json:"config1"`
}
func main() {
var data Data
_ = json.Unmarshal([]byte(jsonStr), &data)
fmt.Printf("%+v\n", data)
}
Output:
{Config1:{Parameters:map[Pm1:value_1 Pm2:value_2 Pm3:value_3] Dynamic:{Parameters:map[a:value_1 b:value_2 c:value_3 d:value_4] Epoch:value} Epoch:value}}

Is there a way to decode JSON in Go with converting types?

I'm working with an API which can return one value as int if it's zero and as string if it's not zero and I want a decoder which will decode these two JSONs correctly into struct
{
"id": 1,
"rating": 0
}
{
"id": 2,
"rating": "2"
}
type User struct {
Id int64 `json:"id"`
Rating int64 `json:"rating,string"`
}
So it should try to convert any JSON type (int, float, string) to type specified in struct and raise error only if it isn't possible. Standard json.Decoder doesn't do that.
Or maybe there is some more customizable json parsing library?
You are trying to parse an int64 from a JSON string. You can do this using custom types that implement the JSON Unmarshaler interface.
e.g.
type User struct {
Id int64 `json:"id"`
Rating Int64String `json:"rating"`
}
type Int64String int64
func (i Int64String) MarshalJSON() ([]byte, error) {
return json.Marshal(strconv.FormatInt(int64(i), 10))
}
func (i *Int64String) UnmarshalJSON(data []byte) error {
var jstring string
err := json.Unmarshal(data, &jstring)
if err != nil {
return err
}
*(*int64)(i), err = strconv.ParseInt(jstring, 0, 64)
return err
}
Playground
I recommend you to change the api, if thats not possible then you can use the interface type for the rating and manually check for the type during parsing (ints will return as float64):
package main
import (
"fmt"
"encoding/json"
)
type User struct {
Id int `json:"id"`
Rating interface{} `json:"rating"`
}
func main() {
usersJson := `[{"id": 1, "rating": 0}, {"id": 2,"rating": "2"}]`
var users []User
err := json.Unmarshal([]byte(usersJson), &users)
if err != nil {
fmt.Println("err: ",err)
return
}
for _, u := range users {
switch u.Rating.(type) {
case float64:
fmt.Println("its an float64", u.Rating.(float64))
case string:
fmt.Println("its an string", u.Rating.(string))
}
}
}
Solved like this:
type Int64 struct {
Value int64
}
func (this *Int64) UnmarshalJSON(bytesValue []byte) error {
stringValue := string(bytesValue)
if len(stringValue) > 2 {
if stringValue[0] == '"' && stringValue[len(stringValue)-1] == '"' {
stringValue = stringValue[1 : len(stringValue)-1]
}
}
var err error
this.Value, err = strconv.ParseInt(stringValue, 10, 64)
return err
}
func (this *Int64) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf("%v", this.Value)), nil
}

Parsing json without index name in golang

I have a json in the following format
{
"status_code": 200,
"status_message": "OK",
"response": {
"Messages": [
"CODE_NOT_AVAILABLE"
],
"UnknownDevices": {
"": [
"6",
"7",
"8",
"9",
"10"
]
}
}
}
As we see we are missing one index key, after unknownDevices, i am trying to unmarshal this json using golan in the following way
package main
import (
"encoding/json"
"fmt"
)
type pushWooshResponse struct {
Status int `json:"status_code"`
Status_msg string `json:"status_message"`
Response response
}
type response struct {
Message []string `json:"Messages"`
UnknownDevices devices
}
type devices struct {
Udevices []string `json:""`
}
func main() {
itemInfoR := `{"status_code":200,"status_message":"OK","response":{"Messages":["CODE_NOT_AVAILABLE"],"UnknownDevices":{"devices":["6","7","8","9","10"]}}}`
itemInfoBytes := []byte(itemInfoR)
var ItemInfo pushWooshResponse
er := json.Unmarshal(itemInfoBytes, &ItemInfo)
if er != nil {
fmt.Println("Error", er.Error())
} else {
fmt.Println(ItemInfo)
}
}
Output is
{200 OK {[CODE_NOT_AVAILABLE] {[]}}}
Everything is working fine except for the last part, which i am not able to unmarshall this. Can you help me out to umarshal the last part of the json.
There may be other options, but whenever you see strange JSON you can always fall back to implementing your own custom (un)marshalling by implementing json.Unmarshaler and/or json.Marshaler.
Perhaps something like:
type devices struct {
Udevices []string
}
func (d *devices) UnmarshalJSON(b []byte) error {
var x map[string][]string
err := json.Unmarshal(b, &x)
if err == nil {
// perhaps check that only a single
// key exists in the map as well
d.Udevices = x[""]
}
return err
}
func (d devices) MarshalJSON() ([]byte, error) {
x := map[string][]string{"": d.Udevices}
return json.Marshal(x)
}
Playground

Golang modify json without struct

type Struct struct {
Value string `json:"value"`
Value1 string `json:"value_one"`
Nest Nested `json:"nest"`
}
type Nested struct {
Something string `json:"something"`
}
I want to add elements which are not in the structs definitions without creating another struct type. For example
Struct.Extra1 = Nested{"yy"}
Struct.Nested.Extra2 = "zz"
Which will result
{
"Value": "xx",
"Value1": "xx",
"Extra1": {
"Something", "yy"
},
"Nest": {
"Something": "xx",
"Extra2": "zz"
}
}
SOLUTION1:
I thought of adding omitempty to achieve this but it makes the structs complex.
type Struct struct {
Value string
Value1 string
Nest Nested
Extra1 Nested `json:"omitempty"`
}
type Nested struct {
Something string
Extra2 string `json:"omitempty"`
}
SOLUTION2:
myextras := make(map[string]interface{})
// get Struct.Nested in map[string]interface{} format
myextras = Struct.Nest
myextras["Extra2"] = "zz"
// get Struct in map[string]interface{} format
struct["Nest"] = myextras
struct["Extra1"] = Nested{"yy"}
// solves the problem with lots of type casting but doesn't support json tag naming
Is there a better solution to add nested elements which are not represented in struct datatype with json-tag support and could be used to output to user.
If someone is not happy with the solution provided:
Try tidwall/sjson. It provides functions for quick JSON editing without having to define any structure. It saved me a bunch of time yesterday :D
Example usage:
value, _ := sjson.Set(`{"name":{"last":"Anderson"}}`, "name.last", "Smith")
println(value)
// Output:
// {"name":{"last":"Smith"}}
Based on this answer: Can I use MarshalJSON to add arbitrary fields to a json encoding in golang?
You could do something like (demo: http://play.golang.org/p/dDiTwxhoNn):
package main
import (
"encoding/json"
"fmt"
"log"
)
type Book struct {
Title string
Author string
// extra is used for additional dynamic element marshalling
extra func() interface{}
}
type FakeBook Book
func (b *Book) SetExtra(fn func() interface{}) {
b.extra = fn
}
func (b *Book) MarshalJSON() ([]byte, error) {
if b.extra == nil {
b.extra = func() interface{} { return *b }
}
return json.Marshal(b.extra())
}
func main() {
ms := &Book{
Title: "Catch-22",
Author: "Joseph Heller",
}
ms.SetExtra(func() interface{} {
return struct {
FakeBook
Extra1 struct {
Something string `json:"something"`
} `json:"extra1"`
}{
FakeBook: FakeBook(*ms),
Extra1: struct {
Something string `json:"something"`
}{
Something: "yy",
},
}
})
out, err := json.MarshalIndent(ms, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
mb := &Book{
Title: "Vim-go",
Author: "Fatih Arslan",
}
mb.SetExtra(func() interface{} {
return struct {
FakeBook
Something string `json:"something"`
}{
FakeBook: FakeBook(*mb),
Something: "xx",
}
})
out, err = json.MarshalIndent(mb, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
mc := &Book{
Title: "Another-Title",
Author: "Fatih Arslan",
}
out, err = json.MarshalIndent(mc, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
}
yes. there is a type json.Raw which not a struct but []byte. you can manage it out of struct, in any marshal/unmarshal way.
UPDATE
use any way to edit json string on the fly
package main
import (
"fmt"
"encoding/json"
"strings"
)
type Struct struct {
Value string `json:"value"`
Value1 string `json:"value_one"`
Nest json.RawMessage`json:"nest"`
}
func main() {
s := Struct{Value1: "struct string"}
buf, _ := json.Marshal(s)
fmt.Println(string(buf))
s2 := strings.ReplaceAll(string(buf), "null", `{"extra2":{"anykey":3.1415926535}}`)
fmt.Println(s2)
}
https://play.golang.org/p/lOCxJBs5iRJ
The map approach is the only sane way to do it, everything else (like json.RawMessage fields would require an extra marshalling step anyway.
if you don't want install extra package, you can change or add new values with this code :
package main
import "fmt"
import "strings"
import "encoding/json"
func main() {
s := []byte(`{ "level1a":"aaa", "level1b":{"level2a":"bbb", "level2b":{"level3":"aaa"} } }`)
var j interface{}
json.Unmarshal(s, &j)
SetValueInJSON(j, "level1a", "new value 1a")
SetValueInJSON(j, "level1b.level2a", "new value 2a")
SetValueInJSON(j, "level1b.level2b.level3", "new value 3")
SetValueInJSON(j, "level1b.level2c", "new key")
s,_ = json.Marshal(j)
fmt.Println(string(s))
// result: {"level1a":"new value 1a","level1b":{"level2a":"new value 2a","level2b":{"level3":"new value 3"},"level2c":"new key"}}
}
func SetValueInJSON(iface interface{}, path string, value interface{}) interface{} {
m := iface.(map[string]interface{})
split := strings.Split(path, ".")
for k, v := range m {
if strings.EqualFold(k, split[0]) {
if len(split) == 1 {
m[k] = value
return m
}
switch v.(type) {
case map[string]interface{}:
return SetValueInJSON(v, strings.Join(split[1:], "."), value)
default:
return m
}
}
}
// path not found -> create
if len(split) == 1 {
m[split[0]] = value
} else {
newMap := make(map[string]interface{})
newMap[split[len(split)-1]] = value
for i := len(split) - 2; i > 0; i-- {
mTmp := make(map[string]interface{})
mTmp[split[i]] = newMap
newMap = mTmp
}
m[split[0]] = newMap
}
return m
}