Custom UnmarshalJSON: array object to map - json

I have json files like this:
{
"Systems":[
{
"ID":74,
"Data1":0.1,
"Data2":4
},
{
"ID":50,
"Data1":31,
"Data2":3
}
],
"Error":false
}
I would like to unmarshal in Go to something like this (note map):
type Info struct {
Systems map[int]System `json:"Systems"` // key should be ID json field
Error bool `json:"Error"`
}
type System struct {
Data1 float32 `json:"Data1"`
Data2 int `json:"Data2"`
}
Here is my (wrong) code:
package main
import (
"encoding/json"
"fmt"
)
type Info struct {
Systems map[int]System `json:"Systems"` // key should be ID json field
Error bool `json:"Error"`
}
type System struct {
ID int `json:"ID"`
Data1 float32 `json:"Data"`
Data2 int `json:"Data2"`
}
func main() {
file := "{\"Systems\":[{\"ID\":74,\"Data1\":0.1,\"Data2\":4},{\"ID\":50,\"Data1\":31,\"Data2\":3}],\"Error\":true}"
info := Info{}
bytes := []byte(file)
err := json.Unmarshal(bytes, &info)
if err != nil {
fmt.Printf("=> %v\n", err)
}
fmt.Printf("INFO: %+v\n", info)
}
func (d *Info) UnmarshalJSON(buf []byte) error {
var tmp interface{}
if err := json.Unmarshal(buf, &tmp); err != nil {
return err
}
d.Error = tmp.(map[string]interface{})["Error"].(bool)
d.Systems = make(map[int]System)
for _, v := range tmp.(map[string]interface{})["Systems"].([]interface{}) {
d.Systems[v.(map[string]interface{})["ID"].(int)] = v.(map[string]interface{}).(System)
}
return nil
}
https://play.golang.org/p/L_Gx-f9ycjW

You can try this
package main
import (
"encoding/json"
"fmt"
)
func main() {
var str = `{
"Systems":[
{
"ID":74,
"Data1":0.1,
"Data2":4
},
{
"ID":50,
"Data1":31,
"Data2":3
}
],
"Error":false
}`
var t Info
err := json.Unmarshal([]byte(str), &t)
if err != nil {
panic(err)
}
bts, err := json.MarshalIndent(t, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(bts))
}
type Info struct {
Systems map[int]System `json:"Systems"`
Error bool `json:"Error"`
}
type System struct {
ID int `json:"ID,omitempty"`
Data1 float32 `json:"Data1"`
Data2 int `json:"Data2"`
}
func (info *Info) UnmarshalJSON(data []byte) error {
var t struct {
Systems []System `json:"Systems"`
Error bool `json:"Error"`
}
err := json.Unmarshal(data, &t)
if err != nil {
return err
}
info.Systems = make(map[int]System, 0)
for _, v := range t.Systems {
info.Systems[v.ID] = v
}
return nil
}
https://play.golang.org/p/qB3vF08cmW8
Output:
{
"Systems": {
"50": {
"ID": 50,
"Data1": 31,
"Data2": 3
},
"74": {
"ID": 74,
"Data1": 0.1,
"Data2": 4
}
},
"Error": false
}

You can't decode json directly in the struct you proposed because it doesn't match the json structure.
What you can do is decoding the json into this:
type Info struct {
Systems []*System `json:"Systems"` // array here
Error bool `json:"Error"`
Index map[int]*System // not json mapped
}
type System struct {
ID int `json:"ID"`
Data1 float32 `json:"Data1"`
Data2 int `json:"Data2"`
}
and populate the Index field in postprocessing with something like this:
var info Info
json.Unmarshal(dataIn, &info)
info.Index = map[int]*System{} // initialize an empty map
for _, s := range info.Systems {
info.Index[s.ID] = s
}
fmt.Println(info.Index[50].Data1)
you can find a full example here https://play.golang.org/p/B8O6nfI258-

Related

Decoding a dynamic field in Go

I get a response from an external API that has a field which can have 2 values:
{"field": []}
or
{"field": {"key1": "value", "key2": "value"}}
I set the struct to be
type Object Struct {
Field map[string]string `json:"field,omitempty"`
}
And then call my own implemented function to decode the response
func decode(response *http.Response) (*Object, error) {
var response Object
err := json.NewDecoder(response.Body).Decode(&response)
if err != nil {
return nil, err
}
return &response, nil
}
But this works only for the second response ( when field not empty is). For the first response I get an error.
you can do a custom marshaler type for the Field. Example:
type keys struct {
Key1 string
Key2 string
}
type mytype struct {
EmptySlice bool
Keys *keys
}
func (m *mytype) UnmarshalJSON(b []byte) error {
if bytes.Equal(b, []byte("[]")) {
m.EmptySlice = true
return nil
}
m.Keys = &keys{}
return json.Unmarshal(b, &m.Keys)
}
type Object struct {
Field mytype `json:"field"`
}
func main() {
input := []string{
`{"field": []}`,
`{"field": {"key1": "value", "key2": "value"}}`,
}
for i, s := range input {
var o Object
err := json.Unmarshal([]byte(s), &o)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%d: %+v\n", i+1, o)
}
}
https://go.dev/play/p/OqSKfUXHFyb
You can use a custom type and implement the UnmarshalJSON interface method on that type.
For example:
type Field struct {
arr []string
m map[string]string
}
func (f *Field) UnmarshalJSON(b []byte) error {
var m map[string]string
err := json.Unmarshal(b, &m)
if err == nil {
f.m = m
return nil
}
var arr []string
err = json.Unmarshal(b, &arr)
if err == nil {
f.arr = arr
return nil
}
return fmt.Errorf("type of property not array or map")
}
https://go.dev/play/p/JuFE--hWAjw

How do I json unmarshal slice inside a slice

I am trying to unmarshal some pretty ugly json but can't figure out how. I have:
package main
import "fmt"
import "encoding/json"
type PublicKey struct {
ID int `json:"id"`
Key string `json:"key"`
MyData []struct {
ID string `json:"id"`
Value int `json:"value"`
}
}
func main() {
b := `[
{
"id": 1,
"key": "my_key"
},
[
{
"id": "some_id",
"value": 12
},
{
"id": "anorther_id",
"value": 13
}
]
]`
var pk []PublicKey
err := json.Unmarshal([]byte(b), &pk)
if err != nil {
fmt.Println(err)
}
fmt.Println(pk)
}
For the result I am getting:
[{1 my_key []} {0 []}]
The second slice is empty when it shouldn't be.
EDIT:
The error I get is:
json: cannot unmarshal array into Go struct field PublicKey.key of type main.PublicKey
https://play.golang.org/p/cztXOchiiS5
That is some truly hideous JSON! I have two approaches to handling the mixed array elements and I like the 2nd one better. Here's the first approach using interface and a type switch:
package main
import (
"encoding/json"
"errors"
"fmt"
)
type PublicKey struct {
ID int `json:"id"`
Key string `json:"key"`
}
type MyData struct {
ID string `json:"id"`
Value int `json:"value"`
}
type MixedData struct {
Key []PublicKey
MyData [][]MyData
}
func (md *MixedData) UnmarshalJSON(b []byte) error {
md.Key = []PublicKey{}
md.MyData = [][]MyData{}
var obj []interface{}
err := json.Unmarshal([]byte(b), &obj)
if err != nil {
return err
}
for _, o := range obj {
switch o.(type) {
case map[string]interface{}:
m := o.(map[string]interface{})
id, ok := m["id"].(float64)
if !ok {
return errors.New("public key id must be an int")
}
pk := PublicKey{}
pk.ID = int(id)
pk.Key, ok = m["key"].(string)
if !ok {
return errors.New("public key key must be a string")
}
md.Key = append(md.Key, pk)
case []interface{}:
a := o.([]interface{})
myData := make([]MyData, len(a))
for i, x := range a {
m, ok := x.(map[string]interface{})
if !ok {
return errors.New("data array contains unexpected object")
}
val, ok := m["value"].(float64)
if !ok {
return errors.New("data value must be an int")
}
myData[i].Value = int(val)
myData[i].ID, ok = m["id"].(string)
if !ok {
return errors.New("data id must be a string")
}
md.MyData = append(md.MyData, myData)
}
default:
// got something unexpected, handle somehow
}
}
return nil
}
func main() {
b := `[
{
"id": 1,
"key": "my_key"
},
[
{
"id": "some_id",
"value": 12
},
{
"id": "another_id",
"value": 13
}
]
]`
m := MixedData{}
err := json.Unmarshal([]byte(b), &m)
if err != nil {
fmt.Println(err)
}
fmt.Println(m)
}
https://play.golang.org/p/g8d_AsH-pYY
Hopefully there aren't any unexpected other elements, but they can be handled similarly.
Here is the second that relies more on Go's internal JSON parsing with the help of json.RawMessage. It makes the same assumptions about the contents of the array. It assumes that any objects will Unmarshal into PublicKey instances and any arrays consist of only MyData instances. I also added how to marshal back into the target JSON for symmetry:
package main
import (
"encoding/json"
"fmt"
"os"
)
type PublicKey struct {
ID int `json:"id"`
Key string `json:"key"`
}
type MyData struct {
ID string `json:"id"`
Value int `json:"value"`
}
type MixedData struct {
Keys []PublicKey
MyData [][]MyData
}
func (md *MixedData) UnmarshalJSON(b []byte) error {
md.Keys = []PublicKey{}
md.MyData = [][]MyData{}
obj := []json.RawMessage{}
err := json.Unmarshal([]byte(b), &obj)
if err != nil {
return err
}
for _, o := range obj {
switch o[0] {
case '{':
pk := PublicKey{}
err := json.Unmarshal(o, &pk)
if err != nil {
return err
}
md.Keys = append(md.Keys, pk)
case '[':
myData := []MyData{}
err := json.Unmarshal(o, &myData)
if err != nil {
return err
}
md.MyData = append(md.MyData, myData)
default:
// got something unexpected, handle somehow
}
}
return nil
}
func (md *MixedData) MarshalJSON() ([]byte, error) {
out := make([]interface{}, len(md.Keys)+len(md.MyData))
i := 0
for _, x := range md.Keys {
out[i] = x
i++
}
for _, x := range md.MyData {
out[i] = x
i++
}
return json.Marshal(out)
}
func main() {
b := `[
{
"id": 1,
"key": "my_key"
},
[
{
"id": "some_id",
"value": 12
},
{
"id": "another_id",
"value": 13
}
]
]`
m := MixedData{}
err := json.Unmarshal([]byte(b), &m)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(m)
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
if err := enc.Encode(m); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
https://play.golang.org/p/ryZzaWKNcN0
Here's an approach that combines json.RawMessage with the trick of using the default unmarshaler in a type that implements json.Unmarshaler by creating a new temporary type that aliases the target type.
The idea is that we unmarshal the incoming array into a raw message and ensure that the array length is what we expect. Then we unmarshal the individual array elements into the custom struct types using their JSON tag annotations. The end result is that we can unmarshal the PublicKey type in the usual way and the UnmarshalJSON code is not terribly difficult to follow once you understand the tricks.
For example (Go Playground):
type PublicKey struct {
ID int `json:"id"`
Key string `json:"key"`
Data []MyData
}
type MyData struct {
ID string `json:"id"`
Value int `json:"value"`
}
func (pk *PublicKey) UnmarshalJSON(bs []byte) error {
// Unmarshal into a RawMessage so we can inspect the array length.
var rawMessage []json.RawMessage
err := json.Unmarshal(bs, &rawMessage)
if err != nil {
return err
}
if len(rawMessage) != 2 {
return fmt.Errorf("expected array of length 2, got %d", len(rawMessage))
}
// Parse the first object as PublicKey using the default unmarshaler
// using a temporary type that is an alias for the target type.
type PublicKey2 PublicKey
var pk2 PublicKey2
err = json.Unmarshal(rawMessage[0], &pk2)
if err != nil {
return err
}
// Parse the second object as []MyData in the usual way.
err = json.Unmarshal(rawMessage[1], &pk2.Data)
if err != nil {
return err
}
// Finally, assign the aliased object to the target object.
*pk = PublicKey(pk2)
return nil
}
func main() {
var pk PublicKey
err := json.Unmarshal([]byte(jsonstr), &pk)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", pk)
// main.PublicKey{ID:1, Key:"my_key", Data:[]main.MyData{main.MyData{ID:"some_id", Value:12}, main.MyData{ID:"anorther_id", Value:13}}}
}

golang json decode with field name

For following JSON
{
"Jon": {
"Age": 15
},
"Mary": {
"Age": 17
}
}
how can i map it into golang struct, normally, the structure will be
type Person struct {
Name string `json:??,string`
Age int `json:Age, int`
}
as the json field name is the attribute of struct, thank you in advance.
You have to use custom JSON marshalling
package main
import (
"encoding/json"
"log"
"fmt"
)
type Person struct {
Name string `json:??,string`
Age int `json:Age, int`
}
type People map[string]*Person
func (p *People) UnmarshalJSON(data []byte) error {
var transient = make(map[string]*Person)
err := json.Unmarshal(data, &transient)
if err != nil {
return err
}
for k, v := range transient {
v.Name = k
(*p)[k] = v
}
return nil
}
func main() {
jsonInput := `
{
"Jon": {
"Age": 15
},
"Mary": {
"Age": 17
}
}
`
var people People = make(map[string]*Person)
err := people.UnmarshalJSON([]byte(jsonInput))
if err != nil {
log.Fatal(err)
}
for _, person := range people {
fmt.Printf("%v -> %v\n", person.Name, person.Age)
}
}

How do I decode the following JSON?

I have a JSON object of the format
{
"results": [
{
"hits": [
{
"title": "Juliette DELAUNAY",
"author:url": "abc.com"
}
]
}
]
}
To decode in my go program, I have made the following structs
type results struct{
Result []result `json:"results"`
}
type result struct{
Hits []hit `json:"hits"`
}
type hit struct{
Name string `json:"title"`
Url string `json:"author:url"`
}
var m =make(map[string]string)
var t results
But when I try to do the following,
decoder := json.NewDecoder(resp.Body)
err = decoder.Decode(&t)
if err != nil {
fmt.Println(err)
}
for _,x := range t.Result[0].Hits{
m[x.Name] = x.Url
fmt.Println(x.Name,x.Url)
}
It gives a runtime error saying index is out of range. What am I doing wrong? Are my structs incorrect for the given json?
EDIT : The JSON file I need to decode
var jsonStr = []byte(`{"requests":[{"indexName":"recherchepepitesAtoZ","params":"query=x&hitsPerPage=2817&maxValuesPerFacet=42&page=0&facets=%5B%22field_frenchtech_hub_pepite%22%2C%22field_categorie%22%2C%22field_frenchtech_hub_pepite%22%5D&tagFilters="}]}`)
req, err := http.NewRequest("POST", "http://6y0slgl8yj-dsn.algolia.net/1/indexes/*/queries?x-algolia-agent=Algolia%20for%20vanilla%20JavaScript%20(lite)%203.20.4%3Binstantsearch.js%201.10.4%3BJS%20Helper%202.18.0&x-algolia-application-id=6Y0SLGL8YJ&x-algolia-api-key=6832a361e1e1628f8ddb2483623104c6", bytes.NewBuffer(jsonStr))
//req.Header.Set("X-Custom-Header", "application/x-www-form-urlencoded")
req.Header.Set("Content-Type", "application/json")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
Here is a slightly modified version that works on my machine and go playground:
GoPlayground
package main
import (
"encoding/json"
"fmt"
"strings"
)
type results struct {
Result []result `json:"results"`
}
type result struct {
Hits []hit `json:"hits"`
}
type hit struct {
Name string `json:"title"`
Url string `json:"author:url"`
}
var m = make(map[string]string)
func main() {
jsonSample := `{
"results": [
{
"hits": [
{
"title": "Juliette DELAUNAY",
"author:url": "abc.com"
}
]
}
]
}`
var t results
decoder := json.NewDecoder(strings.NewReader(jsonSample))
err := decoder.Decode(&t)
if err != nil {
fmt.Println(err)
}
for _, x := range t.Result[0].Hits {
m[x.Name] = x.Url
fmt.Println(x.Name, x.Url)
}
}

go "encoding/json" : marshal json field

I have a PostgreSQL schema with json field's (DisplayInfo, and FormatInfo). Structure of this field's is dynamic.
I'can read and render it only as string (string type in render struct) :
[
{
"ID":9,
"Name":"120 №1",
"DisplayInfo":"{\"path\": \"http://path/to/img.png\"}",
"Format":{
"Code":"frame-120",
"Width":120,
"Height":60,
"FormatInfo":"[{\"name\": \"\\u0413\\u043b\\u0430\\u0432\\u043d\\u043e\\u0435 \\u0438\\u0437\\u043e\\u0431\\u0440\\u0430\\u0436\\u0435\\u043d\\u0438\\u0435\", \"field_type\": \"img\", \"key\": \"path\"}]"
},
"Weight":0.075,
"Application":8,
"Url":"//path/to/game",
"Referrer":""
}
]
but i want output field DisplayInfo as JSON object. How ?
My render code:
func renderJSON(w http.ResponseWriter, obj models.Model) {
js, err := json.Marshal(obj)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Write(js)
}
UPD 1 : Structure of this field's is dynamic. DisplayInfo may have 'path' field, or may not. They may have additional fields.
UPD 2. I wana output DisplayInfo and FormatInfo as json-object(not string), as part of whole object, like this:
[
{
"ID":9,
"Name":"120 №1",
"DisplayInfo":{"path": "http://path/to/img.png"},
"Format":{
"Code":"frame-120",
"Width":120,
"Height":60,
"FormatInfo":[{"name": "\\u0413\\u043b\\u0430\\u0432\\u043d\\u043e\\u0435 \\u0438\\u0437\\u043e\\u0431\\u0440\\u0430\\u0436\\u0435\\u043d\\u0438\\u0435", "field_type": "img", "key": "path"}]
},
"Weight":0.075,
"Application":8,
"Url":"//path/to/game",
"Referrer":""
}
]
UPD 3: Structures
Actual structure is :
type BannerSerializer struct {
ID int
Name string
DisplayInfo string
Format formatSerializer
Weight float32
Application int
Url string
Referrer string
}
Then i trying this structure:
type BannerSerializer struct {
ID int
Name string
DisplayInfo json.RawMessage
Format formatSerializer
Weight float32
Application int
Url string
Referrer string
}
DisplayInfo serialize as base64 string (or like base64, don't know)
Use a pointer to json.RawMessage:
type Data struct {
Obj *json.RawMessage
}
Playground: http://play.golang.org/p/Qq9IUBDLzJ.
Assuming you have access to change models.Model, you can create your own type with a custom Unmarshaler that just returns the raw string:
type JSONString string
func (s JSONString) MarshalJSON() ([]byte, error) {
return []byte(s), nil
}
Working example:
package main
import (
"encoding/json"
"fmt"
)
type JSONString string
func (s JSONString) MarshalJSON() ([]byte, error) {
return []byte(s), nil
}
type Model struct {
ID int
Name string
DisplayInfo JSONString
}
func main() {
data := []byte(`{
"ID":9,
"Name":"120 №1",
"DisplayInfo":"{\"path\": \"http://path/to/img.png\"}"
}`)
var obj Model
err := json.Unmarshal(data, &obj)
if err != nil {
panic(err)
}
// Here comes your code
js, err := json.Marshal(obj)
if err != nil {
panic(err)
}
fmt.Println(string(js))
}
Output:
{"ID":9,"Name":"120 №1","DisplayInfo":{"path":"http://path/to/img.png"}}
Playground: http://play.golang.org/p/6bcnuGjlU8
You'd have to unmarshal it, here's an example:
var data []*struct {
ID int
DisplayInfo string
}
if err := json.Unmarshal([]byte(j), &data); err != nil {
log.Fatal(err)
}
for _, d := range data {
var displayInfo struct{ Path string }
if err := json.Unmarshal([]byte(d.DisplayInfo), &displayInfo); err != nil {
log.Fatal(err)
}
fmt.Println(d.ID, displayInfo.Path)
}
playground