Unmarshalling of JSON with dynamic keys - json

Question

I have a scenario where the JSON that has dynamic set of fields that need to get unmarshalled in to a struct.
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
type Status struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status map[string]Status `json:"status"`
}
func main() {
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var person Person
if err := dec.Decode(&person); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(person)
fmt.Println(person.Status["bvu62fu6dq"])
}
}
The output:
{john 23 map[]}
{false}
When it gets unmarshalled, the nested status struct is not being correctly resolved to the value in the JSON (shows false even with true value in JSON), is there any issue in the code?

Answer 1

Your types don't really match with the JSON you have:
type Status struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status map[string]Status `json:"status"`
}
Maps to JSON that looks something like this:
{
"name": "foo",
"age": 12,
"status": {
"some-string": {
"Status": true
}
}
}
The easiest way to unmarshal data with a mix of known/unknown fields in a go type is to have something like this:
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"` // skip this key
}
Then, first unmarshal the known data:
var p Person
if err := json.Unmarshal([]byte(jsonStream), &p); err != nil {
panic(err)
}
// then unmarshal the rest of the data
if err := json.Unmarshal([]byte(jsonStream), &p.Random); err != nil {
panic(err)
}
Now the Random map will contain every and all data, including the name and age fields. Seeing as you've got those tagged on the struct, these keys are known, so you can easily delete them from the map:
delete(p.Random, "name")
delete(p.Random, "age")
Now p.Random will contain all the unknown keys and their respective values. These values apparently will be an object with a field status, which is expected to be a boolean. You can set about using type assertions and convert them all over to a more sensible type, or you can take a shortcut and marshal/unmarshal the values. Update your Person type like so:
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"`
Statuses map[string]Status `json:"-"`
}
Now take the clean Random value, marshal it and unmarshal it back into the Statuses field:
b, err := json.Marshal(p.Random)
if err != nil {
panic(err)
}
if err := json.Unmarshal(b, &p.Statuses); err != nil {
panic(err)
}
// remove Random map
p.Random = nil
The result is Person.Statuses["bvu62fu6dq"].Status is set to true
Demo
Cleaning this all up, and marshalling the data back
Now because our Random and Statuses fields are tagged to be ignored for JSON marshalling (json:"-"), marshalling this Person type won't play nice when you want to output the original JSON from these types. It's best to wrap this logic up in a custom JSON (un)-Marshaller interface. You can either use some intermediary types in your MarshalJSON and UnmarshalJSON methods on the Person type, or just create a map and set the keys you need:
func (p Person) MarshalJSON() ([]byte, error) {
data := make(map[string]interface{}, len(p.Statuses) + 2) // 2 being the extra fields
// copy status fields
for k, v := range p.Statuses {
data[k] = v
}
// add known keys
data["name"] = p.Name
data["age"] = p.Age
return json.Marshal(data) // return the marshalled map
}
Similarly, you can do the same thing for UnmarshalJSON, but you'll need to create a version of the Person type that doesn't have the custom handling:
type intermediaryPerson struct {
Name string `json:"name"`
Age int `json:"age"`
Random map[string]interface{} `json:"-"`
}
// no need for the tags and helper fields anymore
type Person struct {
Name string
Age int
Statuses map[string]Status // Status type doesn't change
}
func (p *Person) UnmarshalJSON(data []byte) error {
i := intermediaryPerson{}
if err := json.Unmarshal(data, &i); err != nil {
return err
}
if err := json.Unmarshal(data, &i.Random); err != nil {
return err
}
delete(i.Random, "name")
delete(i.Random, "age")
stat, err := json.Marshal(i.Random)
if err != nil {
return err
}
// copy known fields
p.Name = i.Name
p.Age = i.Age
return json.Unmarshal(stat, &p.Statuses) // set status fields
}
In cases like this, it's common to create a type that handles the known fields and embed that, though:
type BasePerson struct {
Name string `json:"name"`
Age int `json:"age"`
}
and embed that in both the intermediary and the "main"/exported type:
type interPerson struct {
BasePerson
Random map[string]interface{} `json:"-"`
}
type Person struct {
BasePerson
Statuses map[string]Status
}
That way, you can just unmarshal the known fields directly into the BasePerson type, assign it, and then handle the map:
func (p *Person) UnmarshalJSON(data []byte) error {
base := BasePerson{}
if err := json.Unmarshal(data, &base); err != nil {
return err
}
p.BasePerson = base // takes care of all known fields
unknown := map[string]interface{}{}
if err := json.Unmarshal(data, unknown); err != nil {
return err
}
// handle status stuff same as before
delete(unknown, "name") // remove known fields
// marshal unknown key map, then unmarshal into p.Statuses
}
Demo 2
This is how I'd go about it. It allows for calls to json.Marshal and json.Unmarshal to look just like any other type, it centralises the handling of unknown fields in a single place (the implementation of the marshaller/unmarshaller interface), and leaves you with a single Person type where every field contains the required data, in a usable format. It's a tad inefficient in that it relies on unmarshalling/marshalling/unmarshalling the unknown keys. You could do away with that, like I said, using type assertions and iterating over the unknown map instead, faffing around with something like this:
for k, v := range unknown {
m, ok := v.(map[string]interface{})
if !ok {
continue // not {"status": bool}
}
s, ok := m["status"]
if !ok {
continue // status key did not exist, ignore
}
if sb, ok := s.(bool); ok {
// ok, we have a status bool value
p.Statuses[k] = Status{
Status: sb,
}
}
}
But truth be told, the performance difference won't be that great (it's micro optimisation IMO), and the code is a tad too verbose to my liking. Be lazy, optimise when needed, not whenever

Answer 2

Type doesn't meet with your json value.
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
For above json your code should look like below snnipet to work (some modifications in your existing code).
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
const jsonStream = `{
"name": "john",
"age": 23,
"bvu62fu6dq": {
"status": true
}
}`
type bvu62fu6dq struct {
Status bool
}
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Status bvu62fu6dq `json:"bvu62fu6dq"`
}
func main() {
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var person Person
if err := dec.Decode(&person); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(person)
fmt.Println(person.Status)
}
}
Based on your json data you have to map with type fields.
Run code snippet