Marshal and unmarshal JSON bodies without touching all fields - json

Question

I'm looking for a way to unmarshal a JSON body without having to specify targets for all fields. And then be a able to "remarshal" the body with implicit fields untouched.
Something like this would be good, but doesn't work as a expected: (https://play.golang.org/p/fnVOKrmiFj)
package main
import (
"encoding/json"
"fmt"
)
type Transaction struct {
Field1 string `json:"field1"`
X map[string]interface{} `json:"-"`
}
func main() {
body := []byte(`{"field1": "value1", "field2": "value2"}`)
fmt.Printf("%+v\n", string(body))
var unmarshalledTransaction Transaction
json.Unmarshal(body, &unmarshalledTransaction)
fmt.Printf("%+v\n", unmarshalledTransaction)
remarshalledTransaction, _ := json.Marshal(&unmarshalledTransaction)
fmt.Printf("%+v\n", string(remarshalledTransaction))
}
Gives the output
{"field1": "value1", "field2": "value2"}
{Field1:value1 X:map[]}
{"field1":"value1"}
My expected result would be that unmarshalledTransaction contains the "leftover" fields in the X fields. And they are then restored when Marshalling again.
Can this be done?

Answer 1

You would need to implement the MarshalJSON and UnmarshalJSON interfaces, and write your own logic to remap the fields to the appropriate spots:
func (t *Transaction) MarshalJSON() ([]byte, error) {
data := t.X
data["field1"] = t.Field1
return json.Marshal(data)
}
func (t *Transaction) UnmarshalJSON(data []byte) error {
m := make(map[string]interface{})
json.Unmarshal(data, &m)
t.Field1 = m["field1"].(string)
delete(m, "field1")
t.X = m
return nil
}
https://play.golang.org/p/KBGAsXB0xA

Answer 2

If you want a generic solution (that would work with any struct without knowing the fields in advance), you can implement a function that would un-marshal the body into a struct and also return the "leftover" fields.
For that you'd also need to implement a function that would convert any given struct to a map (to be used to then manipulate maps in a generic way instead of known-in-advance structs).
Like so:
func structToMap(object interface{}) (map[string]interface{}, error) {
tempJson, err := json.Marshal(object)
if err != nil {
return nil, err
}
var theMap map[string]interface{}
err = json.Unmarshal(tempJson, &theMap)
if err != nil {
return nil, err
}
return theMap, nil
}
And then:
func unmarshalWithLeftovers(jsonBody []byte, target interface{}) (map[string]interface{}, error) {
err := json.Unmarshal(jsonBody, target)
if err != nil {
return nil, err
}
structMap, err := structToMap(target)
if err != nil {
return nil, err
}
var leftOvers map[string]interface{}
err = json.Unmarshal(jsonBody, &leftOvers)
if err != nil {
return nil, err
}
for k, _ := range structMap {
delete(leftOvers, k)
}
return leftOvers, nil
}
You can then combine the struct and the leftovers map in a similar fashion to re-marshal everything.
See here a working example with the same type and json string that you used in your question:
https://play.golang.org/p/Fot6YVurHH