Unmarshal JSON in JSON in Go - json

I want to unmarshal a JSON object where one field contains a JSON string into one coherent object. How do I do that in Go?
Example:
Input:
{
"foo":1,
"bar":"{\\"a\\":\\"Hello\\"}"
}
Go type:
type Child struct {
A string `json:"a"`
}
type Main struct {
Foo int `json:"foo"`
Bar Child `json:"bar"`
}
I guess I'd need to implement a custom UnmarshalJSON implementation on one of the types, but its twisting my head to figure out on which one and how.

I guess you want to treat this as if the JSON String were just part of the surrounding JSON object? If so, then yes, as you suggest, a custom UnmarshalJSON method on Child should accomplish this.
func (c *Child) UnmarshalJSON(p []byte) error {
var jsonString string
if err := json.Unmarshal(p, &jsonString); err != nil {
return err // Means the string was invalid
}
type C Child // A new type that doesn't have UnmarshalJSON method
return json.Unmarshal([]byte(jsonString), (*C)(c))
}
See it in the playground

if i were to create a custom UnmarshalJson for that data, I would create an auxiliary struct auxMain that has the same fields as the main struct but with Bar field as string. Then it unmarshals the JSON data into this auxiliary struct, extracting the Foo field and the Bar field as a string. After that, it unmarshals the Bar field as string into the Child struct, and assigns the extracted Foo field and the Child struct to the Main struct.
It's a round about way but seems to work in the playground.
func (m *Main) UnmarshalJSON(b []byte) error {
type auxMain struct {
Foo int `json:"foo"`
Bar string `json:"bar"`
}
var a auxMain
if err := json.Unmarshal(b, &a); err != nil {
return err
}
var child Child
if err := json.Unmarshal([]byte(a.Bar), &child); err != nil {
return err
}
m.Foo = a.Foo
m.Bar = child
return nil
}
try it out in the PlayGround and see: https://go.dev/play/p/wWIceUxu1tj
Don't know if this is what you are looking for.

Related

Array of struct/map without keys

Welcome all.
Tell me how to get a flat array in GO.
That is, conditionally, I have a structure without keys of the form:
type DashboardHeatMapStruct struct {
float64
string
}
Next, I give it in response to rest in the form of JSON and get output of the form:
[[0,"#AEAEAE"],[0.01,"#0e00ff"],[0.65,"#00ffcf"],[0.7,"#00ffcf"],[0.75,"#00ff9c"],[0.8,"#00ff0a"],[0.85,"#b3ff00"],[0.9,"#ffdc00"],[0.95,"#ff6d00"],[1,"#c60000"]]
Declare a struct type to represent the elements of the JSON array.
type DashboardHeatMapStruct struct {
t float64
c string
}
Implement the json.Unmarshaler interface on that type:
func (d *DashboardHeatMapStruct) UnmarshalJSON(p []byte) error {
// p is expected to be JSON array with float and
// string values. Create slice to match.
v := []interface{}{&d.t, &d.c}
// Unmarshal to JSON array to the slice. The JSON decoder
// follows the pointers in the slice to set the struct members.
return json.Unmarshal(p, &v)
}
Implement the json.Marshler interface to encode back to JSON.
func (d DashboardHeatMapStruct) MarshalJSON() ([]byte, error) {
v := []interface{}{d.t, d.c}
return json.Marshal(v)
}
Unmarshal to a slice of DashboardHeatMapStruct:
var result []DashboardHeatMapStruct
if err := json.Unmarshal(data, &result); err != nil {
// handle error
}
Run it on the playground.

How do I capitalize all keys in a JSON array?

I'm reading a file.json into memory. It's an array of objects, sample:
[
{"id":123123,"language":"ja-JP","location":"Osaka"}
,{"id":33332,"language":"ja-JP","location":"Tokyo"}
,{"id":31231313,"language":"ja-JP","location":"Kobe"}
]
I want to manipulate certain keys in this JSON file, so that they start with uppercase. Meaning
"language" becomes "Language" each time it's found. What I've done so far is to make a struct representing each object, as such:
type sampleStruct struct {
ID int `json:"id"`
Language string `json:"Language"`
Location string `json:"Location"`
}
Here, I define the capitalization. Meaning, id shouldn't be capitalized, but location and language should.
Rest of the code is as such:
func main() {
if len(os.Args) < 2 {
fmt.Println("Missing filename parameter.")
return
}
translationfile, err := ioutil.ReadFile(os.Args[1])
fileIsValid := isValidJSON(string(translationfile))
if !fileIsValid {
fmt.Println("Invalid JSON format for: ", os.Args[1])
return
}
if err != nil {
fmt.Println("Can't read file: ", os.Args[1])
panic(err)
}
}
func isValidJSON(str string) bool {
var js json.RawMessage
return json.Unmarshal([]byte(str), &js) == nil
}
// I'm unsure how to iterate through the JSON objects and only uppercase the objects matched in my struct here.
func upperCaseSpecificKeys()
// ...
Desired output, assuming the struct represents the whole data object, transform each key as desired:
[
{"id":123123,"Language":"ja-JP","Location":"Osaka"}
,{"id":33332,"Language":"ja-JP","Location":"Tokyo"}
,{"id":31231313,"Language":"ja-JP","Location":"Kobe"}
]
The documentation on json.Unmarshal says (with added emphasis):
To unmarshal JSON into a struct, Unmarshal matches incoming object
keys to the keys used by Marshal (either the struct field name or its
tag), preferring an exact match but also accepting a case-insensitive
match
See example here: https://play.golang.org/p/1vv8PaQUOfg
One way is to implement custom marshal method, although not very flexible:
type upStruct struct {
ID int `json:"id"`
Language string
Location string
}
type myStruct struct {
ID int `json:"id"`
Language string `json:"language"`
Location string `json:"location"`
}
func (m myStruct) MarshalJSON() ([]byte, error) {
return json.Marshal(upStruct(m))
}
....
func main() {
var mySArr []myStruct
// 1. Unmarshal the input
err := json.Unmarshal([]byte(myJson), &mySArr)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Input: \n%+v\n", mySArr)
// 2. Then, marshal it using our custom marshal method
val, err := json.Marshal(mySArr)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Output: \n%v\n", string(val))
}
Link to working code: https://play.golang.org/p/T4twqPc34k0
Thanks to mkopriva

Go: Converting JSON string to map[string]interface{}

I'm trying to create a JSON representation within Go using a map[string]interface{} type. I'm dealing with JSON strings and I'm having a hard time figuring out how to avoid the JSON unmarshaler to automatically deal with numbers as float64s. As a result the following error occurs.
Ex.
"{ 'a' : 9223372036854775807}" should be map[string]interface{} = [a 9223372036854775807 but in reality it is map[string]interface{} = [a 9.2233720368547758088E18]
I searched how structs can be used to avoid this by using json.Number but I'd really prefer using the map type designated above.
The go json.Unmarshal(...) function automatically uses float64 for JSON numbers. If you want to unmarshal numbers into a different type then you'll have to use a custom type with a custom unmarshaler. There is no way to force the unmarshaler to deserialize custom values into a generic map.
For example, here's how you could parse values of the "a" property as a big.Int.
package main
import (
"encoding/json"
"fmt"
"math/big"
)
type MyDoc struct {
A BigA `json:"a"`
}
type BigA struct{ *big.Int }
func (a BigA) UnmarshalJSON(bs []byte) error {
_, ok := a.SetString(string(bs), 10)
if !ok {
return fmt.Errorf("invalid integer %s", bs)
}
return nil
}
func main() {
jsonstr := `{"a":9223372036854775807}`
mydoc := MyDoc{A: BigA{new(big.Int)}}
err := json.Unmarshal([]byte(jsonstr), &mydoc)
if err != nil {
panic(err)
}
fmt.Printf("OK: mydoc=%#v\n", mydoc)
// OK: mydoc=main.MyDoc{A:9223372036854775807}
}
func jsonToMap(jsonStr string) map[string]interface{} {
result := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &result)
return result
}
Example - https://goplay.space/#ra7Gv8A5Heh
Related questions - create a JSON data as map[string]interface with the given data

JSON unmarshal integer field into a string

I am struggling with deserializing a integer into a string struct field.
The struct field is a string and is expected to be assignable from users of my library. That's why I want it to be a string, since for the purpose of writing it to the database I actually don't care about the value inside.
The users can supply text, but some just assign integers.
Consider this struct:
type Test struct {
Foo string
}
Sometimes I end up with a JSON value that is valid but won't deserialize into the struct due to the Foo field being a integer instead of a string:
{ "foo": "1" } // works
{ "foo": 1 } // doesn't
json.Unmarshal will blow up with the following error:
json: cannot unmarshal number into Go struct field test.Foo of type string
See the reproduction: https://play.golang.org/p/4Qau3umaVm
Now in every other JSON library (in other languages) I have worked in so far, if the target field is a string and you get a integer the deserializer will usually just wrap the int in a string and be done with it. Can this be achieved in Go?
Since I can't really control how the data comes in I need to make json.Unmarshal unsensitive to this - the other solution would be to define Foo as interface{} which needlessly complicates my code with type assertions etc..
Any ideas on how to do this? I basically need the inverse of json:",string"
To handle big structs you can use embedding.
Updated to not discard possibly previously set field values.
func (t *T) UnmarshalJSON(d []byte) error {
type T2 T // create new type with same structure as T but without its method set!
x := struct{
T2 // embed
Foo json.Number `json:"foo"`
}{T2: T2(*t)} // don't forget this, if you do and 't' already has some fields set you would lose them
if err := json.Unmarshal(d, &x); err != nil {
return err
}
*t = T(x.T2)
t.Foo = x.Foo.String()
return nil
}
https://play.golang.org/p/BytXCeHMvt
You can customize how the data structure is Unmarshaler by implementing the json.Unmarshaler interface.
The simplest way to handle unknown types is to unnmarshal the JSON into an intermediate structure, and handle the type assertions and validation during deserialization:
type test struct {
Foo string `json:"foo"`
}
func (t *test) UnmarshalJSON(d []byte) error {
tmp := struct {
Foo interface{} `json:"foo"`
}{}
if err := json.Unmarshal(d, &tmp); err != nil {
return err
}
switch v := tmp.Foo.(type) {
case float64:
t.Foo = strconv.Itoa(int(v))
case string:
t.Foo = v
default:
return fmt.Errorf("invalid value for Foo: %v", v)
}
return nil
}
https://play.golang.org/p/t0eI4wCxdB

Deserialize patterned fields In Golang

I am writing golang struct, which are compatible with some json structure. However, those most of the fields are know, there will be few fields following some specific patterns(like "x-{randomName}") in the json definition, which I also want to get deserialized to a certain field as map[string]interface{} as well.
Is there any descent way to achieve it?
It's less efficient, but you could unmarshal twice to avoid manually mapping the fields. Once to put all the properly tagged fields into the struct, and then again into a map[string]interface{} to get everything else. If you don't care about the duplicate fields, you don't even need to filter the second map.
You can even do this in an UnmarshalJSONmethod to automatically populate the struct
type S struct {
A string `json:"a"`
B string `json:"b"`
All map[string]interface{}
}
func (s *S) UnmarshalJSON(b []byte) error {
// create a new type to hide the UnmarshalJSON method
// otherwise we'll recurse indefinitely.
type ss S
err := json.Unmarshal(b, (*ss)(s))
if err != nil {
return err
}
// now unmarshal again into the All map
err = json.Unmarshal(b, &s.All)
if err != nil {
return err
}
return nil
}
http://play.golang.org/p/VBVlRjNlHy