Cast nested []interface{} to []map[string]interface{} - json

Question

My goal is to generate from a json a map[string]interface structure , where all nested []interface{} are casted into []map[string]interface{} . This is because we are using the module https://github.com/ivahaev/go-xlsx-templater to fill xlsx from json . And all the data is expected to be into a map[string]interface{} struct where all the nested []interface{} are []map[string]interface.
Having a json as an input like the following :
{
"totalAmount": 4,
"subtotal": 4,
"Vendors": [{
"MethodOfTenders": [{
"order": 1,
"fees": 2
}, {
"order": 1,
"fees": 2
}],
"subtotalFees": 4
},
{
"MethodOfTenders": [{
"order": 1,
"fees": 2
}, {
"order": 1,
"fees": 1
}],
"subtotalFees": 3
}
]
}
When unmarshalling into an map[string]interface{} .I get the following struct :
map[string]interface{}:
"totalAmount" : 4 interface{}(float64)
"subtotal" : 4 interface{}(float64)
"Vendors" : interface{}([]interface{})
[0]: interface(map[string]interface{})
"MethodOfTenders" : interface{}([]interface{})
[0] : interface(map[string]interface{})
"order" : 1 interface{}(float64)
"fees" : 2 interface{}(float64)
[1] : interface(map[string]interface{})
"order" : 1 interface{}(float64),
"fees" : 2 interface{}(float64)
"subtotalFees" : 4 interface{}(float64)
[1]: interface(map[string]interface{})
"MethodOfTenders" : interface{}([]interface{})
[0] : interface(map[string]interface{})
"order" : 1 interface{}(float64)
"fees" : 2 interface{}(float64)
[1] : interface(map[string]interface{})
"order" : 1 interface{}(float64),
"fees" : 2 interface{}(float64)
"subtotalFees" : 3 interface{}(float64)
After doing some parsing , ranging each []interface{} and creating a a []map[string]interface{} to store each one of the nested map[string]interface .
I got the desired result where all []interfaces{} are []map[string]interface{}
map[string]interface{}:
"totalAmount" : 4 interface{}(float64)
"subtotal" : 4 interface{}(float64)
"Vendors" : interface{}([]map[string]interface{})
[0]: interface(map[string]interface{})
"MethodOfTenders" : interface{}([]map[string]interface{})
[0] : interface(map[string]interface{})
"order" : 1 interface{}(float64)
"fees" : 2 interface{}(float64)
[1] : interface(map[string]interface{})
"order" : 1 interface{}(float64),
"fees" : 2 interface{}(float64)
"subtotalFees" : 4 interface{}(float64)
[1]: interface(map[string]interface{})
"MethodOfTenders" : interface{}([]map[string]interface{})
[0] : interface(map[string]interface{})
"order" : 1 interface{}(float64)
"fees" : 2 interface{}(float64)
[1] : interface(map[string]interface{})
"order" : 1 interface{}(float64),
"fees" : 2 interface{}(float64)
"subtotalFees" : 3 interface{}(float64)
Is there any way to walk al the map[string]interface recursively and change the []interfaces{} for []map[string]interface ?
EDIT:
Here the repository with the full template and json , currently working with the reflection approach .
template and json repo

Answer 1

Update
So I've since noticed you added a link to a repo containing what you have so far. The template is weird (some of the values in there aren't part of your sample data). I've removed all fields except for the ones that were actually in the sample data JSON you have. As it turns out, the package you are using is quite rough around the edges (it does take care of type assertions in nested values, but not at the top level). I'd personally either fork the package and fix that issue, but in the mean while, if you know what fields you want to iterate over, I got it all to work in about 5 minutes with this:
var jsonMap map[string]interface{}
data := sampleData()
err := json.Unmarshal(data, &jsonMap)
if err != nil {
panic("Final log")
}
ctx := jsonMap
v := jsonMap["Vendors"]
vs := v.([]interface{})
vendors := make([]map[string]interface{}, 0, len(vs))
for _, v := range vs {
m := v.(map[string]interface{})
vendors = append(vendors, m)
}
jsonMap["Vendors"] = vendors
doc := xlst.New()
err = doc.ReadTemplate("export_support_template.xlsx")
if err != nil {
fmt.Println("ERROR OPENING THE TEMPLATE: ", err)
panic("error opening template")
}
err = doc.Render(ctx)
if err != nil {
fmt.Println("ERROR RENDERING THE TEMPLATE: ", err)
panic("error rendering template")
}
err = doc.Save("report.xlsx")
As you can see, I just unmarshalled the data. I then focused on the Vendors key, cast it to a slice (v.([]inteface{}), created a new variable of type []map[string]interface{}, and copied over the data casting the elements of the slice to maps in this simple loop:
for _, v := range vs {
m := v.(map[string]interface{})
vendors = append(vendors, m)
}
Then, I just reassigned the Vendors key in the original map (ctx["Vendors"] = vendors), and passed that in to the template. Less than 10 lines of code needed, and everything worked like a charm. No need for reflection, or any other magic. Just straightforward type casts. Without this, the package you're using does complain about the use of range on an interface{} type (instead of first checking to see if the key can be successfully cast to a []interface{} first). I removed all other functions from your main file (except for the sample data one), ran go build and executed ./json_marshaller. It churned out a correctly populated xlsx file no problem. Easy.
I'd still recommend you actually create a PR to the templater package so you don't have to handle this stuff yourself. It should be a fairly straightforward change, but for the time being: this approach works perfectly well.
Include your template
Looking at the repository of the go-xlsx-templater, I don't really see why you'd need anything other than this:
data := map[string]interface{}
_ = json.Unmarshal(input, &data)
The reason why this is perfectly fine, is because go-xlsx-templater performs the type conversion of all interface{} values within this map already. It will traverse the map, and check if any of the values are of the type []interface{}, it'll then iterate over this slice and check for map[string]interface{} values inside the slice. It's all in the source code. Going by their own documentation, it should work just fine if your template looks somewhat like this:
| Total: | {{ totalAmount }} |
| Subtotal: | {{ subtotal }} |
| | Vendor subtotal | fees | order |
| {{ range Vendors }} |
| | | {{ MethodsOfTender.Fees }} | {{ MethodsOfTender.order }} |
| | {{ subtotalFees }} |
| {{ end }} |
You should end up with the spreadsheet being populated correctly:
| Total: | 4 | | |
| Subtotal: | 4 | | |
| | Vendor subtotal | fees | order |
| | | 2 | 1 |
| | | 2 | 1 |
| | 4 | | |
| | | 2 | 1 |
| | | 1 | 1 |
| | 3 | | |
Essentially, the data-set you're showing is no different to the example the repo gives in their very own main README file. The context data itself contains nested maps, and a screenshot of a spreadsheet template showing how to use it. Putting it bluntly, I feel like your question is pretty much a case of RTFM...
Without your template, however, there's no way to be sure, so I'm leaving my initial answer in full below...
If I understand what you're asking correctly, you're looking for a way to use type assertions/casts to extract keys from a map of type map[string]interface{} and use the values of type []interface{} as []map[string]interface{}. In this particular example, you need the top level key Vendors to be accessible as []map[string]interface{} (as opposed to interface{} or []interface{}). If that's what you're looking for, you can do this quite easily:
data := map[string]interface{}{}
if err := json.Unmarshal(jsonBytes, &data); err != nil {
// handle error
}
nested := map[string][]map[string]interface{}{} // subset of data that needs to be accessible as slices of maps
for k, v := range data {
if s, ok := v.([]interface); ok {
// this key is of type []interface, see if we can use it as slice of maps
if mapS, err := sliceAnyToMap(s); err == nil {
nested[k] = mapS
} // handle error if needed
}
}
func sliceAnyToMap(s []any) ([]map[string]interface{}, error) {
ret := make([]map[string]interface{}, 0, len(s))
for _, v := range s {
if m, ok := v.(map[string]interface{}); ok {
ret = append(ret, m)
} else {
return nil, errors.New("slice contains non-map data")
}
}
return ret, nil
}
With this, you'll end up with a variable called nested which will contain keys like Vendors, with the data accessible as a []map[string]interface{}. This can be passed on to your templater to populate the spreadsheet with the vendors data.
Demo here
Now, inside some of the Vendors key, there is some data which is in turn a a map, which still can't be accessed directly. Breaking up this sliceAnyToMap stuff into a function will make it easier to convert all data you need fairly easily.
Now, having said all this, I do personally think this is very much an X-Y problem. The data you've shown here looks to be of a very well defined structure. Rather than faffing around with map[string]interface{} and the like, it would be a whole lot easier to read, write, and maintain if you were to use actual types. Based on what you've included in your question, a couple of types like this would be all it takes:
type Data struct {
Total float64 `json:"totalAmount"`
Subtotal float64 `json:"subtotal"`
Vendors []Vendor `json:"Vendors"`
}
type Vendor struct {
MethodOfTenders []MOT `json:"MethodOfTenders"`
SubtotalFees float64 `json:"subtotalFees"`
}
type MOT struct {
Fees float64 `json:"fees"`
Order float64 `json:"order"`
}
With these types, you can quickly, and easily parse the given input into a format that is much, much easier to use further down the line:
data := Data{}
if err := json.Unmarshal(input, &data); err != nil {
// handle error
}
for _, vendor := range data.Vendors {
fmt.Printf("Vendor subtotal fees: %f\n", vendor.SubtotalFees)
for i, mot := range vendor.MethodOfTenders {
fmt.Printf("MOT %d:\nOrder: %f\nFees: %f\n\n", i+1, mot.Order, mot.Fees)
}
fmt.Println("-----")
}
Demo here
Last couple of things I'm wondering is whether or not all numberic values should be float64. Sure, it makes sense for amounts and/or fees (assuming they're monetary values), but the Order field probably contains an order number, which I don't expect to be a value like 0.123.
Finally, if you need to pass this data on to a templater of sorts which expects a map, then it's a fairly trivial thing to do using the aforementioned types all the same. There's a couple of ways to do this. The easy way being:
func (d Data) ToMap() (map[string]interface{}, error) {
raw, err := json.Marshal(d)
if err != nil {
return nil, err
}
asMap := map[string]interface{}{}
if err := json.Unmarshal(raw, &asMap); err != nil {
return nil, err
}
return asMap, nil
}
JSON marshalling is quick to implement, but it's a tad inefficient, so you if performance matters a lot, then you could spend a few minutes writing some methods like:
func (d Data) ToMap() map[string]interface{} {
vendors := make([]map[string]interface{}, 0, len(d.Vendors))
for _, v := range d.Vendors {
vendors = append(vendors, v.ToMap())
}
return map[string]interface{}{
"totalAmount": d.Total,
"subtotal": d.Subtotal,
"Vendors": vendors,
}
}
func (v Vendor) ToMap() map[string]interface{} {
mot := make([]map[string]interface{}, 0, len(v.MethodsOfTender))
for _, m := range v.MethodsOfTender {
mot = append(mot, m.ToMap())
}
return map[string]interface{
"MethodsOfTender": mot,
"subtotalFees": v.SubtotalFees,
}
}
func (m MOT) ToMap() map[string]interface{} {
return map[string]interface{}{
"fees": m.Fees,
"order": m.Order,
}
}
Then, turning your Data object into a map is as simple as:
dataMap := data.ToMap()

Answer 2

package main
import (
"encoding/json"
"fmt"
)
func main() {
s := `{
"totalAmount": 4,
"subtotal": 4,
"Vendors": [{
"MethodOfTenders": [
{
"order": 1,
"fees": 2
}, {
"order": 1,
"fees": 2
}
],
"subtotalFees": 4
},
{
"MethodOfTenders": [
{
"order": 1,
"fees": 2
}, {
"order": 1,
"fees": 1
}
],
"subtotalFees": 3
}
]
}
`
var data map[string]interface{}
if err := json.Unmarshal([]byte(s), &data); err != nil {
fmt.Println(err)
}
fmt.Printf("%+v\n", data)
}
Playground
When Unmarshal from json it is generating expected output which is required by go-xlsx-templater
Note: Created a fix for the issue stated in the question here raised PR. After the fix no conversion is required.
After applying the fix the remaining code with sampleData() function which is not included in the example
package main
import (
"encoding/json"
"fmt"
"log"
"os"
"reflect"
"strings"
xlst "github.com/ivahaev/go-xlsx-templater"
)
func main() {
wd, err := os.Getwd()
if err != nil {
log.Fatalf(err.Error())
}
path := strings.Join([]string{wd, ""}, "")
doc := xlst.New()
err = doc.ReadTemplate(path + "/export_support_template.xlsx")
if err != nil {
fmt.Println("ERROR OPENING THE TEMPLATE: ", err)
panic("error opening template")
}
var ctx map[string]interface{}
data := sampleData()
err := json.Unmarshal(data, &ctx)
if err != nil {
fmt.Println("Final log")
}
err = doc.Render(ctx)
if err != nil {
fmt.Println("ERROR RENDERING THE TEMPLATE: ", err)
panic("error rendering template")
}
err = doc.Save(path + "/report.xlsx")
if err != nil {
fmt.Println("ERROR SAVING THE TEMPLATE: ", err)
panic("error saving template")
}
}

Answer 3

The trick is performed using reflection. Go down recursively and check the kind of values:
// "Casts" map values to the desired type recursively
func castMap(m map[string]any) map[string]any {
for k := range m {
switch reflect.ValueOf(m[k]).Kind() {
case reflect.Map:
mm, ok := m[k].(map[string]any)
if !ok {
panic(fmt.Errorf("Expected map[string]any, got %T", m[k]))
}
m[k] = castMap(mm)
case reflect.Slice, reflect.Array:
ma, ok := m[k].([]any)
if !ok {
panic(fmt.Errorf("Expected []any, got %T", m[k]))
}
m[k] = castArray(ma)
default:
// fmt.Printf("%s: %T, kind %v\n", k, m[k], reflect.ValueOf(m[k]).Kind())
continue
}
}
return m
}
// "Casts" slice elements to the desired types recursively
func castArray(a []any) []map[string]any {
res := []map[string]any{}
for i := range a {
switch reflect.ValueOf(a[i]).Kind() {
case reflect.Map:
am, ok := a[i].(map[string]any)
if !ok {
panic(fmt.Errorf("Expected map[string]any, got %T", a[i]))
}
am = castMap(am)
res = append(res, am)
default:
panic(fmt.Errorf("Expected map[string]any, got %T", a[i]))
}
}
return res
}
Full example with main is here: https://go.dev/play/p/MEQRe-f3dY1
It's output:
before: map[string]interface {}{"Vendors":[]interface {}{map[string]interface {}{"MethodOfTenders":[]interface {}{map[string]interface {}{"fees":2, "order":1}, map[string]interface {}{"fees":2, "order":1}}, "subtotalFees":4}, map[string]interface {}{"MethodOfTenders":[]interface {}{map[string]interface {}{"fees":2, "order":1}, map[string]interface {}{"fees":1, "order":1}}, "subtotalFees":3}}, "subtotal":4, "totalAmount":4}
after: map[string]interface {}{"Vendors":[]map[string]interface {}{map[string]interface {}{"MethodOfTenders":[]map[string]interface {}{map[string]interface {}{"fees":2, "order":1}, map[string]interface {}{"fees":2, "order":1}}, "subtotalFees":4}, map[string]interface {}{"MethodOfTenders":[]map[string]interface {}{map[string]interface {}{"fees":2, "order":1}, map[string]interface {}{"fees":1, "order":1}}, "subtotalFees":3}}, "subtotal":4, "totalAmount":4}
See? "Vendors" was []interface {}, became []map[string]interface {}
"Vendors[].MethodOfTenders" were []interface {}, became []map[string]interface {}
The functions panic if they see something unexpected. Feel free to modify them to returning error if needed.
UPDATE
Here is the same recursive algoritm using type assertions.
// "Casts" map values to the desired type recursively
func castMap(m map[string]any) map[string]any {
for k := range m {
mm, ok := m[k].(map[string]any)
if ok {
m[k] = castMap(mm)
continue
}
ma, ok := m[k].([]any)
if ok {
m[k] = castArray(ma)
continue
}
}
return m
}
// "Casts" slice elements to the desired types recursively
func castArray(a []any) []map[string]any {
res := []map[string]any{}
for i := range a {
am, ok := a[i].(map[string]any)
if ok {
am = castMap(am)
res = append(res, am)
} else {
panic(fmt.Errorf("Expected map[string]any, got %T", a[i]))
}
}
return res
}
Full code https://go.dev/play/p/IbOhQqpisie
BENCHMARK
One of the commenters made a claim that reflection is expensive. This is not the case here:
goos: windows
goarch: amd64
pkg: example.org/try/test
cpu: Intel(R) Core(TM) i7-8550U CPU # 1.80GHz
BenchmarkCastReflect-8 382574 2926 ns/op 2664 B/op 29 allocs/op
BenchmarkCastType-8 418257 2934 ns/op 2664 B/op 29 allocs/op
The benchmark is here: https://go.dev/play/p/Ro8PeVQy8kA
Playground doesn't execute benchmarks. It should be run on a real CPU