Enumerate slice entries - json

If I have this output
Stuff {
"Items": [
{
"title": "test1",
"id": "1",
},
{
"title": "test",
"id": "2",
},
],
"total": 19
},
}
But want this instead:
stuff {
"Items": [
1:{
"title": "test1",
"id": "1",
},
2:{
"title": "test",
"id": "2",
},
],
"total": 19
},
}
Currently, my structs are build like this:
Stuff struct {
Items []Items `json:"items"`
Total int `json:"total"`
} `json:"stuff"`
type Items struct {
Title string `json:"title"`
ID string `json:"id"`
}
I initiate a slice by:
stuff := make([]Items, 10) // Lets say I have 10 entries
And append by:
Stuff.Items = stuff
Stuff.Total = len(Stuff.Items)
Now I am unsure on how to numerate that. So for every item entries, there should be a number, starting from 1 - 10 (In this example)

Given your Stuff and Items type declarations, here's a simple data structure and its JSON dump:
s := Stuff{Items: []Items{Items{"test1", "1"}, Items{"test2", "2"}}, Total: 10}
j, err := json.MarshalIndent(s, "", " ")
if err != nil {
log.Fatal(err)
}
fmt.Println(string(j))
JSON:
{
"items": [
{
"title": "test1",
"id": "1"
},
{
"title": "test2",
"id": "2"
}
],
"total": 10
}
To see what you want instead, there's no magic json package call to do that. You'll have to create a new data structure that reflects the structure of your desired output.
In this case a simple map will do:
m := make(map[string]Items)
for _, item := range s.Items {
m[item.ID] = item
}
And now if you JSON-dump this map, you get:
{
"1": {
"title": "test1",
"id": "1"
},
"2": {
"title": "test2",
"id": "2"
}
}
Note that I'm not wrapping it with Stuff now, because Stuff has different fields. Go is statically typed, and each struct can only contain the fields you declared it to have.

Related

Golang iterate over nested struct

i want to iterate over a nested json struct in golang. The issue is, i do not exactly know how nested the structure will be, becuase there are multiple jsons. In this case for example the output should be:
"available": false
"type": "foo"
"name": "foo"
"street": "foo"
....
Is that possible?
{
"informations": {
"available": false,
"provide": {
"informations": {
"customer": {
"type": "foo",
"address": {
"name": "foo",
"street": "foo",
"zipcode": "123",
"city": "foo"
}
}
}
}
}
}
I think you can do this with a map[string]interface{}. Because, as you said that you don't know exactly the structure of the JSON. But, again, you need to know the JSON structure each time while extracting data. Also, you need to do type assertion for inner map[string]interface{}. You can look at the answer from here for type assertion.
Here's the example code that I wrote:
package main
import (
"encoding/json"
"fmt"
)
var j = `{
"informations": {
"available": false,
"provide": {
"informations": {
"customer": {
"type": "foo",
"address": {
"name": "foo",
"street": "foo",
"zipcode": "123",
"city": "foo"
}
}
}
}
}
}`
func main() {
var data map[string]interface{}
err := json.Unmarshal([]byte(j), &data)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println(data["informations"])
information := data["informations"]
provide := information.(map[string]interface{})["provide"]
fmt.Println(provide)
}
Output will be:
map[available:false provide:map[informations:map[customer:map[address:map[city:foo name:foo street:foo zipcode:123] type:foo]]]]
map[informations:map[customer:map[address:map[city:foo name:foo street:foo zipcode:123] type:foo]]]
Go Playground

Is there any way to unmarshal JSON into a map and preserve keys order

I need to unmarshal JSON data into the map by preserving the order of keys and values.
This is a JSON data example:
{
"31ded736-4076-4b1c-b38f-7e8d9fa78b41": {
"Name": "Requested",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f55", "GroupName": "First task" , "NumQuestion": "10" ,"Score":"10"},
{"ID": "17acf9a1-b2c7-46c6-b975-759b9d9f538d", "GroupName": "Test1" , "NumQuestion": "20" ,"Score":"5" }
]
},
"115f7d04-3075-408a-b8ce-c6e46fe6053f": {
"Name": "To do",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f56", "GroupName": "First task" , "NumQuestion": "5" ,"Score":"10"}
]
},
"9bcf1415-3a41-43b6-a871-8de1939a75c4": {
"Name": "In Progress",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f57", "GroupName": "Second task" , "NumQuestion": "10" ,"Score":"5"}
]
},
"2f6c1455-6cf9-4009-b86b-de0a0d2204a1": {
"Name": "Done",
"Items": [
{ "ID": "c7ac5b7f-59b0-45e3-82fb-b3b0afc05f58", "GroupName": "Third task" , "NumQuestion": "20" , "Score":"7"}
]
}
}
This is my code:
var GroupTestListUpdate = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
type Item struct {
ID string `json:"id"`
GroupName string `json:"groupName"`
NumQuestion string `json:"numQuestion"`
Score string `json:"score"`
}
type Input struct {
Name string `json:"name"`
Items []Item `array:"item"`
}
var objmap map[string]Input
reqBody, err := ioutil.ReadAll(r.Body)
if err != nil{
http.Error(w, "can't read body", http.StatusBadRequest)
return
}
json.Unmarshal(reqBody, &objmap)
fmt.Println(objmap)
})
Output:
map[115f7d04-3075-408a-b8ce-c6e46fe6053f:{To do [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f56 First task 5 10}]} 2f6c1455-6cf9-4009-b86b-de0a0d2204a1:{Done [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f58 Third task 20 7}]} 31ded736-4076-4b1c-b38f-7e8d9fa78b41:{Requested [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f55 First task 10 10} {17acf9a1-b2c7-46c6-b975-759b9d9f538d Test1 20 5}]} 9bcf1415-3a41-43b6-a871-8de1939a75c4:{In Progress [{c7ac5b7f-59b0-45e3-82fb-b3b0afc05f57 Second task 10 5}]}]
Question:
I want to know how to preserve the order of keys and values.
I got the same output every time I tried this code, So I wonder that the order in the map is not completely random?
Yes.
You can override the UnmarshalJSON method on a custom map type that will allow you to preserve the order.
As an example, here is a library that does just that: https://github.com/iancoleman/orderedmap.

Protobuf custom options not showing in JSON made by protojson library

I'm trying to extract Protobuf custom options from a FileDescriptorSet generated by the protoc compiler. I'm unable to do so using protoreflect. So, I tried to do so using the protojson library.
PS : Importing the Go-generated code is not an option for my use case.
Here's the Protobuf Message I'm testing with :
syntax = "proto3";
option go_package = "./protoze";
import "google/protobuf/descriptor.proto";
extend google.protobuf.FieldOptions {
string Meta = 50000;
}
extend google.protobuf.FileOptions {
string Food = 50001;
}
option (Food) = "cheese";
message X {
int64 num = 1;
}
message P {
string Fname = 1 [json_name = "FNAME"];
string Lname = 2 [json_name = "0123", (Meta) = "Yo"];
string Designation = 3;
repeated string Email = 4;
string UserID = 5;
string EmpID = 6;
repeated X z = 7;
}
// protoc --go_out=. filename.proto
Here's how far I got :
package main
import (
"fmt"
"io/ioutil"
"os/exec"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/types/descriptorpb"
)
func main() {
exec.Command("protoc", "-oBinaryFile", "1.proto").Run()
Fset := descriptorpb.FileDescriptorSet{}
byts, _ := ioutil.ReadFile("File")
proto.Unmarshal(byts, &Fset)
byts, _ = protojson.Marshal(Fset.File[0])
fmt.Println(string(byts))
}
And here's the output JSON
{
"name": "1.proto",
"dependency": [
"google/protobuf/descriptor.proto"
],
"messageType": [
{
"name": "X",
"field": [
{
"name": "num",
"number": 1,
"label": "LABEL_OPTIONAL",
"type": "TYPE_INT64",
"jsonName": "num"
}
]
},
{
"name": "P",
"field": [
{
"name": "Fname",
"number": 1,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "FNAME"
},
{
"name": "Lname",
"number": 2,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "0123",
"options": {}
},
{
"name": "Designation",
"number": 3,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "Designation"
},
{
"name": "Email",
"number": 4,
"label": "LABEL_REPEATED",
"type": "TYPE_STRING",
"jsonName": "Email"
},
{
"name": "UserID",
"number": 5,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "UserID"
},
{
"name": "EmpID",
"number": 6,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "EmpID"
},
{
"name": "z",
"number": 7,
"label": "LABEL_REPEATED",
"type": "TYPE_MESSAGE",
"typeName": ".X",
"jsonName": "z"
}
]
}
],
"extension": [
{
"name": "Meta",
"number": 50000,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"extendee": ".google.protobuf.FieldOptions",
"jsonName": "Meta"
},
{
"name": "Food",
"number": 50001,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"extendee": ".google.protobuf.FileOptions",
"jsonName": "Food"
}
],
"options": {
"goPackage": "./protoze"
},
"syntax": "proto3"
}
So, data about my custom options showed up in the extensions. But what I really wanted was the value of those Custom Options in the "options" as well. (Which in my case was (Food) = "Cheese" and I want Cheese)
Can someone tell me how I can extract my custom options from the FileDescriptorSet using Protoreflect or by using Protojson.
I tried a lot to try and extract it using Protoreflect but failed !
Although not specifically an answer to how to get the custom options in a generated JSON, I believe I have an answer to what sounds like your underlying question: how to access the custom options without loading the generated Go code. This is thanks to dsnet's answer to my question on the golang issues board. Needless to say all the credit for this tricky solution goes to him. The punchline is to Marshal and then Unmarshal the options using a runtime-populated protoregistry.Types that actually knows about the custom options.
I made a complete demonstration of this approach working in this repo, and the key section (all the guts of which come from dsnet's example) is here:
func main() {
protogen.Options{
}.Run(func(gen *protogen.Plugin) error {
gen.SupportedFeatures = uint64(pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL)
// The type information for all extensions is in the source files,
// so we need to extract them into a dynamically created protoregistry.Types.
extTypes := new(protoregistry.Types)
for _, file := range gen.Files {
if err := registerAllExtensions(extTypes, file.Desc); err != nil {
panic(err)
}
}
// run through the files again, extracting and printing the Message options
for _, sourceFile := range gen.Files {
if !sourceFile.Generate {
continue
}
// setup output file
outputfile := gen.NewGeneratedFile("./out.txt", sourceFile.GoImportPath)
for _, message := range sourceFile.Messages {
outputfile.P(fmt.Sprintf("\nMessage %s:", message.Desc.Name()))
// The MessageOptions as provided by protoc does not know about
// dynamically created extensions, so they are left as unknown fields.
// We round-trip marshal and unmarshal the options with
// a dynamically created resolver that does know about extensions at runtime.
options := message.Desc.Options().(*descriptorpb.MessageOptions)
b, err := proto.Marshal(options)
if err != nil {
panic(err)
}
options.Reset()
err = proto.UnmarshalOptions{Resolver: extTypes}.Unmarshal(b, options)
if err != nil {
panic(err)
}
// Use protobuf reflection to iterate over all the extension fields,
// looking for the ones that we are interested in.
options.ProtoReflect().Range(func(fd protoreflect.FieldDescriptor, v protoreflect.Value) bool {
if !fd.IsExtension() {
return true
}
outputfile.P(fmt.Sprintf("Value of option %s is %s",fd.Name(), v.String()))
// Make use of fd and v based on their reflective properties.
return true
})
}
}
return nil
})
}
// Recursively register all extensions into the provided protoregistry.Types,
// starting with the protoreflect.FileDescriptor and recursing into its MessageDescriptors,
// their nested MessageDescriptors, and so on.
//
// This leverages the fact that both protoreflect.FileDescriptor and protoreflect.MessageDescriptor
// have identical Messages() and Extensions() functions in order to recurse through a single function
func registerAllExtensions(extTypes *protoregistry.Types, descs interface {
Messages() protoreflect.MessageDescriptors
Extensions() protoreflect.ExtensionDescriptors
}) error {
mds := descs.Messages()
for i := 0; i < mds.Len(); i++ {
registerAllExtensions(extTypes, mds.Get(i))
}
xds := descs.Extensions()
for i := 0; i < xds.Len(); i++ {
if err := extTypes.RegisterExtension(dynamicpb.NewExtensionType(xds.Get(i))); err != nil {
return err
}
}
return nil
}

One of the elements of the array as an object

The question is how to get the "from" element? The rest is not a problem
I know that in https://github.com/json-iterator/ it can be done, but I could not figure out how it works there
Json:
{
"ab": 123456789,
"cd": [
[
4,
1234,
123456,
1000000001,
1234567890,
"text",
{
"from": "123456"
}
],
[
4,
4321,
654321,
1000000001,
9876543210,
"text",
{
"from": "654321"
}
]
]
}
Golang:
type test struct {
Ab int `json:"ab"`
Cd [][]interface{} `json:"cd"`
}
var testvar test
json.Unmarshal(Data, &testvar)
testvar.Cd[0][6]["from"].(string)
Error:
invalid operation: testvar.Cd[0][6]["from"] (type interface {} does not support indexing)
It's simple: it is a map[string]interface{} hence
m, ok := testvar.Cd[0][6].(map[string]interface{})
fmt.Println(m, ok, m["from"])
https://play.golang.org/p/XJGs_HsxMfZ

Create struct for complex JSON array in Golang

I have the following JSON array that I'm trying to convert to a struct.
[
{
"titel": "test 1",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}, {
"titel": "test 2",
"event": "some value",
"pair": "some value",
"condition": [
"or",
[
"contains",
"url",
"/"
]
],
"actions": [
[
"option1",
"12",
"1"
],
[
"option2",
"3",
"1"
]
]
}
]
This is the struct I've got so far:
type Trigger struct {
Event string `json:"event"`
Pair string `json:"pair"`
Actions [][]string `json:"actions"`
Condition []interface{} `json:"condition"`
}
type Triggers struct {
Collection []Trigger
}
However, this does not really cover the "Condition" part. Ideally id like to have a structure for that as well.
Assuming that there can be only a single condition per item in the root array, you can try a struct below. This could make using Condition clear.
https://play.golang.org/p/WxFhBjJmEN
type Trigger struct {
Event string `json:"event"`
Pair string `json:"pair"`
Actions [][]string `json:"actions"`
Condition Condition `json:"condition"`
}
type Condition []interface{}
func (c *Condition) Typ() string {
return (*c)[0].(string)
}
func (c *Condition) Val() []string {
xs := (*c)[1].([]interface{})
ys := make([]string, len(xs))
for i, x := range xs {
ys[i] = x.(string)
}
return ys
}
type Triggers struct {
Collection []Trigger
}