I want to decode a request JSON that the value of a field could be a single string or array (list).
I know how to parse if separately. but I'm looking for a way to have a dynamic decoder to parse both of them.
The value field in the JSON is the case I'm talking about
Sample for single-value:
{
"filter":{
"op":"IN",
"field":"name",
"value": "testDuplicate"
}
}
Sample for multi-value:
{
"filter":{
"op":"IN",
"field":"name",
"value":["testDuplicate","tt"]
}
}
Structs for single value:
type DocumentTypeSearchRequest struct {
Filter DocTypeFilter `json:"filter"`
}
type DocTypeFilter struct {
Op string `json:"op"`
Field string `json:"field"`
Value string `json:"value"`
}
Structs for multi-value:
type DocumentTypeSearchRequest struct {
Filter DocTypeFilter `json:"filter"`
}
type DocTypeFilter struct {
Op string `json:"op"`
Field string `json:"field"`
Value []string `json:"value"`
}
One solution is to try to decode with one of them if failed use another one, but I'm not sure if that is the proper way to handle this problem.
You need a custom unmarshaler. I've talked about a similar situation in this video:
type Value []string
func (v *Value) UnmarshalJSON(p []byte) error {
if p[0] == '[' { // First char is '[', so it's a JSON array
s := make([]string, 0)
err := json.Unmarshal(p, &s)
*v = Value(s)
return err
}
// else it's a simple string
*v = make(Value, 1)
return json.Unmarshal(p, &(*v)[0])
}
Playground link
With this definition of Value, your struct would become something like:
type DocTypeFilter struct {
Op string `json:"op"`
Field string `json:"field"`
Value Value `json:"value"`
}
and now your Value field will always be a slice, but it may contain only a single value when your JSON input is a string, and not an array.
Related
I tried to set an optional json config in nest struct, when i need this json it will appear, otherwise it will not exist.
type Test struct {
Data NestTest `json:"data"`
}
type NestTest struct {
NestData1 string `json:"data1"`
NestData2 string `json:"data2,omitempty"`
}
test := Test{
Data: NestTest{
NestData1: "something",
},
}
b, err := json.Marshal(test)
fmt.Sprintf("the test struct json string is: %s", string(b))
output:
{"data":{"data1":"something","data2":""}}
expect:
{"data":{"data1":"something"}}
All fields are optional when unmarshalling (you won't get an error if a struct field doesn't have an associated value in the JSON). When marshaling, you can use omitempty to not output a field if it contains its type's zero value:
https://pkg.go.dev/encoding/json#Marshal
var Test struct {
Data string `json:"data,omitempty" validate:"option"`
}
Here is the struct:
type UP struct {
Rxinfo []struct {
Gatewayid string `json:"gatewayID"`
Uplinkid string `json:"uplinkID"`
Name string `json:"name"`
Time time.Time `json:"time"`
Rssi int `json:"rssi"`
Lorasnr float64 `json:"loRaSNR"`
Location struct {
Latitude int `json:"latitude"`
Longitude int `json:"longitude"`
Altitude int `json:"altitude"`
} `json:"location"`
} `json:"rxInfo"`
Adr bool `json:"adr"`
Fcnt int `json:"fCnt"`
Data string `json:"data"`
}
When I code like this:
up := UP{}
if err := json.Unmarshal(msg.Payload(), &up); err != nil {
fmt.Printf("Message could not be parsed (%s): %s", msg.Payload(), err)
}
val := reflect.ValueOf(up).FieldByName("Name")
fmt.Printf("%v",val)
It returns <invalid reflect.Value>.
The reason why the logs return an <invalid reflect.Value> error is because the field Name is located inside a slice of structs. If you want to get the value of a field from a slice of struct, you have to specify which index of the slice you want to get value from.
For example, your payload looks like this:
{
"rx_info":[
{
"gateway_id":"1",
"up_link_id":"2",
"name":"sunstrider",
"time":"2021-05-25T19:37:00Z",
"rssi":-21,
"lo_ra_snr":0.342,
"location":{
"latitiude":413,
"longitude":124,
"altitude":41
}
},
{
"gateway_id":"2",
"up_link_id":"4",
"name":"sunstrider 2",
"time":"2021-06-25T19:37:00Z",
"rssi":-41,
"lo_ra_snr":0.562,
"location":{
"latitiude":213,
"longitude":321,
"altitude":443
}
}
],
"address":true,
"fcnt":53,
"data":"this is the data"
}
Notice how the field Name is inside an object that's inside an array. If your payload looks like this, then your code should look like this:
package main
import (
"encoding/json"
"fmt"
"reflect"
"time"
)
type UP struct {
RxInfo []RxInfo `json:"rx_info"`
Address bool `json:"address"`
FCNT int `json:"fcnt"`
Data string `json:"data"`
}
type RxInfo struct {
GatewayID string `json:"gateway_id"`
UplinkID string `json:"uplink_id"`
Name string `json:"name"`
Time time.Time `json:"time"`
RSSI int `json:"rssi"`
LoRaSNR float64 `json:"lo_ra_snr"`
Location Location `json:"location"`
}
type Location struct {
Latitude int `json:"latitude"`
Longitude int `json:"longitude"`
Altitude int `json:"altitude"`
}
func main() {
up := UP{}
if err := json.Unmarshal([]byte(payload), &up); err != nil {
fmt.Printf("Message could not be parsed (%s): %s", payload, err)
}
// because the "Name" field is inside an array, we have to
// bind the payload array to an array of a predefined struct
valSlice := reflect.ValueOf(up).FieldByName("RxInfo").Interface().([]RxInfo)
// getting the value from the zeroth index
fmt.Println(valSlice[0].Name)
// getting the value from the first index
fmt.Println(valSlice[1].Name)
// since your goal is to get the value of a field, I suggest you
// get the value using the structure of the struct using
// "dot" operator rather than of using the FieldByName()
valString := up.RxInfo[0].Name
fmt.Println(valString)
}
Also, a couple of tips to keep your code clean:
You should always access the value of the payload using the structure of your struct. Only use reflect.ValueOf(variable).FieldByName("name_of_field") when you don't know the complete structure of your payload.
If you have a deeply nested struct, split them up into multiple tiny structs (like my example above) to make it easier to read and maintain.
Standardize your naming convention. This applies when you're naming variables, structs, struct fields, struct tags, etc. I personally use camel-case for field names and snake-case for json struct tags.
This is the Go code that I have:
func main(){
s := string(`{"Id": "ABC123",
"Name": "Hello",
"RelatedItems":[
{"RId":"TEST123","RName":"TEST1","RChildren":"Ch1"},
{"RId":"TEST234","RName":"TEST2","RChildren":"Ch2"}]
}`)
var result map[string]interface{}
json.Unmarshal([]byte(s), &result)
fmt.Println("Id:", result["Id"])
Rlist := result["RelatedItems"].([]map[string]interface{})
for key, pist := range pist {
fmt.Println("Key: ", key)
fmt.Println("RID:", pist["RId"])
}
}
The struct is down below
type Model struct {
Id string `json:"Id"`
Name string `json:"ModelName"`
RelatedItems []RelatedItems `json:"RelatedItems"`
}
type RelatedItems struct {
RId string `json:"PCId"`
RName string `json:"PCName"`
RChildren string `json:"string"`
}
How would I get an output that would let me choose a particular field from the above?
eg:
Output
Id: ABC123
key:0
RID:TEST123
key:1
RID:TEST234
I am seeing this error
panic: interface conversion: interface {} is nil, not []map[string]interface {}
Based on the posted content,
I'm clear that you are facing issues retrieving data from the nested JSON string.
I've taken your piece of code and tried to compile and reproduce the issue.
After observing, I have a few suggestions based on the way the code has been written.
When the datatype present in the s is known to be similar to the type Model, the result could have been declared as type Model.
That results in var result Model instead of map[string]interface{}.
When the data that's gonna be decoded from the interface{} is not known, the usage of switch would come into rescue without crashing the code.
Something similar to:
switch dataType := result["RelatedItems"].(type){
case interface{}:
// Handle interface{}
case []map[string]interface{}:
// Handle []map[string]interface{}
default:
fmt.Println("Unexpected-Datatype", dataType)
// Handle Accordingly
When we try to Unmarshal, we make sure to look into the json tags that are provided for the fields of a structure. If the data encoded is not having the tags we provided, the data will not be decoded accordingly.
Hence, the result of decoding the data from s into result would result in {ABC123 [{ } { }]} as the tags of the fields Name, RId, RName, RChildren are given as ModelName, PCId, PCName, string respectively.
By the above suggestions and refining the tags given, the piece of code would be as following which would definitely retrieve data from nested JSON structures.
s := string(`{"Id": "ABC123",
"Name": "Hello",
"RelatedItems":[
{"RId":"TEST123","RName":"TEST1","RChildren":"Ch1"},
{"RId":"TEST234","RName":"TEST2","RChildren":"Ch2"}]
}`)
var result Model
json.Unmarshal([]byte(s), &result)
fmt.Println(result)
type Model struct {
Id string `json:"Id"`
Name string `json:"Name"`
RelatedItems []RelatedItems `json:"RelatedItems"`
}
type RelatedItems struct {
RId string `json:"RId"`
RName string `json:"RName"`
RChildren string `json:"RChildren"`
}
This results in the output: {ABC123 Hello [{TEST123 TEST1 Ch1} {TEST234 TEST2 Ch2}]}
Why would you unmarshal to a map anyway and go through type checks?
type Model struct {
Id string `json:"Id"`
Name string `json:"ModelName"`
RelatedItems []RelatedItems `json:"RelatedItems"`
}
type RelatedItems struct {
RId string `json:"PCId"`
RName string `json:"PCName"`
RChildren string `json:"string"`
}
s := `{"Id": "ABC123",
"Name": "Hello",
"RelatedItems":[
{"RId":"TEST123","RName":"TEST1","RChildren":"Ch1"},
{"RId":"TEST234","RName":"TEST2","RChildren":"Ch2"}]
}`
var result Model
if err := json.Unmarshal([]byte(s), &result); err != nil {
log.Fatal(err.Error())
}
fmt.Println("Id: ", result.Id)
for index, ri := range result.RelatedItems {
fmt.Printf("Key: %d\n", index)
fmt.Printf("RID: %s\n", ri.RId)
}
My golang program have this structure of structs:
type JSONDoc struct {
Count int `json:"count"`
Objects []uintptr `json:"objects"`
}
type ObjectA struct {
FieldA string
}
type ObjectB struct {
FieldB string
}
I don't know what object types can be in JSONDoc.Objects, i'm need to store multiple structs in json array. Reflect returns pointers to structs, i'm appending them to struct, but encoding/json package in result json replace pointer with integer address.
Also unsafe.Pointer cannot be parsed by encoding/json too.
Just want result json to look as
{
"count":2,
"objects":
[
{"FieldA":"..."},
{"FieldB":"..."}
]
}
How can i store pointers to structs that will be correctly encoded to json?
You could use interface{}, for example :
type JSONDoc struct {
Count int `json:"count"`
Objects []interface{} `json:"objects"`
}
func main() {
doc := JSONDoc{Count: 2}
doc.Objects = append(doc.Objects, &ObjectA{"A"}, &ObjectB{"B"})
b, err := json.MarshalIndent(&doc, "", "\t")
fmt.Println(string(b), err)
}
playground
I have struct
type tySurvey struct {
Id int64 `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
I do json.Marshal write JSON bytes in HTML page. jQuery modifies name field in object and encodes object using jQueries JSON.stringify and jQuery posts string to Go handler.
id field encoded as string.
Sent: {"id":1} Received: {"id":"1"}
Problem is that json.Unmarshal fails to unmarshal that JSON because id is not integer anymore.
json: cannot unmarshal string into Go value of type int64
What is best way to handle such data? I do not wish to manually convert every field. I wish to write compact, bug free code.
Quotes is not too bad. JavaScript does not work well with int64.
I would like to learn the easy way to unmarshal json with string values in int64 values.
This is handled by adding ,string to your tag as follows:
type tySurvey struct {
Id int64 `json:"id,string,omitempty"`
Name string `json:"name,omitempty"`
}
This can be found about halfway through the documentation for Marshal.
Please note that you cannot decode the empty string by specifying omitempty as it is only used when encoding.
use json.Number
type tySurvey struct {
Id json.Number `json:"id,omitempty"`
Name string `json:"name,omitempty"`
}
You could also create a type alias for int or int64 and create a custom json unmarshaler
Sample code:
Reference
// StringInt create a type alias for type int
type StringInt int
// UnmarshalJSON create a custom unmarshal for the StringInt
/// this helps us check the type of our value before unmarshalling it
func (st *StringInt) UnmarshalJSON(b []byte) error {
//convert the bytes into an interface
//this will help us check the type of our value
//if it is a string that can be converted into a int we convert it
///otherwise we return an error
var item interface{}
if err := json.Unmarshal(b, &item); err != nil {
return err
}
switch v := item.(type) {
case int:
*st = StringInt(v)
case float64:
*st = StringInt(int(v))
case string:
///here convert the string into
///an integer
i, err := strconv.Atoi(v)
if err != nil {
///the string might not be of integer type
///so return an error
return err
}
*st = StringInt(i)
}
return nil
}
func main() {
type Item struct {
Name string `json:"name"`
ItemId StringInt `json:"item_id"`
}
jsonData := []byte(`{"name":"item 1","item_id":"30"}`)
var item Item
err := json.Unmarshal(jsonData, &item)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%+v\n", item)
}