Retrieve element from nested JSON string - json

This is the Go code that I have:
func main(){
s := string(`{"Id": "ABC123",
"Name": "Hello",
"RelatedItems":[
{"RId":"TEST123","RName":"TEST1","RChildren":"Ch1"},
{"RId":"TEST234","RName":"TEST2","RChildren":"Ch2"}]
}`)
var result map[string]interface{}
json.Unmarshal([]byte(s), &result)
fmt.Println("Id:", result["Id"])
Rlist := result["RelatedItems"].([]map[string]interface{})
for key, pist := range pist {
fmt.Println("Key: ", key)
fmt.Println("RID:", pist["RId"])
}
}
The struct is down below
type Model struct {
Id string `json:"Id"`
Name string `json:"ModelName"`
RelatedItems []RelatedItems `json:"RelatedItems"`
}
type RelatedItems struct {
RId string `json:"PCId"`
RName string `json:"PCName"`
RChildren string `json:"string"`
}
How would I get an output that would let me choose a particular field from the above?
eg:
Output
Id: ABC123
key:0
RID:TEST123
key:1
RID:TEST234
I am seeing this error
panic: interface conversion: interface {} is nil, not []map[string]interface {}

Based on the posted content,
I'm clear that you are facing issues retrieving data from the nested JSON string.
I've taken your piece of code and tried to compile and reproduce the issue.
After observing, I have a few suggestions based on the way the code has been written.
When the datatype present in the s is known to be similar to the type Model, the result could have been declared as type Model.
That results in var result Model instead of map[string]interface{}.
When the data that's gonna be decoded from the interface{} is not known, the usage of switch would come into rescue without crashing the code.
Something similar to:
switch dataType := result["RelatedItems"].(type){
case interface{}:
// Handle interface{}
case []map[string]interface{}:
// Handle []map[string]interface{}
default:
fmt.Println("Unexpected-Datatype", dataType)
// Handle Accordingly
When we try to Unmarshal, we make sure to look into the json tags that are provided for the fields of a structure. If the data encoded is not having the tags we provided, the data will not be decoded accordingly.
Hence, the result of decoding the data from s into result would result in {ABC123 [{ } { }]} as the tags of the fields Name, RId, RName, RChildren are given as ModelName, PCId, PCName, string respectively.
By the above suggestions and refining the tags given, the piece of code would be as following which would definitely retrieve data from nested JSON structures.
s := string(`{"Id": "ABC123",
"Name": "Hello",
"RelatedItems":[
{"RId":"TEST123","RName":"TEST1","RChildren":"Ch1"},
{"RId":"TEST234","RName":"TEST2","RChildren":"Ch2"}]
}`)
var result Model
json.Unmarshal([]byte(s), &result)
fmt.Println(result)
type Model struct {
Id string `json:"Id"`
Name string `json:"Name"`
RelatedItems []RelatedItems `json:"RelatedItems"`
}
type RelatedItems struct {
RId string `json:"RId"`
RName string `json:"RName"`
RChildren string `json:"RChildren"`
}
This results in the output: {ABC123 Hello [{TEST123 TEST1 Ch1} {TEST234 TEST2 Ch2}]}

Why would you unmarshal to a map anyway and go through type checks?
type Model struct {
Id string `json:"Id"`
Name string `json:"ModelName"`
RelatedItems []RelatedItems `json:"RelatedItems"`
}
type RelatedItems struct {
RId string `json:"PCId"`
RName string `json:"PCName"`
RChildren string `json:"string"`
}
s := `{"Id": "ABC123",
"Name": "Hello",
"RelatedItems":[
{"RId":"TEST123","RName":"TEST1","RChildren":"Ch1"},
{"RId":"TEST234","RName":"TEST2","RChildren":"Ch2"}]
}`
var result Model
if err := json.Unmarshal([]byte(s), &result); err != nil {
log.Fatal(err.Error())
}
fmt.Println("Id: ", result.Id)
for index, ri := range result.RelatedItems {
fmt.Printf("Key: %d\n", index)
fmt.Printf("RID: %s\n", ri.RId)
}

Related

How to set optional json in nest struct

I tried to set an optional json config in nest struct, when i need this json it will appear, otherwise it will not exist.
type Test struct {
Data NestTest `json:"data"`
}
type NestTest struct {
NestData1 string `json:"data1"`
NestData2 string `json:"data2,omitempty"`
}
test := Test{
Data: NestTest{
NestData1: "something",
},
}
b, err := json.Marshal(test)
fmt.Sprintf("the test struct json string is: %s", string(b))
output:
{"data":{"data1":"something","data2":""}}
expect:
{"data":{"data1":"something"}}
All fields are optional when unmarshalling (you won't get an error if a struct field doesn't have an associated value in the JSON). When marshaling, you can use omitempty to not output a field if it contains its type's zero value:
https://pkg.go.dev/encoding/json#Marshal
var Test struct {
Data string `json:"data,omitempty" validate:"option"`
}

How to use FieldByName to extract nested struct value

Here is the struct:
type UP struct {
Rxinfo []struct {
Gatewayid string `json:"gatewayID"`
Uplinkid string `json:"uplinkID"`
Name string `json:"name"`
Time time.Time `json:"time"`
Rssi int `json:"rssi"`
Lorasnr float64 `json:"loRaSNR"`
Location struct {
Latitude int `json:"latitude"`
Longitude int `json:"longitude"`
Altitude int `json:"altitude"`
} `json:"location"`
} `json:"rxInfo"`
Adr bool `json:"adr"`
Fcnt int `json:"fCnt"`
Data string `json:"data"`
}
When I code like this:
up := UP{}
if err := json.Unmarshal(msg.Payload(), &up); err != nil {
fmt.Printf("Message could not be parsed (%s): %s", msg.Payload(), err)
}
val := reflect.ValueOf(up).FieldByName("Name")
fmt.Printf("%v",val)
It returns <invalid reflect.Value>.
The reason why the logs return an <invalid reflect.Value> error is because the field Name is located inside a slice of structs. If you want to get the value of a field from a slice of struct, you have to specify which index of the slice you want to get value from.
For example, your payload looks like this:
{
"rx_info":[
{
"gateway_id":"1",
"up_link_id":"2",
"name":"sunstrider",
"time":"2021-05-25T19:37:00Z",
"rssi":-21,
"lo_ra_snr":0.342,
"location":{
"latitiude":413,
"longitude":124,
"altitude":41
}
},
{
"gateway_id":"2",
"up_link_id":"4",
"name":"sunstrider 2",
"time":"2021-06-25T19:37:00Z",
"rssi":-41,
"lo_ra_snr":0.562,
"location":{
"latitiude":213,
"longitude":321,
"altitude":443
}
}
],
"address":true,
"fcnt":53,
"data":"this is the data"
}
Notice how the field Name is inside an object that's inside an array. If your payload looks like this, then your code should look like this:
package main
import (
"encoding/json"
"fmt"
"reflect"
"time"
)
type UP struct {
RxInfo []RxInfo `json:"rx_info"`
Address bool `json:"address"`
FCNT int `json:"fcnt"`
Data string `json:"data"`
}
type RxInfo struct {
GatewayID string `json:"gateway_id"`
UplinkID string `json:"uplink_id"`
Name string `json:"name"`
Time time.Time `json:"time"`
RSSI int `json:"rssi"`
LoRaSNR float64 `json:"lo_ra_snr"`
Location Location `json:"location"`
}
type Location struct {
Latitude int `json:"latitude"`
Longitude int `json:"longitude"`
Altitude int `json:"altitude"`
}
func main() {
up := UP{}
if err := json.Unmarshal([]byte(payload), &up); err != nil {
fmt.Printf("Message could not be parsed (%s): %s", payload, err)
}
// because the "Name" field is inside an array, we have to
// bind the payload array to an array of a predefined struct
valSlice := reflect.ValueOf(up).FieldByName("RxInfo").Interface().([]RxInfo)
// getting the value from the zeroth index
fmt.Println(valSlice[0].Name)
// getting the value from the first index
fmt.Println(valSlice[1].Name)
// since your goal is to get the value of a field, I suggest you
// get the value using the structure of the struct using
// "dot" operator rather than of using the FieldByName()
valString := up.RxInfo[0].Name
fmt.Println(valString)
}
Also, a couple of tips to keep your code clean:
You should always access the value of the payload using the structure of your struct. Only use reflect.ValueOf(variable).FieldByName("name_of_field") when you don't know the complete structure of your payload.
If you have a deeply nested struct, split them up into multiple tiny structs (like my example above) to make it easier to read and maintain.
Standardize your naming convention. This applies when you're naming variables, structs, struct fields, struct tags, etc. I personally use camel-case for field names and snake-case for json struct tags.

Does json.Unmarshal require your result structure to match exactly the JSON passed in?

I have a JSON string I want to unmarshal:
{
"id":1720,
"alertId":1,
"alertName":"{stats} Test Lambda Alert",
"dashboardId":5,
"panelId":2,
"userId":0,
"newState":"alerting",
"prevState":"ok",
"time":1523983581000,
"text":"",
"regionId":0,
"tags":[],
"login":"",
"email":"",
"avatarUrl":"",
"data":{
"evalMatches":[
{
"metric":"{prod}{stats} Lambda Alert Test",
"tags":null,
"value":16.525333333333332
}
]
}
}
I get the raw stream via a request: bodyBytes, err := ioutil.ReadAll(resp.Body)
I was hoping I could just specify a struct that pulls the values I care about, e.g.,
type Result struct {
ID string `json:"id"`
Time int64 `json:"time"`
}
However, when I try this, I get errors.
type Result struct {
ID string `json:"id"`
Time string `json:"time"`
}
var result Result
err2 := json.Unmarshal(bodyBytes, &result)
if err2 != nil {
log.Fatal(fmt.Sprintf(`Error Unmarshalling: %s`, err2))
}
fmt.Println(result.ID)
Error Unmarshalling: json: cannot unmarshal array into Go value of type main.Result
I suspect this error may be due to what's actually returned from ioutil.ReadAll(), since it has the above JSON string wrapped in [ ] if I do a fmt.Println(string(bodyBytes)), but if I try to json.Unmarshal(bodyBytes[0], &result), I just get compile errors, so I'm not sure.
If I want to unmarshal a JSON string, do I have to specify the full structure in my type Result struct? Is there a way around this? I don't want to be bound to the JSON object I receive (if the API changes upstream, it requires us to modify our code to recognize that, etc.).
You can unmarshal into structs that represent only some fields of your JSON document, but the field types have to match, as the error clearly states:
cannot unmarshal number into Go struct field Result.id of type string
You cannot unmarshal a number into a string. If you define the ID field as any numeric type it'll work just fine:
package main
import (
"encoding/json"
"fmt"
)
var j = []byte(`
{
"id":1720,
"prevState":"ok",
"time":1523983581000,
"text":"",
"regionId":0
}
`)
type Result struct {
ID int `json:"id"` // or any other integer type, or float{32,64}, or json.Number
Time int64 `json:"time"`
}
func main() {
var r Result
err := json.Unmarshal(j, &r)
fmt.Println(r, err)
}
Try it on the playground: https://play.golang.org/p/lqsQwLW2dHZ
Update
You have just edited your question with the actual error you receive. You have to unmarshal JSON arrays into slices. So if the HTTP response in fact returns a JSON array, unmarshal into []Result:
var j = []byte(`
[
{
"id":1720,
"prevState":"ok",
"time":1523983581000,
"text":"",
"regionId":0
}
]
`)
var r []Result
err := json.Unmarshal(j, &r)
fmt.Println(r[0], err)
https://play.golang.org/p/EbOVA8CbcFO
To generate Go types that match your JSON document pretty well, use https://mholt.github.io/json-to-go/.

convert interface{} directly to int in Golang, where interface stores a number as string

I got a map[string]interface{} because decoding to JSON; with normal data, interface most be only a number but in type string, like this :
var a interface{}
a="3"
And then all data will be stored into a struct.
type someStruct struct {
ID string
Number1 int
Number2 int
Number3 int
Number4 int
}
So I need to convert interface to int, but can´t do it easily and efficiently because only the code would be https://play.golang.org/p/oktbvTUbk93, pretty annoying and code does not seem to be readable if you take in mind the fact that I should handle all of possible errors
I would want to convert it directly to int, I´ve been searching for solutions but any of this convertions works as I want https://play.golang.org/p/Dw67U6kZwHC
In case you wondering why I don´t just decode it into struct directly, it´s because it´s dynamic data, the actual decoding occurs like this :
type dataIn struct {
Code int `json:"code"`
ID string `json:"id"`
Data interface{} `json:"data"`
}
And then I handle Data according to code and id, and they all are different data structures, so I can´t directly handle it with JSON
Create a helper function which does parsing and validating for you, in one place:
func parseInt(i interface{}) (int, error) {
s, ok := i.(string)
if !ok {
return 0, errors.New("not string")
}
return strconv.Atoi(s)
}
And you can use this where needed. Here's a complete example, in which I also used another helper function, which takes care of the error handling:
m := map[string]interface{}{
"number1": "1",
"number2": "2",
"number3": "3",
"number4": "4",
"ID": "asdsa",
"Title": "asdas",
}
getInt := func(key string) int {
n, err := parseInt(m[key])
if err != nil {
panic(err) // Decide what you wanna do with error
}
return n
}
// converting to struct
data := element{
ID: m["ID"].(string),
Title: m["Title"].(string),
Number1: getInt("number1"),
Number2: getInt("number2"),
Number3: getInt("number3"),
Number4: getInt("number4"),
}
fmt.Printf("%+v\n", data)
Output of the above (try it on the Go Playground):
{ID:asdsa Title:asdas Number1:1 Number2:2 Number3:3 Number4:4}
Also note that the open source package github.com/icza/dyno should help you handle dynamic objects at ease. (Disclosure: I'm the author.) It has a dyno.GetInteger() function for example, which is capable of extracting an int64 value out of multiple types (such as integers, floats, strings etc.).
Sounds like what you need is a custom unmarshal json method on your main struct. First, unmarshal into your main struct to get your code and id, then use those in a switch statement to determine which struct to use for the rest of the data and unmarshal into that, stirring that struct in your Data field.
I still didn't get the part where you stated the struct is generated dynamically. Anyways, you can have a struct method attached, that does the int conversion. If the Data field, which is of type interface{} is always gonna hold integers, you an try this:
type DataIn struct {
Code int `json:"code"`
ID string `json:"id"`
Data interface{} `json:"data"`
}
func (s DataIn) toInt() int {
switch t := s.Data.(type)
case int:
i, _ := strings.Atoi(fmt.Sprintf("%v",s.Data))
return i
}
// using it
sampleData := someStruct{
Number1: datain.toInt(),
}

Golang json decoding fails on Array

I have an object like:
a = [{
"name": "rdj",
"place": "meh",
"meh" : ["bow", "blah"]
}]
I defined a struct like:
type first struct {
A []one
}
type one struct {
Place string `json: "place"`
Name string `json: "name"`
}
when I use the same in code like:
func main() {
res, _ := http.Get("http://127.0.0.1:8080/sample/")
defer res.Body.Close()
var some first
rd := json.NewDecoder(res.Body)
err := rd.Decode(&some)
errorme(err)
fmt.Printf("%+v\n", some)
}
I get the below error:
panic: json: cannot unmarshal array into Go value of type main.first
My understanding is:
type first defines the array and within that array is data structure defined in type one.
The JSON is an array of objects. Use these types:
type one struct { // Use struct for JSON object
Place string `json: "place"`
Name string `json: "name"`
}
...
var some []one // Use slice for JSON array
rd := json.NewDecoder(res.Body)
err := rd.Decode(&some)
The the types in the question match this JSON structure:
{"A": [{
"name": "rdj",
"place": "meh",
"meh" : ["bow", "blah"]
}]}
#rickydj,
If you need to have "first" as a separate type:
type first []one
If you do not care about having "first" as a separate type, just cut down to
var some []one
as #Mellow Marmot suggested above.