looping with nested []struct in Go? - json

I have a structure I'm working with, and I'm not sure how to loop through it properly. I would like to access the field names, but all it is doing is just incrementally counting at each loop.
Here is my structure:
type ImgurJson struct {
Status int16 `json:"status"`
Success bool `json:"success"`
Data []struct {
Width int16 `json:"width"`
Points int32 `json:"points"`
CommentCount int32 `json:"comment_count"`
TopicId int32 `json:"topic_id"`
AccountId int32 `json:"account_id"`
Ups int32 `json:"ups"`
Downs int32 `json:"downs"`
Bandwidth int64 `json:"bandwidth"`
Datetime int64 `json:"datetime"`
Score int64 `json:"score"`
Account_Url string `json:"account_url"`
Topic string `json:"topic"`
Link string `json:"link"`
Id string `json:"id"`
Description string`json:"description"`
CommentPreview string `json:"comment_preview"`
Vote string `json:"vote"`
Title string `json:"title"`
Section string `json:"section"`
Favorite bool `json:"favorite"`
Is_Album bool `json:"is_album"`
Nsfw bool `json:"nsfw"`
} `json:"data"`
}
Here is my function:
func parseJson(file string) {
jsonFile, err := ioutil.ReadFile(file)
if err != nil {
...
}
jsonParser := ImgurJson{}
err = json.Unmarshal(jsonFile, &jsonParser)
for field, value := range jsonParser.Data {
fmt.Print("key: ", field, "\n")
fmt.Print("value: ", value, "\n")
}
}
How do I loop through a nested, []struct in Go and return the fields? I've seen several posts about reflection, but I don't understand if that would assist me or not. I can return the values of each field, but I don't understand how to map the field name to the key value.
Edit:
Renamed "keys" to "field", sorry! Didn't realise they were called fields.
I would like to be able to print:
field: Width
value: 1234
I would like to learn how to do this so I can later call a specific field by name so I can map it to a SQL column name.

This code based off an example here should do it for you; http://blog.golang.org/laws-of-reflection
import "reflect"
for _, value := range jsonParser.Data {
s := reflect.ValueOf(&value).Elem()
typeOfT := s.Type()
for i := 0; i < s.NumField(); i++ {
f := s.Field(i)
fmt.Print("key: ", typeOfT.Field(i).Name, "\n")
fmt.Print("value: ", f.Interface(), "\n")
}
}
Note that in your original code the loop is iterating items in the slice called Data. Each of those things an object of that anonymous struct type. You're not dealing with the fields at that point, from there, you can leverage the reflect package to print the names and values of fields in the struct. You can't just range over a struct natively, the operation isn't defined.

This is an alternative approach that we discussed in the comments:
Keep in mind that while this is faster than reflection, it's still better and more efficient to use the struct fields directly and make it a pointer (Data []*struct{....}).
type ImgurJson struct {
Status int16 `json:"status"`
Success bool `json:"success"`
Data []map[string]interface{} `json:"data"`
}
//.....
for i, d := range ij.Data {
fmt.Println(i, ":")
for k, v := range d {
fmt.Printf("\t%s: ", k)
switch v := v.(type) {
case float64:
fmt.Printf("%v (number)\n", v)
case string:
fmt.Printf("%v (str)\n", v)
case bool:
fmt.Printf("%v (bool)\n", v)
default:
fmt.Printf("%v (%T)\n", v, v)
}
}
}
playground

You can also iterate using normal golang for loop on nested struct.
type ImgurJson struct {
Status int16 `json:"status"`
Success bool `json:"success"`
Data []struct {
Width int16 `json:"width"`
Points int32 `json:"points"`
CommentCount int32 `json:"comment_count"`
TopicId int32 `json:"topic_id"`
AccountId int32 `json:"account_id"`
Ups int32 `json:"ups"`
Downs int32 `json:"downs"`
Bandwidth int64 `json:"bandwidth"`
Datetime int64 `json:"datetime"`
Score int64 `json:"score"`
Account_Url string `json:"account_url"`
Topic string `json:"topic"`
Link string `json:"link"`
Id string `json:"id"`
Description string`json:"description"`
CommentPreview string `json:"comment_preview"`
Vote string `json:"vote"`
Title string `json:"title"`
Section string `json:"section"`
Favorite bool `json:"favorite"`
Is_Album bool `json:"is_album"`
Nsfw bool `json:"nsfw"`
} `json:"data"`
}
func parseJson(file string) {
jsonFile, err := ioutil.ReadFile(file)
if err != nil {
...
}
jsonParser := ImgurJson{}
err = json.Unmarshal(jsonFile, &jsonParser)
for i :=0; i<len(jsonParser.Data); i++ {
fmt.Print("key: ", jsonParser.Data[i].TopicId, "\n")
}
}

Related

How to filter JSON data based off an empty field?

I'm calling an API and trying to parse the body response to filter out data. I've created a conditional for-loop to go through each object and check whether this field: Relationships.CurrentConfigurationVersion.Data is empty.
Here is my first attempt:
func (s *Server) getEmptyWorkSpaces(w http.ResponseWriter, r *http.Request) {
// omitted API calls for brevity
// body holds the JSON response
body, err := ioutil.ReadAll(resp.Body)
// label my struct type as data
var data WorkspacesJSON
err = json.Unmarshal(body, data)
if err != nil {
panic(err)
}
var data2 []*WorkspacesJSON
for _, v := range data.Data {
if v.Relationships.CurrentConfigurationVersion.Data == " " {
data2 = append(data2, v)
}
}
}
The error occurs in the conditional statement because I'm comparing a struct type to a string.
cannot convert "" (untyped string constant) to struct{ID string "json:"id""; Type string "json:"type""}
Attempt 2
After some searching around I tried another attempt from information I learned:
func (s *Server) getEmptyWorkSpaces(w http.ResponseWriter, r *http.Request) {
// omitted API calls for brevity
// body holds the JSON response
body, err := ioutil.ReadAll(resp.Body)
// label my struct type as data, this time attach []*
var data []*WorkspacesJSON
// added & in this new attempt
err = json.Unmarshal(body, &data)
if err != nil {
panic(err)
}
var data2 []*WorkspacesJSON
for _, v := range data.Data {
if v.Relationships.CurrentConfigurationVersion.Data == " " {
data2 = append(data2, v)
}
}
}
The compiler throw another error but this time targeting v.Relationships.CurrentConfigurationVersion.Data saying:
v.Relationships undefined (type *WorkspacesJSON has no field or method Relationships)compilerMissingFieldOrMethod
I'm not sure why I'm getting this error because in my first attempt this was not a problem? Clearly I am not understanding this.
Here is my type struct:
It's fairly long but the only field of important is Data.Relationships.CurrentConfigurationVersion.Data
type WorkspacesJSON struct {
Data []struct {
ID string `json:"id"`
Type string `json:"type"`
Attributes struct {
AllowDestroyPlan bool `json:"allow-destroy-plan"`
AutoApply bool `json:"auto-apply"`
AutoDestroyAt interface{} `json:"auto-destroy-at"`
CreatedAt time.Time `json:"created-at"`
Environment string `json:"environment"`
Locked bool `json:"locked"`
Name string `json:"name"`
QueueAllRuns bool `json:"queue-all-runs"`
SpeculativeEnabled bool `json:"speculative-enabled"`
StructuredRunOutputEnabled bool `json:"structured-run-output-enabled"`
TerraformVersion string `json:"terraform-version"`
WorkingDirectory string `json:"working-directory"`
GlobalRemoteState bool `json:"global-remote-state"`
UpdatedAt time.Time `json:"updated-at"`
ResourceCount int `json:"resource-count"`
ApplyDurationAverage int `json:"apply-duration-average"`
PlanDurationAverage int `json:"plan-duration-average"`
PolicyCheckFailures int `json:"policy-check-failures"`
RunFailures int `json:"run-failures"`
WorkspaceKpisRunsCount int `json:"workspace-kpis-runs-count"`
LatestChangeAt time.Time `json:"latest-change-at"`
Operations bool `json:"operations"`
ExecutionMode string `json:"execution-mode"`
VcsRepo struct {
Branch string `json:"branch"`
IngressSubmodules bool `json:"ingress-submodules"`
Identifier string `json:"identifier"`
DisplayIdentifier string `json:"display-identifier"`
OauthTokenID string `json:"oauth-token-id"`
WebhookURL string `json:"webhook-url"`
RepositoryHTTPURL string `json:"repository-http-url"`
ServiceProvider string `json:"service-provider"`
} `json:"vcs-repo"`
VcsRepoIdentifier string `json:"vcs-repo-identifier"`
Permissions struct {
CanUpdate bool `json:"can-update"`
CanDestroy bool `json:"can-destroy"`
CanQueueDestroy bool `json:"can-queue-destroy"`
CanQueueRun bool `json:"can-queue-run"`
CanQueueApply bool `json:"can-queue-apply"`
CanReadStateVersions bool `json:"can-read-state-versions"`
CanCreateStateVersions bool `json:"can-create-state-versions"`
CanReadVariable bool `json:"can-read-variable"`
CanUpdateVariable bool `json:"can-update-variable"`
CanLock bool `json:"can-lock"`
CanUnlock bool `json:"can-unlock"`
CanForceUnlock bool `json:"can-force-unlock"`
CanReadSettings bool `json:"can-read-settings"`
CanManageTags bool `json:"can-manage-tags"`
} `json:"permissions"`
Actions struct {
IsDestroyable bool `json:"is-destroyable"`
} `json:"actions"`
Description interface{} `json:"description"`
FileTriggersEnabled bool `json:"file-triggers-enabled"`
TriggerPrefixes []interface{} `json:"trigger-prefixes"`
Source string `json:"source"`
SourceName interface{} `json:"source-name"`
SourceURL interface{} `json:"source-url"`
TagNames []interface{} `json:"tag-names"`
} `json:"attributes"`
Relationships struct {
Organization struct {
Data struct {
ID string `json:"id"`
Type string `json:"type"`
} `json:"data"`
} `json:"organization"`
CurrentRun struct {
Data struct {
ID string `json:"id"`
Type string `json:"type"`
} `json:"data"`
Links struct {
Related string `json:"related"`
} `json:"links"`
} `json:"current-run"`
LatestRun struct {
Data struct {
ID string `json:"id"`
Type string `json:"type"`
} `json:"data"`
Links struct {
Related string `json:"related"`
} `json:"links"`
} `json:"latest-run"`
Outputs struct {
Data []interface{} `json:"data"`
} `json:"outputs"`
RemoteStateConsumers struct {
Links struct {
Related string `json:"related"`
} `json:"links"`
} `json:"remote-state-consumers"`
CurrentStateVersion struct {
Data struct {
ID string `json:"id"`
Type string `json:"type"`
} `json:"data"`
Links struct {
Related string `json:"related"`
} `json:"links"`
} `json:"current-state-version"`
CurrentConfigurationVersion struct {
Data struct {
ID string `json:"id"`
Type string `json:"type"`
} `json:"data"`
Links struct {
Related string `json:"related"`
} `json:"links"`
} `json:"current-configuration-version"`
AgentPool struct {
Data interface{} `json:"data"`
} `json:"agent-pool"`
Readme struct {
Data struct {
ID string `json:"id"`
Type string `json:"type"`
} `json:"data"`
} `json:"readme"`
} `json:"relationships"`
Links struct {
Self string `json:"self"`
} `json:"links"`
} `json:"data"`
Links struct {
Self string `json:"self"`
First string `json:"first"`
Prev interface{} `json:"prev"`
Next string `json:"next"`
Last string `json:"last"`
} `json:"links"`
Meta struct {
StatusCounts struct {
Pending int `json:"pending"`
PlanQueued int `json:"plan-queued"`
Planning int `json:"planning"`
Planned int `json:"planned"`
Confirmed int `json:"confirmed"`
ApplyQueued int `json:"apply-queued"`
Applying int `json:"applying"`
Applied int `json:"applied"`
Discarded int `json:"discarded"`
Errored int `json:"errored"`
Canceled int `json:"canceled"`
CostEstimating int `json:"cost-estimating"`
CostEstimated int `json:"cost-estimated"`
PolicyChecking int `json:"policy-checking"`
PolicyOverride int `json:"policy-override"`
PolicyChecked int `json:"policy-checked"`
PolicySoftFailed int `json:"policy-soft-failed"`
PlannedAndFinished int `json:"planned-and-finished"`
PostPlanRunning int `json:"post-plan-running"`
PostPlanCompleted int `json:"post-plan-completed"`
PreApplyRunning int `json:"pre-apply-running"`
PreApplyCompleted int `json:"pre-apply-completed"`
Fetching int `json:"fetching"`
None int `json:"none"`
Total int `json:"total"`
} `json:"status-counts"`
Pagination struct {
CurrentPage int `json:"current-page"`
PageSize int `json:"page-size"`
PrevPage interface{} `json:"prev-page"`
NextPage int `json:"next-page"`
TotalPages int `json:"total-pages"`
TotalCount int `json:"total-count"`
} `json:"pagination"`
} `json:"meta"`
}
I'm stuck in an eternal loop of those two same errors above, and trying to create hacky functions to get the job done, but no luck.
I'm very new to Go, I have experience with Python and working with JSON in Py is much easier for me but I want to try this using Go. Thanks in advance for your guidance.
How can I filter my JSON body using a conditional to check if a field is empty?
EDIT:
In my second attempt, How would I be able to select the field I'm looking for, Relationships.CurrentConfigurationVersion.Data? Once I figure that out I think I will be okay. Currently its saying that WorkspaceJSON has no field of method forRelationships. This was not the case for my first attempt.
There are lots of ways to check is struct empty or not, which is discussed here: How to check for an empty struct?
Also the append part of the code, must have the same type; as the following code:
data2 := WorkspacesJSON{}
for _, v := range data.Data {
if fmt.Sprintf("%v", v.Relationships.CurrentConfigurationVersion.Data) == "{ }" {
data2.Data = append(data2.Data, v)
}
}

Can´t unmarshal json to an exported struct from other package

I wan´t to unmarshal json to an exported struct from other package but I does not work propertly
package anyPackage
type DataStruct struct{
Size int `json:"size"`
Material string `json:"material"`
Date time.Time
}
package main
import (
"fmt"
"log"
"encoding/json"
"customPackage/anyPackage"
)
type NewStruct struct{
Name string `json:"name"`
Code int `json:"code"`
ExtraData anyPackage.DataStruct
}
func main(){
blob := `{ "name":"John", "code":12546, "material":"wood","size":456 }`
var aux NewStruct
if err := json.Unmarshal([]byte(blob), &aux); err != nil {
log.Fatal(err)
}
fmt.Printf("%+v", aux)
}
In that case, name and code are correctly unmarshal, but material and size don´t, they are empty
You should be able to fix the issue by embedding the DataStruct instead of having a separate field.
type NewStruct struct{
Name string `json:"name"`
Code int `json:"code"`
anyPackage.DataStruct
}
The fact that the type is in another package is irrelevant to unmarshalling JSON. The problem is that your data structure doesn't match the JSON. Your structure is effectively:
struct {
Name string `json:"name"`
Code int `json:"code"`
ExtraData struct {
Size int `json:"size"`
Material string `json:"material"`
Date time.Time
}
}
Which would equate to JSON like:
{
"name":"John",
"code":12546,
"ExtraData": {
"material":"wood",
"size":456
}
}
But that is not your JSON structure. Either your data structure or your JSON needs to be modified such that they match.

Unmarshal json request body to a structure which has a struct member of custom interface type

Lets consider the following code
type A struct {
Column1 string `json:"column1"`
Entity CustomInterface `json:"entity"`
}
type CustomInterface interface {
GetType() string
}
type Entity1 struct {
ColumnX string `json:"columnx"`
ColumnY string `json:"columny"`
}
type Entity2 struct {
ColumnP string `json:"columnp"`
ColumnQ string `json:"columnq"`
}
func (*e Entity1) GetType() string {
return "ENTITY1"
}
func (*e Entity2) GetType() string {
return "ENTITY2"
}
Now if I am trying to bind an instance of A type as follows
var bodyJSON A
ShouldBindWith(&bodyJson, binding.JSON)
I am getting the following error
json: cannot unmarshal object into Go struct field A.entity of type package.CustomInterface
Am I doing anything very silly here?
PS: I just started exploring go. Apologies if this question is very noob level.
The json.Unmarshal function by itself does not let you unmarshal into interface types except for the empty interfaces (interface{}) that don't have any methods:
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
However, in some simple cases the following scheme can work.
type CustomerEntity struct {
CustomerName string `json:"customer_name"`
Address string `json:"customer_address"`
}
type EmployeeEntity struct {
EmployeeName string `json:"employee_name"`
ID int `json:"employee_id"`
}
If we know that an entity is either an employee or a customer, then we can define an Entity that embeds each:
type Entity struct {
CustomerEntity
EmployeeEntity
}
We can give it methods to check whether it's a customer or an employee:
func (s Entity) IsCustomer() bool {
return s.CustomerEntity != CustomerEntity{}
}
func (s Entity) IsEmployee() bool {
return s.EmployeeEntity != EmployeeEntity{}
}
Really, these are just checking that at least one field is set.
Then we unmarshal the following JSON:
{
"entity": {
"employee_name": "Bob",
"employee_id": 77
}
}
Here's a complete example:
import (
"encoding/json"
"fmt"
)
type Example struct {
Entity Entity `json:"entity"`
}
type Entity struct {
CustomerEntity
EmployeeEntity
}
func (s Entity) IsCustomer() bool {
return s.CustomerEntity != CustomerEntity{}
}
func (s Entity) IsEmployee() bool {
return s.EmployeeEntity != EmployeeEntity{}
}
type CustomerEntity struct {
CustomerName string `json:"customer_name"`
CustomerAddress string `json:"customer_address"`
}
type EmployeeEntity struct {
EmployeeName string `json:"employee_name"`
EmployeeID int `json:"employee_id"`
}
func main() {
var example Example
if err := json.Unmarshal([]byte(`{"entity":{"employee_name":"Bob", "employee_id":77}}`), &example); err != nil {
panic("won't fail")
}
fmt.Printf("%#v\n", example)
if example.Entity.IsCustomer() {
fmt.Printf("customer %s lives at %d\n", example.Entity.CustomerName, example.Entity.CustomerAddress)
}
if example.Entity.IsEmployee() {
fmt.Printf("employee %s has id %d\n", example.Entity.EmployeeName, example.Entity.EmployeeID)
}
}
which outputs
main.Example{Entity:main.Entity{CustomerEntity:main.CustomerEntity{CustomerName:"", CustomerAddress:""}, EmployeeEntity:main.EmployeeEntity{EmployeeName:"Bob", EmployeeID:77}}}
employee Bob has id 77
as we might expect.
There are a few caveats. First, this won't work if there's overlap in either the JSON or the Go field names for the entity types. Second, nothing stops you from (accidentally) initializing some fields in both the customer and employee types and causing it to return true for both IsCustomer and IsEmployee.
If your JSON data has a "type" field, then you could use it to decide what is held instead:
type Entity struct {
Type string `json:"type"`
CustomerEntity
EmployeeEntity
}
although this has the same drawbacks as the other solution mentioned above.

Golang JSON Marshal/Unmarshal on struct containing byte arrays [duplicate]

type TestObject struct {
kind string `json:"kind"`
id string `json:"id, omitempty"`
name string `json:"name"`
email string `json:"email"`
}
func TestCreateSingleItemResponse(t *testing.T) {
testObject := new(TestObject)
testObject.kind = "TestObject"
testObject.id = "f73h5jf8"
testObject.name = "Yuri Gagarin"
testObject.email = "Yuri.Gagarin#Vostok.com"
fmt.Println(testObject)
b, err := json.Marshal(testObject)
if err != nil {
fmt.Println(err)
}
fmt.Println(string(b[:]))
}
Here is the output:
[ `go test -test.run="^TestCreateSingleItemResponse$"` | done: 2.195666095s ]
{TestObject f73h5jf8 Yuri Gagarin Yuri.Gagarin#Vostok.com}
{}
PASS
Why is the JSON essentially empty?
You need to export the fields in TestObject by capitalizing the first letter in the field name. Change kind to Kind and so on.
type TestObject struct {
Kind string `json:"kind"`
Id string `json:"id,omitempty"`
Name string `json:"name"`
Email string `json:"email"`
}
The encoding/json package and similar packages ignore unexported fields.
The `json:"..."` strings that follow the field declarations are struct tags. The tags in this struct set the names of the struct's fields when marshaling to and from JSON.
Ru it on the playground.
When the first letter is capitalised, the identifier is public to any
piece of code that you want to use.
When the first letter is lowercase, the identifier is private and
could only be accessed within the package it was declared.
Examples
var aName // private
var BigBro // public (exported)
var 123abc // illegal
func (p *Person) SetEmail(email string) { // public because SetEmail() function starts with upper case
p.email = email
}
func (p Person) email() string { // private because email() function starts with lower case
return p.email
}
In golang
in struct first letter must uppercase
ex. phonenumber -> PhoneNumber
======= Add detail
First, I'm try coding like this
type Questions struct {
id string
questionDesc string
questionID string
ans string
choices struct {
choice1 string
choice2 string
choice3 string
choice4 string
}
}
golang compile is not error and not show warning. But response is empty because something
After that, I search google found this article
Struct Types and Struct Type Literals
Article then... I try edit code.
//Questions map field name like database
type Questions struct {
ID string
QuestionDesc string
QuestionID string
Ans string
Choices struct {
Choice1 string
Choice2 string
Choice3 string
Choice4 string
}
}
Is work.
Hope for help.

Using []struct with Json

I'm trying to parse JSON to a []struct, the JSON is retrieved from https://api.github.com/events
However when I try to access each struct within the array I get an error:
type GITHUB_EVENT does not support indexing
How am I able to access each struct within the array?
func httpGetEvents() {
eventDataRAW := httpPageGet("https://api.github.com/events", true)
eventDataJSON := new(GITHUB_EVENT)
_ = json.Unmarshal([]byte(eventDataRAW), &eventDataJSON)
fmt.Println(eventDataJSON[0].Id)
}
//--------------------------------------------------------------------------------------//
type GITHUB_EVENT []struct {
Id string `json:"id"`
Type string `json:"type"`
Actor struct {
Id int `json:"id"`
Login string `json:"login"`
GravatarId string `json:"gravatar_id"`
Url string `json:"url"`
AvatarUrl string `json:"avatar_url"`
} `json:"actor"`
Repo struct {
Id int `json:"id"`
Name string `json:"name"`
Url string `json:"url"`
} `json:"repo"`
Payload struct {
PushId int `json:"push_id"`
Size int `json:"size"`
DistinctSize int `json:"distinct_size"`
Ref string `json:"ref"`
Head string `json:"head"`
Before string `json:"before"`
Commits []struct {
Sha string `json:"sha"`
Author struct {
Email string `json:"email"`
Name string `json:"name"`
} `json:"author"`
Message string `json:"message"`
Distinct bool `json:"distinct"`
Url string `json:"url"`
} `json:"commits"`
} `json:"payload"`
Public bool `json:"public"`
CreatedAt string `json:"created_at"`
}
This statement:
eventDataJSON := new(GITHUB_EVENT)
declares eventDataJSON as *GITHUB_EVENT (a pointer to a slice), and initializes it as a nil pointer. After unmarshaling, you could access the first event by explicitly deref-ing the pointer before indexing:
(*eventDataJSON)[0].Id
However, the more conventional approach is to use make:
eventDataJSON := make(GITHUB_EVENT, 0)
which declares eventDataJSON as a GITHUB_EVENT, and initializes it as an empty slice. (Remember that make is for special built-in types such as slices, maps, and channels).
You could pass a pointer to this slice to json.Unmarshal:
_ = json.Unmarshal([]byte(eventDataRAW), &eventDataJSON)
...and then index the slice directly:
fmt.Println(eventDataJSON[0].Id)
Additionally, json.Marshal will take care of allocating the output value, so you could declare eventDataJSON and skip any explicit initialization:
var eventDataJSON GITHUB_EVENT
Example: http://play.golang.org/p/zaELDgnpB2