I'm sending requests to a JSON API, and it either returns an error...
{
"error": {
"code": 404,
"message": "Document not found.",
"status": "NOT_FOUND"
}
}
or the data.
{
"name": "projectname",
"fields": {
"userId": {
"stringValue": "erw9384rjidfge"
}
},
"createTime": "2018-06-28T00:52:25.638791Z",
"updateTime": "2018-06-28T00:52:25.638791Z"
}
Here are the corresponding structs
type HttpError struct {
Code int `json:"code"`
Message string `json:"message"`
Status string `json:"status"`
}
type Document struct {
Name string `json:"name"`
Fields struct {
UserID struct {
StringValue string `json:"stringValue"`
} `json:"userId"`
} `json:"fields"`
CreateTime time.Time `json:"createTime"`
UpdateTime time.Time `json:"updateTime"`
}
Once I get the response, how do I cleanly/concisely unmarshal to the correct struct? I've seen a lot of ugly solutions (maybe Go's fault instead of the writers).
func getDocument() {
resp, _ := httpClient.Get("example.com")
defer resp.Body.Close()
bodyBytes, _ := ioutil.ReadAll(resp.Body)
var data map[string]interface{}
// How to unmarshal to either HttpError or Document??
err = json.Unmarshal([]byte(bodyBytes), &data)
}
By the way, I can't use the Go Firestore client library because reasons.
You can use an struct type inside your unmarshal method; with pointers to establish what's been unmarshalled.
Note: This code assumes there is no overlap of top level json keys... error / name / fields / etc.
type outer struct {
*HttpError `json:"error"`
*Document
}
var out outer
if err := json.Unmarshal(bodyBytes, &out); err != nil {
// error handling
}
if out.HttpErr != nil {
// handle error json case
}
// Here you can use out.Document, probably worth check if it is nil first.
Runnable example
Related
I'm trying to unmarshal the following JSON string
token = `{
"id": 1,
"token": {
"id": 2248637,
"metadata": {
"name": "Name #1",
"formats": "[{\"mimeType\": \"model/gltf-binary\", \"uri\": \"uri1\", {\"mimeType\": \"image/gif\", \"uri\": \"uri2\"}]"
}
}`
I can unmarshal it with 2 phases like this. However, I would like to use custom unmarshalJSON but I fail. I got error
My code as follow:
type FileFormat struct {
MIMEType string
URI string
}
func (format *FileFormat) UnmarshalJSON(data []byte) error {
var aux []interface{}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
format.MIMEType = aux[0].(string)
format.URI = aux[1].(string)
return nil
}
type TokenMetadata struct {
Name string `json:"name"`
Formats []FileFormat `json:"formats"`
}
type Token struct {
ID TokenID `json:"tokenId"`
Metadata TokenMetadata `json:"metadata"`
}
func main() {
var tokenRes OwnedToken
if err := json.Unmarshal([]byte(token), &tokenRes); err != nil {
fmt.Println(err)
}
}
And the error is
json: cannot unmarshal string into Go struct field TokenMetadata.token.metadata.formats of type []main.FileFormat
How can I fix this problem? Many thanks!
The JSON array of file formats is double encoded. Declare a Go type corresponding to the array. Double decode in the UnmarshalJSON method for that type.
type FileFormats []FileFormat
func (ff *FileFormats) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
return json.Unmarshal(
[]byte(s),
(*[]FileFormat)(ff))
}
type TokenMetadata struct {
Name string `json:"name"`
Formats FileFormats `json:"formats"`
}
Note: The conversion from from *FileFormats to *[]FileFormat is required to prevent recursion.
I'm new to Golang.
golang version: 1.17.8
validator: "github.com/go-playground/validator/v10"
I want to validate an incoming JSON payload after loaded into nested struct data structure.
Here's my incoming JSON payload,
{
"name": "Yomiko",
"address": {
"city": "Tokyo",
"street": "Shibaura St"
},
"children":[
{
"lastName": "Takayashi"
}
],
"isEmployed": false
}
Here's my user.go file,
package main
type User struct {
Name string
Address *Address `validate:"required"`
Children []*Child
IsEmployed *bool `validate:"required"`
}
type Address struct {
City string `validate:"required"`
Street string `validate:"required"`
}
type Child struct {
Title string `validate:"required"`
FirstName string
LastName string `validate:"required"`
}
Here's my test function,
func TestUserPayload(t *testing.T) {
actualUserPayload := NewUserPayloadFromFile("userpayload.json")
validate := validator.New()
err := validate.Struct(actualUserPayload)
if err != nil {
t.Error("Validation Error: ", err)
}
}
This test passes. However, I expected it to fail as Child.Title is marked as required. I expected the following error,
Validation Error: Key: 'Child.Title' Error:Field validation for 'Title' failed on the 'required' tag
However, when I loop through the children slice and validate each child struct as follows the test fails as expected,
func TestUserPayload(t *testing.T) {
actualUserPayload := NewUserPayloadFromFile("userpayload.json")
validate := validator.New()
err := validate.Struct(actualUserPayload)
if err != nil {
t.Error("Validation Error: ", err)
}
children := actualUserPayload.Children
for _, child := range children {
err := validate.Struct(child)
if err != nil {
t.Error("Validation Error: ", err)
}
}
}
Is there a straightforward way to do this validation of the items in a slice of structs?
According to the documentation of the validator package, you can use dive in your struct tag to get this behavior. This causes the validator to also validate the nested struct/slice/etc.
So you would need to update your User struct to this:
type User struct {
Name string
Address *Address `validate:"required"`
Children []*Child `validate:"dive"`
IsEmployed *bool `validate:"required"`
}
Here it is working in Go Playground
I'm working on a Go wrapper for an API and I noticed that two of the JSON fields stay empty when they don't have any data.
Basically the API returns a set of information on a given url, and if it was visited at least once, everything is okay and I get a full json that I then Unmarshal into a struct:
{
"stats":{
"status":1,
"date":"09.07.2019",
"title":"Test",
"devices":{
"dev":[
{
"tag":"Desktop"
}
],
"sys":[
{
"tag":"GNU/Linux "
},
{
"tag":"Windows 10"
}
],
"bro":[
{
"tag":"Firefox 67.0"
},
{
"tag":"Chrome 62.0"
}
]
},
"refs":[
{
"link":"www.google.com"
}
]
}
}
This is the struct I'm using:
type Stats struct {
Stats struct {
Status int `json:"status"`
Date string `json:"date"`
Title string `json:"title"`
Devices struct {
Dev []struct {
Tag string `json:"tag"`
} `json:"dev"`
Sys []struct {
Tag string `json:"tag"`
} `json:"sys"`
Bro []struct {
Tag string `json:"tag"`
} `json:"bro"`
} `json:"devices"`
Refs []struct {
Link string `json:"link"`
} `json:"refs"`
} `json:"stats"`
}
When a new url is given, then things become a little bit weird:
{
"stats": {
"status": 1,
"date": "09.07.2019",
"title": "Test2",
"devices": [
],
"refs": [
]
}
}
As you can see, the fields "dev", "sys" and "bro" just disappear because they're not used and when I try to Unmarshal the JSON into the same struct I get json: cannot unmarshal array into Go struct field Stats.device of type [...]
I tried to use two different structs to handle both the responses but I'm sure that there's a way to handle them gracefully with just one.
Any help would be appreciated, thanks!
I finally managed to make it work with an ugly workaround.
I changed my struct to
type Stats struct {
Status int `json:"status"`
Date string `json:"date"`
Title string `json:"title"`
Devices interface{} `json:"devices"`
Refs interface{} `json:"refs"`
}
Then I can finally Unmarshal the JSON in both cases, but I get a map[string]interface{} when an object is passed and an empty interface{} when an empty array is passed. In order to fix this inconsistency, I simply check for the data type and force the use of a JSON intermediate conversion in order to unpack the map[string]interface{} value inside a custom Devices struct:
// Devices contains devices information
type Devices struct {
Dev []struct {
Tag string `json:"tag"`
Clicks string `json:"clicks"`
} `json:"dev"`
Sys []struct {
Tag string `json:"tag"`
Clicks string `json:"clicks"`
} `json:"sys"`
Bro []struct {
Tag string `json:"tag"`
Clicks string `json:"clicks"`
} `json:"bro"`
}
The algorithms I use are the following:
//ForceDevicesToRightType uses a json conversion as intermediary for filling the Stats.Devices
// struct with map[string]interface{} values
func ForceDevicesToRightType(dev interface{}) (Devices, error) {
temp, err := json.Marshal(dev)
if err != nil {
return Devices{}, err
}
// Use a temporary variable of the right type
var devices Devices
err = json.Unmarshal(temp, &devices)
if err != nil {
return Devices{}, err
}
return devices, nil
}
// ForceRefsToRightType uses a json conversion as intermediary for filling the Stats.Refs
// struct with map[string]interface{} values
func ForceRefsToRightType(refs interface{}) (Refs, error) {
temp, err := json.Marshal(refs)
if err != nil {
return Refs{}, err
}
// Use a temporary variable of the right type
var references Refs
err = json.Unmarshal(temp, &references)
if err != nil {
return Refs{}, err
}
return references, nil
}
Since the compiler knows that both Devices and Refs fields are interface{} I cannot simply access any methods after the conversion, so I simply make a cast of the right type and everything works fine.
For example, if I wanted to access the Dev sub-struct, this is the proper way:
y, _ := GetStats()
fmt.Println(y.Devices.(Devices).Dev)
It's ugly, but it works.
Thank you very much for your help, I hope that this method will save you an headache!
I want to make a validation api in order to validate a set of json requests regarding specific set of rules. To do that I want to use just one endpoint and call functions that correspond to the specific json struct. I know that there is no method overloading in go so I am kind of stumped.
...
type requestBodyA struct {
SomeField string `json:"someField"`
SomeOtherField string `json:"someOtherField"`
}
type requestBodyB struct {
SomeDifferentField string `json:"someDifferentField"`
SomeOtherDifferentField string `json:"someOtherDifferentField"`
}
type ValidationService interface {
ValidateRequest(ctx context.Context, s string) (err error)
}
type basicValidationService struct{}
...
So in order to validate lots of different json requests, is it better to create structs for each and every json request? Or should I create these dynamically? How can I know what kind of request is sent if I only have one endpoint?
If you have a single endpoint/rpc that has to accept different JSON types, you'll need to tell it how to distinguish between them, somehow. One option is to have something like:
type request struct {
bodyA *requestBodyA
bodyB *requestBodyB
}
Then, populate these fields in a container JSON object appropriately. The json module will only populate bodyA if a bodyA key is present, otherwise leaving it a nil, and so on.
Here's a more complete example:
type RequestBodyFoo struct {
Name string
Balance float64
}
type RequestBodyBar struct {
Id int
Ref int
}
type Request struct {
Foo *RequestBodyFoo
Bar *RequestBodyBar
}
func (r *Request) Show() {
if r.Foo != nil {
fmt.Println("Request has Foo:", *r.Foo)
}
if r.Bar != nil {
fmt.Println("Request has Bar:", *r.Bar)
}
}
func main() {
bb := []byte(`
{
"Foo": {"Name": "joe", "balance": 4591.25}
}
`)
var req Request
if err := json.Unmarshal(bb, &req); err != nil {
panic(err)
}
req.Show()
var req2 Request
bb = []byte(`
{
"Bar": {"Id": 128992, "Ref": 801472}
}
`)
if err := json.Unmarshal(bb, &req2); err != nil {
panic(err)
}
req2.Show()
}
Another option is to do it more dynamically with maps, but it's likely that the method above will be sufficient.
I'm calling a REST API from my Go program which takes n number of hotel ids in the request and returns their data as a JSON. The response is like the following when say I pass 2 ids in the request, 1018089108070373346 and 2017089208070373346 :
{
"data": {
"1018089108070373346": {
"name": "A Nice Hotel",
"success": true
},
"2017089208070373346": {
"name": "Another Nice Hotel",
"success": true
}
}
}
Since I'm new to Golang I using a JSON Go tool available at http://mholt.github.io/json-to-go/ to get the struct representation for the above response. What I get is:
type Autogenerated struct {
Data struct {
Num1017089108070373346 struct {
Name string `json:"name"`
Success bool `json:"success"`
} `json:"1017089108070373346"`
Num2017089208070373346 struct {
Name string `json:"name"`
Success bool `json:"success"`
} `json:"2017089208070373346"`
} `json:"data"`
}
I cannot use the above struct because the actual id values and the number of ids I pass can be different each time, the JSON returned will have different keys. How can this situation be mapped to a struct ?
Thanks
Use a map:
type Item struct {
Name string `json:"name"`
Success bool `json:"success"`
}
type Response struct {
Data map[string]Item `json:"data"`
}
Run it on the playground
Here is some sample code that utilizes Mellow Marmots answer and shows how to iterate over the items in the response.
test.json
{
"data": {
"1018089108070373346": {
"name": "A Nice Hotel",
"success": true
},
"2017089208070373346": {
"name": "Another Nice Hotel",
"success": true
}
}
}
test.go
package main
import (
"encoding/json"
"fmt"
"os"
)
// Item struct
type Item struct {
Name string `json:"name"`
Success bool `json:"success"`
}
// Response struct
type Response struct {
Data map[string]Item `json:"data"`
}
func main() {
jsonFile, err := os.Open("test.json")
if err != nil {
fmt.Println("Error opening test file\n", err.Error())
return
}
jsonParser := json.NewDecoder(jsonFile)
var filedata Response
if err = jsonParser.Decode(&filedata); err != nil {
fmt.Println("Error while reading test file.\n", err.Error())
return
}
for key, value := range filedata.Data {
fmt.Println(key, value.Name, value.Success)
}
}
Which outputs:
1018089108070373346 A Nice Hotel true
2017089208070373346 Another Nice Hotel true