GoLang JSON payload preparation with multiple data - json

I want to create JSON payload in given below format. I want a code or pattern that prepares the given format.
{
transactiontype: 'DDDDD'
emailType: 'QQQQQQ'
template: {
templateUrl: 'xry.kk'
templateName: 'chanda'
}
date: [
{
UserId: 1
Name: chadnan
},
{
UserId: 2
Name: kkkkkk
}
]
}

Hope this helps :
type Template struct {
TemplateURL string `json:"templateUrl" param:"templateUrl"`
TemplateName string `json:"templateName" param:"templateName"`
}
type Date struct {
UserId string `json:"UserId" param:"UserId"`
Name string `json:"Name" param:"Name"`
}
type NameAny struct {
*Template
TransactionType string `json:"transactiontype" param:"transactiontype"`
EmailType string `json:"emailType" param:"emailType"`
Data []Date `json:"date" param:"date"`
}
Data, _ := json.Marshal(NameAny)
Json(c, string(Data))(w, r)

You can use an online tool to convert a json into a valid Go struct: https://mholt.github.io/json-to-go/
Given your JSON, the Go struct is:
type AutoGenerated struct {
Transactiontype string `json:"transactiontype"`
EmailType string `json:"emailType"`
Template struct {
TemplateURL string `json:"templateUrl"`
TemplateName string `json:"templateName"`
} `json:"template"`
Date []struct {
UserID int `json:"UserId"`
Name string `json:"Name"`
} `json:"date"`
}
After the conversion, use the json.Marshal (Go Struct to JSON) and json.Unmarshal (JSON to Go Struct)
Complete example with your data: https://play.golang.org/p/RJuGK4cY1u-

// Transaction is a struct which stores the transaction details
type Transaction struct {
TransactionType string `json:"transaction_type"`
EmailType string `json:"email_type"`
Template Template `json:"template"`
Date []Date `json:"date"`
}
//Template is a struct which stores the template details
type Template struct {
TemplateURL string `json:"template_url"`
TemplateName string `json:"template_name"`
}
// Date is a struct which stores the user details
type Date struct {
UserID int `json:"user_id"`
Name string `json:"name"`
}
Above given structs are the correct data structure for storing your json body you can use json decoder for perfectly storing the data into struct
func exampleHandler(w http.ResponseWriter, r *http.Request) {
var trans Transaction
decoder := json.NewDecoder(r.Body)
err := decoder.Decode(&trans)
if err != nil {
log.Println(err)
}
}

Related

error: json: unsupported type: func() time.Time in Golang

i am new in golang, just try some API in Echo Framework and got some error.
My Models :
package models
import (
"net/http"
"quotes/db"
)
type Quote struct {
Id int `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
}
func GetAll() (Response, error) {
var quotes Quote
var res Response
ctx := db.Init()
ctx.Find(&quotes)
res.Status = http.StatusOK
res.Message = "Success"
res.Data = ctx
return res, nil
}
My Schema table
package schema
type Quotes struct {
Id int `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
}
My Response type for Api
package models
type Response struct {
Status int `json:"status"`
Message string `json:"message"`
Data interface{} `json:"data"`
}
i tried to add this in Models and Schema :
CreatedAt time.Time `gorm:"type:timestamp" json:"created_at,string,omitempty"`
UpdatedAt time.Time `gorm:"type:timestamp" json:"updated_at,string,omitempty"`
DeletedAt time.Time `gorm:"type:timestamp" json:"deleted_at,string,omitempty"`
And Still Not Works, any solutions?
I expect the api work with no errors
When using gorm, you need to embed a gorm.Model struct, which includes fields ID, CreatedAt, UpdatedAt, DeletedAt.
Reference
// gorm.Model definition
type Model struct {
ID uint `gorm:"primaryKey"`
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt gorm.DeletedAt `gorm:"index"`
}
Not familiar with echo but read below to understand how you use gorm.
In your case you can try doing the following:
package schema
type Quote struct {
gorm.Model
Title string `json:"title"`
Description string `json:"description"`
}
Then to get all the quotes:
func GetAll() (Response, error) {
var quotes []schema.Quote // slice
ctx := db.Init()
// Assuming
// ctx, err := gorm.Open(....)
// https://gorm.io/docs/query.html
result := db.Find(&quotes)
if result.Error != nil {
return Response{
Status: http.StatusInternalServerError,
Message: "Query failed",
},result.Error
}
if result.RowsAffected == 0 {
return Response{
Status: http.StatusNotFound,
Message: "No records found",
},nil
}
return Response{
Status: http.StatusOK,
Message: "Success",
Data: quotes,
},nil
}
Keep in mind that the Data field has type interface{}, which means it can hold a value of any type. If the value wasn't a slice you would be using the & operator you take the address of the Quote value. A slice is already a pointer to underlying slice so need to use the & operator.
If you want to access the slice of Quote values from the Data field, you will need to use a type assertion to convert the value from the interface{} type to the []Quote type. Here's an example of how you could do this:
// Assume that response.Data holds a slice of Quote values
quotes, ok := response.Data.([]Quote)
if !ok {
// Handle the case where response.Data is not a slice of Quote
}
Warning: Since you are returning a slice, then any changes to the returned slice will be modifying the initial slice too. If you wanted to avoid this then copy the slice values to a new slice:
quotesCopy = make([]schema.Quote, len(quotes))
copy(quotesCopy, quotes)

Server does't get all the JSON from remote REST API

My Go server does not retrieve all the JSON data from a remote API
I even tried creating a custom http.Client to make the request...but still it won't retrieve ALL the JSON data,and even tried extending the respective timeouts
Here's my code:
var netTransport = &http.Transport{
Dial: (&net.Dialer{
Timeout: 50 * time.Second,
}).Dial,
TLSHandshakeTimeout: 50 * time.Second,
}
var netClient = &http.Client{
Timeout: time.Second * 40,
Transport: netTransport,
}
res, getErr := netClient.Get(url)
if getErr != nil {
log.Fatal(getErr)
}
data := JSONData{}
if err := json.NewDecoder(res.Body).Decode(&data); err != nil {
log.Fatal(err)
}
data.Username = user
fmt.Println(data)
JSONData is defined as follows:
type Owner struct {
Login string
}
// Item is the single repository data structure
type Item struct {
ID int
Name string
FullName string `json:"full_name"`
Owner Owner
Description string
CreatedAt string `json:"created_at"`
}
// JSONData contains the GitHub API response
type JSONData struct {
Count int `json:"total_count"`
Username string
Items []Item
}
The JSONData needs to define JSON tags.
type Owner struct {
Login string `json:"login"`
}
// Item is the single repository data structure
type Item struct {
ID int `json:"id"`
Name string `json:"name"`
FullName string `json:"full_name"`
Owner Owner `json:"owner"`
Description string `json:"description"`
CreatedAt string `json:"created_at"`
}
// JSONData contains the GitHub API response
type JSONData struct {
Count int `json:"total_count"`
Username string `json:"username"`
Items []Item `json:"items"`
}
If no tag is specified, the name is taken as-is:
type Owner struct {
Login string
}
that would be marshaled (and correspondingly unmarshalled) into similar:
{"Login": "some login"}

How to navigate a nested json with unknown structure in Go?

I am trying to parse and get selected data from a deep nested json data in Go Lang. I'm having issues navigating through the structure and accessing the data. The data is too deep and complex to be parsed with a-priori known structures in Go.
Here is the URL of the file:
-https://www.data.gouv.fr/api/1/datasets/?format=csv&page=0&page_size=20
I did some parsing with map interfaces and using a json string:
resultdata := map[string]interface {}
json.Unmarshal([]byte(inputbytestring), &resultdata) //Inputstring is the string containing the JSON data of the above URL
The problem:
How can turn resultdata into a map (of strings), so I can use methods available for maps?
The JSON data is nested and has several levels. How is it possible to access the lower level JSON fields? is it possible to unmarshal the data recursively?
Once you have data as a map[string]interface{}, you can use type assertions to get to the lower levels of data.
There's a good explanation here of how to do this at https://blog.golang.org/json-and-go
Here's an example to get you most of the way:
https://play.golang.org/p/P8cGP1mTDmD
package main
import (
"encoding/json"
"fmt"
"log"
)
func main() {
jsonData := `{
"string": "string_value",
"number": 123.45,
"js_array": ["a", "b", "c"],
"integer": 678,
"subtype": {
"number_array": [1, 2, 3]
}
}`
m := map[string]interface{}{}
err := json.Unmarshal([]byte(jsonData), &m)
if err != nil {
log.Fatal(err)
}
for key, value := range m {
switch v := value.(type) {
case int:
fmt.Printf("Key: %s, Integer: %d\n", key, v)
case float64:
fmt.Printf("Key: %s, Float: %v\n", key, v)
case string:
fmt.Printf("Key: %s, String: %s\n", key, v)
case map[string]interface{}:
fmt.Printf("Key: %s, Subtype: %+v\n", key, v)
case []interface{}:
//TODO: Read through each item in the interface and work out what type it is.
fmt.Printf("Key: %s, []interface: %v\n", key, v)
default:
fmt.Printf("Key: %s, unhandled type: %+v\n", key, v)
}
}
}
Alternatively https://mholt.github.io/json-to-go/ does a decent job of turning examples of JSON data into Go structs that can be used for marshalling.
Putting the example in, I get something that isn't too bad.
type AutoGenerated struct {
Data []struct {
Acronym interface{} `json:"acronym"`
Badges []interface{} `json:"badges"`
CreatedAt string `json:"created_at"`
Deleted interface{} `json:"deleted"`
Description string `json:"description"`
Extras struct {
} `json:"extras"`
Frequency string `json:"frequency"`
FrequencyDate interface{} `json:"frequency_date"`
ID string `json:"id"`
LastModified string `json:"last_modified"`
LastUpdate string `json:"last_update"`
License string `json:"license"`
Metrics struct {
Discussions int `json:"discussions"`
Followers int `json:"followers"`
Issues int `json:"issues"`
NbHits int `json:"nb_hits"`
NbUniqVisitors int `json:"nb_uniq_visitors"`
NbVisits int `json:"nb_visits"`
Reuses int `json:"reuses"`
Views int `json:"views"`
} `json:"metrics"`
Organization struct {
Acronym string `json:"acronym"`
Class string `json:"class"`
ID string `json:"id"`
Logo string `json:"logo"`
LogoThumbnail string `json:"logo_thumbnail"`
Name string `json:"name"`
Page string `json:"page"`
Slug string `json:"slug"`
URI string `json:"uri"`
} `json:"organization"`
Owner interface{} `json:"owner"`
Page string `json:"page"`
Private bool `json:"private"`
Resources []struct {
Checksum struct {
Type string `json:"type"`
Value string `json:"value"`
} `json:"checksum"`
CreatedAt string `json:"created_at"`
Description interface{} `json:"description"`
Extras struct {
} `json:"extras"`
Filesize int `json:"filesize"`
Filetype string `json:"filetype"`
Format string `json:"format"`
ID string `json:"id"`
LastModified string `json:"last_modified"`
Latest string `json:"latest"`
Metrics struct {
NbHits int `json:"nb_hits"`
NbUniqVisitors int `json:"nb_uniq_visitors"`
NbVisits int `json:"nb_visits"`
Views int `json:"views"`
} `json:"metrics"`
Mime string `json:"mime"`
PreviewURL string `json:"preview_url"`
Published string `json:"published"`
Title string `json:"title"`
Type string `json:"type"`
URL string `json:"url"`
} `json:"resources"`
Slug string `json:"slug"`
Spatial interface{} `json:"spatial"`
Tags []interface{} `json:"tags"`
TemporalCoverage interface{} `json:"temporal_coverage"`
Title string `json:"title"`
URI string `json:"uri"`
} `json:"data"`
Facets struct {
Format [][]interface{} `json:"format"`
} `json:"facets"`
NextPage string `json:"next_page"`
Page int `json:"page"`
PageSize int `json:"page_size"`
PreviousPage interface{} `json:"previous_page"`
Total int `json:"total"`
}
If you want inline decode of nested data for quick uses follow the below method:
myJsonData := `{
"code": "string_code",
"data": {
"id": 123,
"user": {
"username": "my_username",
"age": 30,
"posts": [ "post1", "post2"]
}
}
}`
Let's say you have the above nested and unknown JSON data that want to be read and parsed, first read that intomap[string]interface{}:
m := map[string]interface{}{}
err := json.Unmarshal([]byte(myJsonData), &m)
if err != nil {
log.Fatal(err)
}
Now if you want to access code
fmt.Println(m["code"])
For id in nested data block of JSON:
fmt.Println(m["data"].(map[string]interface{})["id"].(float64))
For username in the second level nested user block of JSON:
fmt.Println(m["data"].(map[string]interface{})["user"].(map[string]interface{})["username"].(string))
For age in the second level nested user block of JSON:
fmt.Println(m["data"].(map[string]interface{})["user"].(map[string]interface{})["age"].(float64))
For post1 in the third level nested posts block of JSON:
fmt.Println(m["data"].(map[string]interface{})["user"].(map[string]interface{})["posts"].([]interface{})[0].(string))
Please check the example in playground

How to simplify marshaling of deeply-nested JSON

I have a JSON which contain some static and dynamic data. Below is a sample of JSON
{
"request": { /*Static data start */
"data": {
"object": { /*Static data ends here*/
"user": { /*Dynamic data start here */
"userid": "andmmdn",
"ipaddr": "1.1.1.1",
"noofusers": "100",
"qos": "34",
"id": "kldjflkdfjlkdjfkld",
"domain": "xyz.com" /*Dynamic data ends here */
}
}
}
}
}
Below is the code which can create this JSON
package main
import (
"fmt"
"encoding/json"
)
//ReqJSON struct
type ReqJSON struct {
Request Request `json:"request"`
}
//Request struct
type Request struct {
Data Data `json:"data"`
}
//Data struct
type Data struct {
Object Object `json:"object"`
}
//Object struct
type Object struct {
User User `json:"user"`
}
//User struct
type User struct {
UserID string `json:"userid"`
IPAddr string `json:"ipaddr"`
Noofusers string `json:"noofusers"`
Qos string `json:"qos"`
ID string `json:"id"`
Domain string `json:"domain"`
}
func main() {
test := ReqJSON {
Request{
Data: Data{
Object: Object{
User: User{
UserID: "andmmdn",
IPAddr: "1.1.1.1",
Noofusers: "100",
Qos: "34",
ID: "kldjflkdfjlkdjfkld",
Domain: "xyz.com",
},
},
},
},
}
jsonEncode, _ := json.Marshal(test)
jsonIdent, _ := json.MarshalIndent(&test, "", "\t")
fmt.Println(string(jsonEncode))
fmt.Println(string(jsonIdent))
}
As you can see from above it contain struct which doesn't make much sense as they act more like placeholder for nesting the data. So how we make it more optimized. As all the data is being taken care in the last struct. What approach for unmarshaling of the data should be applied as the response will be in same format and want to use the last struct for the same.
Any thoughts on the approach.
Also how to make a generic struct as I multiple API which uses same struct below is an example
//ReqJSON for populating data
type ReqJSON struct {
Request struct {
Data struct {
Object struct {
Auth Auth `json:"auth"`
} `json:"object"`
} `json:"data"`
} `json:"request"`
}
//ReqJSON for populating data
type ReqJSON struct {
Request struct {
Data struct {
Object struct {
Error Error `json:"error"`
} `json:"object"`
} `json:"data"`
} `json:"request"`
}
If you don't need the wrapping types for anything besides marshaling/unmarshaling, you can define them anonymously:
type ReqJSON struct {
Request struct {
Data struct {
Object struct {
User User `json:"user"`
} `json:"object"`
} `json:"data"`
} `json:"request"`
}
type User struct {
UserID string `json:"userid"`
IPAddr string `json:"ipaddr"`
Noofusers string `json:"noofusers"`
Qos string `json:"qos"`
ID string `json:"id"`
Domain string `json:"domain"`
}
And, borowing from icza's answer, you can add accessor methods to ReqJSON:
func (j *ReqJSON) User() User { return j.Request.Data.Object.User }
func (j *ReqJSON) SetUser(u User) { j.Request.Data.Object.User = u }
func main() {
var j ReqJSON
j.SetUser(User{
UserID: "_id",
IPAddr: "1.1.1.1",
Noofusers: "100",
Qos: "34",
ID: "kldjflkdfjlkdjfkld",
Domain: "xyz.com",
})
b, err := json.MarshalIndent(j, "", " ")
fmt.Println(err, string(b))
}
That sounds about right. The solution is a little verbose / redundant, but so is the data format you have to deal with.
To work with that easily, you may create helper functions and use them:
func wrap(u User) *ReqJSON {
return &ReqJSON{Request: Request{Data: Data{Object: Object{User: u}}}}
}
func unwrap(r *ReqJSON) User {
return r.Request.Data.Object.User
}
But other than that, you can't really simplify other things.
So marshaling a User is like:
var u User
data, err := json.Marshal(wrap(u))
Unmarshaling is:
var r *ReqJSON
err := json.Unmarshal(data, &r)
// Check error
u := unwrap(r) // Here we have the user
You can't eliminate the complexity, but you could potentially hide some of it inside of a custom Marshaler:
type Request struct {
UserID string `json:"userid"`
IPAddr string `json:"ipaddr"`
Noofusers string `json:"noofusers"`
Qos string `json:"qos"`
ID string `json:"id"`
Domain string `json:"domain"`
}
func (r *Request) MarshalJSON() ([]byte, error) {
type request struct {
Data struct {
Object struct {
User struct {
Request
} `json:"user"`
} `json:"object"`
} `json:"data"`
}
structure := request{Data: data{Object: object{User: user{r}}}}
return json.Marshal(structure)
}
The same approach can be employed in reverse for UnmarshalJSON, if desired.

cannot unmarshal string into Go struct field

I have the following (incomplete type) (which is a response from docker API from manifests endpoint v2 schema 1 https://docs.docker.com/registry/spec/manifest-v2-1/)
type ManifestResponse struct {
Name string `json:"name"`
Tag string `json:"tag"`
Architecture string `json:"architecture"`
FsLayers []struct {
BlobSum string `json:"blobSum"`
} `json:"fsLayers"`
History []struct {
V1Compatibility struct {
ID string `json:"id"`
Parent string `json:"parent"`
Created string `json:"created"`
} `json:"v1Compatibility"`
} `json:"history"`
}
While getting the following response:
{ "schemaVersion": 1,
"name": "library/redis",
"tag": "latest",
"architecture": "amd64",
"history": [
{
"v1Compatibility": "{\"id\":\"ef8a93741134ad37c834c32836aefbd455ad4aa4d1b6a6402e4186dfc1feeb88\",\"parent\":\"9c8b347e3807201285053a5413109b4235cca7d0b35e7e6b36554995cfd59820\",\"created\":\"2017-10-10T02:53:19.011435683Z\"}"
}
]
}
While trying to deserialize it using the following snippet of code:
var jsonManResp ManifestResponse
if err = json.NewDecoder(res.Body).Decode(&jsonManResp); err != nil {
log.Fatal(err)
}
I get the following error:
json: cannot unmarshal string into Go struct field .v1Compatibility of type struct { ID string "json:\"id\""; Parent string "json:\"parent\""; Created string "json:\"created\"" }
Full playground code: https://play.golang.org/p/tHzf9GphWX
What might be the problem?
The problem is that v1Compatibility is a string value in the JSON; the value happens to be JSON content, but it's inside a JSON string, so can't be unmarshalled all in one step. You could instead unmarshal it in two passes:
type ManifestResponse struct {
Name string `json:"name"`
Tag string `json:"tag"`
Architecture string `json:"architecture"`
FsLayers []struct {
BlobSum string `json:"blobSum"`
} `json:"fsLayers"`
History []struct {
V1CompatibilityRaw string `json:"v1Compatibility"`
V1Compatibility V1Compatibility
} `json:"history"`
}
type V1Compatibility struct {
ID string `json:"id"`
Parent string `json:"parent"`
Created string `json:"created"`
}
And then:
var jsonManResp ManifestResponse
if err := json.Unmarshal([]byte(exemplar), &jsonManResp); err != nil {
log.Fatal(err)
}
for i := range jsonManResp.History {
var comp V1Compatibility
if err := json.Unmarshal([]byte(jsonManResp.History[i].V1CompatibilityRaw), &comp); err != nil {
log.Fatal(err)
}
jsonManResp.History[i].V1Compatibility = comp
}
Working playground example here: https://play.golang.org/p/QNsu5_63E0