How to do a generic decode with golang - json

I've seen some go code that looks like this :
type userForm struct {
Name string `json:"name" validate:"min=2"`
Surname string `json:"surname" validate:"min=2"`
Phone string `json:"phone" validate:"min=10"`
Email string `json:"email,omitempty" validate:"omitempty,email"`
}
type emailForm struct {
Email string `json:"email" validate:"email"`
}
func decodeUserForm(r *http.Request) (userForm, error) {
var body userForm
d := json.NewDecoder(r.Body)
if err := d.Decode(&body); err != nil {
return body, NewHTTPError(err, 400, "unable to decode user form")
}
return body, validateStruct(body)
}
func decodeEmailForm(r *http.Request) (emailForm, error) {
var body emailForm
d := json.NewDecoder(r.Body)
if err := d.Decode(&body); err != nil {
return body, NewHTTPError(err, 400, "unable to decode email form")
}
return body, validateStruct(body)
}
I find two functions redundant. Is there a simpler way to merge those two into a more generic function? Is it good practice in Go?

func decodeForm(r *http.Request, dst interface{}) error {
if err := json.NewDecoder(r.Body).Decode(dst); err != nil {
return NewHTTPError(err, 400, "unable to decode email form")
}
return validateStruct(dst)
}
Then use it as so:
var body emailForm
if err := decodeForm(r, &body); err != nil {
panic(err)
}

Related

Get proxy response body as clean string in golang

I read this solution for resolve body data from a proxy.
Golang: how to read response body of ReverseProxy?
But I cannot read the body as a plain string, maybe the encoding is not right or other cryption.
My question is how to encode or transform the body to readable html string?
Currently I get:
n8�������♠�♠�A��b:J���g>-��ˤ���[���.....
Code example:
reverseProxy := httputil.NewSingleHostReverseProxy(url)
reverseProxy.ModifyResponse = func (resp *http.Response) (err error) {
var contentType = resp.Header.Get("Content-Type")
if strings.HasPrefix(contentType, "text/html") {
b, err := ioutil.ReadAll(resp.Body) //Read html
if err != nil {
return err
}
err = resp.Body.Close()
if err != nil {
return err
}
body := ioutil.NopCloser(bytes.NewReader(b))
resp.Body = body
resp.ContentLength = int64(len(b))
log.Printf(string(b))
}
return nil
}

How to pass struct as a parameter to a function

How can I do something Like this?
I am trying to pass a struct as a parameter to function in Go.
func handleEntityProperties(w http.ResponseWriter, r *http.Request) {
const sliceSize = 100
var entityProperties struct {
Instance string `json:"instance"`
Entities []struct {
Id string `json:"id"`
Properties map[string]string `json:"properties"`
Type string `json:"type"`
} `json:"entities"`
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
if !json.Valid([]byte(body)) {
fmt.Fprintf(w, "invalid json")
return
}
err = json.Unmarshal(body, &entityProperties)
sendData(entityProperties.Entities[0:100])
return
}
func sendData(entities struct) {
log.Println("Doing things with entities ", entities)
}
as you can see in code I am trying to send first 100 elements of entityProperties.Entities struct to a sendData. I know this is syntactically wrong.
Just declare your type outside of the functions:
type entity struct {
Id string `json:"id"`
Properties map[string]string `json:"properties"`
Type string `json:"type"`
}
And reuse it in handleEntityProperties() and in the signature of sendData():
func handleEntityProperties(w http.ResponseWriter, r *http.Request) {
const sliceSize = 100
var entityProperties struct {
Instance string `json:"instance"`
Entities []entity `json:"entities"`
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
if !json.Valid([]byte(body)) {
fmt.Fprintf(w, "invalid json")
return
}
err = json.Unmarshal(body, &entityProperties)
sendData(entityProperties.Entities[0:sliceSize])
return
}
func sendData(entities []entity) {
log.Println("Doing things with entities ", entities)
}
Also note that there is no guarantee that the client will send at least 100 entities, so you should check that else the slicing expression might result in a runtime panic:
max := 100
if len(entityProperties.Entities) < max {
max = len(entityProperties.Entities)
}
sendData(entityProperties.Entities[:max])
Also that check for invalid JSON is unnecessary: if the JSON is invalid, json.Unmarshal() will report a (non-nil) error and you'll know it.
Going further, you don't even have to read the complete body into memory (into a byte slice), you may use json.Decoder to read from it directly (without the intermediate memory buffer) like this:
dec := json.NewDecoder(r.Body)
if err := dec.Decode(&entityProperties); err != nil {
// handle error
}
And the final return statement is also unnecessary.
So an improved version may look like this:
func handleEntityProperties(w http.ResponseWriter, r *http.Request) {
var entityProperties struct {
Instance string `json:"instance"`
Entities []entity `json:"entities"`
}
dec := json.NewDecoder(r.Body)
if err := dec.Decode(&entityProperties); err != nil {
// handle error
http.Error(w, "invalid json", http.StatusBadRequest)
return
}
max := 100
if len(entityProperties.Entities) < max {
max = len(entityProperties.Entities)
}
sendData(entityProperties.Entities[:max])
}

golang json decode with empty request body

In the following http handler, I try to distinguish whether the request body is empty
type Request struct {
A bool `json:"lala"`
B bool `json:"kaka"`
C int32 `json:"cc"`
D int32 `json:"dd"`
}
var (
opts Request
hasOpts bool = true
)
err = json.NewDecoder(r.Body).Decode(&opts)
switch {
case err == io.EOF:
hasOpts = false
case err != nil:
return errors.New("Could not get advanced options: " + err.Error())
}
However, even with r.Body equals '{}', hasOpts is still true. Is this to be expected? In that case, how should I detect empty request body?
Read the body first, to check its content, then unmarshal it:
body, err := ioutil.ReadAll(r.Body)
if err != nil {
return err
}
if len(body) > 0 {
err = json.Unmarshal(body, &opts)
if err != nil {
return fmt.Errorf("Could not get advanced options: %s", err)
}
}

Golang: Decode json string to struct from mysql db

I'm trying to get informations from my db, and one of my fields is actually JSON stored as a string and I would like to get it as a struct.
This is my row's struct :
//there is json flag because I use it to get data from redis too
type InfoClip struct {
ClipId string `json:clipId`
StreamUrl string `json:streamUrl`
StartTimeCode int `json:startTimeCode`
EndTimeCode int `json:endTimeCode`
CreatedAt string `json:createdAt`
Metas string `json:metas` // here I get a string instead of a 'metas' struct
SourceId string `json:sourceId`
ProviderName string `json:providerName`
ProviderReference string `json:providerReference`
PublicationStatus string `json:publicationStatus`
UserId string `json:userId`
Name string `json:name`
FacebookPage string `json:facebookPage`
TwitterHandle string `json:twitterHandle`
PermaLinkUrl string `json:permalinkUrl`
Logo string `json:logo`
Link string `json:link`
}
This is my metas struct :
type metas struct {
Title string `json:title`
Tags []string `json:tags`
categories []string `json:permalink`
}
This is how I'm trying to get this field
func decodeJsonSql (met string) (*metas, error) {
m := metas{}
if err := json.Unmarshal([]byte(met), &m); err != nil {
fmt.Printf("Error decode metas: ", err)
return nil, err
} else {
return &m, err
}
}
func CheckIdSql(mediaId string) (error){
datab, err := sql.Open("mysql", "tcp()")
if err != nil {
fmt.Printf("[SQL ERROR] Cannot Open db => ", err)
return err
}
if err := datab.Ping(); err != nil {
fmt.Printf("[SQL ERROR] db connection => ", err)
return err
}
fmt.Printf("[SQL ONLINE] =>", datab)
defer datab.Close()
q := "SELECT c.id AS clipId, c.streamUrl, c.startTimecode, c.endTimecode, c.createdAt, s.metas,... FROM clips WHERE c.id = ?"
rows, err := datab.Query(q, mediaId)
if err != nil || err == sql.ErrNoRows {
fmt.Printf("SQL Err: %s", err)
return err
}
clips := InfoClip{}
for rows.Next() {
rows.Scan(&clips.ClipId, &clips.StreamUrl, &clips.StartTimeCode, &clips.EndTimeCode, &clips.CreatedAt, &clips.Metas, ...)
}
ret, err := decodeJsonSql(clips.Metas)
if err != nil{
return err
}
clips.Metas = ret
fmt.Printf("\n\n[SQL DEBUG RESPONSE]: %v", clips)
return nil
}
But this process is pretty heavy, surely there is an easier way?
Thanks.
You can make your metas struct implement the sql.Scanner interface
It should look something like this:
func (m *metas) Scan(src interface{}) error {
strValue, ok := src.(string)
if !ok {
return fmt.Errorf("metas field must be a string, got %T instead", src)
}
return json.Unmarshal([]byte(strValue), m)
}
After that you can use it as an InfoClip field and pass it directly to Scan and drop the decodeJsonSql:
type InfoClip struct {
// [...]
Metas metas `json:metas`
// [...]
}
and
q := "SELECT c.id AS clipId, c.streamUrl, c.startTimecode, c.endTimecode, c.createdAt, s.metas,... FROM clips WHERE c.id = ?"
row := datab.QueryRow(q, mediaId)
clips := InfoClip{}
err := row.Scan(&clips.ClipId, &clips.StreamUrl, &clips.StartTimeCode, &clips.EndTimeCode, &clips.CreatedAt, &clips.Metas) // [...]
if err != nil {
fmt.Printf("SQL Err: %s", err)
return err
}
(BTW, as you can see, I replaced datab.Query with datab.QueryRow as you are expecting only one result)

Getting a Bad response - 422 from the server while sending a POST request in GO

I am trying to send a POST request using this function -
{
func (Client *Client) doModify(method string, url string, createObj interface{}, respObject interface{}) error {
bodyContent, err := json.Marshal(createObj)
if err != nil {
return err
}
client := Client.newHttpClient()
req, err := http.NewRequest(method, url, bytes.NewBuffer(bodyContent))
if err != nil {
return err
}
Client.setupRequest(req)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Content-Length", string(len(bodyContent)))
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return errors.New(fmt.Sprintf("Bad response from [%s], go [%d]", url, resp.StatusCode))
}
byteContent, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
return json.Unmarshal(byteContent, respObject)
}
}
I am calling my function like this -
{
func TestContainerCreate(t *testing.T) {
client := newClient(t)
container, err := client.Container.Create(&Container{
Name: "name",
ImageUuid: "xyz",
})
if err != nil {
t.Fatal(err)
}
defer client.Container.Delete(container)
}
}
The Create function calls internally calls the doCreate function which calls the doModify function pasted on the top .
{
func (self *ContainerClient) Create(container *Container) (*Container, error) {
resp := &Container{}
err := self.Client.doCreate(container_TYPE, container, resp)
return resp, err
}
}
{
func (Client *Client) doCreate(schemaType string, createObj interface{}, respObject interface{}) error {
if createObj == nil {
createObj = map[string]string{}
}
schema, ok := Client.Types[schemaType]
if !ok {
return errors.New("Unknown schema type [" + schemaType + "]")
}
return Client.doModify("POST", collectionUrl, createObj, respObject)
}
}
This gives me a 422 bad response.On doing further research, When doing a CURL, with "name" and "imageUuid" first letter as small case, gives a 201 created status but when passing "Name" and "ImageUuid" first letter as capital gives 422 bad response. Could there be issue with the json struct defined for container, or case of these entities being defined or something else?
{
curl -X POST -v -s http://localhost:8080/v1/containers -H 'Content-Type: application/json' -d '{"name" : "demo", "imageUuid" : "docker:nginx"}' | python -m 'json.tool'
}
Container struct definition looks like this -
{
type Container struct {
Resource
ImageId string `json:"ImageId,omitempty"`
ImageUuid string `json:"ImageUuid,omitempty"`
MemoryMb int `json:"MemoryMb,omitempty"`
Name string `json:"Name,omitempty"`
}
type ContainerCollection struct {
Collection
Data []Container `json:"data,omitempty"`
}
}
string(len(bodyContent)) isn't doing what you think it is. You're converting a single int to a utf-8 string. You want to use the strconv package to get the numerical representation.
Also note that you can't omitempty an int, since 0 is a valid value.