How to JSON-decode lowercased names into my struct? - json

I'm starting to go crazy trying to get Go to decode this json request body. Here's a sample request:
curl -X POST -d "{\"username\":\"foo\", \"password\":\"bar\"}" http://localhost:3000/users
And here's my handler:
mux.HandleFunc("/users", func(rw http.ResponseWriter, req *http.Request) {
var body struct {
username string
password string
}
// buf := make([]byte, req.ContentLength)
// req.Body.Read(buf)
// fmt.Println(string(buf))
//
// The above commented out code will correctly print:
// {"username":"foo", "password":"bar"}
err := json.NewDecoder(req.Body).Decode(&body)
if err != nil {
rw.WriteHeader(http.StatusNotAcceptable)
return
}
fmt.Printf("%+v\n", body)
// prints -> {username: password:}
})
Like the comment suggests, I can verify that req.Body is indeed correct -- but for whatever reason, json.NewDecoder(req.Body).Decode(&body) never fills out the fields of body.
Any help would be greatly appreciated!

The problem is that the json decoder does not deal with private struct fields. The fields in your body structs are private.
Rewrite it like so and it will work:
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
basically the json:"username" is a way to tell the json decoder how to map the json name of the object to the struct name. In this instance, for decoding only, it is not necessary - the json decoder is smart enough to make the translation of the upper/lower case.
But if you use the object to encode json as well, you need it, or you'll have upper case field names in the resulting json.
You can use the json struct tags for a few more useful things, like omitting empty field from encoded json, or skipping fields entirely.
You can read more about the JSON struct tags in the documentation for json.Marshal: http://golang.org/pkg/encoding/json/#Marshal

Related

Get JSON representation of request body as a string

I would like to take an arbitrary http.Request and get the body as a json string. I know this involves the json package, but it seems json.Decode needs a specific struct passed in by reference. How can I decode an arbitrary request body (and then stringify the result)?
func RequestBodyJsonString(r *http.Request) string {
}
Use ioutil.ReadAll to get data in byte slice then type conversion to string to get json string
bytedata, err := ioutil.ReadAll(r.Body)
reqBodyString := string(data)
An example in go playground here

How not to allow empty fields when Unmarshalling json to a struct

I do have this struct for my request
type RequestBody struct {
ForkliftID string `json:"forklift_id"`
WarehouseID string `json:"warehouse_id,omitempty"`
TaskID string `json:"task_id"`
}
If I don't send the "forklift_id" in the json request body, the unmarshalling assigns "" without returning error, I wonder if there is something like
`json: "forklift_id,notempty"`
so the unmarshalling returns an error if the field is not present or empty.
Thanks in advance
I assume what you did (treating empty value from payload as an error) is for validation purposes. If so, I think you can go with #colminator answer, or try to use 3rd party library that designed to solve this particular problem. One example library is https://github.com/go-playground/validator.
type RequestBody struct {
ForkliftID string `json:"forklift_id" validate:"required"`
WarehouseID string `json:"warehouse_id" validate:"required"`
TaskID string `json:"task_id"`
}
// ...
var payload RequestBody
// ...
validate := validator.New()
err := validate.Struct(payload)
if err != nil {
// handle the error
}
The field with tag validate:"required" will be validated when validate.Struct() called.
There are also a lot of useful validation rules available other than required.
For a more detailed example, take a look at the example source code
Another alternative solution would be, by performing explicit checks across those struct's fields. Example:
// ...
if payload.ForkliftID == "" {
err = fmt.Errorf("Forklift ID cannot be empty")
}
if payload.WarehouseID == "" {
err = fmt.Errorf("Warehouse ID cannot be empty")
}
Change the field type to *String. After unmarshaling, if the value is nil, then you know the JSON field was not provided.
See here for a more thorough example of unmarshaling error-checking.

Print POST JSON data

I have an issue printing the JSON from the POST.
I use gorilla/mux for routing
r := mux.NewRouter()
r.HandleFunc("/test", Point).Methods("POST")
http.ListenAndServe(":80", r)`
and in Point function I have
func Point(w http.ResponseWriter, r *http.Request) {
var callback Decoder
json.NewDecoder(r.Body).Decode(&callback)
}
But I can use this method only when I know the structure and I want to figure out how to log.Print the whole JSON as a string. I tried
func Point(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
log.Println(r.Form)
}
But it prints an empty map. Please help to figure out this.
In the assumption that you are building a standard API endpoint which receives some JSON and you would like to do something with it you should approach it the following way.
Edit:
As mentioned in the comments when you use the ioutil.ReadAll()
function as in this example it will read everything that is sent in the
post request into the application memory. It's a good idea to check
this in a production application (e.g limiting payload size).
1.) Make a struct which will hold the data coming from an API post request in GoLang
2.) Convert your request body into a byte array []byte
3.) Unmarshal your []byte into a single instance of your struct made earlier.
I'll put an example below:
1. Make a struct which you'll put your JSON into.
Let's take an example of a simple blog post.
The JSON object looks like this and has a slug, a title, and description
{
"slug": "test-slug",
"title": "This is the title",
"body": "This is the body of the page"
}
So your struct would look like this:
type Page struct {
Slug string `json:"slug"`
Title string `json:"title"`
Body string `json:"body"`
}
2 - 3. Get the body of your request and convert it to byte[] Then take that string and Unmarshal it into an instance of your struct.
The data of a post request is the request 'Body'.
In Golang the request in almost all cases (unless you're using something fancy outside of the defaults) will be an http.Request object. This is the 'r' which you normally have in your normal code and it holds the 'Body' of our POST request.
import (
"encoding/json"
"github.com/go-chi/chi" // you can remove
"github.com/go-chi/render" // you can remove but be sure to remove where it is used as well below.
"io/ioutil"
"net/http"
)
func GiveMeAPage(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("A page"))
}
So what we are going to do here is convert an io.ReadCloser, which is what the http.Request.Body is, to a []byte as the Unmarshal function takes a []byte type. I've commented inline below for you.
func Create(w http.ResponseWriter, r *http.Request) {
var p Page //Create an instance of our struct
//Read all the data in r.Body from a byte[], convert it to a string, and assign store it in 's'.
s, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// use the built in Unmarshal function to put the string we got above into the empty page we created at the top. Notice the &p. The & is important, if you don't understand it go and do the 'Tour of Go' again.
err = json.Unmarshal(s, &p)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// From here you have a proper Page object which is filled. Do what you want with it.
render.JSON(w, r, p) // This is me using a useful helper function from go-chi which 'Marshals' a struct to a json string and returns it to using the http.ResponseWriter.
}
As a side note. Please don't use Decoder to parse JSON unless you are using JSON streams. You are not here, and it's unlikely you will
for a while. You can read about why that is
here
If you just want the raw JSON data without parsing it, http.Request.Body implements io.Reader, so you can just Read from it. For example with ioutil.ReadAll.
Something like (untested):
func Point(w http.ResponseWriter, r *http.Request) {
data, err := ioutil.ReadAll(r.Body)
// check error
// do whatever you want with the raw data (you can `json.Unmarshal` it too)
}

Preserve json.RawMessage through multiple marshallings

Background
I'm working with JSON data that must be non-repudiable.
The API that grants me this data also has a service to verify that the data originally came from them.
As best as I can tell, they do this by requiring that the complete JSON they originally sent needs to be supplied to them inside another JSON request, with no byte changes.
Issue
I can't seem to preserve the original JSON!
Because I cannot modify the original JSON, I have carefully preserved it as a json.RawMessage when unmarshalling:
// struct I unmarshal my original data into
type SignedResult struct {
Raw json.RawMessage `json:"random"`
Signature string `json:"signature"`
...
}
// struct I marshal my data back into
type VerifiedSignatureReq {
Raw json.RawMessage `json:"random"`
Signature string `json:"signature"`
}
// ... getData is placeholder for function that gets my data
response := SignedResult{}
x, _ := json.Unmarshal(getData(), &response)
// do some post-processing with SignedResult that does not alter `Raw` or `Signature`
// trouble begins here - x.Raw started off as json.RawMessage...
y := json.Marshal(VerifiedSignatureReq{Raw: x.Raw, Signature: x.Signature}
// but now y.Raw is base64-encoded.
The problem is that []bytes / RawMessages are base64-encoded when marshaled. So I can't use this method, because it completely alters the string.
I'm unsure how to ensure this string is correctly preserved. I had assumed that the json.RawMessage specification in my struct would survive the perils of marshaling an already marshaled instance because it implements the Marshaler interface, but I appear mistaken.
Things I've Tried
My next attempt was to try:
// struct I unmarshal my original data into
type SignedResult struct {
Raw json.RawMessage `json:"random"`
Signature string `json:"signature"`
...
}
// struct I marshal my data back into
type VerifiedSignatureReq {
Raw map[string]interface{} `json:"random"`
Signature string `json:"signature"`
}
// ... getData is placeholder for function that gets my data
response := SignedResult{}
x, _ := json.Unmarshal(getData(), &response)
// do some post-processing with SignedResult that does not alter `Raw` or `Signature`
var object map[string]interface{}
json.Unmarshal(x.Raw, &object)
// now correctly generates the JSON structure.
y := json.Marshal(VerifiedSignatureReq{Raw: object, Signature: x.Signature}
// but now this is not the same JSON string as received!
The issue with this approach is that there are minor byte-wise differences in the spacing between the data. It no longer looks exactly the same when catted to a file.
I cannot use string(x.Raw) either because it escapes certain characters when marshaled with \.
You will need a custom type with its own marshaler, in place of json.RawMessage for your VerifiedSignatureReq struct to use. Example:
type VerifiedSignatureReq {
Raw RawMessage `json:"random"`
Signature string `json:"signature"`
}
type RawMessage []byte
func (m RawMessage) MarshalJSON() ([]byte, error) {
return []byte(m), nil
}

How to deal with Struct having different json key than json response

I have a struct VideoInfo that has a key in it called embedCode. The API I am querying returns the embed code as embed_code. During unmarshalling the response how do I ensure embed_code goes into embedCode?
Also is there an easy way to take a large json string and automatically turn it into a struct, or can one only use a map?
With respect to remapping the field names use the corresponding annotation in the structure declaration:
type VideoInfo struct {
EmbedCode string `json:"embed_code"`
}
The marshaller/un-marshaller will only process public field, so you need to capitalise the field name.
With respect to converting the whole structure, yes it is easy. Declare an instance to un-marshal into and pass a reference to the json.Unmarshal method (from a test):
data, _ := json.Marshal(request)
var resp response.VideoInfo
if err := json.Unmarshal(data, &resp); err != nil {
t.Errorf("unexpected error, %v", err)
}
At first, struct's field must be start from capital letter to be public. So you need something like that:
type VideoInfo struct {
EmbedCode string `json:"embed_code"`
}
And look at documentation for more info.