In a rest api, when the body is set to "{}", the json Decoder will not generate an error. This makes it necessary to check if the target struct is still nil.
I need to check if the library is supposed to work like this, or if this is an issue with it.
// Client Side this request
req, err := http.NewRequest("POST", "url", strings.NewReader("{}") )
// Curl equivalent:
curl -X POST -d '{}' http://api:8080/r/primitives/multiply
// Server side
type Operands struct {
Values []float64 `json:"Values"`
}
func handler(req *http.Request, res *http.Response) (string, error) {
operands := Operands{}
err := json.NewDecoder(req.Body).Decode(&operands)
if err != nil {
res.StatusCode = 400
res.Status = http.StatusText(res.StatusCode)
http.StatusText(res.StatusCode)
return "", err
}
operands.Values[0] // It will fail here.
}
Edit 1: The decoder works fine with and empty body "" with the error being generated, and works fine with a correct body like this one: {"Values" : [ 5.0 , 2.0 ]}
Edit 2: The issue here is that with a "{}" body, it will not return an error when decoding, instead it will keep the target struct as nil.
{} is just an empty Json object, and it will decode fine to your Operandsstruct, as the struct is not required to have anything in the Operands array.
You need to validate that yourself, e.g.
err := json.NewDecoder(req.Body).Decode(&operands)
if err != nil || len(operands.Values) == 0{
Related
I have a Set data structure implemented in Go with the basic operations like Add, Remove, Difference, Union. I am trying to send a http request using the json encoder to encode the request body which contains the object of the form map[string]Set. The Set data structure is defined below:
type Set map[interface{}]struct{}
func NewSet() Set {
set := make(Set)
return set
}
The encoder looks like this:
func (req *Request) BodyContentInJson (val interface{}) error {
buf := bytes.NewBuffer(nil)
enc := json.NewEncoder(buf)
if err := enc.Encode(val); err != nil {
return err
}
req.Obj = val
req.Body = buf
req.BodySize = int64(buf.Len())
return nil
}
This code fails at
if err := enc.Encode(val); err != nil {
return err
}
giving an error:{"errors":["json: unsupported type: Set"]}. Also, the type of val is map[string]interface{}when I debugged it.
How could I possibly encode and marshal/unmarshal JSON content here using the Go's encoder ?
You could write your own UnmarshalJSON method on the *Set type which would then be used by the json.Encoder to encode the json data into the Set. Here's a simple example https://play.golang.org/p/kx1E-jDu5e.
By the way, the reason you're getting the error is because a map key of type interface{} is not supported by the encoding/json package. (https://github.com/golang/go/blob/master/src/encoding/json/encode.go#L697-L699)
How can I fetch the json response from the POST method? Currently I'm only able to fetch Status - 401 Unauthorized and StatusCode - 401
func postUrl(url string, byt []byte) (*http.Response, error) {
tr := &http.Transport{
DisableCompression: true,
}
client := &http.Client{Transport: tr, Timeout: 10 * time.Second}
req, err := http.NewRequest("POST", url, bytes.NewBuffer(byt))
req.Header.Set("X-Custom-Header", "myvalue")
req.Header.Set("Content-Type", "application/json")
req.Header.Add("Authorization", "Basic "+basicAuth("username", "password"))
resp, err := client.Do(req)
return resp, err
}
Above code produces the output:
{
"errorMessages": [
"You do not have the permission to see the specified issue.",
"Login Required"
],
"errors": {}
}
The way to read the response (if there is one) is the same regardless of what status you get.
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
As Frank said, regardless of the status code in the response you can simply read its body to use whatever content it has.
Particularly for the case of a JSON message, you have two options depending on whether you know the JSON message structure in advance (or want your code to depend on it).
If you know the structure and are ok with hard-coding it (plus you gain some type safety and better client code) you can have:
type ErrorResponse struct {
Messages []string `json:"errorMessages"`
}
And then when you detect an error status code unmarshal the response body as that struct:
if resp.StatusCode % 100 != 2 {
var error ErrorResponse
err := json.Unmarshall(resp.Body, &error)
// check err != nil ...
// user error.ErrorMessages for whatever you want
}
Alternatively if you don't want to depend on the JSON structure (to some degree) you can try to unmarshall it to a map[string]interface{} and try to use that in the generic way you think you can (generally not very useful).
This question is not related to http responses and http methods. Decode json string (wich is http response body in that case) with json decoder.
Simple example
(not directly related to your code snippet)
type Transition struct {
Transition map[string]int
}
func main() {
resp, err := postUrl(url, byt)
if err != nil {
log.Fatal(err)
}
var trans Transition
decoder := json.NewDecoder(resp.Body)
if err := decoder.Decode(&trans); err != nil {
log.Fatal(err)
}
fmt.Printf("%+v\n", trans)
}
I'm having a little bit of trouble handling types in Golang. I'm making a POST router.
Consider the following struct:
type DataBlob struct {
Timestamp string
Metric_Id int `json:"id,string,omitempty"`
Value float32 `json:"id,string,omitempty"`
Stderr float32 `json:"id,string,omitempty"`
}
This is my POST router using json.Unmarshal() from a decoded stream:
func Post(w http.ResponseWriter, req * http.Request) {
body, err := ioutil.ReadAll(req.Body)
if err != nil {
panic()
}
var t DataBlob
err = json.Unmarshal(body, &t)
if err != nil {
panic()
}
fmt.Printf("%s\n", t.Timestamp)
fmt.Printf("%d\n", int(t.Metric_Id))
fmt.Printf("%f\n", t.Value)
fmt.Printf("%f\n", t.Stderr)
}
It seems that no matter what I make my values in my POST request:
{
"timestamp": "2011-05-16 15:36:38",
"metric_id": "28",
"value": "4.5",
"stderr": "8.5"
}
All the non-String values print as 0 or 0.000000, respectively. It also doesn't matter if I try to type convert inline after-the-fact, as I did with t.Metric_Id in the example.
If I edit my struct to just handle string types, the values print correctly.
I also wrote a version of the POST router using json.NewDecoder():
func Post(w http.ResponseWriter, req * http.Request) {
decoder := json.NewDecoder(req.Body)
var t DataBlob
err := decoder.Decode(&t)
if err != nil {
panic()
}
fmt.Printf("%s\n", t.Timestamp)
fmt.Printf("%d\n", t.Metric_Id)
fmt.Printf("%f\n", t.Value)
fmt.Printf("%f\n", t.Stderr)
}
This is building off of functionality described in this answer, although the solution doesn't appear to work.
I appreciate your help!
You need to change the names of your Datablob values. You've told the JSON decoder that they're all named "id". You should try something like the following. Also, take a look at the json.Marshal description and how it describes the tags for structs and how the json library handles them. https://golang.org/pkg/encoding/json/#Marshal
type DataBlob struct {
Timestamp string
Metric_Id int `json:"metric_id,string,omitempty"`
Value float32 `json:"value,string,omitempty"`
Stderr float32 `json:"stderr,string,omitempty"`
}
I am working on some code to parse the JSON data from an HTTP response. The code I have looks something like this:
type ResultStruct struct {
result []map[string]string
}
var jsonData ResultStruct
err = json.Unmarshal(respBytes, &jsonData)
The json in the respBytes variable looks like this:
{
"result": [
{
"id": "ID 1"
},
{
"id": "ID 2"
}
]
}
However, err is not nil. When I print it out it says unexpected end of JSON input. What is causing this? The JSON seems to valid. Does this error have something to do with my custom struct?
Thanks in advance!
The unexpected end of JSON input is the result of a syntax error in the JSON input (likely a missing ", }, or ]). The error does not depend on the type of the value that you are decoding to.
I ran the code with the example JSON input on the playground. It runs without error.
The code does not decode anything because the result field is not exported. If you export the result field:
type ResultStruct struct {
Result []map[string]string
}
then the input is decoded as shown in this playground example.
I suspect that you are not reading the entire response body in your application. I suggest decoding the JSON input using:
err := json.NewDecoder(resp.Body).Decode(&jsonData)
The decoder reads directly from the response body.
You can also get this error if you're using json.RawMessage in an unexported field. For example, the following code produces the same error:
package main
import (
"encoding/json"
"fmt"
)
type MyJson struct {
Foo bool `json:"foo"`
bar json.RawMessage `json:"bar"`
}
type Bar struct {
X int `json:"x"`
}
var respBytes = []byte(`
{
"foo": true,
"bar": { "x": 10 }
}`)
func main() {
var myJson MyJson
err := json.Unmarshal(respBytes, &myJson)
if err != nil {
fmt.Println(err)
return
}
myBar := new(Bar)
err = json.Unmarshal(myJson.bar, myBar)
fmt.Println(err)
}
If you export "MyJson.bar" field (e.g. -> "MyJson.Bar", then the code works.
it is not the case here; but if you are getting this error loading json from a file it Will occur if the byte slice for the buffer is not initialized the the byte size of the file. [when you're new like me that happens! ] Since this is the first search result I got it still took some digging to figure out. In this use case the error is a bit misleading.
type GenesisResultStruct []GenesisField
fileinfo, _ := genesis.Stat()
bs := make([]byte, fileinfo.Size())
//bs := []byte {} // wrong!!
_, error := genesis.Read(bs)
if error != nil {
fmt.Println("genesis read error: ", error)
os.Exit(1)
}
var jsonData GenesisResultStruct
eGen = json.Unmarshal(bs, &jsonData)
if eGen != nil {
fmt.Println("genesis unmarshal error: ", eGen)
os.Exit(1)
}
Faced the same issue today.
You can also get this error if the respBytes is nil or there are no brackets [] if you are unmarshalling it to a slice.
In that case, you need to explicitly set respBytes.
As you are unmarshalling it to a slice, brackets [] are expected in the byte-slice
if src == nil {
src = []byte("[]")
}
So I have the following, which seems incredibly hacky, and I've been thinking to myself that Go has better designed libraries than this, but I can't find an example of Go handling a POST request of JSON data. They are all form POSTs.
Here is an example request: curl -X POST -d "{\"test\": \"that\"}" http://localhost:8082/test
And here is the code, with the logs embedded:
package main
import (
"encoding/json"
"log"
"net/http"
)
type test_struct struct {
Test string
}
func test(rw http.ResponseWriter, req *http.Request) {
req.ParseForm()
log.Println(req.Form)
//LOG: map[{"test": "that"}:[]]
var t test_struct
for key, _ := range req.Form {
log.Println(key)
//LOG: {"test": "that"}
err := json.Unmarshal([]byte(key), &t)
if err != nil {
log.Println(err.Error())
}
}
log.Println(t.Test)
//LOG: that
}
func main() {
http.HandleFunc("/test", test)
log.Fatal(http.ListenAndServe(":8082", nil))
}
There's got to be a better way, right? I'm just stumped in finding what the best practice could be.
(Go is also known as Golang to the search engines, and mentioned here so others can find it.)
Please use json.Decoder instead of json.Unmarshal.
func test(rw http.ResponseWriter, req *http.Request) {
decoder := json.NewDecoder(req.Body)
var t test_struct
err := decoder.Decode(&t)
if err != nil {
panic(err)
}
log.Println(t.Test)
}
You need to read from req.Body. The ParseForm method is reading from the req.Body and then parsing it in standard HTTP encoded format. What you want is to read the body and parse it in JSON format.
Here's your code updated.
package main
import (
"encoding/json"
"log"
"net/http"
"io/ioutil"
)
type test_struct struct {
Test string
}
func test(rw http.ResponseWriter, req *http.Request) {
body, err := ioutil.ReadAll(req.Body)
if err != nil {
panic(err)
}
log.Println(string(body))
var t test_struct
err = json.Unmarshal(body, &t)
if err != nil {
panic(err)
}
log.Println(t.Test)
}
func main() {
http.HandleFunc("/test", test)
log.Fatal(http.ListenAndServe(":8082", nil))
}
There are two reasons why json.Decoder should be preferred over json.Unmarshal - that are not addressed in the most popular answer from 2013:
February 2018, go 1.10 introduced a new method json.Decoder.DisallowUnknownFields() which addresses the concern of detecting unwanted JSON-input
req.Body is already an io.Reader. Reading its entire contents and then performing json.Unmarshal wastes resources if the stream was, say a 10MB block of invalid JSON. Parsing the request body, with json.Decoder, as it streams in would trigger an early parse error if invalid JSON was encountered. Processing I/O streams in realtime is the preferred go-way.
Addressing some of the user comments about detecting bad user input:
To enforce mandatory fields, and other sanitation checks, try:
d := json.NewDecoder(req.Body)
d.DisallowUnknownFields() // catch unwanted fields
// anonymous struct type: handy for one-time use
t := struct {
Test *string `json:"test"` // pointer so we can test for field absence
}{}
err := d.Decode(&t)
if err != nil {
// bad JSON or unrecognized json field
http.Error(rw, err.Error(), http.StatusBadRequest)
return
}
if t.Test == nil {
http.Error(rw, "missing field 'test' from JSON object", http.StatusBadRequest)
return
}
// optional extra check
if d.More() {
http.Error(rw, "extraneous data after JSON object", http.StatusBadRequest)
return
}
// got the input we expected: no more, no less
log.Println(*t.Test)
Playground
Typical output:
$ curl -X POST -d "{}" http://localhost:8082/strict_test
expected json field 'test'
$ curl -X POST -d "{\"Test\":\"maybe?\",\"Unwanted\":\"1\"}" http://localhost:8082/strict_test
json: unknown field "Unwanted"
$ curl -X POST -d "{\"Test\":\"oops\"}g4rB4g3##$%^&*" http://localhost:8082/strict_test
extraneous data after JSON
$ curl -X POST -d "{\"Test\":\"Works\"}" http://localhost:8082/strict_test
log: 2019/03/07 16:03:13 Works
I was driving myself crazy with this exact problem. My JSON Marshaller and Unmarshaller were not populating my Go struct. Then I found the solution at https://eager.io/blog/go-and-json:
"As with all structs in Go, it’s important to remember that only
fields with a capital first letter are visible to external programs
like the JSON Marshaller."
After that, my Marshaller and Unmarshaller worked perfectly!
I found the following example from the docs really helpful (source here).
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
func main() {
const jsonStream = `
{"Name": "Ed", "Text": "Knock knock."}
{"Name": "Sam", "Text": "Who's there?"}
{"Name": "Ed", "Text": "Go fmt."}
{"Name": "Sam", "Text": "Go fmt who?"}
{"Name": "Ed", "Text": "Go fmt yourself!"}
`
type Message struct {
Name, Text string
}
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var m Message
if err := dec.Decode(&m); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Printf("%s: %s\n", m.Name, m.Text)
}
}
The key here being that the OP was looking to decode
type test_struct struct {
Test string
}
...in which case we would drop the const jsonStream, and replace the Message struct with the test_struct:
func test(rw http.ResponseWriter, req *http.Request) {
dec := json.NewDecoder(req.Body)
for {
var t test_struct
if err := dec.Decode(&t); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
log.Printf("%s\n", t.Test)
}
}
Update: I would also add that this post provides some great data about responding with JSON as well. The author explains struct tags, which I was not aware of.
Since JSON does not normally look like {"Test": "test", "SomeKey": "SomeVal"}, but rather {"test": "test", "somekey": "some value"}, you can restructure your struct like this:
type test_struct struct {
Test string `json:"test"`
SomeKey string `json:"some-key"`
}
...and now your handler will parse JSON using "some-key" as opposed to "SomeKey" (which you will be using internally).
I like to define custom structs locally. So:
// my handler func
func addImage(w http.ResponseWriter, r *http.Request) {
// define custom type
type Input struct {
Url string `json:"url"`
Name string `json:"name"`
Priority int8 `json:"priority"`
}
// define a var
var input Input
// decode input or return error
err := json.NewDecoder(r.Body).Decode(&input)
if err != nil {
w.WriteHeader(400)
fmt.Fprintf(w, "Decode error! please check your JSON formating.")
return
}
// print user inputs
fmt.Fprintf(w, "Inputed name: %s", input.Name)
}
type test struct {
Test string `json:"test"`
}
func test(w http.ResponseWriter, req *http.Request) {
var t test_struct
body, _ := ioutil.ReadAll(req.Body)
json.Unmarshal(body, &t)
fmt.Println(t)
}