I would like to take an arbitrary http.Request and get the body as a json string. I know this involves the json package, but it seems json.Decode needs a specific struct passed in by reference. How can I decode an arbitrary request body (and then stringify the result)?
func RequestBodyJsonString(r *http.Request) string {
}
Use ioutil.ReadAll to get data in byte slice then type conversion to string to get json string
bytedata, err := ioutil.ReadAll(r.Body)
reqBodyString := string(data)
An example in go playground here
Related
Using Go, I'm attempting to fetch a few JSON responses concurrently from multiple endpoints. I'd like to attach each of these responses to fields in a struct or map and return this struct/map as a JSON object. (Backend for Frontend pattern). So I will make a web request to the Go application with some sort of identifier. It will in turn make several web requests, and compile the data into one large object to return as a response.
I'm using Fiber as my framework but any generic web framework would be similar:
app.Get("/requests/:identifier", func(c *fiber.Ctx) error {
identifier := c.Params("identifier")
timeout := 1600 * time.Millisecond
client := httpclient.NewClient(httpclient.WithHTTPTimeout(timeout))
res, err := client.Get("https://www.example.com/endpoint?id=" + identifier, nil)
if err != nil{
logger.Error("Timout value exceeded")
return c.Status(503).SendString("Socket Timeout")
}
logger.Info("Fetch success: ")
// Heimdall returns the standard *http.Response object
body, err := ioutil.ReadAll(res.Body)
code := 200
response := &main_response{
Service1: body,
}
return c.Status(code).JSON(response)
})
The problem I'm having is, I have no need to unmarshal this data in Go, as I have no use for it (I am simply passing it along). Do I have to unmarshal it just so I can set it as a field in my response struct like this?
type main_response struct {
Service1 []byte `json:"service1"`
Service2 map[string]string `json:"service2"`
Service3 map[string]interface{} `json:"service3"`
}
(I've tried a few different ways to accomplish this. Trying to use a byte array seems to base64 encode the responses)
I will want to marshal that struct to JSON before returning it, so perhaps I have little choice as I can't think of a way to tell Go "only marshal the main struct, everything else is already JSON". It almost feels like I'd be better off building a string at this point.
Use json.RawMessage to copy a []byte containing JSON directly to the response JSON document:
type main_response struct {
Service1 json.RawMessage `json:"service1"`
...
}
response := &main_response{
Service1: body,
}
return c.Status(code).JSON(response)
I have an issue printing the JSON from the POST.
I use gorilla/mux for routing
r := mux.NewRouter()
r.HandleFunc("/test", Point).Methods("POST")
http.ListenAndServe(":80", r)`
and in Point function I have
func Point(w http.ResponseWriter, r *http.Request) {
var callback Decoder
json.NewDecoder(r.Body).Decode(&callback)
}
But I can use this method only when I know the structure and I want to figure out how to log.Print the whole JSON as a string. I tried
func Point(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
log.Println(r.Form)
}
But it prints an empty map. Please help to figure out this.
In the assumption that you are building a standard API endpoint which receives some JSON and you would like to do something with it you should approach it the following way.
Edit:
As mentioned in the comments when you use the ioutil.ReadAll()
function as in this example it will read everything that is sent in the
post request into the application memory. It's a good idea to check
this in a production application (e.g limiting payload size).
1.) Make a struct which will hold the data coming from an API post request in GoLang
2.) Convert your request body into a byte array []byte
3.) Unmarshal your []byte into a single instance of your struct made earlier.
I'll put an example below:
1. Make a struct which you'll put your JSON into.
Let's take an example of a simple blog post.
The JSON object looks like this and has a slug, a title, and description
{
"slug": "test-slug",
"title": "This is the title",
"body": "This is the body of the page"
}
So your struct would look like this:
type Page struct {
Slug string `json:"slug"`
Title string `json:"title"`
Body string `json:"body"`
}
2 - 3. Get the body of your request and convert it to byte[] Then take that string and Unmarshal it into an instance of your struct.
The data of a post request is the request 'Body'.
In Golang the request in almost all cases (unless you're using something fancy outside of the defaults) will be an http.Request object. This is the 'r' which you normally have in your normal code and it holds the 'Body' of our POST request.
import (
"encoding/json"
"github.com/go-chi/chi" // you can remove
"github.com/go-chi/render" // you can remove but be sure to remove where it is used as well below.
"io/ioutil"
"net/http"
)
func GiveMeAPage(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("A page"))
}
So what we are going to do here is convert an io.ReadCloser, which is what the http.Request.Body is, to a []byte as the Unmarshal function takes a []byte type. I've commented inline below for you.
func Create(w http.ResponseWriter, r *http.Request) {
var p Page //Create an instance of our struct
//Read all the data in r.Body from a byte[], convert it to a string, and assign store it in 's'.
s, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// use the built in Unmarshal function to put the string we got above into the empty page we created at the top. Notice the &p. The & is important, if you don't understand it go and do the 'Tour of Go' again.
err = json.Unmarshal(s, &p)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// From here you have a proper Page object which is filled. Do what you want with it.
render.JSON(w, r, p) // This is me using a useful helper function from go-chi which 'Marshals' a struct to a json string and returns it to using the http.ResponseWriter.
}
As a side note. Please don't use Decoder to parse JSON unless you are using JSON streams. You are not here, and it's unlikely you will
for a while. You can read about why that is
here
If you just want the raw JSON data without parsing it, http.Request.Body implements io.Reader, so you can just Read from it. For example with ioutil.ReadAll.
Something like (untested):
func Point(w http.ResponseWriter, r *http.Request) {
data, err := ioutil.ReadAll(r.Body)
// check error
// do whatever you want with the raw data (you can `json.Unmarshal` it too)
}
Is this possible way to add nested json "as is". The nested json doesn't have any structure and might be different. I need to put the nested json data directly to the root node.
https://play.golang.org/p/MzBt7DLQEpD
type RootJson struct {
NestedJson []byte
AdditionalField string
}
func main() {
nestedJson := []byte("{\"number\": 1, \"string\": \"string\", \"float\": 6.56}")
rootJson := RootJson{nestedJson, "additionalField"}
payload, _ := json.Marshal(&rootJson)
fmt.Println(string(payload))
}
Yes, it's possible. Use the json.RawMessage type which implements custom marshaling / unmarshaling, which "renders" it as-is into the JSON output. It's just a plain byte slice:
type RawMessage []byte
Its value should be the UTF-8 encoded byte sequence of the raw JSON text (exactly what you get when you do a conversion, e.g. []byte("someText")).
type RootJson struct {
NestedJson json.RawMessage
AdditionalField string
}
With this, the output will be (try it on the Go Playground):
{"NestedJson":{"number":1,"string":"string","float":6.56},
"AdditionalField":"additionalField"}
(Indentation added by me.)
Currently, I try to parse JSON to map[string][]interface{}, but unmarshalling returns an error. According to (https://golang.org/pkg/encoding/json/), to unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
-[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
I wonder if golang is able to unmarshal map[string][]interface{}. The following is code snippet. I am new to Golang, thanks for help in advance.
// emailsStr looks like "{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}"
emailsRaw := make(map[string][]*entities.Email)
err := json.Unmarshal([]byte(emailsStr), &emailsRaw)
Error message:
&json.UnmarshalTypeError{Value:"number", Type:(*reflect.rtype)(0x151c7a0), Offset:44, Struct:"", Field:""}
The Go encoding/json package will only unmarshal dynamically to a map[string]interface{}. From there, you will need to use type assertions and casting to pull out the values you want, like so:
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
dynamic := make(map[string]interface{})
json.Unmarshal([]byte(jsonStr), &dynamic)
firstEmail := dynamic["unknown.0"].([]interface{})[0].(map[string]interface{})["email_address"]
fmt.Println(firstEmail)
}
(https://play.golang.org/p/VEUEIwj3CIC)
Each time, Go's .(<type>) operator is used to assert and cast the dynamic value to a specific type. This particular code will panic if anything happens to be the wrong type at runtime, like if the contents of unknown.0 aren't an array of JSON objects.
The more idiomatic (and robust) way to do this in Go is to annotate a couple structs with json:"" tags and have encoding/json unmarshal into them. This avoids all the nasty brittle .([]interface{}) type casting:
type Email struct {
Email string `json:"email_address"`
}
type EmailsList struct {
IsSchemaConforming bool `json:"isSchemaConforming"`
SchemaVersion int `json:"schemaVersion"`
Emails []Email `json:"unknown.0"`
}
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
emails := EmailsList{}
json.Unmarshal([]byte(jsonStr), &emails)
fmt.Printf("%+v\n", emails)
}
(https://play.golang.org/p/iS6e0_87P2J)
A better approach will be to use struct for main schema and then use an slice of email struct for fetching the data for email entities get the values from the same according to requirements. Please find the solution below :-
package main
import (
"fmt"
"encoding/json"
)
type Data struct{
IsSchemaConforming bool `json:"isSchemaConforming"`
SchemaVersion float64 `json:"schemaVersion"`
EmailEntity []Email `json:"unknown.0"`
}
// Email struct
type Email struct{
EmailAddress string `json:"email_address"`
}
func main() {
jsonStr := `{"isSchemaConforming":true,"schemaVersion":0,"unknown.0":[{"email_address":"test1#uber.com"},{"email_address":"test2#uber.com"}]}`
var dynamic Data
json.Unmarshal([]byte(jsonStr), &dynamic)
fmt.Printf("%#v", dynamic)
}
I'm starting to go crazy trying to get Go to decode this json request body. Here's a sample request:
curl -X POST -d "{\"username\":\"foo\", \"password\":\"bar\"}" http://localhost:3000/users
And here's my handler:
mux.HandleFunc("/users", func(rw http.ResponseWriter, req *http.Request) {
var body struct {
username string
password string
}
// buf := make([]byte, req.ContentLength)
// req.Body.Read(buf)
// fmt.Println(string(buf))
//
// The above commented out code will correctly print:
// {"username":"foo", "password":"bar"}
err := json.NewDecoder(req.Body).Decode(&body)
if err != nil {
rw.WriteHeader(http.StatusNotAcceptable)
return
}
fmt.Printf("%+v\n", body)
// prints -> {username: password:}
})
Like the comment suggests, I can verify that req.Body is indeed correct -- but for whatever reason, json.NewDecoder(req.Body).Decode(&body) never fills out the fields of body.
Any help would be greatly appreciated!
The problem is that the json decoder does not deal with private struct fields. The fields in your body structs are private.
Rewrite it like so and it will work:
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
basically the json:"username" is a way to tell the json decoder how to map the json name of the object to the struct name. In this instance, for decoding only, it is not necessary - the json decoder is smart enough to make the translation of the upper/lower case.
But if you use the object to encode json as well, you need it, or you'll have upper case field names in the resulting json.
You can use the json struct tags for a few more useful things, like omitting empty field from encoded json, or skipping fields entirely.
You can read more about the JSON struct tags in the documentation for json.Marshal: http://golang.org/pkg/encoding/json/#Marshal