I have been trying to setup a JSON config file to setup dynamic routes for my application. The idea is that I will be able to setup my own URL structure depending on who is using the service. I have a struct that takes in JSON and that works fine. I am using gorilla mux.
type CustomRoute struct {
Name string
Method string
Path string
HandleFunc string
}
The JSON is basically identical to the struct and it goes in fine.
The issue I have is getting the HandleFunc section.
Here is the code:
func NewRouter() *mux.Router {
routerInstance := mux.NewRouter().StrictSlash(true)
/*
All routes from the routing table
*/
// r = []CustomRoute with the JSON data
r := loadRoute()
for _, route := range r {
var handler http.Handler
handler = route.HandlerFunc
handler = core.Logger(handler, route.Name)
routerInstance.
Methods(route.Method).
Path(route.Path).
Name(route.Name).
Handler(handler)
}
return routerInstance
}
I always get the following error (as one would expect)
cannot use route.HandlerFunc (type string) as type http.Handler in assignment:
string does not implement http.Handler (missing ServeHTTP method)
I was told to use something like:
var functions = map[string]interface{}{
"HandleFunc1": HandleFunc1,
}
But I have no idea how to make this work
Thanks to RayenWindspear I was able to fix the problem. It was very simple (like everything). The map code should look like this:
var functions = map[string]http.HandlerFunc{
"HandleFunc1": HandleFunc1,
}
I am using a mux for subdomains, so my examples may be a bit off. The map you are being told to use is done something like this:
type Handlers map[string]http.HandlerFunc
func (handlers Handlers) ServeHTTP(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if handle := handlers[path]; handle != nil {
handle.ServeHTTP(w, r)
} else {
http.Error(w, "Not found", 404)
}
}
Related
Please forgive my question, I'm new to Golang and possibly have the wrong approach.
I'm currently implementing a Terraform provider for an internal service.
As probably expected, that requires unmarshalling JSON data in to pre-defined Struct Types, e.g:
type SomeTypeIveDefined struct {
ID string `json:"id"`
Name String `json:"name"`
}
I've got myself in to a situation where I have a lot of duplicate code that looks like this
res := r.(*http.Response)
var tempThing SomeTypeIveDefined
dec := json.NewDecoder(res.Body)
err := dec.Decode(&tempThing)
In an effort to reduce duplication, I decided what I wanted to do was create a function which does the JSON unmarshalling, but takes in the Struct Type as a parameter.
I've trawled through several StackOverflow articles and Google Groups trying to make sense of some of the answers around using the reflect package, but I've not had much success in using it.
My latest attempt was using reflect.StructOf and passing in a set of StructFields, but that still seems to require using myReflectedStruct.Field(0) rather than myReflectedStruct.ID.
I suspect there may be no way until something like Generics are widely available in Golang.
I considered perhaps an interface for the structs which requires implementing an unmarshal method, then I could pass the interface to the function and call the unmarshal method. But then I'm still implementing unmarshal on all the structs anyway.
I'm just wondering what suggestions there may be for achieving what I'm after, please?
Create a helper function with the repeated code. Pass the destination value as a pointer.
func decode(r *http.Repsonse, v interface{}) error {
return json.NewDecoder(res.Body).Decode(v)
}
Call the helper function with a pointer to your thing:
var tempThing SomeTypeIveDefined
err := deocde(r, &tempThing)
You can do this with interfaces:
func decodeResponse(r *http.Response, dest interface{}) error {
dec := json.NewDecoder(r.Body)
return dec.Decode(dest)
}
func handler(...) {
res := r.(*http.Response)
var tempThing SomeTypeIveDefined
if err:=decodeResponse(res,&tempThing); err!=nil {
// handle err
}
...
}
You don't need to implement an unmarshal for the structs, because the stdlib decoder will use reflection to set the struct fields.
I'm new to Go so please bear with me if this is a trivial problem. I am using a home grown "type registry" to map type names to their type, so as to generate them dynamically based on use cases that point to the various type names (I'm basically trying for a simple solution to polymorphic Aggregation JSON response structures in Elasticsearch, but of course this could apply to many other dynamic/polymorphic situations).
I'm using the solution provided by dolmen in this question: is there a way to create an instance of a struct from a string? :
var typeRegistry = make(map[string]reflect.Type)
func registerType(typedNil interface{}) {
t := reflect.TypeOf(typedNil).Elem()
typeRegistry[t.Name()] = t
}
func init() {
registerType((*playlistIDAggregation)(nil))
registerType((*srcIDAggregation)(nil))
registerType((*assetIDAggregation)(nil))
}
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Interface()
}
I then want to use my dynamically generated struct as the target for the JSON unmarshalling of the Aggregations node in my ES response:
playlistIDAgg := makeInstance("playlistIDAggregation")
err = json.Unmarshal(esResponse.Aggregations, &playlistIDAgg)
This isn't working like I want it to, as the Unmarshal is trying to unmarshall into an empty interface instead of the underlying struct type. it's putting the data under "data" nodes in the playlistIDAgg variable, and those data fields are of course map[string]interface{}. Am I just missing the way to type assert my playlistIDAgg interface or is there a better way to do this?
EDIT---
The questions in the comments made me realize an edit to this question was long overdue.
In my particular case, the structs I defined, to bind to my Bucket aggregations returned by Elasticsearch, have a similar structure and vary only by their root JSON tag, which ES uses to name the aggregation and strongly type it. E.g.
type <name>Aggregation struct {
Agg BucketAggregationWithCamIDCardinality `json:"<name>"`
}
So, rather than a type registry, my particular problem could be solved by dynamically setting the JSON tag on the struct based on the particular use case.
In addition, a heavier but more robust option would be to leverage Oliver Eilhard's Elasticsearch Go client lib, called Elastic, which has built-in support for all the ES aggregation response structures:
https://github.com/olivere/elastic/
I changed makeInstance function by getting address of element and add target structs with exposed fields.
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Addr().Interface()
}
Here is the working code
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type playlistIDAggregation struct {
PlaylistID string
}
type srcIDAggregation struct {
SrcID string
}
type assetIDAggregation struct {
AssetID string
}
var typeRegistry = make(map[string]reflect.Type)
func registerType(typedNil interface{}) {
t := reflect.TypeOf(typedNil).Elem()
typeRegistry[t.Name()] = t
}
func init() {
registerType((*playlistIDAggregation)(nil))
registerType((*srcIDAggregation)(nil))
registerType((*assetIDAggregation)(nil))
}
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Addr().Interface()
}
func main() {
playlistIDAgg := makeInstance("playlistIDAggregation")
fmt.Printf("Type = %[1]T => %#[1]v\n", playlistIDAgg)
err := json.Unmarshal([]byte(`{"PlayListID": "dummy-id"}`), &playlistIDAgg)
if err != nil {
panic(err)
}
fmt.Printf("Type = %[1]T => %#[1]v\n", playlistIDAgg)
}
https://play.golang.org/p/dn19_iG5Xjz
I have an issue printing the JSON from the POST.
I use gorilla/mux for routing
r := mux.NewRouter()
r.HandleFunc("/test", Point).Methods("POST")
http.ListenAndServe(":80", r)`
and in Point function I have
func Point(w http.ResponseWriter, r *http.Request) {
var callback Decoder
json.NewDecoder(r.Body).Decode(&callback)
}
But I can use this method only when I know the structure and I want to figure out how to log.Print the whole JSON as a string. I tried
func Point(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
log.Println(r.Form)
}
But it prints an empty map. Please help to figure out this.
In the assumption that you are building a standard API endpoint which receives some JSON and you would like to do something with it you should approach it the following way.
Edit:
As mentioned in the comments when you use the ioutil.ReadAll()
function as in this example it will read everything that is sent in the
post request into the application memory. It's a good idea to check
this in a production application (e.g limiting payload size).
1.) Make a struct which will hold the data coming from an API post request in GoLang
2.) Convert your request body into a byte array []byte
3.) Unmarshal your []byte into a single instance of your struct made earlier.
I'll put an example below:
1. Make a struct which you'll put your JSON into.
Let's take an example of a simple blog post.
The JSON object looks like this and has a slug, a title, and description
{
"slug": "test-slug",
"title": "This is the title",
"body": "This is the body of the page"
}
So your struct would look like this:
type Page struct {
Slug string `json:"slug"`
Title string `json:"title"`
Body string `json:"body"`
}
2 - 3. Get the body of your request and convert it to byte[] Then take that string and Unmarshal it into an instance of your struct.
The data of a post request is the request 'Body'.
In Golang the request in almost all cases (unless you're using something fancy outside of the defaults) will be an http.Request object. This is the 'r' which you normally have in your normal code and it holds the 'Body' of our POST request.
import (
"encoding/json"
"github.com/go-chi/chi" // you can remove
"github.com/go-chi/render" // you can remove but be sure to remove where it is used as well below.
"io/ioutil"
"net/http"
)
func GiveMeAPage(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("A page"))
}
So what we are going to do here is convert an io.ReadCloser, which is what the http.Request.Body is, to a []byte as the Unmarshal function takes a []byte type. I've commented inline below for you.
func Create(w http.ResponseWriter, r *http.Request) {
var p Page //Create an instance of our struct
//Read all the data in r.Body from a byte[], convert it to a string, and assign store it in 's'.
s, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// use the built in Unmarshal function to put the string we got above into the empty page we created at the top. Notice the &p. The & is important, if you don't understand it go and do the 'Tour of Go' again.
err = json.Unmarshal(s, &p)
if err != nil {
panic(err) // This would normally be a normal Error http response but I've put this here so it's easy for you to test.
}
// From here you have a proper Page object which is filled. Do what you want with it.
render.JSON(w, r, p) // This is me using a useful helper function from go-chi which 'Marshals' a struct to a json string and returns it to using the http.ResponseWriter.
}
As a side note. Please don't use Decoder to parse JSON unless you are using JSON streams. You are not here, and it's unlikely you will
for a while. You can read about why that is
here
If you just want the raw JSON data without parsing it, http.Request.Body implements io.Reader, so you can just Read from it. For example with ioutil.ReadAll.
Something like (untested):
func Point(w http.ResponseWriter, r *http.Request) {
data, err := ioutil.ReadAll(r.Body)
// check error
// do whatever you want with the raw data (you can `json.Unmarshal` it too)
}
I'm making an JSON API wrapper client that needs to fetch paginated results, where the URL to the next page is provided by the previous page. To reduce code duplication for the 100+ entities that share the same response format, I would like to have a single client method that fetches and unmarshalls the different entities from all paginated pages.
My current approach in a simplified (pseudo) version (without errors etc):
type ListResponse struct {
Data struct {
Results []interface{} `json:"results"`
Next string `json:"__next"`
} `json:"d"`
}
func (c *Client) ListRequest(uri string) listResponse ListResponse {
// Do a http request to uri and get the body
body := []byte(`{ "d": { "__next": "URL", "results": []}}`)
json.NewDecoder(body).Decode(&listResponse)
}
func (c *Client) ListRequestAll(uri string, v interface{}) {
a := []interface{}
f := c.ListRequest(uri)
a = append(a, f.Data.Results...)
var next = f.Data.Next
for next != "" {
r := c.ListRequest(next)
a = append(a, r.Data.Results...)
next = r.Data.Next
}
b, _ := json.Marshal(a)
json.Unmarshal(b, v)
}
// Then in a method requesting all results for a single entity
var entities []Entity1
client.ListRequestAll("https://foo.bar/entities1.json", &entities)
// and somewehere else
var entities []Entity2
client.ListRequestAll("https://foo.bar/entities2.json", &entities)
The problem however is that this approach is inefficient and uses too much memory etc, ie first Unmarshalling in a general ListResponse with results as []interface{} (to see the next URL and concat the results into a single slice), then marshalling the []interface{} for unmarshalling it directly aftwards in the destination slice of []Entity1.
I might be able to use the reflect package to dynamically make new slices of these entities, directly unmarshal into them and concat/append them afterwards, however if I understand correctly I better not use reflect unless strictly necessary...
Take a look at the RawMessage type in the encoding/json package. It allows you to defer the decoding of json values until later. For example:
Results []json.RawMessage `json:"results"`
or even...
Results json.RawMessage `json:"results"`
Since json.RawMessage is just a slice of bytes this will be much more efficient then the intermediate []interface{} you are unmarshalling to.
As for the second part on how to assemble these into a single slice given multiple page reads you could punt that question to the caller by making the caller use a slice of slices type.
// Then in a method requesting all results for a single entity
var entityPages [][]Entity1
client.ListRequestAll("https://foo.bar/entities1.json", &entityPages)
This still has the unbounded memory consumption problem your general design has, however, since you have to load all of the pages / items at once. You might want to consider changing to an Open/Read abstraction like working with files. You'd have some Open method that returns another type that, like os.File, provides a method for reading a subset of data at a time, while internally requesting pages and buffering as needed.
Perhaps something like this (untested):
type PagedReader struct {
c *Client
buffer []json.RawMessage
next string
}
func (r *PagedReader) getPage() {
f := r.c.ListRequest(r.next)
r.next = f.Data.Next
r.buffer = append(r.buffer, f.Data.Results...)
}
func (r *PagedReader) ReadItems(output []interface{}) int {
for len(output) > len(buffer) && r.next != "" {
r.getPage()
}
n := 0
for i:=0;i<len(output)&&i< len(r.buffer);i++ {
json.Unmarshal(r.buffer[i], output[i] )
n++
}
r.buffer = r.buffer[n:]
return n
}
I try to write simple message protocol in go and i've encountered a problem. I have a lot of message types and i want to have a dictionary like this to manipulate with messages:
var dict map[reflect.Type]int = map[reflect.Type]int{
reflect.TypeOf(DataMessage{}): 1000,
reflect.TypeOf(TextMessage{}): 1001,
//....
}
func GetMessageTypeId(value interface{}) int {
if id, ok := dict[reflect.TypeOf(value)]; ok {
return id
} else {
return -1
}
}
func GetValueByTypeId(typeId int) interface{} {
for typeDec, id := range dict {
if id == typeId {
return reflect.Zero(typeDec).Interface()
}
}
fmt.Println("Unknown message type", typeId)
return nil
}
It works fine, but when i instantiate message with GetValueByTypeId and try to unmarshall json into it - i receive map[string]interface instead of my message.
I've made simple example to reproduce the problem:
http://play.golang.org/p/QEyDN9vztr
Please read this article: http://research.swtch.com/interfaces, especially the "Memory Optimizations".
The interface{} by definition consists of two pointers - to method table (e.g. type) and to data it holds. So for
var destination3 interface{} = reflect.Zero(reflect.TypeOf(Message{})).Interface()
it is empty method table (as interface{} has no methods) and reference to Message{}. Taking reference from it returns the reference to this struct so the unmarhal overwrites it with whatever matches interface{}.
If the data interface{} variable holds is a pointer itself, then it is optimized in a way that this pointer is used instead creating interface{} structure. So getting reference to it gives the reference to original variable.
http://play.golang.org/p/KsIS29rUAX
package main
import "fmt"
func main() {
var m1 struct{ Data string }
var m2 interface{}
var m3 interface{}
m2 = &m1
m3 = m1
fmt.Printf("&m1=%p m2=%p &m3=%p\n", &m1, m2, &m3)
}
In your case, using Zero is equivalent to m3 in the example above. Using New is equivalent to m2.
I've found the way how to do what i need
val := reflect.New(reflect.TypeOf(Message{}))
json.Unmarshal(data, val.Interface())
return val.Elem().Interface()
http://play.golang.org/p/8g9FSg3MSj
But was was wrong wit the first version???
It Looks like reflect.Zero(type) should be equivalent to reflect.New(type).Elem() - am i wrong?