I'm using the Beego framework to build a web application, and I'm trying to hand it some JSON encoded data. Roughly, this is what I have:
import (
"github.com/astaxie/beego"
)
type LoginController struct {
beego.Controller
}
func (this *LoginController) Post() {
request := this.Ctx.Request
length := request.ContentLength
p := make([]byte, length)
bytesRead, err := this.Ctx.Request.Body.Read(p)
if err == nil{
//blah
} else {
//tell me the length, bytes read, and error
}
}
Per this tutorial, the above Should Just Work (tm).
My problem is this: bytesRead, err := this.Ctx.Request.Body.Read(p) is returning 0 bytes read and the err.Error() is EOF.
The request.ContentLength, however, is a sane number of bytes (19 or more, depending on what data I type in).
I can't figure out why the request would appear to have some length, but would fail on Read. Any ideas?
If you are trying to reach a JSON payload in Beego, you'll want to call
this.Ctx.Input.RequestBody
That returns a []byte array of the sent payload. You can then pass it to a function like:
var datapoint Datapoint
json.Unmarshal(this.Ctx.Input.RequestBody, &datapoint)
Where datapoint is the struct you are attempting to unmarshall your data into.
Related
Using Go, I'm attempting to fetch a few JSON responses concurrently from multiple endpoints. I'd like to attach each of these responses to fields in a struct or map and return this struct/map as a JSON object. (Backend for Frontend pattern). So I will make a web request to the Go application with some sort of identifier. It will in turn make several web requests, and compile the data into one large object to return as a response.
I'm using Fiber as my framework but any generic web framework would be similar:
app.Get("/requests/:identifier", func(c *fiber.Ctx) error {
identifier := c.Params("identifier")
timeout := 1600 * time.Millisecond
client := httpclient.NewClient(httpclient.WithHTTPTimeout(timeout))
res, err := client.Get("https://www.example.com/endpoint?id=" + identifier, nil)
if err != nil{
logger.Error("Timout value exceeded")
return c.Status(503).SendString("Socket Timeout")
}
logger.Info("Fetch success: ")
// Heimdall returns the standard *http.Response object
body, err := ioutil.ReadAll(res.Body)
code := 200
response := &main_response{
Service1: body,
}
return c.Status(code).JSON(response)
})
The problem I'm having is, I have no need to unmarshal this data in Go, as I have no use for it (I am simply passing it along). Do I have to unmarshal it just so I can set it as a field in my response struct like this?
type main_response struct {
Service1 []byte `json:"service1"`
Service2 map[string]string `json:"service2"`
Service3 map[string]interface{} `json:"service3"`
}
(I've tried a few different ways to accomplish this. Trying to use a byte array seems to base64 encode the responses)
I will want to marshal that struct to JSON before returning it, so perhaps I have little choice as I can't think of a way to tell Go "only marshal the main struct, everything else is already JSON". It almost feels like I'd be better off building a string at this point.
Use json.RawMessage to copy a []byte containing JSON directly to the response JSON document:
type main_response struct {
Service1 json.RawMessage `json:"service1"`
...
}
response := &main_response{
Service1: body,
}
return c.Status(code).JSON(response)
I have a very simple Go webserver. It's job is to receive an inbound json payload. It then publishes the payload to one or more services that expect a byte array. The payload doesn't need to be checked. Just sent over.
In this case, it receives an inbound job and sends it to Google PubSub. It might be another service - it doesn't really matter. I'm trying to find the most efficient way to convert the object to a byte array without first decoding it.
Why? Seems a bit wasteful to decode and convert to JSON on one server, only to unmarshal it later. Plus, I don't want to maintain two identical structs in two packages.
How is it possible to convert the io.ReadCloser to a byte array so I only need to unmarshal once. I tried something like this answer but don't think that's the most efficient way either:
From io.Reader to string in Go
My http server code looks like this:
func Collect(d DbManager) http.HandlerFunc {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
code := 422
obj := Report{}
response := Response{}
response.Message = "Invalid request"
decoder := json.NewDecoder(r.Body)
decoder.Decode(&obj)
if obj.Device.MachineType != "" {
msg,_ := json.Marshal(obj)
if d.Publish(msg, *Topic) {
code = 200
}
response.Message = "Ok"
}
a, _ := json.Marshal(response)
w.WriteHeader(code)
w.Write(a)
return
})
}
You convert a Reader to bytes, by reading it. There's not really a more efficient way to do it.
body, err := ioutil.ReadAll(r.Body)
If you are unconditionally transferring bytes from an io.Reader to an io.Writer, you can just use io.Copy
I'm running into an issue with attempting to manage a dynamodb instance using godynamo.
My code is meant to take a gob encoded byte array and put it into dynamodb.
func (c *checkPointManager) CommitGraph(pop *Population) {
var blob, err = pop.GobEncodeColorGraphs()
fitness := pop.GetTotalFitness()
if err != nil {
log.Fatal(err)
}
put1 := put.NewPutItem()
put1.TableName = "CheckPoint"
put1.Item["fitnessScore"] = &attributevalue.AttributeValue{N: string(fitness)}
put1.Item["population"] = &attributevalue.AttributeValue{N: string(1)}
put1.Item["graph"] = &attributevalue.AttributeValue{B: string(blob)}
body, code, err := put1.EndpointReq()
if err != nil || code != http.StatusOK {
log.Fatalf("put failed %d %v %s\n", code, err, body)
}
fmt.Printf("values checkpointed: %d\n %v\n %s\n", code, err, body)
}
Every time I run this code though, I get the following error.
can not be converted to a Blob: Base64 encoded length is expected a multiple of 4 bytes but found: 25
Does godynamo not handle making sure a binary array specifically converts to base64? Is there an easy way for me to handle this issue?
"Client applications must encode binary values in base64 format" according to the binary data type description of Amazon DynamoDB Data Types.
Your code could encode the value if you want, see golang's base64 package:
https://golang.org/pkg/encoding/base64
The godynamo library provides functions that will encode it for you, have a look at AttributeValue:
// InsertB_unencoded adds a new plain string to the B field.
// The argument is assumed to be plaintext and will be base64 encoded.
func (a *AttributeValue) InsertB_unencoded(k string) error {
How can I parse this .Net JSON date with Go?
The value comes back unassigned.
It appears to parse up to the date field.
package main
import (
"encoding/json"
"fmt"
"time"
)
type MyStruct struct {
FirstField string
SomeTime time.Time
LastField string
}
type MyStructSlice struct {
MyStructs []MyStruct
}
func main() {
var s MyStructSlice
str := `{"MyStructs":[{"FirstField":"123", "SomeTime":"\/Date(1432187580000-0500)\/", "LastField":"456"}]}`
json.Unmarshal([]byte(str), &s)
fmt.Println(s)
}
Go Playground
I am going to provide a few suggestions. You will have to write the code yourself though ;)
First of all is it possible to change .NET application that produced this JSON to generate something more parsable? If you make it output datetime in RFC3339 format (something like 1990-12-31T15:59:12-08:00) then go will automatically converts it to time.Time instance thanks to http://golang.org/pkg/time/#Time.UnmarshalJSON
If you cannot change the client then you will have to parse this date yourself:
extract time part (1432187580000) from the string. this looks like number of milliseconds (ms) since UNIX epoch. You can convert it to time.Time instance using time.Unix(sec, nsec).
(optional). The time that was created in the last step already accurately represent a point in time. However if you want to add the original timezone to it (e.g. to print it) you will need to:
parse the offset part (-0500) from the string
create time.FixedZone
call http://golang.org/pkg/time/#Time.In on the instance of time.Time created in the first step
Example: http://play.golang.org/p/Pkahyg2vZa
First you model is wrong, it doesn't model the data structure in your JSON. It should be:
type Data struct {
Data []MyStruct `json:"data`
}
type MyStruct struct {
SomeTime string
}
It works with this, try it on the Go Playground.
Problem is that we still have the time as string.
Now if you want SomeTime to be time.Time, you need to parse it yourself, you can do it by implementing json.Unmarshaler:
type Data struct {
Data []MyStruct `json:"data`
}
type MyStruct struct {
SomeTime time.Time
}
func (m *MyStruct) UnmarshalJSON(data []byte) error {
// First unmashal it into a string:
ms := struct{ SomeTime string }{}
if err := json.Unmarshal(data, &ms); err != nil {
return err
}
s := ms.SomeTime
// s is of format: "/Date(1432187580000-0500)/"
// extract millis and time zone offset hours
i1 := strings.Index(s, "(")
i3 := strings.Index(s, ")")
i2 := strings.Index(s, "-")
if i2 < 0 {
i2 = strings.Index(s, "+")
}
if i1 < 0 || i2 < 0 || i3 < 0 {
return errors.New("Invalid format")
}
millis, err := strconv.ParseInt(s[i1+1:i2], 10, 64)
if err != nil {
return err
}
m.SomeTime = time.Unix(0, millis*1000000)
// Apply timezone:
zoneHours, err := strconv.ParseInt(s[i2:i3], 10, 64)
if err != nil {
return err
}
zone := time.FixedZone("something", int(zoneHours)*3600)
m.SomeTime = m.SomeTime.In(zone)
return nil
}
Try it on the Go Playground.
The value comes back unassigned for many reasons. First of all your MyStruct does not look right. Your JSON has data key as the parent key, which consists an array of objects.
But your struct for some reason does not even resembles this. It has to have Date as a field at least. So make your struct look like the json you are receiving.
Another thing is that your time.Time will not be able to parse this string: Date(1432187580000-0500) in a normal date.
P.S. now that you have updated your struct to normal way, you have to find a way to parse your strange sting to a date. If you have power over your .net application, I would rather recommend changing json to a normal timestamp or something that can be easily parsable.
If not, then you have to change SomeTime time.Time to SomeTime string and then parse string to a normal timestamp and then parse this timestamp to a date.
I am trying to define a struct that can hold an array of any type like so:
type APIResonse struct {
length int
data []interface{}
}
I want the data property to be capable of holding an array of any type/struct so I can have a single response type, that will eventually be serialized to json. So what I want to be able to write is something like the following:
someStruct := getSomeStructArray()
res := &APIResponse{
length: len(someStruct),
data: someStruct,
}
enc, err := json.Marshal(res)
Is this possible in Go? I keep getting cannot use cs (type SomeType) as type []interface {} in assignment. Or do I have to create a different response type for every variation of data? Or maybe I am going about this wrong entirely / not Go-like. Any help would be much appreciated!
There are a couple of problems with that code.
You need to use interface{}, not []interface{}, also [] is called a slice, an array is a fixed number of elements like [10]string.
And your APIResponse fields aren't exported, so json.Marshal will not print out anything.
func main() {
d := []dummy{{100}, {200}}
res := &APIResponse{
Length: len(d),
Data: d,
}
enc, err := json.Marshal(res)
fmt.Println(string(enc), err)
}
playground