sending JSON with go - json

I'm trying to send a JSON message with Go.
This is the server code:
func (network *Network) Join(
w http.ResponseWriter,
r *http.Request) {
//the request is not interesting
//the response will be a message with just the clientId value set
log.Println("client wants to join")
message := Message{-1, -1, -1, ClientId(len(network.Clients)), -1, -1}
var buffer bytes.Buffer
enc := json.NewEncoder(&buffer)
err := enc.Encode(message)
if err != nil {
fmt.Println("error encoding the response to a join request")
log.Fatal(err)
}
fmt.Printf("the json: %s\n", buffer.Bytes())
fmt.Fprint(w, buffer.Bytes())
}
Network is a custom struct. In the main function, I'm creating a network object and registering it's methods as callbacks to http.HandleFunc(...)
func main() {
runtime.GOMAXPROCS(2)
var network = new(Network)
var clients = make([]Client, 0, 10)
network.Clients = clients
log.Println("starting the server")
http.HandleFunc("/request", network.Request)
http.HandleFunc("/update", network.GetNews)
http.HandleFunc("/join", network.Join)
log.Fatal(http.ListenAndServe("localhost:5000", nil))
}
Message is a struct, too. It has six fields all of a type alias for int.
When a client sends an http GET request to the url "localhost:5000/join", this should happen
The method Join on the network object is called
A new Message object with an Id for the client is created
This Message is encoded as JSON
To check if the encoding is correct, the encoded message is printed on the cmd
The message is written to the ResponseWriter
The client is rather simple. It has the exact same code for the Message struct. In the main function it just sends a GET request to "localhost:5000/join" and tries to decode the response. Here's the code
func main() {
// try to join
var clientId ClientId
start := time.Now()
var message Message
resp, err := http.Get("http://localhost:5000/join")
if err != nil {
log.Fatal(err)
}
fmt.Println(resp.Status)
dec := json.NewDecoder(resp.Body)
err = dec.Decode(&message)
if err != nil {
fmt.Println("error decoding the response to the join request")
log.Fatal(err)
}
fmt.Println(message)
duration := time.Since(start)
fmt.Println("connected after: ", duration)
fmt.Println("with clientId", message.ClientId)
}
I've started the server, waited a few seconds and then ran the client. This is the result
The server prints "client wants to join"
The server prints "the json: {"What":-1,"Tag":-1,"Id":-1,"ClientId":0,"X":-1,"Y":-1}"
The client prints "200 OK"
The client crashes "error decoding the response to the join request"
The error is "invalid character "3" after array element"
This error message really confused me. After all, nowhere in my json, there's the number 3. So I imported io/ioutil on the client and just printed the response with this code
b, _ := ioutil.ReadAll(resp.Body)
fmt.Printf("the json: %s\n", b)
Please note that the print statement is the same as on the server. I expected to see my encoded JSON. Instead I got this
"200 OK"
"the json: [123 34 87 104 97 116 ....]" the list went on for a long time
I'm new to go and don't know if i did this correctly. But it seems as if the above code just printed the slice of bytes. Strange, on the server the output was converted to a string.
My guess is that somehow I'm reading the wrong data or that the message was corrupted on the way between server and client. But honestly these are just wild guesses.

In your server, instead of
fmt.Fprint(w, buffer.Bytes())
you need to use:
w.Write(buffer.Bytes())
The fmt package will format the Bytes() into a human-readable slice with the bytes represented as integers, like so:
[123 34 87 104 97 116 ... etc

You don't want to use fmt.Print to write stuff to the response. Eg
package main
import (
"fmt"
"os"
)
func main() {
bs := []byte("Hello, playground")
fmt.Fprint(os.Stdout, bs)
}
(playground link)
Produces
[72 101 108 108 111 44 32 112 108 97 121 103 114 111 117 110 100]
Use the Write() method of the ResponseWriter instead
You could have found this out by telneting to your server as an experiment - always a good idea when you aren't sure what is going on!

Related

How do I solve Golang filepath.walkfunc problem?

I'm trying to solve a task where I must to find one file with data in CSV format among other files with similar names and same size and print a number on 5th row 3rd column (indexes 4 and 2)
So I wrote this code
package main
import (
"encoding/csv"
"fmt"
"os"
"path/filepath"
)
var s [][]string
func walkfunc(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
buf, err1 := os.Open(path)
if err1 == nil {
var err2 error
r := csv.NewReader(buf)
s, err2 = r.ReadAll()
if err2 == nil {
fmt.Printf("found: %v", s[4][2])
}
}
defer buf.Close()
return nil
}
func main() {
const root = "./task/"
if err := filepath.Walk(root, walkfunc); err != nil {
fmt.Printf("error: %v", err)
}
}
And I got this in output
GOROOT=/usr/local/go #gosetup
GOPATH=/usr/local/go/bin #gosetup
/usr/local/go/bin/go build -o /private/var/folders/j2/ybr0drz13yq31dc67zmvkb1w0000gn/T/GoLand/___go_build_qwasd3_go /Users/user/Downloads/zadacha/qwasd3.go #gosetup
/private/var/folders/j2/ybr0drz13yq31dc67zmvkb1w0000gn/T/GoLand/___go_build_qwasd3_go
panic: runtime error: index out of range [4] with length 3
goroutine 1 [running]:
main.walkfunc({0x14000018120?, 0x0?}, {0x14000098d88?, 0x10247fe40?}, {0x0?, 0x0?})
/Users/user/Downloads/zadacha/qwasd3.go:23 +0x28c
path/filepath.walk({0x14000018120, 0xe}, {0x1024c9cf8, 0x140000685b0}, 0x1024c9338)
/usr/local/go/src/path/filepath/path.go:433 +0xd0
path/filepath.walk({0x10248d4a8, 0x7}, {0x1024c9cf8, 0x140000684e0}, 0x1024c9338)
/usr/local/go/src/path/filepath/path.go:457 +0x1fc
path/filepath.Walk({0x10248d4a8, 0x7}, 0x1024c9338)
/usr/local/go/src/path/filepath/path.go:520 +0x6c
main.main()
/Users/user/Downloads/zadacha/qwasd3.go:37 +0x30
Process finished with the exit code 2
What am I doing wrong?
I was trying to run this code on MacBook.
The needed file contains table with numbers and I need to print a number on 5th row and 3rd column.
As other comments have pointed out, you need to check each CSV to make sure it's actually as big as you expect it to be. You could also add a simple check to try and make sure it's a CSV file before opening it by looking for a ".csv" extension.
Though, to directly address your error... The CSV reader may be able to interpret a plain txt file as CSV and not return an err, like:
buf := strings.NewReader(`A regular text file with 3 lines.
Line2
Line3
`)
r := csv.NewReader(buf)
records, err := r.ReadAll()
if err != nil {
fmt.Println("could not read all of CSV file!")
return err
}
fmt.Println(records)
prints:
[[A regular text file with 3 lines.] [Line2] [Line3]]
Just assuming that it's a CSV with the correct number of rows and columns:
fmt.Println("found", records[4][2])
gives the panic message you shared:
panic: runtime error: index out of range [4] with length 3
You at least need to check that your CSV has 5 rows, and if it does, then check if the 5th row has 3 columns before you try to read that field:
if len(records) < 5 {
fmt.Println(path, "does not have 5 rows")
return nil
}
if len(records[4]) < 3 {
fmt.Println(path, "5th row does not have 3 columns")
return nil
}
fmt.Println("found", records[4][2])
You could also do, inside your walkfunc, a basic check of the file path itself to see if it looks like a CSV:
if strings.ToLower(path[len(path)-4:]) != ".csv" {
fmt.Println(path, "is not a CSV")
return nil
}
I show all this code, plus a fully worked/integrated example in this Playground.

Strange number when using fmt.Println in Golang

I'm new to Golang and have been doing alright but I have a strange issue that I have not encountered before when using fmt. This strange behavior is when I'm printing a string. At the end of the string (which has sub-strings) it is also printing out what appears to be the len() of each string although the number don't add up. Can anyone explain why this is happening and how to stop it?
Any help is greatly appreciated
Here is the code:
package main
import (
"fmt"
//"log"
"strings"
)
var e = "[{8888a8558921d75ec8bc362efbe9a76b82ec002337534e9f06ce92cbf8c27c8888 localhost:3303 4d50f447-7c93-42df-a03e-89c09626950a}]"
func main() {
tl := strings.Trim(e, "[{")
tr := strings.Trim(tl, "}]")
r := strings.TrimSpace(tr)
s := strings.Fields(r)
V_PK := s[0]
SERVER_ADDR := s[1]
A_KEY := s[2]
vv, _ := fmt.Printf("[{\"v_pk\": %q", V_PK)
pp, _ := fmt.Printf(",\"server_addr\": %q", SERVER_ADDR)
kk, _ := fmt.Printf(",\"a_key\": %q}] ", A_KEY)
rstr, _ := fmt.Println(vv, pp, kk)
stringc := string(rstr)
fmt.Println(stringc)
}
Expected output:
[{"v_pk": "8888a8558921d75ec8bc362efbe9a76b82ec002337534e9f06ce92cbf8c27c8888","server_addr": "localhost:3303","a_key": "4d50f447-7c93-42df-a03e-89c09626950a"}]
Actual output:
[{"v_pk": "8888a8558921d75ec8bc362efbe9a76b82ec002337534e9f06ce92cbf8c27c8888","server_addr": "localhost:3303","a_key": "4d50f447-7c93-42df-a03e-89c09626950a"}] 82 36 53
Why on earth would it be printing these string lengths on the end? It's probably obvious that I'm trying to build a JSON string so these numbers on the end are problematic when trying to import the string into a JSON interpreter.
Again, any help is appreciated!
Take a look at the documentation for fmt.Printf and its friends fmt.Println. The documentation reads:
Printf formats according to a format specifier and writes to standard output. It returns the number of bytes written and any write error encountered.
The line in your code
vv, _ := fmt.Printf("[{\"v_pk\": %q", V_PK)
prints the formatted string to standard output, then return the number of bytes written and stores that in vv. If you want to print the formatted string to standard output, just call fmt.Printf and ignore the output:
package main
import (
"fmt"
//"log"
"strings"
)
var e = "[{8888a8558921d75ec8bc362efbe9a76b82ec002337534e9f06ce92cbf8c27c8888 localhost:3303 4d50f447-7c93-42df-a03e-89c09626950a}]"
func main() {
tl := strings.Trim(e, "[{")
tr := strings.Trim(tl, "}]")
r := strings.TrimSpace(tr)
s := strings.Fields(r)
V_PK := s[0]
SERVER_ADDR := s[1]
A_KEY := s[2]
fmt.Printf("[{\"v_pk\": %q, \"server_addr\": %q, \"a_key\": %q}]\n", V_PK, SERVER_ADDR, A_KEY)
}
Or, if you want to store the formatted string to a new string variable, call fmt.Sprintf:
stringc := fmt.Sprintf("[{\"v_pk\": %q, \"server_addr\": %q, \"a_key\": %q}]", V_PK, SERVER_ADDR, A_KEY)
fmt.Println(stringc)
You can check out a working version at the playground.
You might also want to checkout the json package, which can do the parsing and serializing for you with properly defined structs:
package main
import (
"encoding/json"
"fmt"
)
func main() {
type Datum struct {
VPK string `json:"v_pk"`
Server string `json:"server_addr"`
AKey string `json:"a_key"`
}
data := []Datum{
{VPK: "8888a8558921d75ec8bc362efbe9a76b82ec002337534e9f06ce92cbf8c27c8888",
Server: "localhost:3303",
AKey: "4d50f447-7c93-42df-a03e-89c09626950a",
}}
json, err := json.MarshalIndent(data, "", " ")
if err != nil {
// deal with error
}
fmt.Println(string(json))
}
Check it out at the go playground.
fmt.Printf returns the number of bytes written. The variables vv, pp, kk are the number of bytes written by those three Printf calls, and the three numbers printed are those numbers.

How to disable default error logger in Go-Gorm

I am using GORM with MySQL, I have encountered and handled the error Error 1062: Duplicate entry. The problem is that it's still printed to the console.
Code in gym/models/auth.go:49:
func AddAuth(username, password string) error {
passwordHash, err := auth.HashPassword(password, argon2Conf)
if err != nil {
return err
}
userAuth := Auth{
Username: username,
Password: passwordHash,
}
return db.Create(&userAuth).Error
}
I am handling the error in the handler function:
func SignUpHandler(c *gin.Context) {
var form user
if err := c.ShouldBind(&form); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
if err := models.AddAuth(form.Username, form.Password); err == nil {
c.JSON(http.StatusOK, gin.H{"status": "you are signed in"})
} else {
// I think I have handled the sql error here
c.JSON(http.StatusBadRequest, gin.H{"error": "sign in failed"})
}
}
When I send a POST request, the error is correctly handled and I get the correct response with {"error": "sign in failed"}. But the console still prints this error message:
(/...../gym/models/auth.go:49)
[2019-04-28 23:37:06] Error 1062: Duplicate entry '123123' for key 'username'
[GIN] 2019/04/28 - 23:37:06 | 400 | 136.690908ms | ::1 | POST /signup
I am confused since I handled the error but it still gets printed. How to prevent this error from getting printed to the error log? Or am I handle the error correct?
UPDATE: for gorm v2:
Use the Logger in gorm.Config:
db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
For gorm v1:
Use db.LogMode to silence the default error logger.
LogMode set log mode, true for detailed logs, false for no log, default, will only print error logs.
db.LogMode(false) should do the job!
I don't have enough reputation to comment but just to add to the answer by ifnotak, you can log sql conditionally by controlling it through an environment variable. This can be handy during debugging.
gormConfig := &gorm.Config{}
if !logSql {
// I use an env variable LOG_SQL to set logSql to either true or false.
gormConfig.Logger = logger.Default.LogMode(logger.Silent)
}
db, err := gorm.Open(dialector, gormConfig)

how to parse response from AWS Go API

I am using the following sample program:
func getEnv(appName string, env string) {
svc := elasticbeanstalk.New(session.New(), &aws.Config{Region: aws.String("us-east-1")})
params := &elasticbeanstalk.DescribeConfigurationSettingsInput{
ApplicationName: aws.String(appName), // Required
EnvironmentName: aws.String(env),
}
resp, err := svc.DescribeConfigurationSettings(params)
if err != nil {
fmt.Println(err.Error())
return
}
v := resp.ConfigurationSettings
fmt.Printf("%s", v)
}
It's printing out the following response; this looks like a valid json except for the missing quote makes. ex: ApplicationName and not "ApplicationName".
How do I parse this? or get a valid json from AWS?
ConfigurationSettings: [{
ApplicationName: "myApp",
DateCreated: 2016-01-12 00:10:10 +0000 UTC,
DateUpdated: 2016-01-12 00:10:10 +0000 UTC,
DeploymentStatus: "deployed",
Description: "Environment created from the EB CLI using \"eb create\"",
EnvironmentName: "stag-myApp-app-s1",
OptionSettings: [
...
resp.ConfigurationSettings is not in JSON format any more, the aws-sdk-go package handled that for you. When you do,
v := resp.ConfigurationSettings
v contains an instance []*ConfigurationSettingsDescription that was parsed from the JSON response, and you don't have to parse it yourself. What you are seeing when you print it out is the Go struct representation. You can just go ahead and use it:
if len(v) > 0 {
log.Println(v[0].ApplicationName)
}
This should print out myApp

Golang sessions with MysqlStore not working

I am having issues setting sessions on one request and getting them on another page request.
I am using the mysqlstore package which was recommended on the gorilla sessions github page under the "Store Implementations" section in there readme file.
https://github.com/srinathgs/mysqlstore
I setup a simple request to test it out and it wasn't working. Here is the code:
HomeController
func (this *HomeController) SetCookie(ctx types.AppContext) web.HandlerType {
return func(c web.C, w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
fmt.Printf("%v\n\n", r)
ctx.SetCookies(r, w)
}
}
func (this *HomeController) GetCookie(ctx types.AppContext) web.HandlerType {
return func(c web.C, w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
fmt.Printf("%v\n\n", r)
ctx.GetCookies(r)
}
}
types.AppContext
func (this *AppContext) SetCookies(r *http.Request, w http.ResponseWriter) {
fmt.Println("Setting cookies")
session, _ := this.store.Get(r, "session-name")
// Set some session values.
session.Values["foo"] = "bar"
session.Values[42] = 43
// Save it before we write to the response/return from the handler.
err := session.Save(r, w)
if err != nil {
fmt.Println("Problem Saving session data")
}
}
func (this *AppContext) GetCookies(r *http.Request) {
session, _ := this.store.Get(r, "session-name")
fmt.Println("Getting cookies")
// get some session values
for k, v := range session.Values {
fmt.Println("Key:", k)
fmt.Println("Value:", v)
}
}
main.go
ctx.SetStore("sessions", "/", 3600, []byte("somesecret"))
....
goji.Get("/set-cookie", homeCtrl.SetCookie(ctx))
goji.Get("/get-cookie", homeCtrl.GetCookie(ctx))
When I visit both pages I get no errors and I can see in the database a session was saved. However, when I try to retrieve the saved cookie, the session.Values map is empty.
When I print out the request I get the following (formatted for easy viewing):
&{
GET /get-cookie HTTP/1.1 1 1
map[
Connection:[keep-alive]
User-Agent:[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.107 Safari/537.36]
Cookie:[session-name=MTQ0MDQ4MjU0M3xCUXdBQWpJM3x3xeRstTXbooJerGOhzP2pvEmqIkisE1XjS76zI365pA==]
Accept-Language:[en,en-CA;q=0.8,en-US;q=0.6,ja;q=0.4]
Cache-Control:[max-age=0]
Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8]
Upgrade-Insecure-Requests:[1]
Accept-Encoding:[gzip, deflate, sdch]
]
0x66a9a0 0 []
false localhost:8000 map[]
map[]
<nil>
map[]
[::1]:62375
/get-cookie
<nil>
}
As you can see I am sending a cookie and it is the "session-name" one. In Chrome when I look at the cookies I can see it locally as well.
Name: session-name
Value: MTQ0MDQ4MjU0M3xCUXdBQWpJM3x3xeRstTXbooJerGOhzP2pvEmqIkisE1XjS76zI365pA
Domain: localhost
Path: /
Expires/Max-Age: 2015-08-25T07:02:23.347Z (it was about 10:50pm - 11:00pm when I tested this out)
I have been looking at this and can't figure out why it doesn't work. This is as simple as I can make it and according to the examples this should work but it doesn't. Is there something I am missing in my code to make it work?
So I figured it out with some digging around. When I was logging some errors within the mysqlstore package I found this little error which led me to the solution:
panic: sql: Scan error on column index 2: unsupported driver -> Scan pair: []uint8 -> *time.Time
I then googled around a bit and found a discussion on the issue here:
https://github.com/go-sql-driver/mysql/issues/9
Basically I didn't read the Examples too well. This was in the packages Readme file the whole time. I forgot to add the query string ?parseTime=true&loc=Local at the end of my mysql connection credentials.
So basically the solution was this when setting up my database:
db, err := sql.Open("mysql", "username:password#/dbname?parseTime=true&loc=Local")