With the following code, if no file argument is given, a panic is thrown for line 9 panic: runtime error: index out of range as expected.
How can I 'catch' this panic and handle it when directly when passing something to it (os.Args[1]) that causes the panic? Much like try/catch in PHP or try/except in Python.
I've had a search here on StackOverflow but I've not found anything that answers this as such.
package main
import (
"fmt"
"os"
)
func main() {
file, err := os.Open(os.Args[1])
if err != nil {
fmt.Println("Could not open file")
}
fmt.Printf("%s", file)
}
A panicking program can recover with the builtin recover() function:
The recover function allows a program to manage behavior of a panicking goroutine. Suppose a function G defers a function D that calls recover and a panic occurs in a function on the same goroutine in which G is executing. When the running of deferred functions reaches D, the return value of D's call to recover will be the value passed to the call of panic. If D returns normally, without starting a new panic, the panicking sequence stops. In that case, the state of functions called between G and the call to panic is discarded, and normal execution resumes. Any functions deferred by G before D are then run and G's execution terminates by returning to its caller.
The return value of recover is nil if any of the following conditions holds:
panic's argument was nil;
the goroutine is not panicking;
recover was not called directly by a deferred function.
Here is an example of how to use this:
// access buf[i] and return an error if that fails.
func PanicExample(buf []int, i int) (x int, err error) {
defer func() {
// recover from panic if one occured. Set err to nil otherwise.
if (recover() != nil) {
err = errors.New("array index out of bounds")
}
}()
x = buf[i]
}
Notice that more often than not, panicking is not the right solution. The Go paradigm is to check for errors explicitly. A program should only panic if the circumstances under which it panics do not happen during ordinary program executing. For instance, not being able to open a file is something that can happen and should not cause a panic while running out of memory is worth a panic. Nevertheless, this mechanism exists to be able to catch even these cases and perhaps shut down gracefully.
Go is not python, you should properly check for args before you use it:
func main() {
if len(os.Args) != 2 {
fmt.Printf("usage: %s [filename]\n", os.Args[0])
os.Exit(1)
}
file, err := os.Open(os.Args[1])
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s", file)
}
Some Golang official packages use panic/defer+recover as throw/catch, but only when they need to unwind a large call stack. In Golang's json package using panic/defer+recover as throw/catch is the most elegant solution.
from http://blog.golang.org/defer-panic-and-recover
For a real-world example of panic and recover, see the json package from the Go standard library. It decodes JSON-encoded data with a set of recursive functions. When malformed JSON is encountered, the parser calls panic to unwind the stack to the top-level function call, which recovers from the panic and returns an appropriate error value (see the 'error' and 'unmarshal' methods of the decodeState type in decode.go).
Search for d.error(
at http://golang.org/src/encoding/json/decode.go
In your example the "idiomatic" solution is to check the parameters before using them, as other solutions have pointed.
But, if you want/need to catch anything you can do:
package main
import (
"fmt"
"os"
)
func main() {
defer func() { //catch or finally
if err := recover(); err != nil { //catch
fmt.Fprintf(os.Stderr, "Exception: %v\n", err)
os.Exit(1)
}
}()
file, err := os.Open(os.Args[1])
if err != nil {
fmt.Println("Could not open file")
}
fmt.Printf("%s", file)
}
First: You wouldn't want to do this. Try-catch-style error handling is no error handling. In Go you would check len(os.Args) first and access element 1 only if present.
For the rare cases you need to catch panics (and your case is not one of them!) use defer in combination with recover. See http://golang.org/doc/effective_go.html#recover
We can manage panic without halting process using recover. By calling recover in any function using defer it will return the execution to calling function. Recover returns two values one is boolean and other one is interface to recover. Using type assertion we can get underlying error value
You can also print underlying error using recover.
defer func() {
if r := recover(); r != nil {
var ok bool
err, ok = r.(error)
if !ok {
err = fmt.Errorf("pkg: %v", r)
}
}
}()
I had to catch panics in a test case. I got redirected here.
func.go
var errUnexpectedClose = errors.New("Unexpected Close")
func closeTransaction(a bool) {
if a == true {
panic(errUnexpectedClose)
}
}
func_test.go
func TestExpectedPanic() {
got := panicValue(func() { closeTransaction(true) })
a, ok := got.(error)
if a != errUnexpectedClose || !ok {
t.Error("Expected ", errUnexpectedClose.Error())
}
}
func panicValue(fn func()) (recovered interface{}) {
defer func() {
recovered = recover()
}()
fn()
return
}
Used from https://github.com/golang/go/commit/e4f1d9cf2e948eb0f0bb91d7c253ab61dfff3a59 (ref from VonC)
Note that the recover treatment of a panic Execution error (such as attempting to index an array out of bounds trigger) might change with go 1.7 after issue 14965
See CL 21214 and its test:
runtime: make execution error panic values implement the Error interface
Make execution panics implement Error as mandated by Run-time panics (specs), instead of panics with strings.
When you recover a panic error, you would be able to do:
if _, ok := recovered.(runtime.Error); !ok {
This is still being evaluated, and as Dave Cheney. mentions:
I don't know what people are currently doing but from my POV this has been broken for a long time and nobody has complained so they are either explicitly relying on the broken behaviour, or nobody cares. Either way I think it's a good idea to avoid making this change.
Related
I am making API calls (potentially thousands in a single job) and as they return and complete, I'd like to be able to write them to a single shared file (say CSV for simplicity) instead of waiting for all of them to complete before writing.
How could I share a single csv.Writer() in a way that effectively writes to a single file shared by many threads. This may be too daunting of a task, but I was curious if there was a way to go about it.
package main
import (
"encoding/csv"
"os"
)
type Row struct {
Field1 string
Field2 string
}
func main () {
file, _ := os.Create("file.csv")
w := csv.NewWriter(file)
// Some operations to create a slice of Row structs that will contain the rows
// To write
var rowsToWrite []Row
// Now lets iterate over and write to file
// Ideally, I'd like to do this in a goroutine but not entirely sure about thread safe writes
for _, r := range rowsToWrite {
go func(row, writer) {
err := writeToFile(row, writer)
if err != nil {
// Handle error
}
}(r, w)
}
}
func writeToFile(row Row, writer ???) error {
// Use the shared writer to maintain where I am at in the file so I can append to the CSV
if err := w.Write(row); err != nil {
return err
}
return nil
}
Lots of back and forth on this one for me 🙂
I originally thought you could use the Write() method on a csv.Writer in a goroutine, but there are issues when the buffer flushes to disk as the buffer is being written to... not exactly sure.
Anyways, to get back to what you were originally asking for...
Still using the same setup to download Todo objects from https://jsonplaceholder.typicode.com, as an example:
type Todo struct {
UserID int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Completed bool `json:"completed"`
}
// toRecord converts Todo struct to []string, for writing to CSV.
func (t Todo) toRecord() []string {
userID := strconv.Itoa(t.UserID)
id := strconv.Itoa(t.ID)
completed := strconv.FormatBool(t.Completed)
return []string{userID, id, t.Title, completed}
}
// getTodo gets endpoint and unmarshalls the response JSON into todo.
func getTodo(endpoint string) (todo Todo) {
resp, err := http.Get(endpoint)
if err != nil {
log.Println("error:", err)
}
defer resp.Body.Close()
json.NewDecoder(resp.Body).Decode(&todo)
return
}
The following:
Will start one "parent" goroutine to start filling the todos channel:
inside that routine, goroutines will be started for each HTTP request and will send the response Todo on todos
the parent will wait till all the request routines are done
when they're done, the parent will close the todos channel
Meanwhile, main has moved on and is ranging over todos, picking a Todo off one-at-a-time and writing it to the CSV.
When the original, "parent" goroutine finally closes todos, the for-loop will break, the writer does a final Flush(), and the program will complete.
func main() {
todos := make(chan Todo)
go func() {
const nAPICalls = 200
var wg sync.WaitGroup
wg.Add(nAPICalls)
for i := 0; i < nAPICalls; i++ {
s := fmt.Sprintf("https://jsonplaceholder.typicode.com/todos/%d", i+1)
go func(x string) {
todos <- getTodo(x)
wg.Done()
}(s)
}
wg.Wait()
close(todos)
}()
w := csv.NewWriter(os.Stdout)
w.Write([]string{"UserID", "ID", "Title", "Completed"})
for todo := range todos {
w.Write(todo.toRecord())
}
w.Flush()
}
I would (personally) not have the same file open for writing at two separate points in the code. Depending on how the OS handles buffered writes, etc., you can end up with "interesting" things happening.
Given how you've described your goals, one might do something like (this is off the top of my head and not rigorously tested):
Create a channel to queue blocks of text (I assume) to be written - make(chan []byte, depth) - depth could be tuneable based on some tests you'd run, presumably.
Have a goroutine open a filehandle for writing on your file, then read from that queueing channel, writing whatever it gets from the channel to that file
you could then have n goroutines writing to the queueing channel, and as long as you don't exceed the channel capacity (outrun your ability to write), you might never need to worry about locks.
If you did want to use locks, then you'd need a sync.Mutex shared between the goroutines responsible for enqueueing.
Season to taste, obviously.
I am writing a program in Golang that interfaces with a modified version of the barefoot mapmatching library which returns results in json via netcat.
My in my actual code json.Unmarshal will only parse the response to the nil value of the struct. But if print the json to console (see code snippet below) and copy paste into goplayground it behaves as expected.
I am wondering if this is an encoding issue that is bypassed when I copy paste from the console as a result.
How do I get my code to process the same string as it is received from barefoot as when it is copy pasted from the console?
Here is the relevant code snippet (structs are identical to goplayground)
body := io_func(conn, cmd)
var obvs []Json_out
json.Unmarshal([]byte(body), &obvs)
fmt.Println(body)
fmt.Println(obvs)
and io_func() if relevant (the response is two lines, with a message on the first and a json string on the second)
func io_func(conn net.Conn, cmd string) string {
fmt.Fprintf(conn, cmd+"\n")
r := bufio.NewReader(conn)
header, _ := r.ReadString('\n')
if header == "SUCCESS\n" {
resp, _ := r.ReadString('\n')
return resp
} else {
return ""
}
}
Following Cerise Limón's advice to properly handle error messages I determined the osm_id value in the JSON was being parsed by json.Unmarshall as number when taking the string from io_func(), although it wasn't doing so when the string was passed in manually in the playground example. Although I don't understand why this is so I would have picked it up with proper error handling.
I altered barefoot code to return the osm_id explicitly in inverted commas since, although only ever composed of digits, I only use it as a string. It now works as expected. Equally I could have changed the type in the struct and convert in Go as needed.
The io_func function creates and discards a bufio.Reader and data the reader may have buffered. If the application calls io_func more than once, then the application may be discarding data read from the network. Fix by creating a single bufio.Reader outside the function and pass that single reader to each invocation of io_func.
Always check and handle errors. The error returned from any of these functions may point you in the right direction for a fix.
func io_func(r *bufio.Reader, conn net.Conn, cmd string) (string, error) {
fmt.Fprintf(conn, cmd+"\n")
header, err := r.ReadString('\n')
if err != nil {
return "", err
}
if header == "SUCCESS\n" {
return r.ReadString('\n')
}
return "", nil
}
...
r := bufio.NewReader(conn)
body, err := io_func(r, conn, cmd)
if err != nil {
// handle error
}
var obvs []Json_out
err = json.Unmarshal([]byte(body), &obvs)
if err != nil {
// handle error
}
fmt.Println(body)
fmt.Println(obvs)
// read next
body, err = io_func(r, conn, cmd)
if err != nil {
// handle error
}
The application uses newline to terminate the JSON body, but newline is valid whitespace in JSON. If the peer includes a newline in the JSON, then the application will read a partial message.
I'm a beginner with Go, and I'm now writing a function which can call an API. The function receives a part of the url (/user, /account, etc) and the struct to convert the returned json to (the structs User or Account for example) as arguments.
So I now have this:
func (self *RestClient) request(action string, return_type interface{}) interface{} {
res, _ := goreq.Request{Uri:self.url + action}.Do()
var item return_type
res.Body.FromJsonTo(&item)
return item
}
And I try to call this function using (with Index being the struct for the return type):
self.request("/api/v1/public/index", Index)
But this doesn't work. I get the following errors:
return_type is not a type
type Index is not an expression
I guess I understand why this is. I think I have to somehow find out the type of return_type and then convert return_type to that type, before using it as the type for item. I have no idea how though.
About the second error I have no idea. I don't even understand what is meant by it.
Could anybody help me out? How can I make this work? Or should this be done in a completely different way all together? All tips are welcome!
A few hints based on this code:
Don't use self - use a meaningful identifier
Don't use interface{} to avoid dealing with the type system
Don't use reflection
Don't ignore errors returned (as from FromJsonTo or goreq.Request)
Don't use a library like goreq unless you are sure you need it (you don't) - you are pulling in lots of code you don't need and it is teaching you bad habits like attempting to use empty interface and reflection to solve simple problems.
Have a look at the definition of FromJsonTo - if you look through this library you'll see it isn't saving you much effort and is adding lots of complexity. Here is how you could do it without the library:
func (c *RestClient) Request(action string, resource interface{}) error {
res, err := http.Get(c.url + action)
if err != nil {
return err
}
defer res.Body.Close()
return json.NewDecoder(res.Body).Decode(resource)
}
Or use an interface and move the decoding to the resource (which could embed a default decoder):
type Decoder interface {
Decode(r io.Reader) error
}
// RequestDecode fetches a request and feeds it to the decoder
func (c *RestClient) RequestDecode(action string, resource Decoder) error {
res, err := http.Get(c.url + action)
if err != nil {
return err
}
defer res.Body.Close()
return resource.Decode(res.Body)
}
I'll first say that you should always check for errors from any function that possibly returns one.
The error you are seeing is because you are trying to declare a variable item as type return_type and that is the name of a function argument.
The other error is from Index being a type declaration and not a concrete value of the Index type.
I agree with Volker's comment but to put it in code, you could use something like this:
func (self *RestClient) request(action string, return_type interface{}) {
res, err := goreq.Request{Uri:self.url + action}.Do()
if err != nil {
// Do something with error here.
}
res.Body.FromJsonTo(return_type)
}
var index Index
rest_client.request("/some/path", &index)
This allows flexibility but could lead to strange cases if you forget to pass a pointer to the value return_type.
I've tried to create a function in Go that used to retry any fail query functions (usually because serialization issue).
func retryer(functionA func(interface{}) (interface{}, []error), maxRetry int, waitBetween time.Duration) interface{} {
//when no error in functionA, retryer returns whatever functionA returns
//when maxRetry is reached, returns nil
}
The functions I want to retry are looked like this
func GetTopStudent(classId string) ([]Student, []error) {
//queries top 10 students for class with classId
}
func GetAverageStudentScores(classId string, from time.Time, until time.Time) ([]Pair, []error) {
//queries all average score using aggregate, grouping by studentId
//Pair behaves like C++ pair<string,string>
}
But, the results is a compile error
cannot use GetTopStudent (type func(string) ([]Student, []error)) as type func(interface{}) (interface {}, []error) in argument to retryer
I've tried to modify it a little and I got another compile error
cannot use GetTopStudent (type func(string) ([]Student, []error)) as type func(string) (interface {}, []error) in argument to retryer
Can anyone help me creating a general function to wrap a function to retry on error?
A better way to solve your problem would be to use closures.
For example, change the type of retryer:
func retryer(f func() error, maxRetry int, waitBetween time.Duration) error {
// retry and wait logic
err := f()
// error handling, retry, and wait logic
return err
}
Now call functions to be retried as:
// ...
classId := "some value"
// ...
var st []Student
var errors []error
err := retryer(func() error {
st, errors = GetTopStudent(classId)
// handle errors
return nil
}, numTries, waitTime)
// use st here
All the answers for something like this are in Javascript, and I'm not sure if it applies in Go.
I've done this
func main() {
db, err := sql.Open("mysql", "db_details")
err = db.Ping()
if err != nil {
fmt.Println("Failed to prepare connection to database")
// log.Fatal("Error:", err.Error())
}
apex.HandleFunc(func(event json.RawMessage, ctx *apex.Context) (interface{}, error) {
fmt.Println(ctx)
return map[string]string{"hello": "world"}, nil
})
}
So I'm trying to hit my Amazon RDS MySql db using golang's sql driver.
I get this error
Error: function response: Response Id: <some_id> Process exited before completing request
From looking around, there are two causes - 1. I need Go's equivalent of context.done, or 2. I need to raise the timeout.
As I'm using Apex, I raised the timeout to be 300s, which is the maximum. No luck there.
I then tried going through the Apex code to see if there was a Context.Done defined or used anywhere - there isn't.
How do I get around this?