Although the output setting has been set to text
~/.aws/config
[default]
output=text
the aws-sdk-go returns json. The question is whether the output could be switched to text.
When:
aws route53 get-hosted-zone --id some-id
is run, the output looks as follows:
NAMESERVERS some-ns
NAMESERVERS some-ns1
NAMESERVERS some-ns2
NAMESERVERS some-ns3
According to the this AWS documentation one could set the configuration:
sess, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-2")},
)
One attempt was to consult this Config struct, but an Output option seems to be omitted.
How to set the output to text?
Note: an issue has added to the github page of the aws-sdk-go as well.
Example
package main
import (
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/route53"
)
func main() {
session, err := session.NewSession()
if err != nil {
log.Fatal(err)
}
r53 := route53.New(session)
listParams := &route53.ListResourceRecordSetsInput{
HostedZoneId: aws.String("some-id"),
}
records, err := r53.ListResourceRecordSets(listParams)
if err != nil {
log.Fatal(err)
}
fmt.Println(records)
}
returns:
{
IsTruncated: false,
MaxItems: "100",
ResourceRecordSets: [
{
Name: "some-domain.",
ResourceRecords: [{
Value: "some-ip"
}],
TTL: 7200,
Type: "A"
}
}
while aws route53 list-resource-record-sets --hosted-zone-id some-id, results in:
RESOURCERECORDSETS some-domain. 7200 A
RESOURCERECORDS some-ip
Problem
While it is possible to set the format of the aws-cli to output, it does not seem to be possible to do the same for the SDK.
Question
How to let the go-aws-sdk return text rather than json?
I have all of the information you need, you just have to unravel it from the response (records).
To get similar results from the last cli command:
for _, recordSet := range records.ResourceRecordSets {
log.Println("RESOURCERECORDSETS " + *recordSet.Name + strconv.Itoa(int(*recordSet.TTL)) + *recordSet.Type)
for _, record := range recordSet.ResourceRecords {
log.Println("RESOURCERECORDS " + *record.Value)
}
log.Println("")
}
Related
I'm trying to solve a task where I must to find one file with data in CSV format among other files with similar names and same size and print a number on 5th row 3rd column (indexes 4 and 2)
So I wrote this code
package main
import (
"encoding/csv"
"fmt"
"os"
"path/filepath"
)
var s [][]string
func walkfunc(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
buf, err1 := os.Open(path)
if err1 == nil {
var err2 error
r := csv.NewReader(buf)
s, err2 = r.ReadAll()
if err2 == nil {
fmt.Printf("found: %v", s[4][2])
}
}
defer buf.Close()
return nil
}
func main() {
const root = "./task/"
if err := filepath.Walk(root, walkfunc); err != nil {
fmt.Printf("error: %v", err)
}
}
And I got this in output
GOROOT=/usr/local/go #gosetup
GOPATH=/usr/local/go/bin #gosetup
/usr/local/go/bin/go build -o /private/var/folders/j2/ybr0drz13yq31dc67zmvkb1w0000gn/T/GoLand/___go_build_qwasd3_go /Users/user/Downloads/zadacha/qwasd3.go #gosetup
/private/var/folders/j2/ybr0drz13yq31dc67zmvkb1w0000gn/T/GoLand/___go_build_qwasd3_go
panic: runtime error: index out of range [4] with length 3
goroutine 1 [running]:
main.walkfunc({0x14000018120?, 0x0?}, {0x14000098d88?, 0x10247fe40?}, {0x0?, 0x0?})
/Users/user/Downloads/zadacha/qwasd3.go:23 +0x28c
path/filepath.walk({0x14000018120, 0xe}, {0x1024c9cf8, 0x140000685b0}, 0x1024c9338)
/usr/local/go/src/path/filepath/path.go:433 +0xd0
path/filepath.walk({0x10248d4a8, 0x7}, {0x1024c9cf8, 0x140000684e0}, 0x1024c9338)
/usr/local/go/src/path/filepath/path.go:457 +0x1fc
path/filepath.Walk({0x10248d4a8, 0x7}, 0x1024c9338)
/usr/local/go/src/path/filepath/path.go:520 +0x6c
main.main()
/Users/user/Downloads/zadacha/qwasd3.go:37 +0x30
Process finished with the exit code 2
What am I doing wrong?
I was trying to run this code on MacBook.
The needed file contains table with numbers and I need to print a number on 5th row and 3rd column.
As other comments have pointed out, you need to check each CSV to make sure it's actually as big as you expect it to be. You could also add a simple check to try and make sure it's a CSV file before opening it by looking for a ".csv" extension.
Though, to directly address your error... The CSV reader may be able to interpret a plain txt file as CSV and not return an err, like:
buf := strings.NewReader(`A regular text file with 3 lines.
Line2
Line3
`)
r := csv.NewReader(buf)
records, err := r.ReadAll()
if err != nil {
fmt.Println("could not read all of CSV file!")
return err
}
fmt.Println(records)
prints:
[[A regular text file with 3 lines.] [Line2] [Line3]]
Just assuming that it's a CSV with the correct number of rows and columns:
fmt.Println("found", records[4][2])
gives the panic message you shared:
panic: runtime error: index out of range [4] with length 3
You at least need to check that your CSV has 5 rows, and if it does, then check if the 5th row has 3 columns before you try to read that field:
if len(records) < 5 {
fmt.Println(path, "does not have 5 rows")
return nil
}
if len(records[4]) < 3 {
fmt.Println(path, "5th row does not have 3 columns")
return nil
}
fmt.Println("found", records[4][2])
You could also do, inside your walkfunc, a basic check of the file path itself to see if it looks like a CSV:
if strings.ToLower(path[len(path)-4:]) != ".csv" {
fmt.Println(path, "is not a CSV")
return nil
}
I show all this code, plus a fully worked/integrated example in this Playground.
I'm setting up testing in Go. I use go-sqlmock to test mysql connection. Now I try to test mysql insert logic. But the error occurs.
I want to know how to resolve this error.
server side: golang
db: mysql
web framework: gin
dao.go
func PostDao(db *sql.DB, article util.Article, uu string) {
ins, err := db.Prepare("INSERT INTO articles(uuid, title,content) VALUES(?,?,?)")
if err != nil {
log.Fatal(err)
}
ins.Exec(uu, article.TITLE, article.CONTENT)
}
dao_test.go
func TestPostArticleDao(t *testing.T) {
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
defer db.Close()
mock.ExpectExec("^INSERT INTO articles*").
WithArgs("bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c", "test", "test").
WillReturnResult(sqlmock.NewResult(1, 1))
article := util.Article{
ID: 1,
TITLE: "test",
CONTENT: "test",
}
PostDao(db, article, "bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c")
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("there were unfulfilled expections: %s", err)
}
}
I expect go test -v runs without error.
But the actual is not.
Here is the error.
=== RUN TestPostArticleDao
2019/08/31 00:08:11 call to Prepare statement with query 'INSERT INTO articles(uuid, title,content) VALUES(?,?,?)', was not expected, next expectation is: ExpectedExec => expecting Exec or ExecContext which:
- matches sql: 'INSERT INTO articles(uuid, title,content) VALUES(?,?,?)'
- is with arguments:
0 - bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c
1 - test
2 - test
- should return Result having:
LastInsertId: 1
RowsAffected: 1
exit status 1
FAIL article/api/dao 0.022s
As #Flimzy suggested, it needs to set ExpectPrepare first.
So I changed dao_test.go in this way:
prep := mock.ExpectPrepare("^INSERT INTO articles*")
prep.ExpectExec().
WithArgs("bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c", "test", "test").
WillReturnResult(sqlmock.NewResult(1, 1))
In my case it worked without asterix:
mock.ExpectExec("INSERT INTO `mytable`").WithArgs(mockdbutils.AnyTime{}, mockdbutils.AnyTime{}, nil, 4455,false).WillReturnResult(sqlmock.NewResult(int64(4455), 1))
mock.ExpectCommit()
I've been trying to get the go yaml package to parse a file with jsonlines entries.
Below is a simple example with three options of data to be parsed.
Option one is a multi-doc yaml example. Both docs parse ok.
Option two is a two jsonline example. The first line parses ok, but the second is missed.
Option three is a two jsonline example, but I've put yaml doc separators in between, to force the issue. Both of these parse ok.
From reading the yaml and json specs, I believe the second option, multiple jsonlines, ought to be handled by a yaml parser.
My questions are:
Should a YAML parser cope with jsonlines?
Am I using the go yaml package correctly?
package main
import (
"bytes"
"fmt"
"reflect"
"strings"
"gopkg.in/yaml.v2"
)
var testData = []string{
`
---
option_one_first_yaml_doc: ok_here
---
option_one_second_yaml_doc: ok_here
`,
`
{option_two_first_jsonl: ok_here}
{option_two_second_jsonl: missing}
`,
`
---
{option_three_first_jsonl: ok_here}
---
{option_three_second_jsonl: ok_here}
`}
func printVal(v interface{}, depth int) {
typ := reflect.TypeOf(v)
if typ == nil {
fmt.Printf(" %v\n", "<null>")
} else if typ.Kind() == reflect.Int || typ.Kind() == reflect.String {
fmt.Printf("%s%v\n", strings.Repeat(" ", depth), v)
} else if typ.Kind() == reflect.Slice {
fmt.Printf("\n")
printSlice(v.([]interface{}), depth+1)
} else if typ.Kind() == reflect.Map {
fmt.Printf("\n")
printMap(v.(map[interface{}]interface{}), depth+1)
}
}
func printMap(m map[interface{}]interface{}, depth int) {
for k, v := range m {
fmt.Printf("%sKey: %s Value(s):", strings.Repeat(" ", depth), k.(string))
printVal(v, depth+1)
}
}
func printSlice(slc []interface{}, depth int) {
for _, v := range slc {
printVal(v, depth+1)
}
}
func main() {
m := make(map[interface{}]interface{})
for _, data := range testData {
yamlData := bytes.NewReader([]byte(data))
decoder := yaml.NewDecoder(yamlData)
for decoder.Decode(&m) == nil {
printMap(m, 0)
m = make(map[interface{}]interface{})
}
}
}
jsonlines is newline delimited JSON. That means the individual lines are JSON, but not multiple lines and certainly not a whole file of multiple lines.
You will need to read the jsonlines input a line at a time, and those lines you should be able to process with go yaml, since YAML is a superset of JSON.
Since you also seem to have YAML end of indicator (---) lines in your test, you
need to process those as well.
When decoding a timestamp field from JSON into a struct on my local OS X machine, the Location of the time.Time field is "empty" rather than UTC. This is problematic for me running unit tests locally (vs. on a CI server where the Location is being set correctly to be UTC).
Here's the example code: https://play.golang.org/p/pb3eMbjSmv
package main
import (
"fmt"
"time"
)
func main() {
// Ignoring the err just for this example's sake!
parsed, _ := time.Parse(time.RFC3339, "2017-08-15T22:30:00+00:00")
fmt.Printf("String(): %v\n", parsed.String())
fmt.Printf("Location(): %v\n", parsed.Location())
}
which outputs
String(): 2017-08-15 22:30:00 +0000 +0000
Location():
So while the offset of the time.Time's Location appears to be correct, its timezone name is just an empty string. Running in on other machines (and The Go Playground) give the expected "UTC" location.
When I run that on my machine, I see
TimeField.String(): 2017-08-15 22:30:00 +0000 +0000
TimeField.Location():
So while the offset of the time.Time's Location appears to be correct, its timezone name is just an empty string. This is using Go 1.5:
go version go1.5 darwin/amd64
I find same behavior using my current setup on Mac and I suspect it will be same behavior on Linux (not sure through)
$ go version
go version devel +31ad583 Wed Aug 10 19:44:08 2016 +0000 darwin/amd64
To make it more deterministic, I suggest using a custom json Unmarshal like so:
package main
import (
"encoding/json"
"fmt"
"strings"
"time"
)
type Time struct {
*time.Time
}
func (t *Time) UnmarshalJSON(b []byte) error {
const format = "\"2006-01-02T15:04:05+00:00\""
t_, err := time.Parse(format, string(b))
if err != nil {
return err
}
*t = Time{&t_}
return nil
}
type Example struct {
TimeField *Time `json:"time_field"`
}
func main() {
inString := "{\"time_field\": \"2017-08-15T22:30:00+00:00\"}"
var ex Example
decoder := json.NewDecoder(strings.NewReader(inString))
decoder.Decode(&ex)
fmt.Printf("TimeField.String(): %v\n", ex.TimeField.String())
fmt.Printf("TimeField.Location(): %v\n", ex.TimeField.Location())
}
Yes, You are right. On The Go Playground the Local is set to UTC inside that sandbox:
Try this working sample code on The Go Playground:
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
fmt.Println(runtime.Version(), runtime.GOARCH, runtime.GOOS) //go1.7 amd64p32 nacl
parsed, err := time.Parse(time.RFC3339, "2017-08-15T22:30:00+00:00")
if err != nil {
panic(err)
}
fmt.Printf("String(): %v\n", parsed.String())
fmt.Printf("Location(): %v\n", parsed.Location())
}
output on The Go Playground:
go1.7 amd64p32 nacl
String(): 2017-08-15 22:30:00 +0000 UTC
Location(): UTC
And try it on your local system, output Location() is empty.
You may use utc := parsed.UTC() with the location set to UTC, like this working sample code The Go Playground:
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
fmt.Println(runtime.Version(), runtime.GOARCH, runtime.GOOS) //go1.7 amd64p32 nacl
parsed, err := time.Parse(time.RFC3339, "2017-08-15T22:30:00+00:00")
if err != nil {
panic(err)
}
fmt.Printf("String(): %v\n", parsed.String())
fmt.Printf("Location(): %v\n", parsed.Location())
utc := parsed.UTC()
fmt.Printf("String(): %v\n", utc.String())
fmt.Printf("Location(): %v\n", utc.Location())
}
Also You may use time.ParseInLocation(time.RFC3339, "2017-08-15T22:30:00+00:00", time.UTC), like this working sample code:
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
fmt.Println(runtime.Version(), runtime.GOARCH, runtime.GOOS) //go1.7 amd64p32 nacl
parsed, err := time.ParseInLocation(time.RFC3339, "2017-08-15T22:30:00+00:00", time.UTC)
if err != nil {
panic(err)
}
fmt.Printf("String(): %v\n", parsed.String())
fmt.Printf("Location(): %v\n", parsed.Location())
}
So the Location() will be UTC.
Thanks to #djd for pointing out that we can skip all the JSON/struct decoding business; the key issue is with time.Parse.
The same issue comes up here where the Location is "empty" rather than UTC (I would've expected UTC based on the docs: https://golang.org/pkg/time/#Parse
In the absence of a time zone indicator, Parse returns a time in UTC.
This answer was taken from the question as of revision 6.
I am using the following sample program:
func getEnv(appName string, env string) {
svc := elasticbeanstalk.New(session.New(), &aws.Config{Region: aws.String("us-east-1")})
params := &elasticbeanstalk.DescribeConfigurationSettingsInput{
ApplicationName: aws.String(appName), // Required
EnvironmentName: aws.String(env),
}
resp, err := svc.DescribeConfigurationSettings(params)
if err != nil {
fmt.Println(err.Error())
return
}
v := resp.ConfigurationSettings
fmt.Printf("%s", v)
}
It's printing out the following response; this looks like a valid json except for the missing quote makes. ex: ApplicationName and not "ApplicationName".
How do I parse this? or get a valid json from AWS?
ConfigurationSettings: [{
ApplicationName: "myApp",
DateCreated: 2016-01-12 00:10:10 +0000 UTC,
DateUpdated: 2016-01-12 00:10:10 +0000 UTC,
DeploymentStatus: "deployed",
Description: "Environment created from the EB CLI using \"eb create\"",
EnvironmentName: "stag-myApp-app-s1",
OptionSettings: [
...
resp.ConfigurationSettings is not in JSON format any more, the aws-sdk-go package handled that for you. When you do,
v := resp.ConfigurationSettings
v contains an instance []*ConfigurationSettingsDescription that was parsed from the JSON response, and you don't have to parse it yourself. What you are seeing when you print it out is the Go struct representation. You can just go ahead and use it:
if len(v) > 0 {
log.Println(v[0].ApplicationName)
}
This should print out myApp