I'm using gin framework and trying do crud operation using grom.I'm trying to get data from MYSQL database. i have db.go to get database instances, some controllers to each table and models
i have a model like this
type Campaigns struct {
ID int `json:"id" form:"id" gorm:"column:CampaignID"`
UserID int `json:"userId" form:"userId" gorm:"column:UserID"`
Name string `json:"name" form:"name" gorm:"column:Name"`
StartDate time.Time `json:"start" form:"start" gorm:"column:StartDate"`
EndDate time.Time `json:"end" form:"end" gorm:"column:EndDate"`
Customer string `json:"customer" form:"customer" gorm:"column:Customer"`
CustomerID int `json:"customerId" form:"customerId" gorm:"column:CustomerID"`
ImpressionsCounter int `json:"ImpressionsCounter" form:"ImpressionsCounter" gorm:"column:ImpressionsCounter"`
MaxImpressions int `json:"maxImpressions" form:"maxImpressions" gorm:"column:MaxImpressions"`
CurrentSpend float64 `json:"currentSpend" gorm:"column:CurrentSpend"`
MaxSpend float64 `json:"maxSpend" form:"maxSpend" gorm:"column:MaxSpend"`
Active bool `json:"active" form:"active" gorm:"column:Active"`
Created time.Time `json:"created" gorm:"column:DateCreated"`
Updated time.Time `json:"updated" gorm:"column:DateCreated"`
}
this is a one controller I'm using
package controllers
import (
"time"
"github.com/op/go-logging"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/go-sql-driver/mysql"
"../models"
)
var log = logging.MustGetLogger("AsAPI")
type AsController struct {
DB gorm.DB
}
func (ac *AsController) SetDB(d gorm.DB) {
ac.DB = d
ac.DB.LogMode(true)
}
// Get all table
func (ac *AsController) ListTable(c *gin.Context) {
var results []models.Campaigns
err := ac.DB.Find(&results)
if err != nil {
log.Debugf("Error when looking up Table, the error is '%v'", err)
res := gin.H{
"status": "404",
"error": "No Table found",
}
c.JSON(404, res)
return
}
content := gin.H{
"status": "200",
"result": "Success",
"Table": results,
}
c.Writer.Header().Set("Content-Type", "application/json")
c.JSON(200, content)
}
To get database connection I'm using
package controllers
import (
"time"
"github.com/op/go-logging"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/go-sql-driver/mysql"
"../models"
)
var log = logging.MustGetLogger("AdsAPI")
type AsController struct {
DB gorm.DB
}
func (ac *AsController) SetDB(d gorm.DB) {
ac.DB = d
ac.DB.LogMode(true)
}
and I'm using following routs
ac := controllers.AdsController{}
ac.SetDB(dc.GetDB())
// Get a Ads resource
router := gin.Default()
router.GET("/table", ac.ListTables)
when i run this I'm getting following error
(/api/controllers/table.go:30)
[2016-03-23 09:56:39] [0.99ms] SELECT * FROM `tables`
2016/03/23 09:56:39 Error when looking up tables, the error is '&{0xc8202140e0 sql: Scan error on column index 3: unsupported driver -> Scan pair: []uint8 -> *time.Time 1 <nil> 0xc82022f860 0xc82022f7c0 0xc82021e140 2 {0xc8201fb4a0} <nil> false map[] map[]}'
[GIN] 2016/03/23 - 09:56:39 | 404 | 1.153811ms | 127.0.0.1 | GET /table
what is the reason for this error ? help me to fix this error ?
You can find the answer in the driver documentation
https://github.com/go-sql-driver/mysql#timetime-support:
The default internal output type of MySQL DATE and DATETIME values is []byte which allows you to scan the value into a []byte, string or sql.RawBytes variable in your programm.
However, many want to scan MySQL DATE and DATETIME values into time.Time variables, which is the logical opposite in Go to DATE and DATETIME in MySQL. You can do that by changing the internal output type from []byte to time.Time with the DSN parameter parseTime=true. You can set the default time.Time location with the loc DSN parameter.
Alternatively you can use the NullTime type as the scan destination, which works with both time.Time and string / []byte.
Are you ever actually opening the database here? I don't see an actual Open() call in there at all...
Related
I am trying to setup a REST API using Gin and Gorm. Following is my main.go:
package main
import (
"app/config"
"app/service"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
// Connect to database
config.ConnectDatabase()
r.GET("/books", service.FindBooks)
// Run the server
r.Run()
}
And in my config/setup.go, I am trying to connect to DB like so
package config
import (
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/mysql"
)
var DB *gorm.DB
// ConnectDatabase : Setup connection to database
func ConnectDatabase() {
database, err := gorm.Open("mysql", "root:password#tcp(127.0.0.1:3306)/test_db")
database.SingularTable(true)
if err != nil {
panic("Failed to connect to database!")
}
DB = database
}
In my service/book.go, I have the following logic:
package service
import (
"errors"
"app/config"
"app/model"
"fmt"
"net/http"
"github.com/gin-gonic/gin"
)
// FindBooks : Get all books
func FindBooks(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"data": FindAllBooks()})
}
// FindAllBooks : Fetch all book from DB
func FindAllBooks() []model.Book {
var book []model.Book
config.DB.Find(&book)
for i := 0; i < len(book); i++ {
fmt.Println(book[i])
}
return book
}
And my model/Book.go is defined as follows:
package model
type Book struct {
id int64 `json:"id" gorm:"primary_key"`
label string `json:"label" gorm:"type:varchar(255)"`
}
When I run the application using go run main.go, following is the log I can see:
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET /books --> feedconsumer/service.FindBooks (3 handlers)
[GIN-debug] Environment variable PORT is undefined. Using port :8080 by default
[GIN-debug] Listening and serving HTTP on :8080
{0 }
[GIN] 2021/03/09 - 12:43:43 | 200 | 2.313864ms | ::1 | GET "/books"
Basically, the {0 } means the object is not fetched actually. What am I missing here?
GORM uses snake_case for defining field of models. Link to docs
So for your case, it is basically:
package model
type Book struct {
ID int64 `json:"id" gorm:"primary_key"`
Label string `json:"label" gorm:"type:varchar(255)"`
}
I guess you could configure logger for GORM like https://gorm.io/docs/logger.html to be able to see actual db query in those logs. At first glance you are doing everything in the right way as described in https://gorm.io/docs/query.html#Retrieving-all-objects
but I personally met some unpredictable GORM behavior which was not so obvious from doc
I'm using Mysql 8. I'm also utilizing 99designs/gqlgen to autogenerate the structs based on the GraphQL schema.
I was to re-use the same structs when scanning MySql responses. And on top of that, while prototyping, I want to have some JSONs in my table.
So the struct is:
type CustomizedItemInput struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus []*CustomizedComponent `json:"skus"`
...
Since storing(providing Value()) is simpler I managed to store Skus into DB as a top-level JSON successfully. Looks like this:
[{"sku": "123", "position": "LEFT"}, {"sku": "456", "position": "RIGHT"}]
Now, how do I get this value out of DB and back into a array of pointers inside the struct without much hustle?
Of course, ideally it should be done without changing the underlying struct because it's autogenerated.
UPDATE:
Adding debugging information. I need to read a DB row into CustomizedItemView which basically mirrors CustomizedItemInput from above:
type CustomizedItemView struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus []*CustomizedComponentView `json:"skus"`
...
Of course, when I say "without hustle" I mean having the DB row extracted into a struct seamlessly. I can add map[string]interface{}{} with all bells and whistles and get the value. But I want to have it neat, like:
var storedCustItem = model.CustomizedItemView{}
err := udb.Get(&storedCustItem, database.SelectCustomizationQuery, userID, custItem.Sku, createdAt)
The error I get is:
2020/10/10 20:38:24 sql: Scan error on column index 8, name "skus": unsupported Scan, storing driver.Value type []uint8 into type *[]*model.CustomizedComponentView
(8 because I removed some fields for the example).
The main problem is that I can't create Scan() for an unnamed type. I have created wrappers for Value() because my inserts are more verbose and I do type conversion with the wrapper type in them:
type CustomizedComponentsIn []*CustomizedComponent
...
func (customizedComponents CustomizedComponentsIn) Value() (driver.Value, error)
...
tx.MustExec(database.SaveCustomizationCommand,
custItem.Sku,
custItem.Name,
model.CustomizedComponentsIn(custItem.Skus)
...
,which is Ok for inserts because there will be some values that do not belong to the input struct.
But I hoped to at least get the value scanned into a View struct automatically.
If you can change the type of the Skus field, the common approach would be to declare a slice type that implements the sql.Scanner and driver.Valuer interfaces and use that instead of the unnamed []*CustomizedComponent type.
For example:
type CustomizedItemInput struct {
Sku string `json:"sku"`
Name string `json:"name"`
Skus CustomizedComponentSlice `json:"skus"`
// ...
}
type CustomizedComponentSlice []*CustomizedComponent
// Value implements driver.Valuer interface.
func (s CustomizedComponentSlice) Value() (driver.Value, error) {
return json.Marshal(s)
}
// Scan implements sql.Scanner interface.
func (s *CustomizedComponentSlice) Scan(src interface{}) error {
var data []byte
switch v := src.(type) {
case string:
data = []byte(v)
case []byte:
data = v
default:
return nil
}
return json.Unmarshal(data, s)
}
If you can't change type of the Skus field you will have to explicitly convert the field during scanning.
For example, given the above named slice type, you could do something like this:
v := new(CustomizedItemView)
row := db.QueryRow("SELECT sku, name, skus FROM customized_item_view WHERE sku = ? LIMIT 1", sku)
err := row.Scan(
&v.Sku,
&v.Name,
// do the conversion here, and any other place where you're scanning Skus...
(*CustomizedComponentSlice)(&v.Skus),
)
if err != nil {
return err
}
fmt.Println(v.Skus) // result
I'm building a web server with Go and I don't know how to process JSON with Go.
Saying that I have a struct as below:
type User struct{
Id int
Name string
Password string
Status int
}
and now I have had an object of the struct User:
user := User{1, "Test", "password", 1}
Now I need to convert user to a JSON object. Here is what I've found:
b, err := json.Marshal(user)
fmt.Println(string(b))
It works well.
Now I want to do two things:
1) remove the Password from the JSON object
2) add a new filed: "code": 200 into the JSON object
What should I do?
If you want to keep the Password property accessible to outer packages, you can set a tag: json:"-" on it. As specified in the docs:
The encoding of each struct field can be customized by the format
string stored under the "json" key in the struct field's tag. The
format string gives the name of the field, possibly followed by a
comma-separated list of options. The name may be empty in order to
specify options without overriding the default field name.
The "omitempty" option specifies that the field should be omitted from
the encoding if the field has an empty value, defined as false, 0, a
nil pointer, a nil interface value, and any empty array, slice, map,
or string.
As a special case, if the field tag is "-", the field is always
omitted. Note that a field with name "-" can still be generated using
the tag "-,".
type User struct {
Id int
Name string
Password string `json:"-"`
Status int
Code int `json:"code"`
}
make Password lower case (and add Code int to your struct):
Try this:
package main
import (
"encoding/json"
"fmt"
)
func main() {
user := User{1, "Test", "password", 1, 200}
b, err := json.Marshal(user)
if err != nil {
panic(err)
}
fmt.Println(string(b))
}
type User struct {
Id int
Name string
password string
Status int
Code int
}
output:
{"Id":1,"Name":"Test","Status":1,"Code":200}
I'm getting really confused by all the different types in Go, but I have a strictly defined struct 'VMR' and I'm trying to do a convert data to it.
I'm querying CouchDB (using the Go SDK) and then trying to assert the returned data into my struct. Of course this isn't working and it's throwing a panic. I'm shooting in the dark trying to figure out what I'm doing wrong.
Here is my function/struct:
type VMR struct {
Name string `json:"name,omitempty"`
InUse bool `json:"inuse"`
Description string `json:"description,omitempty"`
View string `json:"view,omitempty"`
Theme string `json:"theme,omitempty"`
Alias1 int `json:"alias1,omitempty"`
Alias1_Description string `json:"alias1_description,omitempty"`
Host_PIN int `json:"host_pin,omitempty"`
Allow_Guests bool `json:"allow_guests,omitempty"`
Guest_Pin int `json:"guest_pin,omitempty"`
Alias2 int `json:"alias2,omitempty"`
Alias2_Description string `json:"alias2_description,omitempty"`
}
func Get() VMR {
cluster, _ := gocb.Connect("couchbase://ip:port")
bucket, _ := cluster.OpenBucket("bucket", "password")
myQuery := gocb.NewN1qlQuery("SELECT * FROM `bucket` WHERE name='test42' LIMIT 1")
rows, _ := bucket.ExecuteN1qlQuery(myQuery, nil)
var row interface{}
rows.One(&row)
fmt.Printf("Query1: %+v\n", row)
return row.(VMR)
}
Full output:
Query1: map[115:map[alias1_description:Alias 1 description alias2_description:Alias description description:This is the best VMR name:test42 view:ViewOption theme:Theme51 alias1:1 alias2:1 guest_pin:1 host_pin:1 inuse:false]]
Error:
panic: interface conversion: interface is map[string]interface {}, not models.VMR
The viewResults.One(interface{}) call is a wrapper around Next(interface{}) which calls json.Unmarshal under the covers:Github link to relevant code. If you offer an interface{} object to Unmarshal, it will return to you a map[string]interface{} because it has no other option. Try this:
var row VMR
rows.One(&row)
fmt.Printf("Query1: %+v\n", row)
return row
...and that should handle the Unmarshal correctly.
How can I parse this .Net JSON date with Go?
The value comes back unassigned.
It appears to parse up to the date field.
package main
import (
"encoding/json"
"fmt"
"time"
)
type MyStruct struct {
FirstField string
SomeTime time.Time
LastField string
}
type MyStructSlice struct {
MyStructs []MyStruct
}
func main() {
var s MyStructSlice
str := `{"MyStructs":[{"FirstField":"123", "SomeTime":"\/Date(1432187580000-0500)\/", "LastField":"456"}]}`
json.Unmarshal([]byte(str), &s)
fmt.Println(s)
}
Go Playground
I am going to provide a few suggestions. You will have to write the code yourself though ;)
First of all is it possible to change .NET application that produced this JSON to generate something more parsable? If you make it output datetime in RFC3339 format (something like 1990-12-31T15:59:12-08:00) then go will automatically converts it to time.Time instance thanks to http://golang.org/pkg/time/#Time.UnmarshalJSON
If you cannot change the client then you will have to parse this date yourself:
extract time part (1432187580000) from the string. this looks like number of milliseconds (ms) since UNIX epoch. You can convert it to time.Time instance using time.Unix(sec, nsec).
(optional). The time that was created in the last step already accurately represent a point in time. However if you want to add the original timezone to it (e.g. to print it) you will need to:
parse the offset part (-0500) from the string
create time.FixedZone
call http://golang.org/pkg/time/#Time.In on the instance of time.Time created in the first step
Example: http://play.golang.org/p/Pkahyg2vZa
First you model is wrong, it doesn't model the data structure in your JSON. It should be:
type Data struct {
Data []MyStruct `json:"data`
}
type MyStruct struct {
SomeTime string
}
It works with this, try it on the Go Playground.
Problem is that we still have the time as string.
Now if you want SomeTime to be time.Time, you need to parse it yourself, you can do it by implementing json.Unmarshaler:
type Data struct {
Data []MyStruct `json:"data`
}
type MyStruct struct {
SomeTime time.Time
}
func (m *MyStruct) UnmarshalJSON(data []byte) error {
// First unmashal it into a string:
ms := struct{ SomeTime string }{}
if err := json.Unmarshal(data, &ms); err != nil {
return err
}
s := ms.SomeTime
// s is of format: "/Date(1432187580000-0500)/"
// extract millis and time zone offset hours
i1 := strings.Index(s, "(")
i3 := strings.Index(s, ")")
i2 := strings.Index(s, "-")
if i2 < 0 {
i2 = strings.Index(s, "+")
}
if i1 < 0 || i2 < 0 || i3 < 0 {
return errors.New("Invalid format")
}
millis, err := strconv.ParseInt(s[i1+1:i2], 10, 64)
if err != nil {
return err
}
m.SomeTime = time.Unix(0, millis*1000000)
// Apply timezone:
zoneHours, err := strconv.ParseInt(s[i2:i3], 10, 64)
if err != nil {
return err
}
zone := time.FixedZone("something", int(zoneHours)*3600)
m.SomeTime = m.SomeTime.In(zone)
return nil
}
Try it on the Go Playground.
The value comes back unassigned for many reasons. First of all your MyStruct does not look right. Your JSON has data key as the parent key, which consists an array of objects.
But your struct for some reason does not even resembles this. It has to have Date as a field at least. So make your struct look like the json you are receiving.
Another thing is that your time.Time will not be able to parse this string: Date(1432187580000-0500) in a normal date.
P.S. now that you have updated your struct to normal way, you have to find a way to parse your strange sting to a date. If you have power over your .net application, I would rather recommend changing json to a normal timestamp or something that can be easily parsable.
If not, then you have to change SomeTime time.Time to SomeTime string and then parse string to a normal timestamp and then parse this timestamp to a date.