Gorm - Not fetching data - mysql

I am trying to setup a REST API using Gin and Gorm. Following is my main.go:
package main
import (
"app/config"
"app/service"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
// Connect to database
config.ConnectDatabase()
r.GET("/books", service.FindBooks)
// Run the server
r.Run()
}
And in my config/setup.go, I am trying to connect to DB like so
package config
import (
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/mysql"
)
var DB *gorm.DB
// ConnectDatabase : Setup connection to database
func ConnectDatabase() {
database, err := gorm.Open("mysql", "root:password#tcp(127.0.0.1:3306)/test_db")
database.SingularTable(true)
if err != nil {
panic("Failed to connect to database!")
}
DB = database
}
In my service/book.go, I have the following logic:
package service
import (
"errors"
"app/config"
"app/model"
"fmt"
"net/http"
"github.com/gin-gonic/gin"
)
// FindBooks : Get all books
func FindBooks(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"data": FindAllBooks()})
}
// FindAllBooks : Fetch all book from DB
func FindAllBooks() []model.Book {
var book []model.Book
config.DB.Find(&book)
for i := 0; i < len(book); i++ {
fmt.Println(book[i])
}
return book
}
And my model/Book.go is defined as follows:
package model
type Book struct {
id int64 `json:"id" gorm:"primary_key"`
label string `json:"label" gorm:"type:varchar(255)"`
}
When I run the application using go run main.go, following is the log I can see:
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET /books --> feedconsumer/service.FindBooks (3 handlers)
[GIN-debug] Environment variable PORT is undefined. Using port :8080 by default
[GIN-debug] Listening and serving HTTP on :8080
{0 }
[GIN] 2021/03/09 - 12:43:43 | 200 | 2.313864ms | ::1 | GET "/books"
Basically, the {0 } means the object is not fetched actually. What am I missing here?

GORM uses snake_case for defining field of models. Link to docs
So for your case, it is basically:
package model
type Book struct {
ID int64 `json:"id" gorm:"primary_key"`
Label string `json:"label" gorm:"type:varchar(255)"`
}

I guess you could configure logger for GORM like https://gorm.io/docs/logger.html to be able to see actual db query in those logs. At first glance you are doing everything in the right way as described in https://gorm.io/docs/query.html#Retrieving-all-objects
but I personally met some unpredictable GORM behavior which was not so obvious from doc

Related

How to resolve Too many connections

when I run this I get Too many connections error unfortunately I couldn't find the solution, I will generate 50 million data
main.go content
package main
import (
"dopinghafiza-videoplayer-mysql/mysql"
_ "github.com/go-sql-driver/mysql"
)
func main() {
var player mysql.Player
player.Muted = 1
player.UserId = 55
player.PlayBackRate = "2.55"
player.Volume = "4.5888"
player.Volume = "2.55"
connection := mysql.GetConnection()
for i := 0; i < 100000; i++ {
player.Muted = int8(i % 2)
player.UserId = i
mysql.InsertPlayer(player, connection)
}
}
insert.go file content
package mysql
import (
"database/sql"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
type Player struct {
UserId int
Muted int8
PlayBackRate string
Volume string
}
func InsertPlayer(player Player, db *sql.DB) {
res := db.QueryRow("insert into players (muted,user_id,volume,play_back_rate) values (?,?,?,?)", player.Muted, player.UserId, player.Volume, player.PlayBackRate)
db.Close()
fmt.Println(res)
}
You should not close connections; Go comes indeed with some connection pooling.
You should also use Exec() instead of QueryRow, because you are keeping connection open since MySQL still has to send data (although it looks odd).
func InsertPlayer(player Player, db *sql.DB) {
res, err := db.Exec("insert into players (muted,user_id,volume,play_back_rate) values (?,?,?,?)",
player.Muted,
player.UserId,
player.Volume,
player.PlayBackRate)
if err != nil {
// and handle errors!
}
fmt.Println(res.RowsAffected())
}
You shouldn't open and close the connection each time; that's wrong and has a huge cost for your application.
A better method is to use a singleton pattern, open the connection once, use it several times, and close the connection when your application closes, not after each query.
You can find the sample code for the singleton pattern here.
How to create singleton DB class in GoLang
Login to mysql using mysql -u root -p
Use this command to see the value of max_connections show variables like %max_conne%;
Increase the value of max_connections using set global max_connections = 4096;
This will solve your issue.

JSON functions in SQLite

I am trying to get the SQLite JSON functions to work with Go and the github.com/mattn/go-sqlite3 driver but not having much luck.
Using VSCode as my editor I did:
go get github.com/mattn/go-sqlite3
go build --tags "json1" github.com/mattn/go-sqlite3
Wrote my test app which calls the following function
func IsJSONValid(payloadID int) bool {
var isValid int
sql := `select json_valid(json_payload) from exec_payload where payload_id = ?`
err = database.QueryRow(sql, payloadID).Scan(&isValid)
if err != nil {
log.Println(err)
log.Println("Fatal - unable to validate JSON payload")
LogDb("IsJSONValid", "unable to validate JSON payload")
return false
}
if isValid == 0 {
return false
}
return true
}
Executing the code/func I get the following error:
no such function: json_valid
I know I can write a Struct and handle the logic to validate the json data stored in the sqlite table column - however it's pretty convenient to use / have the function already available in the DB - running the same query in the sqlite command line tool executes without any issues.

Golang JSON config for routes

I have been trying to setup a JSON config file to setup dynamic routes for my application. The idea is that I will be able to setup my own URL structure depending on who is using the service. I have a struct that takes in JSON and that works fine. I am using gorilla mux.
type CustomRoute struct {
Name string
Method string
Path string
HandleFunc string
}
The JSON is basically identical to the struct and it goes in fine.
The issue I have is getting the HandleFunc section.
Here is the code:
func NewRouter() *mux.Router {
routerInstance := mux.NewRouter().StrictSlash(true)
/*
All routes from the routing table
*/
// r = []CustomRoute with the JSON data
r := loadRoute()
for _, route := range r {
var handler http.Handler
handler = route.HandlerFunc
handler = core.Logger(handler, route.Name)
routerInstance.
Methods(route.Method).
Path(route.Path).
Name(route.Name).
Handler(handler)
}
return routerInstance
}
I always get the following error (as one would expect)
cannot use route.HandlerFunc (type string) as type http.Handler in assignment:
string does not implement http.Handler (missing ServeHTTP method)
I was told to use something like:
var functions = map[string]interface{}{
"HandleFunc1": HandleFunc1,
}
But I have no idea how to make this work
Thanks to RayenWindspear I was able to fix the problem. It was very simple (like everything). The map code should look like this:
var functions = map[string]http.HandlerFunc{
"HandleFunc1": HandleFunc1,
}
I am using a mux for subdomains, so my examples may be a bit off. The map you are being told to use is done something like this:
type Handlers map[string]http.HandlerFunc
func (handlers Handlers) ServeHTTP(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if handle := handlers[path]; handle != nil {
handle.ServeHTTP(w, r)
} else {
http.Error(w, "Not found", 404)
}
}

Select all is not working in golaong gorm

I'm using gin framework and trying do crud operation using grom.I'm trying to get data from MYSQL database. i have db.go to get database instances, some controllers to each table and models
i have a model like this
type Campaigns struct {
ID int `json:"id" form:"id" gorm:"column:CampaignID"`
UserID int `json:"userId" form:"userId" gorm:"column:UserID"`
Name string `json:"name" form:"name" gorm:"column:Name"`
StartDate time.Time `json:"start" form:"start" gorm:"column:StartDate"`
EndDate time.Time `json:"end" form:"end" gorm:"column:EndDate"`
Customer string `json:"customer" form:"customer" gorm:"column:Customer"`
CustomerID int `json:"customerId" form:"customerId" gorm:"column:CustomerID"`
ImpressionsCounter int `json:"ImpressionsCounter" form:"ImpressionsCounter" gorm:"column:ImpressionsCounter"`
MaxImpressions int `json:"maxImpressions" form:"maxImpressions" gorm:"column:MaxImpressions"`
CurrentSpend float64 `json:"currentSpend" gorm:"column:CurrentSpend"`
MaxSpend float64 `json:"maxSpend" form:"maxSpend" gorm:"column:MaxSpend"`
Active bool `json:"active" form:"active" gorm:"column:Active"`
Created time.Time `json:"created" gorm:"column:DateCreated"`
Updated time.Time `json:"updated" gorm:"column:DateCreated"`
}
this is a one controller I'm using
package controllers
import (
"time"
"github.com/op/go-logging"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/go-sql-driver/mysql"
"../models"
)
var log = logging.MustGetLogger("AsAPI")
type AsController struct {
DB gorm.DB
}
func (ac *AsController) SetDB(d gorm.DB) {
ac.DB = d
ac.DB.LogMode(true)
}
// Get all table
func (ac *AsController) ListTable(c *gin.Context) {
var results []models.Campaigns
err := ac.DB.Find(&results)
if err != nil {
log.Debugf("Error when looking up Table, the error is '%v'", err)
res := gin.H{
"status": "404",
"error": "No Table found",
}
c.JSON(404, res)
return
}
content := gin.H{
"status": "200",
"result": "Success",
"Table": results,
}
c.Writer.Header().Set("Content-Type", "application/json")
c.JSON(200, content)
}
To get database connection I'm using
package controllers
import (
"time"
"github.com/op/go-logging"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/go-sql-driver/mysql"
"../models"
)
var log = logging.MustGetLogger("AdsAPI")
type AsController struct {
DB gorm.DB
}
func (ac *AsController) SetDB(d gorm.DB) {
ac.DB = d
ac.DB.LogMode(true)
}
and I'm using following routs
ac := controllers.AdsController{}
ac.SetDB(dc.GetDB())
// Get a Ads resource
router := gin.Default()
router.GET("/table", ac.ListTables)
when i run this I'm getting following error
(/api/controllers/table.go:30)
[2016-03-23 09:56:39] [0.99ms] SELECT * FROM `tables`
2016/03/23 09:56:39 Error when looking up tables, the error is '&{0xc8202140e0 sql: Scan error on column index 3: unsupported driver -> Scan pair: []uint8 -> *time.Time 1 <nil> 0xc82022f860 0xc82022f7c0 0xc82021e140 2 {0xc8201fb4a0} <nil> false map[] map[]}'
[GIN] 2016/03/23 - 09:56:39 | 404 | 1.153811ms | 127.0.0.1 | GET /table
what is the reason for this error ? help me to fix this error ?
You can find the answer in the driver documentation
https://github.com/go-sql-driver/mysql#timetime-support:
The default internal output type of MySQL DATE and DATETIME values is []byte which allows you to scan the value into a []byte, string or sql.RawBytes variable in your programm.
However, many want to scan MySQL DATE and DATETIME values into time.Time variables, which is the logical opposite in Go to DATE and DATETIME in MySQL. You can do that by changing the internal output type from []byte to time.Time with the DSN parameter parseTime=true. You can set the default time.Time location with the loc DSN parameter.
Alternatively you can use the NullTime type as the scan destination, which works with both time.Time and string / []byte.
Are you ever actually opening the database here? I don't see an actual Open() call in there at all...

How do I do this? Get an struct send to a func as interface and get it back as struct?

I'm new in golang development and have some question regarding something related to this question.
As a learning exercise, I'm trying to create a simple library to handle json based configuration file. As a configuration file to be used for more then one app, it should be able to handle different parameters. Then I have created a type struct Configuration that has the filename and a data interface. Each app will have a struct based on its configuration needs.
In the code bellow, I put all together (lib and "main code") and the "TestData struct" is the "app parameters".
If it doesn't exists, it will set a default values and create the file, and it is working. But when I try to read the file. I try to decode the json and put it back into the data interface. But it is giving me an error and I couldn't figure out how to solve this. Can someone help on this?
[updated] I didn't put the targeted code before, because I though that it would be easier to read in in all as a single program. Bellow is the 'targeted code' for better view of the issue.
As I will not be able to use the TestData struct inside the library, since it will change from program to program, the only way to handle this was using interface. Is there a better way?
library config
package config
import (
"encoding/json"
"fmt"
"os"
)
// Base configuration struct
type Configuration struct {
Filename string
Data interface{}
}
func (c *Configuration) Create(cData *Configuration) bool {
cFile, err := os.Open(cData.Filename)
defer cFile.Close()
if err == nil {
fmt.Println("Error(1) trying to create a configuration file. File '", cData.Filename, "' may already exist...")
return false
}
cFile, err = os.Create(cData.Filename)
if err != nil {
fmt.Println("Error(2) trying to create a configuration file. File '", cData.Filename, "' may already exist...")
return false
}
buffer, _ := json.MarshalIndent(cData.Data, "", "")
cFile.Write(buffer)
return true
}
func (c *Configuration) Read(cData *Configuration) bool {
cFile, err := os.Open(cData.Filename)
defer cFile.Close()
if err != nil {
fmt.Println("Error(1) trying to read a configuration file. File '", cData.Filename, "' may not already exist...")
return false
}
jConfig := json.NewDecoder(cFile)
jerr := jConfig.Decode(&cData.Data)
if jerr != nil {
panic(jerr)
}
return true
}
program using library config
package main
import (
"fmt"
"./config"
)
// struct basic para configuraĆ§Ć£o
type TestData struct {
URL string
Port string
}
func main() {
var Config config.Configuration
Config.Filename = "config.json"
if !Config.Read(&Config) {
Config.Data = TestData{"http", "8080"}
Config.Create(&Config)
}
fmt.Println(Config.Data)
TestData1 := &TestData{}
TestData1 = Config.Data.(*TestData) // error, why?
fmt.Println(TestData1.URL)
}
NEW UPDATE:
I have made some changes after JimB comment about I'm not clear about some concepts and I tried to review it. Sure many things aren't clear for me yet unfortunately. The "big" understanding I believe I got, but what mess my mind up is the "ins" and "outs" of values and formats and pointers, mainly when it goes to other libraries. I'm not able yet to follow the "full path" of it.
Yet, I believe I had some improvement on my code.
I think that I have corrected some points, but still have some big questions:
I stopped sending "Configuration" as a parameter as all "data" were already there as they are "thenselfs" in the instance. Right?
Why do I have use reference in the line 58 (Config.Data = &TestData{})
Why to I have to use pointer in the line 64 (tmp := Config.Data.(*TestData)
Why I CANNOT use reference in line 69 (Config.Data = tmp)
Thanks
The reason you are running into an error is because you are trying to decode into an interface{} type. When dealing with JSON objects, they are decoded by the encoding/json package into map[string]interface{} types by default. This is causing the type assertion to fail since the memory structure for a map[string]interface{} is much different than that of a struct.
The better way to do this is to make your TestData struct the expected data format for your Configuration struct:
// Base configuration struct
type Configuration struct {
Filename string
Data *TestData
}
Then when Decoding the file data, the package will unmarshal the data into the fields that match the closest with the data it finds.
If you need more control over the data unmarshaling process, you can dictate which JSON fields get decoded into which struct members by using struct tags. You can read more about the json struct tags available here: https://golang.org/pkg/encoding/json/#Marshal
You are trying to assert that Config.Data is of type *TestData, but you're assigning it to TestData{"http", "8080"} above. You can take the address of a composite literal to create a pointer:
Config.Data = &TestData{"http", "8080"}
If your config already exsits, your Read method is going to fill in the Data field with the a default json data type, probably a map[string]interface{}. If you assign a pointer of the correct type to Data first, it will decode into the expected type.
Config.Data = &TestData{}
Ans since Data is an interface{}, you do not want to ever use a pointer to that value, so don't use the & operator when marshaling and unmarshaling.