invalid memory address or nil pointer dereference with sql.DB - mysql

I'm learning Go at the moment and trying to make a little SQL-toolset:
type DBUtils struct {
User string
Password string
Host string
Database string
Handle *sql.DB
}
func (dbUtil DBUtils) Connect() {
var err error
dbUtil.Handle, err = sql.Open("mysql", dbUtil.User + ":" + dbUtil.Password + "#tcp(" + dbUtil.Host + ")/" + dbUtil.Database)
if err != nil {
panic(err.Error())
}
err = dbUtil.Handle.Ping()
if err != nil {
panic(err.Error())
}
fmt.Printf("%v", dbUtil)
}
func (dbUtil DBUtils) Close() {
dbUtil.Handle.Close()
}
func (dbUtil DBUtils) GetString(what string, from string, where string, wherevalue string) string {
var username string
fmt.Printf("%v", dbUtil)
stmtOut, err := dbUtil.Handle.Prepare("SELECT " + what + " FROM " + from + " WHERE " + where + " = " + wherevalue)
if err != nil {
panic(err.Error()) // proper error handling instead of panic in your app
}
err = stmtOut.QueryRow(1).Scan(&username)
return username
}
So when using this with the following code:
db := databaseutils.DBUtils{"root", "root", "127.0.0.1:3306", "gotest", nil}
db.Connect() // I get: {root root 127.0.0.1:3306 gotest 0xc42019d600}
fmt.Printf("%v", db) // I get {root root 127.0.0.1:3306 gotest <nil>}
x := db.GetString("username", "users", "id", "1") // Doesn't work: panic: runtime error: invalid memory address or nil pointer dereference
fmt.Println(x)
For me it seems like my DB handle isn't saved properly? Does anyone have an idea - I'm pretty new to go and there are many things looking different to PHP, JS, C++ etc.
Thanks in advance!

Your Connect method is not changing the state of the object you're calling the method on. You're working on a copy of the type. If you want a method to change the object itself, you'll have to define it on a pointer:
func (dbUtil *DBUtils) Connect() {
//instead of
func (dbUtil DBUtils) Connect() {
If you're familiar with C or C++, your current method works similarly to something like:
void connect_db(struct db_utils db_util)
{}
When you call a function like that, you're creating a copy of the argument, and push that on to the stack. The connect_db function will work with that, and after it returns, the copy is deallocated.
compare it to this C code:
struct foo {
int bar;
};
static
void change_copy(struct foo bar)
{
bar.bar *= 2;
}
static
void change_ptr(struct foo *bar)
{
bar->bar *= 2;
}
int main ( void )
{
struct foo bar = {10};
printf("%d\n", bar.bar);//prints 10
change_copy(bar);//pass by value
printf("%d\n", bar.bar);//still prints 10
change_ptr(&bar);
printf("%d\n", bar.bar);//prints 20
return 0;
}
The same thing happens in go. The object on which you define the method can only change state of the instance if it has access to the instance. If not, it can't update that part of the memory.
In case you're wondering, this method doesn't need to be defined on the pointer type:
func (dbUtil DBUtils) Close() {
dbUtil.Handle.Close()
}
The reason for this being that DBUtils.Handle is a pointer type. A copy of that pointer will always point to the same resource.
I will say this, though: given that you're essentially wrapping the handle, and you're exposing the Connect and Close methods, the member itself really shouldn't be exported. I'd change it to lower-case handle

You are right about your DB handle not being saved. You are defining your methods on a value receiver. Therefore you receive a copy inside the Connect method and modify said copy. This copy is then dropped at the end of the Connect method.
You have to define your method on a pointer to your structure:
func (dbUtil *DBUtils) Connect() {
var err error
dbUtil.Handle, err = sql.Open("mysql", dbUtil.User + ":" + dbUtil.Password + "#tcp(" + dbUtil.Host + ")/" + dbUtil.Database)
if err != nil {
panic(err.Error())
}
err = dbUtil.Handle.Ping()
if err != nil {
panic(err.Error())
}
fmt.Printf("%v", dbUtil)
}
For further information see https://golang.org/doc/faq#methods_on_values_or_pointers.
Note: I noticed your usage of QueryRow. It accepts arguments for parameters in your prepare statement. You have no parameters there, so I think you should not pass any parameters. You should also check the Scan result for errors (see https://golang.org/pkg/database/sql/#Stmt).

Related

how to extract single value from json in Golang?

my code
func HostStats() (*host.InfoStat, error) {
infoStat, err := host.Info()
fmt.Printf("All Host info: ", infoStat)
return infoStat, err
}
output
All Host info: %!(EXTRA string= {"hostname":"UDAY-PC","uptime":536323,"bootTime":1559911444,"procs":248,"os":"windows","platform":"Microsoft Windows 10 Pro","platformFamily":"Standalone Workstation","platformVersion":"10.0.17134 Build 17134","kernelVersion":"","virtualizationSystem":"","virtualizationRole":"","hostid":"0b324295-3631-47db-b6e8-83cdba2a1af9"})
I want to parse and show the below value from above:
hostname
Platform
HostId
I tried and below has the additional code:
func HostStats() (*host.InfoStat, error) {
infoStat, err := host.Info()
type Information struct {
Name string
Platform string
HostId string
}
var info []Information
info, err := json.Unmarshal(infoStat, &info)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("ok: %+v", info)
//almost every return value is a struct
fmt.Printf("All Host info: ", infoStat)
return infoStat, err
}
As Adrian mentioned above, you are having problems with your Variables.
In Go you can initialize a variable like you did:
var info string
// And assign a value to it by:
info = "foo"
The Json is unmarshaled into your info variable. The return value of the json.Unmarshal is only an error. So the correct syntax would be:
var info []Information
err := json.Unmarshal(infoStat, &info)
So remember the different ways to initialize vars and assign values to those vars.
You can look at GoDocs for Variables for more Info :)

How to validate a signature signed with PKCS8 private key?

I'm developing a program with Go language for validating signatures.
It receives parameters(public key, signature, plaintext)
and the signature is signed by user's private key in PKCS#8.
I tried to use the function VerifyPKCS1v15 in the package x509
but it didn't work for me.
I guess it may due to the function is oriented for signatures made by pkcs1 private keys.
can anybody help me?
I wanna know if there is any way to validate signatures using pkcs8 public key?
this is my code
func main() {
var plainTest = "P0000000025300000100000000001000026720180705140842"
var hashVal = "15b47c1d79b0be2aae36a05bcd8644af7bfe3dd4e0c23e2b78692fc900998fca"
var signatureStr = "WWFCZsD3BhakkCaLAcTPxMvd3Pom1Glhgcc+xhR7tIDBLvkVk/LtxV+2nHw6b9u0Dcla8U4vUR7KH8zpUS7fNJD9yPDDWxH5PYiw4jQTjziiLHSUpuaGbf8N1Y2jKPXvzq1ZFaEAqCirLSmt5KyD3gQ22ysHgYA2vH44zzBApcxYXVbzLbCIGAR5aL/mvYt7uWsh4FX8dQ49v9SqIm/rRBGEbsscF4HpQApy8VqRGvq6EbwrPCfMcpwIbBHdDUR0mneaNg9GH4hozfMC08SZtAMGDk8J/NQway1FisrjpUeZfMe/hANDH1LmfrbThKDgB7WIpDryCXMTsBKjrqyArQ=="
var pubKeyStr = "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA17hWIujBfiqrd4o0JCEn6N1mzv94VM9LiVHoldvPRDEwCXbeoSebzZElvhkJsPl08o68g1BgRC4LpaGQDcVzwyFMs4DnXEDLapZQpTzkmXaSIYIRYER/U1OgdW5Cq2do/eTrylWdloqWuz5JL2vIr4GFycnEduYVSzFmAqucCvgGEFUxwFxtZ95BVsxfKOt7eFCJWoS0iR2/If5EMG9F6KG6DtDUWg6awN2mIbhm8fqxSF48ehCkPCN4s4YkcUlkmGYEetdBCxbaUh9/S960XjQBK3MXbLIJLgRLoEAdWJ2v6IjaEsw7dQAaMti3QOPr0x7TyHlS7rz/lyjlJjaXEQIDAQAB"
publicKeyBase64, err := base64.StdEncoding.DecodeString(pubKeyStr)
if err != nil {
fmt.Println("base64 error : " + err.Error())
}
fmt.Println("publicKeyBase64: ")
fmt.Println(string(publicKeyBase64))
pub, err := x509.ParsePKIXPublicKey(publicKeyBase64)
if err != nil {
fmt.Println("failed to parse DER encoded public key: " + err.Error())
}
switch pub := pub.(type) {
case *rsa.PublicKey:
fmt.Println("pub is of type RSA:", pub)
case *dsa.PublicKey:
fmt.Println("pub is of type DSA:", pub)
case *ecdsa.PublicKey:
fmt.Println("pub is of type ECDSA:", pub)
default:
panic("unknown type of public key")
}
publicKey, isRSAPublicKey := pub.(*rsa.PublicKey)
if !isRSAPublicKey {
fmt.Println("Public key parsed is not an RSA public key")
}
signatureBytes, _ := hex.DecodeString(signatureStr)
fmt.Println("signatureBytes : " + string(signatureBytes))
validateBytes := sha256.Sum256([]byte(plainTest))
fmt.Println("validateBytes : " + string(validateBytes[:]))
err = rsa.VerifyPKCS1v15(publicKey, crypto.SHA256, validateBytes[:], signatureBytes)
if err != nil {
fmt.Printf("err: %v\n", err)
} else {
fmt.Printf("ok")
}
}
I got the same problem, it turns out my publicKey is wrong.
I mean not the format, but the key is not paired with the private key.
I change to another pair of key, and the code worked.

SQL query results to go inside a nested struct, using Golang

Below is the whole code of the program. It is a service that forwards requests. Currently working. What I am trying to do is get rid of the yml file that is currently storing all configs and move them to db. I don't want to mess with the code much, so my idea was to simply store the db data in the same structs.
// Config contains configuration for this service
type Instance struct {
User string `json:"user"`
Password string `json:"password"`
InstanceId string `json:"instance_id"`
InstanceType string `json:"instance_type"`
InstanceMode string `json:"instance_mode"`
ClientId string `json:"client_id"`
ClientSecret string `json:"client_secret"`
PublicKey string `json:"pubkey"`
Apis Api `json:"apis"`
}
// API struct
type Api struct {
Name string `json:"name"`
Url string `json:"url"`
Version string `json:"version"`
IsBhdGw bool `json:"isBhdGw"`
Key string `json:"key"`
}
// API struc
type InfoResponse struct {
InstanceId string `json:"instance_id"`
InstanceType string `json:"instance_type"`
InstanceMode string `json:"instance_mode"`
ClientId string `json:"client_id"`
ClientSecret string `json:"client_secret"`
PublicKey string `json:"pubkey"`
Apis Api `json:"apis"`
}
type Settings struct {
Port int `json:"port"`
User string `json:"user"`
Secret string `json:"secret"`
Mode string `json:"mode"`
}
type Instances struct {
Instances []Instance
}
var payloadLength int = 3
var instances Instances
var settings Settings
var db *sql.DB
func fetchInstances() []Instance {
rows, err := db.Query("SELECT Instances.InstanceId, Instances.User, Instances.Password, Instances.InstanceType, Instances.InstanceMode, Instances.ClientId, Instances.ClientSecret, Instances.PublicKey, Api.Name, Api.Url, Api.Version, Api.IsBhdGw, Api.Key FROM Instances, Api")
if err != nil {
panic(err.Error())
}
defer rows.Close()
instances := []Instance{}
for rows.Next() {
var instance Instance
var apis Api
err := rows.Scan(&instance.InstanceId, &instance.User, &instance.Password,
&instance.InstanceType, &instance.InstanceMode, &instance.ClientId, &instance.ClientSecret, &instance.PublicKey,
&apis.Name, &apis.Url, &apis.Version, &apis.IsBhdGw, &apis.Key)
if err != nil {
panic(err.Error())
}
instance.Apis = apis
instances = append(instances, instance)
}
return instances
}
When I run this I get a "panic: runtime error: invalid memory address or nil pointer dereference"
You are attempting to connect to an uninitiated database connection. This is due to variable shadowing.
Although you've defined a global db variable:
var db *sql.DB
Your main() method creates a new one when it connects:
db, err := sql.Open(dbDriver, dbUser+":"+dbPass+"#tcp(db:3306)/"+dbName)
So then when you call fetchInstances, which uses the global variable, the global one is still unset.
The easiest solution is to change your code in main() as follows:
var err error
db, err = sql.Open(dbDriver, dbUser+":"+dbPass+"#tcp(db:3306)/"+dbName)
But the better solution is to never use a global variable. Use the locally-defined variable in main, then pass that to fetchInstances.
As a side note: ALWAYS, ALWAYS check your errors. Twice in your code, you fail to check db connection errors, as in:
db, err := sql.Open(dbDriver, dbUser+":"+dbPass+"#tcp(db:3306)/"+dbName)
err = db.Ping()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
If the connection fails, your db.Ping() call will likely panic. Instead, you should do:
db, err := sql.Open(dbDriver, dbUser+":"+dbPass+"#tcp(db:3306)/"+dbName)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
err = db.Ping()
if err != nil {
fmt.Println(err)
os.Exit(1)
}

Slice of pointers to interface for MySQL queries abstractions

I am trying to abstract my uses of my MySQL database and I'm stuck on a error.
There is the object I will take as example:
package models
// Product : The Product's model
type Product struct {
ID int
Name string
Price int
PictureURL string
}
I'm going to try to retrieve the Product id = 1 in my database. For that, let's say that I already have a connection to my database which is represented by the next variable:
var databaseMySQL *sql.DB
In order to query my database, I am using this function:
// QueryMySQL query our MySQL database
func QueryMySQL(sqlquery model.SQLQuery) model.Status {
// prepare the query
stmtOut, err := databaseMySQL.Prepare(sqlquery.Query)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err}
}
defer stmtOut.Close()
// Run the query
err = stmtOut.QueryRow(sqlquery.Args).Scan(sqlquery.Dest)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err}
} else {
return model.Status{Code: http.StatusOK, Error: nil}
}
}
The 2 models used over here are SQLQuery & Status.
package models
type SQLQuery struct {
Query string
Args []interface{}
Dest []*interface{}
}
Status just basically contains an error and an int.
So, since Scan() as the following prototype Scan func(dest ...interface{}) error, I can pass a []*interface{} as parameter.
If I'm right, then I should be able to get my Dest's elements filled by T type element, and then cast/transform them into the types I need?
func GetProductByID(ID int) (model.Product, model.Status) {
// "SELECT ID, Name, Price, PictureURL FROM Products WHERE ID = ?"
var _product model.Product
// HERE IS THE PROBLEM
var dest []*interface{}
append(dest, &_product.Name)
append(dest, &_product.Price)
append(dest, &_product.PictureURL)
// HERE IS THE PROBLEM
status := QueryMySQL(model.SQLQuery{
Query: "SELECT Name, Price, PictureURL FROM Products WHERE ID = ?",
Args: []interface{}{ID},
Dest: dest})
return _product, model.Status{Code: http.StatusOK, Error: nil}
}
PS: this function is just a basic test to try my logic
However, I am getting an error:
cannot use &_product.Name (type *string) as type *interface {} in append:
*interface {} is pointer to interface, not interface
To me, there is two errors:
cannot use &_product.Name (type *string) as type *interface {}
*interface {} is pointer to interface, not interface
First, why can't I use a interface to store my string?
Second, since I am passing a pointer on string, what's the matter with interface{}? it should be *interface{}, doesn't it?
The correct code, thanks to David Budworth for his answer
In addition to the given code, you can pass slice as varargs like I did by adding ... after your slice variable's name
mysqlproduct.go
// GetProductByID returns a Product based on a given ID
func GetProductByID(ID int) (model.Product, model.Status) {
_product := model.Product{
ID: ID}
status := QueryMySQL(&model.SQLQuery{
Query: "SELECT Name, Price, PictureURL FROM Products WHERE ID = ?",
Args: []interface{}{_product.ID},
Dest: []interface{}{&_product.Name, &_product.Price, &_product.PictureURL}})
if status.Code != http.StatusOK {
return _product, status
}
return _product, model.Status{Code: http.StatusOK, Error: ""}
}
mysql.go
// QueryMySQL query our MySQL database
func QueryMySQL(sqlquery *model.SQLQuery) model.Status {
stmtOut, err := databaseMySQL.Prepare(sqlquery.Query)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err.Error()}
}
defer stmtOut.Close()
// Run the query
err = stmtOut.QueryRow(sqlquery.Args...).Scan(sqlquery.Dest...)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err.Error()}
}
defer stmtOut.Close()
return model.Status{Code: http.StatusOK, Error: ""}
}
Interfaces can hold pointers. There is rarely a reason to make a pointer to an interface.
If you change your type to []interface{} it should work.
If you really want pointers to interfaces, for some reason, you'd have to first store the field in an interface, then get a pointer to that.
ie:
var i interface{} = &_product.Name
append(dest, &i)

Golang slices of struct or newbie trouble building REST

and need your help.
Wanted to build simple api and stuck with some problem.
I've choose gin and database/sql with postgres driver
package main
import (
"database/sql"
"fmt"
"github.com/gin-gonic/gin"
_ "github.com/lib/pq"
)
func main() {
router := gin.Default()
router.GET("/search/:text", SearchWord)
router.Run(":8080")
}
I need to make query to DB and make json out of this request.
func checkErr(err error) {
if err != nil {
panic(err)
}
}
type Message struct {
ticket_id int `json:"ticket_id"`
event string `json:"event"`
}
func SearchWord(c *gin.Context) {
word := c.Params.ByName("text")
db, err := sql.Open("postgres", "host=8.8.8.8 user= password= dbname=sample")
defer db.Close()
checkErr(err)
rows, err2 := db.Query("SELECT ticket_id,event FROM ....$1, word)
checkErr(err)
for rows.Next() {
var ticket_id int
var event string
err = rows.Scan(&ticket_id, &event)
checkErr(err)
fmt.Printf("%d | %s \n\n", ticket_id, event)
}
}
This coda working nice, but when i need to make json.
I need to make struct of a row
type Message struct {
ticket_id int `json:"ticket_id"`
event string `json:"event"`
}
an then i need to create slice , and append every rows.Next() loop an than answer to browser with Json...
c.JSON(200, messages)
But how to do that...don't know :(
disclaimer: I am brand new to go
Since you Scanned your column data into your variables, you should be able to initialize a structure with their values:
m := &Message{ticket_id: ticket_id, event: event}
You could initialize a slice with
s := make([]*Message, 0)
And then append each of your message structs after instantiation:
s = append(s, m)
Because I'm not too familiar with go there are a couple things i'm not sure about:
after copying data from query to your vars using rows.Scan does initializing the Message struct copy the current iterations values as expected??
If there is a way to get the total number of rows from your query it might be slighlty more performant to initialize a static length array, instead of a slice?
I think #inf deleted answer about marshalling your Message to json down the line might need to be addressed, and Message field's might need to be capitalized
copied from #inf:
The names of the members of your struct need be capitalized so that
they get exported and can be accessed.
type Message struct {
Ticket_id int `json:"ticket_id"`
Event string `json:"event"` }
I'm going to cheat a little here and fix a few things along the way:
First: open your database connection pool once at program start-up (and not on every request).
Second: we'll use sqlx to make it easier to marshal our database rows into our struct.
package main
var db *sqlx.DB
func main() {
var err error
// sqlx.Connect also checks that the connection works.
// sql.Open only "establishes" a pool, but doesn't ping the DB.
db, err = sqlx.Connect("postgres", "postgres:///...")
if err != nil {
log.Fatal(err)
}
router := gin.Default()
router.GET("/search/:text", SearchWord)
router.Run(":8080")
}
// in_another_file.go
type Message struct {
TicketID int `json:"ticket_id" db:"ticket_id"`
Event string `json:"event" db:"event"`
}
func SearchWord(c *gin.Context) {
word := c.Params.ByName("text")
// We create a slice of structs to marshal our rows into
var messages []*Message{}
// Our DB connection pool is safe to use concurrently from here
err := db.Select(&messages, "SELECT ticket_id,event FROM ....$1, word)
if err != nil {
http.Error(c.Writer, err.Error(), 500)
return
}
// Write it out using gin-gonic's JSON writer.
c.JSON(200, messages)
}
I hope that's clear. sqlx also takes care of calling rows.Close() for you, which will otherwise leave connections hanging.