I am trying to abstract my uses of my MySQL database and I'm stuck on a error.
There is the object I will take as example:
package models
// Product : The Product's model
type Product struct {
ID int
Name string
Price int
PictureURL string
}
I'm going to try to retrieve the Product id = 1 in my database. For that, let's say that I already have a connection to my database which is represented by the next variable:
var databaseMySQL *sql.DB
In order to query my database, I am using this function:
// QueryMySQL query our MySQL database
func QueryMySQL(sqlquery model.SQLQuery) model.Status {
// prepare the query
stmtOut, err := databaseMySQL.Prepare(sqlquery.Query)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err}
}
defer stmtOut.Close()
// Run the query
err = stmtOut.QueryRow(sqlquery.Args).Scan(sqlquery.Dest)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err}
} else {
return model.Status{Code: http.StatusOK, Error: nil}
}
}
The 2 models used over here are SQLQuery & Status.
package models
type SQLQuery struct {
Query string
Args []interface{}
Dest []*interface{}
}
Status just basically contains an error and an int.
So, since Scan() as the following prototype Scan func(dest ...interface{}) error, I can pass a []*interface{} as parameter.
If I'm right, then I should be able to get my Dest's elements filled by T type element, and then cast/transform them into the types I need?
func GetProductByID(ID int) (model.Product, model.Status) {
// "SELECT ID, Name, Price, PictureURL FROM Products WHERE ID = ?"
var _product model.Product
// HERE IS THE PROBLEM
var dest []*interface{}
append(dest, &_product.Name)
append(dest, &_product.Price)
append(dest, &_product.PictureURL)
// HERE IS THE PROBLEM
status := QueryMySQL(model.SQLQuery{
Query: "SELECT Name, Price, PictureURL FROM Products WHERE ID = ?",
Args: []interface{}{ID},
Dest: dest})
return _product, model.Status{Code: http.StatusOK, Error: nil}
}
PS: this function is just a basic test to try my logic
However, I am getting an error:
cannot use &_product.Name (type *string) as type *interface {} in append:
*interface {} is pointer to interface, not interface
To me, there is two errors:
cannot use &_product.Name (type *string) as type *interface {}
*interface {} is pointer to interface, not interface
First, why can't I use a interface to store my string?
Second, since I am passing a pointer on string, what's the matter with interface{}? it should be *interface{}, doesn't it?
The correct code, thanks to David Budworth for his answer
In addition to the given code, you can pass slice as varargs like I did by adding ... after your slice variable's name
mysqlproduct.go
// GetProductByID returns a Product based on a given ID
func GetProductByID(ID int) (model.Product, model.Status) {
_product := model.Product{
ID: ID}
status := QueryMySQL(&model.SQLQuery{
Query: "SELECT Name, Price, PictureURL FROM Products WHERE ID = ?",
Args: []interface{}{_product.ID},
Dest: []interface{}{&_product.Name, &_product.Price, &_product.PictureURL}})
if status.Code != http.StatusOK {
return _product, status
}
return _product, model.Status{Code: http.StatusOK, Error: ""}
}
mysql.go
// QueryMySQL query our MySQL database
func QueryMySQL(sqlquery *model.SQLQuery) model.Status {
stmtOut, err := databaseMySQL.Prepare(sqlquery.Query)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err.Error()}
}
defer stmtOut.Close()
// Run the query
err = stmtOut.QueryRow(sqlquery.Args...).Scan(sqlquery.Dest...)
if err != nil {
return model.Status{Code: http.StatusInternalServerError, Error: err.Error()}
}
defer stmtOut.Close()
return model.Status{Code: http.StatusOK, Error: ""}
}
Interfaces can hold pointers. There is rarely a reason to make a pointer to an interface.
If you change your type to []interface{} it should work.
If you really want pointers to interfaces, for some reason, you'd have to first store the field in an interface, then get a pointer to that.
ie:
var i interface{} = &_product.Name
append(dest, &i)
Related
I am working with Gorm and Graphql. I hadn't run into problems querying data until I tried to connect two existing items using their foreignkey relationships. Here are my two models:
type Report struct {
db.Base
OwnerID string
Patient patients.Patient `gorm:"association_name:Patient;"`
PatientID string
}
type Patient struct {
db.Base
ReportID string
}
I have a function to save the relationship to the database:
func (s *Store) AddPatientToReport(ctx context.Context, id string, patient *patients.Patient) (*Report, error) {
// check against user using context
report, err := s.Report(ctx, id)
if err != nil {
log.Error("Could not find report.")
return nil, err
}
report.PatientID = patient.ID
if err := s.db.Save(&report).Association("Patient").Append(patient).Error; err != nil {
log.WithError(err).Error("add patient failed")
return nil, err
}
return report, nil
}
After the above function I can query the Report and see the patient_id. I can also query the Patient and see the report_id. But the following query to get the whole Patient from the Report just returns empty.
query {
report(id: "report_id") {
id
patientID // this will return the correct patient_id
patient {
id // this field always comes back as an empty string
}
}
}
Here is how the DB is setup:
// NewConn creates a new database connection
func NewConn(cc ConnConf) (*Conn, error) {
db, err := gorm.Open("mysql", cc.getCtxStr())
if err != nil {
return nil, err
}
// Models are loaded from each package. Patients is created before Reports.
if err := db.AutoMigrate(Models...).Error; err != nil {
log.Fatal(err)
}
db.LogMode(cc.Logger)
return &Conn{db}, err
}
I can't figure out how to get the whole patient back. Any suggestions?
Ok, so as it turns out I just needed to ask and then I'd figure it out on my own a few minutes later.
I'd read about Preload() in the Gorm docs, but didn't know where to implement it. I first tried when the DB fired up thinking it would load the associations. But I really needed to use Preload() as I run the query.
result := &Report{}
if err = s.db.Preload("Patient").Where(&query).First(&result).Error; err != nil {
log.WithField("id", id).WithError(err).
Error("could not find report")
return nil, err
}
Now, the graphql query:
query {
report(id: "some_id") {
id
patient {
id // now it returns the id
birthyear // now it returns other fields, too
...
}
}
}
my code
func HostStats() (*host.InfoStat, error) {
infoStat, err := host.Info()
fmt.Printf("All Host info: ", infoStat)
return infoStat, err
}
output
All Host info: %!(EXTRA string= {"hostname":"UDAY-PC","uptime":536323,"bootTime":1559911444,"procs":248,"os":"windows","platform":"Microsoft Windows 10 Pro","platformFamily":"Standalone Workstation","platformVersion":"10.0.17134 Build 17134","kernelVersion":"","virtualizationSystem":"","virtualizationRole":"","hostid":"0b324295-3631-47db-b6e8-83cdba2a1af9"})
I want to parse and show the below value from above:
hostname
Platform
HostId
I tried and below has the additional code:
func HostStats() (*host.InfoStat, error) {
infoStat, err := host.Info()
type Information struct {
Name string
Platform string
HostId string
}
var info []Information
info, err := json.Unmarshal(infoStat, &info)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("ok: %+v", info)
//almost every return value is a struct
fmt.Printf("All Host info: ", infoStat)
return infoStat, err
}
As Adrian mentioned above, you are having problems with your Variables.
In Go you can initialize a variable like you did:
var info string
// And assign a value to it by:
info = "foo"
The Json is unmarshaled into your info variable. The return value of the json.Unmarshal is only an error. So the correct syntax would be:
var info []Information
err := json.Unmarshal(infoStat, &info)
So remember the different ways to initialize vars and assign values to those vars.
You can look at GoDocs for Variables for more Info :)
I'm creating a rest api in golang and making a POST request into a table.
For this I've created a struct. Basically the vars in struct is same as the columns in table named users.
and written a function to make a POST request. And the code is working fine, the params while making POST request is being inserted successfully in the table.
type User struct {
ID int
Name string
Lname string
Country string
}
func insertUser(response http.ResponseWriter, request *http.Request)
{
var userDetails User
decoder := json.NewDecoder(request.Body)
err := decoder.Decode(&userDetails)
defer request.Body.Close()
if err != nil {
returnErrorResponse(response,request, httpError)
} else {
httpError.Code = http.StatusBadRequest
if userDetails.Name == "" {
httpError.Message = "first name can't be empty"
returnErrorResponse(response, request, httpError)
} else if userDetails.Lname == "" {
httpError.Message = "Last name can't be empty"
returnErrorResponse(response, request, httpError)
} else {
isInserted := insertUserInDB(userDetails)
if isInserted {
getAllUsers(response, request)
} else {
returnErrorResponse(response, request, httpError)
}
}
}
}
Here is insertUserInDB(userDetails) definition
func insertUserInDB(userDetails User) bool {
stmt, err := db.Prepare("insert into users set Name=?, Lname=?,
Country=?")
if err != nil {
fmt.Print("helper_methods.go : 118")
fmt.Println(err)
return false
}
_, queryError := stmt.Exec(tableName, userDetails.Name,
userDetails.Lname, userDetails.Country)
if queryError != nil {
fmt.Print("helper_methods.go : 125")
fmt.Println(queryError)
return false
}
return true
}
Is there any way to write a common function to insert record in any of the table in the DB?
Can we create struct dynamically, or any other way?
Please help me out here.
Like in other languages, you can use an ORM library to do the DB translation for you, for example GORM, or you can do the translation yourself. Since you already implemented saving the data manually, see this article for how to retrieve data manually.
If you just want to write a generic method that generates/executes SQL queries by matching struct field names you can use the reflect package of go. You will have to identify the structs fields by using reflect.TypeOf() to get the Type of your passed variable and then iterate over the StructField that you can get by using the Field() method on the Type. The StructField will reveal the Name and ValueOf() will allow you to access the Value. Name and value can then be used to construct the query.
For getting a better understanding I recommend you read some articles on reflect. Like this and this.
I think you can use package Refleciton in Golang
,at first step you have to create a method that generates a query for any given type
of struct, for example i write a method that generates an insert query for given
struct.
I have thus two type
type FirstType struct {
FirstParam string
SecondParam int
}
type SecondType struct {
FirstParam string
SecondParam string
ThirdParam int
}
My insert query generation method is as follow
func generateInsetMethod(input inrerface{}) (string, error) {
if reflect.ValueOf(q).Kind() == reflect.Struct {
query := fmt.Sprintf("insert into %s values(", reflect.TypeOf(q).Name())
v := reflect.ValueOf(q)
for i := 0; i < v.NumField(); i++ {
switch v.Field(i).Kind() {
case reflect.Int:
if i == 0 {
query = fmt.Sprintf("%s%d", query, v.Field(i).Int())
} else {
query = fmt.Sprintf("%s, %d", query, v.Field(i).Int())
}
case reflect.String:
if i == 0 {
query = fmt.Sprintf("%s\"%s\"", query, v.Field(i).String())
} else {
query = fmt.Sprintf("%s, \"%s\"", query, v.Field(i).String())
}
default:
fmt.Println("Unsupported type")
}
}
query = fmt.Sprintf("%s)", query)
return query, nil
}
return ``, QueryGenerationError{}
}
know you can use this function to generate your insert queries and also you can generate other functions for Update and etch query that you wants
Is there an easy way to check if each field of myStruct was mapped by using json.Unmarshal(jsonData, &myStruct).
The only way I could image is to define each field of a struct as pointer, otherwise you will always get back an initialized struct.
So every jsonString that is an object (even an empty one {}) will return an initialized struct and you cannot tell if the json represented your struct.
The only solution I could think of is quite uncomfortable:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
Name *string `json:name`
Age *int `json:age`
Male *bool `json:male`
}
func main() {
var p *Person
err := json.Unmarshal([]byte("{}"), &p)
// handle parse error
if err != nil {
return
}
// handle json did not match error
if p.Name == nil || p.Age == nil || p.Male == nil {
return
}
// now use the fields with dereferencing and hope you did not forget a nil check
fmt.Println("Hello " + *p.Name)
}
Maybe one could use a library like govalidator and use SetFieldsRequiredByDefault. But then you still have to execute the validation and still you are left with the whole pointer dereferencing for value retrieval and the risk of nil pointer.
What I would like is a function that returns my unmarshaled json as a struct or an error if the fields did not match. The only thing the golang json library offers is an option to fail on unknown fields but not to fail on missing fields.
Any idea?
Another way would be to implement your own json.Unmarshaler which uses reflection (similar to the default json unmarshaler):
There are a few points to consider:
if speed is of great importance to you then you should write a benchmark to see how big the impact of the extra reflection is. I suspect its negligible but it can't hurt to write a small go benchmark to get some numbers.
the stdlib will unmarshal all numbers in your json input into floats. So if you use reflection to set integer fields then you need to provide the corresponding conversion yourself (see TODO in example below)
the json.Decoder.DisallowUnknownFields function will not work as expected with your type. You need to implement this yourself (see example below)
if you decide to take this approach you will make your code more complex and thus harder to understand and maintain. Are you actually sure you must know if fields are omitted? Maybe you can refactor your fields to make good usage of the zero values?
Here a fully executable test of this approach:
package sandbox
import (
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
)
type Person struct {
Name string
City string
}
func (p *Person) UnmarshalJSON(data []byte) error {
var m map[string]interface{}
err := json.Unmarshal(data, &m)
if err != nil {
return err
}
v := reflect.ValueOf(p).Elem()
t := v.Type()
var missing []string
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
val, ok := m[field.Name]
delete(m, field.Name)
if !ok {
missing = append(missing, field.Name)
continue
}
switch field.Type.Kind() {
// TODO: if the field is an integer you need to transform the val from float
default:
v.Field(i).Set(reflect.ValueOf(val))
}
}
if len(missing) > 0 {
return errors.New("missing fields: " + strings.Join(missing, ", "))
}
if len(m) > 0 {
extra := make([]string, 0, len(m))
for field := range m {
extra = append(extra, field)
}
// TODO: consider sorting the output to get deterministic errors:
// sort.Strings(extra)
return errors.New("unknown fields: " + strings.Join(extra, ", "))
}
return nil
}
func TestJSONDecoder(t *testing.T) {
cases := map[string]struct {
in string
err string
expected Person
}{
"Empty object": {
in: `{}`,
err: "missing fields: Name, City",
expected: Person{},
},
"Name missing": {
in: `{"City": "Berlin"}`,
err: "missing fields: Name",
expected: Person{City: "Berlin"},
},
"Age missing": {
in: `{"Name": "Friedrich"}`,
err: "missing fields: City",
expected: Person{Name: "Friedrich"},
},
"Unknown field": {
in: `{"Name": "Friedrich", "City": "Berlin", "Test": true}`,
err: "unknown fields: Test",
expected: Person{Name: "Friedrich", City: "Berlin"},
},
"OK": {
in: `{"Name": "Friedrich", "City": "Berlin"}`,
expected: Person{Name: "Friedrich", City: "Berlin"},
},
}
for name, c := range cases {
t.Run(name, func(t *testing.T) {
var actual Person
r := strings.NewReader(c.in)
err := json.NewDecoder(r).Decode(&actual)
switch {
case err != nil && c.err == "":
t.Errorf("Expected no error but go %v", err)
case err == nil && c.err != "":
t.Errorf("Did not return expected error %v", c.err)
case err != nil && err.Error() != c.err:
t.Errorf("Expected error %q but got %v", c.err, err)
}
if !reflect.DeepEqual(c.expected, actual) {
t.Errorf("\nWant: %+v\nGot: %+v", c.expected, actual)
}
})
}
}
You could compare p with a empty struct, instead of comparing each field with nil.
// handle json did not match error
if p == Person{} {
return
}
Since Person{} will initialize with the 0 value of each field, this will result in each property that is pointers to be nil, strings will be "", ints will be 0, and so on.
I have a nested struct which I am using to decode JSON request.
type Service struct {
ID string `json:"id,omitempty" db:"id"`
Name string `json:"name" db:"name"`
Contract struct {
ServiceTime int `json:"service_time"`
Region string `json:"region"`
} `json:"contract" db:"contract"`
}
I am using blob type to store Contract in MySQL. To make it work , I would be needing to create a different struct with Contract as string to insert in DB. Can this be done in better way by using single struct only or is there any other elegant way ?
This depends on what you use to talk to the database, but if you're using database/sql and a driver that provides support for this you can have your Contract type implement the Valuer interface.
type Service struct {
ID string `json:"id,omitempty" db:"id"`
Name string `json:"name" db:"name"`
Contract Contract `json:"contract" db:"contract"`
}
type Contract struct {
ServiceTime int `json:"service_time"`
Region string `json:"region"`
}
func (c *Contract) Value() (driver.Value, error) {
if c != nil {
b, err := json.Marshal(c)
if err != nil {
return nil, err
}
return string(b), nil
}
return nil, nil
}
And then when you're executing the query just pass in the Service.Contract field and the driver should call the Value method.
sql := // INSERT INTO ...
svc := // &Service{ ...
db.Exec(sql, svc.ID, svc.Name, svc.Contract)
And to be able to retrieve the blob back from db and unmarshal it back into Contract you can have it implement the Scanner interface, again, if a type implements this the driver is expected to call it.
func (c *Contract) Scan(src interface{}) error {
var data []byte
if b, ok := src.([]byte); ok {
data = b
} else if s, ok := src.(string); ok {
data = []byte(s)
}
return json.Unmarshal(data, c)
}
// ...
sql := // SELECT * FROM ...
svc := // &Service{ ...
db.QueryRow(sql, 123).Scan(&svc.ID, &svc.Name, &svc.Contract)