I am using go-sql-driver to connect to MySQL database. In one of my table I am using a field called queue_length as BIGINT. This field is NULL by default. When I try to query all the fields from the table using go-sql-driver in Golang, the fields after queue_length are not coming in the result set including queue_length.
In my use case,
Table fields
[unique_id, qid, title, text, queue_length, user_id, date_created, last_updated]
When I execute the following code I am getting values for fields before queue_length but [queue_length, user_id, date_created, last_updated] fields are not coming in result set.
But if I don't select queue_length, all fields are coming in result set. I am scanning queue_length in queueLength variable in code which is of type int64. The value coming from table is NULL. I tried type conversion also. Didn't work.
Is this a bug in go-sql-driver or am I doing anything wrong? Could anyone help me with this?
func GetQueues(uniqueId string) ([]*objects.Queue, error) {
db, err := GetDBConnection()
defer db.Close()
rows, err := db.Query("SELECT qid, title, text, queue_length, user_id, date_created, last_updated FROM queue WHERE unique_id = ?", uniqueId)
if err != nil {
panic(err.Error())
}
defer rows.Close()
var qId string
var title string
var text string
var userId string
var dateCreated string
var lastUpdated string
var queueLength int64
var queue *objects.Queue
var queues []*objects.Queue
for rows.Next() {
err = rows.Scan(&qId, &title, &text, &queueLength, &userId, &dateCreated, &lastUpdated)
queue = &objects.Queue{ QId: qId, Title: title, Text: text, UniqueId: uniqueId, UserId: userId, DateCreated: dateCreated, LastUpdated: lastUpdated, QueueLength: queueLength }
queues = append(queues, queue)
}
err = rows.Err()
if err != nil {
panic(err.Error())
return nil, err
}
return queues, err
}
queueLength should be a NullInt64 instead of an int64.
var queueLength sql.NullInt64
In Go ints can't be nil so there's no way for Scan to save NULL in queueLength.
I use this tool for Null and Zero fields, it includes many data types.
The github url is https://github.com/guregu/null
Related
I'm trying to update a record in the database, and based on the rowsAffected() count I can decided whether the record exists or not. In which case if it doesn't (rowsAffected() == 0) I will run a insert query.
func (u *UserService) NewAddress(l *models.Address) error {
var err error
db, err := database.GetConnection()
if err != nil {
return err
}
defer db.Close()
sql := `UPDATE Locations SET Address = ?, StateId = ? WHERE Id = ?`
query := `INSERT INTO Locations (UserId, StateId, Address, City, StreetName, StreetNumber, Code, Latitude, Longitude) VALUES(?,?,?,?,?,?,?,?,?)`
stmt, err := db.Prepare(sql)
if err != nil {
return err
}
results, err := stmt.Exec("Port St Johns, South Africa", l.StateId, l.Id)
if err != nil {
return err
}
fmt.Println(results.RowsAffected())
if affected, err := results.RowsAffected(); err == nil {
if affected == 0 {
insert, err := db.Query(query, l.UserId, l.StateId, l.Address, l.City, l.StreetName, l.StreetNumber, l.Code, l.Latitude, l.Longitude)
defer insert.Close()
if err != nil {
return err
}
}
}
return nil
}
However, the RowsAffected() func always returns 0 if the value of the column I'm updating e.i Address is the same as the one already stored in the database. It also returns 0 if it can't find the record I'm trying to update.
If I run the same query directly on MySQL console however, even if the update (address) is exactly the same, it always returns Update Rows >= 1 as long as the WHERE clause condition is met.
UPDATE Locations
SET Address = "Port St Johns", StateId = 2
WHERE Id = 102;
Can I do this update any different? I'm trying to avoid a select query to check if the record exists before updating/inserting.
I added a updated timestamp as #mkopriva suggested. That way there's always an updated count even when all the data is the same, the timestamp will change/update
How would it be possible to create a database record (in this case mysql) without literally specifying each variable to be passed in to the model, on or at the model (see following example). With having to create dozens of models, I am trying to avoid having to specify these vars manually with each and every model. I can dynamically collect this list on the controller and am hoping there is some way to just pass one object from the controller to the model (in this failed experiment: res []string).
// controller
_, err := policy.CreateItem(c.DB, res)
// model
func CreateItem(db Connection, res []string) (sql.Result, error) {
// res = ParentID, Title, Description, Sort, Date, StatusID, UserID
tfcim := format.TitlesForCreateInputModel(ItemMin{}) // parent_id, title, description, sort, date, status_id, user_id
phfcm := format.PlaceHoldersForCreateModel(ItemMin{}) // ?,?,?,?,?,?,?
result, err := db.Exec(fmt.Sprintf(`
INSERT INTO %v
(`+tfcim+`)
VALUES
(`+phfcm+`)
`, table),
strings.Join(res, ", ")) // <----------------- How can magic happen here?
return result, err
}
And logically, the output is: flight.go:138: sql: expected 7 arguments, got 1
To note, ItemMin{} is the table struct returning the table keys using 'reflect' in a package named format.
Any functional method, even an entirely different approach (except using an ORM/GORM) would be acceptable. Thanks in advance!
UPDATE: Complete working solution:
// controller
s := make([]interface{}, len(res))
for i, v := range res {
s[i] = v
}
_, err := policy.CreateItem(c.DB, s)
// model
func CreateItem(db Connection, res []interface{}) (sql.Result, error) {
tfcim := format.TitlesForCreateInputModel(ItemMin{}) // parent_id, title, description, sort, date, status_id, user_id
phfcm := format.PlaceHoldersForCreateModel(ItemMin{}) // ?,?,?,?,?,?,?
query := fmt.Sprintf("INSERT INTO %v (%v) VALUES (%v)", table, tfcim, phfcm)
result, err := db.Exec(query, res...)
return result, err
}
As the Exec function has this signature:
Exec(query string, args ...interface{})
You can use this syntax:
query := fmt.Sprintf("INSERT INTO %v (%v) VALUES (%v)", table, tfcim, phfcm)
result, err := db.Exec(query, res...)
So I'm trying to upload image through an API and store it in a MySQL database. I'm unsure how to convert the imageFile to something that I can store in my DB.
I'm trying to store this image into a blob field on the image column of this table:
CREATE TABLE runkdb.uploaded_challenge (
id int NOT NULL AUTO_INCREMENT,
challenge_id int NOT NULL,
user_id int NOT NULL,
created_at DATETIME NOT NULL,
image_caption TEXT,
image_path varchar(255),
image BLOB,
score int DEFAULT 0,
primary key (id)
);
This is part of the API function that tries to store the data:
// Parse the request
imageFile := make([]byte, 0)
image, _, err := req.FormFile("image")
if err != nil {
// No image
glog.Error("Error parsing image")
responseMessage.Message = "Error parsing image"
w.WriteHeader(400)
json.NewEncoder(w).Encode(responseMessage)
return
} else {
//if !strings.HasPrefix(fileHeader.Header["Content-Type"][0], "image") {
// Something wrong with file type
//}
imageFile, err = ioutil.ReadAll(image)
if err != nil {
// Error reading uploaded image from stream
glog.Error("Error reading image")
responseMessage.Message = "Error reading image"
w.WriteHeader(400)
json.NewEncoder(w).Encode(responseMessage)
return
}
}
imageCaption := req.Form.Get("image_caption")
// Create DB connection
txn, err := api.db.Begin()
if err != nil {
glog.Error("Error creating database transaction")
responseMessage.Message = "Error creating database transaction"
w.WriteHeader(500)
json.NewEncoder(w).Encode(responseMessage)
return
}
t := time.Now()
createdAtTimeString := fmt.Sprintf(t.Format("2006-01-02 15:04:05"))
imageByteString := string(imageFile)
query := fmt.Sprintf("INSERT INTO uploaded_challenge (challenge_id, user_id, created_at, image_caption, image) VALUES ('%d', '%d', '%s', '%s', '%s');", id, userID, createdAtTimeString, imageCaption, imageByteString)
print(query)
result, err := txn.Exec(query)
if err != nil {
txn.Rollback()
glog.Error("Error inserting challenge into database")
responseMessage.Message = "Error inserting challenge into database"
w.WriteHeader(406)
json.NewEncoder(w).Encode(responseMessage)
return
}
Solved it by converting my imageFile to a string like this:
encodedImage := base64.StdEncoding.EncodeToString(imageFile)
Then I stored the encodedImage into my database's image column (of type BLOB)
I want to display the number of rows from database using Go. How do I display number of rows?
count, err := db.Query("SELECT COUNT(*) FROM main_table")
The query will return a row into the variable count. So the next you have to do is to read this row and assign the result into a new variable, using the function Scan(). This is how it works.
rows, err := db.Query("SELECT COUNT(*) FROM main_table")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
var count int
for rows.Next() {
if err := rows.Scan(&count); err != nil {
log.Fatal(err)
}
}
fmt.Printf("Number of rows are %s\n", count)
The best option thought would be to use QueryRow() as you expect to read just one row. The code then will be.
var count int
err := db.QueryRow("SELECT COUNT(*) FROM main_table").Scan(&count)
switch {
case err != nil:
log.Fatal(err)
default:
fmt.Printf("Number of rows are %s\n", count)
}
I signed up just to share this as my large datasets were slowing down the program with the constant appends. I wanted to put the rowcount in the rows and figured that something like that must exist, just had to find it.
SELECT
count(1) OVER(PARTITION BY domain_id) AS rowcount,
subscription_id,
domain_id,
...
FROM mytable
WHERE domain_id = 2020
Never used this command before, but it will add the count of the result set that share this parameter. Setting it to one of the query WHERE's makes it the total rows.
I am currently working on a Golang application that talks to a MariaDB database. My service will take in a *User and attempt to update the matching record.
For example:
func (d *database) UpdateUser(user *User) error {
stmt, err := d.Prepare(`UPDATE Users
SET FirstName=?, LastName=?, Email=?, Address1=?, Address2=?,
City=?, State=?, Country=?, PostalCode=?
WHERE Id=?`)
if err != nil {
return err
}
_, err := res, err := stmt.Exec(user.FirstName, user.LastName, user.Email,
user.Address1, user.Address2, user.City, user.State, user.Country,
user.PostalCode, user.Id)
return err
}
However, I do not want to update fields that aren't supplied. I am looking for something closer to this, where it would not update the value if the provided one is an empty string:
`UPDATE Users
SET FirstName=? OR FirstName, LastName=? OR LastName, Email=? OR Email,
Address1=? OR Address1, Address2=? OR Address2, City=? OR City,
State=? OR State, Country=? OR Country, PostalCode=? OR PostalCode
WHERE Id=?`
I have seen that SQL allows for updating using a CASE statement, but I do not see a way to use this without providing the same variable to the Exec() several times.
Like this
UPDATE Users
SET FirstName = case when ? is null then FirstName else ? end
or to pass each parameter just once
UPDATE Users
SET FirstName = coalesce(?, FirstName)