Golang: Mysql Prepare Insert statements do not add rows into db table - mysql

So I'm trying to use a mysql driver to insert data into a database.
Specifically, I'm using this one:
"github.com/go-sql-driver/mysql"
This is my code
func main() {
db, err := sql.Open("mysql", "psanker:123#/education_data")
err = db.Ping()
if err != nil {
fmt.Println("Failed to prepare connection to database")
log.Fatal("Error:", err.Error())
}
defer db.Close()
content, err := ioutil.ReadFile("activities.csv")
lines := strings.Split(string(content), "\r")
//only work so long as I have one district
rows, err := db.Query("SELECT id FROM districts")
var districtId int
defer rows.Close()
for rows.Next() {
err := rows.Scan(&districtId)
check(err)
}
for i, line := range lines {
if i > 1 {
splitStr := strings.Split(line, ",")
var activityCode string
if strings.Contains(splitStr[0], "-") {
//this is an activity
activityCode = splitStr[0]
stmt1, _ := db.Prepare("INSERT INTO activities(code) VALUES(?)")
stmt2, _ := db.Prepare("INSERT INTO activities(name) VALUES(?)")
res, _ := stmt1.Exec(splitStr[0])
stmt2.Exec(splitStr[1])
} else {
//this is a sub activity
stmt1, _ := db.Prepare("INSERT INTO sub_activities(code) VALUES(?)")
stmt2, _ := db.Prepare("INSERT INTO sub_activities(name) VALUES(?)")
stmt1.Exec(splitStr[0])
stmt2.Exec(splitStr[1])
if activityCode != "" {
rows, _ := db.Query("SELECT id from activities where code = ?", activityCode)
var activityId int
for rows.Next() {
err := rows.Scan(&activityId)
check(err)
stmt3, err := db.Prepare("INSERT INTO sub_activities(activity_id) VALUES(?)")
stmt3.Exec(activityId)
}
}
rows, _ := db.Query("SELECT id from sub_activities where code= ?", splitStr[0])
var sub_activityId int
for rows.Next() {
err := rows.Scan(&sub_activityId)
check(err)
stmt5, _ := db.Prepare("INSERT INTO sub_activity_expenditure(district_id) VALUES(?)")
stmt6, _ := db.Prepare("INSERT INTO sub_activity_expenditure(sub_activity_id) VALUES(?)")
stmt7, _ := db.Prepare("INSERT INTO sub_activity_expenditure(expenditure) VALUES(?)")
stmt5.Exec(districtId)
stmt6.Exec(sub_activityId)
stmt7.Exec(splitStr[2])
}
}
}
}
}
When I check Mysql database, there are no rows inserted into the tables. I think I'm accessing the right database because the initial query that gets id from districts is returning 1, which is correct.
What's going on?
EDIT
I have confirmed I'm in the right db by doing this
rows, err = db.Query("SELECT DATABASE();")
var test string
for rows.Next() {
rows.Scan(&test)
fmt.Println(test)
}
Which prints out education_data

As you are trying to insert into a table, I was reminded (in chat at least) that your columns are non-null and you really meant to create one row when you were in fact creating two. It failed on stmt1 and stmt2 because non-nullable columns were not given parameters to replace for all necessary data for insertion to succeed.
Focus on creating one row for stmt1 and stmt2 (combine them into a single stmt1 to execute with 2 parameters).
Make sure strings are wrapped with a single quote.

Related

Incosistent rows updated between MySQL and go-sql-driver/mysql

I'm trying to update a record in the database, and based on the rowsAffected() count I can decided whether the record exists or not. In which case if it doesn't (rowsAffected() == 0) I will run a insert query.
func (u *UserService) NewAddress(l *models.Address) error {
var err error
db, err := database.GetConnection()
if err != nil {
return err
}
defer db.Close()
sql := `UPDATE Locations SET Address = ?, StateId = ? WHERE Id = ?`
query := `INSERT INTO Locations (UserId, StateId, Address, City, StreetName, StreetNumber, Code, Latitude, Longitude) VALUES(?,?,?,?,?,?,?,?,?)`
stmt, err := db.Prepare(sql)
if err != nil {
return err
}
results, err := stmt.Exec("Port St Johns, South Africa", l.StateId, l.Id)
if err != nil {
return err
}
fmt.Println(results.RowsAffected())
if affected, err := results.RowsAffected(); err == nil {
if affected == 0 {
insert, err := db.Query(query, l.UserId, l.StateId, l.Address, l.City, l.StreetName, l.StreetNumber, l.Code, l.Latitude, l.Longitude)
defer insert.Close()
if err != nil {
return err
}
}
}
return nil
}
However, the RowsAffected() func always returns 0 if the value of the column I'm updating e.i Address is the same as the one already stored in the database. It also returns 0 if it can't find the record I'm trying to update.
If I run the same query directly on MySQL console however, even if the update (address) is exactly the same, it always returns Update Rows >= 1 as long as the WHERE clause condition is met.
UPDATE Locations
SET Address = "Port St Johns", StateId = 2
WHERE Id = 102;
Can I do this update any different? I'm trying to avoid a select query to check if the record exists before updating/inserting.
I added a updated timestamp as #mkopriva suggested. That way there's always an updated count even when all the data is the same, the timestamp will change/update

How to create mysql database record dynamically

How would it be possible to create a database record (in this case mysql) without literally specifying each variable to be passed in to the model, on or at the model (see following example). With having to create dozens of models, I am trying to avoid having to specify these vars manually with each and every model. I can dynamically collect this list on the controller and am hoping there is some way to just pass one object from the controller to the model (in this failed experiment: res []string).
// controller
_, err := policy.CreateItem(c.DB, res)
// model
func CreateItem(db Connection, res []string) (sql.Result, error) {
// res = ParentID, Title, Description, Sort, Date, StatusID, UserID
tfcim := format.TitlesForCreateInputModel(ItemMin{}) // parent_id, title, description, sort, date, status_id, user_id
phfcm := format.PlaceHoldersForCreateModel(ItemMin{}) // ?,?,?,?,?,?,?
result, err := db.Exec(fmt.Sprintf(`
INSERT INTO %v
(`+tfcim+`)
VALUES
(`+phfcm+`)
`, table),
strings.Join(res, ", ")) // <----------------- How can magic happen here?
return result, err
}
And logically, the output is: flight.go:138: sql: expected 7 arguments, got 1
To note, ItemMin{} is the table struct returning the table keys using 'reflect' in a package named format.
Any functional method, even an entirely different approach (except using an ORM/GORM) would be acceptable. Thanks in advance!
UPDATE: Complete working solution:
// controller
s := make([]interface{}, len(res))
for i, v := range res {
s[i] = v
}
_, err := policy.CreateItem(c.DB, s)
// model
func CreateItem(db Connection, res []interface{}) (sql.Result, error) {
tfcim := format.TitlesForCreateInputModel(ItemMin{}) // parent_id, title, description, sort, date, status_id, user_id
phfcm := format.PlaceHoldersForCreateModel(ItemMin{}) // ?,?,?,?,?,?,?
query := fmt.Sprintf("INSERT INTO %v (%v) VALUES (%v)", table, tfcim, phfcm)
result, err := db.Exec(query, res...)
return result, err
}
As the Exec function has this signature:
Exec(query string, args ...interface{})
You can use this syntax:
query := fmt.Sprintf("INSERT INTO %v (%v) VALUES (%v)", table, tfcim, phfcm)
result, err := db.Exec(query, res...)

UPDATE multiple rows with different values in one query

I want to update multiple rows with different values, in one query. I know somehow this operation in SQL syntax. But implementing it into Go looks a bit confusing to me. For instance, I want to update the status column based on the menu_id column.
My code works for the first row but other rows value changes to null
updateSql := "UPDATE tb_menu SET tb_menu.status =( case" // set query statement
condition := " WHERE tb_menu.menu_id IN ("
menuList := []interface{}{} //set empty interface array
//counter := 0
for _, row := range status {
menuList = append(menuList, row.Menu_id, row.Status, row.Menu_id)
updateSql += (" WHEN tb_menu.menu_id = ? THEN ? ")
condition += "?,"
}
updateSql += "end)"
condition = strings.TrimSuffix(condition, ",") // remove last "," from query statement
condition += ") "
updateSql += condition
update, err := db.Prepare(updateSql)
if err != nil {
fmt.Println(" error occured on Preparedb function ", err)
res = false
defer db.Close()
return res
}
set, err := update.Exec(menuList...) // query data to db
if err != nil {
fmt.Println("error occured while updating menu list ", err)
res = false
defer db.Close()
return res
}

SQL variable not updating in the select query in go client

I am running the following queries in SQL.
SET #thisid=0;
SET #serial=0;
SELECT #serial := IF((#thisid != `places`.`id`), #serial + 1, #serial) as `serial`, #thisid := `places`.`id`, `places`.`id` FROM `places`;
The variable #serial basically increments only if the new id is not the same as the last one.
Upon running these queries in the terminal and printing the values of #serial and #thisid, the value received is #thisid='id6' #serial=6.
I executed this query in my go code:
if _, err = repo.db.ExecContext(ctx, "SET #thisid=0;"); err != nil {
return
}
if _, err = repo.db.ExecContext(ctx, "SET #serial=0;"); err != nil {
return
}
rows, err = repo.db.QueryContext(ctx, fmt.Sprintf(
"SELECT #serial := IF((#thisid != `places`.`id`), #serial + 1, #serial) as `serial`, #thisid := `places`.`id`, `places`.`id` FROM `places`;",
))
if err != nil {
fmt.Println("error here")
return
}
if err = repo.db.QueryRow("SELECT #serial").Scan(&that); err != nil {
return
}
if err = repo.db.QueryRow("SELECT #thisid").Scan(&this); err != nil {
return
}
Upon printing the values of #thisid and #serial, the value of #thisid is received the same where the value of #serial is received as 0. It doesn't seem to update dynamically.
Go uses a connection pool, which means each query may happen on a different connection. variables like that are scoped to the connection. If you need them to last between queries, you need to use a transaction to ensure you stay within the same connection.
Your query is really arbitrary. MySQL does not guarantee the order of evaluation of expressions in a select. Nor does it guarantee the ordering of the result set.
So, I think you want:
select p.*,
(#rn := if(#id = id, #rn + 1,
if(#id := id, 1, 1)
)
) as serial
from (select p.*
from places p
order by p.id
) p cross join
(select #id := 0, #rn := 0) params;

Count number of rows in golang

I want to display the number of rows from database using Go. How do I display number of rows?
count, err := db.Query("SELECT COUNT(*) FROM main_table")
The query will return a row into the variable count. So the next you have to do is to read this row and assign the result into a new variable, using the function Scan(). This is how it works.
rows, err := db.Query("SELECT COUNT(*) FROM main_table")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
var count int
for rows.Next() {
if err := rows.Scan(&count); err != nil {
log.Fatal(err)
}
}
fmt.Printf("Number of rows are %s\n", count)
The best option thought would be to use QueryRow() as you expect to read just one row. The code then will be.
var count int
err := db.QueryRow("SELECT COUNT(*) FROM main_table").Scan(&count)
switch {
case err != nil:
log.Fatal(err)
default:
fmt.Printf("Number of rows are %s\n", count)
}
I signed up just to share this as my large datasets were slowing down the program with the constant appends. I wanted to put the rowcount in the rows and figured that something like that must exist, just had to find it.
SELECT
count(1) OVER(PARTITION BY domain_id) AS rowcount,
subscription_id,
domain_id,
...
FROM mytable
WHERE domain_id = 2020
Never used this command before, but it will add the count of the result set that share this parameter. Setting it to one of the query WHERE's makes it the total rows.