this is less of a question about golang or mysql, its more a general question. Hope i am still in the right place and someone could help me to wrap my head around this.
I have a struct Role which can have multiple child roles.
type Role struct{
Name string
Children []Role
}
So let's say Role A has a child Role B and Role B has a Child Role C.
In my frontend, the m2m relation is displayed as a multi-select HTML field.
To avoid an infinite loop (A-B-C-A...) I want that the user can not enter one of the related Roles.
For example, Role C should not display Role A and B, because if a user would select those, an infinite loop would happen.
The database in the backend is looking like this:
roles table (main table)
id, name, ...
role_roles (junction table)
role_id, child_id
I created this helper method to detect the ids which should not get displayed. It's checking if the Role C is somewhere in the field child_id then it takes the role_id of this entry and is doing the same again. This works, but it is looking really unprofessional and I was wondering how this could be solved in a more elegant way - and with fewer SQL queries...
// whereIDLoop returns the ids which should get excluded
func whereIDLoop(id int) ([]int, error) {
ids := []int{}
b := builder.GlobalBuilder
rows, err := b.Select("role_roles").Columns("role_id").Where("child_id = ?", id).All()
if err != nil {
return nil, err
}
for rows.Next() {
var id int
if err := rows.Scan(&id); err != nil {
return nil,err
}
ids = append(ids, id)
id2,err := whereIDLoop(id)
if err != nil {
return nil, err
}
if id2 != nil{
ids = append(ids, id2...)
}
}
err = rows.Close()
if err != nil {
return nil, err
}
return ids, nil
}
Thanks for any help.
Cheers Pat
can't say best practice, my suggestion is put the validation logic at application layer.
a helper function in JS to filter the option in multi-select like this
a validation logic in API
a validation logic at repository layer.
track the roles when apply in case there is a circle.
Related
Let's say I have a web app that shows a list of posts. The post struct is:
type Post struct {
Id int64 `sql:",primary"`
Title string
Body string
}
It retrieves the posts with:
var posts []*Post
rows, err := db.QueryContext(ctx, "select * from posts;")
if err != nil {
return nil, oops.Wrapf(err, "could not get posts")
}
defer rows.Close()
for rows.Next() {
p := &Post{}
err := rows.Scan(
&p.Id,
&p.Title,
&p.Body,
)
if err != nil {
return nil, oops.Wrapf(err, "could not scan row")
}
posts = append(posts, p)
}
return posts, nil
All works fine. Now, I want to alter the table schema by adding a column:
ALTER TABLE posts ADD author varchar(62);
Suddenly, the requests to get posts result in:
sql: expected 4 destination arguments in Scan, not 3
which makes sense since the table now has 4 columns instead of the 3 stipulated by the retrieval logic.
I can then update the struct to be:
type Post struct {
Id int64 `sql:",primary"`
Title string
Body string
Author string
}
and the retrival logic to be:
for rows.Next() {
p := &Post{}
err := rows.Scan(
&p.Id,
&p.Title,
&p.Body,
&p.Author
)
if err != nil {
return nil, oops.Wrapf(err, "could not scan row")
}
posts = append(posts, p)
}
which solves this. However, this implies there is always a period of downtime between migration and logic update + deploy. How to avoid that downtime?
I have tried swapping the order of the above changes but this does not work, with that same request resulting in:
sql: expected 3 destination arguments in Scan, not 4
(which makes sense, since the table only has 3 columns at that point as opposed to 4);
and other requests resulting in:
Error 1054: Unknown column 'author' in 'field list'
(which makes sense, because at that point the posts table does not have an author column just yet)
You should be able to achieve your desired behavior by adapting the SQL Query to return the exact fields you want to populate.
SELECT Id , Title , Body FROM posts;
This way even if you add another column Author the query results only contain 3 values.
I am developing a system to enable patient registration with incremental queue number. I am using Go, GORM, and MySQL.
An issue happens when more than one patients are registering at the same time, they tend to get the same queue number which it should not happen.
I attempted using transactions and hooks to achieve that but I still got duplicate queue number. I have not found any resource about how to lock the database when a transaction is happening.
func (r repository) CreatePatient(pat *model.Patient) error {
tx := r.db.Begin()
defer func() {
if r := recover(); r != nil {
tx.Rollback()
}
}()
err := tx.Error
if err != nil {
return err
}
// 1. get latest queue number and assign it to patient object
var queueNum int64
err = tx.Model(&model.Patient{}).Where("registration_id", pat.RegistrationID).Select("queue_number").Order("created_at desc").First(&queueNum).Error
if err != nil && err != gorm.ErrRecordNotFound {
tx.Rollback()
return err
}
pat.QueueNumber = queueNum + 1
// 2. write patient data into the db
err = tx.Create(pat).Error
if err != nil {
tx.Rollback()
return err
}
return tx.Commit().Error
}
As stated by #O. Jones, transactions don't save you here because you're extracting the largest value of a column, incrementing it outside the db and then saving that new value. From the database's point of view the updated value has no dependence on the queried value.
You could try doing the update in a single query, which would make the dependence obvious:
UPDATE patient AS p
JOIN (
SELECT max(queue_number) AS queue_number FROM patient WHERE registration_id = ?
) maxp
SET p.queue_number = maxp.queue_number + 1
WHERE id = ?
In gorm you can't run a complex update like this, so you'll need to make use of Exec.
I'm not 100% certain the above will work because I'm less familiar with MySQL transaction isolation guarantees.
A cleaner way
Overall, it'd be cleaner to keep a table of queues (by reference_id) with a counter that you update atomically:
Start a transaction, then
SELECT queue_number FROM queues WHERE registration_id = ? FOR UPDATE;
Increment the queue number in your app code, then
UPDATE queues SET queue_number = ? WHERE registration_id = ?;
Now you can use the incremented queue number in your patient creation/update before transaction commit.
I'd like to get back the id of the last post inserted into MySql database using sqlx:
resultPost, err := shared.Dbmap.Exec("INSERT INTO post (user_id, description, link) VALUES (?, ?, ?)", userID, title, destPath)
if err != nil {
log.Println(err)
c.JSON(
http.StatusInternalServerError,
gin.H{"error": "internal server error"})
}
fmt.Println("resultPost is:", resultPost)
The problem is that the resultPost is printed as an object:
resultPost is: {0xc420242000 0xc4202403a0}
So I'm wondering what is the correct way to extract the id of the row just inserted?
The return value from Exec, Result is not meant to be accessed directly--it's an object with two methods to call, one of which is LastInsertId().
lastId, err := resultPost.LastInsertId()
if err != nil {
panic(err)
}
fmt.Println("LastInsertId: ", lastId)
Looks like you just need :
resultPost.LastInsertId()
For more information, search for LastInsertId in this documentation
Suppose that I've the following structures that it's the mapped tables.
type Publisher struct{
ID int `db:"id"`
Name string `db:"name"`
Books []*Book
}
type Book struct {
ID int `db:"id"`
Name string `db:"name"`
PublisherID `db:"publisher_id"`
}
So, What if I wanna retrieve all the Publisher with all related Books I would like to get a JSON like this:
[ //Publisher 1
{
"id" : "10001",
"name":"Publisher1",
"books" : [
{ "id":321,"name": "Book1"},
{ "id":333,"name": "Book2"}
]
},
//Publisher 2
{
"id" : "10002",
"name":"Slytherin Publisher",
"books" : [
{ "id":4021,"name": "Harry Potter and the Chamber of Secrets"},
{ "id":433,"name": "Harry Potter and the Order of the Phoenix"}
]
},
]
So I've the following structure that I use to retrieve all kind of query related with Publisher
type PublisherRepository struct{
Connection *sql.DB
}
// GetEbooks return all the books related with a publisher
func (r *PublisherRepository) GetBooks(idPublisher int) []*Book {
bs := make([]Book,0)
sql := "SELECT * FROM books b WHERE b.publisher_id =$1 "
row, err := r.Connection.Query(sql,idPublisher)
if err != nil {
//log
}
for rows.Next() {
b := &Book{}
rows.Scan(&b.ID, &b.Name, &b.PublisherID)
bs := append(bs,b)
}
return bs
}
func (r *PublisherRepository) GetAllPublishers() []*Publisher {
sql := "SELECT * FROM publishers"
ps := make([]Publisher,0)
rows, err := r.Connection.Query(sql)
if err != nil {
// log
}
for rows.Next() {
p := &Publisher{}
rows.Scan(&p.ID,&p.Name)
// Is this the best way?
books := r.GetBooks(p.ID)
p.Books = books
}
return ps
}
So , here my questions
What is the best approach to retrieve all the publisher with the best performance, because a for inside a for is not the best solution, what if I've 200 publisher and in the average of each publisher has 100 books.
Is in GoLang idiomatic PublisherRepository or is there another way to create something to manage the transactions of an entity with pure sql?
1) Bad about this would be the sql request per iteration. So here a solution that does not make an extra request per Publisher:
func (r *PublisherRepository) GetAllPublishers() []*Publisher {
sql := "SELECT * FROM publishers"
ps := make(map[int]*Publisher)
rows, err := connection.Query(sql)
if err != nil {
// log
}
for rows.Next() {
p := &Publisher{}
rows.Scan(&p.ID,&p.Name)
ps[p.ID] = p
}
sql = "SELECT * FROM books"
rows, err := connection.Query(sql)
if err != nil {
//log
}
for rows.Next() {
b := &Book{}
rows.Scan(&b.ID, &b.Name, &b.PublisherID)
ps[b.PublisherID].Books = append(ps[b.PublisherID].Books, b)
}
// you might choose to keep the map as a return value, but otherwise:
// preallocate memory for the slice
publishers := make([]*Publisher, 0, len(ps))
for _, p := range ps {
publishers = append(publishers, p)
}
return publishers
}
2) Unless you create the PublisherRepository only once, this might be a bad idea creating and closing loads of connections. Depending also on your sql client implementation I would suggest (and also have seen it for many other go database clients) to have one connection for the entire server. Pooling is done internally by many of the sql clients, that is why you should check your sql client.
If your sql client library does pooling internally use a global variable for the "connection" (it's not really one connection if pooling is done internally):
connection *sql.DB
func New () *PublisherRepository {
repo := &PublisherRepository{}
return repo.connect()
}
type PublisherRepository struct{
}
func (r *PublisherRepository) connect() *PublisherRepository {
// open new connection if connection is nil
// or not open (if there is such a state)
// you can also check "once.Do" if that suits your needs better
if connection == nil {
// ...
}
return r
}
So each time you create a new PublisherRepository, it will only check if connection already exists. If you use once.Do, go will only create the "connection" once and you are done with it.
If you have other structs that will use the connection as well, you need a global place for your connection variable or (even better) you write a little wrapper package for your sql client, that is in turn used in all your structs.
In your case, the simplest way would be to use json_agg in the query. Like here http://sqlfiddle.com/#!15/97c41/4 (sqlfiddle is slow so here is screenshot http://i.imgur.com/hxMPkUa.png) Not very Go friendly (you need to unmarshal query result data if you want to do something with the books) but all books in one query as you wanted without for loops.
As #TehSphinX said it is better to have single global db connection.
But before implementing strange queries I really suggest you to think: why do you need to return the full list of publishers and their books in one API query? I can't imagine the situation in web or mobile app where your idea might be a good decision. Usually, you just show users list of publishers then users chooses one and you show him the list of books by this publisher. This is "win-win" situation for you and your users - you can make simple queries and your users just get small sets of data that they actually need without paying for unnecessary traffic/wasting browser memory. As you said there can be 200 publishers with 100 books and I'm sure your users don't need 20000 books loaded in one request. Of course, if you are not trying to make your API more data theft friendly.
Even if you have something like a short preview-like list of books for each publisher you should think about pagination for publishers and/or denormalisation of books data for this case (add a column to publishers table with the short list of books in JSON format).
Using the golang example below, how can I query (JOIN) multiple databases.
For example, I want to have the relation db1.username.id = db2.comments.username_id.
id := 123
var username string
err := db.QueryRow("SELECT username FROM users WHERE id=?", id).Scan(&username)
switch {
case err == sql.ErrNoRows:
log.Printf("No user with that ID.")
case err != nil:
log.Fatal(err)
default:
fmt.Printf("Username is %s\n", username)
}
As you are using MySQL, you can select fields across databases. See this related question for
details. For example you should be able to do this:
err := db.QueryRow(`
SELECT
db1.users.username
FROM
db1.users
JOIN
db2.comments
ON db1.users.id = db2.comments.username_id
`).Scan(&username)
You can of course simply fetch all entries from db2.comments using a second database connection and use the values in a query to db1.users. This is, of course, not recommended as it is the job of the database server which it can, most likely, do better than you.