I can't seem to dynamically ORDER BY with db.Select(). I've Googled without any luck...
WORKS
rows, err := db.Query("SELECT * FROM Apps ORDER BY title DESC")
DOES NOT WORK
rows, err := db.Query("SELECT * FROM Apps ORDER BY ? DESC", "title")
I'm not getting any errors, the query simply fails to order.
Placeholders ('?') can only be used to insert dynamic, escaped values for filter parameters (e.g. in the WHERE part), where data values should appear, not for SQL keywords, identifiers etc. You cannot use it to dynamically specify the ORDER BY OR GROUP BY values.
You can still do it though, for example you can use fmt.Sprintf() to assemble the dynamic query text like this:
ordCol := "title"
qtext := fmt.Sprintf("SELECT * FROM Apps ORDER BY %s DESC", ordCol)
rows, err := db.Query(qtext)
Things to keep in mind:
Doing so you will have to manually defend vs SQL injection, e.g. if the value of the column name comes from the user, you cannot accept any value and just insert it directly into the query else the user will be able to do all kinds of bad things. Trivially you should only accept letters of the English alphabet + digits + underscore ('_').
Without attempting to provide a complete, all-extensive checker or escaping function, you can use this simple regexp which only accepts English letters, digits and '_':
valid := regexp.MustCompile("^[A-Za-z0-9_]+$")
if !valid.MatchString(ordCol) {
// invalid column name, do not proceed in order to prevent SQL injection
}
Examples (try it on the Go Playground):
fmt.Println(valid.MatchString("title")) // true
fmt.Println(valid.MatchString("another_col_2")) // true
fmt.Println(valid.MatchString("it's a trap!")) // false
fmt.Println(valid.MatchString("(trap)")) // false
fmt.Println(valid.MatchString("also*trap")) // false
Related
I've been stuck on a GORM issue for about a full day now. I need to be able to filter a messages table on any of 4 things: sender, recipient, keyword, and date range. It also has to paginate. Filtering by sender and recipient is working, and so is pagination. So far this is the query that I have come up with, but it does not seem to work for date ranges or keywords.
Here is how I am selecting from MySQL
db.Preload("Thread").Where(query).Scopes(Paginate(r)).Find(&threadMessages)
I am creating the query like this:
var query map[string]interface{}
Then based on which parameters I am passed, I update the query like this by adding new key values to the map:
query = map[string]interface{}{"user_id": sender, "recipient_id": recipient}
For dates it does not seem to work if I try something like this:
query = map[string]interface{}{"created_at > ?": fromDate}
And for a LIKE condition is also does not seem to work:
query = map[string]interface{}{"contents LIKE ?": keyword}
The reason I chose this approach is that I could not seem to get optional inputs to work in .Where since it takes a string with positional parameters and null positional parameters seem to cause MySQL to return an empty array. Has anyone else dealt with a complicated GORM issue like this? Any help is appreciated at this point.
Passing the map[string]interface{} into Where() only appears to work for Equals operations, or IN operations (if a slice is provided as the value instead).
One way to achieve what you want, is to construct a slice of clause.Expression, and append clauses to the slice when you need to. Then, you can simply pass in all of the clauses (using the ... operator to pass in the whole slice) into db.Clauses().
clauses := make([]clause.Expression, 0)
if mustFilterCreatedAt {
clauses = append(clauses, clause.Gt{Column: "created_at", fromDate})
}
if mustFilterContents {
clauses = append(clauses, clause.Like{Column: "contents", Value: keyword})
}
db.Preload("Thread").Clauses(clauses...).Scopes(Paginate(r)).Find(&threadMessages)
Note: If you're trying to search for content that contains keyword, then you should concatenate the wildcard % onto the ends of keyword, otherwise LIKE behaves essentially the same as =:
clause.Like{Column: "contents", Value: "%" + keyword + "%"}
My final solution to this was to create dynamic Where clauses based on which query params were sent from the client like this:
fields := []string{""}
values := []interface{}{}
If, for example, there is a keyword param:
fields = []string{"thread_messages.contents LIKE ?"}
values = []interface{}{"%" + keyword + "%"}
And to use the dynamic clauses in the below query:
db.Preload("Thread", "agency_id = ?", agencyID).Preload("Thread.ThreadUsers", "agency_id = ?", agencyID).Joins("JOIN threads on thread_messages.thread_id = threads.id").Where("threads.agency_id = ?", agencyID).Where(strings.Join(fields, " AND "), values...).Scopes(PaginateMessages(r)).Find(&threadMessages)
I've already setup and pinged my mysql database connection. It is working and I can return rows using both db.Query and by preparing a query first. I can use the placeholder ? to then specify an id. Is it possible to use the ? as a placeholder for a column name? In the example here I am trying to return all rows from column firstName in table persons.
qry, err := db.Prepare("SELECT ? FROM persons")
if err != nil { log.Fatal(err) }
defer qry.Close()
rows, err :=qry.Query("firstName")
if err != nil { log.Fatal(err) }
defer rows.Close()
I get the following error:
Error 1064: You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version for the right
syntax to use near '?' at line 1
You can't use placeholders for identifiers (such as table and column names), placeholders are for values. You can think of identifiers as being similar to variable or function names in Go so being able to use placeholders for identifiers would be akin to having an eval as in various scripting languages.
This reduces you to using fmt.Sprintf and similar string operations for building the SQL when you don't know the identifiers until runtime:
col := "firstName"
sql := fmt.Sprintf("select %s from persons", col)
but this opens you up to SQL injection and quoting problems so you'd want some sort of whitelist:
quotedColumns := map[string]string{
"firstName": "`firstName`",
"lastName": "`lastName`",
...
}
quoted, ok := quotedColumns[columnName]
if !ok {
// Do something with the error here and run away...
}
sql := fmt.Sprintf("select %s from persons", quoted)
Note that I've included the MySQL backtick quoting in the map's values. There's nothing in the standard interface for quoting/escaping an identifier so you have to do it yourself. If you're already writing the whitelist map by hand then you may as well include the quoting by hand too; otherwise you could write your own quoting function for identifiers by reading the MySQL documentation on quoting and doing a couple (hopefully) simple string operations.
I Need help for fetch records from table using Go.
My Problem is that i'm writing MySQL query and add another where clause i.e HPhone number, Here HPhone number inserted in data base with format like 999-999-9999.And i passed this HPhone Number in format like 9999999999. which is not matching with correct data base field value. And i used SUBSTRING for add hyphen between numbers but it does not get records but when i passed like 999-999-9999 without SUBSTRING it return records.
Here i demonstrate how i used this.
strQry = `SELECT * from table WHERE Depot = ?`
if HPhone != "" {
strQry += ` AND HPhone = ?`
}
queryArgs := []interface{}{RouteAvailability.Depot}
if HPhone != "" {
queryArgs = append(queryArgs, "SUBSTRING("+HPhone+",1,3)"+"-"+"SUBSTRING("+HPhone+",4,3)"+"-"+"SUBSTRING("+HPhone+",7,4)")
}
Help would be appreciated.
Thanks in advance.
Instead of SUBSTRING you can use REPLACE like so:
queryArgs := []interface{}{RouteAvailability.Depot}
if HPhone != "" {
strQry += ` AND REPLACE(HPhone, '-', '') = ?`
queryArgs = append(queryArgs, HPhone)
}
If possible I would suggest you normalize your data, i.e. decide on a canonical format for a particular data type and everytime your program receives some input that contains that data type you format it into its canonical form, that way you can avoid having to deal with SUBSTRING, or REPLACE, or multiple inconsistent formats etc.
This won't work as you are using prepared statements, and the argument you are building when HPhone is not empty will be used in escaped form - so when executing the query, it won't compare the HPhone values with the computed result of some substring, but with a string containing SUBSTRING(9999...
How can I build a dynamic query depending on the parameters that I get?
This example is stupid and the syntax is wrong but you will get the idea of what I want.
I guess that I need to add a slice of variables to the end of the query.
I know how to do it in PHP, but not in golang.
db := OpenDB()
defer db.Close()
var filter string
if name != "" {
filter = filter " AND name = ?"
}
if surname != "" {
filter = filter + " AND surname = ?"
}
if address != "" {
filter = filter + " AND address = ?"
}
err = db.Query("SELECT id FROM users WHERE login = ?" +
filter, login)
To answer your question on how to format the string, the simple answer is to use fmt.Sprintf to structure your string. However see further down for a quick note on using fmt.Sprintf for db queries:
Sprintf formats according to a format specifier and returns the resulting string.
Example:
query := fmt.Sprintf("SELECT id FROM users WHERE login='%s'", login)
err = db.Query(query)
// Equivalent to:
rows, err := db.Query("SELECT id FROM users WHERE login=?", login)
Using this for queries, you're safe from injections. That being said, you might be tempted to modify this and use db.Exec for creations/updates/deletions as well. As a general rule of thumb, if you use db.Exec with fmt.Sprintf and do not sanitize your inputs first, you open yourself up to sql injections.
GoPlay with simple example of why fmt.Sprintf with db.Exec is bad:
https://play.golang.org/p/-IWyymAg_Q
You should use db.Query or db.Prepare in an appropriate way to avoid these sorts of attack vectors. You might have to modify the code sample above to come up with a injection-safe snippet, but hopefully I gave you enough to get started.
I am trying to fetch multiple columns in a mysql database using the go language. Currently I could modify this script, and it would work perfectly well for just fetching one column, and printing it using the http print function. However, when it fetches two things the script no longer works. I don't have any idea what I need to do to fix it. I know that the sql is fine as I have tested that out in the mysql terminal, and it has given me the expected result that I wanted.
conn, err := sql.Open("mysql", "user:password#tcp(localhost:3306)/database")
statement, err := conn.Prepare("select first,second from table")
rows, err := statement.Query()
for rows.Next() {
var first string
rows.Scan(&first)
var second string
rows.Scan(&second)
fmt.Fprintf(w, "Title of first is :"+first+"The second is"+second)
}
conn.Close()
You are using the wrong variable names but I assume that's just a "typo" in your sample code.
Then the correct syntax is:
var first, second string
rows.Scan(&first, &second)
See this on what Scan(dest ...interface{}) means and how to use it.
You should also handle and not ignore the errors.
Finally, you should use Fprintf as it's intended:
fmt.Fprintf(w, "Title of first is : %s\nThe second is: %s", first, second)