Creating GORM dynamic query with optional paramters - mysql

I've been stuck on a GORM issue for about a full day now. I need to be able to filter a messages table on any of 4 things: sender, recipient, keyword, and date range. It also has to paginate. Filtering by sender and recipient is working, and so is pagination. So far this is the query that I have come up with, but it does not seem to work for date ranges or keywords.
Here is how I am selecting from MySQL
db.Preload("Thread").Where(query).Scopes(Paginate(r)).Find(&threadMessages)
I am creating the query like this:
var query map[string]interface{}
Then based on which parameters I am passed, I update the query like this by adding new key values to the map:
query = map[string]interface{}{"user_id": sender, "recipient_id": recipient}
For dates it does not seem to work if I try something like this:
query = map[string]interface{}{"created_at > ?": fromDate}
And for a LIKE condition is also does not seem to work:
query = map[string]interface{}{"contents LIKE ?": keyword}
The reason I chose this approach is that I could not seem to get optional inputs to work in .Where since it takes a string with positional parameters and null positional parameters seem to cause MySQL to return an empty array. Has anyone else dealt with a complicated GORM issue like this? Any help is appreciated at this point.

Passing the map[string]interface{} into Where() only appears to work for Equals operations, or IN operations (if a slice is provided as the value instead).
One way to achieve what you want, is to construct a slice of clause.Expression, and append clauses to the slice when you need to. Then, you can simply pass in all of the clauses (using the ... operator to pass in the whole slice) into db.Clauses().
clauses := make([]clause.Expression, 0)
if mustFilterCreatedAt {
clauses = append(clauses, clause.Gt{Column: "created_at", fromDate})
}
if mustFilterContents {
clauses = append(clauses, clause.Like{Column: "contents", Value: keyword})
}
db.Preload("Thread").Clauses(clauses...).Scopes(Paginate(r)).Find(&threadMessages)
Note: If you're trying to search for content that contains keyword, then you should concatenate the wildcard % onto the ends of keyword, otherwise LIKE behaves essentially the same as =:
clause.Like{Column: "contents", Value: "%" + keyword + "%"}

My final solution to this was to create dynamic Where clauses based on which query params were sent from the client like this:
fields := []string{""}
values := []interface{}{}
If, for example, there is a keyword param:
fields = []string{"thread_messages.contents LIKE ?"}
values = []interface{}{"%" + keyword + "%"}
And to use the dynamic clauses in the below query:
db.Preload("Thread", "agency_id = ?", agencyID).Preload("Thread.ThreadUsers", "agency_id = ?", agencyID).Joins("JOIN threads on thread_messages.thread_id = threads.id").Where("threads.agency_id = ?", agencyID).Where(strings.Join(fields, " AND "), values...).Scopes(PaginateMessages(r)).Find(&threadMessages)

Related

How to query using an IN clause and a `Vec` as parameter in Rust sqlx for MySQL?

Note: this is a similar but NOT duplicate question with How to use sqlx to query mysql IN a slice?. I'm asking for the Rust one.
This is what I try to do.
let v = vec![..];
sqlx::query("SELECT something FROM table WHERE column IN (?)").bind(v)
...
Then I got the following error
the trait bound `std::vec::Vec<u64>: sqlx::Encode<'_, _>` is not satisfied
Answer is in first on FAQ https://github.com/launchbadge/sqlx/blob/master/FAQ.md
How can I do a SELECT ... WHERE foo IN (...) query? In 0.6 SQLx will
support binding arrays as a comma-separated list for every database,
but unfortunately there's no general solution for that currently in
SQLx itself. You would need to manually generate the query, at which
point it cannot be used with the macros.
The error shows Vec is not an Encode that is required to be as a valid DB value. The Encode doc lists all the Rust types that have implemented the trait. Vec is not one.
You can use the following way to bind the parameters in IN with the values of a vector. Firstly, you need to expand the number of '?' in the IN expression to be the same number of the parameters. Then, you need to call bind to bind the values one by one.
let v = vec![1, 2];
let params = format!("?{}", ", ?".repeat(v.len()-1));
let query_str = format!("SELECT id FROM test_table WHERE id IN ( { } )", params);
let mut query = sqlx::query(&query_str);
for i in v {
query = query.bind(i);
}
let row = query.fetch_all(&pool).await?;
Please note if the target database is not MySql, you need to use $n, like $1, $2, instead of ?, as the parameter placeholder.

sequelize fulltext query using parameters

Hi I'm currently trying to query records from db and these are the conditions
I receive 'order by', 'order (desc/asc)', 'limit', 'offset' from the frontend
I also need to search the record using match...against. 'like' is too slow for searching.
There's a mapped model with this query.
so I tried
let order_by = req.query.orderby;
let order = req.query.order;
let page = req.query.pagenum;
let perpage = req.query.parpage;
let searchword = req.query.foodsearch;
let offset = (parseInt(page) - 1) * parpage;
let foods = await models.food.findAll({
limit: parseInt(perpage),
offset: offset,
order: [
[order_by, order]
],
// where: Sequelize.literal
// (
// `MATCH
// (Name, Place, RestoNum, Ingredient, ChefName, Region...)
// AGAINST
// ( ? IN NATURAL LANGUAGE MODE)`,
// { replacements: [ searchword ] }
// )
});
but the commented part seems wrong in this code.
I tried the raw query, but then I can't parameterize those order by, order, offset, limit variables.
I don't want to just add them like ${orderby} because it's risky.
Please let me know if you have any solution for this issue.
Thank you in advance!
You're confusing the Sequelize.literal() and sequelizeInstance.query() APIs.
.literal() only take a string. If you want to use the object notation for your query, your commented code will work. Except that there is no second argument. You will need to concatenate-in or interpolate-in your search term into the AGAINST clause. Also, don't forget your quotes. The output of the literal() is essentially a string. Your MySQL FTS parameter will need the correct type of quotes around it, just as they would appear in your raw SQL query.
.query() DOES take an options parameter. Through this, you don't have to use string interpolation, you can use named replacements or bound-parameters. This will not only allow you to place in your searchword parameter, but whatever ORDER BY clause you want, as well.
I would go with Option 1. That's what we are doing for our FTS, in MS SQL.

Multi-parameter search with mysql and node.js

Let me preface by saying I'm very new to SQL (and back end design) in general. So for those annoyed with noob questions, please be gentle.
BACKGROUND:
I'm trying to build a product test database (storing test data for all our products) where I want a user to be able to refine a search to find test data they actually want. For example, they may start by searching for all products of a certain brand name, and then refine it with a product type, and/or refine it with a date range of when the test was done.
PROBLEM:
I'm having a hard time finding information on how to implement multi-parameter searches with mysql and node.js. I know you can do nested queries and joins and such within pure SQL syntax, but it's not abundantly clear to me how I would do this from node.js, especially when certain search criteria aren't guaranteed to be used.
Ex:
CREATE PROCEDURE `procedureName`(
IN brandname VARCHAR(20),
producttype VARCHAR(30))
BEGIN
SELECT * FROM products
WHERE brand = brandname
AND product_type = producttype;
END
I know how to pass data from node.js to this procedure, but what if the user didn't specify a product type? Is there a way to nullify this part of the query? Something like:
AND product_type = ALL;
WHAT I'VE TRIED:
I've also looked into nesting multiple SQL procedures, but passing in dynamic data to the "FROM" clause doesn't seem to be possible. Ex: if I had a brandname procedure, and a product type procedure, I don't know how/if I can pass the results from one procedure to the "FROM" clause of the other to actually refine the search.
One idea was to create tables with the results in each of these procedures, and pass those new table names to subsequent procedures, but that strikes me as an inefficient way to do this (Am I wrong? Is this a completely legit way to do this?).
I'm also looking into building a query string on the node side that would intelligently decide what search criteria have been specified by the front end, and figure out where to put SQL AND's and JOIN's and what-nots. The example below actually works, but this seems like it could get ugly quick as I add more search criteria, along with JOINS to other tables.
// Build a SQL query based on the parameters in a request URL
// Example request URL: http://localhost:3000/search?brand=brandName&type=productType
function qParams(req) {
let q = "SELECT * FROM products WHERE ";
let insert = [];
if(req.query.brand) {
brandname = req.query.brand; // get brandname from url request
q = q + `brand = ?`, // Build brandname part of WHERE clause
insert.push(brandname); // Add brandname to insert array to be used with query.
};
if(req.query.type) {
productType = req.query.type; // get product type from url request
insert.length > 0 ? q = q + ' AND ' : q = q; // Decide if this is the first search criteria, add AND if not.
q = q + 'product_type = ?'; // Add product_type to WHERE clause
insert.push(productType); // Add product_type variable to insert array.
}
// Return query string and variable insert array
return {
q: q,
insert: insert
};
};
// Send Query
async function qSend(req, res) {
const results = await qParams(req); // Call above function, wait for results
// Send query string and variables to MySQL, send response to browser.
con.query(results.q, results.insert, (err, rows) => {
if(err) throw err;
res.send(rows);
res.end;
})
};
// Handle GET request
router.use('/search', qSend);
CONCISE QUESTIONS:
Can I build 1 SQL procedure with all my search criteria as variables, and nullify those variables from node.js if certain criteria aren't used?
Is there way to nest multiple MySQL procedures so I can pick the procedures applicable to the search criteria?
Is creating tables of results in a procedure, and passing those new table names to other procedures a reasonable way to do that?
Building the query from scratch in node is working, but it seems bloated. Is there a better way to do this?
Googling "multi-parameter search mysql nodejs" is not producing useful results for my question, i.e. I'm not asking the right question. What is the right question? What do I need to be researching?
One option is to use coalesce():
SELECT p.*
FROM products p
WHERE
p.brand = COALESCE(:brandname, p.brand)
AND p.product_type = COALESCE(:producttype, p.producttype);
It may be more efficient do explicit null checks on the parameters:
SELECT p.*
FROM products p
WHERE
(:brandname IS NULL OR p.brand = :brandname)
AND (:producttype IS NULL OR p.product_type = :producttype);

Erlang and mysql

I have a users table in mysql with userid, username, password etc
Using mysql-otp i query like
select * from users where username = ?
Its returns ColumnNames and Rows.
Rows are like
[[38, <<"joe">>, <<"passwordhash">>..]]
Suppose i have a hash to compare
Hash = "passhash".
As its my 3rd day coding in erlang, what i am currently doing/testing is
[[_, _, UserPass, _,..]] = Rows.
Which stores password in UserPass.
Pass = binary_to_list(UserPass).
Which i can then compare like
Hash == Pass.
Is this approach correct or i am doing it all wrong?
There must be a proper way of getting data out of what is supposedly list inside a list.
There must be a proper way of getting data out of what is supposedly
list inside a list
The proper way to get at your target data is to use pattern matching to deconstruct whatever type of collection contains your data. Because mysql-otp returns a list of rows matching the query, where each row itself is a list, the data is in the form of a list of lists. Therefore, in order to match a list of lists your pattern also has to be a list of lists.
As its my 3rd day coding in erlang, what i am currently doing/testing
is
[[_, _, UserPass, _,..]] = Rows.
Bravo.
If you know mysql-otp will only return one row, or you are only interested in the first row, you could simplify the pattern like this:
[_, _, UserPass, _, ...] = hd(Rows).
hd is shorthand for head, i.e. the head of the list. Or, you could accomplish the same thing by hand like this:
[FirstRow | _Tail] = Rows,
[_, _, UserPass, _, ...] = FirstRow.
=======
Another way to extract the password:
UserPass = lists:nth(3, hd(Rows)).

Fetch records with query Args in Go

I Need help for fetch records from table using Go.
My Problem is that i'm writing MySQL query and add another where clause i.e HPhone number, Here HPhone number inserted in data base with format like 999-999-9999.And i passed this HPhone Number in format like 9999999999. which is not matching with correct data base field value. And i used SUBSTRING for add hyphen between numbers but it does not get records but when i passed like 999-999-9999 without SUBSTRING it return records.
Here i demonstrate how i used this.
strQry = `SELECT * from table WHERE Depot = ?`
if HPhone != "" {
strQry += ` AND HPhone = ?`
}
queryArgs := []interface{}{RouteAvailability.Depot}
if HPhone != "" {
queryArgs = append(queryArgs, "SUBSTRING("+HPhone+",1,3)"+"-"+"SUBSTRING("+HPhone+",4,3)"+"-"+"SUBSTRING("+HPhone+",7,4)")
}
Help would be appreciated.
Thanks in advance.
Instead of SUBSTRING you can use REPLACE like so:
queryArgs := []interface{}{RouteAvailability.Depot}
if HPhone != "" {
strQry += ` AND REPLACE(HPhone, '-', '') = ?`
queryArgs = append(queryArgs, HPhone)
}
If possible I would suggest you normalize your data, i.e. decide on a canonical format for a particular data type and everytime your program receives some input that contains that data type you format it into its canonical form, that way you can avoid having to deal with SUBSTRING, or REPLACE, or multiple inconsistent formats etc.
This won't work as you are using prepared statements, and the argument you are building when HPhone is not empty will be used in escaped form - so when executing the query, it won't compare the HPhone values with the computed result of some substring, but with a string containing SUBSTRING(9999...