Hi I'm currently trying to query records from db and these are the conditions
I receive 'order by', 'order (desc/asc)', 'limit', 'offset' from the frontend
I also need to search the record using match...against. 'like' is too slow for searching.
There's a mapped model with this query.
so I tried
let order_by = req.query.orderby;
let order = req.query.order;
let page = req.query.pagenum;
let perpage = req.query.parpage;
let searchword = req.query.foodsearch;
let offset = (parseInt(page) - 1) * parpage;
let foods = await models.food.findAll({
limit: parseInt(perpage),
offset: offset,
order: [
[order_by, order]
],
// where: Sequelize.literal
// (
// `MATCH
// (Name, Place, RestoNum, Ingredient, ChefName, Region...)
// AGAINST
// ( ? IN NATURAL LANGUAGE MODE)`,
// { replacements: [ searchword ] }
// )
});
but the commented part seems wrong in this code.
I tried the raw query, but then I can't parameterize those order by, order, offset, limit variables.
I don't want to just add them like ${orderby} because it's risky.
Please let me know if you have any solution for this issue.
Thank you in advance!
You're confusing the Sequelize.literal() and sequelizeInstance.query() APIs.
.literal() only take a string. If you want to use the object notation for your query, your commented code will work. Except that there is no second argument. You will need to concatenate-in or interpolate-in your search term into the AGAINST clause. Also, don't forget your quotes. The output of the literal() is essentially a string. Your MySQL FTS parameter will need the correct type of quotes around it, just as they would appear in your raw SQL query.
.query() DOES take an options parameter. Through this, you don't have to use string interpolation, you can use named replacements or bound-parameters. This will not only allow you to place in your searchword parameter, but whatever ORDER BY clause you want, as well.
I would go with Option 1. That's what we are doing for our FTS, in MS SQL.
Related
Note: this is a similar but NOT duplicate question with How to use sqlx to query mysql IN a slice?. I'm asking for the Rust one.
This is what I try to do.
let v = vec![..];
sqlx::query("SELECT something FROM table WHERE column IN (?)").bind(v)
...
Then I got the following error
the trait bound `std::vec::Vec<u64>: sqlx::Encode<'_, _>` is not satisfied
Answer is in first on FAQ https://github.com/launchbadge/sqlx/blob/master/FAQ.md
How can I do a SELECT ... WHERE foo IN (...) query? In 0.6 SQLx will
support binding arrays as a comma-separated list for every database,
but unfortunately there's no general solution for that currently in
SQLx itself. You would need to manually generate the query, at which
point it cannot be used with the macros.
The error shows Vec is not an Encode that is required to be as a valid DB value. The Encode doc lists all the Rust types that have implemented the trait. Vec is not one.
You can use the following way to bind the parameters in IN with the values of a vector. Firstly, you need to expand the number of '?' in the IN expression to be the same number of the parameters. Then, you need to call bind to bind the values one by one.
let v = vec![1, 2];
let params = format!("?{}", ", ?".repeat(v.len()-1));
let query_str = format!("SELECT id FROM test_table WHERE id IN ( { } )", params);
let mut query = sqlx::query(&query_str);
for i in v {
query = query.bind(i);
}
let row = query.fetch_all(&pool).await?;
Please note if the target database is not MySql, you need to use $n, like $1, $2, instead of ?, as the parameter placeholder.
I've been stuck on a GORM issue for about a full day now. I need to be able to filter a messages table on any of 4 things: sender, recipient, keyword, and date range. It also has to paginate. Filtering by sender and recipient is working, and so is pagination. So far this is the query that I have come up with, but it does not seem to work for date ranges or keywords.
Here is how I am selecting from MySQL
db.Preload("Thread").Where(query).Scopes(Paginate(r)).Find(&threadMessages)
I am creating the query like this:
var query map[string]interface{}
Then based on which parameters I am passed, I update the query like this by adding new key values to the map:
query = map[string]interface{}{"user_id": sender, "recipient_id": recipient}
For dates it does not seem to work if I try something like this:
query = map[string]interface{}{"created_at > ?": fromDate}
And for a LIKE condition is also does not seem to work:
query = map[string]interface{}{"contents LIKE ?": keyword}
The reason I chose this approach is that I could not seem to get optional inputs to work in .Where since it takes a string with positional parameters and null positional parameters seem to cause MySQL to return an empty array. Has anyone else dealt with a complicated GORM issue like this? Any help is appreciated at this point.
Passing the map[string]interface{} into Where() only appears to work for Equals operations, or IN operations (if a slice is provided as the value instead).
One way to achieve what you want, is to construct a slice of clause.Expression, and append clauses to the slice when you need to. Then, you can simply pass in all of the clauses (using the ... operator to pass in the whole slice) into db.Clauses().
clauses := make([]clause.Expression, 0)
if mustFilterCreatedAt {
clauses = append(clauses, clause.Gt{Column: "created_at", fromDate})
}
if mustFilterContents {
clauses = append(clauses, clause.Like{Column: "contents", Value: keyword})
}
db.Preload("Thread").Clauses(clauses...).Scopes(Paginate(r)).Find(&threadMessages)
Note: If you're trying to search for content that contains keyword, then you should concatenate the wildcard % onto the ends of keyword, otherwise LIKE behaves essentially the same as =:
clause.Like{Column: "contents", Value: "%" + keyword + "%"}
My final solution to this was to create dynamic Where clauses based on which query params were sent from the client like this:
fields := []string{""}
values := []interface{}{}
If, for example, there is a keyword param:
fields = []string{"thread_messages.contents LIKE ?"}
values = []interface{}{"%" + keyword + "%"}
And to use the dynamic clauses in the below query:
db.Preload("Thread", "agency_id = ?", agencyID).Preload("Thread.ThreadUsers", "agency_id = ?", agencyID).Joins("JOIN threads on thread_messages.thread_id = threads.id").Where("threads.agency_id = ?", agencyID).Where(strings.Join(fields, " AND "), values...).Scopes(PaginateMessages(r)).Find(&threadMessages)
I'm working with MySQL for a while and also built an API in NodeJS including mySQL for work
Basically i need a query like "SELECT * FROM table WHERE fieldA=varA AND/OR fieldB=valB"
I'm using "mysql.format(sql, args)" to format my query, so i'm using ? and ?? in my Queries.
I would like to write a basic query, that i could use and feed all the needed fields and values
I tried the following ways
-> "SELECT * FROM table WHERE ?" with "{fieldA: varA, fieldB: varB}" as replacement for ?
that leads to "SELECT * FROM table WHERE fieldA='varA', fieldB='varB'"
-> "SELECT * FROM table WHERE ?? = ?" with "['fieldA', 'fieldB']" and "['varA', 'varB']" as replacements
what leads to "SELECT * FROM table WHERE fieldA, fieldB = 'varA', 'varB'"
For now i "only" need 2 different fields, so i could add fixed "fieldA=? AND/OR fieldB=?" and fill only the values. But i would like a dynamic way and give all the fields i could need in it and also if i use AND or OR in combining.
I didn't find anything like this in the documentation, maybe somebody here had stumbled upon before.
Or might it be the only solution to dynamically add some "AND/OR ?? = ?" to the query and fill the arguments array with fieldName, values one after the other?
Consider theses are the fields that you need to build your dynamic query string.
var fields = [{fieldName:"A",value:"one",operator:"AND"},{fieldName:"B",value:"two",operator:""}];
Now try building your query string with that array. For now I have considered only two fields but you can add as per your needs.
var fields = [{fieldName:"A",value:"one",operator:"AND"},{fieldName:"B",value:"two",operator:""}];
var query = "SELECT * FROM table WHERE ";
fields.forEach((f)=>{
query=query+`${f.fieldName} = ${f.value} `
if(f.operator!="") query=query+f.operator+" "
})
console.log(query)
I have built a method for this now, with hints from Ameers answer. It will generate a query like
SELECT * FROM ?? WHERE ??=? AND/OR ??=? AND/OR ??=?
and then will add the fieldName and values to the arguments array for mysql.format
var fieldValues = [
{ field: 'command', value: command.command, operator: 'OR' },
{ field: 'alias', value: command.command, operator: 'OR' }
];
exists(fieldValues) {
let query = 'SELECT * FROM ??';
let args = ['tablename'];
for (let i = 0; i < fieldValues.length; i++) {
let operator = i == 0 ? 'WHERE' : fieldValues[i].operator;
query += ` ${operator} ?? = ?`;
args.push(fieldValues[i].field);
args.push(fieldValues[i].value);
}
return mysql.query(query, args);
}
mysql.query() here is my wrapper around mysql.connection.query and using promises
Maybe there is a better builtin method, but i didn't find one yet and this way i can still use the builtin mysql-formatter function.
Consider,
that you have fields in DB like first_name, last_name, personal_email, work_email and office_name.
and you want to get all result matching with search fields.
i.e. let search = req.query.search.
It will become easy with ORM like Objection.js
So the query will be like this using objection.js
if (search) {
table_name.where('column_name.last_name', 'like', `%${search}%`)
.orWhere('column_name.first_name', 'like', `%${search}%`)
.orWhere('column_name.personal_email', 'like', `%${search}%`)
.orWhere('column_name.work_email', 'like', `%${search}%`)
.orWhere('column_name.office_name', 'like', `%${search}%`)
}
Note: This code is in nodejs and you can also use table_name you can also add model_name.
Let me preface by saying I'm very new to SQL (and back end design) in general. So for those annoyed with noob questions, please be gentle.
BACKGROUND:
I'm trying to build a product test database (storing test data for all our products) where I want a user to be able to refine a search to find test data they actually want. For example, they may start by searching for all products of a certain brand name, and then refine it with a product type, and/or refine it with a date range of when the test was done.
PROBLEM:
I'm having a hard time finding information on how to implement multi-parameter searches with mysql and node.js. I know you can do nested queries and joins and such within pure SQL syntax, but it's not abundantly clear to me how I would do this from node.js, especially when certain search criteria aren't guaranteed to be used.
Ex:
CREATE PROCEDURE `procedureName`(
IN brandname VARCHAR(20),
producttype VARCHAR(30))
BEGIN
SELECT * FROM products
WHERE brand = brandname
AND product_type = producttype;
END
I know how to pass data from node.js to this procedure, but what if the user didn't specify a product type? Is there a way to nullify this part of the query? Something like:
AND product_type = ALL;
WHAT I'VE TRIED:
I've also looked into nesting multiple SQL procedures, but passing in dynamic data to the "FROM" clause doesn't seem to be possible. Ex: if I had a brandname procedure, and a product type procedure, I don't know how/if I can pass the results from one procedure to the "FROM" clause of the other to actually refine the search.
One idea was to create tables with the results in each of these procedures, and pass those new table names to subsequent procedures, but that strikes me as an inefficient way to do this (Am I wrong? Is this a completely legit way to do this?).
I'm also looking into building a query string on the node side that would intelligently decide what search criteria have been specified by the front end, and figure out where to put SQL AND's and JOIN's and what-nots. The example below actually works, but this seems like it could get ugly quick as I add more search criteria, along with JOINS to other tables.
// Build a SQL query based on the parameters in a request URL
// Example request URL: http://localhost:3000/search?brand=brandName&type=productType
function qParams(req) {
let q = "SELECT * FROM products WHERE ";
let insert = [];
if(req.query.brand) {
brandname = req.query.brand; // get brandname from url request
q = q + `brand = ?`, // Build brandname part of WHERE clause
insert.push(brandname); // Add brandname to insert array to be used with query.
};
if(req.query.type) {
productType = req.query.type; // get product type from url request
insert.length > 0 ? q = q + ' AND ' : q = q; // Decide if this is the first search criteria, add AND if not.
q = q + 'product_type = ?'; // Add product_type to WHERE clause
insert.push(productType); // Add product_type variable to insert array.
}
// Return query string and variable insert array
return {
q: q,
insert: insert
};
};
// Send Query
async function qSend(req, res) {
const results = await qParams(req); // Call above function, wait for results
// Send query string and variables to MySQL, send response to browser.
con.query(results.q, results.insert, (err, rows) => {
if(err) throw err;
res.send(rows);
res.end;
})
};
// Handle GET request
router.use('/search', qSend);
CONCISE QUESTIONS:
Can I build 1 SQL procedure with all my search criteria as variables, and nullify those variables from node.js if certain criteria aren't used?
Is there way to nest multiple MySQL procedures so I can pick the procedures applicable to the search criteria?
Is creating tables of results in a procedure, and passing those new table names to other procedures a reasonable way to do that?
Building the query from scratch in node is working, but it seems bloated. Is there a better way to do this?
Googling "multi-parameter search mysql nodejs" is not producing useful results for my question, i.e. I'm not asking the right question. What is the right question? What do I need to be researching?
One option is to use coalesce():
SELECT p.*
FROM products p
WHERE
p.brand = COALESCE(:brandname, p.brand)
AND p.product_type = COALESCE(:producttype, p.producttype);
It may be more efficient do explicit null checks on the parameters:
SELECT p.*
FROM products p
WHERE
(:brandname IS NULL OR p.brand = :brandname)
AND (:producttype IS NULL OR p.product_type = :producttype);
I have a simple nodejs application which executes the following query.
select * from User where userid in (?)
The userids i get is a JSON array send from client side. How can i use that in this select query ? I tried
1. As itself but not working.
2. Convert this to Javascript array, not working
If you are using node module like mysql, the 2nd approach should work.
var query=select * from User where userid in (?);
var data=['a','b','c'];
var queryData=[data];
conn.query(query, queryData, function (err, results) {})
According to the documentation, "Arrays are turned into list, e.g. ['a', 'b'] turns into 'a', 'b'". So this approach should work (I have used it practically).
If you pass an array to the parameter it works with node mysql2. Parameters are already passed as arrays, so your first parameter needs to be an array [[1,2,3]].
select * from User where userid in (?)
const mysql = require('mysql2/promise');
async function main(){
let db = await mysql.createPool(process.env.MYSQL_URL);
let SQL = 'select * from User where userid in (?)';
let [res, fields] = await db.query(SQL, [[1,2,3]]);
console.log(res)
return res;
}
main().then(() => {process.exit()})
Revisiting this, since the original approach on the question is valid, but with some caveats. If your only escaped argument is the one on the IN clause, then you have to specify it as nested array; something like: [['usrId1', 'usrId2', 'usrIdN']]. This is because the un-escaping functionality expects an array, replacing each '?' with the corresponding array element. So, if you want to replace your only '?' with an array, that array should be the first element of all arguments passed. If you had more than one '?', the syntax is more intuitive, but at the end consistent and the same; in this case, you could have your arguments similar to: ['myOtherArgument1', 'myOtherArgument2', ['usrId1', 'usrId2', 'usrIdN'], 'myOtherArgument3']
Something like this could work!
// get your possible IDs in an array
var ids = [1,2,3,4,5];
// then, create a dynamic list of comma-separated question marks
var tokens = new Array(ids.length).fill('?').join(',');
// create the query, passing in the `tokens` variable to the IN() clause
var query = `SELECT * FROM User WHERE userid IN (${tokens})`;
// perform the query
connection.query(query, ids, (err, data) => {
// do something with `err` or `data`
});
You can do like this:
select * from User where userid in (?,?,?,?)
var array = [];
array.push(value);
array.push(value);
array.push(value);
array.push(value);
then use array as parameter that should be bind.
// get query string data with commas
var param=req.params['ids'];
//damy data var param = [1,2,3,4,5];
var array = params.split(",").map(Number);
//Note in select query don't use " and ' ( inverted commas & Apostrophe)
// Just use ` (Grave accent) first key off numeric keys on keyboard before one
con.query(`select * from TB_NAME where COL IN(?)`,[array],(err,rows,fields)=>{
res.json(rows);
});
let val = ["asd","asd"]
let query = 'select * from testTable where order_id in (?)';
connection.query(query, [val], function (err, rows) {
});
In Node, you need to put array in the array.
Update: Please see this answer. It is the correct way to do what is asked in the question.
The methods I have tried are:
Expand JSON array to a string in the required format. Concatenate it with query using '+'. (Beware of SQL injections)
Dynamically add '?' using length of JSON array holding user ids. Then use the array to provide user ids.
Both works. I then changed my logic with a better approach so now i don't need then 'in' clause anymore.