I use https://github.com/mysqljs/mysql.git library.
I have a mysql db query architecture in which I can not modify the SQL query file one by one to escape each argument for there are too many files, but all the SQL queries will call the query method of a same base mysql instance, so I wonder if I can escape the eventual SQL string in the base mysql query method.
I want to escape the whole SQL string like
select * from tableA where name = 'foo'bar
to
select * from tableA where name = 'foo\'bar'
with some function like mysql_escape("select * from tableA where name = 'foo'bar'") instead of doing this using preparing queries or concating escaped strings.
There isn't a way to do this that wont result in a really inefficient function or some bad hack. Just use parameterized queries, Its basically what they are there for. If you cant use those you use concat strings.
Running mysql_escape on a whole query will require the function to know what characters are part of your query and what characters are part of the input values. You could write some kind of stupid regex to try pull the values from the query and then escape them but its just a bad idea.
Related
I have an JSON string stored in the database and I need to SQL COUNT based on the WHERE condition that is in the JSON string. I need it to work on the MYSQL 5.5.
The only solution that I found and could work is to use the REGEXP function in the SQL query.
Here is my JSON string stored in the custom_data column:
{"language_display":["1","2","3"],"quantity":1500,"meta_display:":["1","2","3"]}
https://regex101.com/r/G8gfzj/1
I now need to create a SQL sentence:
SELECT COUNT(..) WHERE custom_data REGEXP '[HELP_HERE]'
The condition that I look for is that the language_display has to be either 1, 2 or 3... or whatever value I will define when I create the SQL sentence.
So far I came here with the REGEX expression, but it does not work:
(?:\"language_display\":\[(?:"1")\])
Where 1 is replaced with the value that I look for. I could in general look also for "1" (with quotes), but it will also be found in the meta_display array, that will have different values.
I am not good with REGEX! Any suggestions?
I used the following regex to get matches on your test string
\"language_display\":\[(:?\"[0-9]\"\,)*?\"3\"(:?\,\"[0-9]\")*?\]
https://regex101.com/ is a free online regex tester, it seems to work great. Start small and work big.
Sorry it doesn't work for you. It must be failing on the non greedy '*?' perhaps try without the '?'
Have a look at how to serialize this data, with an eye to serializing the language display fields.
How to store a list in a column of a database table
Even if you were to get your idea working it will be slow as fvck. Better off to process through each row once and generate something more easily searched via sql. Even a field containing the comma separated list would be better.
I'm trying to replicate the following LIKE query using a full text search on JSON data;
SELECT * FROM table
WHERE response LIKE '%"prod_id": "foo"%'
AND response LIKE '%"start_date": "2016-07-13"%'
In my database the above query returns 28 rows
This is my attempt:
SELECT * FROM table
WHERE MATCH(response)
AGAINST('+"\"prod_id\": \"foo\"",+"\"start_date\": \"2016-07-13\""')
However this returns over 4,500 rows (the same as running the first query for only the prod_id ~1,900 rows when running the first query on just the date)
It was my understanding that +"text here" would indicate a required word, and that literal double quotes (present in the JSON data) should be escaped, and that , would indicate a split between the two strings I'm looking for. What am I not understanding correctly? Is there any point in running this as a full text query anyway?
Thanks to #Sevle I've tweaked my query like so, and it's returning the correct results;
SELECT * FROM table
WHERE MATCH(response)
AGAINST('+\"prod_id: foo\" +\"start_date: 2016-07-13\"' IN BOOLEAN MODE)
The comma was not helping and I was escaping the wrong characters, and of course I did need IN BOOLEAN MODE to be added. Finally, I removed the double quotes I was searching for in the JSON string.
It may also be worth noting that as I'm using PHP PDO to run this query I also had to make the following tweaks.
Instead of constructing the query like so trying to bind the variables like I normally would;
$query = $db->prepare('...AGAINST('+\"prod_id: :prod_id\" +\"start_date: :start_date\"');
$query->execute(array('prod_id' => 'foo', 'start_date' => '2016-07-13'));
I had to do this, as I found I could not bind variables in full text searches
$sql_against = $db->quote('...AGAINST('+\"prod_id: foo\" +\"start_date: 2016-07-13\"');
$query = $db->prepare("...AGAINST($sql_against IN BOOLEAN MODE)")
I'm trying to write a SQL update to replace a specific xml node with a new string:
UPDATE table
SET Configuration = REPLACE(Configuration,
"<tag>%%ANY_VALUE%%</tag>"
"<tag>NEW_DATA</tag>");
So that
<root><tag>SDADAS</tag></root>
becomes
<root><tag>NEW_DATA</tag></root>
Is there a syntax im missing for this type of request?
Update: MySQL 8.0 has a function REGEX_REPLACE().
Below is my answer from 2014, which still applies to any version of MySQL before 8.0:
REPLACE() does not have any support for wildcards, patterns, regular expressions, etc. REPLACE() only replaces one constant string for another constant string.
You could try something complex, to pick out the leading part of the string and the trailing part of the string:
UPDATE table
SET Configuration = CONCAT(
SUBSTR(Configuration, 1, LOCATE('<tag>', Configuration)+4),
NEW_DATA,
SUBSTR(Configuration, LOCATE('</tag>', Configuration)
)
But this doesn't work for cases when you have multiple occurrences of <tag>.
You may have to fetch the row back into an application, perform string replacement using your favorite language, and post the row back. In other words, a three-step process for each row.
I am having the following problem:
I have a table T which has a column Name with names. The names have the following structure:
A\\B\C
You can create on yourself like this:
create table T ( Name varchar(10));
insert into T values ('A\\\\B\\C');
select * from T;
Now if I do this:
select Name from T where Name = 'A\\B\C';
That doesn't work, I need to escape the \ (backslash):
select Name from T where Name = 'A\\\\B\\C';
Fine.
But how do I do this automatically to a string Name?
Something like the following won't do it:
select replace('A\\B\C', '\\', '\\\\');
I get: A\\\BC
Any suggestions?
Many thanks in advance.
You have to use "verbatim string".After using that string your Replace function will
look like this
Replace(#"\", #"\\")
I hope it will help for you.
The literal A\\B\C must be coded as A\\\\A\\C, and the parameters of replace() need escaping too:
select 'A\\\\B\\C', replace('A\\\\B\\C', '\\', '\\\\');
output (see this running on SQLFiddle):
A\\B\C A\\\\B\\C
So there is little point in using replace. These two statements are equivalent:
select Name from T where Name = replace('A\\\\B\\C', '\\', '\\\\');
select Name from T where Name = 'A\\\\B\\C';
Usage of regular expression will solve your problem.
This below query will solve the given example.
1) S\\D\B
select * from T where Name REGEXP '[A-Z]\\\\\\\\[A-Z]\\\\[A-Z]$';
if incase the given example might have more then one char
2) D\\B\ACCC
select * from T where Name REGEXP '[A-Z]{1,5}\\\\\\\\[A-Z]{1,5}\\\\[A-Z]{1,5}$';
note: i have used 5 as the max occurrence of char considering the field size is 10 as its mentioned in the create table query.
We can still generalize it.If this still has not met your expectation feel free to ask for my help.
You're confusing what's IN the database with how you represent that data in SQL statements. When a string in the database contains a special character like \, you have to type \\ to represent that character, because \ is a special character in SQL syntax. You have to do this in INSERT statements, but you also have to do it in the parameters to the REPLACE function. There are never actually any double slashes in the data, they're just part of the UI.
Why do you think you need to double the slashes in the SQL expression? If you're typing queries, you should just double the slashes in your command line. If you're generating the query in a programming language, the best solution is to use prepared statements; the API will take care of proper encoding (prepared statements usually use a binary interface, which deals with the raw data). If, for some reason, you need to perform queries by constructing strings, the language should hopefully provide a function to escape the string. For instance, in PHP you would use mysqli_real_escape_string.
But you can't do it by SQL itself -- if you try to feed the non-escaped string to SQL, data is lost and it can't reconstruct it.
You could use LIKE:
SELECT NAME FROM T WHERE NAME LIKE '%\\\\%';
Not exactly sure by what you mean but, this should work.
select replace('A\\B\C', '\', '\\');
It's basically going to replace \ whereever encountered with \\ :)
Is this what you wanted?
In postgresql, it's possible to use "Dollar-Quoted String Constants" to avoid escaping single quotes in constant strings:
Without Dollar-Quoted String Constants:
'Jeff''s table'
With Dollar-Quoted String Constants:
$$Jeff's table$$
Is there a MySQL equivalent?
On edit: I'm not looking for a way to sanitize inputs, I'm looking for a way to make queries that generate sql easier to read.
No, because it doesn't really work. An attacker just addes a pair of dollar signs to their injection attempt instead.
The correct way to handle this is a system that uses real query parameters, such that the parameter values are never substituted directly into a query string. This is also generally better for performance because the db can do a better job caching the execution plan you don't end up building string dynamically, which tends to be slow in modern languages.
Not having excess concatenation statements greatly improves the readability of the code, as well. Instead of this:
sql = "SELECT * FROM MyTable WHERE Column1=$$" + somevarialbe + "$$ AND Column2=$$" + OtherVariable";
it's just this:
sql = "SELECT * FROM MyTable wHERE Column1=#Value1 AND Column2=#Value2";
Hmm... Okay, I can see some limited utility for this feature now.
Imagine a ticket tracking system with a stored procedure to get open tickets. You might actually hard-code the literal 'open' into the where clause of the query. Not that this would be a good design — ticket status should be in a table with it's own key, so that you'd hardcode the key rather than the text. But it plants the seed for something valid and more-complicated.
You can enclose your string in double quotes instead, e.g.
"Jeff's table"
Note: If this doesn't work, it means that you've got ANSI_QUOTES SQL mode turned on. The MySQL docs say:
If the ANSI_QUOTES SQL mode is enabled, string literals can be quoted only within single quotation marks because a string quoted within double quotation marks is interpreted as an identifier.