So I am building a database of all my text messages to get information about my habits and I'm having trouble importing the contents of the messages. Whenever there are apostrophes (often) or quotation marks (not as rare as you might think), I get syntax issues.
Is there a way to make MySQL use something other than " or ' to encase strings (specifically, the field is a VARCHAR). If I could use a ~ or some other rarely used character in text messaging my life would become a whole lot easier.
Preferably you should use parameterised queries, then your database connector takes care of sending the strings to the database in the correct way.
If you need to build the queries by concatenating the values into a query, you need to escape the strings correctly to make them string literals in the SQL code.
Stick to one delimiter for strings, don't use apostrophes around some strings and qoutation marks around others, that only makes it harder to escape them correctly. I suggest that you use apostrophes, as that is what the SQL standard specifies.
To escape the strings correctly to be a string literal delimited by apostropes, you should:
Replace all backslashes by double backslashes, then
Replace all apostrophes by a backslash and an apostrophe
For example, to make the string It's an "example" with a backslash(\). into a string literal, it should end up like this in a query:
insert into Table (txt) values ('It\'s an "example" with a backslash(\\).')
Note: This is a correct way to escape strings for MySQL. Other databases may use different characters for escaping and need other characters to be escaped, so using this for any other database may fail, or even worse open up for SQL injection attacks.
Related
The MySQL documentation says that it should be \'. However, both scite and mysql shows that '' works. I saw that and it works. What should I do?
The MySQL documentation you cite actually says a little bit more than you mention. It also says,
A “'” inside a string quoted with “'” may be written as “''”.
(Also, you linked to the MySQL 5.0 version of Table 8.1. Special Character Escape Sequences, and the current version is 5.6 — but the current Table 8.1. Special Character Escape Sequences looks pretty similar.)
I think the Postgres note on the backslash_quote (string) parameter is informative:
This controls whether a quote mark can be represented by \' in a string literal. The preferred, SQL-standard way to represent a quote mark is by doubling it ('') but PostgreSQL has historically also accepted \'. However, use of \' creates security risks...
That says to me that using a doubled single-quote character is a better overall and long-term choice than using a backslash to escape the single-quote.
Now if you also want to add choice of language, choice of SQL database and its non-standard quirks, and choice of query framework to the equation, then you might end up with a different choice. You don't give much information about your constraints.
Standard SQL uses doubled-up quotes; MySQL has to accept that to be reasonably compliant.
'He said, "Don''t!"'
What I believe user2087510 meant was:
name = 'something'
name = name.replace("'", "\\'")
I have also used this with success.
There are three ways I am aware of. The first not being the prettiest and the second being the common way in most programming languages:
Use another single quote: 'I mustn''t sin!'
Use the escape character \ before the single quote': 'I mustn\'t sin!'
Use double quotes to enclose string instead of single quotes: "I mustn't sin!"
just write '' in place of ' i mean two times '
Here's an example:
SELECT * FROM pubs WHERE name LIKE "%John's%"
Just use double quotes to enclose the single quote.
If you insist in using single quotes (and the need to escape the character):
SELECT * FROM pubs WHERE name LIKE '%John\'s%'
Possibly off-topic, but maybe you came here looking for a way to sanitise text input from an HTML form, so that when a user inputs the apostrophe character, it doesn't throw an error when you try to write the text to an SQL-based table in a DB. There are a couple of ways to do this, and you might want to read about SQL injection too.
Here's an example of using prepared statements and bound parameters in PHP:
$input_str = "Here's a string with some apostrophes (')";
// sanitise it before writing to the DB (assumes PDO)
$sql = "INSERT INTO `table` (`note`) VALUES (:note)";
try {
$stmt = $dbh->prepare($sql);
$stmt->bindParam(':note', $input_str, PDO::PARAM_STR);
$stmt->execute();
} catch (PDOException $e) {
return $dbh->errorInfo();
}
return "success";
In the special case where you may want to store your apostrophes using their HTML entity references, PHP has the htmlspecialchars() function which will convert them to '. As the comments indicate, this should not be used as a substitute for proper sanitisation, as per the example given.
Replace the string
value = value.replace(/'/g, "\\'");
where value is your string which is going to store in your Database.
Further,
NPM package for this, you can have look into it
https://www.npmjs.com/package/mysql-apostrophe
I think if you have any data point with apostrophe you can add one apostrophe before the apostrophe
eg. 'This is John's place'
Here MYSQL assumes two sentence 'This is John' 's place'
You can put 'This is John''s place'. I think it should work that way.
In PHP I like using mysqli_real_escape_string() which escapes special characters in a string for use in an SQL statement.
see https://www.php.net/manual/en/mysqli.real-escape-string.php
I was save this string to DB
{"form_5_check":"N\u00e1kladov\u00e9 stredisko"}
But in mysql db is this string:
{"form_5_check":"Nu00e1kladovu00e9 stredisko"}
Pls where are "\" lost ? Thanks a lot
MySQL treats the backslash character as an escape character. If you did something like this:
query = "INSERT INTO foo (json) VALUES ('" + json + "');
you have basically three problems:
the single backslash you have will get interpreted as an escape character, not as content; unless the next character is a quote or another backslash, it will have escaped nothing, and silently disappear.
if your json contained any single quotes, and you are lucky, you will get a syntax error, as the quote that was supposed to contain the value will be closed, and gibberish that SQL can't parse follows.
if your json contained any single quotes, and you're not lucky, you're now a victim of SQL injection attack, the most infamous example being XKCD's.
To avoid all that, make sure that your data is properly sanitised before it hits the database. This can be done in two ways:
The manual, and error-prone way includes always remembering to escape any characters that need it any time you insert a string into a query. This differs between databases. Some databases want a backslash before quotes, while some prefer doubling the quotes and doing nothing to backslashes. Some allow both. Many languages and/or database access libraries have functions that do this in a way appropriate for the database.
The automated, foolproof and very much preferred way is to use parametrised queries and prepared statements that do this for you in a transparent and easy-to-use way. You do not have a specific language tagged, so I can't give you the solution, but the Bobby Tables site has answers for many commonly used programming languages.
Following on from this question MySQL database contains quotes encoded and unencoded and it's breaking javascript
I am executing this MySQL query:
DELETE FROM `example` WHERE `name` = ''12345''
However it fails because the value in the database is '12345'. It seems that old data in the database has a mixture of encoded and unencoded quotes. Is it safe to to update all ' to ' in the database?
In most cases (yours included), store text without any "encoding". That is, do not store htmlentities, store the actual characters, do not store unicode 'codes', store the actual characters, etc.
Do likewise for anything you need to compare to what is in the database.
You will, however, have to escape strings when building SQL statements. Otherwise, you can't get quotes (in text) inside quotes (that are part of the SQL syntax.
That is, you will end up with this SQL when searching for that Irishman:
... WHERE `name` = 'O\'Brian'
Is there any way to perform a SQL injection when single quotes are escaped by two single quotes? I know the MySQL server is using this specific technique to prevent against an attack. I'm trying to log in as a specific user but all of the common injections I've tried for the password have not worked successfully (i.e. ' or '1'='1, ' or ' 1=1, etc.).
No, and yes.
There's no way to have an unsafe values "breakout" of literal values that are enclosed in single quotes, if the value being supplied is "escaped" by preceding single quotes by with an additional single quote.
That is, assuming that your statement is guaranteeing that string literals are enclosed in quotes, as part of the "static" SQL text.
example perl-ish/php-ish
$sql = "... WHERE t.foo = '" . $safe_value . "' ... ";
^ ^
I've underscored here that the single quotes enclosing the literal are part of the SQL text. If $safe_value has been "escaped" by preceding each single quote in the "unsafe" value with another single value to make it "safe"...
$unsafe_value $safe_value
------------- ------------
I'm going I''m going
'she''s' ''she''''s''
1'='1 -- 1''=''1 --
As long as the escaping is handled properly, that we guarantee that potentially unsafe values are are run through the escaping, then including single quotes in data values is not a viable way to "breakout" of a literal with the SQL text.
That's the "no" part of the answer.
The "yes" part of the answer.
One of the biggest problems is making sure this is done EVERYWHERE, and that a mistake has not been made somewhere, assuming that a potentially unsafe string is "safe", and is not escaped. (For example, assuming that values pulled from a database table are "safe", and not escaping them before including them in SQL text.)
Also, the single quote trick is not the only avenue for SQL injection. The code could still be vulnerable.
Firstly, if we're not careful about other parts of the statement, like the single quotes enclosing string literals. Or, if for example, the code were to run the $sql through some other function, before it gets submitted to the database:
$sql = some_other_function($sql);
The return from some_other_function could potentially return SQL text that was in fact vulnerable. (As a ridiculous example, some_other_function might replace all occurrences of two consecutive single quotes with a single single quote. DOH!)
Also, with the vast number of possible unicode characters, if we're ever running through a characterset translation, there's also a possibility that some unicode character could get mapped to a single quote character. I don't have any specific example of that, but dollars to donuts that somewhere, in that plethora of multibyte encodings, there's some unicode character somewhere that will get translated to a single quote in some target.
There's a default character in the target for unmapped characters in the source, and that's usually a question mark (or a white question mark in a black diamond.) It would be a huge problem if the default character in the target (for unmapped characters in the source) was a single quote.
Bottom line: escaping unsafe strings by replacing single quotes with two single quotes goes a long ways towards mediating (mitigating?) SQL injection vulnerabilities. But in and of itself, it doesn't guarantee that code is not vulnerable in some other way.
if the input accepts unicode and is implicitly converted to ascii in the database (not as uncommon as it sounds) then an attacker can simply substitute ʻ or ʼ (0x02BB or 0x02BC) in place of single tick to get around the escaping mechanism and the implicit conversion will map those characters to single ticks (at least that's the case in SQL Server)
The MySQL documentation says that it should be \'. However, both scite and mysql shows that '' works. I saw that and it works. What should I do?
The MySQL documentation you cite actually says a little bit more than you mention. It also says,
A “'” inside a string quoted with “'” may be written as “''”.
(Also, you linked to the MySQL 5.0 version of Table 8.1. Special Character Escape Sequences, and the current version is 5.6 — but the current Table 8.1. Special Character Escape Sequences looks pretty similar.)
I think the Postgres note on the backslash_quote (string) parameter is informative:
This controls whether a quote mark can be represented by \' in a string literal. The preferred, SQL-standard way to represent a quote mark is by doubling it ('') but PostgreSQL has historically also accepted \'. However, use of \' creates security risks...
That says to me that using a doubled single-quote character is a better overall and long-term choice than using a backslash to escape the single-quote.
Now if you also want to add choice of language, choice of SQL database and its non-standard quirks, and choice of query framework to the equation, then you might end up with a different choice. You don't give much information about your constraints.
Standard SQL uses doubled-up quotes; MySQL has to accept that to be reasonably compliant.
'He said, "Don''t!"'
What I believe user2087510 meant was:
name = 'something'
name = name.replace("'", "\\'")
I have also used this with success.
There are three ways I am aware of. The first not being the prettiest and the second being the common way in most programming languages:
Use another single quote: 'I mustn''t sin!'
Use the escape character \ before the single quote': 'I mustn\'t sin!'
Use double quotes to enclose string instead of single quotes: "I mustn't sin!"
just write '' in place of ' i mean two times '
Here's an example:
SELECT * FROM pubs WHERE name LIKE "%John's%"
Just use double quotes to enclose the single quote.
If you insist in using single quotes (and the need to escape the character):
SELECT * FROM pubs WHERE name LIKE '%John\'s%'
Possibly off-topic, but maybe you came here looking for a way to sanitise text input from an HTML form, so that when a user inputs the apostrophe character, it doesn't throw an error when you try to write the text to an SQL-based table in a DB. There are a couple of ways to do this, and you might want to read about SQL injection too.
Here's an example of using prepared statements and bound parameters in PHP:
$input_str = "Here's a string with some apostrophes (')";
// sanitise it before writing to the DB (assumes PDO)
$sql = "INSERT INTO `table` (`note`) VALUES (:note)";
try {
$stmt = $dbh->prepare($sql);
$stmt->bindParam(':note', $input_str, PDO::PARAM_STR);
$stmt->execute();
} catch (PDOException $e) {
return $dbh->errorInfo();
}
return "success";
In the special case where you may want to store your apostrophes using their HTML entity references, PHP has the htmlspecialchars() function which will convert them to '. As the comments indicate, this should not be used as a substitute for proper sanitisation, as per the example given.
Replace the string
value = value.replace(/'/g, "\\'");
where value is your string which is going to store in your Database.
Further,
NPM package for this, you can have look into it
https://www.npmjs.com/package/mysql-apostrophe
I think if you have any data point with apostrophe you can add one apostrophe before the apostrophe
eg. 'This is John's place'
Here MYSQL assumes two sentence 'This is John' 's place'
You can put 'This is John''s place'. I think it should work that way.
In PHP I like using mysqli_real_escape_string() which escapes special characters in a string for use in an SQL statement.
see https://www.php.net/manual/en/mysqli.real-escape-string.php