I am having an issue on inserting this string
input: GyYu4$%acV
posted string: GyYu4$�V
Query error: Incorrect string value: '\xACV' for column.
It seems the best I get is str_replace.
I got the best escape for the issue its about escaping the non-utf8 character that #developerCK mentioned. its utf8_encode
Related
In dealing with the headache of the different rulesets with TEXT escaping and JSON escaping, I've come across the issue where double escaping is required to convert a string to a JSON literal. For example, the original UPDATE looks like this:
UPDATE sourcing_item_data SET data_JSON='{"test": "test \ test"}' WHERE ID = 1;
The above simply removes the '\'.
The problem is I can't see how we get a single backslash into the system. Using two \'s causes the Invalid JSON error. Using three \'s does the same. Using four \'s puts in two \'s.
How does one get a single backslash into a JSON literal from a string with MySQL?
Also, has anyone written a SP or Function that scans a string that's supposed to be converted to MySQL JSON to ensure the string is "scrubbed" for issues (such as this one)?
Thanks!
Four backslashes works.
UPDATE sourcing_item_data SET data_JSON='{"test": "test \\\\ test"}' WHERE ID = 1;
You need to double the backslash to escape it in JSON, and then double each of those to escape in the SQL string.
If you print the JSON value it will show up as two backslashes, but that's because it shows the value in JSON format, which means that the backslash has to be escaped. If you extract the value and unquote it, there will just be one backslash.
select data_JSON->>"$.test" as result
from sourcing_item_data
WHERE id = 1;
shows test \ test
DEMO
i have a json field in my table(MySQL Database).
the following is the structure:
{
"article":{
"Key's 1":{
"value":"24"
}
"Key's of the something's 2":{
"value":"55"
}
}
}
i am trying to extract the "value" field of "Key's 1".
Due special character such as single quote and space, i am unable use JSON_EXTRACT function.
It give the error: Invalid JSON path expression.This error is around character position no : 10
My Query:
select
JSON_EXTRACT(analytics_json,'$.article.Key\'s 1.value')As value
from
tbl_json_data;
Even after place a backslash, i am getting the error.
You may escape each JSON path component in double quotes to handle special characters as well as spaces. The following works:
SELECT
JSON_EXTRACT(analytics_json,'$.article."Key''s 1".value') AS value
FROM tbl_json_data;
Demo
Note that your key name actually has two problems. First, it contains a literal single quote. We can handle that by just doubling up two single quotes. The key name also contains whitespace. By escaping in double quotes we may workaround this problem, but it would probably be best to avoid using JSON keys which have whitespace.
I inserting data from mysql to postgresql using Talend and I get invalid byte sequence error.
I used trim(both CHAR(0x00) from column_name) to remove invalid characters.
It removed most of the errors but I am guessing there are invalid characters in the middle of string. How can I find and remove them using sql?
please look here:
UPDATE cars_tbl
SET description = '{\rtf1'
WHERE (ID=1)
Description field is "blob", where my RTF document is to be stored.
When I check updated data I always find
{
tf1
\r simply disapears. I tried to find solution on the web, but no success. My rtf files are corrupted on many places, because the escape characters used in the string are substituted. How to suppress this substitution and update field with string as is?
Thanx for advice
Lyborko
Backslash is an escape character, so to keep it you need a double backslash:
UPDATE cars_tbl
SET description = '{\\rtf1'
WHERE (ID=1)
As an aside \r is a carriage return.. and it hasn't disappeared in your data; it is responsible for tf1 appearing on the line below the {.
You can achieve this with a more generic approach
use of QUOTE() in mysql
MySQL QUOTE() produces a string which is a properly escaped data value in an SQL statement, out of an user supplied string as argument.
The function achieve this by enclosing the string with single quotes, and by preceding each single quote, backslash, ASCII NUL and control-Z with a backslash.
example
UPDATE cars_tbl
SET description = QUOTE('{\rtf1')
WHERE (ID=1)
UPDATE
to escape your RTF you can also just use REPLACE this way all your \ will become \\
Example
UPDATE cars_tbl
SET description = REPLACE('{\rtf1', '\', '\\')
WHERE (ID=1)
Having followed some tips on escaping apostrophes I am getting an unexpected combination of escape characters in the resulting sql statement. The following rails 4 active record statement is run against 5.5.42-MariaDB:
User.where(["surname LIKE ?", "%#{params[:search]}%"])
Where
params[:search] = "O'Keefe"
A .to_sql generates
SELECT * FROM users WHERE surname LIKE '%O\\'Keefe%'
MySQL/MariaBD expects an apostrophe to be escaped as two single apostrophes '' , or with a single backslash \' so this results in a syntax error. I am looking for help to understand why two backslashes \\' are appearing, and for a solution that will maintain protection against SQL injection.
UPDATE
After further investigation following suggestions below, it appears as though the console .to_sql output SELECT * FROM users WHERE surname LIKE '%O\\'Keefe%' is not what is passed onto MySQL. It failed for me 'cos I simply copied the statement into a mysql console to test execution. There is some black magic on route to the database that converts the double backslash \\' into a valid mysql escape sequence.
So problem 1/2 solved
User.where(["surname LIKE ?", "%#{params[:search]}%"])
is valid syntax that correctly auto-escapes the user input string. But can anyone shed any light on the reason for the generation of the double backslash and how it is modified on its way to database execution?
Try this:
User.where(["surname LIKE ?", "%#{params[:search].gsub("'", "''")}%"])
http://dev.mysql.com/doc/refman/5.0/en/string-literals.html#character-escape-sequences