I would like to know how to handle Japanese characters in a query to a Microsoft Access database. I am trying to use a query selecting variable names written in Japanese using the function odbcQuery from RODBC package in R.
I am working with Windows. My version of RStudio is 1.1.383, and my version of Access is 14.0.7015.1000 (32-bit).
I think R understands the Japanese characters in my query, but when I try to actually carry out the query I get the following error message:
> query <- "SELECT [LOA-FTD_1_5_1_CALCULATE_LOA_query].月日 FROM [LOA-FTD_1_5_1_CALCULATE_LOA_query]"
> sqlQuery(channel,query)
[1] "42000 -3100 [Microsoft][ODBC Microsoft Access Driver] Syntax error in query expression '[LOA-FTD_1_5_1_CALCULATE_LOA_query].<U+6708><U+65E5>'."
[2] "[RODBC] ERROR: Could not SQLExecDirect 'SELECT [LOA-FTD_1_5_1_CALCULATE_LOA_query].<U+6708><U+65E5> FROM [LOA-FTD_1_5_1_CALCULATE_LOA_query]'"
Here, 月日 was converted into U+6708 and U+65E5 in the error message. These are the UTF-8 codes for the two characters, so I guess the string is sent encoded in UTF-8 to MS Access, but MS Access is then unable to read it? Is MS Access even part of the process of carrying out the query?
So it must be an encoding issue, where RStudio and MS Access do not understand each other. When I looked at similar issues with Japanese characters, the problem was usually to display values in a table. Here the variable names are in Japanese, so the query does not work at all.
I am quite lost, so I am open to any idea or remark.
Thank you.
I found an answer that works for me in this post.
The trick (at least in my case) was to set locale to Japanese_Japan.932 before any data importing.
Here is the code for this command:
Sys.setlocale("LC_ALL", locale = "Japanese_Japan.932")
Then I imported my data from Access without having to change encoding, and the Japanese characters are displayed correctly in the resulting data frame. Moreover, this allows Japanese characters in the query to be understood.
Im limiting my clients from entering certain characters.
Right now I allow:
All the numbers
All the english letters
-
_
I do not allow:
'
"
;
What other safe characters and/or unsafe characters exist when doing a query via programming using a textbox or such?
The safest way to work with any SQL (MySQL included) is to use parameterized queries. This will allow your clients to enter any characters they want and prevent any SQL injections.
You didn't mention what language you are working with so here's a .NET example for working with MySQL: http://dev.mysql.com/doc/connector-net/en/connector-net-tutorials-parameters.html
These is one keyword confliction issue in the query module of my application,please see if you can tell me a smart solution.
First,In query module,each query condition contains three parts in UI:
1.field name,its value is fixed,e.g origin,finalDest...
2.operator,it is a select list which includes "like","not like","in","not in","=","!="
3.value,this part is input by user.then in back-end,it will assemble the SQL statement according to UI's query criteria,e.g if user type/select following stuff in UI
Field Name Operator Value
origin like CHI
finalDest in SEL
In back-end,it will generate following SQL:
select * from Booking where origin like '%CHI%' and finalDest in ('SEL').
But there is a bug,e.g if user type some of special symbol in "value",e.g "'","_" etc,it will lead to the generated SQL also contain ' or _ ,e.g:
select * from Booking where origin like '%C_HI%' and finalDest in ('S'EL').
you could see as there is special symbol in "where" block,the SQL can't be executed
For this problem,my solution is add escape character "/" in front of the special symbol before executing it,but what i know is just ' or _ that would conflict with the SQL keywords,do you know if there is any others similar symbol that i need to handle or do you guys have any better idea that can avoid the injection
Sorry,forgot told you what language i am using,i am using java,the DB is mysql,i also use hibernate,there are a lot of people said why i didn't use PreparedStatement,this is a little complex,simply speaking,in my company,we had a FW called dynamic query,we pre-defined the SQL fragment in a XML file,then we will assemble the SQL according to the UI pass in criteria with the jxel expression,as the SQL is kinda of pre-defined stuff,i afraid if change to use PreparedStatement,it will involve a lot of change for our FW,so what we care is just on how to fix the SQL injection issue with a simple way.
The code should begin attempting to stop SQL injection on the server side prior to sending any information to the database. I'm not sure what language you are using, but this is normally accomplished by creating a statement that contains bind variables of some sort. In Java, this is a PreparedStatement, other languages contains similar features.
Using bind variables or parameters in a statement will leverage built in protection against SQL injection, which honestly is going to be better than anything you or I write on the database. If your doing any String concatenation on the server side to form a complete SQL statement, this is an indicator of a SQL injection risk.
0 An ASCII NUL (0x00) character.
' A single quote (“'”) character.
" A double quote (“"”) character.
b A backspace character.
n A newline (linefeed) character.
r A carriage return character.
t A tab character.
Z ASCII 26 (Control+Z). See note following the table.
\ A backslash (“\”) character.
% A “%” character. See note following the table.
_ A “_” character. See note following the table
Reference
Stack Similar Question
You should use bind variables in your SQL statement. As already mentioned this is done with PreparedStatements in Java.
To make sure, only valid column names are used, you can validate the input against the database. MySQL provides schema information like columns of each table as part of the INFORMATION_SCHEMA. For further information, check the MySQL documentation:
"The INFORMATION_SCHEMA COLUMNS Table"
My SQL Server 2008 R2 database has string columns (nvarchar). And some of the old data is showing Ascii. I need to show it to the user in my site and I prefer to convert the data in the database to Unicode. Is there a quick way to do this? are there downsides that I should be aware of?
Examples to my issue:
In the database, I see special chars instead of regulars chars. I have a name of a user which is supposed to be Amédée, and instead it shows Am?d??.
In other cases I see " instead of Quotation mark ("), or the chars &# instead of the word "and".
The ' ' character cannot be included in a name.
I use a log manager to log the error to SQL Server 2008 database. Of course, it will raise another error in the SQL Server because it contains special characters' ' . So what is the best way to handle special characters in SQL Server.
This is because you are using a space in an XML name. Correct your XML code to not have spaces in any tag names, this isn't SQL Server 2008 specific.
To get a column to handle special characters define it as a NVARCHAR instead of a VARCHAR.
Your question is a bit vague. Do you want to prevent the user from having a space? And if so, do you want to not allow there to be a space character, or just get the space character out? If so, do something like replace(#yourInputString, ' ', '').
If this is not what you are looking for, please clarify your question and your exact requirements.