I've read many threads on Stack and other forums, but still can't figure how to make it work.
The configuration
I have a MySQL 8.0 database named test, and configured it with default CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_ci.
Inside there's a table creatively named table, this too configured as utf8mb4 and utf8mb4_0900_as_ci.
The only field is named field and is a blob type, where I write and read encoded data using AES_ENCRYPT/DECRYPT
VBScript classic connect to it using the last ODBC 8 UNICODE driver
I've tried the SET NAMES statement but is not supported in this ODBC version. Instead the charset is included directly in the connection string.
Following other threads, I made a test file
without any external include
saved as UTF8
specified all the possible Vbscript Codepage and Charset at the beginning of the file
the connection string specifies a charset, I've tried both ucs2 and utf8mb4
the html segment specifies the utf charset both in the Content Type and in the form
when reading from the database I specify to CONVERT USING utf8mb4
The page is setup to be the simplest test possible, with a form on top, and the results listed below.
All the simple text is processed correctly, while any emoji is read as ?
For example, if this is the text passed through the form -> hello 😀 😃 😄 😁 😆 😅 😂
this is what is returned -> hello ? ? ? ? ? ? ?
Executing the same SELECT query in Workbench 8, shows the correct text with the emoji, so it's not a problem directly related to the Codepage or Charset, or when it writes the data, but only when it read them.
VarType report the RS("Field") as a simple string / 8.
I've spent the last days studying and testing all the possible solutions, but can't solve it :/
<%#Language="VBScript" CodePage="65001"%>
<% Option Explicit %>
<%
' THE PAGE ENCODING
Response.ContentType = "text/html;charset=UTF-8"
Session.CodePage = 65001
Response.CodePage = 65001
Response.CharSet = "UTF-8"
'----------------------------------------------------------------
dim dbConn, sql, RS
' SIMPLE CONNECTION STRING
' NOTE THAT I'VE TRIED BOTH charset=ucs2 AND charset=utf8mb4
Set dbConn = Server.CreateObject ("ADODB.Connection")
dbConn.Open "DRIVER={MySQL ODBC 8.0 UNICODE Driver}; SERVER=127.0.0.1; PORT=3306; DATABASE=test; Uid=user; Pwd=password; charset=ucs2;"
'----------------------------------------------------------------
%>
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>test MySQL UTF-8</title>
</head>
<body>
<form action="?action=insert" method="post" accept-charset="utf-8">
<textarea name="text"></textarea>
<input type="submit" value="Insert">
</form>
<%
'----------------------------------------------------------------
' INSERT THE STRING WHEN THE FORM IS SUBMITTED
If Request.Querystring("action") = "insert" Then
sql = "INSERT INTO table (Field) VALUES(AES_ENCRYPT('" & Request.Form("text") & "', 'AES_Key'))"
Response.Write(sql & "</br>")
dbConn.execute(sql)
End If
'----------------------------------------------------------------
' LIST ALL THE RECORDS FROM THE TABLE
sql = "SELECT CONVERT(AES_DECRYPT(Field, 'AES_Key') USING utf8mb4) AS Field FROM table"
Set RS = dbconn.execute(sql)
Do Until RS.EOF
Response.Write(RS("Field") & "</br>")
RS.MoveNext
Loop
RS.close
dbConn.close
%>
</body>
</html>
Related
Good Morning!
I've been trying to implement emojis in my applications for a while, but I'm having a lot of difficulties. I have already tested several internet solutions, but none has been effective.
I will try to detail as much as I am doing:
I'm using Classic ASP and MySQL in versions
5.6.40-84.0-log and 5.6.26-log.
The application is hosted on a Plesk Windows (I tried to run locally
and the same goes for the database). Notepad ++ pages have already
been tested in UTF-8 and UTF8 without BOM.
In the HTML <head> there is <meta charset = "utf-8">.
The <form> has the tag accept-charset="UTF-8".
The ASP has Response.AddHeader "Content-Type", "text/html;charset=UTF-8", Response.CodePage=65001,Response.LCID=1060 and Response.Charset="utf-8".
Now for the problems:
In both versions of MySQL, when I change COLLATE to utf8mb4_unicode_* or utf8mb4_bin, it returns to utf8mb4_0900_ai_ci automatically.
In version 5.6.40-84.0-log, if I register the emojis directly in the database, they are like ??????????????.
In version 5.6.26-log, if I register directly with the database, the error returns:
Executing:
INSERT INTO `db`.`table` (` emoji`) VALUES ('tion 123 😀😉😙😐🤐 ão');
Operation failed: There was an error while applying the SQL script to the database.
ERROR 1366: 1366: Incorrect string value: '\ xF0 \ x9F \ x98 \ x80 \ xF0 \ x9F ...' for column 'emoji' at row 1
SQL Statement:
INSERT INTO 'db'. 'Table' ('emoji') VALUES ('tion 123 😀😉😙😐🤐 ão')
In this same version, registering through the of the page, using SET NAMES 'utf8mb4' or SET NAMES' utf8' in the ODBC connection string, the following error is returned:
Microsoft OLE DB Provider for ODBC Drivers error '80004005'
[MySQL] [ODBC 3.51 Driver] [mysqld-5.6.22] Incorrect string value: '\ xE7 \ xE3o' for column 'field' at row 1
EDIT 1:
When I consult SHOW SESSION VARIABLES LIKE 'character_set%'; and SHOW SESSION VARIABLES LIKE "%collation%"; the database it returns the following results:
Thanks!
Translation by Google Translate (rsrs)
For Emoji and some Chinese, you need utf8mb4, not utf8 in MySQL.
Since you are running the old 5.6, you may stumble over another problem. http://mysql.rjweb.org/doc.php/limits#767_limit_in_innodb_indexes
If that E7E3 should have been çã, then you have some latin1-encoded text, too.
Do not mix encodings in a single column.
Your output for SHOW SESSION VARIABLES LIKE 'char%' shows that the connection is just utf8; it needs to be utf8mb4. The main difference between them is in Emoji.
Is this error a Windows MariaDB driver issue?
[MySQL][ODBC 5.1 Driver][mysqld-5.5.5-10.2.12-MariaDB]Source character set not supported by client
I keep getting the error when trying to connect using this asp code:
Set rs = Server.CreateObject("ADODB.Recordset")
rs.ActiveConnection = Connection_STRING
rs.Source = "SELECT * FROM mytable WHERE text-str = 'some text'
rs.CursorType = 1
rs.CursorLocation = 2
rs.LockType = 1
rs.Open()
if (rsL.EOF) then
...
end if
rs.Close()
Set rs = Nothing
It was only after changing text-str datatype form VARCHAR to TEXT that things started to work.
But some time later I found that I couldn't get any text from the table. I could save to it, but not get it back.
Was finally able to resolve it by changing the field's default collation from uft8mb4_general_ci to uft8_general_ci. And now VARCHAR also works.
The error I started after a Wordpress plugin update/removal. Somehow this changed some default settings to the database, and the way I was connecting to it was not working any more.
Installing an updated ODBC driver solved it.
I have set my mysql database, tables and columns to be charset utf8mb4.
I have set the .asp page to have Response.charset="utf-8"
How do write a select where I check if the rs is including åäö characters?
If I use the below and the variable is without åäö then it works, but if the variable contains a word with åäö, then it doesn´t find it!
thepassword="trött"
sql = "SELECT * FROM table where password = '"& thepassword &"' ;"
set rs = conn.Execute (sql)
Any input really appreciated, thanks.
I'm trying to insert an emoji to my MySQL database with a querystring and I don´t know what Im doing wrong?
I have a MySQL database that is set to utf8mb4 and I can insert and emoji to a table with the Mysqlworkbench and it shows as a real emoji. So the database is working as it should. And if I just try to show that on a .asp page then I shows an emoji as it should.
I have set the database, table, and column to be utf8mb4_general_ci, is this right?
And my db connection (koppling.asp):
strConn = "driver={MySQL ODBC 5.2 Unicode Driver};server=localhost;uid=xxx;pwd=xxx;database=emojtest;option=3;charset=utf8mb4;"
set conn = Server.CreateObject("ADODB.Connection")
conn.Open strConn
To my insertemoji.asp page I'm sending the querystring/variable like this.
www.somepage.se/emojitest/insertemoji.asp?text=😜😀😊😃
And my insertemoji.asp looks like this.
<!--#include file="koppling.asp" -->
<%
Response.charset="utf-8"
thetext = request.querystring("text")
sql= "INSERT INTO news (text) VALUES ('"&thetext&"');"
conn.Execute (sql)
response.write thetext (this is showing 😜😀😊😃)
%>
But the emoji is not stored as an emoji, only like this 😜😀😊😃 and if it was encoded right it would be something like this I think \xF0\x9F\x98\x82
So what am I missing? Where do I go wrong?
If it should be stored like \xF0\x9F\x98\x82, how do I then convert/encode it to a real emoji when I display it on my .asp page?
How is an emoji suppose to be stored in the DB?
Should it be like a visible real emoji?
Or like this 😜😀😊😃
Or like this \xF0\x9F\x98\x82
Solved
I finally figured out the problem, I used the MySQL ODBC 5.2 Unicode Driver and when I changed to MySQL ODBC 3.51 Driver it worked!
😜😀😊😃, when Mojibaked, becomes 😜😀😊😃. This because of confusion over the hex F09F989CF09F9880F09F988AF09F9883, which is treated as Emoji in utf8mb4, but that gibberish by latin1.
To see what probably went wrong in the code, see Trouble with UTF-8 characters; what I see is not what I stored
I have moved my server from dreamhost to Godaddy recently
When I was with Dreamhost it was no problem when my visitors submitting a form in my website using other language than English.
For example they use Thai language when they submit the form I receive it with no problem.
but what happen now when they submit any form I will receive a text like ?????????? so I though it might be just the setting on the php.ini file if that was set correctly and I see it was using UTF-8
; PHP's built-in default is text/html
default_mimetype = "text/html"
default_charset = "UTF-8"
and
exif.encode_unicode = "UTF-8"
exif.decode_unicode_motorola = "UCS-2BE"
exif.decode_unicode_intel = "UCS-2LE"
;exif.encode_jis =
exif.decode_jis_motorola = JIS
exif.decode_jis_intel = JIS
but I can't figure out why this is happen while all was set correctly
and each of my html/php header file I have included
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
Any suggestion what I missed?
To put some more information that
If I use the FTP client such as Filezilla there will be another problem that when I edit the file .php or .html the content in the file if it has any other language than English it will go all like question mark ??????? but if I edit it from the web like Net2ftp then it has no problem on that
I had a similar problem with PHP + MySQL, MySQL refused to properly store UTF8 strings. The solution was to tell MySQL each time after connection, to use UTF8. I believe the SQL command to do so was SET CHARACTER SET UTF8, or SET NAMES UTF8 but I'm not sure, it was a long ago.
Edit: Found some old PHP code of mine, I hope it is helpful:
function sql_connect(){
global $host,$user,$pass,$dbname,$conn;
// connect to mysql
$conn=mysql_connect($host,$user,$pass);
// if cannot connect, exit
if(!$conn){
die('Cannot connect to MySQL server: '.mysql_errno().' - '.mysql_error());
};
// if cannot set to utf8, exit
if(!mysql_query('set names utf8;',$conn)){
mysql_close($conn);
die('Cannot set to UTF8: '.mysql_errno().' - '.mysql_error());
};
// if cannot select database, exit
if(!mysql_select_db($dbname,$conn)){
mysql_close($conn);
die('Cannot select database: '.mysql_errno().' - '.mysql_error());
};
};
function sql_to_html($x){
return(htmlspecialchars(stripslashes($x),ENT_QUOTES,"UTF-8"));
};
Don't forget to set PHP's internal encoding to UTF8 using iconv_set_encoding. Your php.ini preferences only affect the HTTP headers that are sent with your document! Finally, make sure you pass the encoding to mangling functions like htmlentities.