Unicode characters are being saved incorrectly - mysql

Question

I have a mysql database with unicode text strings in it. My JSF application (running on tomcat 6) can read these unicode strings out and display them correctly in the browser. All the html charsets are set to UTF-8.
When I save my object, even having made no changes, Hibernate persists it back to the database. If I look directly in the database now, I see that characters with accents have become junk.
My database connection string is:
<property name="hibernate.connection.url"
value="jdbc:mysql://database.address.com/dbname?autoReconnect=true&zeroDateTimeBehavior=convertToNull&useUnicode=true&characterEncoding=utf8"/>
Changing that characterEncoding to UTF-8 instead of utf8 makes no difference.
Actually it was working days ago for sure, but now it isn't and I'm out of ideas for what to check. Do you have any suggestions? I can provide any further info deemed relevant.

Answer 1

Does it only happen on certain tables? Perhaps the DEFAULT CHARSET of those tables is different than the others. (http://dev.mysql.com/doc/refman/5.1/en/charset-table.html)

Answer 2

Some options to resolve that come to my mind:
if using facelets, specify <?xml version="1.0" encoding="UTF-8"?> at the top of the page
if using JSP, specify <%# page pageEncoding="utf-8" %>
if that does not work, make a filter that is mapped to the faces servlet, and does request.setCharacterEncoding("utf-8")

Answer 3

I had the same problem as you but with PostgreSQL. You can use filter as below. It worked for me. I think that it must be better solution for that, but i haven't found yet :/
import java.io.IOException;
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
public class UnicodeFilter implements Filter {
#Override
public void destroy() {}
#Override
public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
// Set the characterencoding for the request and response streams.
req.setCharacterEncoding("UTF-8");
res.setContentType("text/html; charset=UTF-8");
chain.doFilter(req, res);
}
#Override
public void init(FilterConfig arg0) throws ServletException {}
}
In web.xml:
<filter>
<filter-name>utffilter</filter-name>
<filter-class>utils.UnicodeFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>utffilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Answer 4

In Spring a simplest solution i've found is CharacterEncodingFilter.
<filter>
<filter-name>encoding-filter</filter-name>
<filter-class>
org.springframework.web.filter.CharacterEncodingFilter
</filter-class>
<init-param>
<param-name>encoding</param-name>
<param-value>UTF-8</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>encoding-filter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Answer 5

Ensure UTF-8 encoded pages so that certain characters are displayed
and submitted correctly...
Add this filter to web.xml.
<filter>
<filter-name>encodingFilter</filter-name>
<filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
<init-param>
<param-name>encoding</param-name>
<param-value>UTF-8</param-value>
</init-param>
<init-param>
<param-name>forceEncoding</param-name>
<param-value>true</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>encodingFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>