Skip to main content

why the default response character encoding must be ISO-8859-1?

7 replies [Last post]
Anonymous

I am trying to migrate my webapps from tomcat 6.0 to glassfish v3... but I
found the my utf-8 pages are not working well with gf3.

my jsps like this:

<%@page contentType="text/html"%>

...

content

Those pages are working well with apache + tomcat 6.0 but with gf3, Chinese
characters are distorted and I have to right click and manually adjust the
Encoding to Unicode (UTF-8) to display Chinese words correctly. I checked
that the page encoding was somehow adjusted to iso-8859-1.

I remember that exactly same problem was happened with tomcat 5.0.28.

Any idea?

Ken

--
View this message in context: http://www.nabble.com/why-the-default-response-character-encoding-must-b...
Sent from the java.net - glassfish users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Jan Luehe

Ken,

On 03/18/09 07:37 AM, Ken--@newsgroupstats.hk wrote:
> Dear,
>
> Is that character encoding for request? What I am talking is encoding of
> response. Am I correct?
>
> Please find attached jsps. utf8-test1.jsp and utf8-test2.jsp.
>
> http://www.nabble.com/file/p22579795/utf8-test1.jsp utf8-test1.jsp
> http://www.nabble.com/file/p22579795/utf8-test2.jsp utf8-test2.jsp
>
> utf8-test1.jsp is working with tomcat only, gf3 will show distorted words.
> The page encoding of utf8-test1.jsp under gf3 is ISO-8859-1. utf8-test2.jsp
> are working with gf3 and tomcat6.
>
> my webapps contents are latin encoding. In order to work with gf3, I have to
> convert all content encoding to utf-8:
>
> str = new String(str.getBytes("iso-8859-1"), "utf-8");
>
> I got this problem when I was using tomcat 5.0.28. Here is the change log of
> tomcat5.0:
>
> http://web.archive.org/web/20071012233252/http://tomcat.apache.org/tomca...
>
> If ServletResponse.getWriter() is called and no char encoding has been
> specified, set response char encoding to default (ISO-8859-1) so that it is
> reflected in getContentType() and Content-Type header, as required by the
> Servlet Spec (Bugtraq 6152759) (luehe)
>
> I did ask tomcat team and they replied that they just follow servlet spec
> 2.0.
>
> The problem is gone in Tomcat 6.0. (I didn't try tomcat 5.5).
>
> Today, I have to migrate my webapps from tomcat6 to gf3. Please advise any
> config or setting can solve the problem without convert all content to utf8.
>
>

When I access utf8-test1.jsp on GlassFish, I get this response:

Content-Type: text/html;charset=ISO-8859-1

whereas on Tomcat 6, I get:

Content-Type: text/html

"charset=ISO-8859-1" is missing from the Content-Type returned
by Tomcat.

Tomcat may be compliant with Servlet 2.0, but it is *not* compliant
with Servlet 2.5.

In Servlet 2.5, ServletResponse#setCharacterEncoding was enhanced
as follows:

*

Containers must communicate the character encoding used for
* the servlet response's writer to the client if the protocol
* provides a way for doing so. In the case of HTTP, the character
* encoding is communicated as part of the Content-Type
* header for text media types.

and ServletResponse#getWriter was amended as follows:

* If the response's character encoding has not been
* specified as described in getCharacterEncoding
* (i.e., the method just returns the default value
* ISO-8859-1), getWriter
* updates it to ISO-8859-1.

for this reason:

Specifying iso-8859-1 explicitly for text media types is necessary
because many clients don't follow the HTTP spec in applying this
default - they often use a default encoding based on the locale
they're running in or based on a user preference. By telling these
clients explicitly what we're really using we increase our chances of
having the text processed or displayed correctly.

As you can see, GlassFish follows the Servlet 2.5 spec, whereas Tomcat
does not. It is time that a test for this issue is added to the Servlet
compliance
suite, in which case we would not have had this discussion.

Why can't you add this page directive:

<%@page contentType="text/html;charset=utf8"%>

to your JSPs, as you did for utf8-test2.jsp?

You could actually declare this page directive in a JSP prelude,
in which case you would not have to declare it on each and every JSP.

Jan

> Regards,
> Ken
>
>
>
>
> Felipe Gaucho wrote:
>
>> fro Glassfish, the following steps are used to set the encoding:
>>
>> * The getCharacterEncoding() method
>> * A hidden field in the form, specified by the
>> form-hint-field attribute of the parameter-encoding element in the
>> sun-web.xml file
>> * The default-charset attribute of the parameter-encoding
>> element in the sun-web.xml file
>> * The default, which is ISO-8859-1
>>
>> On Tue, Mar 17, 2009 at 5:27 PM, Ken--@newsgroupstats.hk
>> wrote:
>>
>>> I am trying to migrate my webapps from tomcat 6.0 to glassfish v3... but
>>> I
>>> found the my utf-8 pages are not working well with gf3.
>>>
>>> my jsps like this:
>>>
>>> <%@page contentType="text/html"%>
>>>
>>>
>>>
>>> ...
>>>
>>>
>>> content
>>>
>>>
>>>
>>> Those pages are working well with apache + tomcat 6.0 but with gf3,
>>> Chinese
>>> characters are distorted and I have to right click and manually adjust
>>> the
>>> Encoding to Unicode (UTF-8) to display Chinese words correctly. I checked
>>> that the page encoding was somehow adjusted to iso-8859-1.
>>>
>>> I remember that exactly same problem was happened with tomcat 5.0.28.
>>>
>>> Any idea?
>>>
>>> Ken
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/why-the-default-response-character-encoding-must-b...
>>> Sent from the java.net - glassfish users mailing list archive at
>>> Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
>>> For additional commands, e-mail: users-help@glassfish.dev.java.net
>>>
>>>
>>>
>>
>> --
>>
>> Please help to test this application:
>> http://fgaucho.dyndns.org:8080/cejug-classifieds-richfaces
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
>> For additional commands, e-mail: users-help@glassfish.dev.java.net
>>
>>
>>
>>
>
>

[att1.html]

kentsang
Offline
Joined: 2008-01-10

I am Ken--.

Thanks to your quick response. I can specify the charset directive easily but actually all my content are in latin1 encoding so I have to wrap a function to encode all content to utf-8.

str = new String(str.getBytes("iso-8859-1"), "utf-8");

As I said before, my webapps are working under tomcat 6.0.18.

From tomcat web site:
Apache Tomcat 6.x is the current focus of development. It builds upon the improvements made in Tomcat 5.5.x and implements the Servlet 2.5 and JSP 2.1 specifications.

I agreed with you about servlet 2.5.

So, any chance or method to make it work without convert the encoding of my content? Servlet Filter? Those content are get from database resultset, web services and properties files.

Regards,
Ken

Ken--@newsgroupstats.hk

Dear,

Is that character encoding for request? What I am talking is encoding of
response. Am I correct?

Please find attached jsps. utf8-test1.jsp and utf8-test2.jsp.

http://www.nabble.com/file/p22579795/utf8-test1.jsp utf8-test1.jsp
http://www.nabble.com/file/p22579795/utf8-test2.jsp utf8-test2.jsp

utf8-test1.jsp is working with tomcat only, gf3 will show distorted words.
The page encoding of utf8-test1.jsp under gf3 is ISO-8859-1. utf8-test2.jsp
are working with gf3 and tomcat6.

my webapps contents are latin encoding. In order to work with gf3, I have to
convert all content encoding to utf-8:

str = new String(str.getBytes("iso-8859-1"), "utf-8");

I got this problem when I was using tomcat 5.0.28. Here is the change log of
tomcat5.0:

http://web.archive.org/web/20071012233252/http://tomcat.apache.org/tomca...

If ServletResponse.getWriter() is called and no char encoding has been
specified, set response char encoding to default (ISO-8859-1) so that it is
reflected in getContentType() and Content-Type header, as required by the
Servlet Spec (Bugtraq 6152759) (luehe)

I did ask tomcat team and they replied that they just follow servlet spec
2.0.

The problem is gone in Tomcat 6.0. (I didn't try tomcat 5.5).

Today, I have to migrate my webapps from tomcat6 to gf3. Please advise any
config or setting can solve the problem without convert all content to utf8.

Regards,
Ken

Felipe Gaucho wrote:
>
> fro Glassfish, the following steps are used to set the encoding:
>
> * The getCharacterEncoding() method
> * A hidden field in the form, specified by the
> form-hint-field attribute of the parameter-encoding element in the
> sun-web.xml file
> * The default-charset attribute of the parameter-encoding
> element in the sun-web.xml file
> * The default, which is ISO-8859-1
>
> On Tue, Mar 17, 2009 at 5:27 PM, Ken--@newsgroupstats.hk
> wrote:
>>
>> I am trying to migrate my webapps from tomcat 6.0 to glassfish v3... but
>> I
>> found the my utf-8 pages are not working well with gf3.
>>
>> my jsps like this:
>>
>> <%@page contentType="text/html"%>
>>
>>
>>
>> ...
>>
>>
>> content
>>
>>
>>
>> Those pages are working well with apache + tomcat 6.0 but with gf3,
>> Chinese
>> characters are distorted and I have to right click and manually adjust
>> the
>> Encoding to Unicode (UTF-8) to display Chinese words correctly. I checked
>> that the page encoding was somehow adjusted to iso-8859-1.
>>
>> I remember that exactly same problem was happened with tomcat 5.0.28.
>>
>> Any idea?
>>
>> Ken
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/why-the-default-response-character-encoding-must-b...
>> Sent from the java.net - glassfish users mailing list archive at
>> Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
>> For additional commands, e-mail: users-help@glassfish.dev.java.net
>>
>>
>
>
>
> --
>
> Please help to test this application:
> http://fgaucho.dyndns.org:8080/cejug-classifieds-richfaces
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>
>
>

--
View this message in context: http://www.nabble.com/why-the-default-response-character-encoding-must-b...
Sent from the java.net - glassfish users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

Ken--@newsgroupstats.hk

[utf8-test1.jsp]

<%@page contentType="text/html"%>






Test Page for Tomcat UTF-8

content type = <%=response.getCharacterEncoding()%>
<%

// all content of my webapps are come from properties file or database. They
are in iso-8859-1 encoding
byte b[] = {-24, -69, -115, -26, -108, -65, -24, -80, -73};
String str = new String(b, "iso-8859-1");

out.print("Chinese words : ");
out.println(str);
%>


[utf8-test2.jsp]
<%@page contentType="text/html;charset=utf8"%>






Test Page for Tomcat UTF-8

content type = <%=response.getCharacterEncoding()%>
<%

// all content of my webapps are come from properties file or database. They
are in iso-8859-1 encoding
byte b[] = {-24, -69, -115, -26, -108, -65, -24, -80, -73};
String str = new String(b, "iso-8859-1");

// with gf3, I have to convert all iso-8859-1 string to utf-8
str = new String(str.getBytes("iso-8859-1"), "utf-8");

out.print("Chinese words : ");
out.println(str);
%>


--
View this message in context: http://www.nabble.com/why-the-default-response-character-encoding-must-b...
Sent from the java.net - glassfish users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

Martin Gainty

Ken-
i've never seem mandarin render under UTF-8 ..i would try UTF-16

<%@page pageEncoding="UTF-16"%>
"http://www.w3.org/TR/html4/loose.dtd">


to see if cantonese glyph characters are covered check here
http://unicode.org/mail-arch/unicode-ml/Archives-Old/UML003/0362.html

Martin
______________________________________________
Verzicht und Vertraulichkeitanmerkung / Disclaimer and confidentiality note
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
This message is confidential and may be privileged. If you are not the intended recipient, we kindly ask you to please inform the sender. Any unauthorised dissemination or copying hereof is prohibited. This message serves for information purposes only and shall not have any legally binding effect. Given that e-mails can easily be subject to manipulation, we can not accept any liability for the content provided.

> Date: Tue, 17 Mar 2009 09:27:16 -0700
> From: dragonken@gmail.com
> To: users@glassfish.dev.java.net
> Subject: why the default response character encoding must be ISO-8859-1?
>
>
> I am trying to migrate my webapps from tomcat 6.0 to glassfish v3... but I
> found the my utf-8 pages are not working well with gf3.
>
> my jsps like this:
>
> <%@page contentType="text/html"%>
>
>
>
> ...
>
>
> content
>
>
>
> Those pages are working well with apache + tomcat 6.0 but with gf3, Chinese
> characters are distorted and I have to right click and manually adjust the
> Encoding to Unicode (UTF-8) to display Chinese words correctly. I checked
> that the page encoding was somehow adjusted to iso-8859-1.
>
> I remember that exactly same problem was happened with tomcat 5.0.28.
>
> Any idea?
>
> Ken
>
>
>
> --
> View this message in context: http://www.nabble.com/why-the-default-response-character-encoding-must-b...
> Sent from the java.net - glassfish users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>

_________________________________________________________________
Hotmail® is up to 70% faster. Now good news travels really fast.
http://windowslive.com/online/hotmail?ocid=TXT_TAGLM_WL_HM_70faster_032009
[att1.html]

Felipe Gaúcho

fro Glassfish, the following steps are used to set the encoding:

* The getCharacterEncoding() method
* A hidden field in the form, specified by the
form-hint-field attribute of the parameter-encoding element in the
sun-web.xml file
* The default-charset attribute of the parameter-encoding
element in the sun-web.xml file
* The default, which is ISO-8859-1

On Tue, Mar 17, 2009 at 5:27 PM, Ken--@newsgroupstats.hk
wrote:
>
> I am trying to migrate my webapps from tomcat 6.0 to glassfish v3... but I
> found the my utf-8 pages are not working well with gf3.
>
> my jsps like this:
>
> <%@page contentType="text/html"%>
>
>
>
> ...
>
>
> content
>
>
>
> Those pages are working well with apache + tomcat 6.0 but with gf3, Chinese
> characters are distorted and I have to right click and manually adjust the
> Encoding to Unicode (UTF-8) to display Chinese words correctly. I checked
> that the page encoding was somehow adjusted to iso-8859-1.
>
> I remember that exactly same problem was happened with tomcat 5.0.28.
>
> Any idea?
>
> Ken
>
>
>
> --
> View this message in context: http://www.nabble.com/why-the-default-response-character-encoding-must-b...
> Sent from the java.net - glassfish users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>
>

--

Please help to test this application:
http://fgaucho.dyndns.org:8080/cejug-classifieds-richfaces

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

Felipe Gaúcho

you can declare the encoding of the application :
http://wiki.glassfish.java.net/Wiki.jsp?page=FaqHttpRequestParameterEnco...

* I am facing the same issue today :)

2009/3/17 Felipe Gaúcho :
> fro Glassfish, the following steps are used to set the encoding:
>
>  *      The getCharacterEncoding() method
>   *      A hidden field in the form, specified by the
> form-hint-field attribute of the parameter-encoding element in the
> sun-web.xml file
>   *      The default-charset attribute of the parameter-encoding
> element in the sun-web.xml file
>   *      The default, which is ISO-8859-1
>
> On Tue, Mar 17, 2009 at 5:27 PM, Ken--@newsgroupstats.hk
> wrote:
>>
>> I am trying to migrate my webapps from tomcat 6.0 to glassfish v3... but I
>> found the my utf-8 pages are not working well with gf3.
>>
>> my jsps like this:
>>
>> <%@page contentType="text/html"%>
>>
>>
>>
>> ...
>>
>>
>> content
>>
>>
>>
>> Those pages are working well with apache + tomcat 6.0 but with gf3, Chinese
>> characters are distorted and I have to right click and manually adjust the
>> Encoding to Unicode (UTF-8) to display Chinese words correctly. I checked
>> that the page encoding was somehow adjusted to iso-8859-1.
>>
>> I remember that exactly same problem was happened with tomcat 5.0.28.
>>
>> Any idea?
>>
>> Ken
>>
>>
>>
>> --
>> View this message in context: http://www.nabble.com/why-the-default-response-character-encoding-must-b...
>> Sent from the java.net - glassfish users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
>> For additional commands, e-mail: users-help@glassfish.dev.java.net
>>
>>
>
>
>
> --
>
> Please help to test this application:
> http://fgaucho.dyndns.org:8080/cejug-classifieds-richfaces
>

--

Please help to test this application:
http://fgaucho.dyndns.org:8080/cejug-classifieds-richfaces

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net