Skip to main content

Glassfish 4.0 cluster + mod_jk. Servlet streaming content issue

6 replies [Last post]
elioengcomp
Offline
Joined: 2010-07-01
Points: 0

I'm trying to set up a glassfish 4.0 cluster with 2 instances using Apache 2.2 with mod_jk for load balancing. Everything is working as expected, except for a servlet used to serve images.

When I access this servlet on an instance I get the image content without problems (see attachment getImage-instance.jpg).

But when I access this servlet through load balance I get garbage data on the content (see attachment getImage-lb.jpg).

I have noticed that this problem doesn't happen with small images - up to 10KB - just with images larger than that.

I have also noticed that another servlet that serves text content is having a similiar problem, but in that case, the servlet send just part of the file and the request takes nonsenses 1.5 minutes for a 32.6 KB response (!?!?!?)

Here is the part of the servlet code responsible for serving the image content:

try{
// Open streams.
input = new BufferedInputStream(new FileInputStream(fimage),
DEFAULT_BUFFER_SIZE);
output = new BufferedOutputStream(response.getOutputStream(),
DEFAULT_BUFFER_SIZE);

// Write file contents to response.
byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
int length;
while ((length = input.read(buffer)) > 0) {
output.write(buffer, 0, length);
}

output.flush();
}

} catch (Exception e) {
// Tratamento de erros
logger.log(Level.SEVERE, "Falha ao servir imagem", e);
try {
response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
return;
} catch (IOException ex) {
logger.log(Level.SEVERE, "Falha ao responder erro", ex);
}
} finally {
// Close stream
if (input != null)
try {
input.close();
} catch (IOException e) {
logger.log(Level.SEVERE, "Falha ao fechar input", e);
}
}
Here are entries on httpd.conf

LoadModule jk_module modules/mod_jk.so

JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat "%w %V %T"

#HTTPS Configuration
JkExtractSSL On
JkHTTPSIndicator HTTPS
JkSESSIONIndicator SSL_SESSION_ID
JkCIPHERIndicator SSL_CIPHER
JkCERTSIndicator SSL_CLIENT_CERT

JkMount /olhameubebe/* loadbalancer

Listen 443

ServerAdmin some@email.com
DocumentRoot "Your Root folder location"
ServerName www.domain.com:443
ServerAlias domain.com:443
ErrorLog "logs/anyFile-error.log"
CustomLog "logs/anyFile-access.log" common
SSLEngine on
SSLCertificateFile "C:/Program Files (x86)/Apache Software Foundation/Apache2.2/bin/server.crt"
SSLCertificateKeyFile "C:/Program Files (x86)/Apache Software Foundation/Apache2.2/bin/server.key"
JkMount /olhameubebe/* loadbalancer

The worker.properties file looks like

# Define 1 real worker using ajp13
worker.list=worder1,worker2,loadbalancer

# Set properties for worker1 (ajp13)
worker.worker1.type=ajp13
worker.worker1.host=localhost
worker.worker1.port=28080
worker.worker1.socket_keepalive=0

# Set properties for worker2 (ajp13)
worker.worker2.type=ajp13
worker.worker2.host=localhost
worker.worker2.port=28081
worker.worker2.socket_keepalive=0

# Set properties for loadbalancer
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=worker1,worker2
worker.loadbalancer.sticky_session=1
I'm doing this test on windows and I don't see any error/warning on log files.

Does anybody have any idea about what Am I doing wrong?

Thanks in advance!

Does anyone have any idea about what is going wrong?

AttachmentSize
getImage-instance.jpg45.99 KB
getImage-lb.jpg45.99 KB

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
elioengcomp
Offline
Joined: 2010-07-01
Points: 0

After a lot of attemps and experiments, looks like the problem was solved after I remove "Content-Length" from response headers.

So I just had to comment this line from servlet's code:

//response.setHeader("Content-Length", String.valueOf(fimage.length()));

oleksiys
Offline
Joined: 2006-01-25
Points: 0

Please try this patch [1], it has to fix the problem.

WBR,
Alexey.

[1] https://dl.dropboxusercontent.com/u/7319744/nucleus-grizzly-all.jar

oleksiys
Offline
Joined: 2006-01-25
Points: 0

Update:

Actually I can reproduce the problem. Stay tuned...

>> Can't reproduce the issue even with larger files.
>>
>> Can you pls. test if the same issue is reproducible when you use reverse proxy (instead of mod_jk)
>>
>> ProxyPass / ajp://localhost:28080/
>> ProxyPassReverse / ajp://localhost:28080/
>>
>>
>> What is the DEFAULT_BUFFER_SIZE value?
>>
>>Thanks.
>>
>> WBR,
>> Alexey.

elioengcomp
Offline
Joined: 2010-07-01
Points: 0

Nice!

I'll try the patch with servlet's original code (setting "Content-Length" header) and send the results later.

Thanks!

oleksiys
Offline
Joined: 2006-01-25
Points: 0

Any updates?

elioengcomp
Offline
Joined: 2010-07-01
Points: 0

Sorry Alexey, but I didn't have time to execute that test yet.

I'll do that as soon as the release I'm working on has been finished.

Thank you!

Elio