Glassfish 126.96.36.199 randomly stops logging to access logs - GRIZZLY0023?
We are running into an issues with Glassfish 188.8.131.52 where it seems to randomly stop logging to the access logs. We have enabled access logging as follows:
<access-log buffer-size-bytes="0" write-interval-seconds="0" format="%client.name% %datetime% %request% %status% %response.length% %header.referer% %header.user-agent% %header.x-forwarded-for% %header.host%" rotation-interval-in-minutes="60"></access-log>
<virtual-server id="server" log-file="/apps/weblogs/www/server.log" default-web-module="docroot" access-log="/apps/weblogs/www/access" network-listeners="http-listener-1,http-listener-2">
The issue seems to happen randomly. We'll get full logs for weeks and then 1 day it'll stop logging at some point during the day. Seems like the only way to get it to start again is to restart glassfish. We have a farm of glassfish servers and this happens randomly across them.
We cannot correlate when this happens with any events. There doesn't seem to be errors in any log files. The only thought is that it might be due to load or some threading issue. We currently get around 100 req/s on a glassfish server all requests mostly hitting simple Java Web Apps (not EJBs, or EARs).
We run on JDK 7 on RHEL 6.
Anyone have any thoughts what could be causing this?