Skip to main content

glassfish 3.1.2 connection pool gets depleted

Please note these java.net forums are being decommissioned and use the new and improved forums at https://community.oracle.com/community/java.
No replies
miro.m
Offline
Joined: 2012-05-07

Hi,
Im running glassfish 3.1.2 in a clustered configuration (2 nodes + DAS).
Database is MySQL with single master DB and pool of RO slaves accessed through
Load Balancer (with sticky TCP connections).

Few times a day on one of the glassfish nodes DB connections in RO pool drop from steady-pool-size (128) to 1.

At that point server becomes unresponsive and logs full of "Interrupting idle thread".

I was able to capture logs at the point where connections started dropping and there were no exceptions in the server log. Unlike in the case with unadequately sized connection polls where you would hit max-pool-size and won't be able to get any more connections.

One of the few succesful thread dumps (attached) at that point shows no spinning threads eating up CPU as in the case with GZIP compression that we had to disable. Compared to "quiet" state most of the threads are waiting on connector-timer-proxy thread!?

Below is the configuration of the Ro pool that gets depleted. Other pools (RW) are similarily configured with lowerupper limit (they would expand but never get anywhere near their max-pool-size).

Any ideas?

Thanks,
Miro

resources.jdbc-connection-pool.mysqlAggRootPool.allow-non-component-callers=false
resources.jdbc-connection-pool.mysqlAggRootPool.associate-with-thread=false
resources.jdbc-connection-pool.mysqlAggRootPool.connection-creation-retry-attempts=0
resources.jdbc-connection-pool.mysqlAggRootPool.connection-creation-retry-interval-in-seconds=10
resources.jdbc-connection-pool.mysqlAggRootPool.connection-leak-reclaim=true
resources.jdbc-connection-pool.mysqlAggRootPool.connection-leak-timeout-in-seconds=250
resources.jdbc-connection-pool.mysqlAggRootPool.connection-validation-method=table
resources.jdbc-connection-pool.mysqlAggRootPool.datasource-classname=com.mysql.jdbc.jdbc2.optional.MysqlXADataSource
resources.jdbc-connection-pool.mysqlAggRootPool.fail-all-connections=true
resources.jdbc-connection-pool.mysqlAggRootPool.idle-timeout-in-seconds=300
resources.jdbc-connection-pool.mysqlAggRootPool.is-connection-validation-required=true
resources.jdbc-connection-pool.mysqlAggRootPool.is-isolation-level-guaranteed=true
resources.jdbc-connection-pool.mysqlAggRootPool.lazy-connection-association=false
resources.jdbc-connection-pool.mysqlAggRootPool.lazy-connection-enlistment=false
resources.jdbc-connection-pool.mysqlAggRootPool.match-connections=false
resources.jdbc-connection-pool.mysqlAggRootPool.max-connection-usage-count=0
resources.jdbc-connection-pool.mysqlAggRootPool.max-pool-size=512
resources.jdbc-connection-pool.mysqlAggRootPool.max-wait-time-in-millis=60000
resources.jdbc-connection-pool.mysqlAggRootPool.name=mysqlAggRootPool
resources.jdbc-connection-pool.mysqlAggRootPool.non-transactional-connections=false
resources.jdbc-connection-pool.mysqlAggRootPool.object-type=user
resources.jdbc-connection-pool.mysqlAggRootPool.ping=false
resources.jdbc-connection-pool.mysqlAggRootPool.pool-resize-quantity=32
resources.jdbc-connection-pool.mysqlAggRootPool.pooling=true
resources.jdbc-connection-pool.mysqlAggRootPool.res-type=javax.sql.XADataSource
resources.jdbc-connection-pool.mysqlAggRootPool.statement-cache-size=0
resources.jdbc-connection-pool.mysqlAggRootPool.statement-leak-reclaim=true
resources.jdbc-connection-pool.mysqlAggRootPool.statement-leak-timeout-in-seconds=150
resources.jdbc-connection-pool.mysqlAggRootPool.statement-timeout-in-seconds=-1
resources.jdbc-connection-pool.mysqlAggRootPool.steady-pool-size=128
resources.jdbc-connection-pool.mysqlAggRootPool.validate-atmost-once-period-in-seconds=60
resources.jdbc-connection-pool.mysqlAggRootPool.validation-table-name=ovi_aggregate.FPTProductsAggregate
resources.jdbc-connection-pool.mysqlAggRootPool.wrap-jdbc-objects=true
AttachmentSize
crescemon-threads-2012-05-07-1508.txt.gz52.02 KB