Deadlock in Transaction Recovery in Glassfish 188.8.131.52
we just discovered a deadlock between several Glassfish threads when starting the server.
We have a simple @Singleton @Startup EJB with a @PostConstruct method. The code inside this @PostConstruct method tries to read some data from the database using injected JPA EntityManager. The Glassfish main thread hangs exactly while executing this singleton. The deadlock occurs while it tries to obtain a connection from a pool and tries invoke RecoveryManager.waitForResync. There are another two threads taking part on the deadlock: JTS Resync Thread (waiting on EventSemaphore inside of RecoveryManager.proceedWithXARecovery) and Recovery Helper Thread (blocked on com.sun.hk2.component.SingletonInhabitant). The main thread is waiting on EventSemaphore inside of RecoveryManager.waitForResync. So it seems that the main thread and Recovery Helper Thread are blocking each other. Seems that Recovery Helper Thread has to do something and than notify all parties waiting on EventSemaphore, but it can't complete its tasks since it is blocked on SingletonInhabitant.get, which is owned by the main thread.
We configured the transaction service with the following properties in our domain.xml:
<property name="pending-txn-cleanup-interval" value="600"></property>
The stack traces of the main thread and the two JTS threads are attached to the topic.
Is it a bug or did we misconfigured something?