Skip to main content

hs_err with jre - coldfusion

3 replies [Last post]
switcharoonie
Offline
Joined: 2008-08-20
Points: 0

Hi,

I hope this is the right discussion for this post. I have seen a couple of topics that hit on hotspot errors but nothing that hits on jre/coldfusion errors. I have an output from several hs_err*.pid files that have occurred over a couple of weeks and was wondering if anyone could help. This is running ColdFusion 8 as multiserver running solaris 10.

Here's the output.

Thanks.

#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xfe9d7a5c, pid=15727, tid=12
#
# Java VM: Java HotSpot(TM) Server VM (1.6.0_01-b06 mixed mode)
# Problematic frame:
# V [libjvm.so+0x1d7a5c]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#

--------------- T H R E A D ---------------

Current thread (0x0058e800): JavaThread "Poller SunPKCS11-Solaris" daemon [_thread_in_vm, id=12]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000000

Registers:
O0=0x0058e8e8 O1=0x00000000 O2=0x00025c00 O3=0x00578c4c
O4=0x00029dbc O5=0x00029dc0 O6=0xcef7f380 O7=0x00000000
G1=0x00000000 G2=0x00000000 G3=0xfefc6acc G4=0x0002c800
G5=0x00020000 G6=0x00000000 G7=0xfe5b2400 Y=0x00000000
PC=0xfe9d7a5c nPC=0xfe9d7a60

Top of Stack: (sp=0xcef7f380)
0xcef7f380: 00000000 fefc3dc0 000b1d00 00000001
0xcef7f390: fef9a000 00029c00 0002cacc feff0000
0xcef7f3a0: 0058e8e8 00000000 00000022 00578c4c
0xcef7f3b0: 0058e800 20000000 cef7f3f0 cfbd6058
0xcef7f3c0: 0058e800 0058e8e8 cef7f3f0 cfbd603c
0xcef7f3d0: 00000000 00000000 00556fb4 00000000
0xcef7f3e0: 0058e800 00556fa4 0000006c feee413f
0xcef7f3f0: 00000000 00010800 00010800 fefbddd8

Instructions: (pc=0xfe9d7a5c)
0xfe9d7a4c: 92 10 00 19 01 00 00 00 90 10 00 18 15 00 00 97
0xfe9d7a5c: f0 06 60 00 92 1a a1 06 ae 05 00 09 ca 0d e0 00

Stack: [0xcef00000,0xcef80000), sp=0xcef7f380, free space=508k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x1d7a5c]
C [libj2pkcs11.so+0x6060]
C [libj2pkcs11.so+0x5f90] Java_sun_security_pkcs11_wrapper_PKCS11_C_1GetSlotInfo+0x58
j sun.security.pkcs11.wrapper.PKCS11.C_GetSlotInfo(J)Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;+0
j sun.security.pkcs11.wrapper.PKCS11.C_GetSlotInfo(J)Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;+0
j sun.security.pkcs11.SunPKCS11.initToken(Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;)V+12
j sun.security.pkcs11.SunPKCS11.access$100(Lsun/security/pkcs11/SunPKCS11;Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;)V+2
j sun.security.pkcs11.SunPKCS11$TokenPoller.run()V+45
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [libjvm.so+0xe1468]
V [libjvm.so+0x398ee0]
V [libjvm.so+0x18d054]
V [libjvm.so+0x1a127c]
V [libjvm.so+0x6221a4]
V [libjvm.so+0x58ef80]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j sun.security.pkcs11.wrapper.PKCS11.C_GetSlotInfo(J)Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;+0
j sun.security.pkcs11.SunPKCS11.initToken(Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;)V+12
j sun.security.pkcs11.SunPKCS11.access$100(Lsun/security/pkcs11/SunPKCS11;Lsun/security/pkcs11/wrapper/CK_SLOT_INFO;)V+2
j sun.security.pkcs11.SunPKCS11$TokenPoller.run()V+45
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
0x00635800 JavaThread "scheduler-11" [_thread_blocked, id=79]
0x005fcc00 JavaThread "scheduler-10" [_thread_blocked, id=78]
0x0082a400 JavaThread "scheduler-7" [_thread_blocked, id=75]
0x03110800 JavaThread "scheduler-6" [_thread_blocked, id=74]
0x00e71800 JavaThread "scheduler-3" [_thread_in_Java, id=69]
0x00053c00 JavaThread "DestroyJavaVM" [_thread_blocked, id=1]
0x006e7800 JavaThread "jms-bwq-7331688455278509836" daemon [_thread_blocked, id=68]
0x00351000 JavaThread "jms-bwq-3201209212940513056" daemon [_thread_blocked, id=67]
0x000e6800 JavaThread "jms-bwq-8867534153934136545" daemon [_thread_blocked, id=66]
0x00624c00 JavaThread "web-0" [_thread_in_native, id=55]
0x00615400 JavaThread "mipc-1" [_thread_in_native, id=53]
0x00617400 JavaThread "RMI TCP Accept-2517" daemon [_thread_in_native, id=52]
0x0060a000 JavaThread "Thread-12" [_thread_blocked, id=51]
0x00603000 JavaThread "jndi-5" [_thread_blocked, id=50]
0x005fb400 JavaThread "clock" daemon [_thread_blocked, id=47]
0x005f6400 JavaThread "Transaction Domain DefaultDomain" [_thread_blocked, id=46]
0x005f5800 JavaThread "tyrex.util.daemonMaster" daemon [_thread_blocked, id=45]
0x005dd000 JavaThread "jndi-4" [_thread_in_native, id=44]
0x005eb000 JavaThread "jndi-2" [_thread_blocked, id=42]
0x005ea400 JavaThread "jndi-1" [_thread_blocked, id=41]
0x005eec00 JavaThread "jndi-0" [_thread_blocked, id=40]
0x0060b800 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=38]
0x0060b000 JavaThread "jms-fifo-5" [_thread_blocked, id=36]
0x005d9800 JavaThread "jms-fifo-4" [_thread_blocked, id=35]
0x005d9000 JavaThread "jms-fifo-3" [_thread_blocked, id=34]
0x00598000 JavaThread "jms-fifo-2" [_thread_blocked, id=33]
0x005c2400 JavaThread "jms-fifo-1" [_thread_blocked, id=32]
0x005a5c00 JavaThread "GC Daemon" daemon [_thread_blocked, id=19]
0x005a5400 JavaThread "RMI Reaper" [_thread_blocked, id=18]
0x005b8800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=17]
=>0x0058e800 JavaThread "Poller SunPKCS11-Solaris" daemon [_thread_in_vm, id=12]
0x00158400 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=10]
0x00156800 JavaThread "CompilerThread1" daemon [_thread_blocked, id=9]
0x00155800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=8]
0x00154800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=7]
0x00143c00 JavaThread "Finalizer" daemon [_thread_blocked, id=6]
0x00140000 JavaThread "Reference Handler" daemon [_thread_blocked, id=5]

Other Threads:
0x0013cc00 VMThread [id=4]
0x00159800 WatcherThread [id=11]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
PSYoungGen total 141824K, used 105844K [0xf1400000, 0xfc000000, 0xfc000000)
eden space 107584K, 98% used [0xf1400000,0xf7b5d1b0,0xf7d10000)
from space 34240K, 0% used [0xf9e90000,0xf9e90000,0xfc000000)
to space 34176K, 0% used [0xf7d30000,0xf7d30000,0xf9e90000)
PSOldGen total 352256K, used 352255K [0xdbc00000, 0xf1400000, 0xf1400000)
object space 352256K, 99% used [0xdbc00000,0xf13ffff8,0xf1400000)
PSPermGen total 20480K, used 16866K [0xcfc00000, 0xd1000000, 0xdbc00000)
object space 20480K, 82% used [0xcfc00000,0xd0c78a88,0xd1000000)

Dynamic libraries:
0x00010000 /opt/jrun4/bin/jrun
0xff398000 /lib/libpthread.so.1
0xff370000 /lib/libdl.so.1
0xff200000 /lib/libc.so.1
0xff340000 /platform/SUNW,Sun-Fire-280R/lib/libc_psr.so.1
0xfe800000 /opt/jrun4/jre/lib/sparc/server/libjvm.so
0xff300000 /lib/libsocket.so.1
0xff330000 /usr/lib/libsched.so.1
0xff1d0000 /lib/libm.so.1
0xff1a0000 /usr/lib/libCrun.so.1
0xff3ec000 /lib/libthread.so.1
0xff170000 /lib/libdoor.so.1
0xff080000 /lib/libnsl.so.1
0xfe700000 /lib/libm.so.2
0xff140000 /lib/libscf.so.1
0xff060000 /lib/libuutil.so.1
0xff030000 /lib/libgen.so.1
0xff000000 /lib/libmd.so.1
0xfe7e0000 /platform/SUNW,Sun-Fire-280R/lib/libmd_psr.so.1
0xfe6e0000 /lib/libmp.so.2
0xfe6c0000 /opt/jrun4/jre/lib/sparc/native_threads/libhpi.so
0xfe630000 /opt/jrun4/jre/lib/sparc/libverify.so
0xfe5f0000 /opt/jrun4/jre/lib/sparc/libjava.so
0xfe5d0000 /opt/jrun4/jre/lib/sparc/libzip.so
0xfe590000 /usr/lib/locale/en_US.ISO8859-1/en_US.ISO8859-1.so.3
0xfc050000 /opt/jrun4/jre/lib/sparc/libnet.so
0xcfbd0000 /opt/jrun4/jre/lib/sparc/libj2pkcs11.so
0xcfb90000 /usr/lib/libpkcs11.so
0xcfae0000 /usr/lib/libcryptoutil.so.1
0xcf710000 /usr/lib/security/pkcs11_softtoken.so
0xfc010000 /opt/jrun4/jre/lib/sparc/librmi.so

VM Arguments:
jvm_args: -Xmx512m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseParallelGC -Djava.awt.headless=true -Dcoldfusion.rootDir=/opt/jrun4/ -Djmx.invoke.getters=true
java_command:
Launcher Type: generic

Environment Variables:

LD_LIBRARY_PATH=/opt/jrun4/jre/lib/sparc/server:/opt/jrun4/jre/lib/sparc:/opt/jrun4/jre/../lib/sparc:/opt/jrun4/jre/bin:/opt/jrun4/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib
SHELL=/bin/ksh

Signal Handlers:
SIGSEGV: [libjvm.so+0x66b340], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0x66b340], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0x15c3f4], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0x15c3f4], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0x15c3f4], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGHUP: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: [libjvm.so+0x590b9c], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0x590b9c], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGQUIT: [libjvm.so+0x590b9c], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0x590b9c], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0x593ab4], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0x15c3f4], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c

--------------- S Y S T E M ---------------

OS: Solaris 10 6/06 s10s_u2wos_09a SPARC
Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 09 June 2006

uname:SunOS 5.10 Generic_120011-14 sun4u (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:1.13 1.12 1.12

CPU:total 2 has_v8, has_v9, has_vis1, has_vis2, is_ultra3

Memory: 8k page, physical 8388608k(5784848k free)

vm_info: Java HotSpot(TM) Server VM (1.6.0_01-b06) for solaris-sparc, built on Mar 14 2007 01:02:33 by "" with unknown Workshop:0x580

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
switcharoonie
Offline
Joined: 2008-08-20
Points: 0

Thanks for the quick responses. I will check these out and get back to you.

claudio
Offline
Joined: 2003-06-17
Points: 0

When crash occurs, do they all look the same ?

Because this specifically crash, report a thread named "Poller SunPKCS11-Solaris" were running.

Do you use some NSS provider ? Do you need it ?

From this GC log, looks like OLD generation reached its limit, can you increase it ?
[pre]
PSOldGen total 352256K, used 352255K [0xdbc00000, 0xf1400000, 0xf1400000)
object space 352256K, 99% used [0xdbc00000,0xf13ffff8,0xf1400000)
[/pre]

linuxhippy
Offline
Joined: 2004-01-07
Points: 0

> I hope this is the right discussion for this post. I
> have seen a couple of topics that hit on hotspot
Well not really, this forum is for performance, not for stability problems.

> errors but nothing that hits on jre/coldfusion
> errors. I have an output from several hs_err*.pid
> files that have occurred over a couple of weeks and
> was wondering if anyone could help. This is running
> ColdFusion 8 as multiserver running solaris 10.
1.6.0_01 is quite old, the only suggestion I can give you is to try the latest JDK6 update.
Maybe you are hitting a bug that was already fixed.

lg Clemens