Skip to main content

Re: How to enable JMX monitoring

7 replies [Last post]
Anonymous

Hi Tim,

On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:

> Hello, Jan.
>
> Please take a quick look at this recent blog entry:
>
> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>
> This does not count as "official documentation" but the steps there do work.
>

Thanks very much. Much cleaner than anything else so far.

But no luck for me. The visualvm logs suggest some SSL related problems.

Stacktrace below, in case you have a vague idea.

Jan

Trace:

INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
at javax.naming.InitialContext.lookup(InitialContext.java:392)
at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
Caused: java.io.IOException: Failed to retrieve RMIServer stub
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
[catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
java.net.ConnectException: Operation timed out

[ and more ]

> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>
> - Tim
>
> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>
>> Hi,
>>
>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>
>> Hence the question:
>>
>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>
>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>
>> Jan
>>
>

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
tmueller
Offline
Joined: 2005-10-31
Points: 0

This looks like what you get when using a Java SE 6 update earlier than 22.

Tom

On 10/3/2012 2:31 PM, Jan Algermissen wrote:
> Hi Tim,
>
> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>
>> Hello, Jan.
>>
>> Please take a quick look at this recent blog entry:
>>
>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>
>> This does not count as "official documentation" but the steps there do work.
>>
> Thanks very much. Much cleaner than anything else so far.
>
> But no luck for me. The visualvm logs suggest some SSL related problems.
>
> Stacktrace below, in case you have a vague idea.
>
> Jan
>
>
>
> Trace:
>
> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
> java.io.EOFException: SSL peer shut down incorrectly
> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
> at javax.naming.InitialContext.lookup(InitialContext.java:392)
> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
> Caused: java.io.IOException: Failed to retrieve RMIServer stub
> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
> java.net.ConnectException: Operation timed out
>
> [ and more ]
>
>
>
>
>
>
>
>
>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>
>> - Tim
>>
>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>
>>> Hi,
>>>
>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>
>>> Hence the question:
>>>
>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>
>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>
>>> Jan
>>>

Jan Algermissen

On Oct 3, 2012, at 9:40 PM, Tom Mueller wrote:

> This looks like what you get when using a Java SE 6 update earlier than 22.

You mean server side? There I did a fresh install today of

jdk-6u35-linux-x64-rpm.bin

Jan

>
> Tom
>
> On 10/3/2012 2:31 PM, Jan Algermissen wrote:
>> Hi Tim,
>>
>> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>>
>>> Hello, Jan.
>>>
>>> Please take a quick look at this recent blog entry:
>>>
>>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>>
>>> This does not count as "official documentation" but the steps there do work.
>>>
>> Thanks very much. Much cleaner than anything else so far.
>>
>> But no luck for me. The visualvm logs suggest some SSL related problems.
>>
>> Stacktrace below, in case you have a vague idea.
>>
>> Jan
>>
>>
>>
>> Trace:
>>
>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
>> java.io.EOFException: SSL peer shut down incorrectly
>> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
>> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
>> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
>> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
>> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
>> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
>> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
>> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
>> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
>> at javax.naming.InitialContext.lookup(InitialContext.java:392)
>> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
>> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
>> Caused: java.io.IOException: Failed to retrieve RMIServer stub
>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
>> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
>> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
>> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
>> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
>> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
>> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
>> java.net.ConnectException: Operation timed out
>>
>> [ and more ]
>>
>>
>>
>>
>>
>>
>>
>>
>>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>>
>>> - Tim
>>>
>>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>>
>>>> Hi,
>>>>
>>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>>
>>>> Hence the question:
>>>>
>>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>>
>>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>>
>>>> Jan
>>>>
>

tmueller
Offline
Joined: 2005-10-31
Points: 0

Yes, server side. That's not the problem then.
Tom

On 10/3/2012 2:44 PM, Jan Algermissen wrote:
> On Oct 3, 2012, at 9:40 PM, Tom Mueller wrote:
>
>> This looks like what you get when using a Java SE 6 update earlier than 22.
> You mean server side? There I did a fresh install today of
>
> jdk-6u35-linux-x64-rpm.bin
>
> Jan
>
>
>> Tom
>>
>> On 10/3/2012 2:31 PM, Jan Algermissen wrote:
>>> Hi Tim,
>>>
>>> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>>>
>>>> Hello, Jan.
>>>>
>>>> Please take a quick look at this recent blog entry:
>>>>
>>>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>>>
>>>> This does not count as "official documentation" but the steps there do work.
>>>>
>>> Thanks very much. Much cleaner than anything else so far.
>>>
>>> But no luck for me. The visualvm logs suggest some SSL related problems.
>>>
>>> Stacktrace below, in case you have a vague idea.
>>>
>>> Jan
>>>
>>>
>>>
>>> Trace:
>>>
>>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
>>> java.io.EOFException: SSL peer shut down incorrectly
>>> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
>>> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
>>> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
>>> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>>> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
>>> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
>>> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
>>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
>>> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
>>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
>>> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
>>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
>>> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
>>> at javax.naming.InitialContext.lookup(InitialContext.java:392)
>>> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
>>> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
>>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
>>> Caused: java.io.IOException: Failed to retrieve RMIServer stub
>>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
>>> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
>>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
>>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
>>> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
>>> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
>>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
>>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
>>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
>>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
>>> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
>>> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
>>> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
>>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
>>> java.net.ConnectException: Operation timed out
>>>
>>> [ and more ]
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>>>
>>>> - Tim
>>>>
>>>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>>>
>>>>> Hence the question:
>>>>>
>>>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>>>
>>>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>>>
>>>>> Jan
>>>>>

Jan Algermissen

On Oct 3, 2012, at 9:31 PM, Jan Algermissen wrote:

> Hi Tim,
>
> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>
>> Hello, Jan.
>>
>> Please take a quick look at this recent blog entry:
>>
>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>
>> This does not count as "official documentation" but the steps there do work.
>>
>
> Thanks very much. Much cleaner than anything else so far.
>
> But no luck for me. The visualvm logs suggest some SSL related problems.

Update: the certificate of GF seems to be the problem.

Jan

>
> Stacktrace below, in case you have a vague idea.
>
> Jan
>
>
>
> Trace:
>
> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
> java.io.EOFException: SSL peer shut down incorrectly
> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
> at javax.naming.InitialContext.lookup(InitialContext.java:392)
> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
> Caused: java.io.IOException: Failed to retrieve RMIServer stub
> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
> java.net.ConnectException: Operation timed out
>
> [ and more ]
>
>
>
>
>
>
>
>
>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>
>> - Tim
>>
>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>
>>> Hi,
>>>
>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>
>>> Hence the question:
>>>
>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>
>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>
>>> Jan
>>>
>>
>

Jan Algermissen

Hi Tim,

On Oct 3, 2012, at 11:12 PM, Tim Quinn wrote:

> Hi, again, Jan.
>
> I have just installed a very recent build of GlassFish 4 and I can successfully connect to it using jconsole and jvisualvm remotely (after changing the admin password, enabling secure admin, and restarting the domain).
>
> A couple of possibilities come to mind.
>
> First, the version of Java should be recent on both ends. In response to Tom's note I saw that you have a very recent Java installation on the server. What Java version is on your client?

Same as server's.

>
> Second, if as you conjectured, the cert containing localhost is part of the problem, you can try this:
>
> asadmin stop-domain
> asadmin delete-domain domain1
> asadmin create-domain domain1 #(explicitly enter "admin" for the username and respond to the password prompt with a non-empty password)
> asadmin start-domain
> asadmin enable-secure-admin
> asadmin restart-domain
>

Thanks - that is a great trick indeed. I have spend the last hours to dig through the keytool stuff with no real success in trying what 'brute force' does in seconds :-)

Very nice.

However...

> Then try again to connect to the DAS remotely using jvisualvm.

VisualVM still does not do it and e.g. Safari still chockes on the cert saying it has not been validated by a third party. Strange .. isn't the whole idea of self signing that no 3rd party is needed?

Strange that it works for you and not for me. I am on a fresh, hosted root server so there should not be any trouble, actually.

Thanks anyhow. I appreciate it very much!

Jan

>
> Creating the domain again, among other things, creates self-signed certs that have the DAS's actual host name built in (instead of localhost). If that was the problem then recreating the domain might help.
>
> - Tim
>
> On Oct 3, 2012, at 3:20 PM, Jan Algermissen wrote:
>
>>
>> On Oct 3, 2012, at 9:39 PM, Jan Algermissen wrote:
>>
>>>
>>> On Oct 3, 2012, at 9:31 PM, Jan Algermissen wrote:
>>>
>>>> Hi Tim,
>>>>
>>>> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>>>>
>>>>> Hello, Jan.
>>>>>
>>>>> Please take a quick look at this recent blog entry:
>>>>>
>>>>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>>>>
>>>>> This does not count as "official documentation" but the steps there do work.
>>>>>
>>>>
>>>> Thanks very much. Much cleaner than anything else so far.
>>>>
>>>> But no luck for me. The visualvm logs suggest some SSL related problems.
>>>
>>>
>>> Update: the certificate of GF seems to be the problem.
>>
>>
>> I guess the problem is that the cert is for localhost and that does not match the hostname I point my visualvm to.
>>
>> Question is now: how do I get a valid certificate installed. Help welcome.
>>
>> Meanwhile I plough through the blogs on that topic.
>>
>> (What makes me wonder is that what I am trying to do, namely: monitor a GF remotely with VisualVM, seems like a standard use case. But I feel like the only person having problems there. Strange)
>>
>> Thanks for all the help so far!
>>
>>
>> Jan
>>
>>
>>
>>
>>>
>>>
>>> Jan
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>> Stacktrace below, in case you have a vague idea.
>>>>
>>>> Jan
>>>>
>>>>
>>>>
>>>> Trace:
>>>>
>>>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
>>>> java.io.EOFException: SSL peer shut down incorrectly
>>>> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
>>>> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
>>>> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
>>>> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>>>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>>>> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>>>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
>>>> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>>>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>>>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
>>>> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
>>>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
>>>> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
>>>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
>>>> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>>>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
>>>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
>>>> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
>>>> at javax.naming.InitialContext.lookup(InitialContext.java:392)
>>>> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
>>>> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
>>>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
>>>> Caused: java.io.IOException: Failed to retrieve RMIServer stub
>>>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
>>>> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
>>>> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
>>>> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
>>>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
>>>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
>>>> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
>>>> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
>>>> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
>>>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
>>>> java.net.ConnectException: Operation timed out
>>>>
>>>> [ and more ]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>>>>
>>>>> - Tim
>>>>>
>>>>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>>>>
>>>>>> Hence the question:
>>>>>>
>>>>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>>>>
>>>>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>>>>
>>>>>> Jan
>>>>>>
>>>>>
>>>>
>>>
>>
>
> On Oct 3, 2012, at 3:20 PM, Jan Algermissen wrote:
>
>>
>> On Oct 3, 2012, at 9:39 PM, Jan Algermissen wrote:
>>
>>>
>>> On Oct 3, 2012, at 9:31 PM, Jan Algermissen wrote:
>>>
>>>> Hi Tim,
>>>>
>>>> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>>>>
>>>>> Hello, Jan.
>>>>>
>>>>> Please take a quick look at this recent blog entry:
>>>>>
>>>>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>>>>
>>>>> This does not count as "official documentation" but the steps there do work.
>>>>>
>>>>
>>>> Thanks very much. Much cleaner than anything else so far.
>>>>
>>>> But no luck for me. The visualvm logs suggest some SSL related problems.
>>>
>>>
>>> Update: the certificate of GF seems to be the problem.
>>
>>
>> I guess the problem is that the cert is for localhost and that does not match the hostname I point my visualvm to.
>>
>> Question is now: how do I get a valid certificate installed. Help welcome.
>>
>> Meanwhile I plough through the blogs on that topic.
>>
>> (What makes me wonder is that what I am trying to do, namely: monitor a GF remotely with VisualVM, seems like a standard use case. But I feel like the only person having problems there. Strange)
>>
>> Thanks for all the help so far!
>>
>>
>> Jan
>>
>>
>>
>>
>>>
>>>
>>> Jan
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>> Stacktrace below, in case you have a vague idea.
>>>>
>>>> Jan
>>>>
>>>>
>>>>
>>>> Trace:
>>>>
>>>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
>>>> java.io.EOFException: SSL peer shut down incorrectly
>>>> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
>>>> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
>>>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
>>>> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
>>>> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>>>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>>>> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>>>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
>>>> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>>>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>>>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
>>>> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
>>>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
>>>> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
>>>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
>>>> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>>>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
>>>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
>>>> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
>>>> at javax.naming.InitialContext.lookup(InitialContext.java:392)
>>>> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
>>>> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
>>>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
>>>> Caused: java.io.IOException: Failed to retrieve RMIServer stub
>>>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
>>>> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
>>>> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
>>>> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
>>>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
>>>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
>>>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
>>>> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
>>>> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
>>>> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
>>>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
>>>> java.net.ConnectException: Operation timed out
>>>>
>>>> [ and more ]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>>>>
>>>>> - Tim
>>>>>
>>>>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>>>>
>>>>>> Hence the question:
>>>>>>
>>>>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>>>>
>>>>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>>>>
>>>>>> Jan
>>>>>>
>>>>>
>>>>
>>>
>>
>

tjquinn
Offline
Joined: 2005-03-30
Points: 0

I'm sorry this is proving to be so difficult, Jan!

A few notes below.

On Oct 3, 2012, at 4:34 PM, Jan Algermissen wrote:

> Hi Tim,
>
> On Oct 3, 2012, at 11:12 PM, Tim Quinn wrote:
>
>> Hi, again, Jan.
>>
>> I have just installed a very recent build of GlassFish 4 and I can successfully connect to it using jconsole and jvisualvm remotely (after changing the admin password, enabling secure admin, and restarting the domain).
>>
>> A couple of possibilities come to mind.
>>
>> First, the version of Java should be recent on both ends. In response to Tom's note I saw that you have a very recent Java installation on the server. What Java version is on your client?
>
> Same as server's.
>
>>
>> Second, if as you conjectured, the cert containing localhost is part of the problem, you can try this:
>>
>> asadmin stop-domain
>> asadmin delete-domain domain1
>> asadmin create-domain domain1 #(explicitly enter "admin" for the username and respond to the password prompt with a non-empty password)
>> asadmin start-domain
>> asadmin enable-secure-admin
>> asadmin restart-domain
>>
>
> Thanks - that is a great trick indeed. I have spend the last hours to dig through the keytool stuff with no real success in trying what 'brute force' does in seconds :-)
>
> Very nice.
>
> However...
>
>
>> Then try again to connect to the DAS remotely using jvisualvm.
>
> VisualVM still does not do it and e.g. Safari still chockes on the cert saying it has not been validated by a third party. Strange .. isn't the whole idea of self signing that no 3rd party is needed?
You're right that no 3rd-party is needed with self-signed certs, but from the browser's point of view the site is basically vouching for itself, rather than some trusted authority vouching for the site. Any browser worth anything will warn you if the cert is self-signed and should give you the chance to trust it anyway. So I am not surprised Safari is complaining, but it should give you the choice to go to the site anyway.

Do you see the same problem if you try jconsole instead of jvisualvm? I don't know why this would make any difference, really, but if jconsole works but jvisualvm does not then that might indicate something.

Are there any errors logged in the DAS server.log file?

Maybe try launching jvisualvm by specifying

jvisualvm -J-Djavax.net.debug=ssl

or, if that doesn't seem to give helpful information, try

jvisualvm -J-Djavax.net.debug=all

From the errors you posted, it seems as if the server is deciding to abandon the connection, so you might also want to set up that same system property on the server side:

asadmin create-jvm-options "-Djavax.net.debug=ssl"

(or, if needed, replace "ssl" with "all").

- Tim

Jan Algermissen

On Oct 3, 2012, at 9:39 PM, Jan Algermissen wrote:

>
> On Oct 3, 2012, at 9:31 PM, Jan Algermissen wrote:
>
>> Hi Tim,
>>
>> On Oct 3, 2012, at 7:09 PM, Tim Quinn wrote:
>>
>>> Hello, Jan.
>>>
>>> Please take a quick look at this recent blog entry:
>>>
>>> https://blogs.oracle.com/quinn/entry/jmx_monitoring_of_glassfish_servers
>>>
>>> This does not count as "official documentation" but the steps there do work.
>>>
>>
>> Thanks very much. Much cleaner than anything else so far.
>>
>> But no luck for me. The visualvm logs suggest some SSL related problems.
>
>
> Update: the certificate of GF seems to be the problem.

I guess the problem is that the cert is for localhost and that does not match the hostname I point my visualvm to.

Question is now: how do I get a valid certificate installed. Help welcome.

Meanwhile I plough through the blogs on that topic.

(What makes me wonder is that what I am trying to do, namely: monitor a GF remotely with VisualVM, seems like a standard use case. But I feel like the only person having problems there. Strange)

Thanks for all the help so far!

Jan

>
>
> Jan
>
>
>
>
>
>
>>
>> Stacktrace below, in case you have a vague idea.
>>
>> Jan
>>
>>
>>
>> Trace:
>>
>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi://xxxxxxxxxx:8686/jndi/rmi://xxxxxxxxxxxx:8686/jmxrmi)
>> java.io.EOFException: SSL peer shut down incorrectly
>> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
>> Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:882)
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1203)
>> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:654)
>> at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
>> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
>> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
>> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
>> Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
>> at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
>> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
>> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
>> Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
>> javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
>> at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
>> at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
>> at javax.naming.InitialContext.lookup(InitialContext.java:392)
>> at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
>> at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:255)
>> Caused: java.io.IOException: Failed to retrieve RMIServer stub
>> at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
>> [catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
>> at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
>> at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
>> at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:279)
>> at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:183)
>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:293)
>> at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:270)
>> at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
>> at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
>> at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
>> INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(3574)
>> java.net.ConnectException: Operation timed out
>>
>> [ and more ]
>>
>>
>>
>>
>>
>>
>>
>>
>>> The formatting of some of the images there leaves a little to be desired, but I think it's clear enough what's going on.
>>>
>>> - Tim
>>>
>>> On Oct 3, 2012, at 11:38 AM, Jan Algermissen wrote:
>>>
>>>> Hi,
>>>>
>>>> despite the large number of Q&As on the subject out there, I feel pretty lost. Most of them talk about how people eventually, somehow managed to get remote JMX monitoring to work. I just cannot find an authoritative answer, how it *should* be done.
>>>>
>>>> Hence the question:
>>>>
>>>> Given a fresh install of Glassfish 4, what are the things I must do to enable remote monitoring via VisualVM? Is there any official documentation on the subject?
>>>>
>>>> (I am not going through a firewall, so there should not be an issue with the second port VisualVM opens up).
>>>>
>>>> Jan
>>>>
>>>
>>
>