Posted by mlam
on September 21, 2007 at 7:37 PM PDT
Some new features have been added to the CVM's VM Inspector. This entry will give you a quick update on this.
You may or may not have noticed on the phoneME Advanced Downloads page , that there is a phoneME Advanced MR2 binary for WinCE / Windows Mobile 5. That's one of the projects that my esteemed colleagues and I have been busy working on in the past few months. That is part of the reason I have not been able to post much. So, I apologize for that.
Well, after being spoiled with all the rich debugging features available in gdb on Linux, working with WinCE has been ... ummm ... challenging. In the initial phases of bringing up the VM, any number of things could have gone wrong. Without adequate debugging capability, it is hard to figure out what has gone wrong.
One of the common things that can occur when the system isn't stable yet is a hang. When that happens, you really need to get hold of the thread stack traces in order to figure out where the hang is happening. Maybe there's a way to do this on WinCE (and VS2005 i.e. Visual Studio 2005) and I'm just ignorant about it, but what I found was that when the threads are all hung, I can't just force a break on all threads whenever I like in VS2005. Well, I can ... but I don't seem to be getting any thread stack info. What I heard was that when a thread is blocked inside some WinCE API, then VS2005 won't be able to give us a stack trace. Since hangs tend to be cases where the threads are all blocked in locks of some kind (i.e. using WinCE APIs), I was out of luck trying to get stack trace information.
But all is not lost. There's the VM Inspector (and cvmsh) and the thread dump hack for CVM Java threads that I'd introduced to you previously. However, I can't just use them as is yet. So, I added a little bit of enhancements ...
the CVMSH Server
On WinCE, there is no default console application that will allow me to use cvmsh in its previous form. So, I simply looked up an example of some client-server Java examples, and modified cvmsh to support a server feature. I'll admit the code in there isn't the most elegant stuff I have ever written, but it works. As of rev 7328 in the phoneME Advanced repository, cvmsh now supports a server option.
The client-server feature allows me to use a remote client as the source of command input to cvmsh. Hence, I no longer need to rely on a local command shell. There is now a cvmclient application that you can use to connect to cvmsh's server.
Enough talk. Let's get into the useful stuff ...
How do I use it?
Build the Inspector
As explained before , build CVM with CVM_INSPECTOR=true. This can work with non-debug builds (CVM_DEBUG=false) as well, but by default, the inspector is enabled for debug builds (CVM_DEBUG=true).
Run with the Server
At your command line, run CVM as follows:
> cvm -cp testclasses.zip cvmsh --X "startServer"
--X option says that the next set of arguments are cvmsh commands (like those that I've wrote about previously ). The
startServer is a new command that starts a background server thread.
Normally, you would want to run your application. Let's say this is how you would normally run your application:
> cvm -cp yourJar.jar yourApp 1 2 3 yourArgs
To debug your app with cvmsh in server mode, this is how you would run it:
> cvm -cp testclasses.zip:yourJar.jar cvmsh \
--X "startServer" yourApp 1 2 3 yourArgs
This launches cvmsh's server, and then have cvmsh launch your application.
Another new command that I added to cvmsh is the
verbose which tells cvmsh to be more talkative and tell you more personal stuff as it does its thing. Here's how you can give it a try:
> cvm -cp testclasses.zip:yourJar.jar cvmsh \
--X "verbose;startServer" yourApp 1 2 3 yourArgs
Note that all I did was add it to the cvmsh command list with a semicolon (;) as the delimiter between each of the cvmsh commands (in this case,
startServer). In the same way, you can add as many cvmsh commands (along with their arguments) as you like. Give it a try. Go nuts! Here's an example of me going nuts on it:
> cvm -cp testclasses.zip:yourJar.jar cvmsh \
--X "verbose; captureHeapState initialState; gc" \
--X "disableGC;enableGC; port=2000;" \
--X ";startServer" yourApp 1 2 3 yourArgs
I had several
Sometimes I have spaces between the cvmsh commands and sometimes, I don't.
I sometimes have an extra ';' at the end of one command list, and I have an extra one at the start of another.
I sometimes have more than one command in a command list, and in another list, I have only one command.
In one case (
captureHeapState), the command is followed by an argument before the next command. Note that the next command comes after the ';' delimiter.
It's all good. cvmsh reads and executes those commands from left to right. You can put them all in one
--X command list or several as you like. Extra ' ' and ';' delimiters are just ignored by cvmsh. As I said, you can issue any sequence of cvmsh commands that make sense to you. It's just like typing them in on at the cvmsh shell prompt, except that they are all typed out in a command list now delimited (i.e. separated) by ';'s.
Connect to the Server
In your remote client machine, run cvmclient like so:
> java -cp testclasses.zip cvmclient -host 220.127.116.11 -port=2000
This will bring up a prompt just like the cvmsh prompt. If you don't specify the host, the localhost will be used. If you don't specify the port, then port 4321 will be used. 4321 is also the default socket port that cvmsh's server will wait on if you didn't specify the
port=2000 command earlier that changed the port to 2000 instead.
And now, you are ready to use cvmsh just like you did before. But wait a minute ... it's not quite the same. cvmclient only acts as an input mechanism for cvmsh. All the output for cvmsh still goes to stdout/err on the device where cvmsh is running on.
Note also that in the command line for running cvmclient, I used JavaSE's
java instead of CVM. This is intentional. If you have a CVM port for the remote machine where you run cvmclient, then you can run it using CVM too. But chances are, your remote machine is a desktop machine and not a JavaME device. Hence, you're more likely to have JavaSE installed but not a CVM port there. Hence, cvmclient is designed to not use any CVM proprietary APIs for that reason ... so that it can be run using JavaSE. cvmsh, on the other hand, only runs on CVM. This is, of course, because it needs to access proprietary APIs in order to get you the data about what's going on inside CVM.
The cvmsh server feature works with Foundation Profile only and not plain CDC. This is because it uses
java.net.ServerSocket which isn't included in plain CDC.
When I started this blog entry, I said that I wanted a Java thread dump on WinCE. I haven't gotten there yet. To get there, I need to add a thread dump command to cvmsh. I also added a few other things while I was going at it. Here are the new commands:
list all live CVM threads
dump stacks for all live CVM threads
dump the stack of the specified thread
dump some system configuration info
e.g. the sizes and boundaries of the GC heap,
the JIT configurations.
enable/disable verbose output e.g. error messages.
specifies the port number of the socket connection to use.
starts the cvmsh server which waits on the socket.
stops the cvmsh server which waits on the socket.
Give them a try. In cvmsh or cvmclient, type
help to get a list of all the commands and their usage info.
Note that with these new features of cvmsh, we will no longer need the thread dump hack that I talked about previously. One of the reasons that I had to enhance cvmsh to support a server mode was because I can't send signals to WinCE process to trigger my stack dumps. Now, I can request it via cvmsh instead. By the way, I've added more useful info in the thread dumps than was available based on the code I posted previously. So, check it out.
In case you didn't read my previous introduction of cvmsh and the VM Inspector , I would like to remind you that cvmsh is intended as a poor man's debugger/profiler. I intend to continue to add functionality to it over time as I encounter a need ... particularly debugging features that involve inspecting the internals of the VM. However, I don't intend to make it into a full-fledged debugger/profiler. That's what NetBeans and JVMTI are for.
So, if you have your favorite feature that you would like me to add (or that you think is useful but cannot be gotten in other ways), please let me know. Alternatively, cvmsh is a fun small sized project that anyone can work on with some effort. Working on it will also expose you to some hands on CVM internals. So, if you are looking for a good way to get motivated about learning the internals of CVM (aka the phoneME Advanced VM), then consider signing up and contributing to the development of cvmsh.
As I find time, I intend to add more features, and I will also try to write a series of blogs of how to do useful stuff using cvmsh. An example of useful stuff would be like: how to find out why my object is not getting GC'ed? Or how to find out where the deadlock is in my Java code? :-) The first one can be done with cvmsh today, but the second one can't yet. But with a bit more enhancements, we should be able to do that too.
Well, I hope this has been helpful. Have a nice day. :-)