Fixing Java Memory Tools on Mac OS X

A number of the Java memory management tools with the default distribution on Mac OS X Leopard are broken. The information to repair the situation seems to be hard to come by. This post will detail the steps necessary to get jmap and jhat to work on a Mac for Java 6. I assume that Java 5 has the same issue but haven't checked.


Setting up a test Java process to profile.
1. From the terminal: cd /Developer/Examples/Java/JFC/Java2D/
2. From the terminal: java -jar Java2D.jar &
3. After the process starts it should indicate the pid (in this example it is 5510)
4. You can also use jps to discover the pid: jps -l (then look for the Java2D.jar)

Fixing jmap
If you type jmap , you would expect to get a high-level memory dump... instead you get a message like:
attach: task_for_pid(5510) failed (5)
Error attaching to process: Error attaching to process, or no such process

To fix this, do a sudo, such as sudo jmap .
Debugger attached successfully.
Server compiler detected.
JVM version is 1.6.0_05-b13-52

using thread-local object allocation.
Mark Sweep Compact GC

Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 88080384 (84.0MB)
NewSize = 2686976 (2.5625MB)
MaxNewSize = -65536 (-0.0625MB)
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 6
PermSize = 21757952 (20.75MB)
MaxPermSize = 88080384 (84.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 10551296 (10.0625MB)
used = 4592152 (4.379417419433594MB)
free = 5959144 (5.683082580566406MB)
43.52216069002329% used

At this point it looks like it is working, however the file dump doesn't work.

Fixing jhat
When running jhat on the hump dump, you get the following error:
Eliminating duplicate references.....................
Snapshot resolved.
Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
at com.sun.tools.hat.internal.oql.OQLEngine.init(OQLEngine.java:277)
at com.sun.tools.hat.internal.oql.OQLEngine.(OQLEngine.java:51)
at com.sun.tools.hat.internal.server.QueryListener.setModel(QueryListener.java:59)
at com.sun.tools.hat.Main.main(Main.java:189)
Caused by: java.lang.NullPointerException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.tools.hat.internal.oql.OQLEngine.init(OQLEngine.java:256)
... 3 more

It took me a little hunting to uncover the issue. It turns out that jhat uses the Java 6 js engine. If you go to the terminal again and run: jrunscript , it indicates that the script engine for language js can not be found. On the mac, if you execute: jrunscript -q, which provides a listing of script engines you'll see:
Language AppleScript 2.0.1 implemention "AppleScriptEngine" 1.0
Thank you apple!

As a side note if you actually wanted to run AppleScript as a shell, type jrunscript -l AppleScript

Let's get to fixing the issue

1. Download JSR-223's engines https://scripting.dev.java.net/files/documents/4957/37593/jsr223-engines.zip
2. Download Rhino http://www.mozilla.org/rhino/download.html
3. Copy jsr223-engines/javascript/build/js-engine.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/ext/
4. Copy rhino1_7R1/js.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/ext/

** you have to be sudo to do steps 3 and 4.

now jrunscript -q should look like this:
Language EmbeddedECMAScript 1.6 implemention "Mozilla Rhino" 1.6 release 2
Language AppleScript 2.0.1 implemention "AppleScriptEngine" 1.0
Language ECMAScript 1.6 implemention "Mozilla Rhino" 1.6R7

... and jrunscript will put you in a js shell.
... and jhat heap.out now works! Point your browser at http://localhost:7000 and object browse!

Also something I didn't realize until I was preparing for this post... jmap worked for the console dump if I sudo'd the command. However it would not do a file dump. I did the file dump using jconsole and jmx. Now that the scripting issue is fixed, jmap is improved in the following manner:

1. if you type jmap , it now prompts you for your admin password (instead of just failing)
2. jmap -dump:live,format=b.file=heap.out 5510 now works

It makes me wonder what else wasn't working! Happy Coding!

Portal 2 JSR-268

JSR-286 has been signed off! Here comes Portal 2. I don't know of any implementations yet... so we are still waiting to play with it.

Portal 1 Challenges
There are a number of pain points of portal development which include:

1. Accessing resources... this is an issue with security mainly
   
2. Portlet to portlet communication through the server framework
3. Ajax communication.
   

Much of these issues have been addressed through the specification...

What's new in Portal 2
First, Portlet to portlet communication is not as cumbersome and isn't limited to some session hacking. The specification provides 2 mechanisms for communication at the server tier.

1. Events
2. Public Parameters

Events that are provided by an event producing portlet are described in it's XML file. Event consuming portlets describe what events they can consume, then at deploy time or run time the portlets are wired together through an event channel.


Second, resources are finally part of the picture. In the portal 1 days, you would often need to access a resource outside the portlet... well this request didn't have any portlet details, such as window state or portlet perferences and worse... it didn't run under the security of the portal.
Now resources are obtainable within the context of the portal. To a certain extent this provides us now with the ability to do ajax style development which wasn't possible before.

Next, there is the standard web page needs which have been addressed. For instance, because a portlet doesn't own the page, it really can't participate in provide header information... until now. There are multiple respond phases now... one for the header and one for markup. This provides the ability for a portlet to participate in the creation of the page header. Additionally access to cookies and other web oriented needs have been opened up.

On top of this their are portlet filters and portlet url listeners...

If it is not too late... the portal world is about to get a breath of fresh air. Take a look at the specification, it is worth a look if you are in this space. Perhaps when the reference implementation is out... and I have time I'll create a demo of a few things... of course I'll need to include some of the latest spring stuff in the mix. It looks like Spring 3.0 intends to have Portal 2.0 support. As I scan around on the web... it looks like liferay is portal 2 ready... maybe it is time to switch back :)

You don't even know you don't even know

My daughters and I have a phrase we've picked up from some where... probably a movie, but I don't remember which one, which is "you don't even know, you don't even know". We use it liberally to describe a situation where people are talking about stuff they clearly know nothing about, yet they are sharing their opinion about it anyway. It seems strange that this happens... and frankly it happens all the time.

So while speaking in Denver between sessions I attended a great session by Jared Richardson. We got into a discussion on developer perceptions on agile practices such as pair programming, etc. When it occurred to me that this was it... so often developers dislike pairing when they haven't practiced it. They claim they don't have time to write unit test or integration tests... but they haven't attempted it.

It appears that Agile in some circles is getting a bad name... but most of the time it is based on the opinions of someone who thinks they know what the outcome will be... they have thought it through in their head and have come to a conclusion, which has become their opinion on the matter. Sometimes this is further justified through their comments "well I heard that so and so had issues... blah...blah". Sometimes it comes in the early onslaught of pain... either it is new, or their some affect of team storming. The reality is that often new agile teams run a little slower upfront, but end up significantly ahead of non-agile teams. The pain is worth it! (at least from my experience)

So as a community let's make an agreement. If you ain't done it, you can have an opinion... but it doesn't count :) If you think it won't work or if you heard it won't work... you don't even know what you don't even know!

Knowing comes through experience! and as GI Joe says "Knowing is half the battle".

Speaking at OSCON

Well I will be speak this year at OSCON on July 25, 2008 speaking on Spring 2.5 . It will be fun to see how I take my 1:30 hour talk which I usually run over time on and deliver it in :45 mins.

Welcome to Viet Nam! in Bangalore

I mentioned earlier that I would blog on this subject from my India trip. So I had a very unusual situation in Bangalore. One of the attendees (Madhu) who attended the Jax conference requested that I visit his company during my stay in Bangalore. This became difficult based on a number of reasons, which resulted in Madhu and 10 of his co-workers coming to my hotel. The lobby was too noisy, so we ended up upstairs in my hotel room. The picture below is just half of us that are sitting around a bed talking tech.


We discussed a number of technical details... it started with a discussion around ORM tools and the problem they had mapping to a complex and highly normalized data store. They were creating a web admin tool which for simplicity sake I'll characterize as a content management system (CMS). In this space they don't know all the "columns" of data to store. The columns or metadata is stored in a row of a table, then there is an association table which associates the column to a 3rd table, the table of values. They didn't have to get to far in the description for me to know exactly what they were talking about. I have seen it before a number of times... it is the kind of thing a really out of touch architect, or a green DBA would suggest. I shared some horror stories of my past to try to sway them from this approach. I then discussed all the negatives to this approach.. such as:

1. No indexing
2. No join capabilities
3. Very complex queries
4. Did I mention no indices? This data structure just isn't tunable.

The conversation led to putting an ORM tool on top of this :) If anyone knows of ORM tool that does this kind of mapping I would love to know. I explained that I was unaware of a ORM that would handle this situation.

While I was speaking at JAX on the topic of Spring and JPA, an attendee reflected on Ted Newark's article relating ORM to Viet Nam. I was amazed!! People around the world are reading Ted's Blog...

So as we were sitting around the bed discussing alternatives, I mention Ted's article which I just re-read that week ... of course I had to explain a little history regarding the meaning that Viet Nam has for most Americans. We got to the point were I was describing how a project gets so far done the road and the team has invested so much, that when anyone suggests that we are tapped out and need to rethink the projects technical approach management comes in with statements like "Come on guys, we have so much invested. Can't you just get it to work". At this point they were all smiling, looking at each other and someone spoke up and says that is exactly where we are at... to which I said "Welcome to Viet Nam" :)

Speaking at OSCON 2008

I will be speaking on Spring 2.5 at OSCON 2008 this year! The next tid bit of fun will be fitting my 1 1/2 hour talk into a :45 minute talk... should be fun.

Speaking in India at JAX

Just finished speaking in Bangalore at the JAX conference. The experience was very unique. I'll hit some highlights on a later post. There are 2 remember experiences from the trip. First is the driving there. A friend of mine from a previous trip capture what I'm talking about. Second, my last day I had 10 indians from a local company sitting around my hotel bed talking tech... the statement which capped the conversation was "welcome to vietnam"... I'll explain later.