Thursday, December 31, 2009

3 Core Principles from 1998

I was off for the holidays which gave me some time to clean out the storage area. I ran across some notes from a conference I attended in 1998 and 3 core principles stood out that I thought I would share as we start this new year.

Core Principles (as I wrote them many years ago):
1. Smaller is better than larger
2. Understood is better than unknown
3. Progress is better than promises

Smaller is better than larger
There was nothing else written and I left nothing by way of context... regardless of what it meant then, it is clear that this is a great principle when it comes to code. Less code that does the same amount of work is better. The paradox is that it may take a little more time to develop a smaller code solution, but it pays off. The skill is in not being so abstract and so small as to not be readable or maintainable. It is this balance that makes a true software craftsman.

Understood is better than unknown
If there is one word that we would use to describe the issues of software development... it is the word "unknown". Estimating the known is easy. Estimating the unknown, is unknowable. The skill in software development is to separate the unknowns from the knows. Estimate the knowns and provide a SWAG estimate for the unknowns... these are estimates that you need to keep in check. It is best to delay the estimates and work of the unknowns until ( what Kevlin Henney brilliantly describes as) the last responsible moment. Of course this only works for the known unknowns... what will get you is the unknown unknowns:)

Progress is better than promises
This is why I have been a practitioner and trainer for XP and agile practices for many years. It is all about developing business value on a regular and iterative basis. Promises are meaningless... progress is all that matters.

Happy New Year!!

Monday, December 14, 2009

Intellij 9 and Gradle

One of the hidden gems of the Intellij 9 release is it's support for Gradle. Some of the information on the web is out of date and some features are not intuitive. This post will detail some of the nuances and follow it up with a wish list for the next update :)

Gradle Support
The information from JetBrains is out of date. In order to setup gradle after the GA release, all you need to do is configure the gradle home under File -> Other Settings -> Template Setttings.

Gradle Intellij Nuances
After setting up IDEA to work with Gradle, you'll need a build file. I didn't discover a way to create a gradle build file from IDEA. Just create a new file and name it "build.gradle" in the root of the module directory structure.

When writing the build script, Intellij expects to work with gradle in a way that is depreciated. It will pick up on a createTask(..) method. If the cursor is on a task such as this, and you hit ctrl+shft+F10, it will auto-discover and run that task in the script. (cool stuff)

Hotkeys to know:
ctrl+shft+F10 - runs task the cursor is on
shft+F10 - runs the last script (or the last run)
alt+shft+F10 - pops up a run menu (with all previously run gradle run configs)

The figure below shows an alt+shft+F10 when the cursor is on a task called task2.

If you code gradle in a non-depreciated way... then IDEA does not auto-discover the tasks. Below is what one would expect in a gradle build file:

usePlugin 'java'

repositories {

// from the gradle user guide
task hello << {
println 'Hello world'
task intro(dependsOn: hello) << {
println "I'm Gradle"

Tasks would normally be created this way... It is still great that IDEA has gradle support in general regardless of this oversight. After a run configuration is configured with a targeted task, it worlds with no pain. Also you could add:
 defaultTasks "intro"
and gradle runs this as the fallback when IDEA doesn't provide the required task.

Intellij / Gradle features I'm looking forward to:
  1. IDEA understands proper task configuration
  2. IDEA makes it easy to create new projects with a structure maven and gradle expects.
  3. IDEA treats gradle as a peer with Maven and Ant. This includes:
    a. Gradle Window (similar to the ant build and maven projects panels)
    b. Before Launch Gradle tasks in the run configurations

Saturday, December 5, 2009

More Trouble with Java and Apple

Well the latest update for Java from Apple came through recently. Destroying all in its path...
If you followed my advice in the past on getting Java 1.5 working on a Snow Leopard, then the new Apple update destroys that with the follow error for a Java 5 java -version:
Error occurred during initialization of VM
Unable to load native library: libjava.jnilib
Abort trap

I didn't track down the exact issue... however it is easy to detect that the Java update converts the symbolic links for Java 1.5 to point back to Java 6... Apparently Apple is NOT sorry for their crazy choice of ignorantly doing this in the first place. The solution is to completely go through the process out lined at OneSwarm again and re-establish the symbolic links. This will require you to delete the symbolic links that point to "CurrentJDK" for 1.4, 1.4.2, 1.5, and 1.5.0.

After edits an ls -l in the /System/Library/Frameworks/JavaVM.framework/Versions should look like:

drwxr-xr-x 15 root wheel 510 Dec 5 22:18 .
drwxr-xr-x 12 root wheel 408 Dec 5 22:15 ..
lrwxr-xr-x 1 root wheel 5 Dec 5 21:35 1.3 -> 1.3.1
drwxr-xr-x 3 root wheel 102 Jul 20 18:35 1.3.1
lrwxr-xr-x 1 root wheel 5 Dec 5 22:18 1.4 -> 1.4.2
lrwxr-xr-x 1 root wheel 14 Dec 5 22:18 1.4.2 -> 1.4.2-leopard/
drwxr-xr-x@ 9 root wheel 306 Feb 12 2009 1.4.2-leopard
lrwxr-xr-x 1 root wheel 5 Dec 5 22:01 1.5 -> 1.5.0
lrwxr-xr-x 1 root wheel 14 Dec 5 22:00 1.5.0 -> 1.5.0-leopard/
drwxr-xr-x@ 10 root wheel 340 Dec 5 21:59 1.5.0-leopard
lrwxr-xr-x 1 root wheel 5 Dec 5 21:35 1.6 -> 1.6.0
drwxr-xr-x 8 root wheel 272 Nov 8 14:35 1.6.0
drwxr-xr-x 9 root wheel 306 Dec 5 21:35 A
lrwxr-xr-x 1 root wheel 1 Dec 5 21:35 Current -> A
lrwxr-xr-x 1 root wheel 3 Dec 5 21:35 CurrentJDK -> 1.6

Happy coding!

Friday, November 27, 2009

Syntax Highlighting on Blogspot

I've been very interested in providing some syntax highlighting in my blog posts, which I've been slow in general getting out... ( I have 6 or so queued up ). I finally found something I like and thought I would share the love. Special thanks to @ecounysis on twitter from whom I made this discovery. There are no excuses for my future posts... they will all have nice highlighting and be more easily copied (via the clipboard feature).

The place to start is:

From there you'll see the link to:

One important thing to note for those on blogger / blogspot. The initial update of the head tag didn't take for me. The solution for me was to add the highlighting entries at the tail end of head, just prior to the closing tag.

In addition to many of the bundled options... I've added F# and Objective-C from undermyhat.

Thursday, September 3, 2009

Fixing Java on Mac Snow Leopard

I have long documented some of the issues and differences for a Java developer on the Mac OS X. Most of that has been on Leopard, OS X 10.5. For quick links the most significant articles are: Java is a 2nd Class Citizen on MAC OSX, Fixing Java Memory Tools on Mac OS X and Java 7 on Mac OS X.

Monday I was faced with a new situation, Snow Leopard OS X 10.6. There are several issues with Java on the new Mac OSX 10.6 "Snow Leopard". This post will outline the known issues and will provide fixes for several of them (or provide links where fixes are already explained).

Known Issues:
  1. Java 5 is not installed
  2. Java 5 symbolic links are set to Java 6
  3. jmap - some aspects of jmap do not function
  4. javascript runtime is still missing
As a consequence, applications that used Java 5 either don't work, or provide a message as shown here from my favorite stock trading application.

* I'm still looking for more issues, so if you can add to the list, please add comments, email or twitter.
** I have tested the latest BTrace and it works fine.

Java 5 on Snow Leopard
Whither you agree with the removal of Java 5 or not, it is unbelievable to me that the Java 5 symbolic links are pointing to the CurrentJDK which points to 1.6. For those new to this space the Java version live under /System/Library/Frameworks/JavaVM.framework/Versions . I would advise you to either:
  1. replace Java 5 using instructions provided at leopard
  2. simply delete the java 5 symbolic links
As a side note, I use a script for swapping between JVM versions. I modified the original script to just include 3 important functions when sourced in. I'll add the script to the bottom of this post as a reference. The 3 functions are jvms, setJava, unsetJava.
  • jvms reads through the version directory and reports what it reads of the file system.
  • setJava allows you to type in the name of the version and switches JAVA_HOME and path to this version of the JVM.
  • unsetJava reverts the set to the original java version
JMap on Snow Leopard
My initial execution of jmap on snow leopard failed with the following error message:

kensipe$ jmap 3407
Attaching to process ID 3407, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "heapOopSize" in any of the known library names (-)
at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMIntConstants(
at sun.jvm.hotspot.HotSpotTypeDataBase.(
at sun.jvm.hotspot.MacOSXTypeDataBase.(
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
at java.lang.reflect.Method.invoke(
Debugger attached successfully.

The good news is that the debugger attached successfully :)

Information regarding this issue is scarce. It turns out to likely be an issue with the version of Java that is distributed with snow leopard. The JDK 1.6 rev 14 appears (based on non-confirmed web sources) to have this issue. The latest is JDK 1.6 rev 16 should have the fix, however using the apple distro, we are left waiting for Apple for the fix. Although jmap fails with no arguments it does succeed for certain commands such as jmap -histo 3407.
After replacing the JDK 5 with the leopard distro as advertised above and switching to this version with the script above, jmap works fine. However jdk 5 jmap only works against jdk 5 running applications. It fails against java 6.

As record in a posting from a year ago, called Fixing Java Memory Tools on Mac OS X, the javascript library is still not provided by Apple as part of the Java distro. You will need to following the instructions there to fix it. Unfortunately the Rhino download has disappeared from the Mozilla site, so if you didn't keep a copy, ping me and I will send you want I have. You will also need to duplicate this for each version of Java if you install the Java 5 Leopard package.
The test to see if this is setup correctly for you is: jrunscript -q
If that command only returns AppleScript, then you are missing the javascript engine which is need for a number of tools in the JDK.

After these changes, it appears that Snow Leopard is ready for some hard core Java development. Good Luck and Happy Coding!

Java Switching Script that I use:

# *************************************************************************
# Originally written by Shawn Erickson - shawn at
# Last Update: 2008y 08m 8d
# The following functions are meant for use with bash shell which is currently the
# default on Mac OS X 10.4 (starting with 10.3 IIRC) unless otherwise configured.
# Good info on why not sym links:
# from:
# *************************************************************************

### Prompt ###


### Java Environment Functions ###


function availableJVMs()
ls -1 $J_VERSIONS_DIRECTORY | grep ^[0-9].[0-9]

function jvms()
echo "Available JVMs: "$jvms
ls -1 $J_VERSIONS_DIRECTORY | grep ^[0-9].[0-9] # look to remove redundant call
echo " "

echo "Current Java:"
java -version

function setJava()
local target_jvm=""
local jvms=$(availableJVMs)

# Validate that the user requested an available JVM present on the system

for jvm in $jvms ; do
if [ "$jvm" == "$@" ]; then

if [ "$target_jvm" == "" ]; then
echo "Unsupported Java version requested"

# If we get here the user asked for a valid JVM, so configure it

echo "Configuring Shell Environment for Java "$@
export JAVA_VERSION=$@

# First unset any current set java, back to default before doing configuration

# Generate the paths needed for the JVM requested

# We save the original path so we can toggle back if unset

# We save the original JAVA_HOME so we can toggle back if unset

# Update command prompt mode tag to note JVM setting

echo "Current Java:"
java -version

function _unsetJava()
if [ "$CURRENT_MODE_STRING" != "" ]; then

function unsetJava()
echo "Configuring Shell Environment for default Java"

echo "Current Java:"
java -version

Wednesday, August 19, 2009

97 Things Every Project Manager Should Know

The book 97 Things Every Project Manager Should Know put together by Barbee Davis is out, and worth picking up... why the magical 97? Apparently 101 was way over used... or perhaps Barbee couldn't get the last 4 in before it was time to publish :)

This book has some great insights from a number of sources, which is a huge value add to any project manager or senior developer. This is the top advice from experts in the field around team building, running a software project, and generally how to herd cats.

Most books in this space are limited to the experience of the author and his/her circle of influence. The beautiful part of this book, is the fact that Barbee has captured the most interesting and important tid bits stories of success and failures of software projects from a large sample of project managers and developers each providing a view from different industry verticals.

Well done Barbee!

Monday, July 20, 2009

Speaking at JavaZone in Oslo

It's official... I will be heading to Oslo, Norway in September for JavaZone. I will be presenting Debugging your production JVM, which is talk largely focused on BTrace. Also if you didn't realize it yet... BTrace has moved to the kenai site:

Monday, June 22, 2009

If you can't be a good speaker, be a groovy speaker

I will be speaking on groovy and grails at the up coming 2GX conference in New Orleans in October. If you are in the groovy space or you are just looking, you don't want to miss this... all the big names will be there. Most of the leading authors and committers in the groovy, grails and griffon space will be there!

How I got to be among them is beyond me... but it sure is groovy!

My topics include security, where the focus will include defense against hacking techniques such as injection flaws and XSS, followed by a walk through of the security plugin and how to secure urls and services. The second session is on Java memory management, the memory demands of a dynamic language, along with tools on debugging Grails applications in production.

Thursday, June 11, 2009

Personal Career Guiding Principals

I have received a number of requests from team members and session attendees for career path guidance. Here is what I threw together... I would enjoy others comments on what makes a difference for them!

My Guiding Principles

1. Become an expert at what is coming...
I've been lucky to live in an area of the country which is conservative when it comes to technology, which can at times be very dissatisfying. I travel to the coast (usually west) at least once per year for a technical conference. Through observation and networking, I learn the passions of the technical industry (not the vendor hype... beware the vendor hype). I make time to become well versed in these spaces. Within 1 year clients are interested and are looking for advice, within 2 they are looking for consultants. This is absolutely necessary to be a good architect anyway.
Suggestion: No one should work in our business without reading the mythical man month. Also read the pragmatic programmer or the quick guide if not the book:

2. Have the final say on matters that affect you...
Each of us is responsible for ourselves. If a company says no to training, or a conference or... that doesn't mean no to me... it means that this activity will not be subsidized. The next question to me is it worth the full obligation. There are plenty of training courses I've not only paid for, but took vacation time to be at. I currently have a nice large LCD screen, a wireless headset for my office phone and I don't use the corporate PC, I bought a MacBook Pro... all of which I paid for. I am generally unwilling to compromise working with the best tools. It shocks me in fact that Chefs bring their own cutlery, mechanics bring their own tools and software developers have better machines at home then they use at work, with an expectation that all their tools are provided for them by some company. In addition to the MacBook, I have invested significant dollars in software. Although I like open source, in many cases it is not the best. So I have TextMate, Intellij, MS Office, Keynote, etc. This wasn't as easy starting out, now frankly some vendors send me licenses now so that I'm showing their tool when I'm demoing. It is important to note that I do not accept licenses even for free if I can't stand behind the tool.
Suggestion: Invest in you! Be a master craftsman. Read:

3. Seek out Mentors and Network...
Find people who are great at what they do in an area of interest and seek them out, read their books, articles and blogs. Email them. Network with them. Obviously don't annoy them. Perhaps buy them lunch. Challenge them (for instance, "you said xyz, why would you say that?" or "why wouldn't you say yyy?"). Better yet, see what they are thinking about, what they are working on and what they might need help with. Help them! Pass them some work (if they have time), and ask their opinion.
Suggestion: Go to conferences and network.

4. Build your brand...
Create a brand for yourself. Be the Java Memory guy, the F# guy. Dive deep and market yourself. This means; a) find jobs or opportunities to work in this space, b) blog in this space, c) provide presentations in this space d) write articles or a book in this space. Your digital footprint is very important and it takes time to build out. As an example, if you search my name, the first several pages are me. This is significant. The goal here is to in a position to be hired for a gig without a resume. Those asking for you by name don't need a resume, they know what you represent. Reputation is everything! The great thing about a personal brand is it is more reasonable to change than a business brand... so 2 years as the czar of java, followed by...
Suggestion: Read Groundswell.

5. Know who you are and what makes you special...
This takes time for most people. You have to know what your talents are and what you are not good at. When you know what you lack, find teams / team members to compensate. For instance, I am a knowledge sponge and learn very quickly, but I get bored quickly. I am not as good working with a plain piece of paper, but I can perfect and analyze the dickens out of any architecture, system or code put in front of me. Take some personality tests and understand your personality. It's worth it! In the process of learning you.
Suggestion: Read Brain Rules.

6. Learn and consistently work on soft skills
Don't be the average developer... It is ok to be a geek, but be an alpha geek! Most developers will have one of two paths; PM or architect. Either way, the skills are completely different from what made them great as a developer. In the end, developing software is about enabling the business to a) make money or b) save money. Learn their language.
Suggestion: Read How to win friends and influence people and seven habits of highly effective people.

7. Be a Leader!
Leadership is earned... it isn't a title. For a technical person, It consists of having deep technical skills in at least 1 area combined with good morals. People follow people who know what the heck they are talking about and they trust. Knowledge without morals and people don't trust you. Morals without knowledge and you are a really nice guy. It has to be both.

8. Don't be Afraid:
- to say no (especially to things that don't align with you, know what you want)
- to have an opinion or stance on a subject (as soon as you do someone will shoot arrows at you)
- to change your opinion when the evidence dictates... but not based on pressure or opinion
- to make a decision (even in the absence of information)
- to be criticized
Suggestion: Read Who moved my Cheese.

9. Eat like a Bird and poop like an elephant
This is a wonderful phrase by Guy Kawasaki. Birds eat half their weight in food per day... Elephats poop... well they poop a lot. The idea is to read, listen, consume knowledge... then poop... I mean share that knowledge with anyone and everyone. I have a large library (which I read), and an account with Audible and Audio-Tech Book Summaries.
Suggestion: Read made to stick, and rules for revolutionaries.

10. Passion!
Have some... find it! There is nothing more convincing and more contagious than someone with passion. Change you, then change the world!

11. Have fun...
Life is too short

Tuesday, June 9, 2009

BTrace and JStat

I just finished a JavaOne presentation on Debugging your Production JVM. The killer part and climax of the presentation was on BTrace. BTrace just rocks! As good as it is, the documentation and javadoc information is somewhat dated. This post will explain one of my favorite tool sets working together; that being jstat and Btrace.

Jstat is a tool that has been provided in the jdk bin directory since Java 5. It has a number features, which can be examined by using the flag -options:
jstat -options
The most significant in my opinion being jstat -gcutils which provides an output of jvm memory by compartment; survivor spaces, eden, old and perm space. Here are the steps if this is new to you:
1. start an demo application: java -jar Java2D.jar
2. get the pid: jps
3. lauch jstat: jstat -gcutil 1483

0.00 0.00 10.61 59.98 76.81 8 0.057 9 0.460 0.516
Btrace is a tool that allows you to inject probes into a running java process to observe and debug an application. I won't go into "how" to use BTrace, as I've blogged on it in the past and the BTrace document does a good job of explaining it.

The area that is lacking is around the @Export annotation for BTrace scripts. There are no details in the documentation and the sample code comments says "// create a jvmstat counter using @Export"... yeah... right... Even the internal code comments and JavaDocs are just as vague. So if you are lost at this point, jstat use to be called jvmstat and was provided as part of the distribution of jvmstat. Even then there is no information on how to leverage the two... until now!!

Creating the Script
Take a look at the file in the sample directory of BTrace. Any value you want to be exposed to jstat will need to be annotated with @Export. Then you will need to assign a value to this exported value with a static method from BtraceUtils. In the ThreadCounter example this is accomplished with perfLong(""); The next step is to inject this code into a targeted JVM; such as:
btrace 1483 ../samples/ 

Accessing BTrace exports with jstat
The undocumented trick for jstat is that you have to specify -J-Djstat.showUnsupported=true and -name with the name of the exported variable defined by the perf statement. Here is the full command-line:
jstat -J-Djstat.showUnsupported=true -name 1483

Thanks to Sundar at Sun for the enlightenment!

Sunday, May 3, 2009

Closure: I Don't Think That Word Means What You Think It Means

It is curious that my weakest subject in school was English, and that I struggle so much with the miss use of certain words. Words really do matter. As words are misused, it starts to become commonplace which through a domino effect results in the complete lose of their meaning.

The word most in jeopardy today in the domain of programming is Closure.

WikiPedia has a good definition which states: a closure is a first-class function with free variables. The critical phrase is: free variables. I realize how subtle this is... however without the free variables it is just another function... and IMO it doesn't have to be a function that is just the common mechanism. Adding to the list of good examples is the docs on closures in Javascript.

Examples that are wrong or vague are commonplace. First Example: Groovy, which defines it merely as a code block... or a second example, how about Zdeněk Troníček's blog support Closures in Java which defines it as an anonymous function. A Google search will provide a fairly equal number of good and bad examples.

Defining Closure
The point to be made here is that if you pull the "free variables" out of the definition of a closure than all you have an anonymous function and the two words would be synonymous making one redundant to the other. Put another way... Closures are typically anonymous methods, but not all anonymous methods are closures.

Understanding Variable Scope
When defining a variable, our typical scopes are global, part of a class, parameter to a method, or local to a method. This would be complex if we tried to discuss this for all languages, so I will focus on Java. In Java there is no global scope, this is typically handled as a static member of a class. In this case, it lives in memory in a space called perm space. If the variable is part of a class, then the instance is part of heap memory for an instance of that class. If it is a passed parameter or a local member variable, then the storage is part of the stack frame pushed on the stack. As a reminder, as a method returns the stack frame for that method is popped off the stack and the scope has ended for local variables, references, etc. What is interesting about the proposition of a closure, is we want to be able to create a method (method 1) with some "variables" in the scope of another method (method 2). After the return of method 2, which means everything is out of scope, we still want to be able to pass method 1 to other parts of the program with the state of local variables maintaining state. Perhaps you can understand the issue now. If the scope is lost, where and how is the state of the anonymous method 1 being maintained? This is what closure is, and it is up to the language, compiler or run-time to figure that out.

I prefer to consider closures as another level of scope, as oppose to just another function.

Functional Example
It is easier IMO to see this with a functional language, so here is an example:
Function powerFunctionFactory (int power) {
int pwrFunction(int base) {
return pow(base, power);

return pwrFunction;

Function sqr = powerFunctionFactory (2);
Function cube = powerFunctionFactory (3));

sqr (3);
Looking at this example when the function sqr is defined by invoking powerFunctionFactory(2), the scope of the value of 2 should be lost through normal scoping rules, however due to the feature of closure it is not and this technique is possible for a number of languages. Not only must it maintain state, but it must do it for each instance of closure. Notice that the cube function has a power state all it's own.

In closing I just ran across this great post by Neal Gafter, which is titled the definition of Closures. I didn't get to it in time to reference his material, which looks great. Perhaps there will be a follow up.

If we can't agree on the nature of closures containing variables... we will never understand monads :)

Thursday, April 30, 2009

Speaking at JavaOne 2009

Speaking at JavaOne this year, on Debugging Your Production JVM. The session is on June 3, 2009 at 4:10 PM. This session will cover a number of great JVM tools for debugging and managing a run JVM, with a clear focus on BTrace. This is the 3rd time speaking at JavaOne (2001 and 2003 being the times of past). With the Oracle buyout, this could easily be the last for all of us. Welcome to Oracle OpenWorld! The question which remains...

What will happen to Duke?

Wednesday, April 29, 2009

Presentation Worst Practices

With all these "Best Practices" floating around... I was wondering. Is anyone documenting the worst practices? One Google search later, I discover that not only is there a number of them, but the domain name is taken by someone just parking it... boo!! Here are a couple of my favorite:

Defining Worst Practices - in summary says you are violating a principle or natural law.

Scalability Worst Practices - brilliant... a fun read. Golden Hammer, Big Ball of Mud.

REST Worst Practices - Also a good read. Summary... Rest fails with highly de-normalized systems... oh and don't hard code stuff.

and my favorite worst practices for this post:
Presentation Worst Practices

Happy Coding!

Monday, April 27, 2009

Building Spring 3 working with Git

Until today, I've been keeping up on the Spring 3 development off the svn repository with svn. There is a great post by Chris Beams from SpringSource that provides adequate detail and is a great resource to read as a precursor to this post. In general, I'm moving all my development over to Git, and it was time to get Spring 3 inline. It was not without pain. For those who just want a quick reference, I will provide a quick step by step on how to achieve the end goal of setting up a git client to the spring 3 svn repository. Following that I will provide details for those who want a deeper understanding.

Quick Step-by-Step Spring 3 with Git
  1. In the directory you would like the spring project execute : "git svn clone --stdlayout"
  2. change directory to the newly created spring-framework directory
  3. copy forked script from named git-svn-clone-externals to the current directory; make executable and run.
  4. run "git svn show-ignore > .gitignore"
  5. run "git gc"
  6. from the current directory you'll need to create the following directories
  7. mkdir org.springframework.instrument/src/main/resources/
    mkdir org.springframework.instrument/src/main/resources/META-INF
    mkdir org.springframework.instrument.classloading/src/main/resources
    mkdir org.springframework.instrument.classloading/src/main/resources/META-INF
    mkdir org.springframework.asm/src
    mkdir org.springframework.asm/src/main
    mkdir org.springframework.asm/src/main/java
    mkdir org.springframework.core/src/main/resources
    mkdir org.springframework.core/src/main/resources/META-INF
    mkdir org.springframework.expression/src/main/resources
    mkdir org.springframework.expression/src/main/resources/META-INF
    mkdir org.springframework.web/src/main/resources
    mkdir org.springframework.web/src/main/resources/META-INF
    mkdir org.springframework.orm/src/main/resources
    mkdir org.springframework.orm/src/main/resources/META-INF
    mkdir org.springframework.web.portlet/src/main/resources
    mkdir org.springframework.web.portlet/src/main/resources/META-INF
    mkdir org.springframework.test/src/main/resources
    mkdir org.springframework.test/src/main/resources/META-INF
    mkdir org.springframework.integration-tests/src/main
    mkdir org.springframework.integration-tests/src/main/java
    mkdir org.springframework.integration-tests/src/main/resources
    mkdir org.springframework.integration-tests/src/main/resources/META-INF
    mkdir org.springframework.instrument/src/test/java
    mkdir org.springframework.instrument.classloading/src/test/java
    mkdir org.springframework.asm/src/test
    mkdir org.springframework.asm/src/test/java
    mkdir org.springframework.aspects/src/test/java

  8. export ANT_OPTS="-XX:MaxPermSize=256m -Xmx1024m"
  9. change directory to "build-spring-framework" directory
  10. run "ant"
  11. wait 20 minutes :)
The background and detail
The first line is standard git-svn to get the repository. If you are new to Git, this will take a few minutes long then a standard svn checkout. It is copying all the changes throughout history to your local .git repo. Fortunately spring 3 is still young from a history standpoint.

svn external - The first issue
The first thing to note is that part of the build process for spring 3 is a project which is added to this svn project as an external, which led me to this post on why svn externals are evil :) evil or not we have to deal with them. This led to a post by Alieniloquent which has some good explanations, followed by what I eventually settled on, a post by Andre Pang, which includes a script he created, which I forked off github. Kudos to Andre!! The script he created (which I can see using again the future), provides a one execution solution to what Alieniloquent blogged. It is beautifully simple. The script provides the following:
  1. pulls down all the svn externals (spring 3 only has one)
  2. creates a symlink (mocking what svn would have done)
  3. adds the symlink and the .git_externals directory to the excludes file
If you wanted to check your svn externals for other projects use the following command: "git svn propget svn:externals ." in the root of the project directory.

The next step is to get all the svn ignores into the .gitignores file.

version control directory management - the next issue
The explanation behind this issue is that svn allows for and manages empty directories... git does not. git only manages content. There are a number of empty directories which the spring build files expects to be present for the build to succeed. This is based on a standardize build system and a lack of error handling in the build system. If I get some time, I'll send a patch to the springsource guys to fix this. At this time you will have to create all the directories listed.

jvm out of memory issues - the last issue
ant will run out of memory without these changes.... first you will run out of perm space, followed by heap. You may be able to get by with less; I didn't spend much time here. The standard defaults will fail.

Also if you need to update your master branch with git... "git svn rebase" will do the job!

I need to say thanks to those bloggers already noted and to fellow NFJS speaker Matthew McCollough.

Thursday, February 26, 2009

Justification for Functional Programming

It appears that this subject has caught fire in recent years. The interest in St. Louis has resulting in the Lamdba Lounge started by Alex Miller. (Great name by the way). I have started reading (but not finished) Programming Clojure by Stuart Halloway and Programming Scala by Venkat Subramaniam. Additionally I just finshed writing a 10 page article on an introduction to functional programming, which includes 4-5 page on F#. The article is for NFJS who holds the exclusive rights at this time, but I plan to provide a link to it or post it here in its entirity after 90 days.

I should point out that I am no expert in this space, but I have been developing professional for 15+ years with a variety of languages (C, C++, Java, C#, a little Ruby, Perl, Groovy). In the last 6 months I've been "playing" with Scala, Clojure and F#.

The reason for this post is this: This article has been sent to me a couple of times and has been on twitter several. At this point I will make public comments I made in an email response regarding this subject. The first two subjects were points made via email, which make sense to maintain for this blog post.

The Argument to have UI and DB Support
I don't get the ui / db argument. This indicates an all or nothing mentality, which I would hope the industry could get over. A griffon front-end to a java hibernate db access and clojure rules engine make sense to me. Many of the functional languages provide the ability to produce a UI or interact with the DB, however that isn't really important to me as you will see below, but there are good solutions out there where this isn't really a concern.

Functional Notations and Code Beauty
I may be in the minority... but I could care less for "beauty" of code, I'm interested in less code that adds the same or more value. FP adds several notations that provide a conciseness to code, which is a huge value add. As I was writing up an article on F#... I became frustrated with my other programming languages... for instance: why is a switch in Java/Groovy so limiting. Groovy makes it better... but it is still limiting as you compare it to discriminated unions and pattern matching F#.

Getting Functional Programming
People don't get it yet (referring to the nah sayers)... they are in the battle pits and are lacking the big picture vision. This year quad core laptops are expected to be more common. That trend isn't going to stop. The last decade was all about die size and memory. This next decade is all about cores. The need for concurrency tools is a must. The language trend may be slightly ahead of its time.

I'll post more on this subject in the future... I just felt it necessary to provide some response to the value of functional programming. Frankly I've barely scratch the surface of the value.

Tuesday, February 24, 2009

Java 7 on Mac OS X

This blog post is worthy of Retweeting:)

In 2 weeks when I hope I have time... I will be on this!

BTrace Fix on the Mac

The last blog post showed a work around for getting BTrace to work on a Mac, however as explained it fails when trying to probe Grails. I've been working with the BTrace developers and now have a solution. It will likely be some time before the next release. So for those who don't want to build from source, I put a drop on of a build.

Side notes:
1) It appears that all the scripts and assumptions for Java do not take in account the Mac. Groovy, Grails and previously BTrace all have references to a tools.jar in a place which is not correct on a Mac.
2) is new to me... and it appears worth the look. I will likely use in the future. Mostly for presention code drops. I like github for source references, but sometimes it is just quick and easy to tar or zip up what you have and drop it.
3) There is an interesting situation with BTrace which can be frustrating if you are not aware of it. If you have a failed BTrace session... such as a connection refused on the BTrace side, then it is not possible to BTrace it again with a fixed version of BTrace. You have to restart the application to be probed.

Sunday, February 22, 2009

Fixing BTrace on the Mac

Well it is time to fix another Java issue on the Mac... again... Sorry for the sarcasm. I love Java and Groovy and I love my MBP, but as already documented (on fixing Java Memory Tools and Java is a 2nd Class Citizen), these two don't get along as often as I would like. I thought for some crazy reason that I was the only one fighting this issue until all the comments from the fixing the tools article appeared. Glad to know, first that this blog is helpful to others, second that there are number of great people all over the world (offer drinks no less) and third that we are not alone in this fight.

BTrace and the issue
First... if you don't know what btrace is... you have to check it out. In the evolution of debugging tools for the jvm, it is the next big thing! Active in this space is A. Sundararajan, he has a couple of my favorite blogs, one on scripting btrace and another on jmx and btrace. Another great tutorial blog is by Igor Minar.

This blog isn't as much about btrace, as it is about fixing it on the mac. BTrace is new enough though, that it may require just a quick introduction. BTrace is a debugging tool, which injects "probes" (my term) into a running Java process. It is an open source option that has a high possibility of replacing Wily's Introscope. To be fair, Introscope provides a lot of extras which BTrace doesn't at this time... but Introscope is tens of thousands of dollars and BTrace is a free open source tool. I would be shaking in my boots if I was on the Wily team.

We will need to start a Java process in order to illustrate how it works (or in the case of the Mac... doesn't work). For the Java process look in the demo directory, there is a Java2D.jar. On the Mac it is located: /Developer/Examples/Java/JFC/Java2D. At the comand-line type: java -jar Java2D.jar. This will start up a Java process. To get it's pid at another command-line type: jps
If this is the only Java process running on your box, the pid with the jar notation is the pid you are interested in. If you want to be 100% certain type: jps -l . You'll get something like:
7838 Java2D.jar

Now in the btrace bin directory (I'm assuming you followed one of the referenced tutorials), for our example type: ./btrace 7838 ../samples/ . To which you will get:
Connection refused

In the terminal of the running Java process for the jar you would likely see:
btrace DEBUG: adding to boot classpath failed!
btrace DEBUG: error in opening zip file error in opening zip file
at Method)
at java.util.jar.JarFile.(
at java.util.jar.JarFile.(
at com.sun.btrace.agent.Main.main(
at com.sun.btrace.agent.Main.agentmain(
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
at java.lang.reflect.Method.invoke(
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(

Yea... epic fail! That is BTrace failing on the mac. If you want more details, go to the btrace script and edit -Dcom.sun.btrace.debug=false to be true. This will indicate the following stacktrace:
btrace DEBUG: debugMode is true
btrace DEBUG: dumpClasses is true
btrace DEBUG: dumpDir is .
btrace DEBUG: probe descriptor path is .
btrace DEBUG: parsed command line arguments
btrace DEBUG: System ClassPath: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/tools.jar
btrace DEBUG: adding to boot classpath failed!
btrace DEBUG: error in opening zip file error in opening zip file
at Method)
at java.util.jar.JarFile.(
at java.util.jar.JarFile.(
at com.sun.btrace.agent.Main.main(
at com.sun.btrace.agent.Main.agentmain(
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
at java.lang.reflect.Method.invoke(
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(

The line just above the fail point... hmmm.... it is looking for a tools.jar in the $JAVA_HOME/lib directory. As already noted, Apple likes to slice and dice the JDK... In trying to understand the issue, I uncovered an article released by Apple which says there is no tools.jar file... no tools.jar file!!! It is a classes.jar file and to make it fun it isn't in the $JAVA_HOME/lib directory... no, it isn't even under a subdirectory of $JAVA_HOME... I can't even write that without frothing at the mouth. I could start a rant here... but I'm assuming I'm writing to the choir so to speak.

Fixing the Issue
I didn't spend a significant amount of time on it... but I didn't get this working by changing the btrace scripts... something is in the code. I have another recommendation towards the end that the script change wouldn't work for as well. So we are left currently with the following work around.

Take the classes.jar and copy it to the lib directory as tools.jar (argghhh!!!) but it works. To be more specific... /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Classes/classes.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/tools.jar. It is important to note that you have to be sudo to make this change and you will have to do this with each Java version you want this to work with. Let's hope that the next version of BTrace will be Mac friendly!

After you get that working... checkout visualvm with the BTrace plugin. It is great stuff and another reason that the script change alone is not a total solution.

Other Issues
OMG... I'm frick'n shaving the yak again... just as I'm getting ready to post this... I was about to trace a grails app. The changes described above, break Groovy and Grails. So pick one :) You can be Groovy or you can BTrace but you can't have both yet. I guess I'll begin digging through code and send a patch tonight...

Thursday, February 19, 2009

Speaking on the NFJS Tour 2009

I will be speaking at a number of shows with NFJS this year. Already scheduled is Milwaukee, St. Louis, Minneapolis, Boston, Seattle. I will likely be at Omaha, Atlanta, Denver, Columbus, and Raliegh. That's just the first several months :).

It is always around this time of the year I like to revisit my goals. Commitment comes easy around January of a new year. Around mid to end of Feb, is a good time to check to see which goals you are seriously committed to. One commitment, I've maintained for several years is a commitment to learning and a commitment to excellence. This is easier when surrounded with like-minded and committed individuals who help to hold you accountable. It isn't always easy... just worth it. That is one of the reasons I love NFJS. The caliber of the speakers are top notch and world class.

For some the economy will be a an easy excuse to not attend. Or perhaps an employer will not pay for it this year. What is clear to me is those with the skills and the networks will find it easiest to find the next opportunity. This is the time to pick up new skills. As a local venue, no show makes it easier or less expensive than NFJS.

See you there !

Friday, February 13, 2009

Intellij 8.1 with Git Support

Finally... a good java editor with good git support. I've been waiting for a long time (in internet time) for git integration in the editor. Download Intellij 8.1 for Git support.

I have a number of project demos which are already in Git. I expected that Intellij would just pick up on this fact... and it did not. I then created a new project, did a Git init from Intellij and all was good.

To get IDEA to recognize a project already in Git, under the Version Control menu, select Enable Version Control Integration, then Git.

So far... the Git integration is fantastic!!! Love it! Well done JetBrains!
Happy Coding

Thursday, February 5, 2009

Java Memory Management Details

Needing to gather some details to help answer a question regarding JVM memory and the the distinctions of Perm Space. I thought it might be helpful to post the links of what I thought were incredibly great posts from the past... It is now 2009 and the best posts on this matter appear to come from 2006. Hmm.... perhaps there has been a JDK release in some time :)

On the subject of; the reason the permanent space exists. Jon explains the details of instances of objects and their dependencies on instances of classes and the need for their order as it relates to garbage collection. He also discussions the klassKlass... interesting stuff.

On the subject of String intern, XML parsing and it's effect on Perm Space. haakon or what I assume to be Jenny details the effect of String.intern() and how an out of memory can occur. She does a great job of explaining their troubleshooting approach.

And my favorite book on the JVM spec is Inside the JVM. It turns out there are free chapters of this on the web now and chapter 5 is the chapter detailing the taxonomy of the JVM memory space.

The weakest coverage of any subject based on my searches, is the subject of root sets. Brian Goetz's article on Java Theory and Practice is rock solid.

Monday, January 26, 2009

Java is a 2nd Class Citizen on MAC OSX

Having made several comments over the last several months, and in particular over the last couple of days regarding Java on the Mac, several people have asked me to explain myself. So here goes.

When I first moved to the MBP as my primary development machine, I was very pleased. As a developer, I was amazed at how all my development tools were just there. Ruby, Rails, cvs, subversion, 2 Java JDKs, it just worked out of the box.

When I first discovered the difference
I have a standard demo I use when show other developers some of the nifty debugging tools like visualgc or visualvm. In order to show the tool off, I need to start another Java process. So I standardly go to the demo directory and start the Java2d.jar. Oh but wait... where is that on the mac. This lead to the discovery that the jdk is in several different directories /locations.

JAVA_HOME = /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
Demos: /Developer/Examples/Java/
Headers/Lib: /Developer/SDKs/MacOSX10.5.sdk/System/Library/Frameworks/JavaVM.framework/Versions/xxx
Java Lib: /Library/Java/Extensions; /usr/lib/java
Endorsed Dirs:$JAVA_HOME/lib/endorsed
System jars: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Classes

OK... I'll live with that. I quickly discovered a very useful scipt for switch between JDKs. I've modified the original if anyone interested let me know. But since we are on the subject of being a second class citizen, If you go out to the Sun site and look for the latest jdk, the options are windows, linux, and solaris.

Other Issues in Jakarta
Then the next issue came up... jhat was broken. I wrote a lengthy blog post on how to fix this. The issue... Apple didn't include the javascript libraries, which are necessary for these tools to work.

So now I'm taking a serious look at btrace, which is a great tool. Just trying to do the simple stuff, works fine on Windows... but no joy for the Mac. I'm still looking into it, but it appears that Instrumentation..appendToSystemClassLoaderSearch() fails on the Mac. Could be something else... still looking.

If you are doing standard Java development or Groovy and Grails... you may never notice the Mac difference. When you venture into the debugging and instrumentation realm of Java be prepared for some frustrations. As I get a solution for the btrace issue I will post it.

On the positive side, I enjoy the development experience on the MBP better than my experiences on Windows and linux. The memory management is better and the startup times on Java processes is fantastic.

Saturday, January 24, 2009

JSR-299 Web Bean Review

As several JCP specifications are reaching there final stages, I decided to have a look at two of them. Truth be known, I actually was going to look at one of them jsr-299, the specification previously known as web beans, here forth now known as Java Contexts and Dependency Inject. There was sufficient enough mention of ejb-lite that I decided to have a look at its specification, jsr-318.

Before I get started, my position on the subject of ejbs should be known. I actually thought that ejbs were on the way out... with the lack of testability and the extra unnecessary complexity for most projects prior to ejb3. EJB3 greatly improved this, but IMHO was too little too late. Spring + Tomcat already provided what folks were looking for. Additionally the number of clients using JSF is a dying breed. Once again, JSF 1.0 missed the boat with non-bookmarkable pages, etc. While there are shops sticking with these Sun driven standards which are produced out of the JCP, the majority of support appears to come out of vendor interest and not developer interest. My earlier work on projects with JSF led me to appreciate a couple of frameworks; 1. facelets, 2. a4jsf, and 3. SEAM (when working on jboss). jsr-299 (web beans) seems to be the formal standardization of SEAM.

My thoughts on jsr-299
It appears that @ annotations are the new shiny toy, and there is no end to specifications going crazy with them. jsr-299 defines 34 new annotations. The goal of the specification as defined on the spec home page, is to unify JSF managed bean component model with the EJB component model... well I've stated my position on this. So either this specification is too little too late and a benefit mainly to jboss, or perhaps there is something to be gained beyond this spec stated goal.

There were several interesting ideas out of the specification. The first was the creation of an extensible annotation model. So if there are not enough annotations in the world for you, you can create your own that work with the framework. This is accomplished through the new javax.inject.@BindingType. This appears to provide fine grain autowiring qualifier. One of the examples given defines
class LdapAuthenticator
implements Authenticator { ... }

followed by the declared injection of an LDAP Authenticator:
@LDAP Authenticator authenticator

Another interesting idea is producers... I really like the concept here. Page 3 and 4 illustrate an example where a @SessionScoped @Model Login class has a @Produces @LoggedIn User getCurrentUser() method. Abbreviated below:
@SessionScoped @Model public class Login {
@Current Credentials credentials;
@PersistenceContext EntityManager userDatabase;
private User user;

@Produces @LoggedIn User getCurrentUser() {
if (user==null) {
throw new NotLoggedInException();
} else {
return user;

Then the getCurrentUser() method is invoked for the inject as listed below:
public class DocumentEditor {
@Current Document document;
@LoggedIn User user;
@PersistenceContext EntityManager docDatabase;
public void save() {

The last interesting part of the specification is the event framework. I don't understand why this is in the specification. There appears to be no dependency for the event framework on the other parts of the specification. It doesn't comply with the stated goal of unification of JSF and EJB. It would make sense for this to be separate. Mainly to use it separate from all other aspects of web beans.

Thoughts on jsr-318
I did not review the entire specification yet. I was checking out some of the mentioned ejb-light details mentioned in jsr299. As mentioned, outside of being required by clients to help them with ejbs or ejb light projects, I'm really not interested. There is however one very interesting addition to the ejb specification, that is the @Asynchronous annotation. This is long over due. In addition to its obvious benefits, a method annotated as asynchronous can also return a value defined as java.util.concurrent.Future where V is the return value type. The return of the Future object provides the ability to check status (isDone()) or to cancel the request. Very cool. For a quick list of EJB 3.1 changes, checkout Ken Saks blog.
The concept of EJB-lite is lost on me... I already have that it is Spring.

If you are looking to stay in the world of EJBs and JSF, then these newly defined specifications are of great value to you and your shop. jsr-299 fills a number of gaps in the jee space and provides the power of Spring in a standardized way. Clearly jsr-299 will be part of the new jee 1.6 specification, at least in several of the profiles. If you are not using either JSF or EJB or nether, then it is unclear what value this brings to the community. It will be interesting to see if a vendor implements this specification (or a portion of it), in a world without JSF. It appears that the jsr-299 specification helps vendors more than it helps developers.

Sunday, January 11, 2009

Windows 7 Download Frustration

The danger in posting this blog entry is that I will be counted as a anti-Microsoft person... which is just not true. I happen to like using the best tool for the job which is not all the time from Microsoft (and sometimes it is). In 2008 I switch my primary system to an Apple Mac. There were many factors as to why and frankly I love it! But I'll always be a technical junkie.

With a number of friends (such as Denny Boynton and Ted Neward) posting about their experiences with Windows 7, I thought it was time to take a look. I thought it was going to be a matter of download, get key, start up vmware fusion and I way I go... but I was wrong.

First I went to the public beta site: and selected the 64-bit version in english and got this. WTF?? Repeated attempts resulted in the same. An oops page with a pre-canned search. Where did I go wrong? Well as you can tell, I'm on my Mac. So I pulled out fusion to launch Windows XP for round 2 of the attempt.

I thought this is just wrong, but determined to get a look, I switch to windows and my suspicions were confirmed when I got one page further. I got the download page with a couple of large buttons on the bottm of the page and one read "Download Now". Hey, that's what I want... I want to download now. I clicked the button and... nothing. Click... Nothing... No way... they didn't. Round 2 was in XP, but with firefox.

Round 3 as you would expect is XP with IE. That combination was successful and I'm now 29% into my download.

BTW... In the process of testing a few more times in writing up this blog, the round 1 mac failure was fixed to the point where you will get download page (nice response time msft), however the download button fails.

Why is it necessary to be like this? Why is it so hard to put up a link to a download which is platform neutral? Wouldn't Microsoft want to attract customers from other platforms? Does it always have to be all or nothing?