Ken's Technical Thoughts

Constant Pain with Non-Constant Constants

2011-12-20T11:49:00.000-08:00

This thought has crossed my mind before... since it is on my mind now I thought I might share. Perhaps it will help someone on their software development career path... or perhaps it is just me venting...

The Art of Coding

One of the greatest challenges of coding is being able to understand it. Skill in coding is being able to understand the problem well enough to create a solution (otherwise known as coding). Great Skill is being able to write a solution so that someone else (including the future you) can read and understand what it is they are looking at. This concept isn't new, but yet we seem to be either challenged with the concept... or distracted. Even back in 1952 (yes 1952!!), in the computer history archive a set of lecture notes which in them states: "It is quite difficult to decipher coded programs even with notes and even if you yourself made the program several months ago." It also states: "I think the answer is simple: to make it easy one must make coding comprehensible". Well said!

The Point of Constants

One area that seems to be a constant :) struggle is constants. There are at least a couple of justifications for using constants I can think of: 1) a point of reference for tools (autocomplete with intellisense in IDEs and refactoring), 2) A common holder of value for which if it changes will be realized across the system. The main point of a constant isn't DRY or reuse... a common point of confusion for some reason.

IMO the stronger argument is point 2. if you were coding in VI or emacs, point 1 may not be of value, but point 2 certainly would be. If you name the reference of a constant the same name as the value of the constant... then you lose any the value of point 2.
It must also be said that constants, although they add value, they make code less readable (see examples below). If your constants have no value, they don't achieve points 1 and 2, then they need to be whacked!

Examples of Losing with Constants

This is real production code... some variables changed to obscure identify.



        // example 1   Really??  where does this end?
 String EMPTY_STRING = "";

        // this isn't readable... it takes minutes (instead of seconds) to understand its value
        // example 2
 String ADVERTISE_PROPERTY_PREFIX = OUR_PROPERTY_PREFIX + "advertise.";
 String ADVERTISE_TRADER_ADDRESS_PROPERTY = ADVERTISE_PROPERTY_PREFIX + "host";
 String ADVERTISE_CONTEXT_PROPERTY = ADVERTISE_PROPERTY_PREFIX + "context";
 String ADVERTISE_TYPE_PROPERTY = ADVERTISE_PROPERTY_PREFIX + "type";

       // example 3
       // here's a common one
static final String TABLE_NAME = "email";

 // Column names for all the fields in the email table
static final String ID = TABLE_NAME + ".email_id";
static final String CUSTOMER_ID = TABLE_NAME + ".customer_id";
static final String ADDRESS = TABLE_NAME + ".address";
        // ...
static final String FIELDS = ID + ", " + CUSTOMER_ID + ", " + ADDRESS + ", " + CREATED_BY + ", " + LAST_MOD_BY
  + ", " + CREATE_DATE + ", " + LAST_MOD_DATE + ", " + TERM_DATE + ", " + WEB_USER_ID;

 // List of fields names with formatting used for select statements
static final String SELECT_FIELDS = ID + ", " + CUSTOMER_ID + ", " + ADDRESS + ", " + CREATED_BY + ", "
  + LAST_MOD_BY + ", " + "TO_CHAR(" + CREATE_DATE + ",'" + DATE_TIME_FORMAT + "')," + "TO_CHAR("
  + LAST_MOD_DATE + ",'" + DATE_TIME_FORMAT + "')," + "TO_CHAR(" + TERM_DATE + ",'" + DATE_TIME_FORMAT
  + "')," + WEB_USER_ID;

 // SQL used to create a new EMAIL record
static final String CREATE_SQL = "INSERT INTO " + TABLE_NAME + " (" + FIELDS
  + ") VALUES (?,?,?,?,?,SYSDATE,SYSDATE,NULL,?)";

 // SQL used to retrieve an email record by id
static final String RETRIEVE_BY_ID_SQL = "SELECT " + SELECT_FIELDS + " FROM " + TABLE_NAME + " WHERE " + TERM_DATE
  + " IS NULL AND " + ID + "=?";

For example 1... where does it end? would you create a constant A = "A"? When can EMPTY_STRING not be an empty string or ""? In this case, we get no value and less readability.

Example 2 is the chaining of constants, commonly found when there are a large number of property values being retrieved from a file or sys env. While not as bad as example 1, it is less readable. Having a prefix constant may be of value, but rarely is the chaining of constants of value. It isn't worth trading the potential for a company name change or something similar for readability. Or put another way, it isn't worth trading something that is unlikely to happen for something else which is more likely to be needed (that being the readability of the code)

Example 3 I see all the time... argghhhhh... please... just stop!

Examples of good constants would include, urls and database connections strings or a project defined standard date format.

May your constants always be final!

Advanced Spock Techniques

2011-12-14T15:10:00.000-08:00

In recent years there have been a couple of tools that stand out when it comes to helping me be productive. One of those is the groovy test framework Spock. It is worthy of an introductory blog post... but that isn't this post. One of the challenges to Spock is the documentation isn't a complete as one would hope. The Spock Basics is fantastic when getting started but it is what it claims to be basics. I've been speaking with NFJS on the subject of Spock for the last half of 2011. On 2 occasions I've had the look our red shirted friend in the picture has when asked about controlling aspects of mocks in spock... meaning I didn't have the answer. In this case, google didn't discover it either... a quick email to the Peter Niederwieser reveals the previously undocumented solution. (at least that's my understanding)... and thanks Peter!

Here we will discuss 2 advanced aspects of mocking with spock. If you need more details on mocking in general hit the Spock site... or wait patiently for a future post:)

Problem Setup:

When mocking with Groovy we get 2 benefits from the verification of the mock. 1) order of execution, 2) failure if another method on the mock was executed but was not setup in the demand configuration.


void testDeposit() {

        Account account = new Account(TEST_ACCOUNT_NO, 100)

        def mock = new MockFor(AccountDao)
        mock.demand.findAccount(TEST_ACCOUNT_NO) { account }
        mock.demand.updateAccount(account) { assertEquals 150, account.balance  }

        def dao = mock.proxyDelegateInstance()
        def service = new AccountServiceImpl(dao)

        service.deposit TEST_ACCOUNT_NO, 50
        mock.verify dao
   }

Spock Cleanup

A similar solution in Spock might look like this:


    def "deposit into account test refactor"() {

        def mock = Mock(AccountDao)
        def service = new AccountServiceImpl(mock)

        when:
        service.deposit nmr, amt

        then:
        mock.findAccount(_) >> account
        1 * mock.updateAccount(_)
        0 * mock.createAccount(_)

        and:
        account.balance == total

        where:
        nmr   | account                 | amt | total
        "101" | new Account("101", 100) | 50  | 150
        "101" | new Account("101", 0)   | 50  | 50
    }

However this solution doesn't guarantee order nor do we want to use the 0 * mock. on all possible methods.

Spock Mock Solution

It turns out that we can repeatedly use the then: dsl to determine the order of events. So we can modify the Spock code above with the following changes.


        then:
        1 * mock.findAccount("1234") >> account

        then:
        _ * mock.updateAccount(_)

In this case we are saying that the findAccount() method will be called exactly once and return the account object and "then" the updateAccount will be call any number of times with any type of argument. The test will fail if the updateAccount is ever called prior to the findAccount being invoked first.

We can use this same technique to solve the strictness concern we have as well by adding one more then: block of code as outlined below:


    then:
        1 * mock.findAccount("1234") >> account

        then:
        _ * mock.updateAccount(_)

        then:
        0 * mock._

In this case the last 0 * mock._ says to expect nothing else. Ahhh.. the beauties of a dynamic language:)

Happy Coding... may your builds never fail and your tests always green bar!

MongoDB Grails and Copying Collections

2011-10-13T06:07:00.001-07:00

I currently find myself working on a project where Grails and MongoDB are the technology stack... in fact that in part is some of the reason for being interested in the project. I thought I would share 2 aspects of my learning so far:

Grails + Mongo

We spent at least 2 weeks trying to understand which approach we were going to take for our persistent tier. The general concensus was to use the Mongo GORM. We started down that path and ran into a bug and then a limitation on embedding. We got amazing support on the bug followed by the next release of the GORM option which fixed our initial document embedding concerns. We ran into more problems.

We switched to the morphia driver... once again... challenges caused lots of frustration. The challenges included getting other plugins to work with this driver, the fact that the morphia "domain" objects couldn't be in the standard grails domain folder, which further complicated mocking. Frustrations with getting full capabilities of getting spring security to work. It turns out that with spring security there are a number of relational assumptions (along with assumptions of objects being domain classes.. meaning they live in the grails domain package space) which cause problems when you are using NoSQL.

The morphia had another limitation we couldn't live with... we wanted the ability to have different types of objects mapped to the same collection. It appears to me that the Mongo efforts in the Grails space are bolt ons to a rational model. You can get it to work when thinking with a RDBMS mindset. They seem to break down with a NoSQL mindset... this certainly could be attributed to user error (meaning me). I believe we gave it a good try... however the project is under the gun and we didn't have time to "play" with it or wait for responses.

Alas we created our own solution... one of our significant constraints was to be prepared for extremely large scale. This had us duplicating a lot of documents not common in RDBMS solutions... along with the fact we needed to control the number of queries. Ideally we want 1 query for a page... certainly for the main page which is high laden with dynamic content.

Mongo Copying Documents - Collection to Collection

I'm really getting to like Mongo... a lot. One of the downsides is there is limited documentation... including limited forum comments on it. Because of this, I've decided to post interesting solutions to problems as I discover them... time permitting. so for todays tip. I recently needed to duplicate a "record" in mongo speak a document in a collection. There is so much information out there on how not to duplicate records that a google search at this point usually points you in the wrong direction. My situation was such that we have a CMS component to our application, which sets a publish day on content. What I needed was the ability to duplicate a record (so the BSON id would need to change and set a new publish day.

The first thing to recognize for those new to Mongo is that the Mongo console (the command line client to Mongo) works with javascript... so get your javascript fu ready... no.. not your jQuery fu... your plain jane generic javascript fu:)

Here is the solution I came up with:


db.NewsItem.find({newsType: "Blah"}).forEach( function(x){ x._id = new ObjectId(); x.publishDay=283;  db.NewsItem.insert(x)} );

So it turns out that you can pass a lambda expression to the forEach method of the collection. The trick is to assign a new ObjectId() to the _id, make any other assignments and then insert into your collection of choice... in this case back into the same collection. This comes in handy as well when you want to backup a collection real quick. (yea, I know you could use the mongodump and mongorestore, for a quick temp solution this just seems more convenient)


db.NewsItem.find({publishDay : 285}).forEach( function(x){ db.NewsItemBackup.insert(x)} );

Happy Coding!

JavaOne: Rocking the Gradle

2011-10-05T09:02:00.000-07:00

Presenting Rocking the Gradle at JavaOne tomorrow 12:30pm at the Parc 55... see you there!

Throughput and High Velocity

2011-01-11T14:50:00.000-08:00

Wow… has it really be 11 months since I’ve blogged. I had many ideas for blogs half written or half thought out… but last year was extremely busy. It’s a new year… and I’m back :) Let’s not focus on the past, let’s get right into it.

I had a client recently asking how to improve throughput. It took me a few minutes of questioning to understand what the subject really was. Thoughtput? Are you having scaling issues? It turns out, the interest is closer to developer throughput, although I don’t think they used that term and it may have been broader than that. The interest is in the acceleration of moving an idea into a product. Immediately coming to mind is the book “The Goal”, whose focus it is to increase the throughput of a manufacturing plant. Although the creation of software is nothing like an assembly line (although the analogies seem to be common place), it is however interesting to consider some of the similarities involved with the theory of constraints. The challenge is, in my opinion, that the constraints for software development are more virtual and harder to measure than assembly line constraints.

Consider what slows down the development of software. Of all my years of software development, the issue has never been an issue of programmer to keyboard. In other words, it isn’t how you type or how fast you type, although it certainly needs to be reasonable. It often times isn’t how long a developer has been developing… although again there is a required threshold. Where have there been issues? Let’s see if you list is similar to my list:

Poorly written requirements. The user seems to know what they want, but was unable to communicate it effectively or the BA was unable to capture it appropriately.
Lack of direction, mission and purpose of the entire team… either we have too much time, too much money or a total lack of leadership. Let’s chalk this up to priorities.
Lack of knowing what to do next… BA or programmer… or PM
Lack of knowing how what a programmer is working on fits into the whole
Lack of team coordination
Large team sizes

I’m sure there are plenty more… but what I would say is the #1 issue for teams in trying to achieve high velocity is clarity of thought. I mean the ability to see a problem as the user sees the problem. Being able to see a solution and understand the problem in abstract terms and abstract ways; For a team to have the same thought or to hold within their minds the same mental model / picture. This is a tough task as differing priorities, differing locations, differing experiences and differing thoughts among the team members results in a number of differing ideas to the solution. It brings to mind to me of the scene in the last samurai, where Koyamada tells Tom Cruise “Too Many Mind”… “No Mind”

As Dr. Covey says in 7 Habits of Highly Effective People (which is what we are talking about… right?), you must start with the end in mind. Clarity positions people to be as effective as they can be. In the real world, a user really may not have the clarity themselves or may not understand the problem fully to articulate the desired clarity. This is why an agile development process is so effective. It provides an opportunity for the user to discover the clarity through regular feedback with the developers. Or in other words to detect deltas in the forming solution from the ideal or user desired solution. It also places the developer close to the user, so that through the creation of software when the output drifts from the user mental model it is caught quickly and corrected. Also developers are pairing with, and communicating with other developers regularly so that differences in the mental model are detected early.
What we are seeking for efficiencies sake is the fastest mechanism to detect that something is just not right.

So at the time of development what are the common practices that occur in a corporate development which cause inefficiencies or put another way what are the practices that cause the error detection to be delayed, either the detection itself is delayed or the communication of the detection is delayed.

Developers are several degrees removed from the user. So there are either 2 human “hops” for error recognition and for communication.
Developers are too junior. Junior developers are just less capable of thinking and communicating in abstract software terms and are less able to detect deficiencies in the solution model.
Separation of the team members. Numerous studies show the increase cost of communication of people in just 2 different rooms, its more expensive for 2 different floors and way expensive when there is an ocean between members of the team. There is an additional cost that isn’t well documented in the fact the difference in the mental model or solution set are very difficult off-shore.
Communication is all facilitated through paper. This is the worse of all the communication modalities.

So what can you do to increase throughput? What can you do to increase team velocity?

Hire highly skilled developers, developers skilled in their language who also have a proven record of quickly translating problem sets into solution sets.
Provide them with quick, and easy access to the user
Put the developers in the same work space
Reduce the team size, which results in less possibilities of mental model differences.

You are looking for a team with a high attention to detail and ability to detect differences in the mental model during the development process. Then you are looking for well-lubricated spontaneous communication.

This isn’t fool proof but it does solve one of the largest most common problems I see in corporate software development.

A number of ideas came to mind in the process of writing this blog... and there are some assumptions baked into the content. Based on that, I will likely blog deeper on several of these subjects in the future.

Setting up Clojure 1.1.0 on Mac OSX

2010-02-21T19:59:00.000-08:00

As part of the Lambda Lounge, established by Alex Miller (Thanks!!), we have started a group to study the SICP. We just started and are going virtual... if you would like to participate, email or tweet me (@kensipe). In the process of studying SICP and LISP, I plan on focusing on Clojure. Clojure was previously on my machine as I was reading Stu's book Programming Clojure, however increased usage would require some maturing of my tools. This led to the discovery of Mark Reid's blog on Setting up Clojure for Mac OS X, however I assume based on updates and changes over the last year, it requires some updates.

Read Mark's blog first... getting clojure, the clojure-contrib and jline. Here are the differences:

Create an environment variable CLOJURE_HOME assigned to the directory for clojure
Create the /bin directory for the clj script included below
Add $CLOJURE_HOME/bin in the $PATH
Clojure is distributed with clojure.jar in the CLOJURE_HOME... so I maintain that. It is expected that contrib and jline are in the same directory.
Mark's clj script has been updated below... The previous script invokes classes which are depreciated.

Here is the updated script:


#!/bin/bash
# Runs Clojure using the classpath specified in the `.clojure` file of the
# current directory.
#
# Ken Sipe   2010-02-20
# Inspired by Mark Reid 
# original: http://github.com/mreid/clojure-framework/blob/e1c80cc650f448713243be8272dba1fa3c1a7cea/clj
#
# This scripts expects $JAVA_HOME and $CLOJURE_HOME to be defined for the system it is running on
# The clojure.jar is standardly in the clojure_home dir (and not in the lib directory), so
# the expectation is that clojure-contrib.jar and the jline.jar are placed in the same dir

CLJ_DIR=$CLOJURE_HOME
CLOJURE=$CLJ_DIR/clojure.jar
CONTRIB=$CLJ_DIR/clojure-contrib.jar
JLINE=$CLJ_DIR/jline-0_9_5.jar
CP=$PWD:$CLOJURE:$JLINE:$CONTRIB

# Add extra jars as specified by `.clojure` file
if [ -f .clojure ]
then
   CP=$CP:`cat .clojure`
fi

if [ -z "$1" ]; then
   java -server -cp $CP \
        jline.ConsoleRunner clojure.lang.Repl
else
   java -server -cp $CP clojure.main -i $*
fi

Love the TextMate support and the history support in JLine!

Reporting from SpeakerConf 2010

2010-02-17T11:55:00.000-08:00

I was able to attend speakerconf 2010 this year in Aruba. It was an amazing gathering of talented leaders in the software development space. Each day of the conference started with several 15 – 30 minute talks lasting roughly 3 hours. According to the retro at the end of the conference, most people enjoyed and preferred these short talks as a primer to longer conversations in the afternoon along the pool or beach. Most talks were met with the struggle to maintain time often due to the many follow on questions generated by the subject. As far as I know the subjects were selected by the speaker and they were generally not preannounced. It is impossible to capture the essence of the conference and I make no effort to cover the most valuable part of it… the evening free form discussions. Most of the information went so fast that it was difficult to mentally be engaged and take notes. Here is a glimpse of what was discussed.

Day 1
Stu Halloway
Stu introduced the ADD development… which is alcohol driven. Stu then jumped into clojure, introducing a private testing project he is working on called circumspect. Stu offered up some advice for working in clojure:
* be less clever
* idiomatic clojure
* make adjectives not nouns

The focus on circumspect was to provide an idiomatic testing option for functional testing in clojure. Stu mentioned that through his testing exploration that an test he created actually detected a bug in the clojure code.

Dennis Byrne
Dennis is the first to discuss a topic on hardware and general process constraints in a talk titled Memory Barriers. The challenge is that virtual machines and hardware interfaces reorder code in ways most developers are not aware of. Fences or memory barriers define for the VM what can and can’t be reordered. There was some discussion on the expensive nature of these barriers.

Matt Deiters
Matt introduces a challenging problem that he has been working on in a story about vertex. The problem stems from social network analysis… it doesn’t work in SQL. Their previous implementation in SQL might take an hour to find all the details need for just 2 degrees of separation in a social network. Matt introduces neo4j an open source nosql graph database and neo4jr-simple, which is a RESTful ruby API. I enjoyed the fact that Matt wasn’t a No SQL bigot… He espoused a “Not all SQL” approach, using alternatives where SQL databases fall down.

Fred George
Fred displayed a triangle with management, story and programmers (or workers) at the vertexes. He wanted to focus on the relationship of the story to the programmer. Fred focuses on a Lean process and raises the question if acceptance tests are really lean? This stirred some debate. In the end, It appeared to me that what was being argued in most cases was the definition of acceptance tests vs. other types of tests. I don’t know if I agreed with Fred’s position, which we discussed at length the rest of the conference. My favorite part of his talk was a list of words or phrases that are flags that your project isn’t Agile. They include: Review, I need it in writing, write me a ticket, plan, checklist, code freeze, gant chart.

Brian Goetz
Brian’s talk is on how we find ourselves in this mess with multi-cores. Brian gives a wonderful history of processors defining them in eras: CISC era, RISC era, and Multi-Core era. The big take-aways for me include:
* a move from explicit to implicit parallelism
* moore’s law favors bandwidth over latency
* cost of memory access

With the speed of today’s processors, it takes 200-300 clock cycles to get 1 memory fetch, where it used to be 1 cycle. As latency increases we need more caches. Memory access times by clock cycle are roughly:
* Register ~ 1 cycle
* L1 ~ 3
* L2 ~ 15
* RAM ~ 200 – 300

Brian states “Memory is the new disk”. Clearly memory access is the new problem to solve in computer hardware. Brian also states that there is no programming model for this new multi-core era. Based on the talk, it appears to be that the industry has a new chicken or the egg problem. Hardware is struggling to evolve, because there is no software development model. Software is struggling because there isn’t a new standard. Brian concludes with latency is the enemy and the industries need to provide development teams with new tools. There will be a growing need to understand the memory access model of a running program, which can only be determined at run-time at the chip level. It will be necessary to understand the Integer pipe and memory cache misses. At the end of Brian’s talk, Nygard mentioned Chuck Moore’s Multi-Core chip where each core runs a tiny forth program. It is something I’ve added to the list of things to look into.

Michael Feathers
Michael discusses challenges to object oriented and functional language hybrids. There is background that I didn’t record, but one of the great observations is that functional programming is a “tell, don’t ask” model and object oriented programming is a “ask, don’t tell” model. In the FP approach, programming logic and data seem to bubble up the call tree and in the OO approach the opposite is true, logic seems to be pushed down the tree. Michael then looks at what it looks like conceptually if FP is on top of OO or OO is on top of FP. Michael then introduces the group to an algorithm used to solve morphological watersheds :)

Day 2
Steve Vinoski
Steve introduced us to a product his startup company Verivue, Inc is producing for the big media companies for streaming video. The IO throughput numbers were unbelievable. The software at the core of the product… Erlang!

Obie Fernadez
Obie and Desi introduced us to the agile practices of Hashrocket, indicating that they have evolved past iterations and they question the value added by iteration activities. They develop in one-week windows and use Pivotal Tracker for tracking stories and progress. The term “Iteration” isn’t even in their vocabulary. Later in the week, I challenged Obie to see if the word “week” replaces the word “iteration” as a convenience and to see if they were doing all things an ”iteration” would entail. The short answer is No! Clearly pivotal tracker desires a good look. Another tool Obie mentioned later in the week was Balsamiq... a tool used for screen mocking. Yet another tool to checkout.

Phillip Hanrigou
Phillip starts with the phrase “any sufficiently advance technology should feel like magic”. He continues with the fact that 30% of Amazon’s sales is based on recommendations. This was an interesting talk that had me flashing back to Matt’s talk on neo4j looking for synergies.

Oren Eini
Oren suggests that TDD doesn’t scale… with a beautiful picture of a motorcycle with training wheels. The major point is that unit tests are bad and scenario based tests are good.

Brian Marick
The take away… “I don’t want to care about anything until I absolutely have to care about it”… Love it!

Dave Thomas
Dave introduces us to vector functional languages such as KDB and k. These specialized languages are the workhorses behind querying ½ terabyte of data in 1 second for the hedge fund industry. Dave claims this is the most interesting technology to come along in 25 years. If interested, he suggests getting started with j from jsoftware.com

Eric Yew
Eric gives us a deep dive into GPUs. He was on the NVidia team that built the technology. Eric suggests that GPUs are making the industry rethink previous assumptions that are not correct in the GPU world. How would we code differently if:

There were no costs to threads
And no costs to context switching

The approach suggests that memory lookup is too expensive (seems like we keep hearing this :). It is cheaper to recalculate a value on a 100 threads then to retrieve from memory.

This model supports scale and throughput and not latency. Latency once again is sacrificed.

The memory model is upside down as well from the traditional general processors. There are 3 memory access points:

global - ~ 100 cycles to get to it
shared
local – stacks

It appears that the old paradigm of being abstracted from the hardware is over. The benefit is that the price of 2 Teraflops is near nothing… thank you computer gaming community!

Day 3
Neal Ford
Neal did a wonderful job presenting a presentation on presentation :) at a conference on conferences… it was a very meta way to start the morning. The big take away is http://presentationpatterns.com/ a site and forth-coming book on presentation styles, techniques and anti-patterns. The biggest benefit of the book is its focus on the practical. It will include templates for good practices.

Amanda Laucher
Amanda starts by defining the classifications of assholes and morons which got a lot of mileage the rest of the day. Amanda focused on educating us on terminologies in the category theory space with examples in F#. Here was what I was able to capture:

* Catamorphism - a fold
* Anamorphism – unfold
* Endomorphis – takes a type and returns the same type
* Homomorphism – takes a structure and returns the same structure or shape
* Monoid – A Functor
* Monad – Applied context
* A Monad is a monoid in the category endofunctors

Dave Hoover
Dave discussed the value of Resque and how it help in the development of Groupon.com

George Malamidis
George continues the conversation on memory access times and threads, introducing us to Node.js as a high scale solution for the web.

Michael Nygard
Thoughts on consistency – what a great talk! Michael introduces the concept of an observer and a super observer in a system through an object lesson. He then follows it up with the fact that with in a database there are often inconsistent states… the point is at the end of a transaction there cannot be any inconsistent states. For example, it is possible that an insert into the database would result in a violation of referential integrity… this is ok and necessary until a commit occurs.

Consistency is based on time. Frozen time means no change.

Another practical example:
2 ebay users making the same query at roughly the same time get different query results. This is viewed as being ok, because from the perspective of the observer it is consistent (they have no idea). From the perspective of a super observer there are some inconsistencies. This led to some discussion in mainly of the corporate failures stemming from the complexity created when trying to maintain consistency from the perspective of a super observer (which is a fallacy and trap).

Aslask Hellesoy
Another agile talk… this one focused on visualization of the return on development through CFD graphs. The suggestion was that velocity is a questionable valuation.

References included: CFD Details and a google spreadsheet
Robert Martin
I don’t think you can summarize Uncle Bob… you have to experience him :)

Summary
I made a couple of interesting observation / assessments through the conference. The first is there was a large percentage of the speakers / attendees that had a strong EE background. I truly enjoyed revisiting challenges at this level again.

Here are some of my takeaways from the conference:

We are at the beginning of a new era in field of computing. One in which we do not have a standardize model or tools
There is a need for programming solutions which reduce (or leak) the abstraction from the memory model that is generally preferable in the previous programming model. It will be necessary to indicate that values in memory are near, far or very far. It will be necessary to be able to measure and understand the cache miss rate.
We appear to be at our limits in our pursuit to improve latency. Worse…. Latency is the component that is consistently sacrificed to increase throughput.

IDEA 9, Gradle and Eating Your Own Dogfood

2010-01-05T09:11:00.000-08:00

After reading the comment on a post a few weeks ago regarding Intellij 9 and Gradle, I had to laugh. Having worked on open source projects for 15+ years and presenting at countless conferences and user groups, I have stated roughly the same comment countless times... "It's open source, if you don't like it or you want a change, then do it!". Thanks to Peter for the suggestion and encouragement. It clearly seems that my actions were a surprise as Peter's first comment in a private email exchange was "Great you're doing this!"

Intellij 9 Development Experience
Getting the source and setting up for the first run was fairly painless after you find the download information. It was nice to see that the code repository was git. I ran into 2 issues in getting setup:

Git clone failed with an unclear message initially. It turned out I was trying to clone using the git protocol behind a corporate firewall which was blocking it. As soon I was not behind the firewall, all went well. Be prepared... it is 924 MB.
Setting up the run configuration wasn't complex, but it wasn't as advertised.

Here is what the Building and Running from the IDE from the checkout and build page should read:

Open the project as a "new" directory-based project
Configure a JSDK named "IDEA jdk", pointing to an installation of either JDK 1.6 (recommended) or JDK 1.5. This is on the project tab of the project structure.
Increase the compiler heap size... default is 128M, it works with 512M :)
Use Build | Make Project to build the code
To run the code, use the provided shared run configuration "IDEA".

Important notes:

You will see that there are circular dependencies between 4 of the project modules.
With the default compiler heap size, you will get a compilation error ( which is erroneous )
There are a number of deprecation warnings during compilation... this might be a great place for someone to get involved and do some clean up.

After you have an Intellij running another instance of Intellij :) Here is a list of resources to be familiar with:

Source Repository Layout
View PSI structure (under the tool menu) - PSI is an IDEA AST abstraction, which was the toughest learning curve. Learn it! Then look at the following clases and their usage in the code base: PsiUtilBase, PsiUtil, PsiResolveHelper, PsiResolveHelperImpl

End Result
In the end, I spent 4 hours adding in additional gradle task support into Intellij 9, created a patch and submitted to Peter... a few days later I was informed that it was committed to repository. Enhanced Gradle support is on it's way!

I had mixed feels when IDEA announced it was going open source, but now that it is... Let's get in there and make this an even better editor!

Kudos to JetBrains and it's community!

3 Core Principles from 1998

2009-12-31T18:40:00.000-08:00

I was off for the holidays which gave me some time to clean out the storage area. I ran across some notes from a conference I attended in 1998 and 3 core principles stood out that I thought I would share as we start this new year.

Core Principles (as I wrote them many years ago):
1. Smaller is better than larger
2. Understood is better than unknown
3. Progress is better than promises

Smaller is better than larger
There was nothing else written and I left nothing by way of context... regardless of what it meant then, it is clear that this is a great principle when it comes to code. Less code that does the same amount of work is better. The paradox is that it may take a little more time to develop a smaller code solution, but it pays off. The skill is in not being so abstract and so small as to not be readable or maintainable. It is this balance that makes a true software craftsman.

Understood is better than unknown
If there is one word that we would use to describe the issues of software development... it is the word "unknown". Estimating the known is easy. Estimating the unknown, is unknowable. The skill in software development is to separate the unknowns from the knows. Estimate the knowns and provide a SWAG estimate for the unknowns... these are estimates that you need to keep in check. It is best to delay the estimates and work of the unknowns until ( what Kevlin Henney brilliantly describes as) the last responsible moment. Of course this only works for the known unknowns... what will get you is the unknown unknowns:)

Progress is better than promises
This is why I have been a practitioner and trainer for XP and agile practices for many years. It is all about developing business value on a regular and iterative basis. Promises are meaningless... progress is all that matters.

Happy New Year!!

Intellij 9 and Gradle

2009-12-14T14:18:00.000-08:00

One of the hidden gems of the Intellij 9 release is it's support for Gradle. Some of the information on the web is out of date and some features are not intuitive. This post will detail some of the nuances and follow it up with a wish list for the next update :)

Gradle Support
The information from JetBrains is out of date. In order to setup gradle after the GA release, all you need to do is configure the gradle home under File -> Other Settings -> Template Setttings.

Gradle Intellij Nuances
After setting up IDEA to work with Gradle, you'll need a build file. I didn't discover a way to create a gradle build file from IDEA. Just create a new file and name it "build.gradle" in the root of the module directory structure.

When writing the build script, Intellij expects to work with gradle in a way that is depreciated. It will pick up on a createTask(..) method. If the cursor is on a task such as this, and you hit ctrl+shft+F10, it will auto-discover and run that task in the script. (cool stuff)

Hotkeys to know:
ctrl+shft+F10 - runs task the cursor is on
shft+F10 - runs the last script (or the last run)
alt+shft+F10 - pops up a run menu (with all previously run gradle run configs)

The figure below shows an alt+shft+F10 when the cursor is on a task called task2.

If you code gradle in a non-depreciated way... then IDEA does not auto-discover the tasks. Below is what one would expect in a gradle build file:


usePlugin 'java'

repositories {
mainCentral()
}

// from the gradle user guide
// http://www.gradle.org/0.8/docs/userguide/tutorial_using_tasks.html
task hello << {     
 println 'Hello world'  
}   
task intro(dependsOn: hello) << {   
 println "I'm Gradle"  
}

Tasks would normally be created this way... It is still great that IDEA has gradle support in general regardless of this oversight. After a run configuration is configured with a targeted task, it worlds with no pain. Also you could add:

 defaultTasks "intro"

and gradle runs this as the fallback when IDEA doesn't provide the required task.

Intellij / Gradle features I'm looking forward to:

IDEA understands proper task configuration
IDEA makes it easy to create new projects with a structure maven and gradle expects.
IDEA treats gradle as a peer with Maven and Ant. This includes:
a. Gradle Window (similar to the ant build and maven projects panels)
b. Before Launch Gradle tasks in the run configurations

More Trouble with Java and Apple

2009-12-05T20:04:00.000-08:00

Well the latest update for Java from Apple came through recently. Destroying all in its path...
If you followed my advice in the past on getting Java 1.5 working on a Snow Leopard, then the new Apple update destroys that with the follow error for a Java 5 java -version:
Error occurred during initialization of VM
Unable to load native library: libjava.jnilib
Abort trap

I didn't track down the exact issue... however it is easy to detect that the Java update converts the symbolic links for Java 1.5 to point back to Java 6... Apparently Apple is NOT sorry for their crazy choice of ignorantly doing this in the first place. The solution is to completely go through the process out lined at OneSwarm again and re-establish the symbolic links. This will require you to delete the symbolic links that point to "CurrentJDK" for 1.4, 1.4.2, 1.5, and 1.5.0.

After edits an ls -l in the /System/Library/Frameworks/JavaVM.framework/Versions should look like:


drwxr-xr-x  15 root  wheel  510 Dec  5 22:18 .
drwxr-xr-x  12 root  wheel  408 Dec  5 22:15 ..
lrwxr-xr-x   1 root  wheel    5 Dec  5 21:35 1.3 -> 1.3.1
drwxr-xr-x   3 root  wheel  102 Jul 20 18:35 1.3.1
lrwxr-xr-x   1 root  wheel    5 Dec  5 22:18 1.4 -> 1.4.2
lrwxr-xr-x   1 root  wheel   14 Dec  5 22:18 1.4.2 -> 1.4.2-leopard/
drwxr-xr-x@  9 root  wheel  306 Feb 12  2009 1.4.2-leopard
lrwxr-xr-x   1 root  wheel    5 Dec  5 22:01 1.5 -> 1.5.0
lrwxr-xr-x   1 root  wheel   14 Dec  5 22:00 1.5.0 -> 1.5.0-leopard/
drwxr-xr-x@ 10 root  wheel  340 Dec  5 21:59 1.5.0-leopard
lrwxr-xr-x   1 root  wheel    5 Dec  5 21:35 1.6 -> 1.6.0
drwxr-xr-x   8 root  wheel  272 Nov  8 14:35 1.6.0
drwxr-xr-x   9 root  wheel  306 Dec  5 21:35 A
lrwxr-xr-x   1 root  wheel    1 Dec  5 21:35 Current -> A
lrwxr-xr-x   1 root  wheel    3 Dec  5 21:35 CurrentJDK -> 1.6

Happy coding!

Syntax Highlighting on Blogspot

2009-11-27T13:03:00.000-08:00

I've been very interested in providing some syntax highlighting in my blog posts, which I've been slow in general getting out... ( I have 6 or so queued up ). I finally found something I like and thought I would share the love. Special thanks to @ecounysis on twitter from whom I made this discovery. There are no excuses for my future posts... they will all have nice highlighting and be more easily copied (via the clipboard feature).

The place to start is: http://alexgorbatchev.com/wiki/SyntaxHighlighter:Hosting

From there you'll see the link to: http://blog.cartercole.com/2009/10/awesome-syntax-highlighting-made-easy.html

One important thing to note for those on blogger / blogspot. The initial update of the head tag didn't take for me. The solution for me was to add the highlighting entries at the tail end of head, just prior to the closing tag.

In addition to many of the bundled options... I've added F# and Objective-C from undermyhat.

Fixing Java on Mac Snow Leopard

2009-09-03T13:55:00.000-07:00

I have long documented some of the issues and differences for a Java developer on the Mac OS X. Most of that has been on Leopard, OS X 10.5. For quick links the most significant articles are: Java is a 2nd Class Citizen on MAC OSX, Fixing Java Memory Tools on Mac OS X and Java 7 on Mac OS X.

Monday I was faced with a new situation, Snow Leopard OS X 10.6. There are several issues with Java on the new Mac OSX 10.6 "Snow Leopard". This post will outline the known issues and will provide fixes for several of them (or provide links where fixes are already explained).

Known Issues:

Java 5 is not installed
Java 5 symbolic links are set to Java 6
jmap - some aspects of jmap do not function
javascript runtime is still missing

As a consequence, applications that used Java 5 either don't work, or provide a message as shown here from my favorite stock trading application.

* I'm still looking for more issues, so if you can add to the list, please add comments, email or twitter.
** I have tested the latest BTrace and it works fine.

Java 5 on Snow Leopard
Whither you agree with the removal of Java 5 or not, it is unbelievable to me that the Java 5 symbolic links are pointing to the CurrentJDK which points to 1.6. For those new to this space the Java version live under /System/Library/Frameworks/JavaVM.framework/Versions . I would advise you to either:

replace Java 5 using instructions provided at leopard http://wiki.oneswarm.org/index.php/OS_X_10.6_Snow_Leopard
simply delete the java 5 symbolic links

As a side note, I use a script for swapping between JVM versions. I modified the original script to just include 3 important functions when sourced in. I'll add the script to the bottom of this post as a reference. The 3 functions are jvms, setJava, unsetJava.

jvms reads through the version directory and reports what it reads of the file system.
setJava allows you to type in the name of the version and switches JAVA_HOME and path to this version of the JVM.
unsetJava reverts the set to the original java version

JMap on Snow Leopard
My initial execution of jmap on snow leopard failed with the following error message:


kensipe$ jmap 3407
Attaching to process ID 3407, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "heapOopSize" in any of the known library names (-)
  at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:399)
  at sun.jvm.hotspot.HotSpotTypeDataBase.readVMIntConstants(HotSpotTypeDataBase.java:319)
  at sun.jvm.hotspot.HotSpotTypeDataBase.(HotSpotTypeDataBase.java:88)
  at sun.jvm.hotspot.MacOSXTypeDataBase.(MacOSXTypeDataBase.java:36)
  at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:578)
  at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:499)
  at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:337)
  at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
  at sun.jvm.hotspot.tools.PMap.main(PMap.java:67)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)
  at sun.tools.jmap.JMap.runTool(JMap.java:179)
  at sun.tools.jmap.JMap.main(JMap.java:110)
Debugger attached successfully.

The good news is that the debugger attached successfully :)

Information regarding this issue is scarce. It turns out to likely be an issue with the version of Java that is distributed with snow leopard. The JDK 1.6 rev 14 appears (based on non-confirmed web sources) to have this issue. The latest is JDK 1.6 rev 16 should have the fix, however using the apple distro, we are left waiting for Apple for the fix. Although jmap fails with no arguments it does succeed for certain commands such as jmap -histo 3407.
After replacing the JDK 5 with the leopard distro as advertised above and switching to this version with the script above, jmap works fine. However jdk 5 jmap only works against jdk 5 running applications. It fails against java 6.

javascript
As record in a posting from a year ago, called Fixing Java Memory Tools on Mac OS X, the javascript library is still not provided by Apple as part of the Java distro. You will need to following the instructions there to fix it. Unfortunately the Rhino download has disappeared from the Mozilla site, so if you didn't keep a copy, ping me and I will send you want I have. You will also need to duplicate this for each version of Java if you install the Java 5 Leopard package.
The test to see if this is setup correctly for you is: jrunscript -q
If that command only returns AppleScript, then you are missing the javascript engine which is need for a number of tools in the JDK.

After these changes, it appears that Snow Leopard is ready for some hard core Java development. Good Luck and Happy Coding!

Java Switching Script that I use:



# *************************************************************************
# Originally written by Shawn Erickson - shawn at freetimesw.com
#
#
# Last Update: 2008y 08m 8d
#
# The following functions are meant for use with bash shell which is currently the
# default on Mac OS X 10.4 (starting with 10.3 IIRC) unless otherwise configured.
#  Good info on why not sym links: http://lists.apple.com/archives/Java-dev/2006/Jan/msg00290.html
#
#  from: http://lists.apple.com/archives/Java-dev/2005/Aug/msg00506.html
#  http://homepage.mac.com/shawnce/misc/java_functions_bashrc.txt
#
# *************************************************************************

### Prompt ###

export CURRENT_MODE_STRING="";

### Java Environment Functions ###

J_VERSIONS_DIRECTORY="/System/Library/Frameworks/JavaVM.framework/Versions"
J_COMMANDS_SUBPATH="Commands"
J_HOME_SUBPATH="Home"

function availableJVMs()
{
  ls -1 $J_VERSIONS_DIRECTORY | grep ^[0-9].[0-9]
}

function jvms()
{
  clear
  echo "Available JVMs: "$jvms
  ls -1 $J_VERSIONS_DIRECTORY | grep ^[0-9].[0-9]  # look to remove redundant call
  echo " "

  echo "Current Java:"
  java -version
}

function setJava()
{
  local target_jvm=""
  local jvms=$(availableJVMs)

  # Validate that the user requested an available JVM present on the system

  for jvm in $jvms ; do
      if [ "$jvm" == "$@" ]; then
          target_jvm=$@ 
      fi
  done

  if [ "$target_jvm" == "" ]; then
      echo "Unsupported Java version requested"
      return;
  fi

  # If we get here the user asked for a valid JVM, so configure it

  echo "Configuring Shell Environment for Java "$@
  export JAVA_VERSION=$@

  # First unset any current set java, back to default before doing configuration
  _unsetJava

  # Generate the paths needed for the JVM requested
  local jcmd="${J_VERSIONS_DIRECTORY}/$@/${J_COMMANDS_SUBPATH}"
  local jhome="${J_VERSIONS_DIRECTORY}/$@/${J_HOME_SUBPATH}"

  # We save the original path so we can toggle back if unset
  ORIGINAL_PATH="$PATH"
  PATH="$jcmd:${PATH}"

  # We save the original JAVA_HOME so we can toggle back if unset
  ORIGINAL_JAVA_HOME="$JAVA_HOME"
  JAVA_HOME="$jhome"

  # Update command prompt mode tag to note JVM setting
  CURRENT_MODE_STRING="J$@"

  echo "Current Java:"
  java -version
}

function _unsetJava()
{
  if [ "$CURRENT_MODE_STRING" != "" ]; then
          PATH="$ORIGINAL_PATH"
      JAVA_HOME="$ORIGINAL_JAVA_HOME"
      CURRENT_MODE_STRING=""
  fi
}

function unsetJava()
{
  echo "Configuring Shell Environment for default Java"
   _unsetJava

  echo "Current Java:"
      java -version
}

97 Things Every Project Manager Should Know

2009-08-19T12:14:00.000-07:00

The book 97 Things Every Project Manager Should Know put together by Barbee Davis is out, and worth picking up... why the magical 97? Apparently 101 was way over used... or perhaps Barbee couldn't get the last 4 in before it was time to publish :)

This book has some great insights from a number of sources, which is a huge value add to any project manager or senior developer. This is the top advice from experts in the field around team building, running a software project, and generally how to herd cats.

Most books in this space are limited to the experience of the author and his/her circle of influence. The beautiful part of this book, is the fact that Barbee has captured the most interesting and important tid bits stories of success and failures of software projects from a large sample of project managers and developers each providing a view from different industry verticals.

Well done Barbee!

Speaking at JavaZone in Oslo

2009-07-20T08:40:00.000-07:00

It's official... I will be heading to Oslo, Norway in September for JavaZone. I will be presenting Debugging your production JVM, which is talk largely focused on BTrace. Also if you didn't realize it yet... BTrace has moved to the kenai site: http://kenai.com/projects/btrace/pages/Home

If you can't be a good speaker, be a groovy speaker

2009-06-22T17:30:00.000-07:00

I will be speaking on groovy and grails at the up coming 2GX conference in New Orleans in October. If you are in the groovy space or you are just looking, you don't want to miss this... all the big names will be there. Most of the leading authors and committers in the groovy, grails and griffon space will be there!

How I got to be among them is beyond me... but it sure is groovy!

My topics include security, where the focus will include defense against hacking techniques such as injection flaws and XSS, followed by a walk through of the security plugin and how to secure urls and services. The second session is on Java memory management, the memory demands of a dynamic language, along with tools on debugging Grails applications in production.

Personal Career Guiding Principals

2009-06-11T09:59:00.000-07:00

I have received a number of requests from team members and session attendees for career path guidance. Here is what I threw together... I would enjoy others comments on what makes a difference for them!

My Guiding Principles

1. Become an expert at what is coming...
I've been lucky to live in an area of the country which is conservative when it comes to technology, which can at times be very dissatisfying. I travel to the coast (usually west) at least once per year for a technical conference. Through observation and networking, I learn the passions of the technical industry (not the vendor hype... beware the vendor hype). I make time to become well versed in these spaces. Within 1 year clients are interested and are looking for advice, within 2 they are looking for consultants. This is absolutely necessary to be a good architect anyway.
Suggestion: No one should work in our business without reading the mythical man month. Also read the pragmatic programmer or the quick guide if not the book: http://www.codinghorror.com/blog/files/Pragmatic%20Quick%20Reference.htm

2. Have the final say on matters that affect you...
Each of us is responsible for ourselves. If a company says no to training, or a conference or... that doesn't mean no to me... it means that this activity will not be subsidized. The next question to me is it worth the full obligation. There are plenty of training courses I've not only paid for, but took vacation time to be at. I currently have a nice large LCD screen, a wireless headset for my office phone and I don't use the corporate PC, I bought a MacBook Pro... all of which I paid for. I am generally unwilling to compromise working with the best tools. It shocks me in fact that Chefs bring their own cutlery, mechanics bring their own tools and software developers have better machines at home then they use at work, with an expectation that all their tools are provided for them by some company. In addition to the MacBook, I have invested significant dollars in software. Although I like open source, in many cases it is not the best. So I have TextMate, Intellij, MS Office, Keynote, etc. This wasn't as easy starting out, now frankly some vendors send me licenses now so that I'm showing their tool when I'm demoing. It is important to note that I do not accept licenses even for free if I can't stand behind the tool.
Suggestion: Invest in you! Be a master craftsman. Read: http://blog.objectmentor.com/articles/2009/04/01/master-craftsman-teams

3. Seek out Mentors and Network...
Find people who are great at what they do in an area of interest and seek them out, read their books, articles and blogs. Email them. Network with them. Obviously don't annoy them. Perhaps buy them lunch. Challenge them (for instance, "you said xyz, why would you say that?" or "why wouldn't you say yyy?"). Better yet, see what they are thinking about, what they are working on and what they might need help with. Help them! Pass them some work (if they have time), and ask their opinion.
Suggestion: Go to conferences and network.

4. Build your brand...
Create a brand for yourself. Be the Java Memory guy, the F# guy. Dive deep and market yourself. This means; a) find jobs or opportunities to work in this space, b) blog in this space, c) provide presentations in this space d) write articles or a book in this space. Your digital footprint is very important and it takes time to build out. As an example, if you search my name, the first several pages are me. This is significant. The goal here is to in a position to be hired for a gig without a resume. Those asking for you by name don't need a resume, they know what you represent. Reputation is everything! The great thing about a personal brand is it is more reasonable to change than a business brand... so 2 years as the czar of java, followed by...
Suggestion: Read Groundswell.

5. Know who you are and what makes you special...
This takes time for most people. You have to know what your talents are and what you are not good at. When you know what you lack, find teams / team members to compensate. For instance, I am a knowledge sponge and learn very quickly, but I get bored quickly. I am not as good working with a plain piece of paper, but I can perfect and analyze the dickens out of any architecture, system or code put in front of me. Take some personality tests and understand your personality. It's worth it! In the process of learning you.
Suggestion: Read Brain Rules.

6. Learn and consistently work on soft skills
Don't be the average developer... It is ok to be a geek, but be an alpha geek! Most developers will have one of two paths; PM or architect. Either way, the skills are completely different from what made them great as a developer. In the end, developing software is about enabling the business to a) make money or b) save money. Learn their language.
Suggestion: Read How to win friends and influence people and seven habits of highly effective people.

7. Be a Leader!
Leadership is earned... it isn't a title. For a technical person, It consists of having deep technical skills in at least 1 area combined with good morals. People follow people who know what the heck they are talking about and they trust. Knowledge without morals and people don't trust you. Morals without knowledge and you are a really nice guy. It has to be both.

8. Don't be Afraid:
- to say no (especially to things that don't align with you, know what you want)
- to have an opinion or stance on a subject (as soon as you do someone will shoot arrows at you)
- to change your opinion when the evidence dictates... but not based on pressure or opinion
- to make a decision (even in the absence of information)
- to be criticized
Suggestion: Read Who moved my Cheese.

9. Eat like a Bird and poop like an elephant
This is a wonderful phrase by Guy Kawasaki. Birds eat half their weight in food per day... Elephats poop... well they poop a lot. The idea is to read, listen, consume knowledge... then poop... I mean share that knowledge with anyone and everyone. I have a large library (which I read), and an account with Audible and Audio-Tech Book Summaries.
Suggestion: Read made to stick, and rules for revolutionaries.

10. Passion!
Have some... find it! There is nothing more convincing and more contagious than someone with passion. Change you, then change the world!

11. Have fun...
Life is too short

BTrace and JStat

2009-06-09T14:43:00.000-07:00

I just finished a JavaOne presentation on Debugging your Production JVM. The killer part and climax of the presentation was on BTrace. BTrace just rocks! As good as it is, the documentation and javadoc information is somewhat dated. This post will explain one of my favorite tool sets working together; that being jstat and Btrace.

JStat
Jstat is a tool that has been provided in the jdk bin directory since Java 5. It has a number features, which can be examined by using the flag -options:

jstat -options

The most significant in my opinion being jstat -gcutils which provides an output of jvm memory by compartment; survivor spaces, eden, old and perm space. Here are the steps if this is new to you:
1. start an demo application: java -jar Java2D.jar
2. get the pid: jps
3. lauch jstat: jstat -gcutil 1483
output:


S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
0.00   0.00  10.61  59.98  76.81      8    0.057     9    0.460    0.516

BTrace
Btrace is a tool that allows you to inject probes into a running java process to observe and debug an application. I won't go into "how" to use BTrace, as I've blogged on it in the past and the BTrace document does a good job of explaining it.

The area that is lacking is around the @Export annotation for BTrace scripts. There are no details in the documentation and the sample code comments says "// create a jvmstat counter using @Export"... yeah... right... Even the internal code comments and JavaDocs are just as vague. So if you are lost at this point, jstat use to be called jvmstat and was provided as part of the distribution of jvmstat. Even then there is no information on how to leverage the two... until now!!

Creating the Script
Take a look at the ThreadCounter.java file in the sample directory of BTrace. Any value you want to be exposed to jstat will need to be annotated with @Export. Then you will need to assign a value to this exported value with a static method from BtraceUtils. In the ThreadCounter example this is accomplished with perfLong("btrace.com.sun.btrace.samples.ThreadCounter.count"); The next step is to inject this code into a targeted JVM; such as:

btrace 1483 ../samples/ThreadCounter.java

Accessing BTrace exports with jstat
The undocumented trick for jstat is that you have to specify -J-Djstat.showUnsupported=true and -name with the name of the exported variable defined by the perf statement. Here is the full command-line:

jstat -J-Djstat.showUnsupported=true -name btrace.com.sun.btrace.samples.ThreadCounter.count 1483

Thanks to Sundar at Sun for the enlightenment!

Closure: I Don't Think That Word Means What You Think It Means

2009-05-03T19:15:00.000-07:00

It is curious that my weakest subject in school was English, and that I struggle so much with the miss use of certain words. Words really do matter. As words are misused, it starts to become commonplace which through a domino effect results in the complete lose of their meaning.

The word most in jeopardy today in the domain of programming is Closure.

WikiPedia has a good definition which states: a closure is a first-class function with free variables. The critical phrase is: free variables. I realize how subtle this is... however without the free variables it is just another function... and IMO it doesn't have to be a function that is just the common mechanism. Adding to the list of good examples is the docs on closures in Javascript.

Examples that are wrong or vague are commonplace. First Example: Groovy, which defines it merely as a code block... or a second example, how about Zdeněk Troníček's blog support Closures in Java which defines it as an anonymous function. A Google search will provide a fairly equal number of good and bad examples.

Defining Closure
The point to be made here is that if you pull the "free variables" out of the definition of a closure than all you have an anonymous function and the two words would be synonymous making one redundant to the other. Put another way... Closures are typically anonymous methods, but not all anonymous methods are closures.

Understanding Variable Scope
When defining a variable, our typical scopes are global, part of a class, parameter to a method, or local to a method. This would be complex if we tried to discuss this for all languages, so I will focus on Java. In Java there is no global scope, this is typically handled as a static member of a class. In this case, it lives in memory in a space called perm space. If the variable is part of a class, then the instance is part of heap memory for an instance of that class. If it is a passed parameter or a local member variable, then the storage is part of the stack frame pushed on the stack. As a reminder, as a method returns the stack frame for that method is popped off the stack and the scope has ended for local variables, references, etc. What is interesting about the proposition of a closure, is we want to be able to create a method (method 1) with some "variables" in the scope of another method (method 2). After the return of method 2, which means everything is out of scope, we still want to be able to pass method 1 to other parts of the program with the state of local variables maintaining state. Perhaps you can understand the issue now. If the scope is lost, where and how is the state of the anonymous method 1 being maintained? This is what closure is, and it is up to the language, compiler or run-time to figure that out.

I prefer to consider closures as another level of scope, as oppose to just another function.

Functional Example
It is easier IMO to see this with a functional language, so here is an example:

Function powerFunctionFactory (int power) {
int pwrFunction(int base) {
 return pow(base, power);
}

return pwrFunction;
}

Function sqr = powerFunctionFactory (2);
Function cube = powerFunctionFactory (3));

sqr (3);
cube(3);

Looking at this example when the function sqr is defined by invoking powerFunctionFactory(2), the scope of the value of 2 should be lost through normal scoping rules, however due to the feature of closure it is not and this technique is possible for a number of languages. Not only must it maintain state, but it must do it for each instance of closure. Notice that the cube function has a power state all it's own.

In closing I just ran across this great post by Neal Gafter, which is titled the definition of Closures. I didn't get to it in time to reference his material, which looks great. Perhaps there will be a follow up.

If we can't agree on the nature of closures containing variables... we will never understand monads :)

Speaking at JavaOne 2009

2009-04-30T08:17:00.000-07:00

Speaking at JavaOne this year, on Debugging Your Production JVM. The session is on June 3, 2009 at 4:10 PM. This session will cover a number of great JVM tools for debugging and managing a run JVM, with a clear focus on BTrace. This is the 3rd time speaking at JavaOne (2001 and 2003 being the times of past). With the Oracle buyout, this could easily be the last for all of us. Welcome to Oracle OpenWorld! The question which remains...

What will happen to Duke?

Presentation Worst Practices

2009-04-29T21:10:00.000-07:00

With all these "Best Practices" floating around... I was wondering. Is anyone documenting the worst practices? One Google search later, I discover that not only is there a number of them, but the domain name is taken by someone just parking it... boo!! Here are a couple of my favorite:

Defining Worst Practices - in summary says you are violating a principle or natural law.

Scalability Worst Practices - brilliant... a fun read. Golden Hammer, Big Ball of Mud.

REST Worst Practices - Also a good read. Summary... Rest fails with highly de-normalized systems... oh and don't hard code stuff.

and my favorite worst practices for this post:
Presentation Worst Practices

Happy Coding!

Building Spring 3 working with Git

2009-04-27T16:02:00.000-07:00

Until today, I've been keeping up on the Spring 3 development off the svn repository with svn. There is a great post by Chris Beams from SpringSource that provides adequate detail and is a great resource to read as a precursor to this post. In general, I'm moving all my development over to Git, and it was time to get Spring 3 inline. It was not without pain. For those who just want a quick reference, I will provide a quick step by step on how to achieve the end goal of setting up a git client to the spring 3 svn repository. Following that I will provide details for those who want a deeper understanding.

Quick Step-by-Step Spring 3 with Git

In the directory you would like the spring project execute : "git svn clone --stdlayout https://src.springsource.org/svn/spring-framework"
change directory to the newly created spring-framework directory
copy forked script from github.com/kensipe named git-svn-clone-externals to the current directory; make executable and run.
run "git svn show-ignore > .gitignore"
run "git gc"
from the current directory you'll need to create the following directories

mkdir org.springframework.instrument/src/main/resources/
mkdir org.springframework.instrument/src/main/resources/META-INF
mkdir org.springframework.instrument.classloading/src/main/resources
mkdir org.springframework.instrument.classloading/src/main/resources/META-INF
mkdir org.springframework.asm/src
mkdir org.springframework.asm/src/main
mkdir org.springframework.asm/src/main/java
mkdir org.springframework.core/src/main/resources
mkdir org.springframework.core/src/main/resources/META-INF
mkdir org.springframework.expression/src/main/resources
mkdir org.springframework.expression/src/main/resources/META-INF
mkdir org.springframework.web/src/main/resources
mkdir org.springframework.web/src/main/resources/META-INF
mkdir org.springframework.orm/src/main/resources
mkdir org.springframework.orm/src/main/resources/META-INF
mkdir org.springframework.context.support/src/main/resources
mkdir org.springframework.context.support/src/main/resources/META-INF
mkdir org.springframework.web.portlet/src/main/resources
mkdir org.springframework.web.portlet/src/main/resources/META-INF
mkdir org.springframework.test/src/main/resources
mkdir org.springframework.test/src/main/resources/META-INF
mkdir org.springframework.integration-tests/src/main
mkdir org.springframework.integration-tests/src/main/java
mkdir org.springframework.integration-tests/src/main/resources
mkdir org.springframework.integration-tests/src/main/resources/META-INF
mkdir org.springframework.instrument/src/test/java
mkdir org.springframework.instrument.classloading/src/test/java
mkdir org.springframework.asm/src/test
mkdir org.springframework.asm/src/test/java
mkdir org.springframework.aspects/src/test/java

export ANT_OPTS="-XX:MaxPermSize=256m -Xmx1024m"
change directory to "build-spring-framework" directory
run "ant"
wait 20 minutes :)

The background and detail
The first line is standard git-svn to get the repository. If you are new to Git, this will take a few minutes long then a standard svn checkout. It is copying all the changes throughout history to your local .git repo. Fortunately spring 3 is still young from a history standpoint.

svn external - The first issue
The first thing to note is that part of the build process for spring 3 is a project which is added to this svn project as an external, which led me to this post on why svn externals are evil :) evil or not we have to deal with them. This led to a post by Alieniloquent which has some good explanations, followed by what I eventually settled on, a post by Andre Pang, which includes a script he created, which I forked off github. Kudos to Andre!! The script he created (which I can see using again the future), provides a one execution solution to what Alieniloquent blogged. It is beautifully simple. The script provides the following:

pulls down all the svn externals (spring 3 only has one)
creates a symlink (mocking what svn would have done)
adds the symlink and the .git_externals directory to the excludes file

If you wanted to check your svn externals for other projects use the following command: "git svn propget svn:externals ." in the root of the project directory.

The next step is to get all the svn ignores into the .gitignores file.

version control directory management - the next issue
The explanation behind this issue is that svn allows for and manages empty directories... git does not. git only manages content. There are a number of empty directories which the spring build files expects to be present for the build to succeed. This is based on a standardize build system and a lack of error handling in the build system. If I get some time, I'll send a patch to the springsource guys to fix this. At this time you will have to create all the directories listed.

jvm out of memory issues - the last issue
ant will run out of memory without these changes.... first you will run out of perm space, followed by heap. You may be able to get by with less; I didn't spend much time here. The standard defaults will fail.

Also if you need to update your master branch with git... "git svn rebase" will do the job!

I need to say thanks to those bloggers already noted and to fellow NFJS speaker Matthew McCollough.

Justification for Functional Programming

2009-02-26T13:27:00.001-08:00

It appears that this subject has caught fire in recent years. The interest in St. Louis has resulting in the Lamdba Lounge started by Alex Miller. (Great name by the way). I have started reading (but not finished) Programming Clojure by Stuart Halloway and Programming Scala by Venkat Subramaniam. Additionally I just finshed writing a 10 page article on an introduction to functional programming, which includes 4-5 page on F#. The article is for NFJS who holds the exclusive rights at this time, but I plan to provide a link to it or post it here in its entirity after 90 days.

I should point out that I am no expert in this space, but I have been developing professional for 15+ years with a variety of languages (C, C++, Java, C#, a little Ruby, Perl, Groovy). In the last 6 months I've been "playing" with Scala, Clojure and F#.

The reason for this post is this: http://groovy.dzone.com/news/why-functional-programming. This article has been sent to me a couple of times and has been on twitter several. At this point I will make public comments I made in an email response regarding this subject. The first two subjects were points made via email, which make sense to maintain for this blog post.

The Argument to have UI and DB Support
I don't get the ui / db argument. This indicates an all or nothing mentality, which I would hope the industry could get over. A griffon front-end to a java hibernate db access and clojure rules engine make sense to me. Many of the functional languages provide the ability to produce a UI or interact with the DB, however that isn't really important to me as you will see below, but there are good solutions out there where this isn't really a concern.

Functional Notations and Code Beauty
I may be in the minority... but I could care less for "beauty" of code, I'm interested in less code that adds the same or more value. FP adds several notations that provide a conciseness to code, which is a huge value add. As I was writing up an article on F#... I became frustrated with my other programming languages... for instance: why is a switch in Java/Groovy so limiting. Groovy makes it better... but it is still limiting as you compare it to discriminated unions and pattern matching F#.

Getting Functional Programming
People don't get it yet (referring to the nah sayers)... they are in the battle pits and are lacking the big picture vision. This year quad core laptops are expected to be more common. That trend isn't going to stop. The last decade was all about die size and memory. This next decade is all about cores. The need for concurrency tools is a must. The language trend may be slightly ahead of its time.

I'll post more on this subject in the future... I just felt it necessary to provide some response to the value of functional programming. Frankly I've barely scratch the surface of the value.

Java 7 on Mac OS X

2009-02-24T15:27:00.000-08:00

This blog post is worthy of Retweeting:) http://infernus.org/2009/02/building-java-7-on-mac-os-x/

In 2 weeks when I hope I have time... I will be on this!

BTrace Fix on the Mac

2009-02-24T11:40:00.000-08:00

The last blog post showed a work around for getting BTrace to work on a Mac, however as explained it fails when trying to probe Grails. I've been working with the BTrace developers and now have a solution. It will likely be some time before the next release. So for those who don't want to build from source, I put a drop on drop.io of a build.

Side notes:
1) It appears that all the scripts and assumptions for Java do not take in account the Mac. Groovy, Grails and previously BTrace all have references to a tools.jar in a place which is not correct on a Mac.
2) drop.io is new to me... and it appears worth the look. I will likely use drop.io in the future. Mostly for presention code drops. I like github for source references, but sometimes it is just quick and easy to tar or zip up what you have and drop it.
3) There is an interesting situation with BTrace which can be frustrating if you are not aware of it. If you have a failed BTrace session... such as a connection refused on the BTrace side, then it is not possible to BTrace it again with a fixed version of BTrace. You have to restart the application to be probed.

Fixing BTrace on the Mac

2009-02-22T16:05:00.000-08:00

Well it is time to fix another Java issue on the Mac... again... Sorry for the sarcasm. I love Java and Groovy and I love my MBP, but as already documented (on fixing Java Memory Tools and Java is a 2nd Class Citizen), these two don't get along as often as I would like. I thought for some crazy reason that I was the only one fighting this issue until all the comments from the fixing the tools article appeared. Glad to know, first that this blog is helpful to others, second that there are number of great people all over the world (offer drinks no less) and third that we are not alone in this fight.

BTrace and the issue
First... if you don't know what btrace is... you have to check it out. In the evolution of debugging tools for the jvm, it is the next big thing! Active in this space is A. Sundararajan, he has a couple of my favorite blogs, one on scripting btrace and another on jmx and btrace. Another great tutorial blog is by Igor Minar.

This blog isn't as much about btrace, as it is about fixing it on the mac. BTrace is new enough though, that it may require just a quick introduction. BTrace is a debugging tool, which injects "probes" (my term) into a running Java process. It is an open source option that has a high possibility of replacing Wily's Introscope. To be fair, Introscope provides a lot of extras which BTrace doesn't at this time... but Introscope is tens of thousands of dollars and BTrace is a free open source tool. I would be shaking in my boots if I was on the Wily team.

We will need to start a Java process in order to illustrate how it works (or in the case of the Mac... doesn't work). For the Java process look in the demo directory, there is a Java2D.jar. On the Mac it is located: /Developer/Examples/Java/JFC/Java2D. At the comand-line type: java -jar Java2D.jar. This will start up a Java process. To get it's pid at another command-line type: jps
If this is the only Java process running on your box, the pid with the jar notation is the pid you are interested in. If you want to be 100% certain type: jps -l . You'll get something like:
7897 sun.tools.jps.Jps
7838 Java2D.jar

Now in the btrace bin directory (I'm assuming you followed one of the referenced tutorials), for our example type: ./btrace 7838 ../samples/ThreadCounter.java . To which you will get:
Connection refused

In the terminal of the running Java process for the jar you would likely see:
btrace DEBUG: adding to boot classpath failed!
btrace DEBUG: java.util.zip.ZipException: error in opening zip file
java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:114)
at java.util.jar.JarFile.(JarFile.java:133)
at java.util.jar.JarFile.(JarFile.java:97)
at com.sun.btrace.agent.Main.main(Main.java:108)
at com.sun.btrace.agent.Main.agentmain(Main.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)

Yea... epic fail! That is BTrace failing on the mac. If you want more details, go to the btrace script and edit -Dcom.sun.btrace.debug=false to be true. This will indicate the following stacktrace:
btrace DEBUG: debugMode is true
btrace DEBUG: dumpClasses is true
btrace DEBUG: dumpDir is .
btrace DEBUG: probe descriptor path is .
btrace DEBUG: parsed command line arguments
btrace DEBUG: System ClassPath: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/tools.jar
btrace DEBUG: adding to boot classpath failed!
btrace DEBUG: java.util.zip.ZipException: error in opening zip file
java.util.zip.ZipException: error in opening zip file
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:114)
at java.util.jar.JarFile.(JarFile.java:133)
at java.util.jar.JarFile.(JarFile.java:97)
at com.sun.btrace.agent.Main.main(Main.java:108)
at com.sun.btrace.agent.Main.agentmain(Main.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:348)

The line just above the fail point... hmmm.... it is looking for a tools.jar in the $JAVA_HOME/lib directory. As already noted, Apple likes to slice and dice the JDK... In trying to understand the issue, I uncovered an article released by Apple which says there is no tools.jar file... no tools.jar file!!! It is a classes.jar file and to make it fun it isn't in the $JAVA_HOME/lib directory... no, it isn't even under a subdirectory of $JAVA_HOME... I can't even write that without frothing at the mouth. I could start a rant here... but I'm assuming I'm writing to the choir so to speak.

Fixing the Issue
I didn't spend a significant amount of time on it... but I didn't get this working by changing the btrace scripts... something is in the code. I have another recommendation towards the end that the script change wouldn't work for as well. So we are left currently with the following work around.

Take the classes.jar and copy it to the lib directory as tools.jar (argghhh!!!) but it works. To be more specific... /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Classes/classes.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/tools.jar. It is important to note that you have to be sudo to make this change and you will have to do this with each Java version you want this to work with. Let's hope that the next version of BTrace will be Mac friendly!

After you get that working... checkout visualvm with the BTrace plugin. It is great stuff and another reason that the script change alone is not a total solution.

Other Issues
OMG... I'm frick'n shaving the yak again... just as I'm getting ready to post this... I was about to trace a grails app. The changes described above, break Groovy and Grails. So pick one :) You can be Groovy or you can BTrace but you can't have both yet. I guess I'll begin digging through code and send a patch tonight...

Speaking on the NFJS Tour 2009

2009-02-19T12:20:00.000-08:00

I will be speaking at a number of shows with NFJS this year. Already scheduled is Milwaukee, St. Louis, Minneapolis, Boston, Seattle. I will likely be at Omaha, Atlanta, Denver, Columbus, and Raliegh. That's just the first several months :).

It is always around this time of the year I like to revisit my goals. Commitment comes easy around January of a new year. Around mid to end of Feb, is a good time to check to see which goals you are seriously committed to. One commitment, I've maintained for several years is a commitment to learning and a commitment to excellence. This is easier when surrounded with like-minded and committed individuals who help to hold you accountable. It isn't always easy... just worth it. That is one of the reasons I love NFJS. The caliber of the speakers are top notch and world class.

For some the economy will be a an easy excuse to not attend. Or perhaps an employer will not pay for it this year. What is clear to me is those with the skills and the networks will find it easiest to find the next opportunity. This is the time to pick up new skills. As a local venue, no show makes it easier or less expensive than NFJS.

See you there !

Intellij 8.1 with Git Support

2009-02-13T11:46:00.000-08:00

Finally... a good java editor with good git support. I've been waiting for a long time (in internet time) for git integration in the editor. Download Intellij 8.1 for Git support.

I have a number of project demos which are already in Git. I expected that Intellij would just pick up on this fact... and it did not. I then created a new project, did a Git init from Intellij and all was good.

To get IDEA to recognize a project already in Git, under the Version Control menu, select Enable Version Control Integration, then Git.

So far... the Git integration is fantastic!!! Love it! Well done JetBrains!
Happy Coding

Java Memory Management Details

2009-02-05T07:17:00.000-08:00

Needing to gather some details to help answer a question regarding JVM memory and the the distinctions of Perm Space. I thought it might be helpful to post the links of what I thought were incredibly great posts from the past... It is now 2009 and the best posts on this matter appear to come from 2006. Hmm.... perhaps there has been a JDK release in some time :)

On the subject of; the reason the permanent space exists. Jon explains the details of instances of objects and their dependencies on instances of classes and the need for their order as it relates to garbage collection. He also discussions the klassKlass... interesting stuff.

On the subject of String intern, XML parsing and it's effect on Perm Space. haakon or what I assume to be Jenny details the effect of String.intern() and how an out of memory can occur. She does a great job of explaining their troubleshooting approach.

And my favorite book on the JVM spec is Inside the JVM. It turns out there are free chapters of this on the web now and chapter 5 is the chapter detailing the taxonomy of the JVM memory space.

The weakest coverage of any subject based on my searches, is the subject of root sets. Brian Goetz's article on Java Theory and Practice is rock solid.

Java is a 2nd Class Citizen on MAC OSX

2009-01-26T12:45:00.000-08:00

Having made several comments over the last several months, and in particular over the last couple of days regarding Java on the Mac, several people have asked me to explain myself. So here goes.

When I first moved to the MBP as my primary development machine, I was very pleased. As a developer, I was amazed at how all my development tools were just there. Ruby, Rails, cvs, subversion, 2 Java JDKs, it just worked out of the box.

When I first discovered the difference
I have a standard demo I use when show other developers some of the nifty debugging tools like visualgc or visualvm. In order to show the tool off, I need to start another Java process. So I standardly go to the demo directory and start the Java2d.jar. Oh but wait... where is that on the mac. This lead to the discovery that the jdk is in several different directories /locations.

JAVA_HOME = /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
Demos: /Developer/Examples/Java/
Bin:/usr/bin/java
Headers/Lib: /Developer/SDKs/MacOSX10.5.sdk/System/Library/Frameworks/JavaVM.framework/Versions/xxx
Java Lib: /Library/Java/Extensions; /usr/lib/java
Endorsed Dirs:$JAVA_HOME/lib/endorsed
System jars: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Classes

OK... I'll live with that. I quickly discovered a very useful scipt for switch between JDKs. I've modified the original if anyone interested let me know. But since we are on the subject of being a second class citizen, If you go out to the Sun site and look for the latest jdk, the options are windows, linux, and solaris.

Other Issues in Jakarta
Then the next issue came up... jhat was broken. I wrote a lengthy blog post on how to fix this. The issue... Apple didn't include the javascript libraries, which are necessary for these tools to work.

So now I'm taking a serious look at btrace, which is a great tool. Just trying to do the simple stuff, works fine on Windows... but no joy for the Mac. I'm still looking into it, but it appears that Instrumentation..appendToSystemClassLoaderSearch() fails on the Mac. Could be something else... still looking.

If you are doing standard Java development or Groovy and Grails... you may never notice the Mac difference. When you venture into the debugging and instrumentation realm of Java be prepared for some frustrations. As I get a solution for the btrace issue I will post it.

On the positive side, I enjoy the development experience on the MBP better than my experiences on Windows and linux. The memory management is better and the startup times on Java processes is fantastic.

JSR-299 Web Bean Review

2009-01-24T13:01:00.000-08:00

As several JCP specifications are reaching there final stages, I decided to have a look at two of them. Truth be known, I actually was going to look at one of them jsr-299, the specification previously known as web beans, here forth now known as Java Contexts and Dependency Inject. There was sufficient enough mention of ejb-lite that I decided to have a look at its specification, jsr-318.

Before I get started, my position on the subject of ejbs should be known. I actually thought that ejbs were on the way out... with the lack of testability and the extra unnecessary complexity for most projects prior to ejb3. EJB3 greatly improved this, but IMHO was too little too late. Spring + Tomcat already provided what folks were looking for. Additionally the number of clients using JSF is a dying breed. Once again, JSF 1.0 missed the boat with non-bookmarkable pages, etc. While there are shops sticking with these Sun driven standards which are produced out of the JCP, the majority of support appears to come out of vendor interest and not developer interest. My earlier work on projects with JSF led me to appreciate a couple of frameworks; 1. facelets, 2. a4jsf, and 3. SEAM (when working on jboss). jsr-299 (web beans) seems to be the formal standardization of SEAM.

My thoughts on jsr-299
It appears that @ annotations are the new shiny toy, and there is no end to specifications going crazy with them. jsr-299 defines 34 new annotations. The goal of the specification as defined on the spec home page, is to unify JSF managed bean component model with the EJB component model... well I've stated my position on this. So either this specification is too little too late and a benefit mainly to jboss, or perhaps there is something to be gained beyond this spec stated goal.

There were several interesting ideas out of the specification. The first was the creation of an extensible annotation model. So if there are not enough annotations in the world for you, you can create your own that work with the framework. This is accomplished through the new javax.inject.@BindingType. This appears to provide fine grain autowiring qualifier. One of the examples given defines
@LDAP
class LdapAuthenticator
implements Authenticator { ... }

followed by the declared injection of an LDAP Authenticator:
@LDAP Authenticator authenticator

Another interesting idea is producers... I really like the concept here. Page 3 and 4 illustrate an example where a @SessionScoped @Model Login class has a @Produces @LoggedIn User getCurrentUser() method. Abbreviated below:
@SessionScoped @Model public class Login {
@Current Credentials credentials;
@PersistenceContext EntityManager userDatabase;
private User user;

@Produces @LoggedIn User getCurrentUser() {
if (user==null) {
throw new NotLoggedInException();
} else {
return user;
}
}

Then the getCurrentUser() method is invoked for the inject as listed below:
@Model
public class DocumentEditor {
@Current Document document;
@LoggedIn User user;
@PersistenceContext EntityManager docDatabase;
public void save() {
document.setCreatedBy(currentUser);
em.persist(document);
}
}

The last interesting part of the specification is the event framework. I don't understand why this is in the specification. There appears to be no dependency for the event framework on the other parts of the specification. It doesn't comply with the stated goal of unification of JSF and EJB. It would make sense for this to be separate. Mainly to use it separate from all other aspects of web beans.

Thoughts on jsr-318
I did not review the entire specification yet. I was checking out some of the mentioned ejb-light details mentioned in jsr299. As mentioned, outside of being required by clients to help them with ejbs or ejb light projects, I'm really not interested. There is however one very interesting addition to the ejb specification, that is the @Asynchronous annotation. This is long over due. In addition to its obvious benefits, a method annotated as asynchronous can also return a value defined as java.util.concurrent.Future where V is the return value type. The return of the Future object provides the ability to check status (isDone()) or to cancel the request. Very cool. For a quick list of EJB 3.1 changes, checkout Ken Saks blog.
The concept of EJB-lite is lost on me... I already have that it is Spring.

Summary
If you are looking to stay in the world of EJBs and JSF, then these newly defined specifications are of great value to you and your shop. jsr-299 fills a number of gaps in the jee space and provides the power of Spring in a standardized way. Clearly jsr-299 will be part of the new jee 1.6 specification, at least in several of the profiles. If you are not using either JSF or EJB or nether, then it is unclear what value this brings to the community. It will be interesting to see if a vendor implements this specification (or a portion of it), in a world without JSF. It appears that the jsr-299 specification helps vendors more than it helps developers.

Windows 7 Download Frustration

2009-01-11T04:17:00.000-08:00

The danger in posting this blog entry is that I will be counted as a anti-Microsoft person... which is just not true. I happen to like using the best tool for the job which is not all the time from Microsoft (and sometimes it is). In 2008 I switch my primary system to an Apple Mac. There were many factors as to why and frankly I love it! But I'll always be a technical junkie.

With a number of friends (such as Denny Boynton and Ted Neward) posting about their experiences with Windows 7, I thought it was time to take a look. I thought it was going to be a matter of download, get key, start up vmware fusion and I way I go... but I was wrong.

First I went to the public beta site: http://www.microsoft.com/windows/windows-7/beta-download.aspx and selected the 64-bit version in english and got this. WTF?? Repeated attempts resulted in the same. An oops page with a pre-canned search. Where did I go wrong? Well as you can tell, I'm on my Mac. So I pulled out fusion to launch Windows XP for round 2 of the attempt.

I thought this is just wrong, but determined to get a look, I switch to windows and my suspicions were confirmed when I got one page further. I got the download page with a couple of large buttons on the bottm of the page and one read "Download Now". Hey, that's what I want... I want to download now. I clicked the button and... nothing. Click... Nothing... No way... they didn't. Round 2 was in XP, but with firefox.

Round 3 as you would expect is XP with IE. That combination was successful and I'm now 29% into my download.

BTW... In the process of testing a few more times in writing up this blog, the round 1 mac failure was fixed to the point where you will get download page (nice response time msft), however the download button fails.

Why is it necessary to be like this? Why is it so hard to put up a link to a download which is platform neutral? Wouldn't Microsoft want to attract customers from other platforms? Does it always have to be all or nothing?

Speaking at Code Mash 2009

2008-12-23T09:08:00.000-08:00

I will be speaking on Spring MVC. The title says Spring 2.5 MVC... But I'm throwing a curve ball... it will be Spring 3.0 MVC. See you there!

Complexity and Architecture

2008-12-22T12:40:00.000-08:00

It has been too long since I had a post... I have a number of scribbled notes of blog ideas, but little time. I'm in the process of starting to write the spring 3 book, which may take me off grid more than normal. I will make an effort to post a couple of times a month (more if there is something cool to document). Currently my mind has been on 3 subjects; Spring, Architecture and languages. This blog is regarding one of the often forgotten concerns of architecture... complexity.

Defining Software Architecture
I have been in a large number of conference BOFs this year. When the audience is slow in starting a conversation, the question we gravitate to is: "What is a Software Architect?" or its keen "What is Software Architecture?". I guess I shouldn't be surprised, but it is amazing how diverse the answers to these questions are. It is a reflection of at least two things... 1) the immaturity of our industry and /or 2) the lack of standardization in this space. The conversation spans from design decisions to application architecture to enterprise architecture.

Martin Fowler defines architecture as "things that people per-ceive as hard to change". This is a really responisble definition.

I like to define it as "making decisions on all the non-funcational requirements" and "managing all the things you can't control"

The area of non-functional requirements would include:

User Experience
Performance / Response Time
Scalability
Security
Frameworks
Complexity

In the category of things you don't control:

On public web: Number of Requests
Number of Threads / Connections
Number of returns from a user generated search
Items in a batch

Complexity in Architecture
The subject that is rarely vocalized and debated in architecture is that of complexity. Neal Ford, in a couple of his talks with NFJS, talks about principles that were documented by anciet philosophers... that of essential and accidental properties. He carries this forward into software architecture in a way that we as an industry really need to talk about. That of software architecture decisions moving the enterprise forward in a way that has essential complexity or in a way that increases the accidental complexity. Neal further clarifies the point with the following comments... We have a hard problem (Essential complexity) and we have made our problem hard (Accidental complexity). Well done Neal! Very easy to envision.

Is complexity one of the non-functional requirements which your architecture team is looking at? If not, It seems likely that what is left is accidental.

What is a concrete example of this?
Let's say that your enterprise has just purchased a shiny new ESB. Suppose there wasn't a project need for it... it was just the stated SOA direction of your organization. When down from on high the edict is brought to your project that you must use the ESB in order to justify the $1M expense. Perhaps it is a fit... but perhaps you are about to fit the square peg in the round hole... in other words accidental complexity.

Where does Accidental Complexity come from?
Neal would say that it comes from vendors who are the pushers of accidental complexity... however there is something deeper here. The reason they are referred to as the pushers is that they are incented to sell you a solution. The solution may not fit your needs exactly, but hey, if they can show you a vision of how it could help, then there is a framework to following and life should be more simplified. So essentially we are saying that any choice which doesn't provide the best solution brings complexity.
We could also say that the more stuff / more solutions we have brings complexity. So by adding more frameworks and options we have added complexity. Conversely, many organizations fall into the trap of standardizing certain aspects of development... like everything is an EJB or we don't use store procedures here. In this situation what happens is that the architect and developers will have to come up with a more complex solution in order to be compliant with the corporate standards.

I'm open to suggestions and challenges... but I think we can conclude that much of our accident complexity comes from not applying the best solution to the problem. I would expect that most teams would not do this deliberately. So the team is either 1) ignorant of the best option (which can be fixed through experience and training) or 2) it is forced upon them.

I will close with the following challenges and questions:
Next time you are looking at a software component... Ask yourself the following
1) Does it simplify the problem?
2) Is there a more simplified solution?
3) Is it testable and verifiable?
4) Does it add complexity upstream or downstream of the problem?

When evaluating solutions... try to put a value on the level of complexity. Here are some ideas:
1) If it moves code or configuration into a proprietary solution then increase its complexity score.
2) If it allows you to write less code for the same functionality then decrease its complexity score.
3) If it enhances testability, decrease its complexity score.
4) How easy is it to monitor and maintain?
5) How easy is it to integrate with and is it standards compliant?

Good luck and Happy Holidays!

Boulder / Denver JUG Talks

2008-11-13T07:02:00.000-08:00

It seems I can't escape Denver. I was there last month at the JUG meeting. Matthew McCullough has some commentary on his blog. It seems Matt did a presentation on iPhone development this month. I will be speaking at NFJS this weekend in Denver. My topics include:

- Java Memory Management and GC
- Hacking - The Dark Arts
- Security Code Review
- Spring / JPA
- Spring 2.5 - Spring without XML
- Architecture and Scaling

Favorite Development Tools of 2008

2008-10-20T20:18:00.000-07:00

Mac
TextMate - What a great tool! Reason enough to own a Mac.
Intellij - Version 8 is looking good.
Groovy - This is becoming my favorite tool.
Grails
git - It is replacing cvs and subversion.
gradle - replacing ant and maven
hudson - looking to see if it replaces cruise control.
Fusion - developing C# is more fun on a Mac.
Spring 2.5 MVC - If you can't do Grails... this is the way to do it.

Tools I'm looking forward to in 2009

Clojure - Interesting Language... I know very little at this point.
F# - Probably the most interesting language on the .Net platform.
Spring 3 and what I call ADD or Annotated Driven Development :)

Tools That need some TLC

I would like to see a refresh on xplanner - This would be a great candidate for Grails.

jps - The Java way when you need to kill something Groovy

2008-10-20T10:41:00.000-07:00

As a speaker, I'm often showing "how" to do something and often requested for "When" / "Why". I just experienced a great example of this with jps. jps is of course is the Java ps tool that is included with JDK5 and 6. So it is already on a Java developer's machine. It is the platform independent way to get PIDs.

How it Started
The Yak-Shaving started when I decided to run a groovy script from TextMate. I conventiently hit +R. Everything ran the way I wanted it to. Except... There wasn't an end to the program script... idiot! So, I closed TextMate... Still Running....

Step 1
First I went to the command line and did a ps... which didn't result in much. I followed this up with a ps -ef. Way too much info... so I ended with a ps -ef | grep JavaVM. Things just got worst. That is a lot of information!

Solution
Along comes jps. A simple jps (no args) and here are the results
9382
10620 GroovyStarter
10314 JConsole
10761 Jps

There it is GroovyStarter... kill 10620.

... and what if you had multiple GroovyStarter scripts running. Well, jps -m will give you the command-line of the script, which includes the groovy script name.

Grails JMX Plugin

2008-10-19T14:15:00.000-07:00

I just created and released version 0.3 of the Grails JMX Plugin. The code is at github; git://github.com/kensipe/jmx-grails-plugin.git . The documentation is at grails; http://grails.org/JMX+Plugin.

Sometime this week, I'll post some tips and tricks for using the JMX Plugin with Grails.

Cooking Chef vs. Polygot Programmer

2008-10-18T11:23:00.000-07:00

Occasionally crazy analogies pop up in my head. Sometimes these ideas don't make sense to anyone but me. Let me know if I've gone off the deep end.

So I was thinking what it was like growing up when I didn't have a microwave and how useful that tool is. Then it occurred to me that as useful as it is, there are things I would never put in the microwave. Take for instance a turkey. Perhaps it is possible to cook a turkey in a microwave, but why? You can cook toast on a gas stove as well, but a toaster just seems more appropriate. When playing chef, each tool provides a specific value. Turkey in the oven, toast in the toaster, and butter melting in the microwave.

Stretching this thought into a conversation on technology, I'm reminded of a term Neal Ford is credited with: The polygot programmer.

Why is it that developers and programmers pick camps? Why does an organization limit their technology choices to one or a small handful? When an organization says they are a "Java Shop" or a ".Net Shop", what I hear is "We only use microwaves to cook here". In many of these shops they are doing with code what can be equivalent to cooking a turkey in a microwave. They can get it to work, but it doesn't taste so good!

Writing a Spring 3 Book

2008-10-03T07:15:00.000-07:00

I'm looking at writing a book on Spring 3. Yes... I know, it doesn't exist yet. My expectation is it should have a likeness to Spring 2.5, which is also lacking in good printed materials. With this assumption, I'm looking to get a head start now and refactor based on actual delivery of product.

If you have some ideas or thoughts... please share. At this point, My goal is not to cover what is already out there. I'm focused on the new approach of annotated development.

I'm also thinking of taking this book and writing a Spring.Net book to follow... Perhaps I will change my mind as I go through the new experience of the first book.

Speaking at October IASA Meeting in St. Louis

2008-10-03T07:08:00.000-07:00

I will be speaking on Architecture and Scale at the Oct 6 Meeting of IASA in St. Louis.

The Need for Better Interfaces (Human to Computer)

2008-08-28T09:33:00.000-07:00

I can't help but believe that there has to be a better way. There has to be more options...

I talking about the I/O options to my computer. Here are a couple of recent stories that will hopefully bring context to what I mean.

Gestures on the Mac
Recently I purchased a new MacBook Pro. You know, one of the new Macs with the multi-touch capabilities. The idea is that if you use one finger you move the mouse, if you use 2 fingers it scrolls (and not just up and down), if you use 3 fingers then it depends somewhat on what application you are using, but generally it means go back or go forward depending on the wave of the hand. This is just awesome stuff and here is what happened to me.
First, the back and forth action of 3 fingers worked in Safari and in finder.... but not in Firefox. I was a little more than dismayed. Generally preferring Firefox, I found myself using Safari more in order to use the gestures. I really wanted this new feature in all applications. Eventually the shiny new toy lost some luster and I mostly use Firefox again. The consequence... sometimes I forget the gestures work in the finder... and I know that I would use it all the time if it always worked. In other words if the bell always rings and I can trust it, I will salivate. And I want to.

A PC with No Gestures
So eventually I found myself working again on another laptop. Man did I look like an idiot for a few minutes. I come to a page that needs scrolling, naturally I reach up to the touchpad with 2 fingers... moving it back and forth to get a scroll thinking for a moment damn computer is locked up or slow... only to discover that I was the component with the issue. It didn't help that my daughter was staring at me funny wondering what I was doing. She was still laughing even after I explained myself.

iTouch as a Birthday Gift
So last week was the same daughters birthday. I bought here a new iTouch. She loves it when I let it leave my hands for her to use. The first couple of days were great. She asked me "how do you turn it on?". I said, "It has one button... push it!". She would ask, "How do I get back out of this application?". I would say, "It has one button... push it!". And of course, she quickly gain the necessary skills and probably has surpassed my abilities on the thing within a few days.

PC is Not an iTouch
So last night she was asking me for some sync help between the laptop and iTouch. After a little conversation, she understands what she needs to do... she needs to hit the sync button on the laptop. What does she do? She taps the screen on the laptop... not once, but a couple of times and then begins to laugh. It was funny.

MacBook as a Fish Tank
Last example. I love to scuba and I love the ocean. When I saw the aquarium screen saver for the mac, it was an obvious purchase. So there is a setting on the screen saver, which reads "Enable Motion Sensor Water Tilting". First you have to turn this on! It uses the motion sensor on the mac so that if you tilt the laptop it appears that physics has applied forces on the water in your laptop the way you would expect. The water tilts! Here's the deal: The optical illusion is so good that even really savvy software developers are impressed and want to know: How?

Call to Action: Better Human Interfaces
If you combine the above stories with the fact that I just finished reading "Dreaming in Code", which outlines how far we haven't come in software development for the last half of a century, I'm left restless. There has to be more... there has to be better... Meaning more we can do and better ways of interfacing with computers. We are just beginning to see it. When you look at Microsoft Surface, Apple's iPhone and the promise of RFID, you begin to see it. It isn't what the general software developer is being train on or developing... They are still working on Dialogs.
Those who claim to be on the cutting edge, they are working on web 2.0 with javascript and cool new frameworks (they are cool). But that isn't what the future demands. The future demands seamless integration computer to human... where the human gestures to something and the machine understands. Where the computer knows what is expected and provides recommendation and choice. In this world, the interface isn't always a screen... and the input isn't a keyboard or a mouse!

Speaking at Websphere Portal Techincal Conference

2008-08-28T09:25:00.000-07:00

I will be introducing the world of Spring to the world of Portlet development at the IBM WebSphere Portal Technical Conference. The conference is in Nashville, TN between Oct 13-16. The topic of the session is Annotated Portal Development with RAD and Spring; session D19.

Spring JavaScript

2008-08-24T12:25:00.000-07:00

I'm sitting in Keith's Talk @ NFJS Orlando on Spring JavaScript . I've worked with a lot of the spring framework on varies projects over time. I have not until now had a chance to look at what SpringSource was doing with JavaScript. This is great technology which desires some blog time in the future.

There is an underlying theme in the approach Keith is describing which I really like. The concept is a design approach where ajax provides value-add to a client which allows for javascript. Yet the application still functions without it.

Impressions from a one hour guided tour. Things I liked:

Choice of dojo as the underlying framework
Design approach
Use of Tiles
Leveraging of yahoo performance rules for script compression and optimization

Things to look forward to:

Binding of multiple decorations to the same html element
Closer look at Spring's CSS Framework

Great example application was demo to show off spring capabilites... but it is actually a very useful application reporting dependencies of osgi bundles. Take a look: http://springsource.com/repository/

Fixing Java Memory Tools on Mac OS X

2008-08-21T14:59:00.000-07:00

A number of the Java memory management tools with the default distribution on Mac OS X Leopard are broken. The information to repair the situation seems to be hard to come by. This post will detail the steps necessary to get jmap and jhat to work on a Mac for Java 6. I assume that Java 5 has the same issue but haven't checked.

Setting up a test Java process to profile.
1. From the terminal: cd /Developer/Examples/Java/JFC/Java2D/
2. From the terminal: java -jar Java2D.jar &
3. After the process starts it should indicate the pid (in this example it is 5510)
4. You can also use jps to discover the pid: jps -l (then look for the Java2D.jar)

Fixing jmap
If you type jmap , you would expect to get a high-level memory dump... instead you get a message like:
attach: task_for_pid(5510) failed (5)
Error attaching to process: Error attaching to process, or no such process

To fix this, do a sudo, such as sudo jmap .
Debugger attached successfully.
Server compiler detected.
JVM version is 1.6.0_05-b13-52

using thread-local object allocation.
Mark Sweep Compact GC

Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 88080384 (84.0MB)
NewSize = 2686976 (2.5625MB)
MaxNewSize = -65536 (-0.0625MB)
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 6
PermSize = 21757952 (20.75MB)
MaxPermSize = 88080384 (84.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 10551296 (10.0625MB)
used = 4592152 (4.379417419433594MB)
free = 5959144 (5.683082580566406MB)
43.52216069002329% used

At this point it looks like it is working, however the file dump doesn't work.

Fixing jhat
When running jhat on the hump dump, you get the following error:
Eliminating duplicate references.....................
Snapshot resolved.
Exception in thread "main" java.lang.RuntimeException: java.lang.NullPointerException
at com.sun.tools.hat.internal.oql.OQLEngine.init(OQLEngine.java:277)
at com.sun.tools.hat.internal.oql.OQLEngine.(OQLEngine.java:51)
at com.sun.tools.hat.internal.server.QueryListener.setModel(QueryListener.java:59)
at com.sun.tools.hat.Main.main(Main.java:189)
Caused by: java.lang.NullPointerException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.tools.hat.internal.oql.OQLEngine.init(OQLEngine.java:256)
... 3 more

It took me a little hunting to uncover the issue. It turns out that jhat uses the Java 6 js engine. If you go to the terminal again and run: jrunscript , it indicates that the script engine for language js can not be found. On the mac, if you execute: jrunscript -q, which provides a listing of script engines you'll see:
Language AppleScript 2.0.1 implemention "AppleScriptEngine" 1.0
Thank you apple!

As a side note if you actually wanted to run AppleScript as a shell, type jrunscript -l AppleScript

Let's get to fixing the issue

Download JSR-223's engines https://scripting.dev.java.net/files/documents/4957/37593/jsr223-engines.zip
Download Rhino http://www.mozilla.org/rhino/download.html
Copy jsr223-engines/javascript/build/js-engine.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/ext/
Copy rhino1_7R1/js.jar to /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/lib/ext/

** you have to be sudo to do steps 3 and 4.

now jrunscript -q should look like this:
Language EmbeddedECMAScript 1.6 implemention "Mozilla Rhino" 1.6 release 2
Language AppleScript 2.0.1 implemention "AppleScriptEngine" 1.0
Language ECMAScript 1.6 implemention "Mozilla Rhino" 1.6R7

... and jrunscript will put you in a js shell.
... and jhat heap.out now works! Point your browser at http://localhost:7000 and object browse!

Also something I didn't realize until I was preparing for this post... jmap worked for the console dump if I sudo'd the command. However it would not do a file dump. I did the file dump using jconsole and jmx. Now that the scripting issue is fixed, jmap is improved in the following manner:

if you type jmap , it now prompts you for your admin password (instead of just failing)
jmap -dump:live,format=b.file=heap.out 5510 now works

It makes me wonder what else wasn't working! Happy Coding!

Portal 2 JSR-268

2008-05-14T20:21:00.000-07:00

JSR-286 has been signed off! Here comes Portal 2. I don't know of any implementations yet... so we are still waiting to play with it.

Portal 1 Challenges
There are a number of pain points of portal development which include:

Accessing resources... this is an issue with security mainly
Portlet to portlet communication through the server framework
Ajax communication.

Much of these issues have been addressed through the specification...

What's new in Portal 2
First, Portlet to portlet communication is not as cumbersome and isn't limited to some session hacking. The specification provides 2 mechanisms for communication at the server tier.

Events
Public Parameters

Events that are provided by an event producing portlet are described in it's XML file. Event consuming portlets describe what events they can consume, then at deploy time or run time the portlets are wired together through an event channel.

Second, resources are finally part of the picture. In the portal 1 days, you would often need to access a resource outside the portlet... well this request didn't have any portlet details, such as window state or portlet perferences and worse... it didn't run under the security of the portal.
Now resources are obtainable within the context of the portal. To a certain extent this provides us now with the ability to do ajax style development which wasn't possible before.

Next, there is the standard web page needs which have been addressed. For instance, because a portlet doesn't own the page, it really can't participate in provide header information... until now. There are multiple respond phases now... one for the header and one for markup. This provides the ability for a portlet to participate in the creation of the page header. Additionally access to cookies and other web oriented needs have been opened up.

On top of this their are portlet filters and portlet url listeners...

If it is not too late... the portal world is about to get a breath of fresh air. Take a look at the specification, it is worth a look if you are in this space. Perhaps when the reference implementation is out... and I have time I'll create a demo of a few things... of course I'll need to include some of the latest spring stuff in the mix. It looks like Spring 3.0 intends to have Portal 2.0 support. As I scan around on the web... it looks like liferay is portal 2 ready... maybe it is time to switch back :)

You don't even know you don't even know

2008-05-14T19:58:00.000-07:00

My daughters and I have a phrase we've picked up from some where... probably a movie, but I don't remember which one, which is "you don't even know, you don't even know". We use it liberally to describe a situation where people are talking about stuff they clearly know nothing about, yet they are sharing their opinion about it anyway. It seems strange that this happens... and frankly it happens all the time.

So while speaking in Denver between sessions I attended a great session by Jared Richardson. We got into a discussion on developer perceptions on agile practices such as pair programming, etc. When it occurred to me that this was it... so often developers dislike pairing when they haven't practiced it. They claim they don't have time to write unit test or integration tests... but they haven't attempted it.

It appears that Agile in some circles is getting a bad name... but most of the time it is based on the opinions of someone who thinks they know what the outcome will be... they have thought it through in their head and have come to a conclusion, which has become their opinion on the matter. Sometimes this is further justified through their comments "well I heard that so and so had issues... blah...blah". Sometimes it comes in the early onslaught of pain... either it is new, or their some affect of team storming. The reality is that often new agile teams run a little slower upfront, but end up significantly ahead of non-agile teams. The pain is worth it! (at least from my experience)

So as a community let's make an agreement. If you ain't done it, you can have an opinion... but it doesn't count :) If you think it won't work or if you heard it won't work... you don't even know what you don't even know!

Knowing comes through experience! and as GI Joe says "Knowing is half the battle".

Speaking at OSCON

2008-05-14T19:53:00.000-07:00

Well I will be speak this year at OSCON on July 25, 2008 speaking on Spring 2.5 . It will be fun to see how I take my 1:30 hour talk which I usually run over time on and deliver it in :45 mins.

Welcome to Viet Nam! in Bangalore

2008-04-24T12:28:00.000-07:00

I mentioned earlier that I would blog on this subject from my India trip. So I had a very unusual situation in Bangalore. One of the attendees (Madhu) who attended the Jax conference requested that I visit his company during my stay in Bangalore. This became difficult based on a number of reasons, which resulted in Madhu and 10 of his co-workers coming to my hotel. The lobby was too noisy, so we ended up upstairs in my hotel room. The picture below is just half of us that are sitting around a bed talking tech.

We discussed a number of technical details... it started with a discussion around ORM tools and the problem they had mapping to a complex and highly normalized data store. They were creating a web admin tool which for simplicity sake I'll characterize as a content management system (CMS). In this space they don't know all the "columns" of data to store. The columns or metadata is stored in a row of a table, then there is an association table which associates the column to a 3rd table, the table of values. They didn't have to get to far in the description for me to know exactly what they were talking about. I have seen it before a number of times... it is the kind of thing a really out of touch architect, or a green DBA would suggest. I shared some horror stories of my past to try to sway them from this approach. I then discussed all the negatives to this approach.. such as:

No indexing
No join capabilities
Very complex queries
Did I mention no indices? This data structure just isn't tunable.

The conversation led to putting an ORM tool on top of this :) If anyone knows of ORM tool that does this kind of mapping I would love to know. I explained that I was unaware of a ORM that would handle this situation.

While I was speaking at JAX on the topic of Spring and JPA, an attendee reflected on Ted Newark's article relating ORM to Viet Nam. I was amazed!! People around the world are reading Ted's Blog...

So as we were sitting around the bed discussing alternatives, I mention Ted's article which I just re-read that week ... of course I had to explain a little history regarding the meaning that Viet Nam has for most Americans. We got to the point were I was describing how a project gets so far done the road and the team has invested so much, that when anyone suggests that we are tapped out and need to rethink the projects technical approach management comes in with statements like "Come on guys, we have so much invested. Can't you just get it to work". At this point they were all smiling, looking at each other and someone spoke up and says that is exactly where we are at... to which I said "Welcome to Viet Nam" :)

Speaking at OSCON 2008

2008-04-17T14:18:00.000-07:00

I will be speaking on Spring 2.5 at OSCON 2008 this year! The next tid bit of fun will be fitting my 1 1/2 hour talk into a :45 minute talk... should be fun.

Speaking in India at JAX

2008-04-17T14:12:00.000-07:00

Just finished speaking in Bangalore at the JAX conference. The experience was very unique. I'll hit some highlights on a later post. There are 2 remember experiences from the trip. First is the driving there. A friend of mine from a previous trip capture what I'm talking about. Second, my last day I had 10 indians from a local company sitting around my hotel bed talking tech... the statement which capped the conversation was "welcome to vietnam"... I'll explain later.

Speaking References and Links

2008-03-08T12:21:00.000-08:00

Speaking at a number of conferences specifically

Folks generally are asking for resources, references and rss feeds for material. I've started leveraging this blog more, as well as http://del.icio.us/kensipe . So if you are looking for supporting information for my 7 habits talk then look at http://del.icio.us/kensipe/7habits or click on the tag.

It is All About the Approach

2008-02-29T09:50:00.000-08:00

Being a pilot this video makes me chuckle a little. But at the same time , I don't know a pilot who hasn't had a rough landing during training or high cross winds or... This video reminds us that it still happen to the professionals.

While I'm not sure how this pilot found himself in this position. Experience tells me that it is all about the setup and approach on final which results in a well executed landing.

Working with software solutions it is the same... a well finished project is based on the approach. If you have early and often feedback, if you have tests which run often you will land safely. Iteration X is dependent on Iteration 0. Make sure you get Iteration 0 right! Then follow the check list!

Paired Teachers (K-8)

2008-02-28T09:37:00.000-08:00

Agile in the Classroom
Being involved in the community, I was at a school board meeting last night (for way too long, but that is another story). There was an interesting presentation. The school last year started a program referred to as "Teacher Leaders". It was a great presentation detailing the outcome of this program which is essentially highly skilled teachers which "pair" (my word not theirs) with other teachers on specific subjects to meet 2 objectives:

Increase student learning
Increase teacher teaching capability

Results of Agile in the Classroom
The results of the presentation were amazing and the whole room seem to get it. It was great that they brought empirical evidence to the table. They defined 4 levels of support for coaching a teacher on a new program, which are:

Theory
Demonstration
Practice
Peered Coaching

They took 2 teachers; Teacher A went through levels 1-3 which is the standard approach, while teacher B had all 4 levels, which included in classroom pair teacher sessions through peer coaching. What was great was the measuring stick... it wasn't the teachers it was the students. In classrooms with teacher A, students had a 35% proficiency on the subject matter. In classrooms with teacher B, students had a 95% proficiency rating. This was consistent with other studies in the field, but they wanted to test it themselves. Additional benefits included an increase in motivation and spirit of the teachers. There was an obvious excitement in the air when several teachers involved in the program shared their experiences.

Agile Converts
Members of the board shared their reluctance in approving the program, which mirrored that of software stakeholders hedging on paired programming. One board member was significantly against it. Now she is a total convert and is looking to see how to expand the program. The reason... The quality of the result. It turns out that pairing produces higher quality.

What is a calendar?

2008-02-25T16:10:00.001-08:00

It is my intention to write a few posts on the need to focus IT on business solutions. I've got several posts in my head... All I need now is time. A situation today spurred me to post a comment on words and the meaning of words, which is critical for one person to understand another person in order to clearly provide a business solution.

It is 10pm... Do you know where your data is?
I have been a heavy user of google and it's products for some time. This is very interesting on its own since when Microsoft came out with Passport and HailStorm years ago... My response was... "heck NO!" you're not keeping all my data. It is interest, as they had some services but nothing significantly compelling to me. Yet, over the years google hasn't asked to store all my stuff... they didn't have a huge marketing campaign, yet some how they have all my data. I now use they're gmail, calendar, I use and share notebooks, I've used their payment capabilities, I blog on their site, I use their phone services through GrandCentral... they know everything about me. I came to this comparison and realization recently and well... I'm not sure if I am scared yet or not, but that is a subject for another post.

"What's in a name? That which we call a rose By any other name would smell as sweet." - Shakespeare
So on to the main point of this post. I have convinced several other partners to switch from yahoo group calendar to the google calendar. One of them was lost in how to get to this shared calendar so I proceeded to explain via email how the google calendar is used. This resulted in the following statement by me:

It is a little complicated to write about because there needs to be a distinction between a "calendar" of time and a "calendar" which represents a thread of interest through time :)

It made me reflect on how many every day terms we use in English are so abstract.

The essences of Chairness
If you ever read Plato, you know what I mean... The word chair means what you usually think it means. However as Plato puts it, what do you think of when someone says chair. Does it have 4 legs, or 3? Does it have a back to it? If you started picking it apart (remove a leg, etc.), at what point does it cease to be a chair? This isn't made to be too philosophical. The point is some times it is difficult for us to use an abstract term and worse, sometimes words are more abstract then we think they are. Where a chair could paint the same mental picture for 2 people, the concept of what is core to the essences of chairness is likely different.

The trouble with Users
Here in lies the trouble with users. There really isn't an issue with users. They mean what they said and they are paying the bills. However often they say calendar and they mean calendar of time and often a developer sees the bigger picture and sees the need for a calendar to represent a thread of interest. There are a number of issues at this point:

The user and developer understand the words of the domain to have different meanings.
Even, as often is the case, the developer is astute enough to realize that their is a disconnect, words do not exist to delineate between the concepts. This leads to:

Lengthy discussions where the user is dumbfounded, where both parties are asking themselves why doesn't he get it!
Frustration

It is all about context, context, context
It is just interesting how often this situation happens. Often the issue is context. Where context provides the meaning or at least a deeper meaning. But sometimes not. In the example of the calendar, it is tough to say that context would provide greater meaning. This issue is further exacerbated with specific domains where context does matter.

Several years ago, I worked on a life science project working with a lab on transcription profiling. During this engagement a couple of us worked with several top Ph.D.s in the field to help automate the process. It was necessary to understand the word "Allele" and its domain. Although there are aspects about an Allele which were easy to agree upon. The other aspects were dependent on who you talked to and on which day... it didn't take long to realize that it was really a matter of context which provided its mean.

Clarity of Thought
If there is a point to this current blog it would be this: In order to write software... even flexible software (in this case meaning software which can understand context of the user), then it is first necessary to understand concretely what is meant, as well as what are the other contexts of meaning. This may mean more in depth investigation on the part of the BA or developer, but it will pay off in dividends!

This provides some context around the value of Agile development. Closer and frequent communication with the user increases the likelihood of understanding what the user meant. Regular feedback by the user provides the opportunity to discover word meaning differences. The greater the number of users involved increases the likelihood of discovering more contexts. Often this is the reason for a number of "discovered" stories along the Agile path of development.

-- Happy Coding!

2008 Technology Predictions

2008-02-11T12:45:00.000-08:00

So after a month of reading a number of articles making there 2008 predictions, I feel compelled to share my predictions. I have the added advantage which comes to those who procrastinate... however there isn't much new over the last month.

Larger portion of the Java community will move towards Java 5 /6, taking advantage of more of the language changes.
Increased adoption of groovy users
Spring framework increases in use... leaving in their dust SEAM and Guice.
Spring MVC and Webflow becomes the new Struts (from a popularity standpoint)

It will come down to WebFlow and JSF
It will be interesting to see what grails does to this prediction

Late 2008 will have a number of negative news feeds on SOA.

This will be geared around failure of large initiatives.
It may be 2009... depends on the economy

Flex use increases

MSFT will start the get the RIA facts in an effort to take over this space.

Vista will continue to have trouble

It will be interesting to see what MSFT does when the XP isn't available for purchase any more.

with the OEM ending in June 2008, I expect that MSFT will extend it... after a period of pain :)

JBoss and OSS in general will increase in use.

The economy will be the driver for this.

Java 6 Web Service Security Alert

2008-02-05T12:23:00.000-08:00

This isn't how I envisioned starting this new blog...

It appears that there is a fresh security warning from Sun regarding security issues with xml parsing in Java 6 (update 3 or earlier). It does not affect Java 5 or eariler.

Details from Sun:
http://sunsolve.sun.com/search/document.do?assetkey=1-66-231246-1

Details from SecLists:
http://seclists.org/bugtraq/2008/Feb/0007.html