Monitoring ActiveMQ Using JMX Over SSH

If you want to know true confusion and frustration I suggest you attempt to monitor ActiveMQ using JMX over SSH port forwarding.

For those who use ActiveMQ, the JMX monitoring is some pretty impressive stuff.  You can dive into connections, queues, topics, subscriptions, etc and get stats about the current state of the system.  However, this nirvana of information is practically unreachable when AMQ is in a data center.  For the first six months or so working on AMQ I was able to get around this using the web console which gives some level of detail on topics and queues but once we moved into trying to debug bigger and bigger problems the console wasn’t cutting it any longer.

The Investigation Begins

As with anything AMQ, I started looking into the JMX documentaion.  Step one was to enable JMX in the activemq.xml file.

<broker useJmx="true" brokerName="BROKER1">

Fire it up, forward port 1099 (the default JMX port), give it a try, and, no go.  Come to find out, JMX uses two ports, one which is defaults to 1099 and one which is negotiated at runtime.  No worries, there is an “Advanced JMX Configuration” seciton in the documentation which lead me to this xml configuration.

<broker useJmx=”true”>

<managementContext>

<managementContext connectorPort=”2011″ jmxDomainName=”test.domain”/>

</managementContext>

</broker>

<managementContext>

    <managementContext connectorPort="2011" jmxDomainName="test.domain"/>
</managementContext>

Awesome, I can change the default 1099 port but nothing else.  The AMQ documentation was failing me, as usual.  Google, here I come.  After some searching around I was able to dig up AMQ-892.  (Power user tip, search AMQ’s Jira site once the documentation has inevitably failed.)  It turns out you can configure the RMI server port and have been able to do so since version 4.1.

<managementContext connectorPort="11099" rmiServerPort="11109" jmxDomainName="org.apache.activemq"/>

I particularly liked the “This necessitates a documentation update in the wiki.” comment from 2007.  Way to jump on that.  Confident that this is bound to work I forward port 11099 and 11109, give it a whirl, and, failure.  It was at this point that I started to despair.

Deeper Down The Rabbit Hole

Quickly running out of options I decided to try running the same configuration on a dev box in the office, and of course I could connect to it flawlessly.  To add insult to injury I tried another dev box and this one failed.  The only difference I could easily see was one was running a VM image and the other was native but this didn’t point at any easily understandable target.  I was done.  I conceded defeat.  I would soldier on without JMX.

Meanwhile, my co-worker had been pressuring me to start packet sniffing with wireshark.  I had resisted, mostly because I dislike getting that low level, but also because RMI is a binary protocol and I didn’t think I could learn much.  However, I was bloody and beaten at this point and gave in. I downloaded the package, fired it up and started sniffing away.  Sure enough the binary was a bunch of gibberish, but tucked away in there was one string, the hostname of the amq server I was trying to connect to.  What the hell is this?!?  That question is left to the reader but at least the issue was clear, the hostname was inside the data center and not something I could connect to from my machine.  It also jived with the test on the local dev boxes.  The box with the VM image had an internal hostname that was not in the office dns, where as the native machine had an addressable hostname.

I attacked the problem with renewed vigor.  I found a JConsole FAQ on the AMQ website that I had previously dismissed for its cryptic message.  However, this time it made sense.  I quickly added

-Djava.rmi.server.hostname=127.0.0.1

to the Java Service Wrapper configuration for AMQ so communication would be forced back through the SSH tunnel, rebooted AMQ, and, wait for it, VICTORY.

The Morale of the Story

So the short answer is add the broker configuration, the management context (the second on up there), and the rmi hostname configuration, port forward 11099 and 11109, and fire up jconsole

jconsole service:jmx:rmi://127.0.0.1:11119/jndi/rmi://127.0.0.1:11099/jmxrmi

As with most of these posts, so easy once you know, but so hard to get there.

Unit Testing Spring Jersey Integration

Recently, I’ve been working on a side project which has given me the opportunity to play with an interesting Java technology stack. After a few weekends of working on the build and moving my way up the stack from database to service layer I was at the point of writing a full integration test. In the past I’ve done integration testing with an iBatis/Spring stack, which, with the help of DBUnit is a fairly simple exercise. However, this project is heavily REST based and the addition of Jersey (JAX-RS) was a serious curveball.

To really test the whole stack an embedded webserver is required.  Looking through the Jersey/Spring unit tests they make use of an exploded WAR using Glassfish.  Assuming this was the optimal solution I added Glassfish to my ivy.xml config and ran an ivy update.  However, once I realized the overwhelming volume of jars being downloaded I decided it was much too heavy weight of a solution.

My next choice was Jetty.  I have had experience with using Jetty in the past but I quickly ran into issues trying to get it working right with Jersey, Spring, and an exploded WAR configuration.  It was then that I saw the GrizzlyServerFactory referenced in another Jersey unit test.  After some prodding and reading into the source code I ended up with the following:

final URI baseUri = UriBuilder.fromUri( "http://localhost/" ).port( 9998 ).build();
final ServletAdapter adapter = new ServletAdapter();
adapter.addInitParameter( "com.sun.jersey.config.property.packages", <your-package-name> );
adapter.addContextParameter( "contextConfigLocation","classpath:applicationContext.xml" );
adapter.addServletListener( "org.springframework.web.context.ContextLoaderListener" );
adapter.setServletInstance( new SpringServlet() );
adapter.setContextPath( baseUri.getPath() );
SelectorThread threadSelector = GrizzlyServerFactory.create( baseUri, adapter );

We start by making a base Uri for localhost at a high numbered port.  Then we build the Grizzly ServletAdaptor.  The first parameter is the package name where your Jersey enabled are located.  The next two lines are taken directly from the web.xml file that is used for the actual WAR.  The first is the config location for the Spring Context and the second is the ContextLoaderListener servlet.  Finally we add the Jersey SpringServlet and the context path.  With these two objects created we can build a SelectorThread and begin using the server using the Jersey Client api that I talked about in my previous post.  When you are done, don’t forget to call stopEndpoint() on the SelectorThread when you are done.

REST calls and JSON results in Java

I recently had the need interact with Twittervision’s RESTful api from Java. As usual, I started googling around for a solution but nothing obviously stood out. There are a handful of RESTful java frameworks, but these focus on providing a RESTful api and not consuming one. Finally, after an annoyingly difficult search I found an impressively simple solution based on Jersey.

The point to realize is that Jersey provides both a sever and a client api. To make use of the client api requires three simple lines:

Client client = Client.create();
WebResource webResource = 
client.resource("http://twittervision.com/user/current_status/bdarfler.json");
String response = webResource.get(String.class);

The result is a string of JSON, but now, how to parse it. After searching around I ended up settling on json-simple. A few simple lines of code and we can get the address for any twitter user.

final JSONObject jsonObj = (JSONObject) JSON_PARSER.parse( response );
if ( jsonObj != null && jsonObj.containsKey( "location" ) )
{
final JSONObject location = (JSONObject) jsonObj.get( "location" );
return location.get( "address" ).toString();
}

So, once again, simple stuff but not as obvious as I was hoping it would be.

Integrate ant with dbdeploy not in the classpath

The need for a schema versioning system at LocaModa finally became a priority this sprint so I’ve spent the past few days researching and diving into implementation.  After comparing LiquiBase, dbdeploy, MIGRATEdb, and dbmigrate I settled on the simple SQL driven solution that dbdeploy provides and started integrating it with our Ant build.  Pramod Sadalage over at Agile DBA put up a nice post on the basics but I quickly hit a wall, the MySQL driver was not in my classpath.

As I mentioned earlier we are using Ivy for our dependancy management which makes it difficult to throw jars into the Ant classpath since they are automatically downloaded at build time.  So far this hasn’t been an issue since most Ant tasks allow you to specify the classpath as a parameter within the task.  Dbdeploy does not.  Goggling around I quickly found a posting that suggested patching the dbdeploy ant task as the only way to go, so off I went.  However, a few hours later, after wandering through the Java classloader abyss I was stymied.

I backed out of the changes and decided a different tactic was needed; back to Goggle I went.  This time I came across a post which suggested all I needed was to add the jar to the taskdef classpath  Could it really be that easy?  Switching back to my build.xml I doctored up the dbdeploy taskdef, kicked off a build and sure enough the damn thing worked.

So, if you want to centrally manage your ant dependencies, just remember to add the necessary classpath references to your taskdefs, like so:

<taskdef name="dbdeploy"
        classname="net.sf.dbdeploy.AntTarget"
        classpath="${dbdeploy.jar}:${mysql-connector.jar}"/>

The Best of the Best Podcasts

I’m constantly surprised at how little press podcasts get.  Everyone and their mother has a DVR and has grown accustom to watching TV on their schedule, yet only a small minority see podcasts as the identical concept applied to radio.  Maybe this stems from a lack of interest in radio as opposed to TV.  Maybe it stems from people only using radio for its musical content, in which case mp3s are as far as they need to g.  Either way, its a shame and in hopes of remedying that in some small way, I have compiled my short list of podcasts which everyone should listen to. (download the opml file)

Public Radio

Public Radio is leading the field in providing their content in podcast form.  I have no concept of when any of these shows air on my local station (or even if they air).  Instead they show up in iTunes and I listen at my leisure.

On The Media

As a meta news show, it is the one and only news show necessary to fully understand how media covers (or fails to cover) the important stories of the week.

7AM ET News Summary Podcast

The daily news in under five minutes.  No Britney, no Timberlake, just the basics. 

Marketplace Morning Report

The daily financial news in under 10 minutes.  Increasingly important in the past six months.

Marketplace Money

A weekly one hour program on the financial sector and personal finance.  Everything you need to know for retirement, debt repayment, etc.

WNYC’s Radio Lab

Hands down the best science podcast.  Incredibly produced and riveting.

This American Life

An hour of touching, poignant, humorous and otherwise evocative stories about life in America. 

Speaking of Faith with Krista Tippett

A wonderful hour exploring every possible view on faith, spirituality, religion, ethics, morals, and all the other big questions in life.

Intelligence Squared

Thoughtful, intelligent, persuasive debates on the hot button topics of today in the Oxford style.

Software Development

The Java Posse

Hands down the best developer centric podcast available.  Four friends with four very different view points come together to recap anything and everything JVM related.

Pragmatic Podcasts

Interviews with authors from the Pragmatic Programers publishing company on their upcoming books.

Railscasts

Interviews with influential members of the Rails community.  The only tolerable rails podcast, more a comment about the other podcasts than about this one.

The Accidental Creative

An inspirational podcast aimed at artistic creatives but carries over well to the technical creative.

JavaWorld’s Java Technology Insider

An infrequent Java podcast with some solid content.

Personal Development

Zencast

Weekly one hour dharma talks, always insightful and inspirational

Buddhist Geeks: Seriously Buddhist, Seriously Geeky

Like the name says, these are a bunch of Buddhists who geek out about Buddhism, and occasionally about tech.

The New Man: Beyond the Macho Jerk and the New Age Wimp

A no bs, post-new age approach to manhood.  I imagine this is one of those love it or hate it podcasts.

Arts

IndieFeed: Performance Poetry

A thrice weekly slam poem delivered directly to your ears, improved only by the host genuine love for the genre and community

Philosophy Bites

Occasional podcast delving deep into a particular philosopher or philosophical concept, all with great British accents.

The Moth Podcast

“Real life stories, told live with out notes.”  The story equivalent of slam poetry.

Fora.tv – Audio Program of the Week

About as close as you can get to TED talks in an audio format.

Business

Harvard Business IdeaCast

Business for the thinking person.  No hype, no buzz words, no marketing speak.

Gen Y Marketing Podcast

A great take on marketing from the Gen Y perspective.  Gotta love the aussie accents.

Entrepreneurial Thought Leaders

A fantastic weekly lecture series at Stanford

No-Install Domain Setup Zen

In this down economy  the chatter about owning your own domain as a way of personal branding has been on the rise.  However, the elephant in the room is what to do with it once you have it?  For a large majority of people, installing and maintaing blogging software like WordPress, Moveable Type, or Drupal is cumbersome, expensive, and unnecessarily time consuming.  Why go this route when everything else online is moving towards the cloud computing model?  I’ll show you how you can set up not only a blog but email, photos and more for little to no cost and without installing a single piece of software.

Getting a Domain

Go out and get yourself a GoDaddy domain using a promo code and you can be on your way for about $7.50 a year.  Nothing beats the price and since they are the 800 pound gorilla of domain hosting you are more likely to find good blog posts (like this one) about how to get your hosted applications up and running on it. 

Putting Up Content

blog.mydomain.com

As I mentioned in my previous post there are a really two ways to get hosted blogging, WordPress and Blogger.  Blogger integrates with GoDaddy freely and easily but is less polished than WordPress.  If you go the WordPress route the cost is $10/yr for redirecting and the set up is a bit confusing but its worth it for the superior platform.

1. Go do your WordPress dashboard and click on the Upgrades menu
2. Gift yourself a $10 credit
3. Under the domains menu add blog.<your-domain>.com
4. Create a CNAME record in GoDaddy that points from blog.<your-domain>.com to <your-blog>.wordpress.com
5. Wait and verify that blog.<your-domain>.com redirects to <your-blog>.wordpress.com. 
6. Follow these directions to make <your-blog>.wordpress.com to redirect to blog.<your-domain>.com

photos.mydomain.com

Unfortunately there is no free lunch here, the only options are Zenfolio and SmugMug.  Zenfolio’s integration seems pretty straight forward and requires a $40/yr subscription.  SmugMug is currently more full featured (for instance it integrates with S3 for storing digital negatives) and more customizable (check out this example) but it also costs more.  Their integration requires a $60/yr subscription.

Google Apps

One of the main reasons for getting your own domain is having an email address with that domain.  Luckily Google and GoDaddy make it very easy with two simple tutorials; one for forwarding the various subdomains and one for setting up your mail server.  Better yet, there is an automated tool for setting up the mail forwarding.

Miscellaneous Tools

wiki.mydomain.com

I’ll let my fellow bloggers extol the virtues of a personal wiki.  Now that you are convinced, go over to Wikidot, make yourself an account and follow the instructions on mapping it to your wiki subdomain.

openid.mydomain.com

More than three years after its inception OpenID is finally taking off with more and more sites providing and accepting the standard.  To centralize your identity at your new domain register at myOpenID and follow their convenient directions.

Tying it Together with Tumblr

Tumblr is one of the most under appreciated sites.  As a blogging platform it ranks last in traffic stats. However, its ability to function as a blog, share feed (via the bookmarklet), life stream (via importing arbitrary rss feeds), or any combination, all while remaining simple to use, is astounding.  For our purposes though, the killer feature is its ability to redirect from a custom domain.  With that setup you can easily start pulling in any feeds you like.  I pull in both my blog and my photoblog feeds but other options might include Twitter, YouTube, digg, etc.  You could also import a share feed from delicious but it pails comparison to using the Tumblr bookmarklet.  With the bookmarklet, content to shows up directly, instead of an imported link, and you can inject your own thoughts using comments.

But the fun doesn’t stop there.  Tumblr allows you to customize every last inch of your site which has resulted in a vibrant community of 3rd party themes and hacks.  After some experimentation, I settled on the nine of mine theme by Sid05.  However, the theme was only the base.  From there, I replaced the default feed with feedburner, added some google analytics, and did some very basic SEO.  Finally, I added comments using DISQUS and used add to any for a share & save link as well as an improved subscription link.

Happy Hunting

Well there you have it, the culmination of all the research that went into creating my homepage.  Use it wisely and don’t forget to drop me a comment.

This Web 2.0 Life

Preface

I’m find myself intrigued by how others use the plethora of social media sites that dot the internet landscape. There are the Facebook only users, those that have branched out to Twitter, others that keep an external blog, even fewer who dabble in more than just a few sites, and then there are the elite that strive for that perfect interconnect that unifies their wide scattered online presence. I consider myself in the latter group and as such I feel the urge to share my setup.

Overview

Before we dive into the details I’d like to give a conceptual view of my setup. While some might be tempted to take the swiss army approach of trying to accomplish everything in Facebook or a single blog I find this limiting. In the case of Facebook, they have a wide array tools but none of which really stand out. In the case of a blog it quickly becomes obvious that, while possible, it was not meant to consolidate everything. Instead I like using a wide range of simple, single minded tools and then, through extensive use of rss, push and pull the content where it belongs. 

Self Generated Content

Words

When it comes to traditional blogging sites they tend to fall into two camps, the ones that focus on community and the ones that focus on the individual blogger. When I’m writing my personal musings and occasional moanings I like the feel of a community as well as a flexible set of privacy settings which is why I have my personal blog on LiveJournal. LiveJournal rose to ascension when I was in undergrad and as such its the platform where most of my friends still remain, as well as where many other people my age tend to reside. Additionally, given its nice set of privacy controls I can decide if the world, my friends, or only my closest confidents can see what I write. 

On the other hand, when I started to consider blogging on a more professional level I was instantly drawn to WordPress. In the hosted blog space for individual bloggers the same three names tend to fly around; WordPress, Typepad, and Blogger. I wasn’t looking to pay so that quickly ruled out Typepad. As for WordPress vs Blogger the contrasts are pretty obvious. WordPress spends a lot of time on the fit and finish were as Blogger spends a lot of time on the advanced customization. At the end of the day WordPress’s clean interface and themes won out. 

Finally, lets not forget micro-blogging. In keeping with my simple and and focused mentality the clear winner here is Twitter. Pownce has been gobbled up by SixAppart and end of life’d while Jaiku and Plurk both try and be too clever (with added rss stream imports for Jaiku and crazy timeline visualization for Plurk) and lack in a solid desktop client story.  Interestingly enough the open source site identi.ca looks really slick but at this point I don’t think I can justify the cost of moving all my friends over to identi.ca and I have no interest in keeping a mirror of my twitter account.

Pictures

When it comes to photos things actually get a bit trickier. For a long time I was a pro user at Flickr which I found fantastic as an amateur photographer. As one would expect, the social aspect is top notch, and since it has an overwhelming mind share I have been solicited on it a number of times by people wanting to use my photos which is quite nice. However, as I became more serious about my photography I realized I needed something more serious for my photos. It was at that point that I made the switch to SmugMug. At the time it was the only serious option for professional photographers short of rolling their own solution. Additionally its multi-tiered membership levels gave me room for growth as a photographer.  

At the same time, I recognized the importance of setting up a photography blog to promote my photography beyond just posting them online.  There are a handful of photoblog sites but they all work under the paradigm of uploading photos which gets annoying quick, they really need to work on SmugMug and Flickr integration.  Instead, I prefer simply linking to them from my SmugMug account. Additionally, I couldn’t find one that can hang off my own domain.  So, even though a few of them are pretty slick looking I was drawn back to my blog stand by, WordPress.

With my professional pictures safely on SmugMug I was left with the issue of where to put my random snapshots of friends and events. I could have continued to use my Flickr account but I have serious issues with Yahoo!’s ridiculous username/account name clusterfu*k whereby I end up with a different Yahoo! name from my Flickr name from my Flickr url, its abysmal for someone who likes to have a unified web presence. The solution was to go where the people are. For photos of friends I end putting them in Facebook. For photos of family I usually put them on Picasa. However, more and more my family members are finding their was to Facebook so Picasa might be on its way out of my toolbox.

Sharing Others Content

The flip side of all this content generation is sharing other’s content. There are many options in this space with a plethora of sites all jockeying for a share of the social-bookmarking space. However, I find the social-bookmarking paradigm a poor fit for sharing of user content since here is no easy way of sharing the actual content, just links to it.  Google Reader, has a nice solution in this space where you can share items in your feeds and they end up on a public share page.  However, this requires that the content you want to share be in a feed to begin with and the public pages are not the prettiest things to look at.  In my opinion the stand out winner in this space is Tumblr.  Using a simple bookmarklet I can post pictures, text, videos or links from any site to a nicely styled and easily configurable site that works for both the rss junkie and the casual web browsing friend.  They also play nicely with using custom domains.

Social Networking

The last piece of this puzzle is social networking.  At this point its damn near impossible not to be on Facebook, and of course I am, but thats not the only social networking site either.  LinkedIn, is quickly becoming the social network of repute for the business world and the place to go for both job hunters and head hunters.  Finally, there is Twitter.  While it doesn’t try to be the center of your personal or professional universe like Facebook and LinkedIn it is still the go to place for connecting and communicating with friends and acquaintances, personal and professional alike.  Its an invaluable, on going conversation.

Consolidation

So thats the survey of applications I use, but now comes the fun part, consolidation.  I tend to group my generated and shared content loosely into two groups, personal and professional, which heavily influences where I pool my content.

For me my personal hub is Facebook and as such its the consolidation point for all of my personal content sharing and generation.  I use the Tumblelog facebook app to bring in my personal Tumblr, the default notes app to bring in my personal blog, and the Picasa Tab app to bring in my Picasa photos.  Additionally, I use the Twitter app to bring in my Twitter messages as status updates.

The professional piece took me a bit longer to figure out.  It wasn’t until I finally purchased my own domain (bdarfler.com) that things really came together.  As I alluded to before, I have my two wordpress blogs hanging off this domain as well as my SmugMug account, but how was I going to consolidate these as well as the professional content that I share?  This is where Tumblr really shines, that domain is actually just a Tumblr page, and nothing more.  Moreover, Tumblr can auto post content from any arbitrary rss feed, which allows my to consolidate content from my blog and photoblog as with the content that I post manually.  The added benefit of all this is getting one rss feed for everything I do professionally online which I can then pull into LinkedIn using the BlogLink app.

Conclusion

So there you have it, the eventual evolution of my online presence.  I’m sure things will continue to fluctuate but at this point I’m fairly satisfied with the whole situation.  If I get ambitious again I’ll write a post exposing more of the details of my setup over at bdarfler.com.  Its pretty interesting what you can do without actually hosting a single file and the Tumblr hacks I have are quite cool as well.  If you have any feedback, ideas, suggestions, etc. I would love to hear them below.

Controlling Log4j Logs with Contexts and Filters

Its a problem we’ve all had. Something has gone awry, you jack up the log level to debug and all of the sudden you are inundated with everything under the sun. While one option could be changing some of the logs to trace level there is another option, using Log4j contexts. Log4j contains two different contexts mapped and nested. In this post I’ll focus on nested.

The first thing to do is to isolate the section of your code that you want to create a context for. Once you have accomplished this you make a static call to NDC.push(). The parameter passed in is the string name for the context you would like create. I recommend pulling these strings out into a constants file ease of use and additionally calling .intern() on them so you can use pointer equality checks later when we code up the filters. When you leave the context you call NDC.pop() and life goes back to normal. Finally, remember to call NDC.remove() when you leave the thread.
The code ends up looking like this:

NDC.push(Constants.HEARTBEAT_CONTEXT);
try{
  if (log.isDebugEnabled()) {
    log.debug("[MASTER/SLAVE] Publishing heartbeat");
  }
  doStuff();
}
finally {
  NDC.pop();
  NDC.remove();
}


Now that we have the logs tagged with the correct context we need to create a log4j filter by extending the filter class and overriding the decide(LoggingEvent event) method. The important concept to remember here is that filters can be chained which means your decision results in either a DENY, ACCEPT, or NEUTRAL decision where deny forces the log to be rejected, accept force the log to be written and neutral passes the decision down the chain. Additionally, you access the context by calling getNDC() on the LoggingEvent object passed in to the decide method.
The code then looks like this:
public class HeartbeatFilter extends Filter
{
  @Override
  public int decide(final LoggingEvent event) {
    if (event.getNDC() == Constants.HEARTBEAT_CONTEXT) {
      return DENY;
    }
    return NEUTRAL;
  }
}
Finally we have to add this filter to our log4j configuration. For this we have to make sure we are using the xml configuration file type as it is more expressive and allows us to define filters to use where as the property configuration file type does not. All we have to do is add a filter tag to an appender in the xml and point it at our class.
It looks something like this:

<appender name="console" class="org.apache.log4j.ConsoleAppender">
  <param name="Target" value="System.out"/>
  <layout class="org.apache.log4j.PatternLayout">
    <param name="ConversionPattern" value="%-5p %c{1} - %m%n"/>
  </layout>
  <filter class="my.package.HeartbeatFilter" />
</appender>

Pimpin’ your Ant build with Ivy

As of late Maven has been making headway in the Java build world.  However, there are some who are either in the position of working with an existing system build with Ant or who don’t want to get locked into the Maven file structure.  For those folks there is Ivy to the rescue.

Ivy is an “agile dependancy manager” which recently released 2.0.0-rc1.  Thought jam packed with lovely features the first one that I have put to use is the ability to manage 3rd party dependancies, automagically downloading them, dealing with version conflicts and ensuring that any dependancies they in turn have are dealt with.

Ivy allows for two configuration files, one for describing where to look for jars and one to describe what jars to look for.  Beyond that, the rest of the work is done within your ant build.xml file.  Lets start with the ivysettings.xml file which describes where to look for jars.

<?xml version="1.0" encoding="UTF-8"?>
<ivysettings>  
    <settings defaultResolver="chained"/>  
    <resolvers>  
        <chain name="chained" returnFirst="true">
            <url name="jboss">  
                <artifact pattern="http://repository.jboss.com/maven2/javax/jms/[module]/[revision]/[module]-[revision].jar" />
            </url> 
            <ibiblio name="ibiblio" m2compatible="true"/>
            <filesystem name="local-filesystem">
                <artifact pattern="/local/file/system/[organization]-[revision]/lib/[module].jar" />
                <artifact pattern="/local/file/system/[organization]-[revision]/[module].jar" />
            </filesystem>
        </chain>  
    </resolvers>  
</ivysettings>  

There are a few things to point out here.  First is the resolvers tag. There are a few different type of resolvers and I have chosen to use the chain resolver here.  If you are lucky, everything you need with be in ibiblio and all you would need would be the middle ibiblio tag.  However, if you are using any jars that are not mainstream you will have to end up specifying the url here.  The url tag I’m using here is to deal with the unfortunate issue of jboss not having the latest javax.jms jar up on ibiblio.  Here we specify the url pattern to down load the jar directly as long as the url points at a valid maven style repo.  Notice you can insert the module revision and organization as parameters.  These are passed in from the ivy.xml file which we will discuss next. Also note that in this case there is already a javax.jms jar in ibiblio, just not the right version so we put the url tag first.  Additionally, you might have to deal with some jars that are proprietary or not in a valid maven style repo.  In this case you will have to download these using ant (show later) or store them in source control.  However you can still leverage Ivy by having it pick them up from the local filesystem.

Now onto the ivy.xml file.

<?xml version="1.0" encoding="ISO-8859-1"?>
<ivy-module version="2.0">
    <info organisation="locamoda" module="lms" status="integration" />
    <configurations>
        <conf name="build"  description="build dependancies"/>
        <conf name="runtime" description="runtime dependancies"/>
        <conf name="junit" extends="build, runtime" description="junit dependances"/>
        <conf name="mysql" description="mysql dependances"/>
        <conf name="cobertura" description="cobertura dependances"/>
        <conf name="findbugs" description="findbugs dependances"/>
    </configurations>
    <dependencies defaultconfmapping="*->default">
        <dependency org="com.google.collections" name="google-collections" rev="0.8" conf="build, runtime" />
        <dependency org="log4j" name="log4j" rev="1.2.14" conf="build, runtime" />
        <dependency org="org.apache.activemq" name="activemq-all" rev="5.1.0" conf="build, runtime" />
        <dependency org="commons-lang" name="commons-lang" rev="2.4" conf="build, runtime" />
        <dependency org="commons-dbcp" name="commons-dbcp" rev="1.2.2" conf="build, runtime" />
        <dependency org="org.mortbay.jetty" name="jetty" rev="6.1.11" conf="build, runtime" />
        <dependency org="junit" name="junit" rev="4.5" conf="build" />
        <dependency org="mysql" name="mysql-connector-java" rev="5.1.6" conf="mysql, runtime"/>
        <dependency org="xerces" name="xercesImpl" rev="2.8.1" conf="runtime"/>
        <dependency org="commons-logging" name="commons-logging" rev="1.1.1" conf="runtime" />
        <dependency org="javax.jms" name="jms" rev="1.1" conf="runtime" />
        <dependency org="org.easymock" name="easymock" rev="2.4" conf="junit" />
        <dependency org="org.easymock" name="easymockclassextension" rev="2.4" conf="junit" />
        <dependency org="org.dbunit" name="dbunit" rev="2.3.0" conf="junit" />
        <dependency org="net.sourceforge.cobertura" name="cobertura" rev="1.9rc1" conf="cobertura, junit" />
        <dependency org="findbugs" name="findbugs" rev="1.3.5" conf="findbugs, build" />
    </dependencies>
</ivy-module>

Lets start with the dependencies first.  Each dependancy has an org, name, rev and an optional conf.  The org, name and rev correspond to the organization, module, and revision in the ivysettings.xml as you might assume.  For your common jars you can find the necessary values by looking up the package at mvnrepository.com of course they call org group, module artifact, and revision version, just to keep you on your toes.  Additionally you should notice the defaultconfmapping=”*->default”, if you forget this things blow up quick and in weird ways. The dependancy thing is pretty sweet but the real power comes when you start in on the configurations.  As you can see I have defined six configurations that correspond to different sets of jars that I need to reference in my build.xml.  Configurations can inherit from eachother using the extends so junit is a superset of build, runtime, and the junit specific jars.  Assigning jars to configurations is as easy as listing the right configurations in the conf attribute of the dependancy tag.

Next we tie this all together in the build.xml.

    <target name="dl-binaries" depends="checkfiles, dl-findbugs, dl-ivy" description="download runtime binaries to external"/>

    <target name="checkfiles" description="checks if files have already been downloaded">
        <available file="${findbugs.dir}" type="dir" property="findbugs.exists" />
        <available file="${ivy.jar.file}" property="ivy.exists" />
    </target>

    <target name="dl-findbugs" description="download findbugs" depends="checkfiles" unless="findbugs.exists">
        <get src="${findbugs.url}" dest="${findbugs.zip}" usetimestamp="true" />
        <unzip src="${findbugs.zip}" dest="${external_home}" />
        <delete file="${findbugs.zip}"/>
    </target>

    <target name="dl-ivy" description="download ivy" depends="checkfiles" unless="ivy.exists">
	<mkdir dir="${ivy.jar.dir}"/>
        <get src="http://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar"
            dest="${ivy.jar.file}" usetimestamp="true"/>
    </target>

    <target name="ivy" depends="dl-ivy" description="configure ivy" >
    	<path id="ivy.lib.path">
            <fileset dir="${ivy.jar.dir}" includes="*.jar"/>
        </path>
        <taskdef resource="org/apache/ivy/ant/antlib.xml"
              uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
        <ivy:settings file="ivysettings.xml" />
    	<ivy:retrieve />

        <!-- Define Paths -->
        <ivy:cachepath pathid="build.path" conf="build" />
        <ivy:cachepath pathid="runtime.path" conf="runtime" />
        <ivy:cachepath pathid="junit.path" conf="junit" />
        <ivy:cachepath pathid="cobertura.path" conf="cobertura" />
        <ivy:cachepath pathid="mysql.path" conf="mysql" />
    	<ivy:cachepath pathid="findbugs.path" conf="findbugs" />
    </target>

To start off we have the ivy target which takes care of a few things.  First it depends on dl-ivy for bootstrap downloading of ivy.  Once Ivy is downloaded it sets up the ivy path, defines the ivy ant tasks, and reads the settings file.  At this point all the magic happens with the <ivy:retrieve /> tag.  This goes out, reads the ivy.xml and grabs all the necessary jars.  Once that is done we can create ant paths for all the different configurations we defined in ivy.xml.  This will become super useful later on.  Another thing to notice here is downloading of findbugs.  This is done for two reasons.  First this is a good example of downloading 3rd party libs using ant that are not in a maven style repo.  This target goes out and gets findbugs, unzips it and then ivy can use it through the local filesystem resolver in the ivysettings.xml. The other reason it is here is that the default way of downloading findbugs through the maven style repo conflicts horribly with the findbugs ant plugin.  The plugin expects the files in a very specific layout which is broken when you download from maven vs downloading it from their site.  I use this idiom for downloading some other picky 3rd party jars as well as some binaries that are useful for development but are not actually needed for java.

Finally we get to see the power of these configurations.

     <target name="compile" depends="init,messages,dbobjs" description="compile all the java source">
       <javac target="${TARGET}" source="${SOURCE}" debug="${DEBUG}" destdir="${build}" includes="**/*.java">
            <src path="${src}"/>
            <classpath>
                   <path refid="build.path" />
            </classpath>
            <!-- <compilerarg value="-Xlint"/> -->
        </javac>
    </target>

All we have to do is add the build.path reference and away we go.  We can do the same for findbugs, mysql, junit, adding runtime dependancies into a jar that we create in our build and so on.  Need to add a new jar?  Look it up in the maven repo, add it to ivy.xml and define it as a build, runtime, junit, etc dependancy and away you go.  Pretty slick!

Groovin’ Up Your Database Tests

While TDD proclaims the gospel of fine grained tests and a healthy dose of mock objects, at some point, interaction with a database becomes inevitable.  Luckily, there is DbUnit to assist us.  DbUnit is a fantastic framework which allows for painless setup of database data sets, ensuring repeatable, self contained unit tests. However, there are already plenty of tutorials on the subject.  Instead, what I want to talk about is the dark side of dbunit… xml.

Using dbunit on any reasonably sized project ends up with an explosion of xml data set files, even if care is taken to factor out common data sets and reuse them when possible.  For the longest time this seemed an unfortunate but necessary evil of using such an amazingly useful library, but that was before I attended the Northeast No Fluff Just Stuff in April.  Before that point I had been interested in Groovy and was looking for places to add it to our code base (actually snuck it into a script that ran in the build) but I hadn’t really pondered the possibilities it provided.  However, NFJS was over flowing with talks and general love for Groovy.  Somewhere between the Groovy unit testing and the Groovy MOP talks it dawned on me.  Why not inline the xml using a Groovy builder?!

Groovy has this fantastic feature called builders which allows for very compact syntax to generate any sort of markup (html, xml, java swing, etc).  Of course the one we want to focus on is the xml builder.  By simply changing the file extension from .java to .groovy on the unit test we are already half way there.  After using Groovy’s syntax for creating xml we simply call .toString() on the underlying writer, wrap it in a new StringReader() and then pass create a new FlatXmlDataSet() with the reader as the only constructor argument.

If you are using Ant for your build you can simply wire up the groovyc task to compile all of your groovy tests and then call junit on the .class files.  If you want to get this working in Eclipse, best of luck, I was able to kinda get it working under windows xp but now that I have switched to os x the groovy plugin screams bloody murder with 64-bit java under os x in Eclipse.