Chris Aniszczyk's (zx) diatribe

When I came home last night, I was delighted to find a package (because I’m a fan of gifts).

What was in the package? A brand spanking new copy of the Equinox OSGi book.

In my biased opinion, this is best OSGi book on the market. I’ve had the honor to see the book evolve while it was written. Through that experience, the OSGi tooling at Eclipse (PDE) improved significantly. The authors were kind enough to nag inform me when things didn’t work the way they desired and I did my best to fix everything.

Why is this the best OSGi book on the market? First off, the authors know their stuff and have been working with OSGi before it was cool. Second, the book comes with one of the best example OSGi applications I’ve seen. Third, kind words are said about me in the Acknowledgments section

The only downside, my last name was spelled wrong in the Acknowledgments section (will be fixed in second printing).

So what are you waiting for, grab the book and play with the Toast example.

If you want hands on training from the authors of the book and me, please attend our tutorial at EclipseCon! It’s not too late to register!

How exciting to finally have the Equinox and OSGi book done (see the book review from DZone). In the Eclipse community, many people complain about the lack of documentation and reference material, well that should be less of a problem given the book and excellent Toast example.

You can download the first chapter for free (and the 6th chapter).

I may be a bit biased (see the kind words about me in the Acknowledgments section of the book), but I believe this is the most comprehensive book on OSGi out there. The authors have many years of practical OSGi experience, from the days to when OSGi wasn’t cool yet (SMF) to when it became the basis of Eclipse. On top of that, I think the best thing that comes out of this book is the open sourced Toast example. Expect to see a lot us within the Eclipse community use this example when we talk about OSGi, Equinox and EclipseRT.

So what are you waiting for, go buy the book and learn about OSGi.

Over the past few weeks, I’ve helped plan and organize a panel at EclipseCon called The Future of Application Servers.

In the past few years, application servers have been evolving rapidly and with the advent of OSGi leaking into application servers, we are seeing modularity being introduced to the typical application developer. The main goals of the panel is to explore what application servers may look like in the future and how application server programming models may evolve. The panel will feature leaders from their respective application server communities… from Eclipse Virgo (dmServer) to IBM Websphere:

• Glyn Normington (VMWare/SpringSource)
• Graham Charters (IBM)
• Rich Sharples (JBoss/Red Hat)
• Mike Lehmann (Oracle)
• Greg Wilkins (webtide)

John R. Rymer from Forrester Research will be moderating the panel.

If you’re interested in the future of application servers, swing by and attend EclipseCon!

Over the holidays, I’ve been having a blast with the BUG from Bug Labs.

What is the BUG you ask? Well, it’s a bit hard to explain but on the high level think of it as programmable LEGOs for adults. It’s essentially a collection of electronic modules you can easily snap together and program to make any device or gadget you want. There are many modules available for the BUG…

Using these modules you can build any device you want. Once you have built creation, you can start developing applications for that particular device. For example, I can have the BUGlocate (GPS) and BUGmotion (accelerometer) modules and develop an application that takes a picture between a certain time and tweets the photo.

In my opinion, the coolest thing about the BUG is that it’s open source and enables modularity via OSGi. Under the covers, it is using the Concierge OSGi framework implementation due to size constraints.

Since I’m a hardened OSGi guy, I found developing applications for the BUG really easy. You interact with each of the modules and other applications via OSGi services. For example, the BUGmotion module exposes a com.buglabs.bug.module.motion.pub.IMotionSubject service which you can use to get information about motion events.

In terms of tooling, you’ll be right at home with the Eclipse-based Dragonfly SDK (if you’re an Eclipse person).

The SDK makes it pretty easy to launch, test, deploy and find applications. If you’re familiar with Eclipse and Java development, it should be really easy to get started. If you’re familiar with OSGi development in Eclipse, you’ll feel like you’re right at home. They also use some aspects of the Plug-in Development Environment (PDE) from Eclipse to make your life easier (although not as much as I’d like them too). On top of the tooling, the BUG also has an “app store” (called BUGNet) for all of its applications. It’s very easy to browse for applications and have the source downloaded into your SDK.

The ease of browsing and downloading BUG applications makes me jealous as an Eclipse user.

On the whole, I’m very pleased with the BUG. The BUG has changed the way I think about hardware and how it relates to modularity. Furthermore, any company who’s mission is to advance modularity by making it fun and allowing you to build cool devices get kudos from me.

On a side note, given the set of BUG modules, anyone have interesting ideas for an application to develop ? My current thinking is something that involves the motion module, the camera module, my refrigerator, twitter and late night snacking.

I don’t know if it’s just the last sprint before the end of the year for people, but there’s been a decent amount of buzz around software versioning lately. Maybe people (like Mirko Jahn) are just so frustrated with the status quo around versioning.

In the first case, the OSGi community has been blogging about the importance of semantic versioning in regards to the blah approach taken by the Jigsaw project. I find that Peter Kriens wrote an eloquent post about the importance of versioning in module systems like OSGi, especially when it comes to the semantics. In response to JSR 294’s versioning policy (vaguely Debian-like strings), Alex Blewitt has written a funny “JSR294 Fantasy Version Series” featuring a Linux and Roman Numeral versioning scheme. I highly recommend reading Alex’s blog series if you’re in need of a laugh.

I also learned today that the Groovy project is going through a modularization stage since the main Groovy JAR is now over 4MB and there’s a desire to break things apart (I wonder what monsters lurk under that big JAR). I hope the Groovy team considers a hard look at OSGi and also the importance of semantic versioning for its modules. Since it seems Groovy uses Maven as part of its build infrastructure, it should be easy to adopt OSGi with the upcoming Maven3 release and Tycho.

Ok, what about modularity and versioning outside the Java world? Well, to my surprise and excitement, there’s some activity in the Javascript space via the CommonJS group. The purpose of the CommonJS group is to build a JavaScript ecosystem for web servers, desktop and command line apps. As part of this effort, the group is developing a module and package system. While I still think they are a bit crazy, I’m happy to see that they are standardizing their versioning scheme (called SemVer) to include semantics.

If you look deeper into CommonJS and their module system, a lot of the metadata that they are defining for their modules map quite nicely to OSGi. Is it time that OSGi considers stepping out of the comfortable Java space and embrace modularity for other languages like JavaScript? There’s a lot of lessons the OSGi community can give to the modularity newcomers.

Has anyone seen anything else good when it comes to modularity and software versioning?

On a side note, sometimes I wonder if we need something like the Agile Manifesto but for Semantic Verisoning. I’d call it a Semantic Versioning Manifesto where people could sign it and support it. What do people think?

There was some activity on the JSR 294 mailing list this morning. The Jigsaw team came down from the mountains and has informed us that…

javac can compile source files belonging to modules inferred from the
modulepath and will enforce full module-private accessibility Real Soon
Now. Long-standing issues with bootstrapping the JDK have also been
addressed, such that modularized JDK libraries are used properly and
early in the build.

The libraries are modularized by mechanisms in the Jigsaw module system
including but not limited to recursive virtual modules, per-method
dependencies, packages split across modules, and structured
arbitrary-length versions. Sun finds these mechanisms useful for
compatibly restructuring vanilla packages in the JDK, and believes they
are applicable to clients of the JDK also.

Clearly, the Jigsaw model goes some way beyond “named and versioned
JARs”. How it overlaps with the OSGi model – designed for new, dynamic,
service-oriented applications – is not clear. Since both Jigsaw and OSGi
are defined in the first instance outside the JCP, it does not appear
that JSR 294 is well-positioned to define a module system that unifies
them. It would be a lowest-common-denominator design with
well-intentioned but irksome compromises. There has been no clear
support in the EG for the Simple Module System proposal, and I would
like to take it no further.

I just about stopped reading when I hit the “structured arbitrary-length versions” feature.

I dug a bit deeper in the Jigsaw Javadoc to see what this means exactly…

Nice… “Vaguely Debian-like version strings, for now.”

I had my rant on software versioning in general last week… this is a terrible idea in my book. Maybe I’m just in a bad mood because my inbox is under attack by Stephanie Meyer who happened to release a book with the same title as my favorite open source project. In module systems, version numbers should encode compatibility to keep us sane when developing and talking about things. Let me give you an example from OSGi land. Let’s say you’re writing an application and need to depend on log4j…

Require-Bundle: org.apache.log4j

You go to deploy your bundle and things work great. Your users are happy. Now you have a colleague take your bundle and deploy it into another system and you get a bunch of crazy exceptions. What happened? Well, first you didn’t specify a version range on your depending for log4j so basically your assumption is that your bundle would work with any version of log4j. At development time, you were using log4j 1.1 and the first system you were deploying to happened to use log4j 1.1 too. However, when your colleague went to reuse your bundle (since we can easily do that now since we have modules), things blow up because the system they went to deploy your bundle happened to have log4j 1.2… and the log4j development team unknowingly broke binary compatibility with the new release. To fix this problem… you can simply scope your depending on log4j…

Require-Bundle: org.apache.log4j;version=[1.1, 1.2)

Let’s use another example from OSGi land… commons collections. Let’s say you needed commons collections… you were developing with version 2.1 and you decided to be smart and scope you version range this time because you thought you learned your lesson.

Require-Bundle: org.apache.commons.collections;version=[2.1, 2.2)

Next what happened was that the commons collections development team released version 3.0 and declared it to be binary compatible with the 2.1 version. The potential environments your bundle can be deployed to is limited based on the scoped dependency above… even though your bundle would work fine in an environment where the commons collections 3.0 version was floating around. The ability to reuse your bundle was reduced.

The crux of the problem here is that you don’t know how to scope your version ranges and in particular… versions really don’t convey anything to you as a module developer. In OSGi land, there’s a four segment version and a mantra of encoding compatibility in your version number. Each segment of the version means something. To get a good idea of how to handle versioning on a large scale, we have a fantastic set of guidelines regarding versioning bundles at Eclipse. If everyone followed the same set of versioning guidelines, you can properly setup your dependencies and as a developer, you immediately can see meaning between what you’re developing against and what you may consume in the future.

My recommendation to the Jigsaw team would be to reconsider their stance on treating module versions like nothing. I would like to see binary compatibility encoding in version numbers and given the fact that Maven is moving to the OSGi versioning scheme… Jigsaw might as well also. It would be nice in Java if we had a consistent way to talk about the versions of software we develop for people to reuse.

Alright, I have to vent and go on a diatribe as its been awhile.

I was in the process of updating my iPhone and was greeted with these updates.

1.34? 1.7f? Why does Apple even bother showing the version number. I don’t even know what version of the software was previously installed. As a user, I only really care if there’s an update and what is included in that update. That’s it.

The way software engineers treat version numbers is sad. Since the inception of software, we haven’t been able to come up with a reasonable versioning scheme that can be shared across technologies. Even worse, versions really mean nothing. On top of that, we confuse marketing version numbers with internal software version numbers. Here are some examples of what I’ve come across.

Let’s start with Java. 1.0.3, 1.1.2, 1.4.2, 1.5.0, 1.6.0? Why both with the “1″ in the version number?

Next up we have is Windows… 3.1, 2000, XP, ME and 7. Awesome.

I remember when Netscape magically went from version 4 to 6. You know, we didn’t really need a version 5 because IE was already at version 6. We can’t let our competitors out version us.

In Linux… 2.3, 2.4.0, 2.6.8, 2.6.8.1… it’s cool when you can start adding version segments. Linux also seems to suffer the Java problem where the first digit is meaningless… maybe Linus’ favorite number is 2. On the bright side, the second segment in the version number at least signifies whether the code is considered stable or not. I’ll give bonus points to Linux for that.

SmallEiffel… -0.87, -0.79, -0.74… nice… software versions that start on the negative scale and apparently count down to zero.

In areas where modules exist and there’s a goal of reusable components (think OSGi and Maven), there’s a standard version scheme with a recommended policy on how to handle version changes. The only problem is that not everyone follows the scheme given their experience where version numbers mean nothing. It also doesn’t help that version schemes may be different amongst similar component technologies.

In the end, there really needs to be two types of versions, one for the marketing department and one for the development department. Confusing the two types is bad. It’s good to see that I’m not the only one that feels like this, there’s some funny commentary by Jeff Atwood on versioning.

Shame on us in the software industry for letting this crap continue.

I was reading over the OSGi survey results and noticed that people found that obtaining valid bundles is still a problem. I agree. I find obtaining bundles or setting up my development target akin to playing a game of Where’s Waldo.

As you can tell, I was never a fan of the game, but for some reason my family loved getting those books.

Last week, the PDE team had a little meeting of the minds to help think of ideas to tackle parts of that problem. There’s a lot more we can do to improve the developer experience around managing bundles since p2 is around now.

The PDE team is looking at improving the target definition story again. One aspect of the improvement is making targets managed by a p2 profile. This would make it easier to manage the target in the long run and potentially expose cool things you can do with targets. The aspect I’m mostly interested in is improving the developer experience and workflow with PDE.

I remember when I first started developing bundles, one big problem was just setting up your environment and finding the correct bundles. I remember staring at my Java source files and seeing evil compile errors for missing imports like this commons collections one.

Wouldn’t it be nice to take advantage of p2 repositories to help solve that problem?

p2 repositories are rich with metadata and you can query them for many interesting things like packages.

You can also query for things like bundles.

Heck, if you’re adventurous, you can also query for features!

Since the target platform is managed by p2, the necessary artifacts will be downloaded and available for you to use.

No more compile errors!

We can see that the commons collections bundle is now part of our dependencies.

That’s just one example of how to improve the bundle developer workflow using p2. The PDE team is exploring many options so if you have ideas on how to improve developer workflow, please let the PDE team know.

I’m presenting at Eclipse Summit Europe 2009 which happens in about a week from today.

I’ll be speaking about OSGi Versioning and Testing with Jeff McAffer and Paul VanderLei. Please consider coming to the talk, it should be fun and will involve discussion of painting fences with neighbors.

Here’s the abstract:

The old saying goes, good fences make for good neighbors. Modular systems are all about the fences, otherwise known as contracts. OSGi-based systems are simply compositions of modules and contracts. To express contracts in OSGi, we use version numbers. From our experience in the Eclipse community, we have found that managing API and these contracts a difficult process.

In the first part of this talk, we’ll discuss the problem of version numbering and managing change. We’ll bring forward real world examples that demonstrate the complexities in managing change, including examples from the Eclipse community where change has caused havoc. There will be a demonstration of some the tools we developed at Eclipse to help manage change in your OSGi-based applications.

In the second part of this talk we draw on experiences and examples from the OSGi and Equinox book to identify key elements of modularity and point out best practices for designing and building modular systems using OSGi. Along the way we highlight pitfalls and anti-patterns and talk about ease of testing as barometer for modularity.

On a side note, over 30 people have signed up for the Eclipse Summit Europe 5K run every morning!

Please don’t forget to sign up if you want to run and burn off some Kirsch.

A few days ago I had one of those moments that made me smile, let me describe it in pictures…

I recently was hacking on the JGit project as I’m working on getting a build ready for the project.

What did I notice? Well, there was some evil UI code in the JGit bundle.

You know what to do with evil UI code? Time to move it into another bundle!

Sweet! Now we have a nice separation between core and ui layers, right?

Oh no… compile errors… what the heck?

It looks like some of the code in the core bundle was referencing UI pieces. This is an obvious bad practice since there is the common use case that people may want to run your code in a headless fashion. However, as a developer, it’s common to start out and just group your code into one bundle. It’s sad but it’s just the reality of development. This reason should also highlight to you why modularity doesn’t come for free… you have to think about your dependencies and architecture. If you produce complex code first and try to modularize it later… you’ll end up in painful situation. Thankfully, when you have good tools that manage your classpath for you, errors like this become evident.

Ok, enough of my diatribes, back to getting a build for JGit in place at Eclipse.

On a side note, I’ll soon blog about using EGit and Gerrit to develop JGit, stay tuned.