Twitter github

Jigsaw Versioning is Ridiculous

There was some activity on the JSR 294 mailing list this morning. The Jigsaw team came down from the mountains and has informed us that…

javac can compile source files belonging to modules inferred from the
modulepath and will enforce full module-private accessibility Real Soon
Now. Long-standing issues with bootstrapping the JDK have also been
addressed, such that modularized JDK libraries are used properly and
early in the build.

The libraries are modularized by mechanisms in the Jigsaw module system
including but not limited to recursive virtual modules, per-method
dependencies, packages split across modules, and structured
arbitrary-length versions.
Sun finds these mechanisms useful for
compatibly restructuring vanilla packages in the JDK, and believes they
are applicable to clients of the JDK also.

Clearly, the Jigsaw model goes some way beyond “named and versioned
JARs”. How it overlaps with the OSGi model – designed for new, dynamic,
service-oriented applications – is not clear. Since both Jigsaw and OSGi
are defined in the first instance outside the JCP, it does not appear
that JSR 294 is well-positioned to define a module system that unifies
them. It would be a lowest-common-denominator design with
well-intentioned but irksome compromises. There has been no clear
support in the EG for the Simple Module System proposal, and I would
like to take it no further.

I just about stopped reading when I hit the “structured arbitrary-length versions” feature.

I dug a bit deeper in the Jigsaw Javadoc to see what this means exactly…

Jigsaw Version

Nice… “Vaguely Debian-like version strings, for now.”

I had my rant on software versioning in general last week… this is a terrible idea in my book. Maybe I’m just in a bad mood because my inbox is under attack by Stephanie Meyer who happened to release a book with the same title as my favorite open source project. In module systems, version numbers should encode compatibility to keep us sane when developing and talking about things. Let me give you an example from OSGi land. Let’s say you’re writing an application and need to depend on log4j…

Require-Bundle: org.apache.log4j

You go to deploy your bundle and things work great. Your users are happy. Now you have a colleague take your bundle and deploy it into another system and you get a bunch of crazy exceptions. What happened? Well, first you didn’t specify a version range on your depending for log4j so basically your assumption is that your bundle would work with any version of log4j. At development time, you were using log4j 1.1 and the first system you were deploying to happened to use log4j 1.1 too. However, when your colleague went to reuse your bundle (since we can easily do that now since we have modules), things blow up because the system they went to deploy your bundle happened to have log4j 1.2… and the log4j development team unknowingly broke binary compatibility with the new release. To fix this problem… you can simply scope your depending on log4j…

Require-Bundle: org.apache.log4j;version=[1.1, 1.2)

Let’s use another example from OSGi land… commons collections. Let’s say you needed commons collections… you were developing with version 2.1 and you decided to be smart and scope you version range this time because you thought you learned your lesson.

Require-Bundle: org.apache.commons.collections;version=[2.1, 2.2)

Next what happened was that the commons collections development team released version 3.0 and declared it to be binary compatible with the 2.1 version. The potential environments your bundle can be deployed to is limited based on the scoped dependency above… even though your bundle would work fine in an environment where the commons collections 3.0 version was floating around. The ability to reuse your bundle was reduced.

The crux of the problem here is that you don’t know how to scope your version ranges and in particular… versions really don’t convey anything to you as a module developer. In OSGi land, there’s a four segment version and a mantra of encoding compatibility in your version number. Each segment of the version means something. To get a good idea of how to handle versioning on a large scale, we have a fantastic set of guidelines regarding versioning bundles at Eclipse. If everyone followed the same set of versioning guidelines, you can properly setup your dependencies and as a developer, you immediately can see meaning between what you’re developing against and what you may consume in the future.

My recommendation to the Jigsaw team would be to reconsider their stance on treating module versions like nothing. I would like to see binary compatibility encoding in version numbers and given the fact that Maven is moving to the OSGi versioning scheme… Jigsaw might as well also. It would be nice in Java if we had a consistent way to talk about the versions of software we develop for people to reuse.

  • http://intellectualcramps.blogspot.com/ David Carver

    My personal opinion is that there are valid times that the major version needs to be incremented regardless if it broke backwards compatibility or not. If your framework implemented some significant new functionality, but also still provides the existing API and framework, it may still make sense to bump the major version. You might consider this a marketing version, but in certain cases I wouldn’t. I don’t necessarily consider major new functionality a minor version increment.

  • http://intellectualcramps.blogspot.com David Carver

    My personal opinion is that there are valid times that the major version needs to be incremented regardless if it broke backwards compatibility or not. If your framework implemented some significant new functionality, but also still provides the existing API and framework, it may still make sense to bump the major version. You might consider this a marketing version, but in certain cases I wouldn’t. I don’t necessarily consider major new functionality a minor version increment.

  • http://aniszczyk.org/ Chris Aniszczyk

    @David Carver, I think what you’re talking about is a marketing version… visible to those people that aren’t developers. If you go back to my example… how do you scope your module dependencies properly if version changes are arbitrary? You limit the potential reuse of your bundle and how developers may perceive changes. If everyone does things differently, it’s a crap-shoot.

    Imagine how powerful things can be if we as developers can know exactly what types of changes happened given the version number of a module?

  • http://aniszczyk.org Chris Aniszczyk

    @David Carver, I think what you’re talking about is a marketing version… visible to those people that aren’t developers. If you go back to my example… how do you scope your module dependencies properly if version changes are arbitrary? You limit the potential reuse of your bundle and how developers may perceive changes. If everyone does things differently, it’s a crap-shoot.

    Imagine how powerful things can be if we as developers can know exactly what types of changes happened given the version number of a module?

  • Fatih Coskun

    What Sun folks are trying to achieve is bringing all the various versioning formats together, such that jigsaw can be used with ANY library system (e.g. debian packages, maven repositories, osgi, etc.). The format has to be so general, that it can accept any of the version strings of the different systems. Of course you have to be able to specify dependency on a particular version, and I think Sun will address this issue soon enough.

  • Fatih Coskun

    What Sun folks are trying to achieve is bringing all the various versioning formats together, such that jigsaw can be used with ANY library system (e.g. debian packages, maven repositories, osgi, etc.). The format has to be so general, that it can accept any of the version strings of the different systems. Of course you have to be able to specify dependency on a particular version, and I think Sun will address this issue soon enough.

  • Jannick

    Disclaimer: No experience using OSGi or similar. Couldn’t this issue be resolved by having the commons 3.0 module meta-data include a binary-compatible-replacement-off-versions: …. kind of description? Proper use of major version numbers is ideal for developers making use of a module, but I would personally hate to operate with different version numbers for marketing and developer use.

  • Jannick

    Disclaimer: No experience using OSGi or similar. Couldn’t this issue be resolved by having the commons 3.0 module meta-data include a binary-compatible-replacement-off-versions: …. kind of description? Proper use of major version numbers is ideal for developers making use of a module, but I would personally hate to operate with different version numbers for marketing and developer use.

  • Konstantin Komissarchik

    OSGi’s rigid versioning scheme is my least favorite feature of an an otherwise good system. I completely agree that it is good to have a meaningful version convention that you publish to your downstream consumers (especially if you are building a library) and there is nothing wrong with publishing suggested versioning conventions for people to adopt, but to suggest that one versioning convention should be good for everyone under all circumstances is absurd.

    I don’t know what genius at OSGi working group decided that four version segments should be enough for everyone. The product that I am currently working on needs five segments in order to align with the version that is assigned to the product (four segments) and to allow for the essential time stamp. Now, you may not agree that product version should align with bundle version and that may be an appropriate stance in some cases, but in my situation, there is no benefit out of having two separate versioning conventions but there is plenty of confusion by everyone involved. Unfortunately, I have no way to improve the situation because whoever designed OSGi versioning decided that four segments should be enough for everyone.

    Now, I don’t know what “debian-like” versioning means, but so far it sounds like Sun is trying to adopt a more flexible versioning system, which is a very good thing in my book.

  • Konstantin Komissarchik

    OSGi’s rigid versioning scheme is my least favorite feature of an an otherwise good system. I completely agree that it is good to have a meaningful version convention that you publish to your downstream consumers (especially if you are building a library) and there is nothing wrong with publishing suggested versioning conventions for people to adopt, but to suggest that one versioning convention should be good for everyone under all circumstances is absurd.

    I don’t know what genius at OSGi working group decided that four version segments should be enough for everyone. The product that I am currently working on needs five segments in order to align with the version that is assigned to the product (four segments) and to allow for the essential time stamp. Now, you may not agree that product version should align with bundle version and that may be an appropriate stance in some cases, but in my situation, there is no benefit out of having two separate versioning conventions but there is plenty of confusion by everyone involved. Unfortunately, I have no way to improve the situation because whoever designed OSGi versioning decided that four segments should be enough for everyone.

    Now, I don’t know what “debian-like” versioning means, but so far it sounds like Sun is trying to adopt a more flexible versioning system, which is a very good thing in my book.

  • http://aniszczyk.org/ Chris Aniszczyk

    @Konstantin Komissarchik, 5… do you really need 5 or is that just the way things originally started in your product? Is that a marketing number?

    The fourth segment in OSGi could include a timestamp and other things like patch information. It’s the three major

    I don’t think it’s crazy to just pick a versioning scheme and stick to it in Java land. Otherwise, it’s going to be difficult to reuse libraries and scope dependencies properly. In the software industry we have solved a lot of problems… it’s sad that we can’t come up with a reusable versioning scheme across a language.

    (btw, here’s the link for Debian version strings)

  • http://aniszczyk.org Chris Aniszczyk

    @Konstantin Komissarchik, 5… do you really need 5 or is that just the way things originally started in your product? Is that a marketing number?

    The fourth segment in OSGi could include a timestamp and other things like patch information. It’s the three major

    I don’t think it’s crazy to just pick a versioning scheme and stick to it in Java land. Otherwise, it’s going to be difficult to reuse libraries and scope dependencies properly. In the software industry we have solved a lot of problems… it’s sad that we can’t come up with a reusable versioning scheme across a language.

    (btw, here’s the link for Debian version strings)

  • Konstantin Komissarchik

    > do you really need 5 or is that just the way things originally started
    > in your product? Is that a marketing number?

    The product version is assigned from higher and is controlled by marketing / product-development people. That gives us four segments, with fourth being the one that’s frequently the most interesting. We cannot concat timestamp into the fourth segment as that’s not allowed by product version presentation rules. We are allowed to use the fifth segment for our own purposes, but OSGi does not allow us.

    > I don’t think it’s crazy to just pick a versioning scheme and stick
    > to it in Java land.

    I think the underlying framework needs to be as flexible as possible when it comes to versions. What’s important is not that everyone uses the same versionining convention, but that everyone (or at least providers of bundles that are likely to be reused) publish their convention and adhere to them.

  • Konstantin Komissarchik

    > do you really need 5 or is that just the way things originally started
    > in your product? Is that a marketing number?

    The product version is assigned from higher and is controlled by marketing / product-development people. That gives us four segments, with fourth being the one that’s frequently the most interesting. We cannot concat timestamp into the fourth segment as that’s not allowed by product version presentation rules. We are allowed to use the fifth segment for our own purposes, but OSGi does not allow us.

    > I don’t think it’s crazy to just pick a versioning scheme and stick
    > to it in Java land.

    I think the underlying framework needs to be as flexible as possible when it comes to versions. What’s important is not that everyone uses the same versionining convention, but that everyone (or at least providers of bundles that are likely to be reused) publish their convention and adhere to them.

  • Miroslav Pokorny

    It appears figuring out versions requires modules to not only declare what they need bt also for modules to also delcare what they are eqivalent too. One cannot jst declare that a modle needs a particular range because a new release of said dependency could be compatible but perhaps had an unforseen increase in the version number that makes it out of range even though it is compatible. I believe a “i-am-backward-compatible-with” would help. Naturally this would probably only work provided the public api has not changed, and specifying such a value would imply that said module was a drop in replacement with no changes.

  • Miroslav Pokorny

    It appears figuring out versions requires modules to not only declare what they need bt also for modules to also delcare what they are eqivalent too. One cannot jst declare that a modle needs a particular range because a new release of said dependency could be compatible but perhaps had an unforseen increase in the version number that makes it out of range even though it is compatible. I believe a “i-am-backward-compatible-with” would help. Naturally this would probably only work provided the public api has not changed, and specifying such a value would imply that said module was a drop in replacement with no changes.

  • http://zwitserloot.com/ Reinier Zwitserloot

    Sun decided that the burden of having a module version number that is NOT the same as the marketing version number is the lesser evil compared to having module version numbers that have no explicitly defined meanings per version sub-number.

    A debatable issue, to be sure. There’s a third way out, of course: Enforcing that marketing no longer decides versions. If you think you can enforce this across the entire java community, you’re delusional.

    So, two fairly legitimate strategies. I have a slight preference for OSGi’s model over sun’s choice, but, eh. Sun wants to be compatible with stuff like debian, which already has semantic meanings for this stuff, and it’s somewhat different to the OSGi model, so I get where they are coming from.

    And yet you start ranting like a crazy man – calling it ‘ridiculous’ and ‘terrible’. You’re not doing OSGi any favours by acting this way.

    FYI, jigsaw allows a similar version bounding scheme as OSGi. jigsaw splits the version string into pieces along punctuation (dots and dashes), and per block you can specify bounds, including numeric ones, so [1.1, 1.2) has similar meaning. The problem is of course that you have to check the exact meaning of each level of number per module, but you yourself said that log4j broke backwards compatibility by accident. They would release the wrong version. Version systems suck unless a given version release is never ever changed, so once log4j released this wrongly labelled version, you’re stuck with it forever.

    Pragmatically speaking, then, this much celebrated intrinsic meaning of version numbers is a guideline at best. Add in projects throwing documentation to the wind and going with their own versioning schemes because marketing said so, and the OSGi situation isn’t all that much different from jigsaw’s proposals, except that jigsaw’s proposal is far more flexible.

    Now, which part of that argument is ‘ridiculous’ and ‘terrible’? I might be overestimating the amount of version fail that’s going to occur, but you can’t deny accidents happen.

    NB: Regarding take-backsies on misversioned stuff, jigsaw’s plan is for (module-name+version, jar*) tuples to be stored in a central repository on the local machine, so with that plan, taking back a released version to reversion it is just not feasible.

    *) It won’t be a jar, but some other file format, but I doubt this is a relevant detail.

  • http://zwitserloot.com/ Reinier Zwitserloot

    Sun decided that the burden of having a module version number that is NOT the same as the marketing version number is the lesser evil compared to having module version numbers that have no explicitly defined meanings per version sub-number.

    A debatable issue, to be sure. There’s a third way out, of course: Enforcing that marketing no longer decides versions. If you think you can enforce this across the entire java community, you’re delusional.

    So, two fairly legitimate strategies. I have a slight preference for OSGi’s model over sun’s choice, but, eh. Sun wants to be compatible with stuff like debian, which already has semantic meanings for this stuff, and it’s somewhat different to the OSGi model, so I get where they are coming from.

    And yet you start ranting like a crazy man – calling it ‘ridiculous’ and ‘terrible’. You’re not doing OSGi any favours by acting this way.

    FYI, jigsaw allows a similar version bounding scheme as OSGi. jigsaw splits the version string into pieces along punctuation (dots and dashes), and per block you can specify bounds, including numeric ones, so [1.1, 1.2) has similar meaning. The problem is of course that you have to check the exact meaning of each level of number per module, but you yourself said that log4j broke backwards compatibility by accident. They would release the wrong version. Version systems suck unless a given version release is never ever changed, so once log4j released this wrongly labelled version, you’re stuck with it forever.

    Pragmatically speaking, then, this much celebrated intrinsic meaning of version numbers is a guideline at best. Add in projects throwing documentation to the wind and going with their own versioning schemes because marketing said so, and the OSGi situation isn’t all that much different from jigsaw’s proposals, except that jigsaw’s proposal is far more flexible.

    Now, which part of that argument is ‘ridiculous’ and ‘terrible’? I might be overestimating the amount of version fail that’s going to occur, but you can’t deny accidents happen.

    NB: Regarding take-backsies on misversioned stuff, jigsaw’s plan is for (module-name+version, jar*) tuples to be stored in a central repository on the local machine, so with that plan, taking back a released version to reversion it is just not feasible.

    *) It won’t be a jar, but some other file format, but I doubt this is a relevant detail.

  • Thomas Hallgren

    @Fatih Coskun
    If that’s what they are trying to do, then they should have a look at the p2 OmniVersion. It may look complex at first glance but that’s what it takes to unify different versioning formats. There’s not much use unifying formats if you cannot understand the semantics.

  • Thomas Hallgren

    @Fatih Coskun
    If that’s what they are trying to do, then they should have a look at the p2 OmniVersion. It may look complex at first glance but that’s what it takes to unify different versioning formats. There’s not much use unifying formats if you cannot understand the semantics.

  • http://wagenknecht.org/blog/ Gunnar Wagenknecht

    Miroslav Pokorny :
    […] I believe a “i-am-backward-compatible-with” would help.[…]

    I think you hit the nail on its head. The issue of specifying version ranges at import time just delegates the problem.

    Chris, you had some very good examples in your blog. But in all cases it was *never* the clients fault. In all examples the producers just didn’t get the versioning right. It really is the task of a producer to decide which versions they want to be compatible to. This should be as dead easy as declaring it at export time, i.e. a package export should say “hey, I’m version 3.0 and I’m binary compatible to 2.1, 2.2 and whatever was out there before, if not my module vendor screwed up”.

  • http://wagenknecht.org/blog/ Gunnar Wagenknecht

    Miroslav Pokorny :
    […] I believe a “i-am-backward-compatible-with” would help.[…]

    I think you hit the nail on its head. The issue of specifying version ranges at import time just delegates the problem.

    Chris, you had some very good examples in your blog. But in all cases it was *never* the clients fault. In all examples the producers just didn’t get the versioning right. It really is the task of a producer to decide which versions they want to be compatible to. This should be as dead easy as declaring it at export time, i.e. a package export should say “hey, I’m version 3.0 and I’m binary compatible to 2.1, 2.2 and whatever was out there before, if not my module vendor screwed up”.

  • Luke Patterson

    Jannick :
    Couldn’t this issue be resolved by having the commons 3.0 module meta-data include a binary-compatible-replacement-off-versions:

    This discussion wouldn’t be complete without this tidbit from Peter Kriens:

    “… I believe life would have been easier if we would have had export ranges because the exporter of a package knows exactly to which version he is compatible, being the responsible person for that package. If he changes a package, he is the one thinking and deciding about backward compatibility. The importer is almost always guessing. That is, if I make a change to a package called p, I can knowingly say that I am package p version 2, but I am completely compatible to version 1 I therefore can export p;[1,2]. In contrast, if I import package p I have no clue if version 1 and 2 will be compatible except for the convention of version number interpretation. Alas, export versions did not make it to R4.”

    http://www.osgi.org/blog/2008/02/research-challenges-for-osgi.html

  • Luke Patterson

    Jannick :
    Couldn’t this issue be resolved by having the commons 3.0 module meta-data include a binary-compatible-replacement-off-versions:

    This discussion wouldn’t be complete without this tidbit from Peter Kriens:

    “… I believe life would have been easier if we would have had export ranges because the exporter of a package knows exactly to which version he is compatible, being the responsible person for that package. If he changes a package, he is the one thinking and deciding about backward compatibility. The importer is almost always guessing. That is, if I make a change to a package called p, I can knowingly say that I am package p version 2, but I am completely compatible to version 1 I therefore can export p;[1,2]. In contrast, if I import package p I have no clue if version 1 and 2 will be compatible except for the convention of version number interpretation. Alas, export versions did not make it to R4.”

    http://www.osgi.org/blog/2008/02/research-challenges-for-osgi.html

  • http://aniszczyk.org/ Chris Aniszczyk

    Yes, I think that this would have been easier if we had export ranges or some other metadata in the MANIFEST.MF indicating binary compatibility.

    However, this still doesn’t give us any consistent semantics about versions… it would be up to the decision of each bundle supplier how to treat their version values. It’s a step forward, but I think we would all be doing ourselves a favor if we just chose a reasonable versioning scheme and stuck with it. What do we really get out of coming up with version numbers for our components? A chance to express some creativity? Ideally, I would love to have it happen automatically for us based on changes in the code.

  • http://aniszczyk.org Chris Aniszczyk

    Yes, I think that this would have been easier if we had export ranges or some other metadata in the MANIFEST.MF indicating binary compatibility.

    However, this still doesn’t give us any consistent semantics about versions… it would be up to the decision of each bundle supplier how to treat their version values. It’s a step forward, but I think we would all be doing ourselves a favor if we just chose a reasonable versioning scheme and stuck with it. What do we really get out of coming up with version numbers for our components? A chance to express some creativity? Ideally, I would love to have it happen automatically for us based on changes in the code.

  • http://aniszczyk.org/ Chris Aniszczyk

    Reinier Zwitserloot :
    Now, which part of that argument is ‘ridiculous’ and ‘terrible’? I might be overestimating the amount of version fail that’s going to occur, but you can’t deny accidents happen.

    My biggest problem was that Sun hasn’t really reached out to the OSGi community for help in building a module system. They just came down from the mountains with their solution. However, that’s a topic of another post. Also, I thought people would be a bit more thick-skinned by now working in open source.

    My point is that versioning is critical to module systems and they can’t just be an afterthought. Having a consistent version scheme so you can express contracts is mandatory. Just look at the versioning landscape in Java… it’s a mess. I would just like to clean things up given a chance.

  • http://aniszczyk.org Chris Aniszczyk

    Reinier Zwitserloot :

    Now, which part of that argument is ‘ridiculous’ and ‘terrible’? I might be overestimating the amount of version fail that’s going to occur, but you can’t deny accidents happen.

    My biggest problem was that Sun hasn’t really reached out to the OSGi community for help in building a module system. They just came down from the mountains with their solution. However, that’s a topic of another post. Also, I thought people would be a bit more thick-skinned by now working in open source.

    My point is that versioning is critical to module systems and they can’t just be an afterthought. Having a consistent version scheme so you can express contracts is mandatory. Just look at the versioning landscape in Java… it’s a mess. I would just like to clean things up given a chance.

  • Alex Buckley

    The Jigsaw version scheme is very similar to OmniVersion. The OmniVersion link has very little to say on semantics; the meaning of individual segments is deferred to the underlying system (Mozilla, RedHat, Apache…). The OmniVersion comparator assumes structured input of arbitrary length, just like the Jigsaw comparator.

  • Alex Buckley

    The design and implementation of the Jigsaw module system is carried out in OpenJDK's Project Jigsaw. The project's mailing list would have been an excellent place to comment on the Jigsaw version scheme. Unfortunately, there has been very little activity in the OpenJDK project by members of the OSGi community. This may be because the goals of the Jigsaw module system are very different to the goals of the OSGi module system, as Sun has documented in recent public presentations (Strange Loop 2009, Devoxx 2009).

  • Alex Buckley

    The Jigsaw version scheme is very similar to OmniVersion. The OmniVersion link has very little to say on semantics; the meaning of individual segments is deferred to the underlying system (Mozilla, RedHat, Apache…). The OmniVersion comparator assumes structured input of arbitrary length, just like the Jigsaw comparator.

  • Alex Buckley

    The design and implementation of the Jigsaw module system is carried out in OpenJDK's Project Jigsaw. The project's mailing list would have been an excellent place to comment on the Jigsaw version scheme. Unfortunately, there has been very little activity in the OpenJDK project by members of the OSGi community. This may be because the goals of the Jigsaw module system are very different to the goals of the OSGi module system, as Sun has documented in recent public presentations (Strange Loop 2009, Devoxx 2009).

  • Pingback: Is there a future for software versioning? | All the small things()