I've been thinking what is the best way to create a fully bottom-up build infrastructure for Eclipse-based product. What I mean by "fully" is:
- Developer commits code.
- A new build starts automatically.
- If the build succeeds - run tests.
- If tests are successful - tag the code.
Advantages of this scheme are the following:
- I'll always know what's the status of the product in CVS.
- I can choose the "best" package and promote it without invoking any builds.
Looks simple, isn't it? Now, how would I achieve this with what I have in my hands... In my hands I have:
- CVS repository
- Hudson
- Eclipse
Unfortunately, Hudson can't be used for 100% when building Eclipse plug-ins, since it's rather used as a wrapper to the Eclipse PDE builder. Here, I don't use the ability of Hudson to poll SCM, checkout & tag, present history, etc... OK, I can setup some CVS handler that invokes build in Hudson by accessing URL. So, the first two problems are solved. Hudson executes build script that uses PDE headless builder to build & test my plug-ins. If the build & test where unsuccessful, then I'll see very sad
Mr. Hudson and everything is good so far. But if the build was OK, and number of failed tests is less or equal than in the previous build - then I have to tag the sources. And here I miss the "Team -> Release" action in Ant :-) Even if there was such an action the problem still remains, since plug-ins are versioned with 'HEAD' (and I don't want to use common timestamp version for all plug-ins, but rather for those that have changed from the last tag).
As a conclusion: I see a lot of black magic work and hacks to get this done. Or... am I missing some pretty solution here?
12 comments:
The problem with tagging after the build (especially on CVS) is that someone could make commits while build is running or even while tagging is happening. So, you really need to tag beforehand. You can delete the tag if build failed.
I agree with @eugene - better to tag first, remove later. The system you describe works reasonably well with a Subversion repo, since the tagging seems to work in a different way to CVS-style tagging.
Tagging post-build can work fine with CVS provided the build tool checks out code based on the timestamp the build was triggered (any real continuous integration tool should do this rather than blindly checking out HEAD - I imagine Hudson does it under the covers).
If code is committed during a long-running build, you need to trigger another build anyway whether you tag first or last.
hmm this looks like protecting CVS repo from developers. In small well organized team it should be easier to make sure everyone runs tests before comitting. In your scenario - imagine that some other code-unrelated issue causes build failure. That'd block developers from tagging even if they're not guilty.
The Athena common builder can do what you are asking. I've also done this using the Ant4Eclipse project which doesn't require running eclipse headless just to build plugins or features. (http://ant4eclipse.sourceforge.net).
Since Nick is a build committer on PDT, I'd suggest migrating PDT builds to Hudson if you haven't and implementing the builds the way you want. I.e. code checked into Head gets rebuilt as soon as their are changes, tests are run, and code is tagged based on the code that was checked out.
You can customize version for plug-ins on PDE headless build level in customTargets.xml preSetup target.
Unfortunately I don't know anything that would work out of the box with CVS tags.
One thing still missing in the Athena/Hudson setup is implementing the weekly tag&release automation that PDT (and GEF and GMF) currently enjoys.
It's an ugly piece of shell script, which means I need to find time (and/or sufficient pain/motivation) to rewrite it in Ant or Groovy so it's more portable and friendly.
But it's not black magic - it's just a scheduled `cvs rtag` for all the plugins which have changed since the last time we rtagged, followed by an update to the map file(s).
Bob's your uncle, pre-build weekly (or ad hoc as needed) tag&release, same as the Team > Release action in Eclipse. Coming soon to Athena.
Oh, and FYI, because of how Hudson on build.eclipse runs, you will never be able to use the tag-on-good-build plugins. Hudson is not a committer on your project, so it does NOT have commit (or rtag) rights to your sources, whether they're in cvs, svn, or git.
Only workarounds that spring to mind are:
* run tagging as a committer on a schedule that's close to what Hudson uses for building, or
* concoct some sort of handshaking/conch-passing setup where Hudson can tell you it's time to tag, and your crontab watches for that message to respond accordingly.
The commercial Bamboo from Atlassian does all of this. It has a real good interface, is based on Subversion and even integrates information from Jira - their issue management system - in the relevant places. If/when errors occurs, new issue can be created easily...
@Eugene Kuleshov, @oisin: AFAIK tagging not updated source is legal in CVS. The tag will apply on versions that present in a local copy.
@Tony Madsen: is there something that makes better integration with Eclipse PDE builder in Bamboo comparing to Hudson?
Another approach is to build your Eclipse product using maven eg. tycho's mojos.
I am currently building using this approach:
1. Develop Eclipse RCP product by using Eclipse PDE for developing features and when feature is finished, then check code into Subversion repository.
2. A Hudson job is setup to poll the product's subversion locations for changes and if a change is detected, then it will kick of a build that will build the product using tycho's mojos, using the install goal. This build includes running junit tests and the plugins and product are installed into local maven repository.
3. We also have another Hudson job that will run at midnight that will build the product using the maven deploy goal to deploy to remote repository and is also setup to run code quality tools, eg. emma, pmd, and findbugs.
4. The product that is produced from the nightly build is then downloaded from the Maven repository on a regular basis during the day and unzipped to a location that is set as the Target Platform in the Eclipse workspace.
We have found that this approach works quite well for a team of 10 developers. We tried using m2eclipse to build the product inside of Eclipse instead of using PDE, but we found that the building took too long and too fragile, so we reverted to using PDE inside of Eclipse.
I wrote an article about how we did Eclipse Headless builds with Zed. It covers some of the ideas you're exploring.
Also agree 100% with the above comments. We always tag before the build, and build based on the tag.
Post a Comment