I recently released my first open source library for Java, MDBI. I learnt a lot about the Java open-source ecosystem as part of this process, and this blog summarises that in the hope that it will be useful to others. Specifically, the post will explain how to set up a project using the modern Gradle build system to build code and deploy it to the standard Maven Central repository from the command line really easily.
In the Haskell ecosystem, everyone uses Cabal and Hackage, which are developed by the same people and tightly integrated. In contrast, Java’s ecosystem is a bit more fragmented: build systems and package repositiories are managed by different organisations, and you need to do a bit of integration work to join everything up.
No-one can publish directly to Maven Central: instead you need to publish your project to an “approved repository”, from where it will be synced to Central. Sonatype OSS is an approved repository that Sonatype (the company that runs Maven Central) provide free of charge specifically for open-source projects. We will use this to get our artifacts into Central, so go and follow the sign-up instructions now.
Your application will be manually reviewed by a Sonatype employee and approved within one or two working days. If you want an example of what this process looks like you can take a look at the ticket I raised for my MDBI project.
Sonatype OSS is a functional enough way to get your artifacts onto Central, but it has some irritating features. In particular, when you want to make a release you need to first push your artifacts to OSS, and then use an ugly and confusing web interface called Sonatype Nexus to actually “promote” this to Central. I wanted the release to Central to be totally automated, and the easiest way to use that is to have a 3rd party deal with pushing to and then promoting from OSS. For this reason, you should also sign up with Bintray (you can do this with one click if you have a GitHub account).
Bintray is run by a company called JFrog and basically seems to be a Nexus alternative. JFrog run a Maven repository called JCenter, and it’s easy to publish to that via Bintray. Once it’s on JCenter we’ll be able to push and promote it on Sonatype OSS fully automatically.
We also need to create a Bintray “package” within your Bintray Maven repository. Do this via the Bintray interface — it should be self-explanatory. Use the button on the package page to request it be linked to JCenter (this was approved within a couple of hours for me).
We’ll also need a GPG public/private key pair. Let’s set that up now:
- Open up a terminal and run
gpg --gen-key. Accept all the defaults about the algorithm to use, and enter a name, email and passphrase of your choosing.
If you run
gpg --list-public-keysyou should see something like this:
/Users/mbolingbroke/.gnupg/pubring.gpg -------------------------------------- pub 2048R/3117F02B 2015-11-18 uid Max Bolingbroke sub 2048R/15245385 2015-11-18
Whatever is in place of
3117F02Bis the name of your key. I’ll call this
$KEYNAMEfrom now on.
gpg --keyserver hkp://pool.sks-keyservers.net --send-keys $KEYNAMEto publish your key.
gpg -a --export-key $KEYNAMEand
gpg -a --export-secret-key $KEYNAMEto get your public and secret keys as ASCII text. Edit your Bintray account and paste these into the “GPG Signing” part of the settings.
- Edit your personal Maven repository on Bintray and select the option to “GPG Sign uploaded files automatically”. Don’t use Bintray’s public/private key pair.
Now you have your Bintray and OSS accounts we can move on to setting up Gradle.
The key problem we’re trying to solve with our Gradle build is producing a set of JARs that meet the Maven Central requirements. What this boils down to is ensuring that we provide:
- The actual JAR file that people will run.
- Source JARs containing the code that we built.
- Javadoc JARs containing compiled the HTML help files.
- GPG signatures for all of the above. (This is why we created a GPG key above.)
- A POM file containing project metadata.
To satisfy these requirements we’re going to use gradle-nexus-plugin. The resulting (unsigned, but otherwise Central-compliant) artifacts will then be uploaded to Bintray (and eventually Sonatype OSS + Central) using gradle-bintray-plugin. I also use one more plugin — Palantir’s gradle-gitsemver — to avoid having to update the Gradle file whenever the version number changes. Our Gradle file begins by pulling all those plugins in:
Now we have the usual Gradle configuration describing how to build the
JAR. Note the use of the
semverVersion() function (provided by the
gradle-gitsemver plugin) which returns a version number derived from
from the most recent Git tag of the form
vX.Y.Z. Despite the name of
the plugin, there is no requirement to actually adhere to the principles
of Semantic Versioning to use it: the only
requirements for the version numbers are syntactic.
(Obviously your group, project name, description, dependencies etc will differ from this. Hopefully it’s clear which parts of this example Gradle file you’ll need to change for your project and which you can copy verbatim.)
Now we need to configure
gradle-nexus-plugin to generate the POM. Just
by the act of including the plugin we have already arranged for the
appropriate JARs to be generated, but the plugin can’t figure out the
full contents of the POM by itself.
Note that I’ve explicitly turned off the automatic artifact signing capability of the Nexus plugin. Theoretically we should be able to keep this turned on, and sign everything locally before pushing to Bintray. This would mean that we wouldn’t have to give Bintray our private key. In practice, if you sign things locally Bintray seems to mangle the signature filenames so they become unusable…
Finally, we need to configure the Bintray sync:
We do this conditionally because we still want people to be able to use
the Gradle file even if they don’t have your a username and password set
up. In order to make these credentials available to the script when run
on your machine, you need to create a
with contents like this:
# These 3 are optional: they'll be needed if you ever use the nexus plugin with 'sign = true' (the default) signing.keyId= signing.password= signing.secretKeyRingFile= nexusUsername= nexusPassword= bintrayUsername= bintrayApiKey=
You can see the complete, commented, Gradle file that I’m using in my project on Github.
Your first release
We should now be ready to go (assuming your Sonatype OSS and JCenter setup requests have been approved). Let’s make a release! Go to the terminal and type:
git tag v1.0.0 gradle bintrayUpload
If everything works, you’ll get a
BUILD SUCCESSFUL message after a
minute or so. Your new version should be visible on the Bintray package
page (and JCenter) immediately, and will appear on Maven Central shortly
If you want to go the whole hog and have your continuous integration (e.g. the excellent Travis) make these automatic deploys after every passing build, this guide for SBT looks useful. However, I didn’t go this route so I can’t say it how it could be adapted for Gradle.
A nice benefit of publishing to Maven Central is that javadoc.io will host your docs for free totally automatically. Check it out!
Overall I found the process of painlessly publishing Java open source to Maven Central needlessly confusing, with many more moving parts than I was expecting. The periods of waiting for 3rd parties to approve my project were also a little frustrating, though it fairness the turnaround time was quite impressive given that they were doing the work for free. Hopefully this guide will help make the process a little less frustrating for other Gradle users in the future.