I recently released my first open source library for Java, MDBI. I learnt a lot about the Java open-source ecosystem as part of this process, and this blog summarises that in the hope that it will be useful to others. Specifically, the post will explain how to set up a project using the modern Gradle build system to build code and deploy it to the standard Maven Central repository from the command line really easily.

Getting started

In the Haskell ecosystem, everyone uses Cabal and Hackage, which are developed by the same people and tightly integrated. In contrast, Java’s ecosystem is a bit more fragmented: build systems and package repositiories are managed by different organisations, and you need to do a bit of integration work to join everything up.

In particular, in order to get started we’re going to have to sign up with two different websites: Sonatype OSS and Bintray:

  • No-one can publish directly to Maven Central: instead you need to publish your project to an “approved repository”, from where it will be synced to Central. Sonatype OSS is an approved repository that Sonatype (the company that runs Maven Central) provide free of charge specifically for open-source projects. We will use this to get our artifacts into Central, so go and follow the sign-up instructions now.

    Your application will be manually reviewed by a Sonatype employee and approved within one or two working days. If you want an example of what this process looks like you can take a look at the ticket I raised for my MDBI project.

  • Sonatype OSS is a functional enough way to get your artifacts onto Central, but it has some irritating features. In particular, when you want to make a release you need to first push your artifacts to OSS, and then use an ugly and confusing web interface called Sonatype Nexus to actually “promote” this to Central. I wanted the release to Central to be totally automated, and the easiest way to use that is to have a 3rd party deal with pushing to and then promoting from OSS. For this reason, you should also sign up with Bintray (you can do this with one click if you have a GitHub account).

    Bintray is run by a company called JFrog and basically seems to be a Nexus alternative. JFrog run a Maven repository called JCenter, and it’s easy to publish to that via Bintray. Once it’s on JCenter we’ll be able to push and promote it on Sonatype OSS fully automatically.

We also need to create a Bintray “package” within your Bintray Maven repository. Do this via the Bintray interface — it should be self-explanatory. Use the button on the package page to request it be linked to JCenter (this was approved within a couple of hours for me).

We’ll also need a GPG public/private key pair. Let’s set that up now:

  1. Open up a terminal and run gpg --gen-key. Accept all the defaults about the algorithm to use, and enter a name, email and passphrase of your choosing.

  2. If you run gpg --list-public-keys you should see something like this:

    /Users/mbolingbroke/.gnupg/pubring.gpg
--------------------------------------
pub   2048R/3117F02B 2015-11-18
uid                  Max Bolingbroke 
sub   2048R/15245385 2015-11-18

    Whatever is in place of 3117F02B is the name of your key. I’ll call this $KEYNAME from now on.

  3. Run gpg --keyserver hkp://pool.sks-keyservers.net --send-keys $KEYNAME to publish your key.
  4. Run gpg -a --export-key $KEYNAME and gpg -a --export-secret-key $KEYNAME to get your public and secret keys as ASCII text. Edit your Bintray account and paste these into the “GPG Signing” part of the settings.
  5. Edit your personal Maven repository on Bintray and select the option to “GPG Sign uploaded files automatically”. Don’t use Bintray’s public/private key pair.

Now you have your Bintray and OSS accounts we can move on to setting up Gradle.

Gradle setup

The key problem we’re trying to solve with our Gradle build is producing a set of JARs that meet the Maven Central requirements. What this boils down to is ensuring that we provide:

  • The actual JAR file that people will run.
  • Source JARs containing the code that we built.
  • Javadoc JARs containing compiled the HTML help files.
  • GPG signatures for all of the above. (This is why we created a GPG key above.)
  • A POM file containing project metadata.

To satisfy these requirements we’re going to use gradle-nexus-plugin. The resulting (unsigned, but otherwise Central-compliant) artifacts will then be uploaded to Bintray (and eventually Sonatype OSS + Central) using gradle-bintray-plugin. I also use one more plugin — Palantir’s gradle-gitsemver — to avoid having to update the Gradle file whenever the version number changes. Our Gradle file begins by pulling all those plugins in:

buildscript {
    repositories {
        jcenter()
        maven { url "http://dl.bintray.com/palantir/releases" }
    }
    dependencies {
        classpath 'com.bmuschko:gradle-nexus-plugin:2.3.1'
        classpath 'com.jfrog.bintray.gradle:gradle-bintray-plugin:1.4'
        classpath 'com.palantir:gradle-gitsemver:0.7.0'
    }
}

apply plugin: 'java'
apply plugin: 'com.bmuschko.nexus'
apply plugin: 'com.jfrog.bintray'
apply plugin: 'gitsemver'

Now we have the usual Gradle configuration describing how to build the JAR. Note the use of the semverVersion() function (provided by the gradle-gitsemver plugin) which returns a version number derived from from the most recent Git tag of the form vX.Y.Z. Despite the name of the plugin, there is no requirement to actually adhere to the principles of Semantic Versioning to use it: the only requirements for the version numbers are syntactic.

version semverVersion()
group 'uk.co.omega-prime'

def projectName = 'mdbi'
def projectDescription = 'Max\'s DataBase Interface: a simple but powerful JDBC wrapper inspired by JDBI'

sourceCompatibility = 1.8

jar {
    baseName = projectName
    manifest {
        attributes 'Implementation-Title': projectName,
                   'Implementation-Version': version
    }
}

repositories {
    mavenCentral()
}

dependencies {
    compile group: 'com.google.code.findbugs', name: 'jsr305', version: '3.0.1'
    testCompile group: 'org.xerial', name: 'sqlite-jdbc', version: '3.8.11.2'
    testCompile group: 'junit', name: 'junit', version: '4.12'
}

(Obviously your group, project name, description, dependencies etc will differ from this. Hopefully it’s clear which parts of this example Gradle file you’ll need to change for your project and which you can copy verbatim.)

Now we need to configure gradle-nexus-plugin to generate the POM. Just by the act of including the plugin we have already arranged for the appropriate JARs to be generated, but the plugin can’t figure out the full contents of the POM by itself.

modifyPom {
    project {
        name projectName
        description projectDescription
        url 'http://batterseapower.github.io/mdbi/'

        scm {
            url 'https://github.com/batterseapower/mdbi'
            connection 'scm:https://batterseapower@github.com/batterseapower/mdbi.git'
            developerConnection 'scm:git://github.com/batterseapower/mdbi.git'
        }

        licenses {
            license {
                name 'The Apache Software License, Version 2.0'
                url 'http://www.apache.org/licenses/LICENSE-2.0.txt'
                distribution 'repo'
            }
        }

        developers {
            developer {
                id 'batterseapower'
                name 'Max Bolingbroke'
                email 'batterseapower@hotmail.com'
            }
        }
    }
}

nexus {
    sign = false
}

Note that I’ve explicitly turned off the automatic artifact signing capability of the Nexus plugin. Theoretically we should be able to keep this turned on, and sign everything locally before pushing to Bintray. This would mean that we wouldn’t have to give Bintray our private key. In practice, if you sign things locally Bintray seems to mangle the signature filenames so they become unusable…

Finally, we need to configure the Bintray sync:

if (hasProperty('bintrayUsername') || System.getenv().containsKey('BINTRAY_USER')) {
    // Used by the bintray plugin
    bintray {
        user = System.getenv().getOrDefault('BINTRAY_USER', bintrayUsername)
        key  = System.getenv().getOrDefault('BINTRAY_KEY', bintrayApiKey)
        publish = true

        pkg {
            repo = 'maven'
            name = projectName
            licenses = ['Apache-2.0']
            vcsUrl = 'https://github.com/batterseapower/mdbi.git'

            version {
                name = project.version
                desc = projectDescription
                released = new Date()

                mavenCentralSync {
                    user     = System.getenv().getOrDefault('SONATYPE_USER', nexusUsername)
                    password = System.getenv().getOrDefault('SONATYPE_PASSWORD', nexusPassword)
                }
            }
        }

        configurations = ['archives']
    }
}

We do this conditionally because we still want people to be able to use the Gradle file even if they don’t have your a username and password set up. In order to make these credentials available to the script when run on your machine, you need to create a ~/.gradle/gradle.properties file with contents like this:

# These 3 are optional: they'll be needed if you ever use the nexus plugin with 'sign = true' (the default)
signing.keyId=
signing.password=
signing.secretKeyRingFile=

nexusUsername=
nexusPassword=

bintrayUsername=
bintrayApiKey=

You can see the complete, commented, Gradle file that I’m using in my project on Github.

Your first release

We should now be ready to go (assuming your Sonatype OSS and JCenter setup requests have been approved). Let’s make a release! Go to the terminal and type:

git tag v1.0.0
gradle bintrayUpload

If everything works, you’ll get a BUILD SUCCESSFUL message after a minute or so. Your new version should be visible on the Bintray package page (and JCenter) immediately, and will appear on Maven Central shortly afterwards.

If you want to go the whole hog and have your continuous integration (e.g. the excellent Travis) make these automatic deploys after every passing build, this guide for SBT looks useful. However, I didn’t go this route so I can’t say it how it could be adapted for Gradle.

A nice benefit of publishing to Maven Central is that javadoc.io will host your docs for free totally automatically. Check it out!

Overall I found the process of painlessly publishing Java open source to Maven Central needlessly confusing, with many more moving parts than I was expecting. The periods of waiting for 3rd parties to approve my project were also a little frustrating, though it fairness the turnaround time was quite impressive given that they were doing the work for free. Hopefully this guide will help make the process a little less frustrating for other Gradle users in the future.