Spring Cloud Release Tools


Table of Contents

1. Spring Cloud Release Tools
1.1. What does it do?
1.1.1. Single project
1.1.2. Meta-release
1.2. What should I do first?
1.3. How to run it (interactive mode)
1.4. How to run it (automatic mode)
1.5. How to run meta-release (automatic-mode)
1.6. Project options
1.7. Examples
1.7.1. Keeping configuration in the project
1.7.2. Specifying A Branch
1.7.3. Using Environment Variables
1.8. FAQ
1.8.1. JSchException: Auth fail

1. Spring Cloud Release Tools

Spring Cloud projects reuse the same pattern of building and deploying the applications. That’s why this tool makes it easy to automate the release / dependency update process of our applications.

1.1 What does it do?

1.1.1 Single project

For a single project

  • Clones the Spring Cloud Release project and picks all versions (Boot + Cloud projects)
  • Modifies the project versions with values from a BOM (e.g. for Spring Cloud it’s Spring Cloud Release)

    • throws an exception when we bump versions to release and there’s a SNAPSHOT version referenced in the POM
  • Performs the build and checks if the docs modules have properly created the documentation

    • throws an exception when in the docs module there’s an unresolved tag in any HTML file
  • Commits changed poms (ONLY FOR NON-SNAPSHOT VERSIONS)
  • Creates a tag for the release / milestone (ONLY FOR NON-SNAPSHOT VERSIONS)
  • Runs the deployment of the artifacts
  • Publishes the docs (to spring-cloud-static for non-snapshots, to gh-pages for snapshots)
  • Reverts back to snapshots, bumps the version by a patch (1.0.1.RELEASE1.0.2.BUILD-SNAPSHOT) (ONLY FOR RELEASE VERSIONS)
  • Closes the milestone on Github (e.g. v1.0.1.RELEASE) (ONLY FOR NON-SNAPSHOT VERSIONS)
[Important]Important

Starting with version that does Sagan integration, you MUST pass the OAuth token, otherwise the application will fail to start

After project release

1.1.2 Meta-release

  • Uses the fixed versions to clone and check out each project (e.g. spring-cloud-sleuth: 2.1.0.RELEASE)
  • From the version analyzes the branch and checks it out. E.g.

    • for spring-cloud-release’s `Finchley.RELEASE version will resolve either Finchley branch or will fallback to master if there’s no Finchley branch.
    • for spring-cloud-sleuth’s `2.1.0.RELEASE version will resolve 2.1.x branch
  • Performs the release tasks per each project
  • Performs the post release tasks at the end of the release
  • Will update and run smoke test samples (for Spring Cloud it will be https://github.com/spring-cloud/spring-cloud-core-tests)
  • Will clone provided test samples and will update all versions to the latest ones
  • Will clone the release train wiki and update it with the latest release versions (for Spring Cloud it will be https://github.com/spring-projects/spring-cloud.wiki.git)
[Important]Important

For the meta-releaser to work we assume that the path to the custom configuration file for each project is always config/releaser.yml.

1.2 What should I do first?

Members of the Spring Cloud Team typically use this tool as follows. They first clone the releaser locally and build the jar manually

$ git clone [email protected]:spring-cloud/spring-cloud-release-tools.git
$ cd spring-cloud-release-tools
$ ./mvnw clean install
[Important]Important

You must set the value of the OAuth token. You can do it either via the command line --releaser.git.oauth-token=…​ or put it as an env variable in .bashrc or .zshrc e.g. export RELEASER_GIT_OAUTH_TOKEN=…​

1.3 How to run it (interactive mode)

Go to your project (e.g. Spring Cloud Sleuth)

$ git clone [email protected]:spring-cloud/spring-cloud-sleuth.git
$ cd spring-cloud-sleuth
$ # example of running the releaser agains Dalston.SR1 tag with 1.0.0.BUILD-SNAPSHOT version of the releaser
$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser

The application will start running from your working directory. Running this code follows the convention that you have the OAuth token environment variable set. It also assumes that you might have some custom configuration in config/releaser.yml file. This setting is optional - if you don’t have that file, nothing will happen.

[Tip]Tip

It is important that you clone the repository you are going to release using SSH in order for the releaser to be able to push tags and commit changes automatically.

You will see text similar to this one

=== WHAT DO YOU WANT TO DO? ===

0) Perform a full release of this project without interruptions
1) Perform a full release of this project in a verbose mode (you'll be asked about skipping steps)
2) Update poms with versions from Spring Cloud Release
3) Build the project
4) Commit, tag and push the tag
5) Deploy the artifacts
6) Publish the docs
7) Go back to snapshots and bump originalVersion by patch
8) Push the commits
9) Close the milestone at Github
10) Create email / blog / tweet etc. templates

You can pick a range of options by using the hyphen - e.g. '2-4' will execute jobs [2,3,4]
You can execute all tasks starting from a job by using a hyphen and providing only one number - e.g. '8-' will execute jobs [8,9,10]
You can execute given tasks by providing a comma separated list of tasks - e.g. '3,7,8' will execute jobs [3,7,8]

You can press 'q' to quit

Just pick a number and continue! Pick either a full release or single steps. You can also pick ranges or multiple steps. You can also provide the range only with the starting step - that you will execute all steps starting from the given one.

[Tip]Tip

Read before picking a number cause it might have changed between tool releases ;)

1.4 How to run it (automatic mode)

Go to your project (e.g. Spring Cloud Sleuth) and execute the application with -h or --help flag.

$ git clone [email protected]:spring-cloud/spring-cloud-sleuth.git
$ cd spring-cloud-sleuth
$ # example of running the releaser agains Dalston.SR1 tag with 1.0.0.BUILD-SNAPSHOT version of the releaser
$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser -h

You will see a help screen looking like more or less like this

Here you can find the list of tasks in order

[release,releaseVerbose,metaRelease,postRelease,updatePoms,build,commit,deploy,docs,snapshots,push,closeMilestone,updateSagan,createTemplates,updateGuides,updateDocumentation]


Option                                Description
------                                -----------
-a, --start-from <String>             Starts all release task starting from the
                                        given task. Requires passing the task
                                        name (either one letter or the full
                                        name)
-b, --build [String]                  Build the project
-c, --commit [String]                 Commit, tag and push the tag
-d, --deploy [String]                 Deploy the artifacts
-f, --full-release [Boolean]          Do you want to do the full release of a
                                        single project? (default: false)
-g, --updateSagan [String]            Updating Sagan with release info
-h, --help [String]
-i, --interactive <Boolean>           Do you want to set the properties from
                                        the command line of a single project?
                                        (default: true)
-m, --closeMilestone [String]         Close the milestone at Github
-o, --docs [String]                   Publish the docs
-p, --push [String]                   Push the commits
-r, --range <String>                  Runs release tasks from the given range.
                                        Requires passing the task names with a
                                        hyphen. The first task is inclusive,
                                        the second inclusive. E.g. 's-m' would
                                        mean running 'snapshot', 'push' and
                                        'milestone' tasks
-s, --snapshots [String]              Go back to snapshots and bump
                                        originalVersion by patch
-t, --createTemplates [String]        Create email / blog / tweet etc. templates
--task-names, --tn <String>           Starts all release task for the given
                                        task names
-u, --updatePoms [String]             Update poms with versions from Spring
                                        Cloud Release
--ud, --updateDocumentation [String]  Updating documentation repository
--ug, --updateGuides [String]         Updating Spring Guides
-x, --meta-release <Boolean>          Do you want to do the meta release?
                                        (default: false)

Examples of usage:

Run 'build' & 'commit' & 'deploy'
java -jar jar.jar -b -c -d

Start from 'push'
java -jar releaser.jar -a push

Range 'docs' -> 'push'
java -jar releaser.jar -r o-p

The Releaser can use two sets of options. The configuration options like releaser.pom.branch and the task switches. For the tasks you can use either the full names or short switches. For example providing range of tasks via switches o-p is equivalent to full name docs-push.

A couple of examples:

Doing the full release in interactive mode (asking for skipping steps). 

$ git clone [email protected]:spring-cloud/spring-cloud-sleuth.git
$ cd spring-cloud-sleuth
$ # example of running the releaser agains Dalston.SR1 tag with 1.0.0.BUILD-SNAPSHOT version of the releaser
$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser --full-release

Doing the full release in non interactive mode (automatic release). 

$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser --full-release --interactive=false

Updating pom, closing milestone & createTemplates in interactive mode. 

$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser -u -m -t

Running all tasks starting from 'push' (automatic). 

$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser -a push -i=false

Running tasks from 'docs' (inclusive) to 'push' (inclusive) (automatic). 

$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser -r d-p -i=false

Running single task 'closeMilestone' (automatic). 

$ java -jar ~/repo/spring-cloud-release-tools/spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --releaser.pom.branch=vDalston.SR1 --spring.config.name=releaser --closeMilestone -i=false

1.5 How to run meta-release (automatic-mode)

All you have to do is run the jar with the releaser and pass the -x=true option to turn on meta-release and a list of fixed versions in the `--"releaser.fixed-versions[project-name]=project-version" format

$ java -jar spring-cloud-release-tools-spring/target/spring-cloud-release-tools-spring-1.0.0.BUILD-SNAPSHOT.jar --spring.config.name=releaser -x=true --"releaser.fixed-versions[spring-cloud-sleuth]=2.0.1.BUILD-SNAPSHOT"
[Important]Important

For the meta release the startFrom or taskNames take into consideration the project names, not task names. E.g. you can start from spring-cloud-netflix project, or build only tasks with names spring-cloud-build,spring-cloud-sleuth.

1.6 Project options

  • releaser.fixed-versions - A String to String mapping of manually set versions. E.g. "spring-cloud-cli" → "1.0.0.RELEASE" will set the spring-cloud-cli.version to 1.0.0.RELEASE regardless of what was set in spring-cloud-release project. Example --releaser.fixed-versions[spring-cloud-cli]=1.0.0.RELEASE. Use these properties to provide versions for the meta release.
  • releaser.meta-release.enabled - You have to turn it on to enable a meta release. Defaults to false
  • releaser.meta-release.git-org-url - The URL of the Git organization. We’ll append each project’s name to it. Defaults to https://github.com/spring-cloud
  • releaser.meta-release.projects-to-skip - List of projects that we should not clone and release. Spring Cloud release train depends on projects that got already released. We default this list to [spring-boot, spring-cloud-stream, spring-cloud-task].
  • releaser.git.update-documentation-repo - If true then will update documentation repository with the current URL. Defaults to true.
  • releaser.git.spring-project-url - URL to the documentation Git repository. Defaults to https://github.com/spring-projects/spring-cloud.
  • releaser.git.spring-project-branch - Branch to check out for the documentation project. Defaults to gh-pages.
  • releaser.git.update-spring-project - If true then will update Project Sagan with the current release train values. Defaults to true.
  • releaser.git.test-samples-project-url - URL to the test samples to be checked against the given release train. Defaults to https://github.com/spring-cloud/spring-cloud-core-tests.
  • releaser.git.test-samples-project-branch - Branch to check out for test samples. Defaults to master.
  • releaser.git.release-train-wiki-url - URL to the project’s release train wiki page. Defaults to https://github.com/spring-projects/spring-cloud.wiki.git.
  • releaser.git.update-release-train-wiki - If true then will update the release train wiki page with the current release train values. Defaults to true.
  • releaser.git.run-updated-samples - If true then will update samples and run the the build. Defaults to true.
  • releaser.git.all-test-samples-urls - URLs to the test samples to be cloned and updated with proper snapshot versions. E.g. "--releaser.git.all-test-samples-urls[spring-cloud-sleuth]=https://github.com/spring-cloud-samples/sleuth-issues/,https://github.com/spring-cloud-samples/sleuth-documentation-apps/". Defaults to Sleuth and Contract samples.
  • releaser.git.update-all-test-samples - If true then will update samples with bumped snapshots after release. Defaults to true.
  • releaser.git.release-train-docs-url - URL to the release train documentation. Defaults to https://github.com/spring-cloud-sample/scripts.
  • releaser.git.release-train-docs-branch - Branch to check out for release train documentation. Defaults to master.
  • releaser.git.update-release-train-docs - If true then will update the release train documentation project and run the generation. Defaults to true.
  • releaser.git.update-spring-guides - If true then will update Spring Guides with the current release train. Defaults to true.

The following properties are used for both meta release and a release of an individual module.

  • releaser.post-release-tasks-only - If set to true will run only post release tasks. Defaults to false.
  • releaser.meta-release.release-train-project-name - Name of the project that represents the BOM of the release train. Defaults to spring-cloud-release.
  • releaser.meta-release.release-train-dependency-names - Names of the projects that should be updated with the version of the release train. Defaults to [spring-cloud, spring-cloud-dependencies, spring-cloud-starter].
  • releaser.git.fetch-versions-from-git - If true then should fill the map of versions from Git. If false then picks fixed versions.
  • releaser.git.clone-destination-dir - Where should the Spring Cloud Release repo get cloned to. If null defaults to a temporary directory.
  • releaser.git.release-train-bom-url - URL to a project containing a BOM. Defaults to Spring Cloud Release Git repository: https://github.com/spring-cloud/spring-cloud-release.
  • releaser.git.documentation-url - URL to the documentation Git repository. Defaults to https://github.com/spring-cloud/spring-cloud-static.
  • releaser.git.documentation-branch - Branch to check out for the documentation project. Defaults to gh-pages.
  • releaser.sagan.update-sagan - If true then will update project sagan with information about this project. Defaults to true.
  • releaser.git.oauth-token - GitHub OAuth token to be used to interact with GitHub repo.
  • releaser.git.username - Optional Git username. If not passed keys will be used for authentication.
  • releaser.git.password - Optional Git password. If not passed keys will be used for authentication.
  • releaser.git.number-of-checked-milestones - In order not to iterate endlessly over milestones we introduce a threshold of milestones that we will go through to find the matching milestone. Defaults to 10.
  • releaser.maven.build-command - Command to be executed to build the project. Defaults to ./mvnw clean install -Pdocs.
  • releaser.maven.deploy-command - Command to be executed to deploy a built project". Defaults to ./mvnw deploy -DskipTests -Pfast.
  • releaser.maven.publish-docs-commands - Command to be executed to deploy a built project. If present {{version}} will be replaced by the proper version. Defaults to the standard Spring Cloud wget and execution of ghpages.
  • releaser.maven.system-properties - Additional system properties that should be passed to any commands. If present {{systemProps}} will be replaced by the contents of this property.
  • releaser.maven.wait-time-in-minutes - Max wait time in minutes for the process to finish. Defaults to 20.
  • releaser.gradle.gradle-props-substitution - a map containing a key which is a property key inside gradle.properties and a value of a project name. E.g. in gradle.properties you have foo=1.0.0.BUILD-SNAPSHOT and you would like spring-cloud-contract version to be set there. Just provide a mapping for the gradle-props-substition looking like this foo=spring-cloud-contract and the result (e.g for sc-contract version 2.0.0.RELEASE) will be an updated gradle.properties with entry foo=2.0.0.RELEASE.
  • releaser.pom.branch - Which branch of Spring Cloud Release should be checked out. Defaults to master.
  • releaser.pom.pom-with-boot-starter-parent - What is the location of the pom.xml that contains the spring-boot-starter-parent as its parent pom. Defaults to spring-cloud-starter-parent/pom.xml.
  • releaser.pom.this-train-bom - What is the location of the pom.xml that contains all the versions for the release train. Defaults to spring-cloud-dependencies/pom.xml.
  • releaser.pom.bom-version-pattern - Regular expression that will match the versions of projects in the BOM pom.xml. Defaults to ^(spring-cloud-.*)\.version$.
  • releaser.pom.ignored-pom-regex - List of regular expressions of ignored poms. Defaults to test projects and samples. Example: "--releaser.pom.ignored-pom-regex=".*\\.git/.*$,.{asterisk}spring-cloud-contract-maven-plugin/src/test/projects/.*$,.*spring-cloud-contract-maven-plugin/target/.*$,.*samples/standalone/[a-z]+/.*$".
  • releaser.working-dir - By default Releaser assumes running the program from the current working directory.
  • releaser.template.template-folder - Tells which subfolder with templates to pick for blog, email etc. generation. Defaults to cloud.
[Tip]Tip

You can pass the options either via system properties or via application arguments. Example for system properties: java -Dreleaser.pom.branch=Camden.SR6 -jar target/spring-cloud-release-tools-spring-1.0.0.M1.jar Example for application arguments: java -jar target/spring-cloud-release-tools-spring-1.0.0.M1.jar --releaser.pom.branch=Camden.SR6

[Important]Important

For the GA release to be successful, it’s important that if the build / deploy command run a script (e.g. scripts/foo.sh) then inside foo.sh if you call a Maven build ./mvnw clean install then remember to pass all arguments of the script there too. E.g. ./mvnw clean install ${@}. That’s because the releaser will pass any system properties to the build / deploy command, such as system properties with keys and we need them to be passed inside the command executed by the releaser.

1.7 Examples

1.7.1 Keeping configuration in the project

If your project has some custom configuration (e.g. Spring Cloud Contract needs a script to be executed to build the project and properly merge the docs) then you can put a file named e.g. releaser.yml under config folder and run your application like this:

$ wget https://repo.spring.io/libs-milestone/org/springframework/cloud/internal/spring-cloud-release-tools-spring/1.0.0.M1/spring-cloud-release-tools-spring-1.0.0.M1.jar -O ../spring-cloud-release-tools-spring-1.0.0.M1.jar
$ java -jar target/spring-cloud-release-tools-spring-1.0.0.M1.jar --spring.config.name=releaser
[Tip]Tip

Notice that we’re downloading the jar to a parent folder, not to target. That’s because target get cleaned during the build process

[Important]Important

For the meta-releaser to work we assume that the path to the configuration file is always config/releaser.yml.

1.7.2 Specifying A Branch

By deafult the releaser will default to using the master branch of spring-cloud-release. If you would like to use another branch you can specify it using the releaser.pom.branch property.

$ java -jar spring-cloud-release-tools-spring-1.0.0.M1.jar --releaser.pom.branch=Camden.SR6

1.7.3 Using Environment Variables

In some cases it might be easier to specify environment variables instead of passing parameters to releaser. For example, you might want to use environment variables if you are going to be releasing multiple projects, this keeps you from having to specify the same parameters for each release

$ export RELEASER_POM_BRANCH=Dalston.RELEASE
$ export RELEASER_GIT_OAUTH_TOKEN=...
$ wget https://repo.spring.io/libs-milestone/org/springframework/cloud/internal/spring-cloud-release-tools-spring/1.0.0.M1/spring-cloud-release-tools-spring-1.0.0.M1.jar -O spring-cloud-release-tools-spring-1.0.0.M1.jar
$ java -jar target/spring-cloud-release-tools-spring-1.0.0.M1.jar --releaser.working-dir=/path/to/project/root

1.8 FAQ

1.8.1 JSchException: Auth fail

I got such an exception

Caused by: org.eclipse.jgit.errors.TransportException: [email protected]:spring-cloud/spring-cloud-sleuth.git: Auth fail
	at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:160) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:137) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	at org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:322) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	at org.eclipse.jgit.transport.TransportGitSsh.openPush(TransportGitSsh.java:167) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:160) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	at org.eclipse.jgit.transport.Transport.push(Transport.java:1275) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:161) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	... 25 common frames omitted
Caused by: com.jcraft.jsch.JSchException: Auth fail
	at com.jcraft.jsch.Session.connect(Session.java:512) ~[jsch-0.1.53.jar!/:na]
	at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:117) ~[org.eclipse.jgit-4.6.0.201612231935-r.jar!/:4.6.0.201612231935-r]
	... 31 common frames omitted

To fix that just call

# to run the agent
$ eval `ssh-agent`
# to store the pass in the agent
$ ssh-add ~/.ssh/id_rsa

before running the app