6. Step-by-step Cloud Foundry Migration

This section details how to migrate applications such that they become compatible with Spring Cloud Pipelines.

6.1 Preview

Click here to check out the slides by Cora Iberkleid where she migrates a set of applications to be compliant with Spring Cloud Pipelines.

6.2 Introduction

This tutorial covers refactoring applications to be compatible with, and take advantage of, Spring Cloud Pipelines.

As an example, we use a simple three-tier application, shown in the following image:

Figure 6.1. Use Case - Logical View

use case logical

At the end of this tutorial, you will be able to quickly create a Concourse pipeline for each application and run successfully through a full lifecycle, from source code commit to production deployment, following the lifecycle stages for testing and deployment recommended by Spring Cloud Pipelines. You will be able to improve application code bases with organized test coverage, a contract-based API, and a versioned database schema, letting Spring Cloud Pipelines carry out stubbed testing and ensure backward compatibility for API and database schema changes.

6.3 Sample Application — Initial State

The sample application is implemented by using Spring Boot applications for the UI and service tiers and MySQL for the database.

The apps are built with Maven and manually pushed to Cloud Foundry. They leverage the three Pivotal Spring Cloud Services: Config Server, Service Discovery, and Circuit Breaker Dashboard. We use Rabbit to propagate Config Server refresh triggers.

The source code for the two Spring Boot applications is stored on GitHub, as is the backing repo for Config Server.

The following image shows an implementation view of the applications and their ancillary services:

Figure 6.2. Use Case - Implementation

use case implementation

6.4 Sample Application — End State

Throughout this tutorial, we add Concourse and JFrog Bintray to manage the application lifecycle.

We also refactor the applications so that they become compatible with Spring Cloud Pipelines requirements and recommendations, including adding and organizing tests and introducing database versioning by using Flyway and introducing API contracts by using Spring Cloud Contract.

6.5 Tutorial — Toolset

Throughout this tutorial, we use the following tools:

  • GitHub: Sample application source code and configuration repositories, a sample stubrunner application repository, and the Spring Cloud Pipelines code base, including the following:

  • Pivotal Web Services: Publicly hosted Cloud Foundry offering free trial accounts and including MySQL, Rabbit, and Pivotal Spring Cloud Services in the Marketplace
  • Concourse
  • JFrog Bintray: Publicly hosted Maven repository offering free OSS accounts
  • Client Tools: On your local machine, you need an IDE as well as the mvn, git, cf, and fly (Concourse) CLIs

6.6 Tutorial — Overview

We separate the migration steps into three stages:

  1. Scaffolding

    • Minimal refactoring to be compatible with basic Spring Cloud Pipelines requirements.
    • At the end of this stage, each application has a corresponding pipeline on Concourse. The pipelines successfully build the applications, store the artifacts in Bintray, tag the GitHub repositories, and deploy the applications to the Test, Stage, and Prod spaces in Cloud Foundry.
  2. Tests

    • Add and organize tests to be compatible with Spring Cloud Pipelines recommendations.
    • Incorporate flyway for database schema versioning and initial data loading.
    • At the end of this stage, the pipelines trigger unit and integration tests during the Build stage, smoke tests in the Test environment, and end-to-end tests in the Stage environment. The pipelines also ensure backward compatibility for the database, such that you can safely roll back the backend service application, even after the database schema has been updated.
  3. Contracts

    • Incorporate Spring Cloud Contract to define the API between the UI and service apps and auto-generate tests and stubs.
    • At the end of this stage, the pipelines catch breaking API changes during the Build stage and ensure backward compatibility for the API, such that you can safely roll back the backend service (producer) app, even after an API change.

6.7 Tutorial — Step-by-step

The remainder of this chapter is the actual tutorial, which consists of a preparation stage and three main stages:

6.7.1 Prep: Before you begin

If you want to simply review the migration steps explained below, you can look at the various branches in the greeting-ui and fortune-service repositories. A branch represents the end-state of each stage, as the following image shows:

Figure 6.3. GitHub Branches

github branches

If you want to use this tutorial as a hands-on lab, fork each of the following repositories:

Then create a new directory on your local machine. You may name it anything you like. We refer to it as $SCP_HOME throughout this tutorial.

In $SCP_HOME, clone your forks of greeting-ui and fortune-service, as well as the following two repositories:

Finally, create a directory called $SCP_HOME/credentials. Leave it empty for now.

6.7.2 Stage One: Scaffolding

In this stage, we make minimal changes to satisfy basic Spring Cloud Pipelines requirements so that the applications can run through the entire pipeline without error. We make “scaffolding” changes only — no code changes.

[Important]Important

You must complete the steps in this stage for both greeting-ui and fortune-service.

1.1 Create GitHub Branches

Create branches in GitHub by using the following git commands:

git branch version
git checkout -b sc-pipelines

The version branch is required to exist, though it can be created as an empty branch. It is used by Spring Coud Pipelines to generate a version number for each new pipeline execution.

The sc-pipelines branch is optional and can be named anything you wish. The intention is for you to use it as a working branch for the changes suggested in this tutorial (hence, you should both create it and check it out).

1.2 Add Maven Wrapper

This step covers how to add the Maven wrapper (which lets your users build without having Maven on the path). To add the Maven wrapper, run the following command:

mvn -N io.takari:maven:wrapper

This commands adds four files to a project:

.
├── mvnw
├── mvnw.cmd
└── .mvn
    └── wrapper
        ├── maven-wrapper.jar
        └── maven-wrapper.properties

Make sure all four files are tracked by Git. One way to do so is to add the following lines to the .gitignore file:

#Exceptions
!/mvnw
!/mvnw.cmd
!/.mvn/wrapper/maven-wrapper.jar
!/.mvn/wrapper/maven-wrapper.properties

1.3 Create the Bintray Maven Repository Package

We use Bintray as the Maven repository. Bintray requires that a package exist before any application artifacts can be uploaded.

Log into the Bintray UI and create the packages as follows (you can use the “Import from GitHub” option to create these):

Figure 6.4. Bintray Packages

bintray packages

1.4 Configure Distribution Management by Using the Bintray Maven Repository

[Important]Important

You must do this step for both application repositories.

Edit the application pom.xml files. Make sure that the Bintray URLs match the URLs of the corresponding packages created in the previous step. The values you use should differ from the following example in that they should point to your repository:

<properties>
...
<distribution.management.release.id>bintray</distribution.management.release.id>
<distribution.management.release.url>https://api.bintray.com/maven/ciberkleid/maven-repo/fortune-service</distribution.management.release.url>
</properties>

...

<distributionManagement>
<repository>
<id>${distribution.management.release.id}</id>
<url>${distribution.management.release.url}</url>
</repository>
</distributionManagement>

Though not required by Spring Cloud Pipelines, it makes sense to also configure your local maven settings with the credentials to your Bintray maven repository. To do so, edit your maven settings file (usually ~/.m2/settings.xml). If the file does not exist, create it.

Note that the id must match the id specified in the previous step. Also, make sure to use your username and API token (not your account password) instead of the sample values shown in the following example:

<?xml version="1.0" encoding="UTF-8"?>
<settings>
  <servers>
    <server>
      <id>bintray</id>
      <username>ciberkleid</username>
      <password>my-super-secret-api-token</password>
   </server>
 </servers>
</settings>

1.5 Push Changes to GitHub

Push the changes you made in the preceding step to GitHub. You should be pushing the following to each of the two application repositories:

  • Four new Maven wrapper files
  • A modified .gitignore file
  • A modified pom.xml file

1.6 Add a Spring Cloud Pipelines Credentials File

In $SCP_HOME/credentials, make two copies of the $SCP_HOME/spring-cloud-pipelines/concourse/credentials-sample-cf.yml file. Rename them as credentials-fortune-service.yml and credentials-greeting-ui.yml.

[Caution]Caution

These files will contain credentials to your GitHub repository, your Bintray repository, and your Cloud Foundry foundation. Hence, we opt to put them in a separate directory. You may choose to store these files in a private Git repository, but do not push them to a public repository.

Edit the Git properties of each credentials file. Make sure to replace the sample values shown in our example. For tools-branch, you can use a fixed release (use v1.0.0.M8 or later for Cloud Foundry). Leave the other values as they are. We update those in later steps. The following listing shows a credentials file:

app-url: [email protected]:ciberkleid/fortune-service.git
app-branch: sc-pipelines
tools-scripts-url: https://github.com/spring-cloud/spring-cloud-pipelines.git
tools-branch: master
build-options: ""

github-private-key: |
  -----BEGIN RSA PRIVATE KEY-----
  MIIJKQIBAAKCAgEAvwkL97vBllOSE39Wa5ppczT1cr5Blmkhadfoa1Va2/IBVyvk
  NJ9PqoTI+BahF2EgzweyiDSvKsstlTsG7QgiM9So8Voi2PlDOrXL6uOfCuAS/G8X
  ...
  -----END RSA PRIVATE KEY-----
git-email: [email protected]
git-name: Cora Iberkleid

Edit the Maven repository properties of each credentials file. Make sure to replace the sample values shown in our example. Bintray requires separate URLs for uploads and downloads. If you use a different artifact repository, such as Artifactory or Nexus, and the repository URL is the same for uploads and downloads, you do not need to set repo-with-binaries-for-upload. The following listing shows the values to add or edit in your credentials file:

m2-settings-repo-id: bintray
m2-settings-repo-username: ciberkleid
m2-settings-repo-password: my-super-secret-api-token

repo-with-binaries: https://ciberkleid:[email protected]/ciberkleid/maven-repo

repo-with-binaries-for-upload: https://api.bintray.com/maven/ciberkleid/maven-repo/fortune-service

1.7 Set the Concourse Pipeline

At this point, all of the build jobs, which run on Concourse workers, should succeed.

To verify this, log in to your Concourse target and set the Concourse pipelines. Update the target name in the following example:

# Set greeting-ui pipeline
fly -t myTarget set-pipeline -p greeting-ui -c "${SCP_HOME}/spring-cloud-pipelines/concourse/pipeline.yml" -l "${SCP_HOME}/credentials/credentials-greeting-ui.yml" -n

# Set fortune-service pipeline
fly -t myTarget set-pipeline -p fortune-service -c "${SCP_HOME}/spring-cloud-pipelines/concourse/pipeline.yml" -l "${SCP_HOME}/credentials/credentials-fortune-service.yml" -n

Log into the Concourse UI and un-pause the pipelines. Start each pipeline. You should see that the build jobs all succeed, similar to the following image:

Figure 6.5. Build Success

concourse build success

In addition, you should see a new dev/<version_number> tag in each GitHub repository and see the app jars uploaded into Bintray.

The test, stage, and prod jobs fail, because we have not yet added scaffolding for deployment to Cloud Foundry. We do that next.

1.8 Add Cloud Foundry manifest

If you are deploying to Cloud Foundry, you may already be routinely including manifest files with your applications. Our sample applications did not have manifest files, so we add them now.

In the greeting-ui repository, create a manifest.yml file as follows:

---
applications:
- name: greeting-ui
  timeout: 120
  services:
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io

In the fortune-service repository, create a manifest.yml file as follows:

---
applications:
- name: fortune-service
  timeout: 120
  services:
  - fortune-db
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io

The TRUST_CERTS variable is used by the Pivotal Spring Cloud Services (Config Server, Service Registry, and Circuit Breaker Dashboard), which we use in this example. The value specified in the preceding example assumes deployment to Pivotal Web Services. Update it accordingly if you are deploying to a different Cloud Foundry foundation, or you can leave it out altogether if you are replacing the Pivotal Spring Cloud Services with alternative implementations (for example, deploying the services as applications and exposing them as user-provided services).

If you wishi, you can add additional values to the manifest files — for example, if additional values are useful for any manual deployment you may still want to do or if you need additional values in your Spring Cloud Pipelines deployment. For example, the following file could be an alternative manifest.yml for fortune-service:

---
applications:
- name: fortune-service
  timeout: 120
  instances: 3
  memory: 1024M
  buildpack: https://github.com/cloudfoundry/java-buildpack.git
  random-route: true
  path: ./target/fortune-service-0.0.1-SNAPSHOT.jar
  services:
  - fortune-db
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    SPRING_PROFILES_ACTIVE: someProfile
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io

Note that Spring Cloud Pipelines ignores random-route and path. instances is honored in stage and prod but is overridden with a value of 1 for test.

1.9 Add the Spring Cloud Pipelines Manifest

The Cloud Foundry manifest created in the previous step includes the logical names of the services to which the applications should be bound, but it does not describe how the services can be provisioned. Hence, we add a second manifest file so that Spring Cloud Pipelines can provision the services.

Add a file called sc-pipelines.yml to each application and include the same list of services as in the corresponding manifest.yml. Add the necessary details such that Spring Cloud Pipelines can construct a cf create-service command.

[Note]Note

The `type: broker' parameter in the next example instructs Spring Cloud Pipelines to provision a service by using `cf create-service'. Other service types are also supported: cups, syslog, route, app, and stubrunner.

More specifically, for greeting-ui, create an sc-pipelines.yml file with the following content:

test:
  services:
    - name: config-server
      type: broker
      broker: p-config-server
      plan: standard
      params:
        git:
          uri: https://github.com/ciberkleid/app-config
      useExisting: true
    - name: cloud-bus
      type: broker
      broker: cloudamqp
      plan: lemur
      useExisting: true
    - name: service-registry
      type: broker
      broker: p-service-registry
      plan: standard
      useExisting: true
    - name: circuit-breaker-dashboard
      type: broker
      broker: p-circuit-breaker-dashboard
      plan: standard
      useExisting: true

The sc-pipelines.yml file for fortune-service is similar, with the addition of the fortune-db service, as follows:

test:
  # list of required services
  services:
    - name: fortune-db
      type: broker
      broker: cleardb
      plan: spark
      useExisting: true
    - name: config-server
      type: broker
      broker: p-config-server
      plan: standard
      params:
        git:
          uri: https://github.com/ciberkleid/app-config
      useExisting: true
    - name: cloud-bus
      type: broker
      broker: cloudamqp
      plan: lemur
      useExisting: true
    - name: service-registry
      type: broker
      broker: p-service-registry
      plan: standard
      useExisting: true
    - name: circuit-breaker-dashboard
      type: broker
      broker: p-circuit-breaker-dashboard
      plan: standard
      useExisting: true

The values in the preceding two examples assume deployment to Pivotal Web Services. If you are deploying to a different Cloud Foundry foundation, update the values accordingly. Also, make sure to replace the config-server URI with the address of your fork of the app-config repository.

[Tip]Tip

Notice the useExisting: true parameter in the preceding example. By default, Spring Cloud Pipelines deletes and re-creates services in the test space. To override this behavior and re-use existing services, we set useExisting: true. This is helpful in cases where services may take time to provision and initialize, where there is no risk in re-using them between pipeline runs, or where it is desirable to retain the service instance from the last pipeline run (for example, a database migration).

1.10 Push changes to GitHub

Push the preceding changes to GitHub. You should be pushing the following to each of the two application repositories:

  • A new app manifest file
  • A new sc-pipelines manifest file

1.11 Create Cloud Foundry Orgs and Spaces

Spring Cloud Pipelines requires that the Cloud Foundry test, stage, and prod spaces exist before a pipeline is run. If you wish, you can use different foundations, orgs, and users for each. For simplicity, in this example, we use a single foundation (PWS), a single org, and a single user.

You can name the orgs and spaces anything you like. Each app requires its own test space. The stage and prod spaces are shared.

For this example, use the following commands to create spaces:

cf create-space scp-test-greeting-ui
cf create-space scp-test-fortune-service
cf create-space scp-stage
cf create-space scp-prod

1.12 Create Cloud Foundry Stage and Prod Service Instances

Spring Cloud Pipelines dynamically creates the services in the test spaces, as defined by the sc-pipelines.yml file we created previously. Optionally, you can add a second section to the sc-pipelines.yml file for the stage environment, and these are created dynamically as well. However, you must always crate prod manually.

For this example, we create both the stage and prod services manually.

Create the services listed in the application manifest files in both scp-stage and scp-prod.

1.13 Update the Spring Cloud Pipelines Credentials File

Update the greeting-ui and fortune-service credentials files with Cloud Foundry information. Replace values in the next example as appropriate for your Cloud Foundry environment.

Notice that the test space name specified is a prefix, unlike the stage and prod space names, which are literals. Spring Cloud Pipelines append the application name to the test space name, thereby matching the test space names we created manually. The stage and prod space names are not prefixes and are not altered by Spring Cloud Pipelines.

Note also the paas-hostname-uuid. The value is included in each created route. This value is optional, but it is useful in shared or multi-tenant environments (such as PWS), as it helps to ensure routes are unique. Change it to a unique uuid.

The following example shows an updated credentials file:

pipeline-descriptor: sc-pipelines.yml

paas-type: cf

paas-hostname-uuid: cyi

# test values
paas-test-api-url: https://api.run.pivotal.io
paas-test-username: [email protected]
paas-test-password: secret
paas-test-org: S1Pdemo12
paas-test-space-prefix: scp-test

# stage values
paas-stage-api-url: https://api.run.pivotal.io
paas-stage-username: [email protected]
paas-stage-password: my-super-secret-password
paas-stage-org: S1Pdemo12
paas-stage-space: scp-stage

# prod values
paas-prod-api-url: https://api.run.pivotal.io
paas-prod-username: [email protected]
paas-prod-password: my-super-secret-password
paas-prod-org: S1Pdemo12
paas-prod-space: scp-prod

1.14 Update the Concourse Pipeline with Updated Credentials Files

Set the Concourse pipelines again, as we did previously, to update them with the values added to the credentials files. The test, stage, and prod jobs should all now succeed and result in output similar to the following image:

Figure 6.6. Test, Stage, & Prod Success

concourse test stage prod success

On Cloud Foundry, you should now see the apps deployed in the test, stage, and prod spaces. The following image shows the deployment of fortune-service to its dedicated test space:

Figure 6.7. Cloud Foundry Test and Prod Deployment

cf test and prod deployed

Notice that the five services declared in its manifest files (sc-pipelines.yml for provisioning and manifest.yml for binding) have also been automatically provisioned. The image also shows the deployment of the same app to the shared prod space. Notice that the instance of the previous version has been renamed as venerable and stopped. If a rollback were deemed necessary, the prod-rollback job in the pipeline could be triggered to remove the currently running version, remove the prod/<version_number> tag from GitHub, and re-start the former (venerable) version.

Stage One Recap and Next Steps

What have we accomplished?

  • By adding the basic scaffolding needed to enable Spring Cloud Pipelines to manage the lifecycle of greeting-ui and fortune-service from source code commit to production deploy, we have made it possible for the application development teams to instantly and easily create pipelines for each application by using a common, standardized template.
  • We can count on the pipelines to:

    • Automatically provision services in test spaces and, optionally, in stage spaces as well.
    • Dynamically clean up the test spaces between pipeline executions.
    • Upload the app artifacts to the maven repo (for example, Bintray).
    • Tag the git repositories with dev/<version_number> and prod/<version_number>.
  • After each successful pipeline run, we can, if necessary, to roll back to the last deployed version byusing the prod-rollback job.

These accomplishments are extremely valuable. However, to derive confidence and reliability from the pipelines, we need to incorporate testing. We do this in Stage Two of the application migration.

6.7.3 Stage Two: Tests

In this stage, we enable Spring Cloud Pipelines to execute tests so that we can increase confidence in the code being deployed. We do so by adding test profiles to the pom.xml files and then organizing or adding tests in a way that corresponds to the profiles. By doing so, we establish standards around testing across development teams in the enterprise.

We will also enable database schema versioning in this stage, thereby providing the foundation for rollback testing during schema changes.

2.1 Add Maven Profiles

For both greeting-ui and fortune-service, add a profiles section to the pom.xml file, as shown in the next listing. Note that we are adding four profiles:

  • default

    • For unit and integration tests. Note that this profile includes all tests except those that are explicitly called by the smoke and e2e profiles.
    • Tests matching this profile run during the build-and-upload job.
  • apicompatibility

    • For ensuring backward compatibility in case of API changes. Note that this is not effective until Stage Three, when we will add contracts. However, we add this profile now to ensure the api compatibility check during the build-and-upload job does not run other tests.
  • smoke

    • For tests to be run against the application deployed in the test space.
  • e2e

    • For tests to be run against the application deployed in the stage space.

The following listing shows the necessary profiles element of a pom.xml file:

  <profiles>
    <profile>
      <id>default</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>**/*Tests.java</include>
                <include>**/*Test.java</include>
              </includes>
              <excludes>
                <exclude>**/smoke/**</exclude>
                <exclude>**/e2e/**</exclude>
              </excludes>
            </configuration>
          </plugin>
          <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
          </plugin>
        </plugins>
      </build>
    </profile>
    <profile>
      <id>apicompatibility</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>**/contracttests/**/*Tests.java</include>
                <include>**/contracttests/**/*Test.java</include>
              </includes>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
    <profile>
      <id>smoke</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>smoke/**/*Tests.java</include>
                <include>smoke/**/*Test.java</include>
              </includes>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
    <profile>
      <id>e2e</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>e2e/**/*Tests.java</include>
                <include>e2e/**/*Test.java</include>
              </includes>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>

2.2 Add and Organize Tests

Next, we ensure that we have a matching test package structure in our apps, as the following image shows:

Figure 6.8. Test Package Structure

test package structure

Note that we are creating matching packages only for the default, smoke, and e2e profiles. We will address the package for the apicompatibility profile in Stage Three.

When working with your own applications, if you have existing tests, you would move the files into one of these packages now and rename them so that they are included by the filters declared in the profiles (that is, the file names end in Test.java or Tests.java)

In the case of our sample apps, we have no tests, so we add some now.

fortune-service Default Tests

Add your unit and integration tests so that they match the default profile, as defined in the fortune-service pom.xml file. These are run on Concourse against the fortune-service application that runs on the Concourse worker in the build-and-upload job.

As an example, we add two tests, one that loads the context and another that verifies the number of rows expected in the database. The following example defines these tests:

package io.pivotal;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import org.springframework.jdbc.core.JdbcTemplate;
import static org.assertj.core.api.Assertions.assertThat;

import static org.junit.Assert.*;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = FortuneServiceApplication.class)
public class FortuneServiceApplicationTests {

    @Test
    public void contextLoads() throws Exception {

    }

    @Autowired
    private JdbcTemplate template;

    @Test
    public void testDefaultSettings() throws Exception {
        assertThat(this.template.queryForObject("SELECT COUNT(*) from FORTUNE",
                Integer.class)).isEqualTo(7);
    }

}
fortune-service Smoke Tests

Add your smoke tests so that they match the smoke profile, as defined in the fortune-service pom.xml file. These run on Concourse against the fortune-service application deployed in the Cloud Foundry scp-test-fortune-service space. Two versions of these tests are executed against the application:

  • the current version, in the test-smoke job.
  • the latest prod version, in the test-rollback-smoke job.

The following image shows the tests in Concourse:

Figure 6.9. fortune-service Smoke Tests

fortune service smoke tests

In the test environment, we choose to verify that fortune-service is retrieving a fortune from fortune-db, and not returning its Hystrix fallback response. The following example defines this test:

package smoke;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SmokeTests.class,
        webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class SmokeTests {

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity<String> response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback response
		BDDAssertions.then(response.getBody()).doesNotContain("The fortuneteller will be back soon.");
	}

}
fortune-service End-to-end (e2e) Tests

Add your end-to-end tests so that they match the e2e profile, as defined in the fortune-service pom.xml file. These tests run on Concourse against the fortune-service application deployed in the Cloud Foundry scp-stage space. This space is shared, so we assume greeting-ui is also present.

The following image shows the tests in Concourse:

Figure 6.10. fortune-service E2E Tests

fortune service e2e tests

In the e2e environment, we choose to use a string replacement to obtain the URL for greeting-ui. We also choose to verify that we hit fortune-db and do not receive Hystrix fallback responses from either application. The following example shows this test:

package e2e;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = E2eTests.class,
		webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class E2eTests {

	// The app is running in CF but the tests are executed from Concourse worker,
	// so the test will deduce the url to greeting-ui: it will assume the same host
	// as fortune-service, and simply replace "fortune-service" with "greeting-ui" in the url

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity<String> response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl.replace("fortune-service", "greeting-ui") + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback responses from both fortune and greeting
		BDDAssertions.then(response.getBody()).doesNotContain("This fortune is no good. Try another.").doesNotContain("The fortuneteller will be back soon.");
	}

}
greeting-ui Default Tests

Add your unit and integration tests so that they match the default profile, as defined in the greeting-ui pom.xml file. These run on Concourse against the greeting-ui application that runs on the Concourse worker in the build-and-upload job.

As an example, we add one test that loads the context:

package io.pivotal;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = GreetingUIApplication.class)
public class GreetingUIApplicationTests {

    @Test
    public void contextLoads() throws Exception {

    }

}
greeting-ui Smoke Tests

Add your smoke tests so that they match the smoke profile, as defined in the greeting-ui pom.xml file. These run on Concourse against the greeting-ui application deployed in the Cloud Foundry scp-test-greeting-ui space. Two versions of these tests run against the app:

  • The current version, in the test-smoke job.
  • The latest prod version, in the test-rollback-smoke job.

The following image shows the tests in Concourse:

Figure 6.11. greeting-ui Smoke Tests

greeting ui smoke tests

Since fortune-service is not deployed to the scp-test-greeting-ui space, we expect to receive the Hystrix fallback response defined in greeting-ui. Hence, our smoke test validates that condition:

package smoke;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SmokeTests.class,
        webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class SmokeTests {

    @Value("${application.url}") String applicationUrl;

    RestTemplate restTemplate = new RestTemplate();

    @Test
    public void should_return_a_fallback_fortune() {
        ResponseEntity<String> response = this.restTemplate
                .getForEntity("http://" + this.applicationUrl + "/", String.class);

        BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

        // Expect the hystrix fallback response
        BDDAssertions.then(response.getBody()).contains("This fortune is no good. Try another.");
    }

}
greeting-ui End-to-end (e2e) Tests

Add your end-to-end tests so that they match the e2e profile, as defined in the greeting-ui pom.xml file. These run on Concourse against the greeting-ui application deployed in the Cloud Foundry scp-stage space. This space is shared, so we assume fortune-service is also present.

The following image shows the tests in Concourse:

Figure 6.12. greeting-ui E2E Tests

greeting ui e2e tests

In the e2e environment, we choose to verify that we hit fortune-service and do not receive the Hystrix fallback response from greeting-ui. The following example shows the test:

package e2e;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = E2eTests.class,
		webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class E2eTests {

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity<String> response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback response
		BDDAssertions.then(response.getBody()).doesNotContain("This fortune is no good. Try another.");
	}

}

2.3 Enable Database Versioning

At this point, we also incorporate Flyway, an OSS database migration tool, to track database schema versions and handle schema changes and data loading.

This change needs to be made only to fortune-service, since fortune-service owns the interaction with fortune-db.

Add Flyway Dependency

We first add the Flyway dependency to the fortune-service pom.xml. We need not add a version as Spring Boot takes care of that for us. The following listing shows the Flyway dependency:

    <dependency>
      <groupId>org.flywaydb</groupId>
      <artifactId>flyway-core</artifactId>
    </dependency>
    <dependency>
Create Flyway Migration

Next, we create a migration directory and our initial migration file, following Flyway’s file naming convention. The following image shows the name of the file in context:

Figure 6.13. fortune-service Flyway File Name

fortune service flyway file name

Note that the filename specifies the version (V1), followed by two underscore characters.

We place our CREATE TABLE and INSERT statements in our src/main/resources/db/migration/V1__init.sql file, as the following listing shows:

CREATE TABLE fortune (
  id BIGINT PRIMARY KEY AUTO_INCREMENT,
  text varchar(255) not null
);

INSERT INTO fortune (text) VALUES ('Do what works.');

INSERT INTO fortune (text) VALUES ('Do the right thing.');

INSERT INTO fortune (text) VALUES ('Always be kind.');

INSERT INTO fortune (text) VALUES ('You learn from your mistakes... You will learn a lot today.');

INSERT INTO fortune (text) VALUES ('You can always find happiness at work on Friday.');

INSERT INTO fortune (text) VALUES ('You will be hungry again in one hour.');

INSERT INTO fortune (text) VALUES ('Today will be an awesome day!');
Disable JPA DDL Initialization

Because we rely on Flyway to create and populate the schema, we need to disable JPA-based database initialization. We can set ddl-auto to validate, which validates the schema against the application entities and throws an error in case of a mismatch but does not actually generate the schema. The following snippet shows how to do so:

spring:
  jpa:
    hibernate:
      ddl-auto: validate

There are a few options for where to store the ddl-auto configuration, both in terms of location (in the fortune-service app or on the app-config GitHub repo) and in terms of file name. For this example, update the application.yml in the fortune-service app for local testing. Additionally, save these values in a new file called application-flyway.yml on your fork of app-config.

By convention, fortune-service picks up the configurations in application-flyway.yml if the string flyway is in the list of active Spring profiles. Consequently, we can add flyway to the SPRING_PROFILES_ACTIVE environment variable in the fortune-service manifest.yml, as the following listing shows:

---
applications:
- name: fortune-service
  timeout: 120
  services:
  - fortune-db
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    SPRING_PROFILES_ACTIVE: flyway
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io
Remove Non-Flyway Data Loading

We can now remove the old code that populated the database. In our sample app, this was found in the io.pivotal.FortuneServiceApplication class. The following listing shows the code we now remove:

@Bean
    CommandLineRunner loadDatabase(FortuneRepository fortuneRepo) {
        return args -> {
//            logger.debug("loading database..");
//            fortuneRepo.save(new Fortune(1L, "Do what works."));
//            fortuneRepo.save(new Fortune(2L, "Do the right thing."));
//            fortuneRepo.save(new Fortune(3L, "Always be kind."));
//            fortuneRepo.save(new Fortune(4L, "You learn from your mistakes... You will learn a lot today."));
//            fortuneRepo.save(new Fortune(5L, "You can always find happiness at work on Friday."));
//            fortuneRepo.save(new Fortune(6L, "You will be hungry again in one hour."));
//            fortuneRepo.save(new Fortune(7L, "Today will be an awesome day!"));
            logger.debug("record count: {}", fortuneRepo.count());
            fortuneRepo.findAll().forEach(x -> logger.debug(x.toString()));
        };

    }

We also no longer need the Fortune entity constructors, so we can comment these out in the io.pivotal.fortune.Fortune class as follows:

//    public Fortune() {
//    }
//
//    public Fortune(Long id, String text) {
//        super();
//        this.id = id;
//        this.text = text;
//    }
Flyway Integration Summary

With that, we have completed the setup for Flyway, and our database schema is now versioned. From this point onward, Spring Boot calls Flyway.migrate() to perform the database migration. As long as we follow Flyway conventions for future schema changes, Flyway takes care of tracking the schema version and migrating the database for us.

From a rollback perspective, Spring Cloud Pipelines includes two jobs in the test phase (test-rollback-deploy and test-rollback-smoke), wherein it validates that the latest prod jar works against the newly updated database. The purpose is to ensure that we can roll back the application in prod if a problem is discovered after the prod database schema has been updated and avoid the burden of rolling back the database.

See Spring Boot database initialization with Flyway for further information, including Flyway configuration options.

2.4 Push Changes to GitHub

For greeting-ui, you should push the following new or modified files:

  • pom.xml
  • src/test/java/e2e/E2eTests.java
  • src/test/java/io/pivotal/GreetingUIApplicationTests.java
  • src/test/java/smoke/SmokeTests.java

For fortune-service, you should be pushing the following new or modified files:

  • pom.xml
  • src/test/java/e2e/E2eTests.java
  • src/test/java/io/pivotal/FortuneServiceApplicationTests.java
  • src/test/java/smoke/SmokeTests.java
  • src/main/resources/db/migration/V1__init.sql
  • src/main/resources/application.yml
  • manifest.yml
  • src/main/java/io/pivotal/FortuneServiceApplication.java
  • src/main/java/io/pivotal/fortune/Fortune.java

For app-config, you should be pushing the following new:

  • application-flyway.yml

2.5 Re-run the Pipelines

Run through the pipelines again and view the output for the jobs that run the default, smoke, and end-to-end (e2e) tests. You should see that the tests we added in this stage were run.

As you run through the pipelines a second time, you should see the smoke tests from the latest prod version run against the database in the test-rollback-smoke job. In this case, there is no schema upgrade. Nonetheless, the tests confirm that the latest prod version of the app can be used with the current database schema.

You can see the database version information stored in the database by Flyway either by querying the database itself or by hitting the flyway endpoint on the fortune-service URL. The following image shows an example from the scp-stage environment:

Figure 6.14. fortune-service Flyway Schema Info

fortune service flyway schema info

Stage Two Recap and Next Steps

What have we accomplished?

  • Increased the effectiveness of the pipelines and our confidence in them by integrating our applications with the testing strategy built into Spring Cloud Pipelines.
  • Established a standard approach to organizing tests, which brings consistency within and across development teams.
  • Enabled auto-managed database versioning and backward compatibility testing that alleviates database schema management throughout the release management lifecycle.

We can now add any unit, integration, smoke, and end-to-end tests to our code base and have a high level of reliability and confidence in our pipelines. We are also better positioned to ensure that our development teams conform to these practices, given the structure established by Spring Cloud Pipelines and the fast feedback and visibility we gain from the pipelines as they execute the tests.

However, we could benefit further by incorporating contracts to define and test the API integration points between applications. We do this in Stage Three of the application migration.

6.7.4 Stage Three: Contracts

In this stage, we introduce contract-based programming practices into our sample application. Doing so improves API management capabilities, including defining, communicating, and testing API semantics. It also lets us catch breaking API changes (that is, we can validate API backward compatibility) in the build phase. This extends the effectiveness of the pipelines, encourages better communication and programming practices across development teams, and provides faster feedback to developers.

We will integrate Spring Cloud Contract and add contracts, stubs, and a stub runner. We will also now complete and make use of the apicompatibility profile defined in Section 6.7.3, “Stage Two: Tests”.

3.1 Create a Contract

We start by creating the contract for the interaction between greeting-ui and fortune-service. The contract should describe the following expectation: greeting-ui makes a GET request to the root URL of fortune-service and expects a response with status 200 and a string (new fortune) in the body

We code this buy using groovy syntax as follows:

import org.springframework.cloud.contract.spec.Contract

Contract.make {
    description("""
should return a fortune string
""")
    request {
        method GET()
        url "/"
    }
    response {
        status 200
        body "new fortune"
    }
}

Save this contract in the fortune-service code base in the following location (which is compliant with Spring Cloud Contract convention): src/test/resources/contracts/<service-name>/<contract-file>. The following image shows the contract file in its location within an IDE:

Figure 6.15. fortune-service Flyway Contract File

fortune service contract file

[Note]Note

You can optionally enable your IDE to assist with contract syntax by adding the Spring Cloud Contract Verifier to your pom.xml file. It is pluggable, and includes groovy and pact by default. The following element shows the dependency to add:

    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-contract-verifier</artifactId>
      <scope>test</scope>
    </dependency>

3.2 Create a Base Class for Contract Tests

Now that we have a coded contract, we want to enable auto-generation of contract-based tests. The auto-generation, which we will configure in the next steps, requires a base class that stubs out the service that satisfies the API call, so that we can run the test without external dependencies (for example, the database). The objective is to focus on testing API semantics.

We create the base class in the fortune-service test package as follows:

package io.pivotal.fortune;

import io.restassured.module.mockmvc.RestAssuredMockMvc;
import org.junit.Before;
import org.mockito.BDDMockito;

public class BaseClass {

    @Before
    public void setup() {
        FortuneService service = BDDMockito.mock(FortuneService.class);
        BDDMockito.given(service.getFortune()).willReturn("foo fortune");
        RestAssuredMockMvc.standaloneSetup(new FortuneController(service));
    }
}

3.3 Enable Automated Contract-based Testing

Now that we have a contract and a base class, we can use the Spring Cloud Contract Maven plugin to auto-generate contract tests, stubs, and a stub jar.

First we add the Spring Cloud Contract version to the list of properties in the fortune-service pom.xml file, since we will reference it when we enable the Spring Cloud Contract maven plugin. To do so, we add the <spring-cloud-contract.version> to the properties element, as follows:

  <properties>
...
    <spring-cloud-contract.version>1.2.1.RELEASE</spring-cloud-contract.version>
...
</properties>

Next, we edit the default profile in the fortune-service pom.xml file to:

  • Add a plugin block for Spring Cloud Contract maven plugin.
  • Configure it to use our base class (io.pivotal.fortune.BaseClass) to generate tests.
  • Configure it to place auto-generated tests in the io.pivotal.fortune.contracttests test package.

Note that the package of the contract tests is included by the include filter in the default profile, so these tests run against the application during the build-and-upload job. For fortune-service, this serves to validate that the application conforms to the contract.

The following listing shows the complete profile:

    <profile>
      <id>default</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>**/*Tests.java</include>
                <include>**/*Test.java</include>
              </includes>
              <excludes>
                <exclude>**/smoke/**</exclude>
                <exclude>**/e2e/**</exclude>
              </excludes>
            </configuration>
          </plugin>
          <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
          </plugin>
          <!--Spring Cloud Contract maven plugin -->
          <plugin>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-contract-maven-plugin</artifactId>
            <version>${spring-cloud-contract.version}</version>
            <extensions>true</extensions>
            <configuration>
              <baseClassForTests>io.pivotal.fortune.BaseClass</baseClassForTests>
              <basePackageForTests>io.pivotal.fortune.contracttests</basePackageForTests>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

When the app is built, the Spring Cloud Contract maven plugin also now produces a stub and a stub jar that contains the contract and stub. This stub jar is uploaded to Bintray, along with the usual app jar. As we see shortly, this stub jar can be used by the greeting-ui development team while they wait for fortune-service to be completed. In other words, this gives the greeting-ui development team a producer to test against that is based on a mutually agreed-upon contract without the lead time of having to wait for fortune-service team to implement anything more than a base class and without having to manually stub out calls to fortune-service based on arbitrary or static responses.

[Tip]Tip

Package the project locally (run mvn package) to observe the tests, stubs, and stub jar that the Spring Cloud Contract Maven plugin generates. See the following image for reference:

Figure 6.16. Generated Tests and Stubs

fortune service generated tests

3.4 Enable backward compatibility API check

To enable Spring Cloud Pipelines to catch any breaking API changes during the the API compatibility check in the build-and-upload job, we add the Spring Cloud Contract Maven plugin to the apicompatibility profile as well.

In this case, we want the plugin to generate tests based on contracts outside of the project (the ones from the latest prod version), so we configure the plugin to download the latest prod stub jar, which contains the old contract. The plugin uses the old contract and the specified base class (which, in our example, is the same as the one in the previous step) to generate contract tests. These tests are run against the new code to validate that it is still compatible with consumers that comply with the prior contract. This ensures backward compatibility for the API.

In short, we edit the apicompatibility profile in the fortune-service pom.xml file to:

  • Add a plugin block for Spring Cloud Contract Maven plugin.
  • Configure it to download the latest prod stub jar from Bintray to obtain the old contract.
  • Configure it to use our base class (io.pivotal.fortune.BaseClass) to generate tests (we use the same one as in the prior step).
  • Configure it to place auto-generated tests in the io.pivotal.fortune.contracttests test package.

Note that the package of the contract tests matches the include filter in the apicompatibility profile, so these tests run against the app during the the API compatibility check during the build-and-upload job. For fortune-service, this serves to validate that the app conforms to the old contract.

The following listing shows the complete profile:

    <profile>
      <id>apicompatibility</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <includes>
                <include>**/contracttests/**/*Tests.java</include>
                <include>**/contracttests/**/*Test.java</include>
              </includes>
            </configuration>
          </plugin>
          <!--Spring Cloud Contract maven plugin -->
          <plugin>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-contract-maven-plugin</artifactId>
            <version>${spring-cloud-contract.version}</version>
            <extensions>true</extensions>
            <configuration>
              <contractsRepositoryUrl>${repo.with.binaries}</contractsRepositoryUrl>
              <contractDependency>
                <groupId>${project.groupId}</groupId>
                <artifactId>${project.artifactId}</artifactId>
                <classifier>stubs</classifier>
                <version>${latest.production.version}</version>
              </contractDependency>
              <contractsPath>/</contractsPath>
              <baseClassForTests>io.pivotal.fortune.BaseClass</baseClassForTests>
              <basePackageForTests>io.pivotal.fortune.contracttests</basePackageForTests>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

Spring Cloud Pipelines dynamically injects the values for ${repo.with.binaries} and ${latest.production.version}. You can run this locally by providing these values manually as system properties in the Maven command.

3.5 Push Changes to GitHub

All changes in Stage Three thus far are in fortune-service. At this point, you should be pushing the following new or modified files:

  • pom.xml
  • src/test/resources/contracts/greeting-ui/shouldReturnAFortune.groovy
  • src/test/java/io/pivotal/fortune/BaseClass.java

3.6 Re-run the fortune-service Pipeline

Run through the fortune-service pipeline to generate stubs. The following output from the build-and-upload job shows the auto-generation of tests and stubs:

Figure 6.17. fortune-service build-and-upload Test and Stub Generation

fortune service build and upload test and stub generation

You should also see output in the build-and-upload job that shows the execution of these tests against the code.

Additionally, you should see the stub jar uploaded to Bintray along with the usual app jar.

Finally, as you run through the pipeline a second time, you should see that the contract tests from the latest prod version run against the new code in the output of the the API compatibility check during the build-and-upload job. In this case, there is no API change. Nonetheless, the tests confirm that the latest prod version of the API can be used with the current code base.

3.7 Enable Stubs for Integration Tests

Now we turn our attention to greeting-ui.

The following image compares the path of a request through greeting-ui in the build phase, both with and without stubs:

Figure 6.18. greeting-ui Build Flow

greeting ui build flow

Without stubs, we expect the response to be the Hystrix fallback response that is hard-coded in greeting-ui. With stubs, however, we can expect the response that was declared in the contract. In this case, the stubs are loaded into the greeting-ui process. This leads us to our next task: Loading the stubs produced by fortune-service.

Enable the in-process Stub Runner

To load the stubs into the greeting-ui process, we must enable the Spring Cloud Contract Stub Runner dependency. This dependency start ans in-process stub runner that automatically configures Wiremock.

Add the following dependency to the greeting-ui pom.xml file:

<dependency>
 <groupId>org.springframework.cloud</groupId>
 <artifactId>spring-cloud-starter-contract-stub-runner</artifactId>
 <scope>test</scope>
</dependency>
Add Integration Tests Aligned with the Contract

Next, we add integration tests to greeting-ui to test for the expected response declared in the contract.

Add the following class to the test package in greeting-ui:

package io.pivotal.fortune;

import io.pivotal.GreetingUIApplication;
import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.contract.stubrunner.spring.AutoConfigureStubRunner;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = GreetingUIApplication.class, webEnvironment = SpringBootTest.WebEnvironment.NONE,
        properties = {"spring.application.name=greeting-ui", "spring.cloud.circuit.breaker.enabled=false", "hystrix.stream.queue.enabled=false"})
@AutoConfigureStubRunner(ids = {"io.pivotal:fortune-service:1.0.0.M1-20180102_203542-VERSION"},
        repositoryRoot = "${REPO_WITH_BINARIES}"
        //workOffline = true
)

public class FortuneServiceTests {

    @Autowired FortuneService fortuneService;

    @Test
    public void shouldSendRequestToFortune() {
        // when
        String fortune = fortuneService.getFortune();
        // then
        BDDAssertions.then(fortune).isEqualTo("foo fortune");
    }

}

At this point, we can get through the build phase for greeting-ui, and the integration tests run against the fortune-service stubs that runs in the greeting-ui process on the Concourse worker.

[Tip]Tip

Notice the configuration of @AutoConfigureStubRunner. You can replace the version with a + sign if using Artifactory or Nexus and it automatically chooses the latest available version on the maven repo.

[Tip]Tip

Setting workOffline=true (commented out but shown earlier for informational purposes) would make the stub runner get the stubs from the local Maven repository. This is useful for local testing. Alternatively, set the $REPO_WITH_BINARIES environment variable to the value used in the credentials file before doing a local Maven build. Then the local build will download the stubs from your remote Maven repository (for example, Bintray).

3.8 Enable Stubs for Smoke Tests

The following image compares the path of a request through greeting-ui in the test phase, both with and without stubs. Note that in the build phase, where the app process runS on the Concourse worker, we ran the stubs in the same process. In the test environment (Cloud Foundry), we run the stubs in a separate process by using a standalone stub runner application. The following image shows the test flow for the greeting-ui application:

Figure 6.19. greeting-ui Test Flow

greeting ui test flow

As in the build phase, without stubs, we expect the response to be the Hystrix fallback response that is hard-coded in greeting-ui. With stubs, however, we can expect the response that was declared in the contract.

We rely on Spring Cloud Pipelines to:

  • Deploy a stub runner application.
  • Provide the stub runner application with the necessary information to download the stubs.
  • Open a port on the stub runner application for each stub.

We rely on the stub runner application to:

  • Download the stubs from our Maven repository (Bintray).
  • Expose each stub on a separate port.
  • Register each stub in the Service Discovery server.

The following steps describe how to configure stubs for smoke tests.

Provide a Stand-alone Stub Runner App Jar

In the Prep step for this tutorial, you cloned the cloudfoundry-stub-runner-boot repo to your local machine. The next step is to build this application and upload it to Bintray to make the jar available to Spring Cloud Pipelines.

As mentioned in Section 6.7.2, “Stage One: Scaffolding” of this tutorial, Bintray requires that a package exist before any application artifacts can be uploaded. Log into the Bintray UI and create a package for cloudfoundry-stub-runner-boot. If you forked this repo, you can use the Import from GitHub option. Otherwise, create the package manually and choose any license (for example, Apache 2.0).

Now you are ready to build and upload this app to Bintray. Edit the following script (which shows cloning, building, and uploading) to match your Bintray URL, the Bintray ID in your ~/.m2/settings/xml file, and the cloudfoundry-stub-runner-boot repository URL (if you chose to fork it):

# Edit to match your Bintray URL and M2 repo ID setting (check your ~/.m2/settings.xml file)
MAVEN_REPO_URL=https://api.bintray.com/maven/ciberkleid/maven-repo/cloudfoundry-stub-runner-boot
MAVEN_REPO_ID=bintray

# Clone cloudfoundry-stub-runner-boot
git clone https://github.com/spring-cloud-samples/cloudfoundry-stub-runner-boot.git
cd cloudfoundry-stub-runner-boot

# Build and upload
./mvnw clean deploy -Ddistribution.management.release.url="${MAVEN_REPO_URL}" -Ddistribution.management.release.id="${MAVEN_REPO_ID}"

You should now see the cloudfoundry-stub-runner-boot artifacts uploaded on Bintray.

Provide Stand-alone Stub Runner Application Manifest

Next, we add a manifest file for the stub runner application for deployment to Cloud Foundry.

Place this file in the greeting-ui repo. The file name and location can be your choice. For this example, we use sc-pipelines/manifest-stubrunner.yml. The following image shows the file in the appropriate folder:

Figure 6.20. greeting-ui Stub Runner Manifest

greeting ui stubrunner manifest

We populate this manifest-stubrunner.yml with the content shown in the next listing so that the stub runner binds to service-registry. The stub runner registers the fortune-service stub there so that greeting-ui can discover it in the same way it discovers the actual fortune-service app endpoint in stage and prod. From the greeting-ui perspective, there is no difference in how it interacts with Eureka and the stub runner application in test and the way it interacts with Eureka and the fortune-service application in stage and prod. The following listing shows the content of manifest-stubrunner.yml:

---
applications:
- name: stubrunner
  timeout: 120
  services:
  - service-registry
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io
----
Provide a Stub Runner Jar and Manifest Information to the Pipeline

Now that we have a jar file and a manifest file for our stub runner application, we need to provide this information to our greeting-ui pipeline so that the pipeline downloads the jar from Bintray and deploys it to Cloud Foundry. We do this through the greeting-ui sc-pipelines.yml file. We add an entry to the list of services in the test section, as follows:

    - name: stubrunner
      type: stubrunner
      coordinates: io.pivotal:cloudfoundry-stub-runner-boot:0.0.1.M1
      pathToManifest: sc-pipelines/manifest-stubrunner.yml

Notice that name matches the name of the application in manifest-stubrunner.yml, coordinates corresponds to the jar coordinates of the Maven repository, and pathToManifest matches our chosen fie name for the stub runner application manifest.

Note also the type is set to stubrunner, which Spring Cloud Pipelines will recognize as a stanalone stub runner app and treat accordingly.

Provide Stub Configuration for the Stub Runner Application

The final steps in the configuration of the stand-alone stub runner app are as follows:

  • Enable the stub runner app to download the fortune-service stub from Bintray.
  • Open a second port on the container to receive requests for this stub.

To accomplish this, we put stub and port configuration information into the properties section of the greeting-ui pom.xml file, by using a property called stubrunner.ids. This property can accept a list of stubrunner IDs. However, for this tutorial, we only have one, as follows:

  <properties>
...
    <!--Tell stub runner app to start this stub-->
    <stubrunner.ids>io.pivotal:fortune-service:1.0.0.M1-20180102_203542-VERSION:stubs:10000</stubrunner.ids>
  </properties>

Spring Cloud Pipelines uses this information in two ways:

  • It provides this information to the stub runner application through the application’s environment variables.

    • Spring Cloud Pipelines also provides the $REPO_WITH_BINARIES as an environment variable for the stub runner application.
    • The stub runner application uses this information to download the stub from Bintray and expose it on the specified port.
  • It opens the additional port specified on the stub runner app and map a new route to it.

    • The format for each route is <stub-runner-app-name>-<hostname-uuid>-<env>-<app-name>-<port>.<domain>.
    • In our example, this would be stubrunner-cyi-test-greeting-ui-10000.cfapps.io.

Since we bound our stub runner application to service-registry (Eureka), the stub runner application registers the stub URL under the FORTUNE-SERVICE application name on Eureka, as the following image shows:

Figure 6.21. greeting-ui Stub Runner Eureka Registration

greeting ui stub runner eureka registration

This completes the process of configuring the stand-alone stub runner application.

[Note]Note

You can automate the port configuration by Spring Cloud Pipelines in the future such that you need not include the port in stubrunner.ids. However, for the moment, we are required to specify the port each stub should use.

Edit Smoke Tests to Align with the Contract

Finally, we edit our smoke tests for greeting-ui to ensure the response does not contain the Hystrix fallback, since we are now expecting a response from the stub. The following listing shows the edited smoke tests:

package smoke;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SmokeTests.class,
        webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class SmokeTests {

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity<String> response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback response
		BDDAssertions.then(response.getBody()).doesNotContain("This fortune is no good. Try another.");
	}

}

In this case, in contrast to the integration test we created earlier for greeting-ui, we do not include @AutoConfigureStubRunner, because we are using a standalone stub runner application.

3.9 Push Changes to GitHub

Push contract-based changes for greeting-ui. You should be pushing the following new or modified files:

  • pom.xml
  • sc-pipelines.yml
  • sc-pipelines/manifest-stubrunner.yml
  • src/test/java/io/pivotal/fortune/FortuneServiceTests.java
  • src/test/java/smoke/SmokeTests.java

At this point, we can run through the full pipeline for greeting-ui and leverage the contract-based stub in both the build and test environments.

Stage Three Recap

What have we accomplished?

By implementing a contract-driven approach with auto-generation of tests and stubs, we have introduced a clean, structured, and reliable way to define, communicate, document, manage, and test APIs. The benefits include the following:

  • Inter-team communication can be simpler.

    • Consumer and producer teams can now communicate requirements through coded contracts.
    • The inventory of contracts serves as a record and reference of the agreed-upon APIs.
  • Developer productivity will increase.

    • Producers can quickly and easily generate contract-based stubs.
    • Consumers no longer have to manually stub out APIs and write tests with arbitrary hard-coded responses. Instead, they can use the auto-generated stubs and test for contract-based responses.
    • Both producers and consumers can validate their code complies with the contract.
    • Producers can verify backward compatibility of API changes.
    • Troubleshooting will be easier.
    • Failure and feedback will be faster.

6.8 Conclusion

This concludes the tutorial on migrating apps for Spring Cloud Pipelines for Cloud Foundry.

Moving forward, the refactoring work shown here can be incorporated into your and your team’s standard practices. We recommend the following practices:

Good:

  • Use Maven or Gradle wrappers.
  • Include a Cloud Foundry manifest file in your application repository.
  • Include a pipeline descriptor (sc-manifest.yml) in your app repository.
  • Create an empty version branch in your application repository.
  • Include artifact repository configuration in the pom.xml file.
  • Align your Cloud Foundry spaces with the Spring Cloud Pipelines model (isolated test space and shared stage and prod spaces).

Better

  • Include default, apicompatibility, smoke, and e2e profiles in the pom.xml file.
  • Organize tests accordingly in your application repository.

Best

  • Use a database migration tool, such as Flyway.
  • Use contract-based API programming.

Implementing all the “good” practices positions you to instantly create pipelines for your applications by using Spring Cloud Pipelines. This is a huge win in terms of consistency and productivity and standardization across development teams. Of course, this is an open source project, so you can modify it to meet your needs.

Implementing the “better” practices ensures the proper tests get run at the proper time. At that point, you can add as much test coverage as you need to have high confidence in your pipelines.

Implementing the “best” practices gives you additional confidence in your pipeline and encourages better programming practices for database version and API management across development teams. It also gives you higher confidence in your pipelines and lets you avoid the cumbersome business of rolling back a database.

Happy coding!