8. Jenkins Pipeline (Cloud Foundry)

[Important]Important

In this chapter we assume that you perform deployment of your application to Cloud Foundry PaaS

The Spring Cloud Pipelines repository contains job definitions and the opinionated setup pipeline using Jenkins Job DSL plugin. Those jobs will form an empty pipeline and a sample, opinionated one that you can use in your company.

All in all there are the following projects taking part in the whole microservice setup for this demo.

8.1 Step by step

This is a guide for Jenkins Job DSL based pipeline.

If you want to just run the demo as far as possible using PCF Dev and Docker Compose

8.1.1 Fork repos

There are 4 apps that are composing the pipeline

You need to fork only these. That’s because only then will your user be able to tag and push the tag to repo.

8.1.2 Start Jenkins and Artifactory

Jenkins + Artifactory can be ran locally. To do that just execute the start.sh script from this repo.

git clone https://github.com/spring-cloud/spring-cloud-pipelines
cd spring-cloud-pipelines/jenkins
./start.sh yourGitUsername yourGitPassword yourForkedGithubOrg

Then Jenkins will be running on port 8080 and Artifactory 8081. The provided parameters will be passed as env variables to Jenkins VM and credentials will be set in your set. That way you don’t have to do any manual work on the Jenkins side. In the above parameters, the third parameter could be yourForkedGithubOrg or yourGithubUsername. Also the REPOS env variable will contain your GitHub org in which you have the forked repos.

Deploy the infra JARs to Artifactory

When Artifactory is running, just execute the tools/deploy-infra.sh script from this repo.

git clone https://github.com/spring-cloud/spring-cloud-pipelines
cd spring-cloud-pipelines/
./tools/deploy-infra.sh

As a result both eureka and stub runner repos will be cloned, built and uploaded to Artifactory.

8.1.3 Start PCF Dev

[Tip]Tip

You can skip this step if you have CF installed and don’t want to use PCF Dev The only thing you have to do is to set up spaces.

[Warning]Warning

It’s more than likely that you’ll run out of resources when you reach stage step. Don’t worry! Keep calm and clear some apps from PCF Dev and continue.

You have to download and start PCF Dev. A link how to do it is available here.

The default credentials when using PCF Dev are:

username: user
password: pass
email: user
org: pcfdev-org
space: pcfdev-space
api: api.local.pcfdev.io

You can start the PCF Dev like this:

cf dev start

You’ll have to create 3 separate spaces (email admin, pass admin)

cf login -a https://api.local.pcfdev.io --skip-ssl-validation -u admin -p admin -o pcfdev-org

cf create-space pcfdev-test
cf set-space-role user pcfdev-org pcfdev-test SpaceDeveloper
cf create-space pcfdev-stage
cf set-space-role user pcfdev-org pcfdev-stage SpaceDeveloper
cf create-space pcfdev-prod
cf set-space-role user pcfdev-org pcfdev-prod SpaceDeveloper

You can also execute the ./tools/cf-helper.sh setup-spaces to do this.

8.1.4 Run the seed job

We already create the seed job for you but you’ll have to run it. When you do run it you have to provide some properties. By default we create a seed that has all the properties options, but you can delete most of it. If you set the properties as global env variables you have to remove them from the seed.

Anyways, to run the demo just provide in the REPOS var the comma separated list of URLs of the 2 aforementioned forks of github-webhook and `github-analytics'.

   

Figure 8.1. Click the 'jenkins-pipeline-seed-cf' job for Cloud Foundry and jenkins-pipeline-seed-k8s for Kubernetes

seed click

   

Figure 8.2. Click the 'Build with parameters'

seed run

   

Figure 8.3. The REPOS parameter should already contain your forked repos (you’ll have more properties than the ones in the screenshot)

seed

   

Figure 8.4. This is how the results of seed should look like

seed built

8.1.5 Run the github-webhook pipeline

We already create the seed job for you but you’ll have to run it. When you do run it you have to provide some properties. By default we create a seed that has all the properties options, but you can delete most of it. If you set the properties as global env variables you have to remove them from the seed.

Anyways, to run the demo just provide in the REPOS var the comma separated list of URLs of the 2 aforementioned forks of github-webhook and github-analytics.

   

Figure 8.5. Click the 'github-webhook' view

seed views

   

Figure 8.6. Run the pipeline

pipeline run

   

[Important]Important

If your build fails on the deploy previous version to stage due to missing jar, that means that you’ve forgotten to clear the tags in your repo. Typically that’s due to the fact that you’ve removed the Artifactory volume with deployed JAR whereas a tag in the repo is still pointing there. Check out this section on how to remove the tag.

   

Figure 8.7. Click the manual step to go to stage (remember about killing the apps on test env). To do this click the ARROW next to the job name

pipeline manual

   

[Important]Important

Most likely you will run out of memory so when reaching the stage environment it’s good to kill all apps on test. Check out the FAQ section for more details!

   

Figure 8.8. The full pipeline should look like this

pipeline finished

   

8.2 Declarative pipeline & Blue Ocean

You can also use the declarative pipeline approach with the Blue Ocean UI. Here is a step by step guide to run a pipeline via this approach.

The Blue Ocean UI is available under the blue/ URL. E.g. for Docker Machine based setup http://192.168.99.100:8080/blue.

   

Figure 8.9. Open Blue Ocean UI and click on github-webhook-declarative-pipeline

blue 1

   

Figure 8.10. Your first run will look like this. Click Run button

blue 2

   

Figure 8.11. Enter parameters required for the build and click run

blue 3

   

Figure 8.12. A list of pipelines will be shown. Click your first run.

blue 4

   

Figure 8.13. State if you want to go to production or not and click Proceed

blue 5

   

Figure 8.14. The build is in progress…​

blue 6

   

Figure 8.15. The pipeline is done!

blue 7

   

[Important]Important

There is no possibility of restarting pipeline from specific stage, after failure. Please check out this issue for more information

[Warning]Warning

Currently there is no way to introduce manual steps in a performant way. Jenkins is blocking an executor when manual step is required. That means that you’ll run out of executors pretty fast. You can check out this issue for and this StackOverflow question for more information.

8.3 Jenkins Cloud Foundry customization

 All the steps below are not necessary to run the demo. They are needed only
when you want to do some custom changes.

8.3.1 All env vars

The env vars that are used in all of the jobs are as follows:

Property NameProperty DescriptionDefault value
 

BINARY_EXTENSION

Extension of the binary uploaded to Artifactory / Nexus. Example: change this to war for WAR artifacts

PAAS_TEST_API_URL

The URL to the CF Api for TEST env

api.local.pcfdev.io

PAAS_STAGE_API_URL

The URL to the CF Api for STAGE env

api.local.pcfdev.io

PAAS_PROD_API_URL

The URL to the CF Api for PROD env

api.local.pcfdev.io

PAAS_TEST_ORG

Name of the org for the test env

pcfdev-org

PAAS_TEST_SPACE_PREFIX

Prefix of the name of the CF space for the test env to which the app name will be appended

sc-pipelines-test

PAAS_STAGE_ORG

Name of the org for the stage env

pcfdev-org

PAAS_STAGE_SPACE

Name of the space for the stage env

sc-pipelines-stage

PAAS_PROD_ORG

Name of the org for the prod env

pcfdev-org

PAAS_PROD_SPACE

Name of the space for the prod env

sc-pipelines-prod

REPO_WITH_BINARIES_FOR_UPLOAD

URL to repo with the deployed jars

http://artifactory:8081/artifactory/libs-release-local

M2_SETTINGS_REPO_ID

The id of server from Maven settings.xml

artifactory-local

JDK_VERSION

The name of the JDK installation

jdk8

PIPELINE_VERSION

What should be the version of the pipeline (ultimately also version of the jar)

1.0.0.M1-${GROOVY,script ="new Date().format('yyMMdd_HHmmss')"}-VERSION

GIT_EMAIL

The email used by Git to tag repo

[email protected]

GIT_NAME

The name used by Git to tag repo

Pivo Tal

PAAS_HOSTNAME_UUID

Additional suffix for the route. In a shared environment the default routes can be already taken

 

AUTO_DEPLOY_TO_STAGE

Should deployment to stage be automatic

false

AUTO_DEPLOY_TO_PROD

Should deployment to prod be automatic

false

API_COMPATIBILITY_STEP_REQUIRED

Should api compatibility step be required

true

DB_ROLLBACK_STEP_REQUIRED

Should DB rollback step be present

true

DEPLOY_TO_STAGE_STEP_REQUIRED

Should deploy to stage step be present

true

JAVA_BUILDPACK_URL

The URL to the Java buildpack to be used by CF

https://github.com/cloudfoundry/java-buildpack.git#v3.8.1

 

BUILD_OPTIONS

Additional options you would like to pass to the Maven / Gradle build

8.3.2 Jenkins Credentials

In your scripts we reference the credentials via IDs. These are the defaults for credentials

Property NameProperty DescriptionDefault value

PAAS_PROD_CREDENTIAL_ID

Credential ID for CF Prod env access

cf-prod

GIT_CREDENTIAL_ID

Credential ID used to tag a git repo

git

GIT_SSH_CREDENTIAL_ID

SSH credential ID used to tag a git repo

gitSsh

GIT_USE_SSH_KEY

if true will pick to use the SSH credential id

false

REPO_WITH_BINARIES_CREDENTIAL_ID

Credential ID used for the repo with jars

repo-with-binaries

PAAS_TEST_CREDENTIAL_ID

Credential ID for CF Test env access

cf-test

PAAS_STAGE_CREDENTIAL_ID

Credential ID for CF Stage env access

cf-stage

If you already have in your system a credential to for example tag a repo you can use it by passing the value of the property GIT_CREDENTIAL_ID

[Tip]Tip

Check out the cf-helper script for all the configuration options!