4. Concourse Pipeline (Cloud Foundry)

[Important]Important

In this chapter we assume that you perform deployment of your application to Cloud Foundry PaaS

The Spring Cloud Pipelines repository contains opinionated Concourse pipeline definition. Those jobs will form an empty pipeline and a sample, opinionated one that you can use in your company.

All in all there are the following projects taking part in the whole microservice setup for this demo.

4.1 Step by step

If you want to just run the demo as far as possible using PCF Dev and Docker Compose

4.1.1 Fork repos

There are 4 apps that are composing the pipeline

You need to fork only these. That’s because only then will your user be able to tag and push the tag to repo.

4.1.2 Start Concourse and Artifactory

Concourse + Artifactory can be run locally. To do that just execute the start.sh script from this repo.

git clone https://github.com/spring-cloud/spring-cloud-pipelines
cd spring-cloud-pipelines/concourse
./setup_docker_compose.sh
./start.sh 192.168.99.100

The setup_docker_compose.sh script should be executed once only to allow generation of keys.

The 192.168.99.100 param is an example of an external URL of Concourse (equal to Docker-Machine ip in this example).

Then Concourse will be running on port 8080 and Artifactory 8081.

Deploy the infra JARs to Artifactory

When Artifactory is running, just execute the tools/deploy-infra.sh script from this repo.

git clone https://github.com/spring-cloud/spring-cloud-pipelines
cd spring-cloud-pipelines/
./tools/deploy-infra.sh

As a result both eureka and stub runner repos will be cloned, built and uploaded to Artifactory.

4.1.3 Start PCF Dev

[Tip]Tip

You can skip this step if you have CF installed and don’t want to use PCF Dev The only thing you have to do is to set up spaces.

[Warning]Warning

It’s more than likely that you’ll run out of resources when you reach stage step. Don’t worry! Keep calm and clear some apps from PCF Dev and continue.

You have to download and start PCF Dev. A link how to do it is available here.

The default credentials when using PCF Dev are:

username: user
password: pass
email: user
org: pcfdev-org
space: pcfdev-space
api: api.local.pcfdev.io

You can start the PCF Dev like this:

cf dev start

You’ll have to create 3 separate spaces (email admin, pass admin)

cf login -a https://api.local.pcfdev.io --skip-ssl-validation -u admin -p admin -o pcfdev-org

cf create-space pcfdev-test
cf set-space-role user pcfdev-org pcfdev-test SpaceDeveloper
cf create-space pcfdev-stage
cf set-space-role user pcfdev-org pcfdev-stage SpaceDeveloper
cf create-space pcfdev-prod
cf set-space-role user pcfdev-org pcfdev-prod SpaceDeveloper

You can also execute the ./tools/cf-helper.sh setup-spaces to do this.

4.1.4 Setup the fly CLI

If you go to Concourse website you should see sth like this:

   

running concourse

   

You can click one of the icons (depending on your OS) to download fly, which is the Concourse CLI. Once you’ve downloaded that (and maybe added to your PATH) you can run:

fly --version

If fly is properly installed then it should print out the version.

4.1.5 Setup your credentials.yml

The repo comes with credentials-sample-cf.yml which is set up with sample data (most credentials) are set to be applicable for PCF Dev. Copy this file to a new file credentials.yml (the file is added to .gitignore so don’t worry that you’ll push it with your passwords) and edit it as you wish. For our demo just setup:

  • app-url - url pointing to your forked github-webhook repo
  • github-private-key - your private key to clone / tag GitHub repos
  • repo-with-binaries - the IP is set to the defaults for Docker Machine. You should update it to point to your setup

If you don’t have a Docker Machine just execute ./whats_my_ip.sh script to get an external IP that you can pass to your repo-with-binaries instead of the default Docker Machine IP.

Below you can see what environment variables are required by the scripts. To the right hand side you can see the default values for PCF Dev that we set in the credentials-sample-cf.yml.

Property NameProperty DescriptionDefault value
 

BUILD_OPTIONS

Additional options you would like to pass to the Maven / Gradle build

PAAS_TEST_API_URL

The URL to the CF Api for TEST env

api.local.pcfdev.io

PAAS_STAGE_API_URL

The URL to the CF Api for STAGE env

api.local.pcfdev.io

PAAS_PROD_API_URL

The URL to the CF Api for PROD env

api.local.pcfdev.io

PAAS_TEST_ORG

Name of the org for the test env

pcfdev-org

PAAS_TEST_SPACE_PREFIX

Prefix of the name of the CF space for the test env to which the app name will be appended

sc-pipelines-test

PAAS_STAGE_ORG

Name of the org for the stage env

pcfdev-org

PAAS_STAGE_SPACE

Name of the space for the stage env

sc-pipelines-stage

PAAS_PROD_ORG

Name of the org for the prod env

pcfdev-org

PAAS_PROD_SPACE

Name of the space for the prod env

sc-pipelines-prod

REPO_WITH_BINARIES_FOR_UPLOAD

URL to repo with the deployed jars

http://192.168.99.100:8081/artifactory/libs-release-local

M2_SETTINGS_REPO_ID

The id of server from Maven settings.xml

artifactory-local

PAAS_HOSTNAME_UUID

Additional suffix for the route. In a shared environment the default routes can be already taken

 

JAVA_BUILDPACK_URL

The URL to the Java buildpack to be used by CF

https://github.com/cloudfoundry/java-buildpack.git#v3.8.1

4.1.6 Build the pipeline

Log in (e.g. for Concourse running at 192.168.99.100 - if you don’t provide any value then localhost is assumed). If you execute this script (it assumes that either fly is on your PATH or it’s in the same folder as the script is):

./login.sh 192.168.99.100

Next run the command to create the pipeline.

./set_pipeline.sh

Then you’ll create a github-webhook pipeline under the docker alias, using the provided credentials.yml file. You can override these values in exactly that order (e.g. ./set-pipeline.sh some-project another-target some-other-credentials.yml)

4.1.7 Run the github-webhook pipeline

   

Figure 4.1. Click Login

concourse login

   

Figure 4.2. Pick main team

concourse team main

   

Figure 4.3. Log in with concourse user and changeme password

concourse user pass

   

Figure 4.4. Your screen should look more or less like this

concourse pipeline

   

Figure 4.5. Unpause the pipeline by clicking in the top lefr corner and then clicking the play button

start pipeline

   

Figure 4.6. Click 'generate-version'

generate version

   

Figure 4.7. Click + sign to start a new build

run pipeline

   

Figure 4.8. The job is pending

concourse pending

   

Figure 4.9. Job is pending in the main screen

job running

   

Figure 4.10. Job is running in the main screen

running pipeline