13. The demo setup (Cloud Foundry)

The demo uses two applications: Github Webhook and Github analytics code. The following image shows how these application communicate with each other:

Figure 13.1. Github Webhook listens to HTTP calls and sends a message to Github Analytics

demo

   

For the demo scenario we have two applications. Github Analytics and Github Webhook. Let’s imagine a case where Github is emitting events via HTTP. Github Webhook has an API that could register to such hooks and receive those messages. Once this happens Github Webhook sends a message by RabbitMQ to a channel. Github Analytics is listening to those messages and stores them in a MySQL database.

Figure 13.2. Github Analytics exposes metrics that are polled by Prometheus

demo metrics

   

Github Analytics has its KPIs (Key Performance Indicators) monitored. In the case of that application the KPI is number of issues.

Figure 13.3. Grafana alerts Slack over Prometheus metrics

demo alerting

   

Let’s assume that if we go below the threshold of X issues then an alert should be sent to Slack.

13.1 Deploying Production Applications to PCF Dev

In a real-world scenario, we would not want to automatically provision services such as RabbitMQ, MySQL, or Eureka each time we deploy a new application to production. Typically, production is provisioned manually (often by using automated solutions). In our case, before you deploy to production, you can provision the pcfdev-prod space by using the cf-helper.sh. To do so, call the following script:

$ ./cf-helper.sh setup-prod-infra

The CF CLI:

  • Logs in to PCF Dev,
  • Targets the pcfdev-prod space
  • Sets up:

    • RabbitMQ (under the rabbitmq-github name)
    • MySQL (under mysql-github-analytics name)
    • Eureka (under github-eureka name)

13.2 Running Prometheus on CF

You can check out Toshiaki Maki’s code on how to automate Prometheus installation on CF.

Go to https://prometheus.io/download/ and download the Linux binary. Then run the following command:

cf push sc-pipelines-prometheus -b binary_buildpack -c './prometheus -web.listen-address=:8080' -m 64m

Also, localhost:9090 in prometheus.yml should be localhost:8080.

The file should resemble the following listing to work with the demo setup (change github-analytics-sc-pipelines.cfapps.io to your github-analytics installation).

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'codelab-monitor'

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first.rules"
  # - "second.rules"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:8080']

  - job_name: 'demo-app'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    metrics_path: '/prometheus'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['github-analytics-sc-pipelines.cfapps.io']

A deployed version for the Spring Cloud Pipelines demo is available here.

13.3 Running Grafana on CF

You can check out Toshiaki Maki’s code to see how to automate Prometheus installation on CF.

Download the tarball from https://grafana.com/grafana/download?platform=linux and set http_port = 8080 in conf/default.ini. Then run the following the command:

cf push sc-pipelines-grafana -b binary_buildpack -c './bin/grafana-server web' -m 64m

The demo uses Grafana Dashboard with an ID of 2471.

A deployed version for the Spring Cloud Pipelines demo is available here