Spring Cloud Data Flow for Cloud Foundry

Spring Cloud Data Flow for Cloud Foundry is a cloud-native orchestration service for composable data microservices on Cloud Foundry. With Spring Cloud Data Flow, developers can create and orchestrate data pipelines for common use cases such as data ingest, real-time analytics, and data import/export.

Quick Start
Fork me on GitHub

Spring Cloud Data Flow for Cloud Foundry offers a collection of patterns and best practices for data microservices running as streaming and batch data pipelines in Cloud Foundry.

Features

  • Consume streaming and batch data microservices as maven artifacts
  • Create, unit-test, troubleshoot and manage data microservices in isolation
  • Perform blue-green deployments on data microservices
  • Develop using: DSL, Shell, REST-APIs, Dashboard, and Flo
  • Take advantage of Cloud Foundry value-adds such as metrics, logging, health checks, and remote management at each data microservice level
  • Scale stream and batch pipelines from cf cli or Apps Manager

Quick Start

Step 1 - Download Spring Cloud Data Flow's Cloud Foundry Server and Shell Applications

Step 2 - Create RabbitMQ Service Instance in Cloud Foundry, this will be used as the messaging middleware

Step 3 - Deploy Spring Cloud Data Flow's Cloud Foundry Server Application to Cloud Foundry

Step 4 - Run and Connect the Shell Application with Server running on Cloud Foundry

Step 5 - Create ‘ticktock’ Stream

dataflow:>stream create ticktock --definition "time | log" --deploy

Step 6 - Launch Dashboard at: http://<your.dataflow.server.on.cloudfoundry>/dashboard

PCF vs. OSS CF Compatibility

Attached below is the compatibility matrix of the recent three Pivotal Cloud Foundry and its open-source equivalent of Cloud Foundry releases.

PCF Release OSS CF Release
PCF 1.10 CF 252 + patches
PCF 1.9 CF 246 + patches
PCF 1.8 CF 239 + patches