Spring Cloud Data Flow for Apache Mesos

Spring Cloud Data Flow for Apache Mesos is a cloud-native orchestration service for composable data microservices on Apache Mesos. With Spring Cloud Data Flow, developers can create and orchestrate data pipelines for common use cases such as data ingest, real-time analytics, and data import/export.

Quick Start
Fork me on GitHub

NOTE: The Spring Cloud Data Flow team have been happy stewards of this project, but now feel that TrustedChoice.com will be a better home, and we are donating the project to them to carry it forward.

Please see the new TrustedChoice.com repository for the latest on this project as the Spring Cloud Data Flow team will no longer maintain it.

Spring Cloud Data Flow for Apache Mesos offers a collection of patterns and best practices for data microservices running as streaming and batch data pipelines in Apache Mesos.

Features

  • Consume streaming and batch data microservices as maven artifacts
  • Create, unit-test, troubleshoot and manage data microservices in isolation
  • Develop using: DSL, Shell, REST-APIs, Dashboard, and Flo
  • Take advantage of Apache Mesos value-adds such as metrics, security, logging, health checks, and remote management at each data microservice level
  • Scale stream and batch pipelines through Marathon APIs or Marathon UI
  • Deploy task and batch jobs through Chronos APIs or Chronos UI

Quick Start

Step 1 - Configure, Deploy and Download a Mesos cluster, Spring Cloud Data Flow for Apache Mesos Server and the Spring Cloud Data Flow Shell Application

Step 2 - Create ‘ticktock’ Stream

dataflow:>stream create --name ticktock --definition "time | log" --deploy

Step 3 - Verify ‘ticktock’ Results in Mesos UI

Step 4 - Launch Dashboard at: http://<SCDF_SERVER_HOST>:<PORT>/dashboard