Spring Cloud Data Flow for Apache Mesos

Spring Cloud Data Flow for Apache Mesos is a cloud-native orchestration service for composable data microservices on Apache Mesos. With Spring Cloud Data Flow, developers can create and orchestrate data pipelines for common use cases such as data ingest, real-time analytics, and data import/export.

Quick Start
Fork me on GitHub

Spring Cloud Data Flow for Apache Mesos offers a collection of patterns and best practices for data microservices running as streaming and batch data pipelines in Apache Mesos.

Features

  • Consume streaming and batch data microservices as maven artifacts
  • Create, unit-test, troubleshoot and manage data microservices in isolation
  • Develop using: DSL, Shell, REST-APIs, Dashboard, and Flo
  • Take advantage of Apache Mesos value-adds such as metrics, security, logging, health checks, and remote management at each data microservice level
  • Scale stream and batch pipelines through Marathon APIs or Marathon UI
  • Deploy task and batch jobs through Chronos APIs or Chronos UI

Quick Start

Step 1 - Configure, Deploy and Download a Mesos cluster, Spring Cloud Data Flow for Apache Mesos Server and the Spring Cloud Data Flow Shell Application

Step 2 - Create ‘ticktock’ Stream dataflow:>stream create --name ticktock --definition "time | log" --deploy

Step 3 - Verify ‘ticktock’ Results in Mesos UI

Step 4 - Launch Dashboard at: http://<SCDF_SERVER_HOST>:<PORT>/dashboard