etl4s

Powerful, whiteboard-style ETL

TRY IT! GET STARTED GITHUB

import etl4s._

/* Define your building blocks */
val fiveExtract = Extract(5)
val timesTwo    = Transform[Int, Int](_ * 2)
val exclaim     = Transform[Int, String](n => s"$n!")
val consoleLoad = Load[String, Unit](println(_))

/* Add config with .requires */
val dbLoad      = Load[String, Unit].requires[String] { dbType => s =>
  println(s"Saving to $dbType DB: $s")
}

/* Stitch your pipeline */
val pipeline =
  fiveExtract ~> timesTwo ~> exclaim ~> (consoleLoad &> dbLoad)

/* Provide config, then run*/
pipeline.provide("sqlite").unsafeRun(())

Whiteboard-style

Model pipelines like you think - with visual, readable chaining

Config-driven

Your pipelines are typed, declarative endpoints - easy to compose and trigger

Type-safe

Prevent bugs by catching type mismatches at compile time