Operators
| Operator | Name | What it does |
|---|---|---|
~> |
Chain | a ~> b - output of a feeds into b |
& / &> |
Fan-out | a & b - run both with the same input (&> runs them concurrently) |
* / *> |
Product | a * b - run on different inputs (*> runs them concurrently) |
>> |
Sequence | a >> b - run in order, keep b's result |
| |
Fan-in | a | b - route an Either input to the matching branch |
+ |
Choice | a + b - route an Either input through independent branches |
<|> |
Fallback | a <|> b - if a throws, run b on the same input |
Chain nodes with ~>
The output of one node feeds the next. This is the backbone of most pipelines:
Send one input to several nodes with & / &>
Both nodes get the same input and their results come back as a pair. Use &> to run them concurrently under an
effect like Future. Here one user goes in and both cards are built from it:
Give each node its own input with * / *>
Where & sends the same input to both, * splits a pair: the first half goes left, the second goes right. Use *>
to run them concurrently. Here the name goes to trimName and the age to bumpAge:
Run steps in order with >>
Runs each step in order and keeps only the last result. Handy for setup steps you run for their side effects:
Route an Either to the matching branch with |
A Left goes to the left node, a Right to the right one, and both produce the same type. Here a legacy id or a uuid comes in, and one account comes out:
Handle each side of an Either with +
Like |, but each side keeps its own output type, so you still have an Either at the end. Here refunds and charges
are handled separately, then merged back with |:
Fall back to another node with <|>
Runs the left node, and if it throws, runs the right one on the original input. Here fetchCached only runs
if fetchLive fails:
Under an effect that tracks errors (like Try), the recovery goes through the effect instead of a thrown exception.
Concurrency needs a concurrent effect
&> and *> only run their branches concurrently when compiled to a concurrent
effect such as Future (or IO), via .compile[Future]. Plain unsafeRun (the
Id interpreter) has no threads, so it runs them in sequence.
Auto-flatten and .zip
Chaining fan-outs auto-flattens the tuple: a & b & c has type
Node[X, (A, B, C)] (not ((A, B), C)), and the same holds for &>:
import etl4s._
val n1 = Node[String, Int](_.length)
val n2 = Node[String, String](_.toUpperCase)
val n3 = Node[String, Boolean](_.nonEmpty)
val flat = n1 & n2 & n3
flat.unsafeRun("hi")
flat has type Node[String, (Int, String, Boolean)], and you will get:
If you already have a node whose output is a nested tuple, .zip flattens it:
nested has type Node[String, ((Int, String), Boolean)] and flatten has type Node[String, (Int, String, Boolean)].
etl4s