Debug taps

A tap is a passive observer you attach to one node of a published flow. It mirrors the node's input (the packet arriving) and output (the packet it produced) into a durable, queryable hit log — so you can see the actual data flowing through a node while the flow runs, and change what you're watching on the fly without editing or republishing the flow.

Taps are the debugging counterpart to flow alerts: alerts capture failures; taps capture data through a chosen node.

Why it works the way it does

The FlowDSL runtime is single-path — selectNext picks exactly one outgoing edge per node — so there is no execution fan-out. A tap therefore does not run as a parallel node; it rides the existing FlowEventSink event stream (the same stream that powers Trace/Live/Alerts). On each node.started / node.done event matching an enabled tap, the TapSink records a hit. Zero runtime change, works on any published flow, toggle without a republish.

• Package: go-flowdsl/flowexec/taps (models, Store, MemStore, TapSink).
• Mongo store: flowexec/storage-mongo/store/tapstore — flow_taps (configs, persistent + shared across the api / admin-api containers via the same DB) and flow_tap_hits (a 24 h TTL, so debug data self-cleans).
• Wiring: the flowexec module mounts the TapSink outermost on the sink chain (broadcaster → taps → alerts → raw) and reloads its match cache on startup + after every tap CRUD.

Match key gotcha: a run's FlowEvent.FlowID is the workflow document ID (e.g. asyncshop-checkout-standard), not the flowexec flow-record ID (flow.<uuid>) the admin API uses. Each tap stores a WorkflowID (resolved from the published version at create time); the sink matches events by that but records each hit under the flow-record ID so the admin trace query works.

See it in action

Recorded on the real Redelay admin — publish a flow, tap a node, run it, watch the packet transform live. (See E2E & demo videos for how this was made.)

Try it in 60 seconds — the debug-playground demo

The flowtemplates module ships examples/debug-playground — a tiny, infra-free flow (no Kafka/Mongo needed) built for exactly this. It takes an order packet {order_id, amount_minor, currency}, adds VAT (gross_minor = amount × 1.23) with a core/math node, and renders a one-line summary with core/template-render.

1. Studio → Create flow → Debug playground → Publish.
2. Flows → Debug: pick the flow, tap the gross node (capture both), Add tap.
3. Launch a run (POST /flows/{id}/runs with {"input":{"order_id":"A-100", "amount_minor":1000,"currency":"PLN"}}, or Run in Studio).
4. The Trace panel shows the gross node's input (amount_minor: 1000) and output (gross_minor: 1230) — the packet transforming, live. Add a second tap on summary to watch the rendered string appear.

Because it uses only in-process core nodes, the demo runs anywhere and is the fixture the taps end-to-end tests exercise (flowexec/taps_e2e_test.go).

Using it — admin

Admin → Flows → Debug (/flows/debug):

1. Pick a flow, then a node to tap.
2. Choose capture (input / output / both) and action (log; log + raise an alert; or log + notify me, an in-app notification each time the node fires). Optionally bound volume on a busy flow with Sample 1-in-N and/or an only when field=value filter, then Add tap.
3. Run the flow (storefront action, POST /flows/{id}/runs, or Studio Run).
4. The Trace panel lists every captured hit — node, side, run, timestamp — each expandable to the full JSON payload. Filter by node; enable 3 s auto-refresh for a live view. Toggle a tap off or delete it at any time.

Using it — API

A hit's payload carries { input } (on node.started) or { output } (on node.done) — the full accumulating packet at that node, so you see everything on the bus (e.g. a checkout node's hit includes rates, customer, shipping_method …), not just the node's headline output.

Roadmap — remaining steps

The feature was approved as option A — passive observer taps (not engine fan-out). Shipped so far: the taps package + Mongo store + module wiring + admin endpoints (go-flowdsl v0.1.9 / flowexec/module v0.2.11), the admin Debug page, and the Studio node badge — a tapped node shows a purple ⦿ TAP badge and the node context menu toggles the tap (Add / Remove debug tap), observe-only. Also shipped (go-flowdsl v0.1.10 / flowexec/module v0.2.12): the clickable canvas badge (click ⦿ TAP to remove — no right-click), redaction (sensitive fields — passwords/tokens/card numbers/… — masked in captured hits by default; email kept as it's usually what you're debugging; never mutates the live event), and retention (DELETE /flows/{id}/taps/hits[?node_id=] + a Clear button; hits also auto-expire after 24 h via a Mongo TTL).

Also shipped: cross-run diff — when a specific node is selected in the trace, the Debug page groups that node's captured hits by run and shows a flattened, field-level diff (added / removed / changed) of two runs' input or output packet at that node — the "why did this run differ here?" view. Built entirely over the existing /taps/hits data (no backend change).

Also shipped (go-flowdsl v0.1.12 / flowexec/module v0.2.14): sampling + field filters to bound hit volume on high-traffic flows. A tap carries an optional sample (record only 1 in every N matching events; the per-tap counter resets on each cache reload) and match_field / match_value (a dotted-path == filter on the node's packet — e.g. shipping_method == courier captures only the courier case). Both apply in TapSink.capture before the hit is recorded (filter first, then the sample counter); pure tap metadata, no runtime change. Set them on the admin Add tap bar (a Sample 1-in-N input and an only when field=value pair); active-tap chips show the 1-in-N and field=value badges.

Also shipped (go-flowdsl v0.1.13 / flowexec/module v0.2.15 / go-modules v0.5.3): the notify tap action — two decoupled reusable pieces, both in the framework:

• Dispatch event. A notify-action tap publishes a channel-agnostic flow.tap.notify event on the bus each time its node fires (flowexec module's WithNotifyHook). The topic + payload (taps.NotifyTopic / taps.NotifyEvent) are the shared contract in the dep-light taps package. flowexec never knows how the notification is delivered — it only says a tapped node fired, so any flow or module can subscribe (in-app, email, SMS, webhook, Slack…).
• Notify module. The redelay notifications module (go-modules) consumes flow.tap.notify and delivers an in-app notification (persist + SSE fan-out) to the tap's target user — the admin who set the tap (Tap.NotifyUser, captured from claims at create time). Pick Log + Notify me in the Add-tap action.

That completes the debug-taps roadmap. Every reusable piece — the tap models + sink, the redaction, the dispatch event, and the notify delivery — lives in the Redelay framework (go-flowdsl + go-modules), not in any downstream app.

See also: Observability (Trace / Live / Lifecycle / Alerts).