FlowDSL Sync
FlowDSL Sync Architecture
How FlowDSL diagrams stay in sync with Go module code — from Studio edit to deployed container.
Status
Draft — proposed architecture, not yet implemented.
The Problem
FlowDSL .flowdsl.yaml files in backend/flows/ currently describe business flows as documentation only. The runtime doesn't read them. There's no enforcement that Go handlers implement all declared nodes, and changing an edge's delivery mode requires manual code changes. This document defines an architecture where the diagram is the authoritative source of wiring.
Core Assumptions
- Redeploy is acceptable. A container restart when a flow changes is fine. Hot-reload is a later optimization, not a requirement.
- Compile-time safety over runtime safety. If a node in the diagram has no handler, the build should fail — not the server at startup.
- Modules own their nodes. Each Go module declares which FlowDSL
operationIdvalues it implements. The framework discovers them automatically. - Delivery mode is diagram-owned. Changing
direct→durableon an edge should require zero Go code changes — only the YAML changes.
Architecture Overview
┌──────────────────────────────────────────────────────────────────────┐
│ 1. DIAGRAM LAYER (Studio / .flowdsl.yaml) │
│ │
│ backend/flows/user_registration.flowdsl.yaml │
│ backend/flows/module_submission.flowdsl.yaml │
│ backend/flows/email_notification.flowdsl.yaml │
│ ... │
│ │
│ Source of truth for: nodes, edges, delivery modes, retry policies, │
│ idempotency keys, packet types. │
└──────────────────────┬───────────────────────────────────────────────┘
│
│ go run ./cmd/generate/ flows
│ (runs in CI after any .flowdsl.yaml change)
▼
┌──────────────────────────────────────────────────────────────────────┐
│ 2. CODEGEN LAYER (backend/flows/gen/) │
│ │
│ user_registration_flow_gen.go │
│ module_submission_flow_gen.go │
│ email_notification_flow_gen.go │
│ │
│ Generated per flow: │
│ - Typed XxxFlowNodes struct (one field per node operationId) │
│ - RegisterXxxFlow(bus, nodes) wiring function │
│ - Edge wiring is compiled — delivery mode → go-events call │
│ - Checked into git (diff = what the diagram change produced) │
└──────────────────────┬───────────────────────────────────────────────┘
│
│ go build ./... (fails if nodes unimplemented)
▼
┌──────────────────────────────────────────────────────────────────────┐
│ 3. NODE HANDLER LAYER (backend/modules/*/flows.go) │
│ │
│ community/flows.go → func (m *Module) UserRegistrationNodes() │
│ marketplace/flows.go → func (m *Module) ModuleSubmissionNodes() │
│ email/flows.go → func (m *Module) EmailNotificationNodes() │
│ │
│ Developer writes: pure business logic functions. │
│ No transport knowledge. Same service layer as HTTP handlers. │
│ Scaffold generates stubs for new operationIds automatically. │
└──────────────────────┬───────────────────────────────────────────────┘
│
│ compiled into binary
▼
┌──────────────────────────────────────────────────────────────────────┐
│ 4. RUNTIME (started in cmd/api/main.go) │
│ │
│ flows.RegisterAll(bus, modules) │
│ → starts goroutines for durable/ephemeral/stream edges │
│ → direct edges become inline function calls │
│ → wiring is compiled in, not dynamic │
└──────────────────────────────────────────────────────────────────────┘
What Gets Generated
For each .flowdsl.yaml, the generator produces one *_flow_gen.go file.
Example: user_registration_flow_gen.go
// Code generated from user_registration.flowdsl.yaml. DO NOT EDIT.
// Source: backend/flows/user_registration.flowdsl.yaml
// Generated: 2026-04-10T11:30:00Z
package flows
import (
"context"
"time"
"github.com/redelay/go-framework/flowdsl"
)
// UserRegistrationFlowNodes holds all node handlers for the user_registration flow.
// Every field must be set — the compiler enforces it at construction time.
type UserRegistrationFlowNodes struct {
// source: receives POST /register
ReceiveRegistration flowdsl.SourceHandler
// transform: validates email, password, account_type
ValidateRegistrationInput flowdsl.RouterHandler // valid → continue, invalid → reject
// router: routes by account_type (developer / community / customer)
RouteRegistrationByType flowdsl.RouterHandler
// action: creates core user + community profile in MongoDB
CreateUserAccount flowdsl.NodeHandler
// action: queues verification email via email.send event
SendVerificationEmail flowdsl.NodeHandler
// publish: emits users.registered to Kafka
PublishUserRegistered flowdsl.PublishHandler
// terminal: returns 400 with validation errors
RejectInvalidRegistration flowdsl.TerminalHandler
}
// RegisterUserRegistrationFlow wires the user_registration flow using the provided handlers.
// Edge delivery modes are compiled in from the diagram — changing the YAML regenerates this function.
func RegisterUserRegistrationFlow(bus flowdsl.EventBus, nodes UserRegistrationFlowNodes) {
// Edge: RegistrationSubmitted → ValidateRegistration [direct]
// Delivery: in-process, microsecond latency, no durability
// (triggered externally by the HTTP handler calling nodes.ReceiveRegistration)
// Edge: ValidateRegistration.valid → RouteByAccountType [direct]
// Edge: ValidateRegistration.invalid → RejectInvalidRegistration [direct]
// These are handled by the RouterHandler return value (port name)
// Edge: RouteByAccountType.* → CreateUserAccount [durable]
// Delivery: MongoDB-backed packet, retry x3, exponential backoff 2s
// IdempotencyKey: {{payload.email}}-register
bus.Subscribe(flowdsl.DurableConsumer{
Topic: "flow.user_registration.create_account",
GroupID: "flow.user_registration",
IdempotencyKey: func(p map[string]any) string {
return fmt.Sprintf("%v-register", p["email"])
},
RetryPolicy: flowdsl.RetryPolicy{
MaxAttempts: 3,
Backoff: flowdsl.ExponentialBackoff,
InitialDelay: 2 * time.Second,
},
Handler: func(ctx context.Context, packet map[string]any) error {
out, err := nodes.CreateUserAccount(ctx, flowdsl.NodeInput{Packet: packet})
if err != nil { return err }
return bus.Publish(ctx, "flow.user_registration.send_verification", out.Packet("out"))
},
})
// Edge: CreateUserAccount → SendVerificationEmail [durable, retry x3]
bus.Subscribe(flowdsl.DurableConsumer{
Topic: "flow.user_registration.send_verification",
GroupID: "flow.user_registration",
RetryPolicy: flowdsl.RetryPolicy{MaxAttempts: 3, Backoff: flowdsl.ExponentialBackoff, InitialDelay: 2 * time.Second},
Handler: func(ctx context.Context, packet map[string]any) error {
out, err := nodes.SendVerificationEmail(ctx, flowdsl.NodeInput{Packet: packet})
if err != nil { return err }
return bus.Publish(ctx, "flow.user_registration.publish_registered", out.Packet("out"))
},
})
// Edge: SendVerificationEmail → PublishUserRegistered [stream, topic: users.registered]
bus.Subscribe(flowdsl.DurableConsumer{
Topic: "flow.user_registration.publish_registered",
GroupID: "flow.user_registration",
Handler: func(ctx context.Context, packet map[string]any) error {
return nodes.PublishUserRegistered(ctx, flowdsl.NodeInput{Packet: packet}, "users.registered")
},
})
}
What changes when you edit the diagram
| Change in Studio | Generated diff | Go code change needed |
|---|---|---|
Change direct → durable on an edge | bus.Subscribe(DurableConsumer{...}) added | None |
| Add retry policy | RetryPolicy{...} in consumer | None |
| Add idempotency key | IdempotencyKey: func(p) ... added | None |
| Change stream topic name | topic: "new-name" in consumer | None |
| Add new node | New field in XxxFlowNodes struct | Implement the handler (build fails until you do) |
| Remove a node | Field removed from struct | Remove the handler assignment (build warns) |
| Rewire edge from A→B to A→C | Consumer topic changes | None (if C handler already exists) |
Module Node Contract
Each module that contributes nodes to flows implements FlowNodeProvider:
// go-framework/modules/module.go (new interface)
// FlowNodeProvider is implemented by modules that register FlowDSL node handlers.
// The generated XxxFlowNodes struct is populated from these registrations.
type FlowNodeProvider interface {
// FlowNodes returns all operationId → handler mappings this module provides.
// Called at startup by flows.RegisterAll().
FlowNodes() map[string]flowdsl.AnyHandler
}
Module implementation pattern
backend/modules/community/
handler.go ← HTTP handlers (unchanged)
service.go ← business logic (unchanged)
flows.go ← NEW: FlowDSL node handler wrappers
same Service calls, different transport adapter
// backend/modules/community/flows.go
// FlowNodes implements modules.FlowNodeProvider.
func (m *Module) FlowNodes() map[string]flowdsl.AnyHandler {
return map[string]flowdsl.AnyHandler{
"receive_registration": flowdsl.Source(m.flowReceiveRegistration),
"validate_registration_input": flowdsl.Router(m.flowValidateRegistration),
"route_registration_by_type": flowdsl.Router(m.flowRouteByAccountType),
"create_user_account": flowdsl.Node(m.flowCreateUserAccount),
"send_verification_email": flowdsl.Node(m.flowSendVerificationEmail),
"publish_user_registered": flowdsl.Publish(m.flowPublishUserRegistered),
"reject_invalid_registration": flowdsl.Terminal(m.flowRejectInvalidRegistration),
}
}
// flowCreateUserAccount wraps m.Service.Register — same logic as the HTTP handler,
// different input adapter (NodeInput instead of http.Request).
func (m *Module) flowCreateUserAccount(ctx context.Context, in flowdsl.NodeInput) (flowdsl.NodeOutput, error) {
var req CommunityRegisterInput
if err := in.Decode(&req); err != nil {
return flowdsl.NodeOutput{}, err
}
member, user, err := m.Service.Register(ctx, &req)
if err != nil {
return flowdsl.NodeOutput{}, err
}
return flowdsl.Out("out", map[string]any{
"id": member.GetID().Hex(),
"email": user.Email,
"account_type": string(member.AccountType),
}), nil
}
The pattern: HTTP handlers and FlowDSL node handlers share the same Service layer. Only the transport adapter differs — httputil.ReadJSON(r, &input) becomes in.Decode(&input).
CI Pipeline
# .github/workflows/ci.yml
- name: Validate FlowDSL diagrams
run: go run ./cmd/flows/ validate
# Fails if any .flowdsl.yaml has invalid syntax or broken references
- name: Regenerate flow bridge code
run: go run ./cmd/generate/ flows
# Reads backend/flows/*.flowdsl.yaml → writes backend/flows/gen/*_flow_gen.go
- name: Check for uncommitted codegen drift
run: git diff --exit-code backend/flows/gen/
# Fails if .flowdsl.yaml was changed without regenerating
# This enforces: diagram edit → commit → regenerate → commit gen → PR
- name: Build
run: go build ./...
# Fails if any operationId in a generated XxxFlowNodes struct is unimplemented
- name: Test
run: go test ./...
The drift check is the key gate: if a developer edits a .flowdsl.yaml in Studio but forgets to regenerate, CI catches it before merge. The generated files are committed to git, making the diagram's impact on wiring fully visible in PR diffs.
Generator CLI: backend/cmd/flows/
go run ./cmd/flows/ validate Validates all .flowdsl.yaml files (spec correctness)
go run ./cmd/flows/ check Checks all operationIds have registered handlers
go run ./cmd/flows/ generate Regenerates backend/flows/gen/*_flow_gen.go
go run ./cmd/flows/ scaffold <flow> Generates handler stubs for unimplemented nodes in a flow
go run ./cmd/flows/ diff Shows what a diagram edit changed in human-readable form
go run ./cmd/flows/ push Uploads flows to Redelay cloud API
Startup Wiring
The generated RegisterAll function wires all flows using the node registry:
// backend/flows/gen/register_all_gen.go ← generated
package flows
func RegisterAll(bus flowdsl.EventBus, reg *modules.Registry) {
// Collect all node handlers from FlowNodeProvider modules
nodes := flowdsl.BuildNodeRegistry(reg)
RegisterUserRegistrationFlow(bus, UserRegistrationFlowNodes{
ReceiveRegistration: nodes.Require("receive_registration"),
ValidateRegistrationInput: nodes.RequireRouter("validate_registration_input"),
RouteRegistrationByType: nodes.RequireRouter("route_registration_by_type"),
CreateUserAccount: nodes.Require("create_user_account"),
SendVerificationEmail: nodes.Require("send_verification_email"),
PublishUserRegistered: nodes.RequirePublish("publish_user_registered"),
RejectInvalidRegistration: nodes.RequireTerminal("reject_invalid_registration"),
})
RegisterModuleSubmissionFlow(bus, ModuleSubmissionFlowNodes{ ... })
RegisterEmailNotificationFlow(bus, EmailNotificationFlowNodes{ ... })
// ...
}
In cmd/api/main.go:
func main() {
inst, srv, err := server.Default()
if err != nil { log.Fatal(err) }
// Wire all FlowDSL flows after module startup
if err := inst.AfterStart(func(ctx context.Context) error {
return flows.RegisterAll(inst.EventBus, inst.Modules)
}); err != nil {
log.Fatal(err)
}
if err := app.Run(inst, srv); err != nil { log.Fatal(err) }
}
Cloud Deployment Model
Today (Git-deployed)
edit YAML in Studio → export to backend/flows/ → commit → CI runs generator
→ go build → docker build → docker push → container restart picks up new wiring
Later (Cloud-managed flows)
Studio → PUT /api/flows/{id} → Redelay cloud stores YAML
→ cloud runs generator → cloud rebuilds and redeploys container automatically
→ OR: container watches /flows/ endpoint and triggers self-restart on change
The cloud layer owns:
- Kafka topic provisioning for
streamedges - MongoDB collection provisioning for
durableedges - Redis stream provisioning for
ephemeraledges - Auto-scaling workers proportional to queue depth per edge
What to Build (Ordered)
| # | Component | Location | Unblocks |
|---|---|---|---|
| 1 | FlowNodeProvider interface | go-framework/modules/module.go | Module node registration |
| 2 | flowdsl.NodeInput/Output/AnyHandler types | go-framework/flowdsl/node.go | Handler function signatures |
| 3 | flowdsl.NodeRegistry | go-framework/flowdsl/registry.go | Node discovery at startup |
| 4 | Flow YAML parser (FlowDSL.com format) | go-framework/flowdsl/spec/ | Generator reads flows |
| 5 | cmd/flows/ validate | backend/cmd/flows/ | CI gate on diagram validity |
| 6 | cmd/generate/ flows | backend/cmd/generate/ | Codegen from YAML |
| 7 | Generated XxxFlowNodes structs | backend/flows/gen/ (generated) | Compile-time enforcement |
| 8 | flows.go node wrappers per module | backend/modules/*/flows.go | Handlers wired to flows |
| 9 | flows.RegisterAll() | backend/flows/gen/ (generated) | Startup wiring |
| 10 | cmd/flows/ scaffold | backend/cmd/flows/ | Auto-stub for new nodes |
| 11 | Direct edge execution | go-framework/flowdsl/ | Simplest flow paths |
| 12 | Durable edge execution (go-events) | go-framework/flowdsl/ | Reliable business paths |
| 13 | Stream/Ephemeral edge execution | go-framework/flowdsl/ | Full delivery mode support |
| 14 | /api/flows/ module + Studio integration | go-framework/modules/flows/ | Cloud diagram management |