Guides

Creating a Go Module

Step-by-step guide to creating a new module for the redelay go-framework.

Creating a Go Module

This guide walks through creating a new module for the Redelay Go framework.

Quick start: If you have a module.yaml, you can scaffold the entire module in one command:

shell

redelayctl validate-module module.yaml  # check for errors first
redelayctl scaffold module.yaml ./modules/mymodule/

This generates module.go, plus routes.go when the YAML declares routes and handlers.go when it declares CRUD operations. That split predates the layout convention below — fold Routes() into module.go and rename handlers.go to handler.go as you flesh the module out. See Module Definition Files for the full tooling pipeline.

Directory Structure

Every module lives under modules/<name>/ (or <name>/ in go-modules/). The canonical layout splits each concern into its own file. Only create files that have content — empty stubs are noise.

text

mymodule/
├── module.go     # factory + Module struct + lifecycle + Routes()
├── ir.go         # //go:embed module.yaml
├── config.go     # Config struct + DefaultConfig() (env vars)
├── schemas.go    # HTTP request/response DTOs
├── model.go      # domain types persisted to DB
├── service.go    # business logic
├── crud.go       # Mongo / ClickHouse persistence (Store)
├── handler.go    # HTTP handlers (the API layer)
├── consumers.go  # event subscriptions (Consumers() + handler funcs)
├── events.go     # typed.EventDefinition vars + payload structs published here
├── lookup.go     # cross-module adapter (optional)
├── module.yaml   # module manifest (events, packets, nodes, settings)
├── flowdsl/      # FlowDSL nodes companion module (optional)
└── admin/        # admin-only submodule (optional)

Why split this way?module.go becomes a tiny "wiring" file (factory + struct

lifecycle + Routes()). Everything else follows the single-responsibility principle: HTTP shape lives in schemas.go, persistence in crud.go, business logic in service.go, event subscription in consumers.go, event publishing in events.go. New contributors find what they need without grepping.

File responsibilities:

File	Contents	Notes
`module.go`	`init()` factory, `Module` struct, `Startup`/`Shutdown`, `Routes()`, `Configure()`	Keep this file thin — wiring only
`ir.go`	`//go:embed module.yaml` + `var moduleYAML []byte`	One-liner — separate so other files don't need `embed`
`config.go`	`Config` struct, `DefaultConfig()` reads all env vars	Single source of truth for env-var names
`schemas.go`	Request/response DTOs (e.g. `UserCreateInput`, `UserResponse`)	Public API surface; visible in OpenAPI spec
`model.go`	DB-persisted types with `bson` tags	Internal; never returned directly from handlers
`service.go`	Business logic methods on `*Service`	Calls into `Store`, transforms model→response
`crud.go`	`Store` struct + Mongo/CH persistence	Single place for DB queries; mockable via `StoreIface`
`handler.go`	`func (m *Module) handleX(w, r)`	HTTP-only concerns: parse body → service → write JSON
`consumers.go`	`Consumers()` + handler funcs that read `transport.EventMessage`	Each handler unmarshals payload, calls service
`events.go`	`typed.EventDefinition` vars + `publishX()` helpers + payload structs	All "this module publishes…" lives here
`lookup.go`	Adapters consumed by other modules (e.g. `users.LookupProvider`)	Optional; only when another module discovers via type assertion
`module.yaml`	Manifest read at build/spec time	Drives modspec browser, settings UI, AsyncAPI

Not every file is required. A pure-event worker has no handler.go or schemas.go. A read-only adapter has no crud.go or events.go. Start with module.go + ir.go + module.yaml; add files as the module grows.

Submodules (`flowdsl/`, `admin/`)

flowdsl/ and admin/ subdirectories are independent modules — each has its own factory, module.yaml, and lifecycle. They follow the same file split as the parent module:

text

mymodule/
├── module.go            # parent module
├── …
├── flowdsl/
│   ├── module.go        # FlowDSL nodes companion module
│   ├── ir.go
│   └── module.yaml
└── admin/
    ├── module.go        # admin routes mounted under /admin
    ├── ir.go
    ├── schemas.go
    ├── handler.go
    └── module.yaml

A bare flowdsl/ that only registers nodes typically needs only module.go + ir.go + module.yaml. An admin/ with several CRUD endpoints follows the full canonical split. Do not invent empty files — the rule is one concern per file, not every concern always present.

Step 1: Create `config.go`

Define a Config struct and a DefaultConfig() function that reads environment variables with a module-specific prefix.

package mymodule

import "github.com/redelay/go-framework/config"

type Config struct {
    Collection string `json:"collection" yaml:"collection"`
    SomeOption int    `json:"some_option" yaml:"some_option"`
}

func DefaultConfig() *Config {
    return &Config{
        Collection: config.GetEnv("MYMODULE_COLLECTION", "my_items"),
        SomeOption: config.GetInt("MYMODULE_SOME_OPTION", 42),
    }
}

Convention: Use the <MODULE_NAME>_ prefix for all environment variables.

Step 2: Create `model.go`

Define MongoDB document structs. Embed crud.BaseModel for automatic ID, timestamps, and soft-delete support.

package mymodule

import "github.com/redelay/go-framework/crud"

type Item struct {
    crud.BaseModel `bson:",inline"`
    Name           string `bson:"name" json:"name"`
    Description    string `bson:"description,omitempty" json:"description,omitempty"`
}

Step 3: Create `service.go`

Encapsulate all business logic in a Service struct backed by crud.MongoCRUD[T].

package mymodule

import (
    "context"

    "github.com/redelay/go-framework/crud"
    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/mongo"
    "go.uber.org/zap"
)

type Service struct {
    cfg    *Config
    crud   *crud.MongoCRUD[*Item]
    logger *zap.Logger
}

func NewService(cfg *Config, col *mongo.Collection, logger *zap.Logger) *Service {
    return &Service{
        cfg:    cfg,
        crud:   crud.NewMongoCRUD[*Item](col),
        logger: logger.Named("mymodule"),
    }
}

func (s *Service) Create(ctx context.Context, name string) (*Item, error) {
    return s.crud.Create(ctx, &Item{Name: name})
}

func (s *Service) EnsureIndexes(ctx context.Context) error {
    return s.crud.EnsureIndexes(ctx,
        mongo.IndexModel{Keys: bson.D{{Key: "name", Value: 1}}},
    )
}

Step 4: Create `module.go`

This is the most important file. It:

Defines the Module struct implementing modules.Module
Registers via init() for autodiscovery

package mymodule

import (
    "context"

    "github.com/redelay/go-framework/modules"
    "go.mongodb.org/mongo-driver/mongo"
    "go.uber.org/zap"
)

func init() {
    modules.RegisterFactory("mymodule", func(deps modules.ModuleDeps) (modules.Module, error) {
        cfg := DefaultConfig()
        return NewModule(deps.DB, cfg, deps.Logger), nil
    })
}

type Module struct {
    Service *Service
    db      *mongo.Database
    cfg     *Config
    logger  *zap.Logger
}

func NewModule(db *mongo.Database, cfg *Config, logger *zap.Logger) *Module {
    return &Module{db: db, cfg: cfg, logger: logger}
}

func (m *Module) Manifest() *modules.Manifest {
    return &modules.Manifest{
        Name:        "mymodule",
        Description: "My custom module",
        Version:     "1.0.0",
        Provides:    []string{"mymodule"},
    }
}

func (m *Module) Startup(ctx context.Context) error {
    col := m.db.Collection(m.cfg.Collection)
    m.Service = NewService(m.cfg, col, m.logger)
    return m.Service.EnsureIndexes(ctx)
}

func (m *Module) Shutdown(ctx context.Context) error {
    return nil
}

Step 5: Wire It Up

In your application, add a blank import of your module's package (your app's module path, not the framework's) to trigger the init() registration:

import (
    _ "github.com/yourorg/myapp/modules/mymodule"
)

app := app.New("myapp", logger)
app.DiscoverAndRegister(modules.ModuleDeps{DB: db, Logger: logger})
app.Start(ctx)

Optional Interfaces

Modules can implement additional interfaces to participate in framework features:

Interface	Purpose
`RoutesProvider`	Register HTTP API routes via `Routes(router Router)`
`EventsProvider`	Define events and consumers
`CRUDProvider`	Declare CRUD capabilities
`MCPProvider`	Expose MCP tools and resources
`CLIProvider`	Contribute CLI commands (auto-exposed via MCP)
`ConfigProvider`	Runtime configuration definitions
`SettingsProvider`	Admin-UI-editable settings
`SchemasProvider`	Publish Go types into `/openapi.json#/components/schemas` without owning an HTTP route — needed when FlowDSL nodes reference types via `$ref`. See section below.
`FlowDSLProvider`	Contribute FlowDSL workflow fragments
`TemplateProvider`	Contribute starter flow documents to the Studio's "Create flow" picker
`DynamicRoutesProvider`	Mount HTTP routes at runtime (typically the flowexec dispatcher for flow-driven endpoints)
`DynamicChannelsProvider`	Announce event channels at runtime (flowexec for subscriber deployments)
`MigrationsProvider`	Database migration scripts
`Configurable`	Resolve cross-module dependencies at startup

Example: Adding HTTP Routes

func (m *Module) Routes(router modules.Router) {
    router.Group("/items", func(r modules.Router) {
        r.Handle("GET", "", m.handleList)
        r.Handle("POST", "", m.handleCreate)
        r.Handle("GET", "/{id}", m.handleGet)
        r.Handle("PUT", "/{id}", m.handleUpdate)
        r.Handle("DELETE", "/{id}", m.handleDelete)
    })
}

The server mounts module routes under ROUTE_PREFIX (default /api/v1), so this serves /api/v1/items — don't repeat the prefix in the group path.

Routes registered this way are automatically collected by the OpenAPI spec generator and exposed via MCP.

Example: Publishing Schemas Without Routes (`SchemasProvider`)

When a module ships FlowDSL nodes whose port specs reference Go types ($ref: "openapi:default#/components/schemas/UserCreateInput"), the OpenAPI spec generator needs to find those types in components/schemas. The reflection registry only sees types declared on actual HTTP routes (Body(...) / Response(...)) — types used ONLY by FlowDSL nodes need a separate publishing path.

Implement SchemasProvider:

// OpenAPISchemas publishes Go types into /openapi.json's
// components/schemas. Each entry is a pointer to a zero-value struct;
// the openapi module reflects them via its existing schema registry.
func (m *Module) OpenAPISchemas() []any {
    return []any{
        &UserCreateInput{},
        &UserUpdateInput{},
        &UserResponse{},
        &UserListResponse{},
    }
}

Without this, a flow-driven endpoint whose packet refs UserCreateInput would render "Schema not found in loaded specs" in Studio and 404 on the spec lookup. Modules that already declare every type via Go HTTP handlers don't need to duplicate them here — only types referenced by FlowDSL nodes / packets / dynamic routes that have no matching Body() reflection path do.

File-split rule for wire shapes. Request/response/conflict types do NOT live in handler.go. They go in schemas.go even when only one handler reads them — keeps the handler file focused on routing and business logic, and makes the wire surface easy to scan in one file. The same rule applies to admin sub-modules (module/admin/schemas.go) and flowdsl companion modules.

Example: Exposing MCP Tools (`MCPProvider`)

Implement MCPProvider to expose your module's capabilities as tools for AI assistants via the MCP server at /mcp. This lets Claude, Copilot, Cursor, and other MCP clients discover and invoke your module's operations.

func (m *Module) MCPTools() []modules.MCPTool {
    return []modules.MCPTool{
        {
            Name:        "search_items",
            Description: "Search items by name or category",
            InputSchema: map[string]any{
                "type":     "object",
                "required": []string{"query"},
                "properties": map[string]any{
                    "query": map[string]any{
                        "type":        "string",
                        "description": "Search term",
                    },
                },
            },
            Handler: func(ctx context.Context, args map[string]any) (any, error) {
                query, _ := args["query"].(string)
                return m.service.Search(ctx, query)
            },
        },
    }
}

func (m *Module) MCPResources() []modules.MCPResource { return nil }

The MCP server auto-discovers your tools at startup — no extra wiring needed. Use InputSchema to define a JSON Schema for the tool's parameters so AI clients understand what arguments to provide.

Tip: You can also expose MCPResource entries for static data (e.g. a config summary or schema document) that AI assistants can read without invoking a tool.

Example: Publishing and Consuming Events (`EventsProvider`)

Implement EventsProvider when your module publishes or consumes events. This serves two purposes:

IR metadata — Events() populates the modspec browser and AsyncAPI spec
Runtime wiring — Consumers() registers handlers before bus.Start()

First, define a typed.EventDefinition (can be in events.go):

package mymodule

import "github.com/redelay/go-events/typed"

type ItemCreatedPayload struct {
    ItemID string `json:"item_id"`
    Name   string `json:"name"`
}

var ItemCreatedEvent = typed.EventDefinition[ItemCreatedPayload]{
    Name:       "item.created",
    EntityType: "item",
    Action:     "created",
    Topic:      "item.created",
}

Then implement EventsProvider in module.go. The module also needs access to the EventBus from ModuleDeps:

func init() {
    modules.RegisterFactory("mymodule", func(deps modules.ModuleDeps) (modules.Module, error) {
        cfg := DefaultConfig()
        return NewModule(deps.DB, cfg, deps.Logger, deps.EventBus), nil
    })
}

type Module struct {
    Service  *Service
    db       *mongo.Database
    cfg      *Config
    logger   *zap.Logger
    eventBus modules.EventBus
}

func NewModule(db *mongo.Database, cfg *Config, logger *zap.Logger, bus modules.EventBus) *Module {
    return &Module{db: db, cfg: cfg, logger: logger, eventBus: bus}
}

// Events returns IR metadata for modspec and AsyncAPI generation.
func (m *Module) Events() []*ir.Event {
    return []*ir.Event{
        {Name: "item.created", EntityType: "item", Action: "created"},
    }
}

// Consumers returns runtime subscriptions — wired before bus.Start().
func (m *Module) Consumers() []*modules.ConsumerRegistration {
    return []*modules.ConsumerRegistration{
        {
            EventName: "order.completed",
            Topic:     "order.completed",
            GroupID:   "mymodule",
            Handler:   m.handleOrderCompleted,
        },
    }
}

func (m *Module) handleOrderCompleted(ctx context.Context, msg *modules.EventMessage) error {
    var p OrderCompletedPayload
    if err := json.Unmarshal(msg.Payload, &p); err != nil {
        return err
    }
    // ... business logic
    return nil
}

Publish an event from a service method. Give the Service the bus — add an eventBus field and pass m.eventBus through from Startup:

type Service struct {
    cfg      *Config
    crud     *crud.MongoCRUD[*Item]
    eventBus modules.EventBus
    logger   *zap.Logger
}

func NewService(cfg *Config, col *mongo.Collection, bus modules.EventBus, logger *zap.Logger) *Service {
    return &Service{cfg: cfg, crud: crud.NewMongoCRUD[*Item](col), eventBus: bus, logger: logger.Named("mymodule")}
}

// in (m *Module) Startup:
//     m.Service = NewService(m.cfg, col, m.eventBus, m.logger)

func (s *Service) Create(ctx context.Context, name string) (*Item, error) {
    item, err := s.crud.Create(ctx, &Item{Name: name})
    if err != nil {
        return nil, err
    }
    if s.eventBus != nil {
        msg, _ := ItemCreatedEvent.NewMessage(
            item.ID.Hex(),
            typed.SystemActor,
            ItemCreatedPayload{ItemID: item.ID.Hex(), Name: item.Name},
        )
        _ = s.eventBus.Publish(ctx, msg)
    }
    return item, nil
}

Guard eventBus != nil when EventBus is optional for your module. Apps without an event transport configured will pass nil — the guard makes the module work in both modes.

Example: Cross-Module Dependencies (`Configurable`)

If your module depends on another module (e.g., auth needs users), implement Configurable:

func (m *Module) Configure(registry *modules.Registry) error {
    usersModule := registry.Get("users")
    if usersModule == nil {
        return fmt.Errorf("mymodule: requires 'users' module")
    }
    // Type-assert to access the concrete module
    return nil
}

Use DependsOn in the manifest to ensure correct startup order:

func (m *Module) Manifest() *modules.Manifest {
    return &modules.Manifest{
        Name:      "mymodule",
        DependsOn: []string{"users"},
    }
}

Example: Contributing CLI Commands (`CLIProvider`)

Implement CLIProvider to contribute commands to the project CLI. Commands are automatically exposed as MCP tools (prefixed cli_) so AI assistants can invoke them too.

func (m *Module) CLICommands() []modules.CLICommand {
    return []modules.CLICommand{
        {
            Name:        "create-item",
            Description: "Create a new item from the command line",
            Args: []modules.CLIArg{
                {Name: "name", Description: "Item name", Required: true},
                {Name: "category", Description: "Item category", Required: false, Default: "general"},
                {Name: "published", Description: "Mark as published", Flag: true, Default: "false"},
            },
            Handler: m.cliCreateItem,
        },
    }
}

func (m *Module) cliCreateItem(ctx context.Context, args map[string]string) error {
    // args["category"] and args["published"] are read the same way.
    item, err := m.Service.Create(ctx, args["name"])
    if err != nil {
        return err
    }
    fmt.Printf("Created item: %s (%s)\n", item.Name, item.ID.Hex())
    return nil
}

The project needs a cmd/redelayctl/main.go entry point that blank-imports the same modules as the API and calls app.RunCLI():

package main

import (
    "os"

    _ "github.com/redelay/go-framework/modules/auth"
    _ "github.com/redelay/go-framework/modules/users"
    // ... other modules
    "github.com/redelay/go-framework/app"
)

func main() {
    if err := app.RunCLI(); err != nil {
        os.Exit(1)
    }
}

Usage:

shell

go run ./cmd/redelayctl create-item --name="Widget" --category=tools --published
go run ./cmd/redelayctl help  # list all available commands

Example: Contributing Flow Templates (`TemplateProvider`)

Any module can ship one or more starter flows that show up in the FlowDSL Studio's "Create flow" → "From template" picker. The flowexec module walks the registry at request time, so all you need is to implement one method — no extra wiring, no migrations, no HTTP routes.

import (
    "encoding/json"
    _ "embed"

    "github.com/redelay/go-flowdsl/flowexec"
    "github.com/redelay/go-flowdsl/ir"
)

//go:embed flows/welcome-email.flowdsl.json
var welcomeEmailJSON []byte

func (m *Module) Templates() []flowexec.FlowTemplate {
    var doc ir.Workflow
    if err := json.Unmarshal(welcomeEmailJSON, &doc); err != nil {
        m.logger.Warn("malformed embedded template", zap.Error(err))
        return nil
    }
    return []flowexec.FlowTemplate{{
        // Conventionally `<module>/<slug>` so collisions across
        // modules are obvious. First-wins on duplicates so a project
        // module can override a framework template by reusing the ID.
        ID:          "mymodule/welcome-email",
        Name:        "User signup → welcome email",
        Description: "Listens for users.user_created and sends a templated welcome email.",
        Source:      "mymodule",          // your module name — surfaces as a chip
        Tags:        []string{"events", "email", "onboarding"},
        Document:    &doc,                // full IR workflow — spec form is converted on-demand
    }}
}

The Document field is an *ir.Workflow — same shape Redelay uses internally for every flow. Author it as a JSON file, embed it via //go:embed, and unmarshal lazily. Keeping the document inline (not in MongoDB) means templates are zero-IO to list and survive without a database.

Authoring the JSON. Use the same structure the assistant module ships (see backend/modules/assistant/flows/default.flowdsl.json):

json

{
  "id": "welcome-email",
  "name": "User signup → welcome email",
  "description": "...",
  "nodes": [
    {"id": "src",  "name": "On users.user_created", "kind": "start",
     "action_ref": "redelay/event-source",
     "config": {"eventName": "users.user_created", "groupID": "welcome-email"}},
    {"id": "send", "name": "Send Templated Email", "kind": "action",
     "action_ref": "redelay/email-send-templated"},
    {"id": "end",  "name": "Sent",                  "kind": "end"}
  ],
  "edges": [
    {"id": "e1", "from": "src",  "to": "send", "delivery_mode": "durable"},
    {"id": "e2", "from": "send", "to": "end",  "delivery_mode": "direct"}
  ],
  "meta": {
    "positions": {
      "src":  {"x":   0, "y": 0},
      "send": {"x": 360, "y": 0},
      "end":  {"x": 720, "y": 0}
    }
  }
}

Rules of thumb:

Every action_ref must match an id: from a real flowdsl_nodes entry. Open /modules after boot to see what's registered.
Every config key must match the node's settings_schema.properties — unknown keys are accepted by the JSON parser but ignored at runtime.
Drop the optional packet field on edges unless the value's $ref exactly matches one of the connected port schemas. Mismatches surface as validation warnings in the Studio canvas.
Add meta.positions for every node so the canvas renders something sensible — without positions, every node stacks on the origin.

The Studio fetches the lightweight metadata (id, name, description, source, tags) up-front for the picker grid; the full document is only fetched when the user clicks Create from template. Keep individual templates small and focused — five tight starters beat one sprawling kitchen-sink flow.

For a full library example, see go-modules/flowtemplates — five production-shaped starters covering events→DB, scheduled API polling, DB→LLM→alert, welcome email, and failed-login burst alerts.

Checklist

module.yaml — Module definition (optional but recommended for validation + diagrams)
config.go — Config struct + DefaultConfig() with env prefix
model.go — MongoDB models with crud.BaseModel
service.go — Business logic + EnsureIndexes()
module.go — init() with RegisterFactory + Module implementing modules.Module
Blank import in your app's main package
DependsOn set if you rely on other modules
If using events: events.go with typed.EventDefinition, implement EventsProvider
If contributing CLI commands: implement CLIProvider, create cmd/redelayctl/main.go entry point
If shipping starter flows: implement TemplateProvider, embed flows/*.json (see reference)
redelayctl validate-module module.yaml — passes with no errors

ClickHouse modules

For analytics-oriented modules, use crud.ClickHouseCRUD[T] instead of MongoCRUD. Models implement ClickHouseModel (requires TableName() string) and use ch struct tags for column mapping:

type PageView struct {
    Timestamp time.Time `ch:"timestamp"`
    Domain    string    `ch:"domain"`
    Count     uint64    `ch:"count"`
}

func (p *PageView) TableName() string { return "page_views" }

For high-throughput writes, wrap with a batch writer:

writer := crud.BatchWriterFor(chCrud,
    crud.WithBatchSize(500),
    crud.WithFlushInterval(5 * time.Second),
)
writer.Enqueue(&PageView{...}) // non-blocking
defer writer.Stop()            // drains remaining items

See the go-framework reference for the full ClickHouse CRUD and batch writer API.

Edit this pageorReport an issue

Deploying to production

Ship a Redelay Go backend to Docker Swarm — a Mode B image built with SSH-forwarded private modules, a swarm stack that survives node reboots, and a push-to-deploy GitHub Actions workflow that fails on an automatic rollback.

Adding Events

How to publish events, consume events, and test event-driven behaviour in a Redelay Go module.