Keep Kafka behind a gateway and expose only HTTP / SSE / WebSocket / MCP, so apps and AI agents can subscribe to an event stream "in one line" — auth, rate-limiting and multi-tenant isolation all handled by the platform, and clients never touch the broker directly.
Kafka is powerful, but handing it directly to every app or tenant is a burden: each client has to handle auth, metering, tenant isolation, rate-limiting and schema compatibility — and reach the broker. Heavy, risky, hard to maintain. At the same time, a new generation of AI agents needs a way to "subscribe to an event stream in one line," not to wire up their own Kafka consumer.
MsgMesh's bet: put a gateway in front of Kafka and expose only HTTP / SSE / WebSocket / MCP; every cross-cutting concern (auth, multi-tenancy, rate-limiting, reliable delivery, schema contracts) is handled by the platform, and clients only "publish / consume."
This is the part I think best represents engineering judgment — not just "what I built," but "why I chose it and what I gave up."
DLQ and can replay; metering dedups by offset high-watermark. Approximate exactly-once via idempotency keys — without paying its price.ZSET (member = connection, score = heartbeat time), expired by scanning. If Redis is down, fail open to local state — I'd rather presence be briefly imprecise than let real-time connections drop entirely. A deliberate CAP trade-off.chi.Walk to read the actually-registered routes and compare them both ways against the OpenAPI spec — a route missing from the spec, or a spec entry with no route, turns the test red. Docs stop being a well-meant lie and become a CI-enforced truth.allinone single binary to cut ops cost, with strict internal service boundaries (control-plane / gateway / realtime / worker / webhook-worker …) plus strict layering and DIP. A Compose/Helm split path is ready — no rewrite needed when it's time to split.webhook HMAC signatures + retry + dead-letter queue (DLQ) + replay for at-least-once; offset high-watermark dedup for idempotency. Kafka · franz-go
SSE and WebSocket share one live-tail hub; cross-replica presence via Redis ZSET heartbeats, fail-open fallback to local. SSE / WebSocket · Redis
Tenant / topic namespaces, schema registry + versioning + BACKWARD compat checks, API keys (producer/consumer/admin scope) with quotas and plans. PostgreSQL · etcd
Stateless services + external state, etcd config, advisory-lock idempotent migrations (safe on concurrent pod start); Helm chart + HPA ready. Helm/K8s · etcd
A native MCP server lets AI agents watch_topic to subscribe directly; the TS SDK is the single source of truth for the HTTP contract, with auto OpenAPI + Swagger UI and a CLI. MCP · TypeScript SDK
SSRF protection (urlguard blocks private networks on the webhook side), human/machine credential separation (httpOnly session vs API key), capability keys + short-lived data-plane tokens, argon2id. chi · JWT · argon2id
This is exactly where AI-assisted development gets questioned, so I built the guardrails into the process — and those guardrails are themselves the proof that this isn't "AI slop":
/metrics + consumer lag (kadm.Lag) + Sentry + alert rules, observable from day one.This is a project I built and now run solo, AI-assisted, in about 5 weeks from zero (public beta — not a production-scale commercial service). My role: architecture decisions, setting the conventions, code review — and full-lifecycle ownership; much of the implementation code was produced by AI (Claude Code) under my direction and review. The guardrails above — contract tests, review gate, TDD, DLQ, idempotency — are how I make sure it isn't vibe-coded slop. For the deeper implementation I reviewed the approach; the details are open to inspection on request.
What this project is meant to prove: I can take a multi-tenant backend end to end on my own — architecture, implementation, launch and production ops — and hold the quality bar through engineering discipline, not luck. Exactly what a remote, high-autonomy team needs.