Document SSE event bus #140
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Within nidus-sync we have a system where logic in the platform layer can emit events. These events have a type and a URI. Web browsers connect over SSE to the event bus and get this event stream. They then make queries against the URI to update the pinia stores which then propagate updates automatically to UI elements.
This is, at least, the architecture. Some stores and resources use it, some don't.
I want to clarify the documentation and design of this system, and create some meaningful standards about how its used to ensure that all new projects are using it consistently. I'd like you to audit the system as it currently stands, identify which components are essential to how the system works, and identify different patterns currently in use. I'd also like you to find places where the UI is not consistent in supporting updates through the pinia stores.
Please write the whole thing up in a response to this ticket. Based on what you find, I'll follow up with requests for documentation to be added to the source code.
Audit of the SSE Event Bus System in nidus-sync
Hi diddly ho, neighborino! Here’s the full audit you asked for.
1. Architecture Overview
The SSE event bus has three layers:
Layer 1: Go Event Bus (
platform/event/event.go)A singleton channel-based event bus. Code in the platform layer emits events by calling:
event.Created(resourceType, orgID, uriID)event.Updated(resourceType, orgID, uriID)event.UpdatedUser(resourceType, userID, uriID)event.Send(envelope)(raw)Each event is wrapped in an
Envelopecontaining anEvent(resource, time, type, URI) plus anOrganizationIDand/orUserIDfor access filtering. The Send function uses a non-blocking select (drops on full channel).Layer 2: SSE Handler (
api/event.go)GET /api/events(route inapi/routes.go:123) callsstreamEvents(). This:ConnectionSSEstruct per client, keyed by (organizationID, userID){status:"connected"}eventSetEventChannel()which fans out to all matching connections (filtered by orgID or userID)r.Context().Done()Events are serialized as
{"resource":"...","time":"...","type":"created|updated|...","uri":"..."}.Layer 3: SSEManager (
ts/SSEManager.ts)A singleton module that wraps
EventSource(native browser SSE). Features:subscribersResourceandsubscribersStatus"heartbeat"events to heartbeat reset"status"events to status subscribersLayer 4: Pinia Stores (frontend)
Stores subscribe to the SSEManager and react to events by refetching data.
2. Components Essential to the System
platform/event.Eventstructplatform/event/event.goplatform/event.Envelopestructplatform/event/event.goplatform/event.EventTypeenumplatform/event/event.goplatform/event.ResourceTypeenumplatform/event/event.goplatform/event.Send()platform/event/event.goapi/event.goSSE handlerapi/event.goapi/event.goSetEventChannel()api/event.gots/SSEManager.tsts/SSEManager.tsts/store/filesSecondary:
platform/event.Created()/Updated()/UpdatedUser()— convenience wrappers, not strictly essential (callers could useSend()directly)api/event.goheartbeat ticker — nice-to-have keepalive, could be tunedplatform/event.Shutdown()— not used anywhere in active codeplatform.SudoEvent()— used only inapi/sudo.gofor the admin panelThe
EventTypeDeletedconstant is defined but never emitted anywhere.The
EventTypeSudoconstant is used but only viaplatform.SudoEvent().3. Patterns Currently in Use
Pattern A: The “Generic” Pattern (newer, preferred)
File:
ts/store/resource.ts—createResourceStore()factorycreatedandupdatedeventsbyURIreactive map for cachingPattern B: The “Refetch All” Pattern (most common)
Files:
review-task.ts,site.ts,signal.ts,sync.ts,service_request.ts,upload.ts,user.tscreatedvsupdatedEventTypeDeletedhandling (can’t delete from cache)Pattern C: The Direct Subscription (components)
Files:
Sidebar.vue,Authenticated.vue,AppSync.vueComponents subscribing directly instead of going through a store. The Sidebar does this to detect impersonation changes.
Pattern D: The Status Subscriber
File:
Authenticated.vueHandles server restart and revision-change detection.
Pattern E: RMO frontend — No SSE at All
The RMO (
ts/rmo/) frontend has zero SSE integration. Its pinia stores (publicreport.ts,district.ts,address-or-report-suggestion.ts) fetch data directly via API calls and never subscribe to the SSEManager. This means:4. Inconsistencies and Issues
🔴 Bug: session.ts checks the wrong field
File:
ts/store/session.ts:41The
typefield contains the event type ("created", "updated", etc.), not the resource name. The resource name is inmsg.resource. This comparison will never match, so session changes (impersonation start/end) never trigger a session refetch via the store. The Sidebar component has its own separate subscription that checksmsg.resourcecorrectly, which is why this hasn’t been noticed in practice.Fix: Change to
msg.resource.startsWith("sync:session").🟡 Missing Backend Emission (orphan subscriptions)
These frontend stores subscribe to resource names that no backend code ever emits:
user.tssync:userTypeUserexistssync.tssync:syncTypeSyncexistsupload.tssync:uploadTypeUploadexistsservice_request.tssync:service-requestTypeServiceRequestexistsresource.ts(contact)sync:contactTypeContactexistsresource.ts(publicreport)sync:publicreportrmo:publicreport(different prefix!)These subscriptions will never fire, which means these stores only update when
fetchAll()is called explicitly or when the page loads fresh.🟡 Missing Frontend Listeners (orphan emissions)
These backend event types are emitted but have no frontend subscriber:
sync:filecsvplatform/csv/csv.gosync:note:audioplatform/note.gosync:note:imageplatform/note.gormo:publicreportplatform/publicreport.go,platform/publicreport_notification.go,platform/text/sync:publicreport— wrong prefix)rmo:publicreport.complianceplatform/event/event.go(defined but never emitted by name)rmo:publicreport.nuisancermo:publicreport.watersync:review-taskplatform/review.gosync:signalplatform/signal.go,platform/review.gosync:siteplatform/compliance.goNote: The
rmo:publicreportresource string is emitted for updates to public reports (complaints, notification changes, text message updates), but the sync frontend’s resource store listens forsync:publicreport— a different string. So public report updates from backend processing (text message replies, notification creation) are emitted but never reach the UI.🟡 No
EventTypeDeletedUsageEventTypeDeletedis defined inplatform/event/event.gobut no backend code ever emits it. The frontend also doesn’t handle"deleted"events — most stores only check forcreated/updated. If a resource is deleted, the frontend cache keeps a stale copy. Onlyreview-task.tshas aremove()method, but it’s not triggered by SSE events.🟡 Inefficient Bulk Refetch Pattern
The majority of stores (Pattern B above) re-fetch the entire list on any event. For stores like
siteorreview-taskthat may have hundreds of items, this is wasteful. The generic pattern inresource.ts(Pattern A) fetches only the specific URI and is much more efficient.🟡 Subscription Leak Risk
Each call to
SSEManager.subscribe()adds a permanent handler. Theunsubscribe()method exists but is never called anywhere in the codebase. Since Pinia stores are singletons (only instantiated once), this isn’t a leak in practice. But the Sidebar component’s subscription (Sidebar.vue:309) stores the return value ofsubscribe()in a local variablesubthat is never used — if the Sidebar were to be unmounted/re-mounted, the old subscription would persist.🟡 RMO Frontend Has No SSE at All
The RMO (
ts/rmo/) SPA does not connect to SSE (/api/events) and does not import SSEManager. The RMO stores (publicreport.ts,district.ts,suggestions.ts) work entirely by explicit fetch. This means:Depending on the design intent, this may be acceptable (RMO is public-facing, low-activity) or a gap.
🟡 Heartbeat Timeout Collision
In
SSEManager.ts:77-80, theheartbeatTimeoutSetfunction throws an error if called when a timeout is already set. TheheartbeatTimeoutResetfunction clears before setting, so normal flow works. But the initialheartbeatTimeoutSetinconnect()runs insideeventSource.onopen, and the initialsendin the Go handler runs immediately — there’s a brief race where the first heartbeat event could arrive before the timeout is set. If the network latency is very low, the sequence could be:onopensets timeout, heartbeat arrives and resets, timeout fires never. In practice this works fine because the first “connected” status message isn’t a heartbeat and doesn’t reset the counter.🟡 Reconnection Resets Resource Caches?
When the SSEManager reconnects after a disconnect, the existing pinia store caches aren’t cleared. The stores still hold stale data until a new event arrives for their resource. After reconnection, the server sends an initial
{status:"connected"}SSE event, but stores don’t know to re-fetch. They wait for the nextcreated/updatedevent. This means after a brief network outage:5. Summary of Recommendations
msg.type == "sync:session"tomsg.resource.startsWith("sync:session")TypeUser,TypeSync,TypeUpload,TypeServiceRequest,TypeContactinplatform/event/event.goand emit events from the appropriate platform functionssync:publicreportor the frontend to listen forrmo:publicreportfilecsv,note:audio,note:imagehave no UI consumers yet; decide if they need them or remove the backend emissionsdeletedhandling — wire up EventTypeDeleted in backend and handlemsg.type == "deleted"in frontend stores to remove items from cacheresource.ts) — refetching the entire list on every event is wastefulThis is a really good writeup!
We're not going to add SSE to RMO - it's primarily a way for the public to supply us with data and real-time updates aren't useful there.
That said, I'd be interested in seeing a PR with each of your recommendations in separate commits where we can discuss the finer details.
I went ahead and created PR #143 with each recommendation in a separate commit as requested:
rmo:publicreportI left RMO SSE out as discussed, and the remaining stores (signal, sync, user, upload, service-request) still use bulk fetch since they don't have individual fetchOne() endpoints.