doc: vision worker auto-scaling design and implementation plan #127

Merged

eliribble merged 1 commit from feature/vision-worker-auto-scale into main

2026-07-16 21:27:12 +00:00

ned commented

2026-07-16 21:16:31 +00:00

Member

This PR addresses issue #126 — researching and documenting what it would take to have the system automatically spin up and spin down GPU instances on Digital Ocean.

What’s Included

`doc/nidus-reveal/vision-worker/design.md`

Design document covering:

Provider abstraction interface for multi-cloud flexibility
godo-based Digital Ocean driver for GPU droplet provisioning
Full droplet lifecycle state machine (provision → deploy → run → drain → destroy)
SSH-based binary deployment with ephemeral per-worker Ed25519 keys
Auto-scaling triggers: manual Sudo button, queue depth threshold
Cost controls: max run cost ($20), monthly spend cap ($500), max droplet lifetime (8h)
Emergency stop mechanism
12 open questions and decision points documented for operator review

`doc/nidus-reveal/vision-worker/implementation.md`

Implementation plan with 9 phases and ~19.5 engineer-days estimate:

Provider abstraction interface
Digital Ocean driver (godo)
Database extensions (new columns, tables, queries)
SSH deployment system (key gen, SCP, systemd)
Auto-scaler engine (evaluation loop, scale-up/down)
Cost controls and safety
API and Sudo UI integration
Testing and hardening (mock DO server, chaos tests)
Observability (metrics, logging, notifications)

Both documents extend the existing Nidus Reveal design and supplement the main implementation plan from issue #108.

This PR addresses issue #126 — researching and documenting what it would take to have the system automatically spin up and spin down GPU instances on Digital Ocean. ## What’s Included ### `doc/nidus-reveal/vision-worker/design.md` Design document covering: - Provider abstraction interface for multi-cloud flexibility - godo-based Digital Ocean driver for GPU droplet provisioning - Full droplet lifecycle state machine (provision → deploy → run → drain → destroy) - SSH-based binary deployment with ephemeral per-worker Ed25519 keys - Auto-scaling triggers: manual Sudo button, queue depth threshold - Cost controls: max run cost ($20), monthly spend cap ($500), max droplet lifetime (8h) - Emergency stop mechanism - 12 open questions and decision points documented for operator review ### `doc/nidus-reveal/vision-worker/implementation.md` Implementation plan with 9 phases and ~19.5 engineer-days estimate: 1. Provider abstraction interface 2. Digital Ocean driver (godo) 3. Database extensions (new columns, tables, queries) 4. SSH deployment system (key gen, SCP, systemd) 5. Auto-scaler engine (evaluation loop, scale-up/down) 6. Cost controls and safety 7. API and Sudo UI integration 8. Testing and hardening (mock DO server, chaos tests) 9. Observability (metrics, logging, notifications) Both documents extend the existing Nidus Reveal design and supplement the main implementation plan from issue #108.

ned referenced this pull request

2026-07-16 21:16:43 +00:00

Research into using go to manage Digital Ocean droplets #126