No description
Find a file
Eli Ribble 91fe22a536 Attach client-dictated sentence punctuation to audio word timings
Make the client the single authority on sentence breaks, mirroring the
transcript text the user is shown. A spoken "period" arrives from the
recognizer as a standalone '.' token; a long pause commits the current
utterance as a sentence end. Both are now folded onto the preceding word's
text (so "Hello" + "." becomes one word "Hello."), and each committed
utterance boundary gets a terminal period appended to its final word.

The uploaded word stream therefore carries each sentence's terminal
punctuation, so the backend can group words into transcript lines by
punctuation alone instead of re-deriving boundaries from pause timing.
2026-09-03 07:28:56 -07:00
doc/design Drop section 11 per review: backend extraction comparison removed 2026-07-22 01:42:12 +00:00
Nidus Attach client-dictated sentence punctuation to audio word timings 2026-09-03 07:28:56 -07:00
Nidus Notes Unit Tests Attach client-dictated sentence punctuation to audio word timings 2026-09-03 07:28:56 -07:00
Nidus Notes.xcodeproj Recommended changes to project format 2026-09-01 18:14:52 -07:00
SpeechTest Compile custom language model and use it in Nidus 2025-10-10 10:24:08 -07:00
SpeechTrainer Various fixes while building demo video 2025-12-10 05:52:56 -07:00
swift-h3@4954df5b00 Nest swift-h3 as a submodule 2026-09-01 15:44:35 -07:00
.gitignore Update .gitignore from gitignore.io 2025-04-28 16:48:05 -07:00
.gitmodules Nest swift-h3 as a submodule 2026-09-01 15:44:35 -07:00
.pre-commit-config.yaml Add pre-commit hook for formatting swift files 2025-04-28 16:48:05 -07:00
.swift-format Configure swift-format a bit 2025-04-28 16:48:05 -07:00
41e0159d-8dbb-48a1-9fcb-7f5107631365.png Compile custom language model and use it in Nidus 2025-10-10 10:24:08 -07:00
bf20563e-aed5-426f-8000-e1ed2c935832-normalized.m4a Compile custom language model and use it in Nidus 2025-10-10 10:24:08 -07:00
lefthook.yml Switch to Apple built-in formatter in lefthook 2025-12-05 13:28:49 -07:00
LICENSE Initial commit 2025-04-28 16:45:21 -07:00
Nidus-Notes-Info.plist Add custom info plist for encryption export control 2025-05-28 09:19:51 -07:00
README.md Add initial README 2026-08-25 09:57:56 -07:00

Nidus

Native iOS/SwiftUI iPhone application for mosquito-control (vector) field technicians. Technicians survey sites, record "nidus" (breeding-source) observations, inspections, and treatments — largely speech-driven: audio is transcribed, then parsed into structured knowledge graphs. Data syncs to a backend mirroring Placer's "FieldSeeker" system.

Built with SwiftData + SQLite (SQLite.swift + SQLiteMigrationManager), SwiftUI, MapKit/H3 hex cells, Apple's NaturalLanguage framework, and Sentry crash reporting (self-hosted GlitchTip).

Application entry

Nidus/NidusApp.swift boots a RootController/RootStore, configures Sentry, registers for remote notifications, and forces portrait orientation. Uses the classic controller/store pattern rather than pure SwiftUI MVVM.

Architecture

Code is organized under Nidus/:

  • Controller/ — 11 controllers (Root, AudioRecording, Camera, Network, Notes, Region, Database, Settings, AudioPlayback, Error, Toast) coordinating stores/services.
  • Store/ — SwiftData @Model persistence plus in-memory KnowledgeGraph (parsed field data), NotesStore, RegionStore, SettingsStore, audio recording/playback stores.
  • Service/KnowledgeService (NL speech→knowledge extraction), NetworkService (actor: uploads audio/images, login, fetch-updates), DatabaseService (SQLite), plus camera/photo/movie capture, natural language, SPC observer, and device lookup services.
  • Model/ — note types (NidusNote, MosquitoSource, ServiceRequest, TrapData, PictureNote, AudioNote, etc.) and FieldSeeker.swift (Codable entities matching the FieldSeeker backend schema).
  • Database/ — SQLite schema, migrations, and a hand-written Query.swift layer.
  • API/ — upload payloads (NidusNotePayload, ImagePayload, AudioPayload) and backend responses (IosClientResponse, FieldseekerResponse).
  • View/ — SwiftUI views (map/breadcrumb/route/inspection-summary tabs, edit views, detail sheets, camera UI).

Data model / field capture

The domain is organized around the session tree: Site Survey → Safety/Access constraints → Observations → Features → Inspections/Treatments. Capture is dual-mode — voice triggers ("begin feature", "safety assessment", "negative observation") or tap navigation — and values are parsed from natural speech:

  • Enum matching ("aggressive animal" → SafetyConstraint), pattern parsing ("twenty by forty by six" → dimensions), quantity+noun (groups), free-text into structured fields.
  • Raw transcript is preserved as evidence but not shown to the technician during capture.
  • A rule-based parser (KnowledgeService.ExtractKnowledge) — not ML — walks lemmatized tokens/grams to extract habitat, water conditions, dimensions/volume, density, life stage, genus/species, dip counts, treatment product/amount, and safety/access mitigators.

Backend sync

NetworkService handles login (server URL/credentials from settings), pulls incremental updates (IosClientResponse with a since timestamp returning service requests, mosquito sources, traps), and uploads audio files and pictures (with background URL-session support via BackgroundDownloadWrapper). Photos are attributed to the currently-active tree node.

Recent work / current state

The most recent artifact (2026-08-25) is a data-model & utterances package for a "Placer MVCD Phase 1" rollout: 20260825-placer-nidus-notes-data-model-package/ containing

  • data-model.md — the canonical session-tree data model (entities, attributes, enums, triggers, recognition rules);
  • utterances.yaml — 20 real technician utterances (10 structured + 10 conversational) with field extractions, covering dry features, access/safety constraints, and groups;
  • utterances-review.md — field-by-field extraction breakdown per example.

This indicates the current focus is formalizing the data model and validating the rule-based extractor against real transcription data for the Placer Mosquito & Vector Control District Phase 1 rollout.

Notable observations (risks / debt)

  • Two persistence stacks coexist: SwiftData @Model (NidusNote) and hand-rolled SQLite (DBSchema/Query.swift). Some types bridge both (e.g. NidusNote.toPayload() vs MosquitoSource Codable). Likely migration/debt hotspot.
  • Typo/legacy residue: Store/KnowledgGraph.swift (misspelled filename); Note.swift defines a NoteCategory/Note protocol while Model/Note/ also has NoteProtocol.swift — two overlapping note abstractions.
  • Hand-written extraction logic in KnowledgeService is brittle by nature (regex/gram heuristics, ordinal/cardinal parsing); the utterances corpus is meant to harden it.
  • Tight coupling: RootController holds 11 sub-controllers and direct references to LocationDataManager; preview/test seams are thin (RootControllerPreview, forPreview helpers).
  • Small unit-test surface (21 Testing-framework tests, mostly inspection*/mosquitoSource* extraction cases) relative to the parsing complexity.
  • Stray non-source artifacts committed at repo root (.heif/.m4a media files, a .jpeg), plus two peripheral Xcode projects (SpeechTest, SpeechTrainer) alongside the main Nidus Notes.xcodeproj.

Bottom line

A mature, speech-first field-capture app with a sophisticated rule-based NL parser, currently formalizing its data model and validating extraction against a real technician utterance corpus for the Placer MVCD Phase 1 rollout. Main structural risks: dual persistence stacks, overlapping note abstractions, and coupling in the root controller.