Product input needed: FieldSeeker technician names → Nidus users (identity, aliases, disabled-by-default) #257

New issue

Open

opened 2026-09-29 14:18:25 +00:00 by eliribble · 1 comment

eliribble commented

2026-09-29 14:18:25 +00:00

Owner

Context

Today a Nidus Inspection/Treatment cannot say who performed it as a real identity: the FieldSeeker translation drops the source's technician name, so the review UI shows nothing (only creator_id, which is null for imports). The goal is to attribute work to actual Nidus users — enabling account avatars and "what has this user done" queries without free-text matching.

Proposed approach: at import time, resolve the source technician name against a user-alias table; if unrecognized (and it looks like a person), create a new Nidus user disabled by default, so it cannot log in until an account owner fleshes it out and enables it.

Before building, this issue records the reconnaissance on the FieldSeeker technician data and the product decisions needed. Counts below are from the shared dev database (nidus-sync, one district) — read-only.

What FieldSeeker gives us

The technician is a free-text name string on each source row — no id, no foreign key, and no personnel/employee table exists in the fieldseeker schema. So identity is name-string-only, which is why an alias table is unavoidable.

Table	Column	Rows	Distinct values
timecard	fieldtech	720,303	94
mosquitoinspection	fieldtech	507,705	142
treatment	fieldtech	288,472	142
trapdata	fieldtech	226,492	39
locationtracking	fieldtech	786	54
samplecollection	fieldtech	2	1
qamosquitoinspection	fieldtech	2	1

Secondary columns that also carry a technician-ish name: trapdata.idbytech (24 distinct), servicerequest.assignedtech (46), servicerequest.techclosed (37), pool.survtech (22), servicerequest.entrytech (16), pointlocation.assignedtech (11), inspectionsample.idbytech (4). trapdata.sortbytech is entirely empty.

Other name-ish fields present but not the field technician: creator / editor (121 / 116 distinct ArcGIS account names, themselves mixed — see below), and created_user / last_edited_user (1–2 distinct — effectively blank in this district).

How reliable are the names?

Across the seven fieldtech tables: 1,743,762 rows, 161 distinct values.

Records per exact name	# names
≥ 100	128
10–99	19
2–9	12
exactly 1	2

Spelling is very consistent: across all 161 values there is exactly one case/whitespace near-duplicate cluster (bryan ferguson / Bryan Ferguson / Bryan Ferguson ). Names are reused across tables too (66 appear in 2 tables, 31 in 3, 25 in 4, 26 in 5, 1 in 6).

So a small, stable vocabulary — a curated alias table would map nearly everything exactly.

The catch: the vocabulary mixes people with zones, labs, and opaque ids

Category	# names	Records	Share
`Zone NN Delta` (e.g. `Zone20 Delta`)	16	654,816	38%
`Lab N`	8	292,766	17%
16-hex ids (e.g. `bb63513e7297e68f1c0a`)	64	242,878	14%
other (people + misc tokens)	73	553,302	32%

~68% of fieldtech rows carry a zone, lab, or hex token — not a person. Zones/labs are how the district records route/broadcast work; the hex ids are unexplained. Naively "create a user per distinct value" would mint disabled users for Zone20 Delta, Lab 3, and 64 hex strings.

Misc non-person tokens also appear: Website, Admin, Admin Delta, DMVCD Admin, FrontierPadmin, FSAdministrator, Front Desk, No Value, -1, Tester, AT, JV, SPM, Biologists, HTC 1, HTC 2, Na, and one free-text sentence.

Alias / mismatch evidence (the problem the alias table must solve)

Real cases observed, small in number:

Bryan Ferguson / bryan ferguson / bryan furguson
A. Davis / A. Davi / A. DAvis / ADavisDelta
Rick Alvarez / Rick Alverez · Jesus Solano / Jesus Jolano
Conlin Reis / Conlin / CReisDelta · Ryan Spratt / RSprattDelta
Mir Bear-Johnson / Mir Delta · Hector Cardenas / HectorDelta
Arturo Bolanos / Arturo Bolano · Kalvin Chavez / Kalvin Chaves · Juan Pablo Ortega / Pablo Ortega

The ArcGIS creator/editor accounts use a further variant form — e.g. CRodriguezDelta, JPerezDelta, LMcGillDelta, zone20Delta — so if we ever key on those too, aliases must cover that shape as well.

Product questions

Classification rule. How do we decide a raw source value is a person before creating a user, versus a zone/lab/system token to ignore? Zones + labs are ~55% of rows. Curated deny-list, curated allow-list, a name-shape heuristic, or a combination?
Hex ids. What are the 64-ish 16-hex values (~250k rows, 14%)? Real technician accounts, FieldSeeker user GUIDs, or junk? Include or exclude?
Authoritative column(s). Is fieldtech the technician for our purposes, or should we also resolve assignedtech / idbytech / survtech / testtech / techclosed? Those add ~24 further distinct names on top of fieldtech.
Auto-create scope. Create a disabled user for every unrecognized person-shaped name regardless of volume, or gate on a record threshold? Confirm disabled-by-default as the invariant.
Alias model. Key aliases by (external system, raw string) → user? Per-organization or global? Who curates them — an admin UI, or manual SQL/import — and is there an approval step?
Zones and labs. Should they map to some other entity (a Zone/Lab concept, a crew), or be explicitly ignored? (They are meaningful grouping — 55% of records — so "ignore" may lose information.)
Merge semantics. When two spellings are the same person, is that one user with two aliases, or two users? Who can merge/unmerge?
Backfill. Do we re-run the FieldSeeker import to attribute historical records (≈1.7M fieldtech rows this district)? Scope and timing?
Profile fields. Do we seed the user's display name from the source name, or leave the profile empty for the owner to fill in before enabling?
Generalization. This is one district. Do we assume the same vocabulary shape elsewhere, or make the classification/alias config per district?

Suggested next step

Settle Q1/Q2 (classification, and what the hex ids are) first — they determine what "unrecognized technician" even means. After that, the schema (user disabled default + user_alias keyed by external system + raw string) and the import resolve-or-create path can be designed.

(Recon methodology: read-only queries against nidus-sync; categories assigned by value shape — leading Zone/Lab, 16+ hex, otherwise "other".)

## Context Today a Nidus Inspection/Treatment cannot say *who* performed it as a real identity: the FieldSeeker translation drops the source's technician name, so the review UI shows nothing (only `creator_id`, which is null for imports). The goal is to attribute work to actual Nidus users — enabling account avatars and "what has this user done" queries without free-text matching. Proposed approach: at import time, resolve the source technician name against a user-alias table; if unrecognized (and it looks like a person), create a new Nidus user **disabled by default**, so it cannot log in until an account owner fleshes it out and enables it. Before building, this issue records the reconnaissance on the FieldSeeker technician data and the product decisions needed. Counts below are from the shared dev database (`nidus-sync`, one district) — read-only. ## What FieldSeeker gives us The technician is a **free-text name string** on each source row — no id, no foreign key, and **no personnel/employee table exists** in the `fieldseeker` schema. So identity is name-string-only, which is why an alias table is unavoidable. | Table | Column | Rows | Distinct values | |---|---|---|---| | timecard | fieldtech | 720,303 | 94 | | mosquitoinspection | fieldtech | 507,705 | 142 | | treatment | fieldtech | 288,472 | 142 | | trapdata | fieldtech | 226,492 | 39 | | locationtracking | fieldtech | 786 | 54 | | samplecollection | fieldtech | 2 | 1 | | qamosquitoinspection | fieldtech | 2 | 1 | Secondary columns that also carry a technician-ish name: `trapdata.idbytech` (24 distinct), `servicerequest.assignedtech` (46), `servicerequest.techclosed` (37), `pool.survtech` (22), `servicerequest.entrytech` (16), `pointlocation.assignedtech` (11), `inspectionsample.idbytech` (4). `trapdata.sortbytech` is entirely empty. Other name-ish fields present but not the field technician: `creator` / `editor` (121 / 116 distinct **ArcGIS account** names, themselves mixed — see below), and `created_user` / `last_edited_user` (1–2 distinct — effectively blank in this district). ## How reliable are the names? Across the seven `fieldtech` tables: **1,743,762 rows, 161 distinct values.** | Records per exact name | # names | |---|---| | ≥ 100 | 128 | | 10–99 | 19 | | 2–9 | 12 | | exactly 1 | 2 | Spelling is **very consistent**: across all 161 values there is exactly **one** case/whitespace near-duplicate cluster (`bryan ferguson` / `Bryan Ferguson` / `Bryan Ferguson `). Names are reused across tables too (66 appear in 2 tables, 31 in 3, 25 in 4, 26 in 5, 1 in 6). So a small, stable vocabulary — a curated alias table would map nearly everything exactly. ## The catch: the vocabulary mixes people with zones, labs, and opaque ids | Category | # names | Records | Share | |---|---|---|---| | `Zone NN Delta` (e.g. `Zone20 Delta`) | 16 | 654,816 | 38% | | `Lab N` | 8 | 292,766 | 17% | | 16-hex ids (e.g. `bb63513e7297e68f1c0a`) | 64 | 242,878 | 14% | | other (people + misc tokens) | 73 | 553,302 | 32% | **~68% of `fieldtech` rows carry a zone, lab, or hex token — not a person.** Zones/labs are how the district records route/broadcast work; the hex ids are unexplained. Naively "create a user per distinct value" would mint disabled users for `Zone20 Delta`, `Lab 3`, and 64 hex strings. Misc non-person tokens also appear: `Website`, `Admin`, `Admin Delta`, `DMVCD Admin`, `FrontierPadmin`, `FSAdministrator`, `Front Desk`, `No Value`, `-1`, `Tester`, `AT`, `JV`, `SPM`, `Biologists`, `HTC 1`, `HTC 2`, `Na`, and one free-text sentence. ## Alias / mismatch evidence (the problem the alias table must solve) Real cases observed, small in number: - `Bryan Ferguson` / `bryan ferguson` / `bryan furguson` - `A. Davis` / `A. Davi` / `A. DAvis` / `ADavisDelta` - `Rick Alvarez` / `Rick Alverez` · `Jesus Solano` / `Jesus Jolano` - `Conlin Reis` / `Conlin` / `CReisDelta` · `Ryan Spratt` / `RSprattDelta` - `Mir Bear-Johnson` / `Mir Delta` · `Hector Cardenas` / `HectorDelta` - `Arturo Bolanos` / `Arturo Bolano` · `Kalvin Chavez` / `Kalvin Chaves` · `Juan Pablo Ortega` / `Pablo Ortega` The ArcGIS `creator`/`editor` accounts use a further variant form — e.g. `CRodriguezDelta`, `JPerezDelta`, `LMcGillDelta`, `zone20Delta` — so if we ever key on those too, aliases must cover that shape as well. ## Product questions 1. **Classification rule.** How do we decide a raw source value is a *person* before creating a user, versus a zone/lab/system token to ignore? Zones + labs are ~55% of rows. Curated deny-list, curated allow-list, a name-shape heuristic, or a combination? 2. **Hex ids.** What are the 64-ish 16-hex values (~250k rows, 14%)? Real technician accounts, FieldSeeker user GUIDs, or junk? Include or exclude? 3. **Authoritative column(s).** Is `fieldtech` *the* technician for our purposes, or should we also resolve `assignedtech` / `idbytech` / `survtech` / `testtech` / `techclosed`? Those add ~24 further distinct names on top of `fieldtech`. 4. **Auto-create scope.** Create a disabled user for every unrecognized *person-shaped* name regardless of volume, or gate on a record threshold? Confirm disabled-by-default as the invariant. 5. **Alias model.** Key aliases by (external system, raw string) → user? Per-organization or global? Who curates them — an admin UI, or manual SQL/import — and is there an approval step? 6. **Zones and labs.** Should they map to some other entity (a Zone/Lab concept, a crew), or be explicitly ignored? (They are meaningful grouping — 55% of records — so "ignore" may lose information.) 7. **Merge semantics.** When two spellings are the same person, is that one user with two aliases, or two users? Who can merge/unmerge? 8. **Backfill.** Do we re-run the FieldSeeker import to attribute historical records (≈1.7M `fieldtech` rows this district)? Scope and timing? 9. **Profile fields.** Do we seed the user's display name from the source name, or leave the profile empty for the owner to fill in before enabling? 10. **Generalization.** This is one district. Do we assume the same vocabulary shape elsewhere, or make the classification/alias config per district? ## Suggested next step Settle Q1/Q2 (classification, and what the hex ids are) first — they determine what "unrecognized technician" even means. After that, the schema (user `disabled` default + `user_alias` keyed by external system + raw string) and the import resolve-or-create path can be designed. *(Recon methodology: read-only queries against `nidus-sync`; categories assigned by value shape — leading `Zone`/`Lab`, 16+ hex, otherwise "other".)*

eliribble commented

2026-10-27 15:42:06 +00:00

Author

Owner

This has been superceded by doc/design/integrations/fieldseeker/technician-identity.md

This has been superceded by [doc/design/integrations/fieldseeker/technician-identity.md](https://source.gleipnir.technology/Gleipnir/nidus-sync/src/branch/somber-dolphin/doc/design/integrations/fieldseeker/technician-identity.md)