- Go 99.9%
|
|
||
|---|---|---|
| .forgejo/workflows | ||
| cmd/vectorsurv | ||
| gateway | ||
| gen | ||
| .gitignore | ||
| application.go | ||
| application_file.go | ||
| application_file_test.go | ||
| application_push.go | ||
| application_push_test.go | ||
| application_test.go | ||
| CHANGELOG.md | ||
| client.go | ||
| client_test.go | ||
| compose.go | ||
| compose_test.go | ||
| config.go | ||
| config_test.go | ||
| conformance_test.go | ||
| flake.lock | ||
| flake.nix | ||
| go.mod | ||
| go.sum | ||
| lefthook.yml | ||
| README.md | ||
| resources.go | ||
| resources_test.go | ||
| sandbox_roundtrip_test.go | ||
| transport.go | ||
| types.go | ||
| units.go | ||
| units_test.go | ||
vectorsurv-go
A Go client library for the VectorSurv API. It covers the full published API surface: every resource's list/get/create/update operations, read-only reference lookups, and the action endpoints (validation, pool/collection linking, calculations, label generation, and more).
The library is a strict gateway: it knows nothing about the applications that consume it and takes and returns plain structs.
Layout
| Path | Kind | Contents |
|---|---|---|
*.go (this package, vectorsurv) |
hand-written | config, authentication, the HTTP transport, the sandbox guard, error decoding and the pagination policy. It embeds gateway.Client, so every generated resource method is a method here. |
gateway/ |
generated | the resource methods and wire types, over an injected Transport. Never edited by hand; generated by gen/. |
gen/ |
hand-written | the generator, which reads gen/swagger.json and emits gateway/ and cmd/vectorsurv/. |
cmd/vectorsurv/ |
generated + main.go, conformance.go, README.md |
the CLI, including the sandbox conformance runner and the worked example of a client that owns its configuration. |
conformance_test.go |
hand-written | the read surface's conformance check, driven from gen/tables.go. |
The import edge is one-way: the root imports gateway, never the reverse.
Configuration
The library's whole contract is vectorsurv.Config, and vectorsurv.New(cfg)
is the only way in. The library reads no environment variable and no file:
a client program decides where its configuration comes from — its own flags,
the environment, a config file, a database row — and hands over a Config.
| Field | Type | Domain |
|---|---|---|
BaseURL |
string |
The API root, e.g. https://sandbox.api.vectorsurv.org. Empty means vectorsurv.SandboxBaseURL. Any host that is not the sandbox — compared as a hostname, so scheme and port do not matter — is refused with vectorsurv.ErrNotSandbox unless AllowProduction is set. |
AgencyToken |
string |
A VectorSurv agency token, sent directly as the bearer. Wins over Username/Password when both are configured. |
Username |
string |
With Password: the alternative credential form, authenticating through POST /login. |
Password |
string |
See Username. |
AllowProduction |
bool |
Default false. Permits a BaseURL that is not the sandbox; reaching another host is always a deliberate opt-in. It admits reads of that host only. |
AllowProductionWrites |
bool |
Default false. Permits writes — POST, PUT, DELETE — against a BaseURL that is not the sandbox. Deliberately separate from AllowProduction, so a deployment that reads production cannot silently write to it: see writes, sandbox-first. |
HTTPClient |
*http.Client |
Optional. nil means a client with a 30-second timeout. Set it to reuse a transport, route through a proxy, or relax TLS. |
MaxResponseBytes |
int |
Caps how much of a response body is read, so a hostile or runaway server cannot exhaust memory. Zero means vectorsurv.DefaultMaxResponseBytes (64 MiB — well above the API's largest real response, about 6.3 MB for GET /v1/agency, where every agency carries its full boundary MultiPolygon). A body over the cap is an error naming the limit and the endpoint, never a body decoded from a truncated read. Negative is refused by New. |
New enforces these invariants:
- One credential form must be present — an
AgencyToken, or both aUsernameand aPassword— or it fails with a "no credentials" error. - A token wins over username and password: when an agency token is set it is
sent as the bearer and
/loginis never called, so there is nothing to refresh and a 401 is reported as it is. The/loginform caches the returned bearer JWT and refreshes it once on a 401. - The base URL is resolved and guarded before the first request.
- The response cap is bounded and resolved before the first request: the
configured
MaxResponseBytes, orvectorsurv.DefaultMaxResponseByteswhen it is zero, bounds every body the client reads — including/login's. A longer body is an error naming the limit and the endpoint, never a truncation that surfaces later as a decode failure.
Pagination defaults apply to every list read: pages of
vectorsurv.DefaultPageSize (500) are walked to exhaustion, and a walk is
refused after vectorsurv.DefaultMaxPages (1000) pages, so an endpoint that
keeps promising more pages is an error rather than a silent truncation (see
Pagination).
Writes, sandbox-first
| Allowances set | The sandbox | Any other host |
|---|---|---|
| neither | read, write | refused (ErrNotSandbox) |
AllowProduction |
read, write | read; a write is refused (ErrNotSandboxWrite) |
AllowProduction and AllowProductionWrites |
read, write | read, write |
AllowProductionWrites does not admit the host by itself: a BaseURL that is
not the sandbox is still refused before the first request unless
AllowProduction is set too. The split exists so that a deployment which has
decided to read production data has not thereby decided to write it, and so
that a new write path is proven on the sandbox — which needs no allowance at all
— before anyone has to make that decision. A refused write never leaves the
client: the guard runs before the request is built.
client, err := vectorsurv.New(vectorsurv.Config{
BaseURL: "https://sandbox.api.vectorsurv.org",
AgencyToken: districtToken,
})
A worked client — including which environment variables it maps onto these
fields — is the CLI in cmd/vectorsurv:
see cmd/vectorsurv/README.md.
Library usage
import (
"context"
"source.gleipnir.technology/Gleipnir/vectorsurv-go"
"source.gleipnir.technology/Gleipnir/vectorsurv-go/gateway"
)
ctx := context.Background()
client, err := vectorsurv.New(vectorsurv.Config{AgencyToken: token})
if err != nil {
return err
}
// List records. Every page is read, and the result says so.
sites, err := client.ListSites(ctx) // gateway.ListResult[gateway.Site]
rows := sites.Rows // every gateway.Site
if !sites.Complete {
// the server promised more pages than it produced
}
cols, err := client.ListCollections(ctx) // gateway.ListResult[gateway.Collection]
agencies, err := client.ListAgencies(ctx) // gateway.ListResult[gateway.Agency]
// Fetch a single record by ID.
col, err := client.GetCollection(ctx, 12345) // *gateway.Collection
site, err := client.GetSite(ctx, 67890) // *gateway.Site
// Create a record. The request body type is <Type>Input (or <Type> when the
// create body matches the record shape).
newApp := &gateway.ApplicationInput{}
agency := 3
newApp.Agency = &agency
newApp.Local = "NEW"
app, err := client.CreateApplication(ctx, newApp) // *gateway.Application
// Update a record. id selects the row; the body type is <Type>Input or a
// dedicated <Type>Input2 when the PUT body differs from the POST body.
updated, err := client.UpdateApplication(ctx, 42, &gateway.ApplicationInput2{...})
// Delete (deactivate) a record.
err = client.DeleteApplication(ctx, 42)
// Nested resources take their parent's key first.
abundance, err := client.ListAbundances(ctx, 12345) // gateway.ListResult[gateway.Abundance]
one, err := client.GetAbundance(ctx, 12345, 7) // abundance 7 in collection 12345
Pagination
A list read walks the API's {total, page, pageSize, totalPages, rows}
envelope to exhaustion and returns a gateway.ListResult[T]: Rows holds every
record collected, and Complete reports whether the walk covered every page the
server reported. A truncated read is therefore observable rather than
indistinguishable from a complete one. The walk is bounded by a page cap, so an
endpoint that keeps reporting further pages is an error, never a silent
truncation.
gateway.ListAll[T] is the query-bearing entry point, for sparse fieldsets and
filters; the generated List methods call it with no extra query:
apps, err := gateway.ListAll[gateway.Application](ctx, client.Client, "/v1/application",
url.Values{"query[agency]": {"18"}})
One read is the exception, because its endpoint is not paginated: a list whose
response is a bare JSON array rather than the envelope (ListUserPreferences,
ListUserMemberships) returns []T directly. The whole collection is the one
response, so there are no pages to walk and no ListResult to carry a
completeness that would be a constant.
Request body types
Every resource that supports create/update has a matching input type so you
don't send server-managed fields (ids, timestamps, add_date, …) on writes:
CreateX(ctx, in *gateway.XInput)— POST body derived from the endpoint's request schema.UpdateX(ctx, id, in *gateway.XInput)— PUT body. When the PUT schema differs from the POST schema a separateXInput2(or similar) is generated.
When a resource's create/update body is identical to its record shape, the
*X type itself is used instead of an XInput.
Reference lookups
Many read-only reference tables are exposed as List/Get methods, e.g.:
lures, err := client.ListLures(ctx) // gateway.ListResult[gateway.Lure]
trap, err := client.GetTrap(ctx, 5) // *gateway.Trap
species, err := client.ListSpecies(ctx) // gateway.ListResult[gateway.Species]
targets, err := client.ListTestTargets(ctx) // gateway.ListResult[gateway.TestTarget]
Actions
Endpoints that do more than plain CRUD are exposed as named methods, for example:
err := client.ValidateSite(ctx, &gateway.SiteInput{...}) // validate without saving
err = client.LinkArthropodPoolToCollection(ctx, 11, 22) // link pool 11 to collection 22
pos, err := client.ArthropodPoolIsPositive(ctx, 11) // positive pool check
result, err := client.CreateResistanceCalculation(ctx, &gateway.CreateResistanceCalculationInput{Tests: []int{1, 2}})
num, err := client.GetNextServiceVisitNumber(ctx, &gateway.GetNextServiceVisitNumberInput{...})
ver, err := client.GetVersion(ctx) // API version
Action methods return the API's response when one is declared, otherwise just
error. Actions that return the paginated envelope (for example
ListUserMemberships) page to exhaustion like the List methods.
Pointers
Fields that are optional or server-managed are pointers (*int, *float64,
*bool) so a nil pointer means "absent" and you can distinguish a zero from a
missing value. Text fields are plain strings.
The write path
Three operations carry the write path, and the two composers are pure — no clock, no database, no network — so the same input always composes the same result:
| Call | Reads | Writes |
|---|---|---|
Compose(treatments, period, units) |
— | — (PUR: a monthly report's buckets and payload) |
ComposeApplications(treatments, opts, units) |
— | — (one plan per treatment) |
PushApplications(ctx, client, report, resolver, units) |
/v1/unit, /v1/application, /v1/application/mix, /v1/application/technician |
the mixes and applications that are not already there |
report, err := vectorsurv.ComposeApplications(treatments, vectorsurv.ApplicationOptions{
Period: vectorsurv.Period{Month: 9, Year: 2026},
Agency: &agency, // nil means the credential's primary agency
Equipment: equipmentID, // agency-scoped; the caller's setting
MixQuantity: vectorsurv.MixQuantityMirror,
PurposeFor: purposeFor, // the caller's category -> purpose policy
}, units) // the caller's unit system
if err != nil {
return err
}
resolver, err := vectorsurv.NewRosterTechnicianResolver(ctx, client, &agency)
if err != nil {
return err
}
result, err := vectorsurv.PushApplications(ctx, client, report, resolver, units)
PushApplications is idempotent against VectorSurv itself, not just against the
caller: it indexes the agency's applications by their local field and the
agency's mixes by their product's DPR registration number before it creates
anything, so a re-run creates nothing and reports what it skipped instead. Both
indexes are read whole — a walk the server leaves short is an error, never an
index that looks complete.
Filing a composed PUR report is the two curated creates beside the generated
surface, CreateUseReport and CreateUseReportItem: (*Payload).ResolveUnits
fills the unit ids from the vocabulary and (*Payload).ItemInputs builds the
create bodies. They are curated rather than generated because a create body's
field presence is part of the request — an omitted agency or commodity code is
not an explicit null — and because a line's amount unit is a resolved id, never a
guessed one. (The PUR endpoints answer HTTP 500 on the sandbox and on production
alike, so this half is composed and tested offline; the Applications half is
proven live — see the sandbox round trip.)
What the library does not know
The library is a strict gateway: it never learns the caller's model. Five things a caller owns reach it as data, as a callback, or as an implementation — never as a rule baked into the library:
| Seam | Type | Why |
|---|---|---|
| The idempotency key | ApplicationTreatment.LocalKey |
The key is the caller's own identity for the source record, written to VectorSurv's free-text local field. The library treats it as an opaque string; a treatment that carries no key is skipped (SkipNoIdempotencyKey) rather than filed without one, because a record the library cannot recognise on a re-run is a record it must not create. |
| Category → purpose | ApplicationOptions.PurposeFor |
Which vector-control purpose a product belongs to is the caller's product taxonomy, not VectorSurv's. The library ships the vocabulary constants (PurposeAdulticide, PurposeLarvicide, PurposeHerbicide) and calls the policy; a category the policy does not cover is skipped with SkipCategoryUnmapped. A missing policy is an error, never a silent empty report. |
| The organisation's units | UnitSystem |
Only the caller knows how its organisation renders a measurement and what its units convert by. ConvertAreaExact renders square metres in the organisation's area unit; ConvertAmount renders an amount in the organisation's units and names the dimension it measures (DimensionMass, DimensionVolume, DimensionCount, or empty); QuantityConversion converts between two of its units in one dimension (a discrete count has none). The library resolves a unit text against /v1/unit itself, using the rendering to recognise a unit the vocabulary names differently. |
| The mix's code | ApplicationTreatment.MixCode |
The code is the label the caller's district shows for a product — "<name> (<product id>)" is one such convention. The library carries it verbatim; a treatment with no code is skipped (SkipNoMixCode) rather than filed under an invented name. |
| Who an application is attributed to | TechnicianResolver |
The library implements two policies — the roster resolver matches a creator to an agency technician by normalised name and creates one only when the caller supplies a certification number (VectorSurv requires one), and the supervisor resolver wraps it to file an unattributable treatment under a configured supervisor instead of dropping it — and the push knows neither: it calls Resolve and honours the reason it gets back. |
ApplicationTreatment.Description is the caller's too: the label an application
carries, omitted from the request when empty.
The sandbox round trip
TestSandboxApplicationsRoundTrip (sandbox_roundtrip_test.go) is the write
path's live proof and the only test here that writes anything: it files a real
mix and application against the VectorSurv sandbox, reads both back, and proves a
re-run creates nothing. It is the read-before-create that needs a real API —
VectorSurv's local field is free text with no uniqueness, so the API will
happily accept a duplicate.
It skips unless the environment carries credentials and it sets no allowance, so
it can only ever reach the sandbox — pointing it at production fails in New:
set -a; . ~/.config/vectorsurv.env; set +a
go test -run TestSandboxApplicationsRoundTrip -v .
CLI
A CLI in cmd/vectorsurv exercises every library function. Build or run it
with the Go toolchain from the module root:
go run ./cmd/vectorsurv <command>
Authentication is the command's own: it reads the environment and builds the
vectorsurv.Config — every variable it reads, and the field each maps to, is
listed in cmd/vectorsurv/README.md.
List commands
Every resource and reference lookup has a flat list command named after the pluralized record type. Output is JSON.
go run ./cmd/vectorsurv sites
go run ./cmd/vectorsurv collections
go run ./cmd/vectorsurv agencies
go run ./cmd/vectorsurv applications
go run ./cmd/vectorsurv lures
go run ./cmd/vectorsurv testtargets
Group commands (CRUD)
Each resource that supports it has a group command (named after the singular
record type) with get, create, update, and delete subcommands. get
and delete take the record's ID (plus parent keys for nested resources);
create and update read a JSON body from stdin.
go run ./cmd/vectorsurv site get 12345
go run ./cmd/vectorsurv site delete 12345
# create/update read JSON from stdin:
echo '{"name":"Downtown","code":"DTN"}' | go run ./cmd/vectorsurv site create
echo '{"name":"Downtown","code":"DTN","comments":"relocated"}' | go run ./cmd/vectorsurv site update 12345
# nested resources take the parent key first:
go run ./cmd/vectorsurv abundance get 12345 7
Action commands
Action endpoints are exposed as flat commands (camelCased method names). Int and string path parameters are positional; when the action takes a body it is read from stdin as JSON.
go run ./cmd/vectorsurv getversion
go run ./cmd/vectorsurv validatesite
go run ./cmd/vectorsurv linkarthropodpooltocollection 11 22
go run ./cmd/vectorsurv getpopulationclassificationpoint "38.544907, -121.740517"
go run ./cmd/vectorsurv getnextservicevisitnumber # body from stdin
Flags
-v, --verbose— enable debug (request) logging.-h, --help— command help (also on subcommands).
Every environment variable the command reads, and the Config field each maps
to, is in cmd/vectorsurv/README.md.
Conformance against the sandbox
conform reads every list endpoint of the library and reports, per read,
the record type's field count, the rows decoded, the pages walked and whether
the read was complete:
go run ./cmd/vectorsurv conform \
--parent ListAbundances.collection=12345 \
--parent ListTicks.collection=12345 \
--parent ListLandUsageValues.id=7 \
--parent ListUserMemberships.id=9
conformance: 95 list reads against https://sandbox.api.vectorsurv.org
METHOD RECORD FIELDS ROWS PAGES COMPLETE OUTCOME PATH
ListAgencies Agency 14 612 13 true ok /v1/agency
ListPreferences Preference 7 21 1 true ok /v1/preference
ListUserPreferences UserPreference 9 21 - - ok /v1/user/preference
...
ListCaseCounts CaseCount 22 - - - not permitted /v1/case-count
ListPesticideUseReports PesticideUseReport 7 - - - server-side /v1/pesticide/use-report
ListResistanceColonyStates ResistanceColonyState 9 - - - needs input /v1/resistance/colony/state
95 reads: 85 conformed, 2 not permitted for this credential, 5 server-side, 3 need an input, 0 defects
A bare read — ListUserPreferences above — reports no pages and no completeness,
because its endpoint has neither: the rows decoded are the only fact it carries.
The report lines are written as each read finishes, never buffered to the end: a single read (the pesticide catalogue is ~75k rows) can take tens of seconds, and a terminal shows which read is in flight — on stderr — while it runs:
conformance: 95 list reads against https://sandbox.api.vectorsurv.org
[35/95] ListLures /v1/lure
[36/95] ListPesticides /v1/pesticide
The report is stdout and those markers are stderr, so
conform > report.txt still leaves a clean report while the terminal keeps
showing progress.
The list of reads is not hand-written: cmd/vectorsurv/surface.go is generated
from the same tables as the resource methods, so a resource added to
gen/tables.go is read by this command as soon as the library is regenerated,
and go test fails if that registry and the emitted methods disagree.
A read that did not conform is classified by observable cause, and only a defect fails the run:
| Outcome | Cause |
|---|---|
not permitted |
the credential may not read it — HTTP 401/403 |
server-side |
the server failed — HTTP 5xx |
needs input |
the server asked for a parameter this run was not given — an HTTP 422 naming a field, or a path parent id |
DEFECT |
anything else: a decode error, a shape the record cannot hold, an empty record, a truncated or short walk |
The classification is by cause, never by an endpoint list, so a new failure is never excused by name: a decode error or a wrong shape is always a defect. Each failing read is still reported, under its cause, with its path and reason.
Nothing but the report goes to stdout and nothing but progress markers to
stderr: no environment value ever reaches either, a log or an error message, and
-v logs request URLs only — never the Authorization header.
Credentials
conform is a client program like any other: it reads the environment itself
and hands a vectorsurv.Config to the library, which reads neither the
environment nor any file. The variables it reads — VECTORSURV_BASE_URL,
VECTORSURV_AGENCY_TOKEN, VECTORSURV_USERNAME, VECTORSURV_PASSWORD,
VECTORSURV_ALLOW_PRODUCTION and INSECURE_HTTPS — and the Config field
each one maps to are listed in
cmd/vectorsurv/README.md.
cd vectorsurv-go
VECTORSURV_AGENCY_TOKEN=… nix develop -c go run ./cmd/vectorsurv conform --parent ...
An unset base URL resolves to the sandbox, never production, so the run is
sandboxed unless the operator deliberately opts in with
VECTORSURV_ALLOW_PRODUCTION=1.
Regenerating the client
The generated files carry a "DO NOT EDIT" header. To regenerate them after the VectorSurv API changes:
# update gen/swagger.json from https://api.vectorsurv.org/swagger.json
curl -s https://api.vectorsurv.org/swagger.json -o gen/swagger.json
go run ./gen
The generator gofmts its own output, so go run ./gen alone reproduces the
committed files; regenerating an unchanged spec leaves a clean diff. It reads
the curated resource/lookup/action tables in gen/tables.go; add or adjust
entries there if an endpoint should be modeled differently, then re-run
go run ./gen.
The tables in gen/tables.go choose each endpoint's operations, but the spec
itself is incomplete or wrong in places, and the corrections live in
gen/extensions.go, a generator input like the tables — a curl refresh of
swagger.json cannot drop them, and a spec that later declares the same field
or shape takes over on its own. An entry does one of three things:
- adds a field the API returns that the spec does not declare (a
technician's agency, a use-report county's state), with
fields; - fixes a field's type where the spec declares it wrongly, with
fixes— a preference'soptions, declared a single object but returned as an object on/v1/user/preferenceand an array on/v1/preference; and a signal word's numericidand a test method's or status's numericweight, each declared a string the API never sends; - records a list's shape where the spec declares it differently from the
API:
pagedfor a list the spec declares a bare object while the API returns the paginated envelope (the endpoint's own summary calls it "a paginated response"), andbarefor one the spec declares as an envelope — or a bare record — while the API returns a bare JSON array.
Add an entry there rather than editing a generated file or the vendored spec; the generator fails, rather than emitting a quietly wrong record, if an entry stops matching what it emits.
Three shapes are worth knowing when reading gateway/types.go:
- An object the spec fixes no property for, and the API fills per record (a
preference's
flags, a site'slabels) is emitted asmap[string]any: the document names neither its keys nor their types, so there is no struct to emit. It used to be emitted as an empty struct. A field that genuinely holds more than one JSON shape — a preference'soptionsis an object on/v1/user/preferenceand an array on/v1/preference— is emitted asanyand marked with the union it holds. - An array whose element schema the spec omits — the GeoJSON
coordinatesof a shape — is emitted asjson.RawMessageand carried verbatim, because its nesting depends on the geometry type. A field the spec declares as aoneOfof scalars (a name or an id) is emitted asanyand marked with the union it came from, since no single Go type accepts both:
State any `json:"state"` // union of string|integer
Options any `json:"options"` // union of array|object
Coordinates json.RawMessage `json:"coordinates"`
- A read whose endpoint returns a bare JSON array rather than the paginated
envelope (
ListUserPreferences,ListUserMemberships) is emitted as[]T, not a one-pageListResult[T]: the collection is the one response, there are no pages to walk, and aCompleteconstant would be exactly the vacuity the paged reads exist to avoid.
prefs, err := client.ListUserPreferences(ctx) // []gateway.UserPreference
Development
The repository ships a devshell with the pinned Go toolchain:
nix develop -c go test ./...
go test includes the read surface's conformance check
(conformance_test.go): it walks every list read the
library exposes — enumerated from gen/tables.go and from the methods actually
emitted — against an httptest fake serving the paginated envelope (or, for a
read curated as a bare array, the collection in one JSON array), asserting that
the record type carries fields, that a response carrying every declared field
decodes into every one of them, and that a paged walk returns three pages, six
rows and Complete, a bare read the rows it was served. A resource added to
gen/tables.go and emitted without coverage fails it, as does an emitted List
method with no entry in the tables, as does a read the sandbox runner above
would skip. The same test parses the emitted types and fails on any struct with
no fields or any bare any field the emitted source does not mark as a declared
union.
go test also carries the write path's live proof,
TestSandboxApplicationsRoundTrip. It skips without
credentials, so CI never files anything and a developer needs no account to run
the suite; run it deliberately with the sandbox's credentials when the write path
changes.
License
See LICENSE.