Download Planet order metadata on startup

eliribble commented

2026-07-15 20:59:33 +00:00

Owner

After #111 we've got working integration with Planet, and #118 contains information about how the auth system works. Integration testing has shown that with a Planet API token nidus-sync is currently showing authentication with planet as successful.

Our account already has some orders. If you look at the logic in go-planet/cmd/planet under the "order list" command you can see how the requests are made. I'd like you to make a first draft of downloading and storing our order information in the nidus-sync database. Since we own the go-planet library as well, feel free to suggest changes to that library to facilitate the work.

After #111 we've got working integration with Planet, and #118 contains information about how the auth system works. Integration testing has shown that with a Planet API token nidus-sync is currently showing authentication with planet as successful. Our account already has some orders. If you look at the logic in go-planet/cmd/planet under the "order list" command you can see how the requests are made. I'd like you to make a first draft of downloading and storing our order information in the nidus-sync database. Since we own the go-planet library as well, feel free to suggest changes to that library to facilitate the work.

ned was assigned by eliribble

2026-07-15 20:59:33 +00:00

ned referenced this issue from a commit

2026-07-15 21:03:38 +00:00

feat: download and store Planet order metadata from API on startup

ned referenced this issue from a pull request that will close it,

2026-07-15 21:03:43 +00:00

Download and store Planet order metadata on startup #120

ned commented

2026-07-15 21:04:07 +00:00

Member

Hi diddly ho neighborino! 👋

I have put together a first draft — PR #120.

What it does: On startup, if PLANET_API_TOKEN is set, nidus-sync:

Calls the Planet Orders API to list all orders (same endpoint as planet order list)
Fetches full details for each order via planet order get
Stores everything in a new planet_order table with the raw JSON preserved for forward compatibility

The sync runs as a short-lived background goroutine and uses ON CONFLICT upserts so it is safe to re-run.

No go-planet changes needed — the existing ListOrders and GetOrder methods were sufficient.

Things still to sort out:

Need to run go-jet to regenerate models from the new migration
The PLANET_API_TOKEN env var needs to be added to the secrets file
Periodic re-sync / job integration is still TODO

Would love your eyes on it when you get a chitty-chat break!

Hi diddly ho neighborino! 👋 I have put together a first draft — PR #120. **What it does:** On startup, if `PLANET_API_TOKEN` is set, nidus-sync: 1. Calls the Planet Orders API to list all orders (same endpoint as `planet order list`) 2. Fetches full details for each order via `planet order get` 3. Stores everything in a new `planet_order` table with the raw JSON preserved for forward compatibility The sync runs as a short-lived background goroutine and uses `ON CONFLICT` upserts so it is safe to re-run. **No go-planet changes needed** — the existing `ListOrders` and `GetOrder` methods were sufficient. **Things still to sort out:** - Need to run go-jet to regenerate models from the new migration - The `PLANET_API_TOKEN` env var needs to be added to the secrets file - Periodic re-sync / job integration is still TODO Would love your eyes on it when you get a chitty-chat break!

ned referenced this issue from a commit

2026-07-15 21:10:54 +00:00

feat: download and store Planet order metadata from API on startup

ned referenced this issue from a commit

2026-07-15 21:13:34 +00:00

feat: download and store Planet order metadata from API on startup

eliribble closed this issue

2026-07-15 21:14:36 +00:00

ned referenced this issue from a pull request that will close it,

2026-07-15 21:37:36 +00:00

feat: download and store Planet order metadata from API on startup #122

ned commented

2026-07-15 22:27:14 +00:00

Member

This was completed via other PRs. Main now has:

planet/client.go — Syncer wrapping the go-planet Client
planet/store.go — Storer with UpsertOrder, SyncAll
planet/download.go — download order results to filesystem
planet.StartSync(ctx, pool) wired into main.go startup
db/migrations/00178_planet_schema.sql — planet.order, planet.item, planet.item_asset, planet.order_item, planet.sensor tables

Closing as completed.

This was completed via other PRs. Main now has: - planet/client.go — Syncer wrapping the go-planet Client - planet/store.go — Storer with UpsertOrder, SyncAll - planet/download.go — download order results to filesystem - planet.StartSync(ctx, pool) wired into main.go startup - db/migrations/00178_planet_schema.sql — planet.order, planet.item, planet.item_asset, planet.order_item, planet.sensor tables Closing as completed.

ned commented

2026-07-15 22:39:02 +00:00

Member

Found and fixed a bug triggered by adding go vet ./... to the CI pipeline:

platform/message.go:17 — fmt.Errorf(…%%w, phone_number) was missing its error argument, silently swallowing the normalized phone number error. go vet caught it.

Fixed in commit 8b46d502. However, the nix-based CI is still failing (run #617). I cannot access the action logs (Forgejo v15.0.2 API limitation — 404 on jobs/logs endpoints). The failure happens ~30s in, past checkout/clone/copy-to-/tmp but during the nix develop + go build phase. Likely candidates to investigate:

Nix can't reach the binary cache or nixpkgs-unstable
go build tries to resolve private modules via HTTPS and auth fails inside nix
The nix build takes longer than the runner allows

If someone with runner-shell access can grab the raw logs from a failed run, we can zero in on the exact cause.

Found and fixed a bug triggered by adding `go vet ./...` to the CI pipeline: **platform/message.go:17** — `fmt.Errorf(…%%w, phone_number)` was missing its error argument, silently swallowing the normalized phone number error. `go vet` caught it. Fixed in commit `8b46d502`. However, the nix-based CI is still failing (run #617). I cannot access the action logs (Forgejo v15.0.2 API limitation — 404 on jobs/logs endpoints). The failure happens ~30s in, past checkout/clone/copy-to-/tmp but during the nix develop + go build phase. Likely candidates to investigate: - Nix can't reach the binary cache or nixpkgs-unstable - go build tries to resolve private modules via HTTPS and auth fails inside nix - The nix build takes longer than the runner allows If someone with runner-shell access can grab the raw logs from a failed run, we can zero in on the exact cause.

eliribble referenced this issue from a commit

2026-08-29 22:17:58 +00:00

feat: download and store Planet order metadata from API on startup

Rows
Columns

Download Planet order metadata on startup #119